Compare commits

..
Author SHA1 Message Date
Palash Debnath 7566f3667b test: reject unresolved MSI authoring and invalid component identity 2026-09-07 11:55:46 +05:30
Palash Debnath 77f56f4f8f Merge commit '5a1ada07' into codex/diagnose-windows-wix 2026-09-07 11:52:22 +05:30
Palash Debnath 5a1ada0757 fix(windows): give binary registry keypaths explicit stable GUIDs 2026-09-07 11:51:55 +05:30
Palash Debnath b865961257 ci: gate installer authoring with the locked Tauri CLI 2026-09-07 11:51:34 +05:30
Palash Debnath 6396cc2b3f style: format Windows installer authoring and regression tests 2026-09-07 11:49:37 +05:30
Palash Debnath 7dfac1d208 Merge branch 'codex/fix-per-user-msi' into codex/diagnose-windows-wix 2026-09-07 11:48:38 +05:30
Palash Debnath d6789d026a test: render per-user diagnostic from system WiX output 2026-09-07 11:48:38 +05:30
Palash Debnath 997ba9e673 fix(windows): author per-user MSI file keypaths and folder cleanup 2026-09-07 11:48:20 +05:30
Palash Debnath 8213f992b8 ci: add isolated nonpublishing Windows MSI authoring diagnostic 2026-09-07 11:39:11 +05:30
Palash Debnath 53ff367c1f fix(studio): simplify voice cloning setup (#1817)
* fix(studio): simplify voice cloning setup

* docs: link cloning redesign changelog

* fix(studio): keep clone recording controls available

* fix(studio): lock clone capture transitions
2026-09-05 14:12:43 +05:30
Palash Debnath f2302e8c95 fix(desktop): don't adopt a backend running stale code (#1796)
Exports failed with a 422 naming a field the current app never sends — twice, from different users. The cause was the attach handshake: if something already answers on the backend port and reports a matching version, the app adopts it and skips the source sync a normal launch performs. A version string holds steady for a whole release cycle, so a same-version process can still be running weeks-old code, and that code then serves a current UI.

The handshake now compares a fingerprint of the shipped Python sources, read from the same response as the version so a dropped probe can't masquerade as a missing field. A backend predating the mechanism is treated as stale; one that is current but started outside the app is still accepted. Refusals are logged with a greppable marker, since this class previously took two reports and a code audit to identify.

Fixes #1770. Closes the duplicate report tracked in #1792.
2026-09-04 03:22:41 +05:30
Palash Debnath 8b4e4ebf56 fix(windows): start the backend when the install path has non-English characters (#1795)
On Windows with a non-UTF-8 system code page, a venv path containing non-English characters (commonly a CJK username) killed the interpreter during startup, before any VoiceStudio code ran — Python reads .pth files in the active code page, and uv's editable install writes the project path there in UTF-8. The backend could never start, and the setup screen only said it had stalled.

A new or genuinely broken environment now builds at an ASCII-safe short path; an existing environment that starts cleanly is never relocated. When no safe path can be produced, the app says so before downloading rather than after. The failure message names the cause and a remedy that works for the install mode in use, since portable installs ignore the setting managed installs use.

Fixes #1783.
2026-09-04 03:21:42 +05:30
Palash Debnath 7f7a4c5f83 fix(settings): make the generation budget reachable and honest (#1797)
The compute-time error told users to raise a generation timeout that had no control anywhere in the app — the only knob was an environment variable, and on Windows the docs explicitly warn against the usual way of setting one. Both budgets are now editable in Settings under Performance & Device, persisted and applied on the next start.

Two defects found in review and fixed here rather than shipped: an explicit universal budget silently overrode a separately saved CPU budget, so the CPU row would have looked like it worked and done nothing; and a value already set in the environment shadowed the saved preference while the panel still reported success. A shadowed row now says so instead. Long-input warnings also fire on Apple Silicon, which gets the accelerated budget and was the device in one of the duplicate reports.

Fixes #1787. Closes the reports tracked in #1774 and #1778.
2026-09-04 01:55:06 +05:30
Palash Debnath e5916acc01 feat(design): simplify the Voice Design panel (#1793)
The panel printed every chosen value three times and held twelve control rows open before anyone touched it. Details now collapse to a single recipe line that expands on request, gender/age/pitch/style become selects, and English accent and Chinese dialect merge into one grouped field so the combination the engine rejects cannot be selected at all. Starting points show five with an overflow, the bottom-bar slider is labelled Steps, and the panel ends where its content ends.

Also fixes two races found in review: a manual pick or a cleared description now beats an in-flight describe response, and arrow-key chip navigation moves focus without resetting the design.
2026-09-04 00:20:36 +05:30
Palash Debnath 06e69d6d6b fix(desktop): resolve the capability-store dir from the running backend (#1789)
Exports and every other native-picker action 403'd with "Invalid or expired desktop authorization" whenever Tauri and the backend resolved different data directories — a dev backend spawned without the OMNIVOICE_* env, a custom data folder, or portable mode. Tauri now takes the directory the running backend advertises, so the two processes cannot disagree, falling back to its own resolution when the backend is unreachable. One resolver covers all six capability kinds. Also decodes JSON escapes when reading that field, which Windows paths depend on, requires an absolute path so a relative data dir cannot recreate the same split, and keeps the 403 and its log line free of filesystem paths. Fixes #1781.
2026-09-03 22:25:08 +05:30
Palash Debnath e4ce065864 fix(design): enforce dialect/accent exclusivity in the voice-design picker (#1788)
Voice Design rendered EnglishAccent and ChineseDialect as two unlinked controls, so a user could select both and only learn they conflict from a 400 after a round trip. A shared exclusive-groups map now mirrors the engine's rule across every path that builds or restores instruct state: the live picker, free-text entry, saved-profile and imported-session restore, plus a message-matching backstop for a conflict arriving by any other route. Picking one clears the other with a visible reason instead of a silent reset. Fixes #1771.
2026-09-03 21:26:07 +05:30
Jaesik Lee f95e73b710 fix(i18n): ja "Cleaning…" is denoising, not housekeeping (#1775)
The Japanese clone.cleaning status read 掃除中 (tidying up a room) instead of ノイズ除去中, which is what the step actually does: denoising the reference audio. Thanks @j30231!
2026-09-03 20:39:42 +05:30
Jaesik Lee b20cd6cb95 fix(i18n): overhaul the ko locale (#1776)
Corrects 231 machine-translation defects in the Korean locale and translates all 493 previously missing keys, dropping ko to zero in the missing-key baseline. Includes terminology consistency (Cinematic, export, UI scale) and fixes copy that named the wrong control. Maintainer commits merged current main and folded in the five live-preview keys added by #1769. Thanks @j30231!
2026-09-03 20:19:42 +05:30
Matt Van Horn 1515b46adb feat(batch): watch-folder auto-ingest (#1768)
Opt-in watch folder on the batch queue: pick a directory once and new videos are auto-enqueued with the last Add-to-queue settings, with pause/stop controls and copy-in-progress protection. Files stream to the loopback backend as bytes; paths never leave the app. Also gives the batch queue a reachable UI entry point and streams multipart uploads to disk. Maintainer fix: the watched directory handle is opened with full share mode on Windows so users can rename or delete the folder while it is watched, matching macOS/Linux behaviour, with a cross-platform regression test. Thanks @mvanhorn!
2026-09-03 18:47:32 +05:30
Matt Van Horn 999345de41 feat(dub): realtime dub preview (#1769)
Opt-in live preview for dub segments: edits debounce into a streamed /ws/tts synthesis played through the chunk player, with cancellation preserved through buffered playback. Maintainer fixes: /ws/tts added to the backend ticket allowlist (feature was dead off-loopback), handshake failures surface a toast, loopback-only plaintext refusal reverted to keep the documented remote-GPU setup working, PCM16 decode hardened. Thanks @mvanhorn!
2026-09-03 18:27:07 +05:30
Palash Debnath ac287c612f docs(readme): enrich with audio samples, hardware guide, and doc links (#1785)
* docs: polish README hero hierarchy

* docs: enrich README with audio samples, hardware guide, and doc links

* docs: address review findings on Docker port binding, MCP transport, and privacy

* docs: address CodeRabbit review feedback on cURL format and Colab links

* docs: align Docker quick-start with stable tag and named volume mount
2026-09-03 18:00:14 +05:30
Palash Debnath d80286562e docs: polish README hero hierarchy (#1780) 2026-09-03 14:28:42 +05:30
9d30774ee2 feat(audiobook): synced lyrics playback (#1766)
* feat(audiobook): synced-lyrics player

Replace the bare <audio controls> in the audiobook result with a player
that renders the chapter text and highlights the word under
audio.currentTime, karaoke-style. Timing reuses what the render stream
already emits — per-chapter duration_s on the chapter SSE events — with
words even-split inside each chapter (the karaoke burn-in's old-job
fallback, ported from services/karaoke_ass.py); after a reload the whole
book even-splits over the file's own duration. No ASR pass, no new
backend surface, nothing leaves the machine. Download keeps going
through the Tauri-safe downloadMedia util.

New pure helper utils/audiobookLyrics.js mirrors the longform parser's
chapter drop rules so cue indices line up with the stream's chapter
list, and degrades to the proportional split on any drift (script
edited after the render, stopped mid-book). Words are buttons —
click-to-seek, keyboard reachable — restyled as prose in index.css.
audiobook.lyrics translated in all 21 locales.

Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>

* docs(changelog): synced-lyrics audiobook player entry (#1766)

Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>

* style(audiobook): apply current formatter

* fix(audiobook): preserve synced render cues

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Palash Debnath <4178343+debpalash@users.noreply.github.com>
2026-09-02 16:32:23 +05:30
65236774aa feat(dub): project-level drag casting board (#1767)
* feat(dub): project-level drag casting board

Adds an expandable Casting Board to the dub editor's CAST strip: speaker
rows (with the auto-clone chip when the extractor found a usable passage)
and draggable voice chips — Default, clone profiles, design presets.
Dropping a chip on a speaker writes the exact fields the CAST <select>
always has (profile_id + merge_parts/merge_parts_original attribution),
via a shared assignSpeakerProfile helper both views now call, so the
dropdowns stay in sync and job persistence is unchanged. Keyboard path:
focus a speaker row, pick from a listbox (arrows/Enter/Escape).

The pre-existing CAST dropdown strip moves verbatim into the new
CastingBoard.jsx (DubLeftColumn shrinks below 800 lines; the new file
holds the 300-line soft cap). Styles extend the .dub-cast-* cluster in
index.css. Six new i18n keys translated in all 21 locales.

Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>

* docs: changelog + roadmap entries for the casting board (#1767)

Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>

* fix(casting): validate and preserve speaker assignments

* fix(casting): recover cleared merged assignments

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Palash Debnath <4178343+debpalash@users.noreply.github.com>
2026-09-02 15:37:57 +05:30
Palash Debnath 0ed7d2ec22 chore(release): prepare v0.5.2 (#1761)
Synchronize VoiceStudio release metadata, lockfiles, installers, container references, documentation, and the dated v0.5.2 changelog after all planned fixes landed.
2026-09-02 10:26:14 +05:30
Matt Van Horn a95041f1e6 feat(studio): speech-to-speech voice changer (#1765)
Add a bounded, local-first speech-to-speech Convert workflow with shared ASR/TTS admission, duration matching, stale-request cancellation, profile conditioning, watermarking, persistence, localization, and regression coverage.\n\nCo-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
2026-09-02 09:45:40 +05:30
Matt Van Horn 4053397921 feat(dub): karaoke word-highlight caption burn-in (#1764)
Adds opt-in word-timed ASS karaoke captions while preserving the existing line-caption default.\n\nCo-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
2026-09-02 08:32:43 +05:30
Palash Debnath e2446c3e61 fix(release): keep Preview ahead of Stable (#1763)
Closes #1762.
2026-09-02 07:49:22 +05:30
Palash Debnath ccd984f324 Merge pull request #1760 from agudmund/feat/mcp-output-mode-files
feat(mcp): output mode + base-path file lane so agents never carry audio in context
2026-09-02 06:27:46 +05:30
Palash Debnath 89f0a25082 fix(mcp): bound encoded audio before decode 2026-09-02 06:13:11 +05:30
Palash Debnath b351dc5b1a Merge remote-tracking branch 'origin/main' into review/pr-1760
# Conflicts:
#	CHANGELOG.md
2026-09-02 06:07:57 +05:30
Palash Debnath 026410fbc6 test(worker): make loop responsiveness checks deterministic (#1759)
Reviewed by CodeRabbit and Greptile. Required Tests (backend + frontend) gate passed on the current, mergeable head.
2026-09-02 05:43:08 +05:30
Palash Debnath b6a1ee50ff Merge remote-tracking branch 'origin/main' into fix/deterministic-inbound-loop-tests
# Conflicts:
#	tests/test_worker_inbound_transport.py
2026-09-02 05:29:25 +05:30
Ævar GuðmundssonandClaude Fable 5.1 3c3c37615c feat(mcp): output mode + base-path file lane so agents never carry audio in context
An LLM agent pays for every byte it receives, and generate_speech returned
each WAV as base64 inline - a short clip already brushed per-result limits.
This adds two knobs in the OMNIVOICE_* family, the pattern the ElevenLabs
MCP settled on (OUTPUT_MODE + a BASE_PATH security boundary):

- OMNIVOICE_MCP_OUTPUT_MODE = resources (default, the original contract) |
  files | both. In files mode generate_speech returns audio_url (the render
  the backend already keeps, served at /audio/<id>.wav) and, when a base
  path is set, output_path - the WAV written into that directory.
- OMNIVOICE_MCP_BASE_PATH: the one directory agents may read from and
  receive files in. transcribe(audio_path=) and clone_voice(ref_audio_path=)
  read only inside it (relative paths resolve against it, absolute ones must
  lie within it, symlinks resolved before the check); with no base path,
  path arguments are refused with a reason.
- OMNIVOICE_MCP_TIMEOUT_S (default 120): the tools' backend timeout, since a
  CPU host serializes generations and an agent queued behind another render
  outlasted the fixed budget with an empty-message ToolError.

Also: transcribe and clone_voice share one input helper (data-URI tolerance
now covers transcribe too), the upload filename carries the sniffed
extension, and the reply is built with json.dumps instead of hand-rolled
JSON. Tests cover the mode parsing, the boundary (escape and missing-base
refusals), both input lanes, all four reply shapes, and the timeout knob.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-01 23:46:44 +00:00
Palash Debnath c44f2fc042 test: make key revocation ordering deterministic (#1758)
Replaces a scheduler-sensitive 200 ms assertion with a deterministic ordering check against the blocked-write release barrier. Repairs the red post-merge main run from #1751.
2026-09-02 05:09:21 +05:30
Palash Debnath 62604ec9bb test(worker): make loop responsiveness checks deterministic 2026-09-02 04:56:29 +05:30
Palash Debnath 08569397d3 fix(asr): secure configured endpoints and refresh guidance (#1751)
Refreshes README and linked docs with accurate installation, platform, privacy, API, and model-license guidance; documents local gigastt; and pins OpenAI-compatible ASR traffic to the configured secure origin. Closes #1736.
2026-09-02 04:41:36 +05:30
Palash Debnath c9a468e500 Merge pull request #1756 from debpalash/fix/windows-direct-job-owner-1734
fix(windows): remove the sidecar supervisor hop
2026-09-02 04:07:36 +05:30
Palash Debnath aec4694628 test(windows): cover delayed Job exit 2026-09-02 03:52:17 +05:30
Palash Debnath 5df1dc973c Merge remote-tracking branch 'origin/main' into fix/windows-direct-job-owner-1734
# Conflicts:
#	CHANGELOG.md
2026-09-02 03:48:02 +05:30
Palash Debnath 8cc978588d Merge pull request #1754 from debpalash/fix/backend-startup-budget-1749
fix(frontend): align backend startup stall budget
2026-09-02 03:28:13 +05:30
Palash Debnath 6f80110a42 fix(windows): await timed-out Job teardown 2026-09-02 03:15:16 +05:30
Palash Debnath c5ac548100 Merge remote-tracking branch 'origin/main' into fix/windows-direct-job-owner-1734
# Conflicts:
#	CHANGELOG.md
2026-09-02 03:14:09 +05:30
Palash Debnath 26d7a333e8 Merge remote-tracking branch 'origin/main' into fix/backend-startup-budget-1749
# Conflicts:
#	CHANGELOG.md
2026-09-02 03:12:28 +05:30
Palash Debnath 75eb7c6099 test: make repository ID bound deterministic (#1757)
Replaces a runner-speed-dependent security assertion with a deterministic proof that oversized repository IDs are rejected before library validation. Repairs the red post-merge main run from #1755.
2026-09-02 02:51:20 +05:30
Palash Debnath 1d06c6f079 fix: accept ASR-detected dub source codes (#1755)
Accepts every Whisper language code persisted by ASR, including three-letter Cantonese yue, so subsequent dubbing uploads no longer fail validation. Closes #1737.
2026-09-02 02:26:56 +05:30
Palash Debnath 35d2e27b5e Classify RX 6700 XT WSL2 path as unverified (#1752)
Closes #1716.\n\nDefines supported, best-effort override, unverified, and unsupported architecture evidence; marks RX 6700 XT/gfx1031 over WSL2 ROCDXG as unverified; and requires routing, utilization, and CPU-fallback evidence before claiming acceleration.
2026-09-02 01:45:56 +05:30
Palash Debnath 267ded0e79 fix(windows): simplify nested job cleanup 2026-09-02 01:44:52 +05:30
Palash Debnath 00d923b4fa fix: repair incomplete Sherpa model caches (#1753)
Repairs missing and zero-byte Sherpa ONNX cache assets before model loading, with offline regression coverage. Closes #1733.
2026-09-02 01:42:03 +05:30
Palash Debnath 549a56fc2d fix: remove Windows sidecar supervisor hop 2026-09-02 01:38:05 +05:30
Palash Debnath 6e47b15e82 test: pin backend stall timeout boundary 2026-09-02 01:29:22 +05:30
Palash Debnath 5e08dde5e0 fix: align backend startup stall budget 2026-09-02 01:24:27 +05:30
Palash Debnath 3c488f7189 Add Trendshift repository badge (#1742)
Adds the repository owner-requested live Trendshift badge at the top of the README.
2026-09-02 00:55:29 +05:30
Palash Debnath 4e5e8d1f89 Allow OmniVoice slow sidecar startup (#1743)
Fixes #1711.\n\nGives only the OmniVoice subprocess a 120-second readiness budget while retaining the shared 30-second default for all other sidecars, with regression coverage.
2026-09-02 00:22:05 +05:30
Palash Debnath 497d57ee62 Show complete engine disk costs before install (#1728)
Closes #1718.

Adds structured pre-install and measured post-install disk costs, complete sidecar preflight accounting, strict authorization for recursive scans, and localized catalogue details with confidence and deduplication context.
2026-09-01 23:49:05 +05:30
Palash Debnath a4f68e4ea0 Preserve cloned voices during queued SRT imports (#1745)
Closes #1709.

Queues SRT selection until speaker analysis and clone extraction finish, then applies the newest selected subtitle file with stale-result, failure, replacement, retry, and abort guards.
2026-09-01 23:15:43 +05:30
Palash Debnath 51bbf50ce3 Ship a supported per-user Windows installer (#1730)
Closes #1713

Adds a separately identified per-user MSI and updater channel, non-administrator install/uninstall verification, and fail-closed WebView2 handling for current-user installs.
2026-09-01 22:03:33 +05:30
Palash Debnath 0c005abd07 Merge pull request #1731 from debpalash/fix/dictation-listener-registration-1707
Keep dictation events queued across listener reloads
2026-09-01 21:30:02 +05:30
Palash Debnath 67d7236edd fix: retain dictation events until listener acknowledgement 2026-09-01 21:03:35 +05:30
Palash Debnath 8f96692537 Merge remote-tracking branch 'origin/main' into fix/dictation-listener-registration-1707 2026-09-01 20:59:24 +05:30
Palash Debnath 60fe6cb814 Merge pull request #1727 from debpalash/fix/msi-webview2-control-1714
fix: make WebView2 MSI bootstrap controllable
2026-09-01 20:38:53 +05:30
Palash Debnath 88e2d4cb00 Merge remote-tracking branch 'origin/main' into fix/dictation-listener-registration-1707 2026-09-01 20:00:42 +05:30
Palash Debnath 05a8badf86 Merge remote-tracking branch 'origin/main' into fix/msi-webview2-control-1714 2026-09-01 19:43:47 +05:30
Palash Debnath 7d49faccdc Merge pull request #1724 from debpalash/fix/engine-execution-evidence-1717
Expose per-engine execution evidence
2026-09-01 19:09:44 +05:30
Palash Debnath 3b6a05c1a8 Merge remote-tracking branch 'origin/main' into fix/msi-webview2-control-1714
# Conflicts:
#	CHANGELOG.md
2026-09-01 18:58:13 +05:30
Palash Debnath ea4dcb6c61 Merge remote-tracking branch 'origin/main' into fix/dictation-listener-registration-1707 2026-09-01 18:56:06 +05:30
Palash Debnath 88e3e71fd7 Merge remote-tracking branch 'origin/main' into fix/engine-execution-evidence-1717 2026-09-01 18:47:30 +05:30
Palash Debnath a30b71666c Merge pull request #1725 from debpalash/fix/subtitle-timing-controls-1710
fix: make subtitle timing directly adjustable
2026-08-30 22:57:39 +05:30
Palash Debnath fb724b8633 fix: isolate diagnostic engine imports 2026-08-30 22:57:12 +05:30
Palash Debnath 0643965583 style: format dictation listener test 2026-08-30 22:53:49 +05:30
Palash Debnath 958c9de7b9 fix: lease dictation listener readiness 2026-08-30 22:44:11 +05:30
Palash Debnath 6dbfeda2e5 Merge remote-tracking branch 'origin/main' into fix/msi-webview2-control-1714 2026-08-30 22:32:25 +05:30
Palash Debnath 9447ace2c3 fix: invalidate unloaded ASR evidence 2026-08-30 22:21:18 +05:30
Palash Debnath 07cff18dfc Merge pull request #1726 from debpalash/fix/backend-parent-liveness-1707
fix: reap backend when desktop owner exits
2026-08-30 22:12:44 +05:30
Palash Debnath 7e777a3a0f fix(dubbing): accept exact minimum timing boundary 2026-08-30 22:12:28 +05:30
Palash Debnath 9fe68ddb3f docs(installer): qualify fail-closed WebView behavior 2026-08-30 22:10:10 +05:30
Palash Debnath cbc89e2d15 fix(diagnostics): track live execution evidence lifecycle 2026-08-30 22:09:05 +05:30
Palash Debnath 87b052489a Merge remote-tracking branch 'origin/main' into fix/backend-parent-liveness-1707 2026-08-30 21:53:15 +05:30
Palash Debnath e35c6ad987 fix(backend): document parent pipe closure 2026-08-30 21:53:15 +05:30
Palash Debnath 1ceebe5aa2 Merge remote-tracking branch 'origin/main' into fix/subtitle-timing-controls-1710 2026-08-30 21:52:56 +05:30
Palash Debnath 7658865735 fix(i18n): clarify Dutch timing labels 2026-08-30 21:52:56 +05:30
Palash Debnath 2c232ac863 Merge remote-tracking branch 'origin/main' into fix/msi-webview2-control-1714 2026-08-30 21:52:15 +05:30
Palash Debnath 2af47dbd4b fix(installer): make WebView2 download opt-in 2026-08-30 21:51:46 +05:30
Palash Debnath 6b5825c5a5 Merge remote-tracking branch 'origin/main' into fix/engine-execution-evidence-1717 2026-08-30 21:48:02 +05:30
Palash Debnath bfdb8dc7f5 docs: add private production API deployment (#1723)
* docs: add private production API deployment

* docs: harden private API deployment guidance

* docs: make proxy trust configuration executable

* docs: clarify private API trust boundaries

* docs: persist private service application data
2026-08-30 21:29:14 +05:30
Palash Debnath e9c2c451e7 test: resolve parent watchdog at execution time 2026-08-30 21:28:53 +05:30
Palash Debnath d4f89ecdc7 fix: clamp and localize subtitle steppers 2026-08-30 21:28:18 +05:30
Palash Debnath e1101255bd fix: make WebView2 MSI bootstrap controllable 2026-08-30 21:25:12 +05:30
Palash Debnath 0a20aeb0c9 fix: reap backend when desktop owner exits 2026-08-30 21:18:36 +05:30
Palash Debnath 7e50640f97 fix: make subtitle timing directly adjustable 2026-08-30 21:15:52 +05:30
Palash Debnath bab794e13b fix: report only observed engine execution 2026-08-30 21:12:54 +05:30
Palash Debnath 503e4ffde8 Merge remote-tracking branch 'origin/main' into fix/engine-execution-evidence-1717 2026-08-30 21:10:29 +05:30
Palash Debnath 80aafa3a53 fix: preserve startup and media failure evidence (#1722)
* fix: preserve startup and media failure evidence

* fix(dictation): bind readiness to installed revision

* test(dictation): align cache resolution contract

* fix: keep retry failures actionable
2026-08-30 20:49:53 +05:30
Palash Debnath 0ec9c074e8 fix: distinguish opaque loaded sidecars 2026-08-30 20:49:26 +05:30
Palash Debnath 2be73b43e3 feat: expose engine execution evidence 2026-08-30 20:40:35 +05:30
Palash Debnath 4e61c2782d fix(engines): harden skill discovery and CosyVoice setup 2026-08-29 05:10:35 +05:30
Palash Debnath d6a2277e6a Merge pull request #1700 from debpalash/chore/finalize-v0.5.1
chore(release): finalize v0.5.1
2026-08-28 20:21:25 +05:30
Palash Debnath a59fdee094 docs(changelog): preserve unreleased structure 2026-08-28 20:06:32 +05:30
Palash Debnath 3fe2018589 chore(release): finalize v0.5.1 2026-08-28 20:02:55 +05:30
Palash Debnath 00bf55c96f Merge pull request #1699 from debpalash/fix/mps-omnivoice-isolation-1697-1698
fix(macos): isolate OmniVoice MPS generation
2026-08-28 19:46:31 +05:30
Palash Debnath 1e4fdd12e0 fix(worker): preserve isolated OmniVoice synthesis settings 2026-08-28 19:32:20 +05:30
Palash Debnath 3b6e15dad5 fix(macos): isolate OmniVoice MPS generation 2026-08-28 19:21:06 +05:30
Palash Debnath de51120d6a fix(models): handle load OOMs safely (#1696)
* fix(models): handle load OOMs safely (#1695)

* fix(dub): sanitize streamed generation failures

* style(ui): format readiness checklist
2026-08-28 16:00:24 +05:30
Palash Debnath 3e5aa90e3f fix(dev): recover backend worker crashes (#1694)
Run uvicorn directly under the dev wrapper so a worker crash cannot hide behind a live reload parent. Preserve Python source reloads, restart isolated crashes with bounded diagnostics, and keep persistent crash loops loud.

Closes #1690.
2026-08-28 14:31:54 +05:30
Palash Debnath 8cc7c88692 fix(dub): restore audio when the media preview falls back (#1693)
Preserve audible, language-matched playback when WebView video decoding falls back, keep waveform timing aligned with the selected dub, prevent recursive recovery failures, and disable stale audio caching.

Closes #1692.
2026-08-28 13:37:34 +05:30
Palash Debnath e7e97e4b4e fix(ui): give Model Catalogue engine rows room (#1691)
Closes #1689.

Uses the Catalogue workspace width for readable engine rows, keeps status badges out of engine names, and collapses by the named Tauri-scaled container. Compact Settings rows remain unchanged.
2026-08-28 12:38:16 +05:30
Palash Debnath 97bfbfecd1 chore(release): prepare v0.5.1 (#1688)
Prepare the tested main branch for the v0.5.1 patch release with synchronized version sources, mirrors, lockfiles, release notes, and install guidance.

Closes #1687.
2026-08-28 11:27:59 +05:30
Palash Debnath 0c8a40774f feat(ui): give Model Catalogue room to breathe (#1686)
Redesign Model Catalogue as one responsive workspace canvas with a compact wide-screen header, simplified navigation, and a single primary scroll plane.

Closes #1685.
2026-08-28 10:50:45 +05:30
Palash Debnath f45c885171 fix(ui): reflow workspaces after bootstrap (#1684)
* fix(ui): reflow shell after bootstrap

* fix(ui): serialize native scale updates

* test(ui): mount responsive shell after bootstrap
2026-08-28 09:54:01 +05:30
Palash Debnath da8358916c fix(desktop): preflight Linux runtime dependencies (#1681)
Fail before backend/window startup when Linux source hosts lack Enigo’s libxdo linker input or WebKit’s GStreamer audio sink. Print an exact distro package command, sync source-build docs, and lock the probes with deterministic tests.

Closes #1680
Closes #1682
2026-08-28 07:19:36 +05:30
Palash Debnath 0268f43e7f fix(dub): keep language and media tools ready (#1679)
Fixes #1677 and #1678.

Publishes first-run media tools to the live backend, provides precise cross-platform missing-process guidance, and keeps localized source-language selection available before transcription. Includes regression coverage and deterministic model-store test isolation.
2026-08-28 06:32:49 +05:30
Palash Debnath 6d42d2255e Merge pull request #1675 from debpalash/fix/runtime-stability-1652
fix(runtime): prevent overlapping native work and false crashes
2026-08-27 22:45:11 +05:30
Palash Debnath 1103898d7b Merge remote-tracking branch 'origin/main' into fix/runtime-stability-1652
# Conflicts:
#	CHANGELOG.md
2026-08-27 22:30:58 +05:30
Palash Debnath 28220ff2dd Merge pull request #1672 from debpalash/fix/dub-srt-voices-1660
fix(dub): preserve workflow state and browser compatibility
2026-08-27 22:12:52 +05:30
Palash Debnath fa9bfd41be fix(stories): serialize marker preview synthesis 2026-08-27 21:57:41 +05:30
Palash Debnath 0e47053823 fix(dub): scope imported speaker clones to matched cues 2026-08-27 21:57:37 +05:30
Palash Debnath bea72cadbe Merge remote-tracking branch 'origin/main' into fix/runtime-stability-1652
# Conflicts:
#	CHANGELOG.md
2026-08-27 21:50:51 +05:30
Palash Debnath 6624b3fb19 Merge remote-tracking branch 'origin/main' into fix/dub-srt-voices-1660
# Conflicts:
#	CHANGELOG.md
2026-08-27 21:50:46 +05:30
Palash Debnath fc603bcc92 Merge pull request #1673 from debpalash/fix/docker-admin-auth-1651
fix(docker): secure admin access and document WSL ROCm
2026-08-27 21:28:11 +05:30
Palash Debnath 2300d4d5f7 fix(tts): localize shared admission failures 2026-08-27 21:27:40 +05:30
Palash Debnath c7890d2a5a fix(dub): close final media and retry review gaps 2026-08-27 21:25:56 +05:30
Palash Debnath 633e991edc Merge remote-tracking branch 'origin/main' into fix/runtime-stability-1652
# Conflicts:
#	CHANGELOG.md
2026-08-27 21:09:34 +05:30
Palash Debnath 64b3785b0b Merge remote-tracking branch 'origin/main' into fix/docker-admin-auth-1651
# Conflicts:
#	CHANGELOG.md
2026-08-27 21:09:30 +05:30
Palash Debnath 4b3c62d783 Merge remote-tracking branch 'origin/main' into fix/dub-srt-voices-1660
# Conflicts:
#	CHANGELOG.md
2026-08-27 21:09:27 +05:30
Palash Debnath 103a5dbbe4 Merge pull request #1674 from debpalash/fix/voice-clone-reference-lease-1668
fix(generate): lease ad-hoc references across abandoned jobs
2026-08-27 20:47:00 +05:30
Palash Debnath 86f6c9ec8e fix(auth): reject blank server admin keys 2026-08-27 20:46:24 +05:30
Palash Debnath e1a76f9aca fix(dub): close shared workflow review gaps 2026-08-27 20:43:21 +05:30
Palash Debnath 57263edad7 fix(tts): centralize single-generation admission 2026-08-27 20:38:29 +05:30
Palash Debnath 4c0a0a26b7 fix(runtime): bound external debugger restarts 2026-08-27 20:26:52 +05:30
Palash Debnath 634936eb10 fix(rocm): diagnose disabled WSL DXG detection 2026-08-27 20:26:09 +05:30
Palash Debnath 422dbd1313 fix(dub): address cast, language, and cancellation review 2026-08-27 20:25:16 +05:30
Palash Debnath d23f22d56b test(generation): assert cancelled reference workers 2026-08-27 20:22:19 +05:30
Palash Debnath eaa87ff041 fix(docker): require the Studio administrator key 2026-08-27 20:21:42 +05:30
Palash Debnath 0ee9bc35d0 fix(runtime): prevent overlapping native work and false crashes 2026-08-27 20:20:58 +05:30
Palash Debnath a1cd15964c Merge remote-tracking branch 'origin/main' into fix/voice-clone-reference-lease-1668 2026-08-27 20:10:51 +05:30
Palash Debnath 895e62df7c Merge remote-tracking branch 'origin/main' into fix/docker-admin-auth-1651 2026-08-27 20:10:51 +05:30
Palash Debnath 49c175301a Merge remote-tracking branch 'origin/main' into fix/dub-srt-voices-1660 2026-08-27 20:10:07 +05:30
Palash Debnath b92d35ac5d feat: add local speech platform (#1671)
Fixes #1646
2026-08-27 19:51:07 +05:30
Palash Debnath b6bd125f23 fix(dub): keep preview extraction off event loop 2026-08-27 19:45:31 +05:30
Palash Debnath 53331a6f5a fix(dub): normalize uploaded video for preview 2026-08-27 19:45:10 +05:30
Palash Debnath 1d445855d2 fix(dub): make language workflow reliable 2026-08-27 19:43:49 +05:30
Palash Debnath 9dc01d4b8b fix(docker): require admin key in quick starts 2026-08-27 19:31:55 +05:30
Palash Debnath e93b6366e6 fix(dub): preserve voices across SRT imports 2026-08-27 19:14:56 +05:30
Palash Debnath 89cee3f824 fix(generate): lease ad-hoc references across abandoned jobs 2026-08-27 19:06:51 +05:30
Parker Hill 078bad8e0f fix(setup): repair fresh-clone desktop and ROCm source installs (#1664, #1665)
Fix fresh-clone desktop development by creating the required dist placeholder before Tauri starts, and make source setup install the selected CUDA or ROCm PyTorch stack consistently. Adds behavior-level cross-platform regression coverage.\n\nFixes #1664.\nFixes #1665.\n\nThanks @uberclokr for the contribution.
2026-08-27 18:33:55 +05:30
Paolo Antinori 032c5aba86 fix(core): macOS fallback for os.waitid ownership probe (#1656)
Fixes #1656. Adds the safe waitpid fallback, proves child ownership before process-group signals, and credits @paoloantinori.
2026-08-27 17:23:25 +05:30
Palash Debnath a8371baaa8 fix(desktop): serialize backend lifecycle (#1635)
Closes #1635.
2026-08-24 18:11:37 +05:30
Palash Debnath 5a615d2c66 feat(workers): package headless GPU nodes (#1638) (#1648)
Closes #1638.\n\nPackages headless GPU workers with durable enrollment, bounded artifact handling, cross-platform lifecycle cleanup, and regression coverage. Incorporates CodeRabbit, Greptile, CodeQL, and platform-CI findings before merge.
2026-08-24 16:32:56 +05:30
Palash Debnath 12480b81b1 fix(persistence): move longform projects to IndexedDB (#1636) (#1650)
Move unbounded Stories and Audiobook project data to revisioned IndexedDB storage with bounded local fallback, durable clear tombstones, migration/recovery safeguards, and deadline-safe persistence before exits and relaunches.

Closes #1636.
2026-08-24 12:59:24 +05:30
Palash Debnath ef1cb57944 fix: route OmniVoice to ROCm GPUs (#1647) 2026-08-24 03:25:56 +05:30
Palash Debnath 6447788fdf fix(dictation): cancel widget work on unmount (#1645)
Fix the post-merge Vitest failure by invalidating CaptureWidget startup continuations and releasing timers, sockets, recorder/worklet state, and microphone streams on teardown. Adds fail-before/pass-after coverage for active capture, pending fallback, delayed microphone permission, and delayed WebSocket authentication.
2026-08-24 02:44:00 +05:30
Palash Debnath afa361913c fix(generate): surface the real cause on streaming failures (#1633)
Classify and journal local and remote streaming generation failures, return actionable scrubbed guidance, and keep exception details, tokens, and user paths out of logs and NDJSON responses.
2026-08-23 15:55:05 +05:30
Palash Debnath 0d3c81b596 feat(gguf): add Linux ARM64 runtime support (#1641)
Add linux-aarch64 platform detection, Vulkan-preferred source builds with CPU fallback, ARM64-safe PyTorch dependency markers, native artifact CI, regression coverage, and synchronized architecture documentation.
2026-08-23 15:07:52 +05:30
Palash Debnath 98c9e68aae test(worker): carry the capacity limit through the handshake, not a post-connect mutation (#1630)
The at-capacity test set client.config.max_concurrent_tasks = 1 after
connect_worker, racing the server's stream-open ConfigUpdate — which
carries the REGISTERED capacity (2, from the hello). When the frame
lands after the mutation (slow CI runners), the override is clobbered,
the worker honours 2 slots, accepts the second assignment, and the test
reports over-concurrency that never existed. Failed CI twice on
2026-08-21 (#1627 first run + main).

Fix: pass the limit through WorkerConfig so hello -> registration ->
ConfigUpdate all agree from the start; assert the registered capacity
as a regression guard. Demonstrated fail-before: with the old pattern
the server record stays 2 and a delivered config frame reverts the
client gate to 2.
2026-08-21 20:36:12 +00:00
debpalash 772e3e82b4 feat(uninstall): --app/-RemoveApp removes the prebuilt binary install
The default curl|sh and irm|iex installs now put a real app on disk
(/Applications or ~/Applications, ~/.local/bin/VoiceStudio, MSI product).
Both uninstallers gain an opt-in flag that targets exactly those:
- uninstall.sh --app: adds the app bundle / AppImage to the dry-run plan
- uninstall.ps1 -RemoveApp: resolves the MSI product across HKLM/HKCU/
  WOW6432Node and uninstalls it silently under -Yes
2026-08-22 01:58:39 +05:30
debpalash 228019c9a8 fix(install): usage text works through a curl pipe ($0 is not a file) 2026-08-21 21:08:06 +05:30
Palash Debnath e38b5f5741 feat(install): binary-first installer with version picking and --source opt-in (#1628)
* feat(install): prebuilt-app installs by default, --source opt-in, --version picker

- install.sh: default mode now downloads the verified release asset
  (dmg/AppImage + SHA256SUMS check) instead of cloning and building;
  --source keeps the previous clone-and-build flow; --version pins a release
- install.ps1: same split — msi download with checksum verification and
  setup wizard by default; -Source (or VOICESTUDIO_INSTALL_MODE) for the
  source build; VOICESTUDIO_VERSION picks a release
- worker landing page documents the modes

* fix(install): CI smoke covers binary + source modes; hdiutil output parse

- drop -quiet from hdiutil attach (it suppresses the mount-point line the
  script parses — caught by the macOS smoke)
- install.ps1 runs msiexec silently under CI, wizard interactively
- smoke verifies binary installs per OS (app bundle / AppImage / MSI
  registry entry) and keeps full source coverage behind --source

* ci(install): check HKCU/WOW6432Node too — Tauri MSI registers per-user

* fix(install): rename $version — collides with bun installer's $Version under iex
2026-08-21 14:30:33 +00:00
debpalash f60f5f1f59 chore(install): route voicestudio.sh/install* to the installer worker 2026-08-21 17:31:50 +05:30
Palash Debnath 7640f42dce feat(install): one-command installer URL (.sh + .ps1) + 3-OS install smoke (#1627)
* feat(install): one-command installer URL (.sh + .ps1) + 3-OS install smoke

- scripts/install.ps1: Windows source installer (winget deps, uv, bun,
  clone, uv sync, frontend build); honors OMNIVOICE_PYTHON/OMNIVOICE_REGION
- infra/install-redirect: Cloudflare Worker serving /install with
  User-Agent sniffing (curl -> sh, PowerShell -> ps1, browser -> landing
  page); proxies live from main; /install.sh + /install.ps1 aliases
- scripts/install.sh: fix stale advertised URL (main/install.sh never
  existed) and repo-root resolution so a local run no longer clones a
  duplicate repo into ~/VoiceStudio (verified on macOS arm64)
- .github/workflows/install-smoke.yml: run both installers end-to-end on
  ubuntu/macos/windows when they change
- docs-sync: install one-liners lead each platform guide; STRUCTURE.md

(#1626)

* fix(install): don't let a failed bun download pass silently

curl | sh runs an empty script and exits 0 when the download fails, so
a bun.sh hiccup surfaced much later as 'bun: command not found' (seen
on the macos-latest smoke runner). Fetch to a temp file, verify, fall
back to npm -g bun when node exists, and die with the manual command.
Same post-install verification for uv.

* fix(install): UTF-8 BOM for install.ps1 + quiet-style changelog entry

- tests/scripts/test_uninstall_ping.py requires shipped PowerShell
  scripts with non-ASCII text to carry a UTF-8 BOM (Windows PowerShell
  5.1 mis-decodes otherwise); same treatment uninstall.ps1 already gets
- test_changelog_style caps entries at ~400 chars
2026-08-21 11:47:39 +00:00
Palash Debnath 3eeed0cfe9 chore: refresh application dependencies (#1625)
Refreshes frontend, tooling, and Python dependencies; regenerates the frozen lockfiles and worker protocol stubs. Keeps compatibility caps for FastAPI, Oxlint, and Vitest where newer releases break repository contracts. Updates the Uvicorn bind-failure regression test for its new nonzero exit code. Fully tested after merging current main.
2026-08-21 10:30:49 +00:00
Muhammad Ahmad Ali b946fded12 fix(dub): preserve speaker attribution across segment edits (#1624)
Adds per-line subtitle timing, insertion, and bidirectional merge controls while preserving each speaker's attribution across merge/split, restore, translation, and cast edits. Includes regression coverage and localization updates. Closes #1612.
2026-08-21 09:55:04 +00:00
Palash Debnath 7718a7a10b fix: cross-platform dictation delivery (#1610)
Makes dictation delivery, capture, recovery, model fallback, AEC, and localized status behavior reliable across macOS, Windows, and Linux.
2026-08-21 03:33:50 +00:00
Palash Debnath 0687e13b57 test(perf): performance regression budgets as operation-count guards (#1622)
Adds deterministic operation-count regression budgets for streaming TTS, cached dub re-mixes, and native batch dubbing.
2026-08-21 02:53:15 +00:00
Palash Debnath 89d585a36e perf(dub,stream): reuse cached segments, batch the default engine, report real TTFA (#1620)
Reuses verified cached segments, safely batches default-engine dubbing, and reports synthesis-only TTFA/RTF.
2026-08-20 21:12:04 +00:00
Paolo Antinori 3f5114923b feat(pockettts): opt-in 24-layer checkpoints via OMNIVOICE_POCKETTTS_24L (#1613)
Adds an opt-in 24-layer PocketTTS checkpoint path, with French correctly using its required 24-layer model.
2026-08-20 20:42:40 +00:00
Palash Debnath 43f1d46fe6 fix(indextts): accept the config name upstream ships, and keep long text alive (#1619)
* fix(indextts): accept the config name upstream ships, and keep long text alive

Two independent defects, both reported on a working IndexTTS 2.5 install.

Install always failed. IndexTeam/IndexTTS-2.5 ships the model config as
config.yaml — at the pinned revision d0aa86e7 and at HEAD; config_v2_5.yaml
exists in no upstream revision. VoiceStudio demanded that name, so
_weights_floor_ok never found it and the install died claiming 'the download
was likely interrupted' when the download had been perfect. The only way
through was to hand-rename the file. Both names are accepted now, in the
installer and on the load path, so installs created with the workaround keep
working without a reinstall.

Long text was killed at 60s. infer() is one blocking upstream call that puts
nothing on the wire, and IndexTTS was the only sidecar still on the 60s
recv_timeout_s class default while pockettts and omnivoice-subprocess had both
raised theirs. Raising the default alone does not fix it — which is why the
reporter's RECV_TIMEOUT_S=3600 edit didn't help: progress frames are also what
report activity to the GPU pool's execution clock (#1367), so a silent sidecar
still trips the outer generate budget. The sidecar now heartbeats every 5s
while infer() runs (and during the cold model construction), _send takes a
lock so the beat thread can't interleave framing, and the deadline rises to
900s via OMNIVOICE_INDEXTTS_RECV_TIMEOUT_S.

test_indextts25_health_requires_25_config_name asserted the bug — that a
checkout holding only config.yaml is unhealthy — so it is rewritten to the
corrected contract, including that a genuinely truncated download is still
caught.

Fixes #1611

* test(indextts): follow the installed config name in the sidecar loader tests

Two more tests encoded the config_v2_5.yaml assumption, both asserting
cfg_path against a directory where no config existed at all — so they were
pinning the literal name rather than the resolution. They now lay down a real
checkpoints/ tree and assert the resolved path, including that a checkout
carrying the pre-fix hand-renamed config still resolves.

Caught by the full suite; the targeted runs during development did not reach
tests/backend/services/.

* test(indextts): event-driven heartbeat tests, real interleaving proof, precedence pin

Review round on #1619 — all four findings taken.

- The docs line naming 0.5.1 is version-neutral now ('Earlier installs') —
  version labels are the owner's call.
- The heartbeat tests waited on wall-clock sleeps; they now block on a
  per-write Event with a bounded deadline, so scheduler load can't flake them.
- The _send test asserted the lock EXISTS — a tautology. It now drives four
  concurrent writers through a stream that yields between every byte and
  asserts every frame decodes; verified fail-before by removing the lock
  (torn frame) and pass-after.
- The precedence test deleted config.yaml before creating the renamed one, so
  reversed precedence still passed. Both files now coexist for the assertion;
  verified fail-before by reversing _CFG_NAMES.
2026-08-20 19:43:52 +00:00
Paolo Antinorianddebpalash 3223a20f88 fix(openai-compat): reuse cached engine instances in _resolve_engine (#1614)
* fix(openai-compat): reuse cached engine instances in _resolve_engine

The direct engine-ID path in /v1/audio/speech constructed a fresh
backend per request (return cls()). For SubprocessBackend engines that
meant: a new sidecar process, a full torch import and an engine model
reload on EVERY request (measured ~28s floor per pockettts request on
an M3 Pro), plus another atexit hook registration each time — exactly
what get_engine_instance_for()'s docstring warns against.

Route the explicit-ID path through the same cached-singleton seam the
active-engine path already uses. Unknown/unavailable IDs keep their
400s; tts-1/tts-1-hd and the OmniVoiceBackend special case are
unchanged.

* fix(openai-compat): unload the outgoing engine on explicit-ID switches

Review follow-up (Greptile/CodeRabbit on #1614): caching instances without
a switch rule would let each distinct explicit engine ID stay resident,
accumulating sidecars / multi-GB in-process models. Mirror
get_active_tts_backend's MM2-01 switch rule: a different explicit ID
(omnivoice included, which resolves to the active engine) unloads the
outgoing instance first, best-effort.

* fix(openai-compat): evict via the shared single-engine-resident seam, not a router-local cache

The explicit-ID unload cache (13c14e2c) kept its own instance ref keyed by
model id. The shared engine cache is deliberately keyed by CLASS (registry
rebinds, idle sweeps and engine_memory eviction all mutate it), so the
router's id-keyed ref could go stale and keep serving an instance the
lifecycle system no longer tracked — caught by
test_openai_speech_toggle_off_sends_raw_text in full-suite order, and it
also introduced a novel unload path that ignored the
OMNIVOICE_SINGLE_ENGINE_RESIDENT opt-out.

Drop the router-local cache entirely: _resolve_engine returns the shared
cached singleton (get_engine_instance_for), and create_speech calls
evict_other_tts_engines(backend.id) before warming the engine — the exact
seam /generate uses. That covers every transition (explicit id → explicit
id, explicit id → tts-1/omnivoice aliases), honors the policy opt-out, and
leaves no per-router state to drift. Regression pinned at the route level in
test_speech_request_evicts_other_resident_engines.

* chore(changelog): trim the #1614 entry to the one-liner limit

415 chars against the 400 the style test allows — CI would have failed on it.

---------

Co-authored-by: debpalash <4178343+debpalash@users.noreply.github.com>
2026-08-20 19:14:10 +00:00
Palash Debnath 2d37627ab2 fix(setup): tolerate reserved memory in the RAM preflight, add OMNIVOICE_RAM_PREFLIGHT=0 escape hatch (#1621)
* fix(setup): tolerate reserved memory in the RAM preflight, add OMNIVOICE_RAM_PREFLIGHT=0 escape hatch (#1618)

An "8 GB" machine reports ~7.8 GB usable (firmware/iGPU/kernel
reservations), so comparing OS-reported RAM against the marketing-size
8 GB threshold hard-blocked exactly the boundary hardware the minimum is
meant to admit — with no way past the wizard. Both thresholds are now
compared with a 7% reserved-memory allowance, and
OMNIVOICE_RAM_PREFLIGHT=0 downgrades a genuine fail to a warning for
users who accept the OOM risk (same opt-out shape as
OMNIVOICE_ASR_VRAM_PREFLIGHT).

Regression tests: backend/tests/test_ram_preflight_1618.py.
Docs: troubleshooting §1c.

* review: hermetic preflight stubs in tests; correct the escape-hatch doc

Greptile P1: the Settings panel can't set OMNIVOICE_RAM_PREFLIGHT (and the
blocker appears before setup completes anyway) — the doc now points at
PowerShell / shell env only.
CodeRabbit: stub _network_check and media_tools.summary so each RAM
assertion stays fast and offline (26s -> 6s locally).
2026-08-20 19:03:50 +00:00
Palash Debnath 54a88f694b fix(watermark): run AudioSeal eagerly instead of through torch.compile (#1617)
* fix(watermark): run AudioSeal eagerly instead of through torch.compile

AudioSeal vendors moshi's @torch_compile_lazy on SEANetEncoder.forward, so
the first embed of a session — not the model load, which #1576's prefetch
already warms — called torch.compile and dropped into Inductor's C++ codegen.
On a macOS arm64 deployment that compile raised CppCompileError on 10/10
takes: the embed fail-opened and the audio shipped UNMARKED, an EU AI Act
Art. 50(2) provenance gap, after burning 30-40s on the first take and 5-8s on
each later one.

The compile is pure cost even where it succeeds. Measured on an M3 (5s of
24kHz audio, three consecutive embeds): compiled 9.70/0.26/0.23s vs eager
0.30/0.28/0.27s — a ~10s first-embed tax to save ~0.03s afterwards, on CPU
work already bounded by the 30s chunk loop. Both embed and detect now run
inside audioseal's own no_compile() switch, restored on the way out (it is a
process global, and other models are entitled to compile).

Verified end-to-end: first embed 9.70s -> 0.26s, watermark still round-trips
at confidence 1.0 with the OmniVoice message intact.

Fixes #1615

* fix(watermark): collapse the eager-guard globals into one lock-guarded state

CodeQL flagged _eager_saved's module-level initializer as dead, and it was
right: depth 0->1 always writes the field before depth 1->0 reads it, so the
None at import was never observed. Depth and saved-value are only meaningful
together and only under _eager_lock, so they become one dict rather than two
module scalars — which also drops the global statement.

Also splits three semicolon-joined statements in the regression test (Ruff
E702, CodeRabbit).

Mutation re-checked after the refactor: a naive no_compile() body still fails
with 'compile was handed back mid-embed'.
2026-08-20 18:16:00 +00:00
Palash Debnath 3441201be0 fix(dictation): clear granted accessibility blocker (#1609)
* fix(dictation): refresh accessibility blocker

* docs(changelog): note accessibility refresh

* test(dictation): assert the native widget hide on accessibility grant

The recheck regression asserted only that the Accessibility pill text left
the DOM, so it still passed with hideWidgetWindow() removed and the native
capsule stranded on screen. Hold one stable getCurrentWindow().hide spy and
assert it after the poll (fails before the fix, passes after).
2026-08-20 17:10:56 +00:00
Palash Debnath de5d848189 Merge pull request #1598 from debpalash/integrate/open-repairs-20260820
merge: land reviewed repair train
2026-08-20 05:16:03 +00:00
debpalash aa7c2f5801 fix: fail open on watermark dispatch deadlines 2026-08-20 10:32:02 +05:30
debpalash b7f14ce4ad fix: bind deadlines to selected capability device 2026-08-20 10:17:31 +05:30
debpalash 7a928da1a0 fix: size remote deadlines for worker device 2026-08-20 10:14:50 +05:30
debpalash 0961a5e512 docs: explain deadline capability fallback 2026-08-20 10:06:40 +05:30
debpalash 0619df8dff fix(cloning): retain selected passage volume 2026-08-20 10:04:54 +05:30
debpalash a91b27b518 test: make cross-loop event delivery deterministic 2026-08-20 10:01:41 +05:30
debpalash 605236566c fix: align generation routing and CPU deadlines 2026-08-20 09:59:47 +05:30
debpalash 918c400f29 fix: preserve audio on watermark teardown cancellation 2026-08-20 09:48:43 +05:30
debpalash 76d16ac1bd fix: fail open on watermark shutdown submission race 2026-08-20 09:43:58 +05:30
debpalash f22606f3ad Merge PR #1600 follow-up: enforce reference bound without preprocessing 2026-08-20 09:36:57 +05:30
debpalash f8492dd676 fix(cloning): cover the fifteen-second boundary 2026-08-20 09:36:31 +05:30
debpalash f6afa43d07 Merge PR #1600: bound long cloning references
# Conflicts:
#	CHANGELOG.md
2026-08-20 09:31:02 +05:30
debpalash 835a889326 fix(cloning): bound exhaustive reference selection 2026-08-20 09:30:16 +05:30
debpalash 2f3888549b Merge PR #1606: isolate workspace DOM during rapid navigation
# Conflicts:
#	CHANGELOG.md
2026-08-20 09:18:01 +05:30
debpalash e9e4d95d06 perf(cloning): bound reference ASR candidates 2026-08-20 09:17:43 +05:30
debpalash 2048d2793a Merge PR #1604: keep healthy CPU synthesis past five minutes
# Conflicts:
#	CHANGELOG.md
2026-08-20 09:17:30 +05:30
debpalash 52ce462396 test: exercise delayed workspace cleanup 2026-08-20 09:14:09 +05:30
debpalash e300739d78 fix(cloning): select bounded speech with ASR 2026-08-20 09:13:42 +05:30
debpalash 7efae54cf8 fix: budget the selected engine device 2026-08-20 09:13:18 +05:30
debpalash 0257bfcfec fix: isolate workspace DOM lifecycles 2026-08-20 09:10:07 +05:30
debpalash 933c1a2cf1 Merge PR #1605: add resizable Dub editor columns 2026-08-20 09:09:59 +05:30
debpalash c59a787a10 fix(dub): follow RTL splitter direction 2026-08-20 09:07:15 +05:30
debpalash ed6d7a9652 fix: preserve explicit generation watchdog 2026-08-20 09:06:50 +05:30
debpalash 3a1013527f Merge PR #1601 follow-up: normalize original-only exports 2026-08-20 09:04:00 +05:30
debpalash 243220fc3a feat(dub): resize editor columns 2026-08-20 09:03:14 +05:30
debpalash d57b4babc5 fix(cloning): compare trimmed reference activity 2026-08-20 09:02:05 +05:30
debpalash c198a8349a fix: allow bounded CPU synthesis time 2026-08-20 09:01:43 +05:30
debpalash c4f3ca457d Merge latest PR #1599 original-only normalization 2026-08-20 09:00:43 +05:30
debpalash e1e3a477a7 fix: normalize original-only dub exports 2026-08-20 09:00:34 +05:30
debpalash e20add344c Merge PR #1601: close integration review findings 2026-08-20 09:00:01 +05:30
debpalash 0a07202634 Merge PR #1603: dispatch foreign loops through serving loop 2026-08-20 08:57:25 +05:30
debpalash 3cf1007f28 fix(cloning): recover speech after silent trim 2026-08-20 08:57:08 +05:30
debpalash 1044483edf Merge latest PR #1599 trusted output paths 2026-08-20 08:56:29 +05:30
debpalash a04c972b71 fix: keep dub export paths trusted 2026-08-20 08:56:20 +05:30
debpalash 8c1afe6d9d Merge PR #1602: harden packaged frontend recovery 2026-08-20 08:56:16 +05:30
debpalash 809314a459 fix(cloning): select densest bounded speech passage 2026-08-20 08:56:09 +05:30
debpalash 2dcfd0bb55 Merge remote-tracking branch 'contributor/fix/watermark-prefetch-cold-start' into fix/integration-eventbus-loop-clean 2026-08-20 08:53:51 +05:30
debpalash 93616a9c2a fix(watermark): fail open while pool drains 2026-08-20 08:53:42 +05:30
debpalash 4072ec3db4 fix(desktop): retain live shell during backup cleanup 2026-08-20 08:52:36 +05:30
debpalash 0548386cb3 Merge latest PR #1599 effective default fix 2026-08-20 08:52:15 +05:30
debpalash 45ec840ead fix(dubbing): resolve effective export track once 2026-08-20 08:52:07 +05:30
debpalash fcc6e4a843 fix(cloning): retain active speech in bounded references 2026-08-20 08:51:04 +05:30
debpalash 99a98eaefe Merge trusted #1599 download labels 2026-08-20 08:50:07 +05:30
debpalash f72439d6cf fix(security): use trusted audio download labels 2026-08-20 08:49:57 +05:30
debpalash a355ad4ab6 Merge remote-tracking branch 'contributor/fix/event-bus-threadpool-emit' into fix/integration-eventbus-loop-clean 2026-08-20 08:49:38 +05:30
debpalash daefad8769 Merge remote-tracking branch 'contributor/fix/watermark-prefetch-cold-start' into fix/integration-eventbus-loop-clean
# Conflicts:
#	CHANGELOG.md
2026-08-20 08:49:38 +05:30
debpalash afe013a6bc fix(events): dispatch foreign loops through serving loop 2026-08-20 08:48:13 +05:30
debpalash f0764532e2 Merge PR #1601 follow-up: keep display labels outside path analysis 2026-08-20 08:47:11 +05:30
debpalash d11d608c2d Merge latest PR #1599 CodeQL fix 2026-08-20 08:46:58 +05:30
debpalash 109199e024 fix(security): keep download labels out of path boundary 2026-08-20 08:46:46 +05:30
debpalash 9d133870e5 fix(watermark): isolate lifecycle state between tests 2026-08-20 08:46:29 +05:30
debpalash 0d9a392e8d test(desktop): resolve Bash portably 2026-08-20 08:45:27 +05:30
debpalash e6284a5d5f fix(desktop): recover interrupted frontend swap 2026-08-20 08:45:27 +05:30
debpalash 69567e2e56 Merge PR #1601: fix integration migration and initial-load findings 2026-08-20 08:45:26 +05:30
debpalash 8e98e7a1be fix(frontend): preserve migration and retry semantics 2026-08-20 08:44:48 +05:30
debpalash b0785c4e6d Merge updated PR #1599 into integration findings 2026-08-20 08:43:59 +05:30
debpalash 5baf82bfb9 fix(security): validate audio export labels 2026-08-20 08:43:30 +05:30
debpalash f5d33aad8c Merge PR #1599: default exported video to dubbed audio
# Conflicts:
#	CHANGELOG.md
2026-08-20 08:35:57 +05:30
debpalash 539309ea84 fix(cloning): bound long reference audio 2026-08-20 08:35:21 +05:30
debpalash e450a37d4b fix(security): separate export labels from paths 2026-08-20 08:34:38 +05:30
debpalash 77b66abd94 Merge PR #1577 follow-up: serialize watermark pool restarts 2026-08-20 08:34:33 +05:30
debpalash 1a39061849 test(watermark): isolate executor lifecycle 2026-08-20 08:33:39 +05:30
debpalash dd1aa3654d fix(dubbing): default exports to dubbed audio 2026-08-20 08:26:57 +05:30
debpalash 8430f9843c Merge PR #1597: ship LAN web UI in desktop installs
# Conflicts:
#	CHANGELOG.md
2026-08-20 08:21:46 +05:30
debpalash 3299d5986b Merge PR #1596: accept omnivoice ASR alias on ROCm
# Conflicts:
#	CHANGELOG.md
2026-08-20 08:21:28 +05:30
debpalash a53ddc35a8 Merge PR #1595: isolate desktop cleanup script tests 2026-08-20 08:21:11 +05:30
debpalash 62eddfad41 Merge PR #1584: improve dubbing detection, timing, and layout
# Conflicts:
#	CHANGELOG.md
2026-08-20 08:21:08 +05:30
debpalash 5462eeacaa Merge PR #1562: deliver sync endpoint WebSocket events 2026-08-20 08:20:52 +05:30
debpalash 9aa01d4e72 Merge PR #1577: background-prefetch AudioSeal watermark generator 2026-08-20 08:20:48 +05:30
debpalash 17d181e427 fix(watermark): reopen pool per app lifespan 2026-08-20 08:18:24 +05:30
debpalash 1762c57355 fix(watermark): block replacement during shutdown 2026-08-20 08:14:03 +05:30
Palash Debnath e3eda1af2c Merge branch 'main' into fix/issue-1582-omnivoice-asr-alias 2026-08-20 02:42:29 +00:00
debpalash 40b3c4f460 Merge remote-tracking branch 'origin/main' into pr-1584 2026-08-20 08:11:51 +05:30
debpalash eed841a8ca fix(bootstrap): replace packaged frontend safely 2026-08-20 08:09:15 +05:30
debpalash 49ab178db7 Merge remote-tracking branch 'origin/main' into codex/pr1577 2026-08-20 08:04:29 +05:30
debpalash 3482399197 fix(watermark): bound executor shutdown 2026-08-20 08:04:28 +05:30
debpalash 28f69d37aa Merge remote-tracking branch 'origin/main' into fix/issue-1566-deterministic-desktop-test 2026-08-20 08:03:59 +05:30
debpalash df4d016a7d Merge remote-tracking branch 'origin/main' into fix/1589-packaged-lan-ui 2026-08-20 08:03:24 +05:30
debpalash 128b07c923 Merge remote-tracking branch 'origin/main' into pr-1562 2026-08-20 08:03:13 +05:30
Palash Debnath ca7fb9c68d Merge pull request #1594 from debpalash/chore/project-agent-skills
chore(agents): install project development skills
2026-08-20 02:32:26 +00:00
debpalash 3dfe9664cf fix: place desktop test changelog under Fixed 2026-08-20 07:58:51 +05:30
debpalash fd6d21401b Merge remote-tracking branch 'origin/main' into fix/1589-packaged-lan-ui
# Conflicts:
#	CHANGELOG.md
2026-08-20 07:51:49 +05:30
debpalash c9adcb2647 fix(sharing): bundle LAN frontend in desktop installs 2026-08-20 07:50:50 +05:30
debpalash e6d3103ba8 Merge remote-tracking branch 'origin/main' into codex/pr1577
# Conflicts:
#	CHANGELOG.md
2026-08-20 07:50:37 +05:30
debpalash 1e6de9155b Merge remote-tracking branch 'origin/main' into pr-1562 2026-08-20 07:50:24 +05:30
debpalash a41dc8bcac fix(asr): accept omnivoice alias on ROCm 2026-08-20 07:49:43 +05:30
debpalash a8c5ce5c31 Merge remote-tracking branch 'origin/main' into pr-1584
# Conflicts:
#	CHANGELOG.md
2026-08-20 07:49:10 +05:30
debpalash cc9c7cfa18 test(desktop): isolate cleanup script artifacts 2026-08-20 07:47:06 +05:30
debpalash 51163cf260 Merge remote-tracking branch 'origin/main' into chore/project-agent-skills 2026-08-20 07:46:31 +05:30
Palash Debnath 4ce4f05c06 Merge pull request #1559 from Eman-Yousaf/fix/path-security-separator-parity
fix(paths): treat / as a separator on Windows so stored sub-paths resolve
2026-08-20 02:11:45 +00:00
debpalash e77feae817 chore(agents): install project development skills 2026-08-20 07:38:35 +05:30
debpalash e0e19f3dc9 Merge remote-tracking branch 'origin/main' into codex/pr1562 2026-08-20 07:36:37 +05:30
debpalash b73f31b237 test(dub): align persistence and review coverage 2026-08-20 07:33:18 +05:30
debpalash 6bcd3429ac Merge remote-tracking branch 'origin/main' into pr-1584 2026-08-20 07:30:04 +05:30
debpalash 81b6bbc4d3 Merge remote-tracking branch 'origin/main' into codex/pr1577 2026-08-20 07:29:44 +05:30
debpalash fdc02b398e Merge remote-tracking branch 'origin/main' into fix/path-security-separator-parity 2026-08-20 06:40:46 +05:30
Palash Debnath 6e1bb44e0d docs(docker): add product media to Docker Hub overview (#1593)
Add the current v0.5 engine-switching GIF plus Model Catalogue and gallery-save screenshots to the canonical Docker Hub overview using absolute raw GitHub asset URLs. Includes a changelog entry.
2026-08-20 01:10:21 +00:00
debpalash def15b8423 fix(dub): address review findings for responsive dubbing 2026-08-20 06:23:40 +05:30
debpalash 4df7d4e97e fix(watermark): make preload local-only and drain shutdown 2026-08-20 06:14:27 +05:30
debpalash 42b63488e9 Merge remote-tracking branch 'origin/main' into pr-1584 2026-08-20 06:12:15 +05:30
Palash Debnath 4dc90a7f4f docs(docker): refresh Docker Hub overview for v0.5 authentication (#1592)
Refresh current v0.5.0/0.5 tag examples, document API-key and share-PIN behavior, and require encrypted private-overlay access for remote deployments. Keeps the Docker install guide and changelog synchronized.
2026-08-20 00:40:18 +00:00
debpalash ee3e87c0a7 Merge remote-tracking branch 'origin/main' into codex/pr1562
# Conflicts:
#	CHANGELOG.md
2026-08-20 06:09:23 +05:30
debpalash 3b64d317ae fix(paths): accept persisted separators on every host 2026-08-20 06:08:28 +05:30
debpalash b8f1d7f19d Merge remote-tracking branch 'origin/main' into codex/pr1577
# Conflicts:
#	CHANGELOG.md
2026-08-20 06:07:47 +05:30
debpalash ee7202b1eb Merge remote-tracking branch 'origin/main' into codex/pr1559 2026-08-20 06:07:23 +05:30
Paolo Antinori 871d68a6ff fix(auth): offer API-key login on server-mode admin 403s (#1569)
Fix server-mode admin authentication recovery without trapping PIN-only deployments, and prevent stale 403 responses from clearing or superseding newly issued sessions. Includes backend/frontend regression coverage, docs, and changelog credit for @paoloantinori.
2026-08-20 00:03:37 +00:00
Palash Debnath b37466b2e5 fix(ci): allowlist Ed25519 type-name false positive (#1591)
Restore weekly full-history gitleaks scans by allowlisting only the exact cryptography type name Ed25519PrivateKey, with an exact-value regression guard and changelog entry.
2026-08-19 22:16:40 +00:00
victordonat0 155b9345b9 Improve dubbing speaker detection, timing, and responsive UI 2026-08-19 02:57:18 +00:00
Paolo Antinori 37c5df6f3a refactor(watermark): /simplify round — grace in one critical section, _env_float
Four-angle /simplify on the cumulative branch diff:

- The idle-reaper grace flag now lives entirely inside _get_generator's
  lock: the prefetch claims it only when THAT call builds the model, and
  every other getter call consumes it. This deletes the duplicated
  call-site clears in embed/detect (detect no longer touches the
  generator's grace at all — it was clearing a flag for a model it never
  uses), and closes the lock-gap window where the prefetch's claim could
  land on an already-used model, which the old comment claimed was
  impossible.

- Shared _env_float(name, default) for main.py's three inline float-env
  parsers (capture delay, watermark delay, MCP start timeout): one
  NaN/negative-rejecting implementation instead of three drifting
  copies; the older two lacked the isfinite guard entirely.

- Test cleanups: dead isinstance-Future assert half removed, the
  fake-audioseal Event-wait simplified to sleep, the reaper-diversion
  guard simplified to a plain no-op lambda, stale setdefault sentence
  dropped from the conftest comment.

Skipped with reason: merging the double will_mark() gate (they guard
different invariants — pool creation vs model load, both tested) and
hoisting the reaper guard to conftest (an autouse module-attr patch
would break tests that verify release_idle_models directly).
2026-08-18 07:33:27 +02:00
Paolo Antinori 3be001f3fd fix(watermark): close the get_watermark_pool None race + assert the grace
CodeRabbit on 28c7bace:

1. (Major) get_watermark_pool's double-checked pattern re-read the
   global after an unlocked null-check, so shutdown_watermark_pool's
   reset could land in between and the caller received None. The
   executor is now captured and returned under _watermark_pool_lock.

2. (Minor) the idle-grace test overwrote _prefetched_unused after the
   embed call, making the embed's clearing unobservable — a failing
   embed would have passed unnoticed. It now asserts the flag directly,
   and a guard diverts any leaked idle reaper (idle_worker resolves
   release_idle_models per call) to a no-op for the test's duration.
2026-08-18 07:12:31 +02:00
Paolo Antinori 28c7bacefb test(watermark): make the idle-grace test immune to a leaked idle reaper
Second CI red on the same test, different assert: the conftest fix killed
the leaked PRELOAD task, but a test lifespan that exits without shutdown
also leaves idle_worker running, and idle_worker calls
release_idle_models on these same module globals from another thread —
re-stamping _last_used mid-test. Each phase of the test now re-
establishes its preconditions immediately before its release call and
pins now= to a far-future monotonic, so an interleaved reaper tick
cannot change the outcome. Verified against the full 5801-test suite
run in one process.
2026-08-17 21:04:47 +02:00
Paolo Antinori fbb258d2e2 fix(watermark): rebuild the pool after the shutdown drain + review round
The shutdown drain killed the module singleton with no replacement, so
any process that keeps running after a lifespan shutdown — the CI suite
does exactly this — dead-submitted on the next watermark op: "cannot
schedule new futures after shutdown" (CI red; independently confirmed
by Greptile P1, CodeRabbit Major, and the plugin code review at 95/100
confidence). shutdown_watermark_pool() now resets the singleton under
its build lock before draining, so the next get_watermark_pool() hands
out a live replacement. Regression test covers
drained-pool-refuses + replacement-accepts.

Same round, minor findings: the drain's except now logs with exc_info
instead of a bare pass (GHAS CodeQL empty-except); the watermark delay
knob rejects negative/non-finite overrides (CodeRabbit); conftest sets
OMNIVOICE_PRELOAD_WATERMARK=0 unconditionally so a stray export from
the runner shell cannot re-enable background warm-ups mid-suite
(CodeRabbit).
2026-08-17 19:56:32 +02:00
Paolo Antinori 6837ba25ac fix(watermark): review follow-ups for #1577 — CI red + bot findings
Two CI failures, both understood:

1. test_shutdown_preload_race_1000 pins the production _cancel_and_await
  _tasks call site by regex; the new fifth handle broke the pattern. The
   guard now pins all FIVE handles (its property — every preload handle
   awaited under one generous bound — is unchanged).

2. test_prefetched_model_gets_one_extra_idle_window flaked only in the
   full suite: many tests boot the app lifespan, and any that exits
   without a lifespan shutdown leaves the deferred watermark-preload
   task pending — 35s later it fires mid-suite in another thread and
   re-stamps _last_used under whatever test is running. conftest now
   defaults OMNIVOICE_PRELOAD_WATERMARK=0 for the test session (a test
   can still opt in), and the grace test neutralizes will_mark so a
   leaked warm-up can't touch it.

Bot findings: Greptile P1 + CodeRabbit — cancelling the preload task
doesn't stop a watermark-pool thread already inside the ~42s cold
import, and nothing drained that pool at shutdown (only the GPU pool
was reset). Shutdown now drains the watermark pool's queue
(shutdown(wait=False, cancel_futures=True)) — bounded abandon, same
documented reality that Python can't kill a running thread. CodeRabbit
Major: the warm-up reads its own delay knob
(OMNIVOICE_PRELOAD_WATERMARK_DELAY, default 35s) instead of reusing the
capture-ASR delay, so a capture env override no longer retimes it.
CodeRabbit Minor: the _prefetched_unused claim/clear transitions now
happen under _generator_lock, so the retention grace can't be granted
to a model that has actually been used; the test fixture resets all
lifecycle globals.

Skipped with reason: gating prefetch on local-checkpoint presence — the
warm-up downloads only what the first embed would download anyway;
time-shifting that download is the feature, not a new network call.
2026-08-17 15:26:49 +02:00
Paolo Antinori 366b55d9d1 perf(watermark): background-prefetch the AudioSeal generator at startup
The first mark_synthetic serialized the audioseal import plus the
generator load INSIDE the first synthesis — measured at ~42s inline on
a cold filesystem (macOS, 2026-08-17 report), pushing a cold first
synthesis to ~87s and 3s past a 90s client timeout. The generator now
warms on a background task ~35s after boot (+5s past the capture-ASR
warm so the two cold imports don't contend), on the watermark pool,
cancellable at shutdown (OMNIVOICE_PRELOAD_WATERMARK=0 opts out; the
pool is only created when will_mark() says watermarking is active, and
setup-half failures log immediately instead of surfacing at shutdown).

Because the prefetch thread races the first embed, the lazy builds now
hold per-model locks — one build per model, no cross-blocking: a
detector load no longer queues behind a ~42s generator build, and
release_idle_models takes both locks in a fixed order. A
prefetch-warmed, never-used generator survives ONE extra idle-reaper
window so a first synthesis shortly after boot still finds it warm;
real embed/detect use clears the grace.

Also: embed/detect failures now log the full traceback (exc_info). The
catch-all printed only the message, which today left a
ModuleNotFoundError('getopt') inside AudioSeal's forward undiagnosable
from the log — audio silently ships unmarked when this fires.
2026-08-17 14:54:13 +02:00
Palash DebnathandClaude Fable 5 2d5f2e800e feat(omnivoice): voice prompts that survive restarts + opt-in FlashInfer (~2.2x) (#1565)
* feat(omnivoice): port upstream VoiceClonePrompt persistence + FlashInfer opt-in

Upstream k2-fsa teardown ports, verified with generated voice samples:

- VoiceClonePrompt.save()/.load() (upstream format v1, weights_only-safe)
  on the vendored model, and a disk layer under the in-memory prompt LRU
  (DATA_DIR/prompt_cache, keyed by ref path+mtime+ref_text+preprocess,
  32 newest kept, OMNIVOICE_PROMPT_DISK_CACHE=0 opts out). First generation
  of a session with a known voice skips the reference re-encode and any
  auto-transcription pass — verified across two real processes (encodes=1
  then encodes=0, same voice).
- omnivoice_flashinfer.py ported (packed CFG attention, fused kernels,
  optional CUDA graphs), schedule adapted to our num_step+1 divergence.
  Opt-in via OMNIVOICE_FLASHINFER=1|graph, CUDA-only, replaces
  torch.compile for the session; missing package / apply failure / runtime
  failure all degrade with a named reason (same #278 contract as compile:
  classify → unapply → retry once, session latch). Measured 2.20x at
  batch=1 on an RTX 4090 with byte-identical text and clean ASR round-trip.
- Docs: OmniVoice guide gains instruct+reference combination semantics
  (consistent instruct stabilizes cloning, reference wins conflicts),
  inline pronunciation control (pinyin / CMU), prompt persistence, and
  corrects the 'no voice design' claim; performance.md documents both new
  env knobs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: point changelog entries at the real PR number (#1565)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(pr): harden FlashInfer lifecycle + prompt-cache writes per review

Bot harvest round 1 (#1565): unapply on apply-failure (half-patched model
could crash the next render); pin eager-mode FlashInfer inference to one
thread too — the attention plan and packed position ids are per-generation
module state, so interleaved _gpu_pool workers would corrupt each other;
restore the CAPTURED pre-apply attention impl (could be flash_attention_2)
instead of assuming sdpa; unique tmp name per prompt-cache write; correct
the _forward_logits layout docstring; resolve VoiceClonePrompt at test
runtime; docs — Known limits keeps only the limitation, performance.md
states the VRAM cost and scopes the fallback claim to classified kernel
failures.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(pr): round-2 review — publish only a fully restored model, redact latch reason, tighten CPU-persistence test

Greptile: the runtime fallback now unapplies BEFORE swapping generate, so
a concurrent render keeps queuing behind the thread-affinity wrapper while
teardown mutates modules. CodeRabbit: FlashInfer failure reasons pass
through core.failure.sanitize before latching/logging (wheel paths embed
the user's home); the save-portability test now creates the tokens on CUDA
when available and asserts the persisted payload itself is CPU-resident.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(pr): fail-closed latch reason when the sanitizer itself breaks

CodeQL empty-except + CodeRabbit round 3: if core.failure.sanitize raises,
the raw reason (home paths, wheel paths) was latched anyway. Now only the
exception class survives with a fixed redaction note; two regression tests
(normal redaction + sanitizer failure).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 16:25:29 +00:00
Palash DebnathandClaude Fable 5 4db02d0c97 fix(test): watermark producer scan must match code, not prose (#1564)
* fix(test): watermark producer scan must match code, not prose

ee35d238 broke main's CI by adding a comment that *mentions*
backend.generate() to worker/transport/server.py — the watermark coverage
guard greps raw source, so the comment made the module a 'producer' that
never marks. Blank COMMENT/STRING token spans before matching (layout
preserved, unparseable files fall back to a raw scan) and apply the same
rule to the allowlist staleness check; a new self-test pins the class.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(test): keep f-string code scannable; require code-level mark_synthetic

Greptile P1 + CodeRabbit on #1564: on Python <=3.11 an entire f-string is
one STRING token, so blanking it would let a synthesis call inside a
replacement field evade the producer scan — f-prefixed strings now stay
raw there (fail closed), while 3.12+ blanks only literal FSTRING_MIDDLE
text. The 'module references mark_synthetic' certification is now also
code-only, so a comment can't satisfy it. Self-test extended with both.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 15:44:58 +00:00
Paolo Antinori be007e9d77 style(frontend): oxfmt useAppData (CI format check) 2026-08-15 15:19:49 +02:00
velixio ee35d2389e fix: require remote voice render parity 2026-08-15 18:49:08 +05:30
velixio 1fda5bdf96 fix: preserve voice identity on remote workers 2026-08-15 18:49:08 +05:30
Paolo Antinori 030bc47515 fix(lint): makeLoader can't call useCallback (rules-of-hooks) — generation state in a ref instead 2026-08-15 15:08:20 +02:00
Palash Debnath 2477dde688 docs: streamline README structure and specifications (#1560)
* docs: streamline README structure and specifications

* docs: address README review findings
2026-08-15 13:06:44 +00:00
Paolo Antinori 31d4db65e8 Merge remote-tracking branch 'upstream/main' into fix/event-bus-threadpool-emit
# Conflicts:
#	CHANGELOG.md
#	frontend/src/hooks/useAppData.js
2026-08-15 14:49:11 +02:00
Paolo Antinori fd30a6c4ad fix(review): last-write-wins loaders + no sleep-polling in the emit test
CodeRabbit #1562 findings, both real:

- makeLoader is now generation-guarded: the initial retry loop overlaps
  freely with WS-triggered reloads, and a slow in-flight response could
  resolve AFTER a fresher reload and overwrite its list with stale data.
  Each invocation bumps a generation; only the newest may setState.
- The regression test awaited the queue via sleep-polling; it now uses
  asyncio.wait_for(q.get()) so a failure surfaces as TimeoutError instead
  of depending on 10ms poll timing (repo rule: no sleeps as sync).
2026-08-15 14:10:38 +02:00
Paolo Antinori dcaed7cbf4 docs(changelog): one-line Unreleased entry with issue ref + credit (Greptile P2) 2026-08-15 13:54:13 +02:00
Paolo Antinori 9615cd5294 fix(events): sync endpoints dropped their WS events — rename/delete left every open tab stale
PUT/DELETE /profiles (rename, delete, revoke consent) and the history/export
mutators are sync FastAPI endpoints: their bodies run in threadpool workers
where asyncio.get_running_loop() raises, so event_bus.emit() hit the
RuntimeError branch and silently dropped the "profiles" event. The UI only
refetches the voice list on that event, so after a rename the list kept stale
state, and a reload during that window could land on an empty panel (no
retry on the initial load either) — which reads to a user as "all my voices
are gone" even though nothing was deleted.

emit() now captures the serving loop in subscribe() and hands off from
foreign threads via call_soon_threadsafe (async callers are unchanged).
Also: the initial list loads in useAppData retry until FIRST success via
retryInitialLoad — a WS-triggered reload failure still keeps the previous
list, but the first load has nothing to keep. Loaders gained {rethrow: true}
for the initial path so the retry actually engages (they swallow errors by
design elsewhere); an integration test pins that wiring.

Tests: tests/test_event_bus_thread_emit.py fails on the old emit (verified
by stashing the fix) and passes with it; a live two-instance probe confirmed
PUT rename → WS event arrives on the fixed build and never on the original.
2026-08-15 13:49:47 +02:00
Palash DebnathandClaude Fable 5 48c9a3b1f8 feat(settings): compute-device override (auto / CUDA / ROCm / XPU / MPS / CPU) (#1557)
* feat(settings): compute-device override — auto | CUDA | ROCm | XPU | MPS | CPU

Auto-detect stays the default; the override kills the 'auto-detect picked
wrong' issue class. Applied at the single choke point (_probe()'s family
selection) so routing, get_best_device(), and every badge inherit it.
Resolution: OMNIVOICE_DEVICE env > Settings pick (prefs.json) > auto (#981
pattern). An override can steer, never invent hardware: a family the host
lacks is noted and ignored; cpu is always honorable. Applies at next
backend start (host caps are immutable per process — same restart contract
as the rest of the Performance tab, RestartBadge shown).

GET/PUT /api/settings/compute-device (admin-gated) reports resolved vs
applied so the panel shows restart-required truthfully and disables itself
under an env pin instead of pretending.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): entry for the compute-device override (#1557)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(device-override): harvest — the override reaches CT2 ASR, full i18n, honest edge states

- _ctranslate2_cuda_ok() and the ASR sidecar now gate on the probe's family,
  so a cpu pin (or ROCm host) can never hand CTranslate2 a CUDA device —
  the override reaches every CT2 loader through one shared gate
- override_ignored exposed by the API and shown by the panel (env pin naming
  a device this machine lacks: auto is in effect, restart won't change it)
- all 8 panel strings + 5 device-family labels translated into all 21
  locales; failed saves keep their error visible through the re-sync
- test isolation: cleanup drops OMNIVOICE_DEVICE before re-probing so no
  overridden caps leak into later tests; panel tests wait for loaded state
- xpu/intel search keywords; oxfmt formatting

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(device-override): round 2 — fail-safe probe fallbacks, complete i18n, combined pin state

- a broken capability probe now means CPU everywhere (CT2 gate + ASR
  sidecar) — never a torch-derived guess that would bypass a cpu pin or
  re-open #1529 on ROCm; regression test added
- env-pinned AND not-detected shows both facts in one subtitle
- device_load_failed/perf_save_failed translated into all 21 locales;
  CJK/th/vi/ar strings no longer say literal 'Auto'
- test_ctranslate2_never_gets_cuda_on_a_rocm_build pins the probe family
  (it was order-dependent on the lru_cache before)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(test): pin the probe family in the faster-whisper OOM-fallback test

Same class as the rocm-build test: it mocked torch but not the probe the
new override gate consults first, so on a cpu-family CI host the CUDA
fallback chain under test was unreachable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 05:08:51 +00:00
Palash DebnathandClaude Fable 5 030d5ea01f docs(engines): a guide for every engine + index; fix two engine-metadata bugs (#1556)
* docs(engines): a guide for every engine + index; fix two engine-metadata bugs

21 new pages under docs/engines/ (10 TTS, 10 ASR, index README) — every
registered engine now has one: what it's for, platform support, model env
vars, quirks with issue refs. Linked from both READMEs' engine sections.

Code fixes found while verifying facts against the registries:
- KittenTTS docstring claimed default voice 'Jasper'; the code default is
  expr-voice-2-f
- the isolated-ASR sidecar read only ASR_MODEL_FW while the download
  preflight read ASR_MODEL_FASTER — set one and the other quietly used a
  different model; both now resolve ASR_MODEL_FW-override → ASR_MODEL_FASTER
- moonshine's install hint named 'useful-moonshine', a package the backend
  never imports; now moonshine-onnx / moonshine-voice

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): entries for the engine guides + sidecar model fix (#1556)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(engines): second-harvest fixes — TLS guidance, matrix/code alignment, CN counts

- README matrix aligned to gpu_compat (the code is the source of truth):
  CosyVoice macOS is CPU not MPS, IndexTTS and GGUF gain their real
  CUDA/CPU/MPS cells
- gpt-sovits guide: prefer https/tunnel for non-loopback servers, plaintext
  warning; first-use download guidance on both OmniVoice pages
- preflight empty-env fallback matches the sidecar (ASR_MODEL_FASTER='' no
  longer resolves a different repo)
- nano installs via uv pip; kitten log level wording; index links install
  guides incl. the Gatekeeper step; README_CN engine counts 16/11

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(readme-cn): the all-engines-local claim now excludes the remote client

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 03:55:59 +00:00
Palash DebnathandClaude Fable 5 b79ba9bd3b docs(readme): lead with download + first clone; seed benchmarks page (#1555)
* docs(readme): lead with download + first clone; seed benchmarks page

Quickstart (installers, install guides, a three-step first-clone walkthrough)
moves above What's-new/Features in both READMEs — visitors get the action
before the pitch. New docs/benchmarks.md anchors measured per-engine/device
numbers on the bench_pipeline.py harness, community-contributed, no estimates.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): entry for the README conversion restructure (#1555)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(bench): emit RTF + CUDA peak VRAM; guard NaN RAM; define the benchmarks schema

Bot harvest on #1555: the tts stage now prints RTF per warm measurement and
CUDA peak VRAM (None elsewhere — no made-up zeros), the stage floor refuses
unmeasurable RAM instead of sailing past a NaN comparison (FLOOR_GB=0
overrides), docs/benchmarks.md columns map 1:1 to what the harness prints,
and the download badges say they open the release page.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(readme): link palash.dev from the maker section

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(bench): name the resolved engine, track VRAM from resolution, comment the guards

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(readme): the quick-switch gif is the hero image

The hero shows motion now; the Launchpad screenshot moves into the 0.5.0
What's-new slot so nothing appears twice.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(bench): peak VRAM is reserved memory; adapter engines name their model

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(bench): subprocess-isolated engines report VRAM n/a, not a parent-side zero

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(bench): out-of-process detection is declarative; sherpa rows name their model

'runs_out_of_process' is now a TTSBackend attribute set by SubprocessBackend
AND omnivoice-gguf (which inherits TTSBackend directly but spawns a binary
per generate — the isinstance check missed it). Duck-typed for the same
module-purge reason as _is_subprocess_isolated. Sherpa-onnx identity comes
from _model_dir's basename when _model_id is absent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(bench): backends self-report model identity via TTSBackend.model_identity()

Greptile enumerated the adapter engines one at a time (mlx _model_id,
sherpa _model_dir, cosyvoice env-only) — the attribute sniffing rots per
engine. The hook fixes the class: each multi-model backend reports its
own identity, the profiler just asks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 02:41:40 +00:00
Palash DebnathandClaude Fable 5 3d0c9605df test(shell): backend-lifecycle fault-injection harness (#1551)
* test(shell): backend-lifecycle fault-injection harness

Runs spawn_backend_and_wait/supervise_backend against REAL dying child
processes and asserts the user receives the correct NAMED diagnosis —
not merely that recovery happens. Wrong/missing explanation was 61% of
the historical "can't reach the backend" class; this rig is the
permanent regression harness for every future lifecycle fix.

Seam: OMNIVOICE_BACKEND_CMD (JSON argv or whitespace form) runs any
command as "the backend" — venv bootstrap and ffmpeg resolution are
skipped, everything else (err-log run offsets, drainer threads, env
pinning, real OS pipes, spawn-failure diagnostics) stays real. Plus
OMNIVOICE_LOG_DIR (per-test log+marker dirs, also a support tool) and
harness-only timing overrides OMNIVOICE_STARTUP_BUDGET_S /
OMNIVOICE_SUPERVISOR_POLL_MS whose production defaults are pinned by
unit tests. Lifecycle fns genericized over tauri::Runtime for the
MockRuntime app; behavior-neutral with the env unset (unit-pinned).

Scenarios (tests/backend_lifecycle.rs, scenario children = this test
binary re-invoking itself; serial by mutex + CI --test-threads=1):
- port conflict (exit 78) → the detectHints-matchable port phrasing
- generic chained traceback → root cause survives into the diagnosis
  and the crash marker
- spawn failure → spawn diagnostic reaches the user, NO bogus marker
- slow start past budget → timeout names the budget + last stderr
- post-Ready crash loop → 3 restarts announced, markers before restarts,
  "kept crashing" diagnosis naming the last exit
- SIGKILL (unix) → named as signal 9
- deliberate kill → supervisor yields silently, no marker, never Failed
- deferred-startup FATAL → the named step reaches the user, forensics,
  and the splash narration

CI: harness added to the 3-OS tauri-cross-platform matrix.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(ci): temporary Windows loader bisect probe for the harness binary

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(shell): embed Common-Controls v6 manifest into Windows test binaries

Bisected on #1551: EVERY integration-test binary of this crate died at
load on Windows with STATUS_ENTRYPOINT_NOT_FOUND (0xc0000139) — cargo
gives test binaries no manifest, so the loader resolves comctl32 v5,
which lacks the TaskDialogIndirect entry point tauri's dialog/tray stack
imports. build.rs now embeds tests/windows-test.manifest via
rustc-link-arg-tests on Windows targets. Bisect probe removed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(test): scenario gate is PID-valued — the parent can't self-inject

CodeRabbit on #1551: in a parallel local `cargo test`, the parent's own
scenario_child test could observe the armed env and start playing the
backend in-process (binding the port, idling 600s). The gate value is
now the arming process's PID; a matching PID stays inert, so only the
spawned child — a different process — runs the scenario.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 18:45:27 +00:00
Palash DebnathandClaude Fable 5 bb813ff676 feat(startup): bind the socket in ~1s and narrate startup step by step (#1550)
* feat(startup): bind the socket in ~1s and narrate startup step by step

The structural fix for the "can't reach the local backend" class (~1 in 5
of every issue ever filed): uvicorn served nothing until torch import
(10-20s cold), the 30-router fan-out, an import-time DB migration, the
cuDNN preload, and alembic all finished — every slow or fragile step
rendered as an unexplained dead backend.

main.py now keeps module scope fast and defers the heavy work:
- _phase_a_build (executor thread): prefs/env restore + #963 migration,
  yt-dlp overlay, cuDNN preload, torchaudio, model_manager, router
  imports — order preserved, literal imports so PyInstaller still traces.
- _phase_a_finalize (event loop, no awaits → atomic wrt requests):
  include_router, mounts, MCP, SPA, openapi bust.
- _phase_b: the old lifespan startup body; handles on app.state so
  shutdown survives a startup that never finished.
- Eager mode (pytest / OMNIVOICE_EAGER_INIT=1) runs everything at import
  — byte-equivalent behavior for the ~100 lifespan-less TestClient sites
  and for embedders (dump_api_routes, probe boot runner opt in).

While starting: /health answers 503 with the current step, new
/startup/progress serves the full ledger (always 200), and
StartupGateMiddleware 503s everything else with the [starting] marker
(same skip-the-Report-button convention as [shutting_down]). A deferred
failure keeps import-crash semantics: traceback to stderr → shell crash
forensics, run sentinel stays uncleared, exit 1 names the failed step.

Shell: startup_progress() probe (marker-header-gated so a foreign
responder can't narrate the splash) feeds per-step log lines into the
launch poll and the supervisor's reconnect wait. --health-check absorbs
the deferred init (60→180s); --diagnose runs Phase A up front so it
still sees restored prefs. Docker HEALTHCHECK semantics unchanged
(curl -f fails on 503 exactly as it did on connection-refused).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(startup): join the Phase A thread on shutdown; async fail-path sleep

Bot-review harvest on #1550: cancelling the deferred-startup task cannot
stop the executor thread inside Phase A's blocking imports — shutdown now
waits (bounded, only when a build started and hasn't finished) on a
thread-completion event so interpreter teardown can't race a mid-import
(#1000 class). The failure path's last-poll beat is now awaited, not
time.sleep — a blocking sleep froze the very loop that beat exists to let
serve. Also: dump_api_routes forces eager (assignment, not setdefault),
and the integration test's child gets DEVNULL instead of an undrained
pipe that could wedge a cold boot.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(startup): close the Phase A submission race; CodeQL nits

Review finds on #1550: shutdown could sample _phase_a_started unset
while the executor callable was queued-but-not-running, skipping the
thread join. started is now set BEFORE submission, the submission is
shielded so a cancel can't strand a queued callable that would never set
_phase_a_finished, and the wrapper sets finished on every exit including
the already-built early return. Contract pinned by
test_phase_a_thread_join_contract. Plus explanatory comments on the new
bare excepts and a consistent return in the gate's websocket branch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 15:23:02 +00:00
Palash DebnathandClaude Fable 5 bc6acec5a3 fix(shell): gate Ready on the deep health probe; pace crash-loop restarts (#1548)
* fix(shell): gate Ready on the deep health probe; pace crash-loop restarts

Two supervisor hardenings from the backend-reliability root-cause pass:

Ready now requires backend_ready() — the identity probe (/system/info
string-sniff) AND the deep probe (/profiles must 200) — at both Ready
transitions (startup poll, supervisor respawn wait). The shallow probe
alone announced a backend whose install/DB had broken underneath as up;
the UI looked alive while every real request 500'd or dead-ended on
"can't reach the backend". Death detection stays process-exit-only, so a
busy-but-alive backend is still never killed.

Supervisor respawns now back off: first respawn immediate (a one-off
crash self-heals fast), then 5s, then 15s, capped — the budget check
ends a hopeless loop, not an unbounded sleep. The pause runs behind the
already-visible "reconnecting" banner and yields within 500ms to app
quit or a deliberate retry-flow replace.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(shell): yield backoff to a tracked replacement child, not just the flag

Greptile P1 on #1548: a completed Retry/Clean&Retry sets the deliberate-
kill flag and track_backend_child clears it — possibly both between two
500ms backoff samples, so the flag alone can be missed and the old
supervisor would free_port() the retry's healthy replacement. The dead
child we observed can never read as alive again, so a live tracked child
during backoff can only be a replacement — yield to it promptly so the
retry's spawn_backend_and_wait can claim the supervisor slot at Ready.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(shell): backoff yields on spawn-generation change, not liveness

Second Greptile pass on #1548: a replacement child that itself exits
before the old supervisor's next 500ms sample read as "still dead" under
the liveness check, so ownership transfer was missed. The spawn
generation (bumped by every track_backend_child, never un-bumped) is
observable regardless of the replacement's fate — snapshot it at death
detection, yield the moment it changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(shell): snapshot spawn generation before observing the exit

Third-pass review find: sampled after try_wait, a replacement tracked in
the gap bakes its own generation into the snapshot and the ownership
transfer is missed. Snapshot first, and re-check once more before
touching the port so the zero-backoff first respawn is covered too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 14:41:21 +00:00
Palash DebnathandClaude Fable 5 94ba362ef2 feat(triage): crash-class recurrence report — the reliability metric (#1549)
* feat(triage): crash-class recurrence report — the reliability metric

scripts/crash_class_report.py measures the "backend died / never came
up" class (the project's #1 lifetime failure, ~1 in 5 of all issues)
filtered to reports from the current version — the definition of done
for the reliability cycle. Buckets by the bug reporter's Build-status
stamp (#1547): current / outdated / unknown (pre-deflection builds), so
deflection-miss noise never pollutes the number the work is judged on.

tests/scripts/test_crash_class_report.py pins the title→sub-class
mapping against the real historical title shapes and locks the stamp
literals to frontend/src/utils/bugReport.js so a reworded marker fails
in CI instead of silently zeroing the metric.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(triage): --version is authoritative; loud fetch-cap warning

Bot-review harvest on #1549: with --version, the Environment Version
line now decides the bucket (extracted to pure classify_build + tests) —
a report stamped "current at filing time" during another version's
window no longer counts toward this version's recurrence. Hitting the
500-issue fetch cap now warns loudly instead of silently understating.
The stamp lockstep test asserts the full Build-status prefix.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 14:20:35 +00:00
Palash DebnathandClaude Fable 5 aabe5783f3 fix(report): offer the latest release before filing from an outdated build (#1547)
* fix(report): offer the latest release before filing from an outdated build

6 in 10 sampled "can't reach the backend" reports came from builds that
were already obsolete when filed, and were closed with "please update" —
pure triage noise. Every Report-bug affordance now funnels through
openBugReport(): on an outdated build it offers the latest release first
(with a "File anyway" escape hatch), and the report body carries a
triage-greppable "**Build status:**" line either way, so current-version
recurrence — the reliability metric — is countable separately from
stale-build reports.

Freshness sources per deployment (behavior identical, implementation per
mode): desktop reads the Rust updater's channel-aware verdict from the
store (no new network path, no CSP widening); browser/dev/Docker make one
bounded latest-release GET, only once the user has initiated the report
flow whose destination is github.com. An 'unknown' dev build stays
silent entirely — never nudged, never claimed current.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(report): anchor version parsing; zh-TW reportBug.title in Traditional

Bot-review harvest on #1547: parseVersionTriple now rejects trailing
non-semver data (1.2.3.4, 1.2.3garbage) instead of silently reading the
leading triple into an outdated/current verdict; the pre-existing zh-TW
reportBug.title was Simplified-script — now properly Traditional.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 13:44:24 +00:00
Palash DebnathandGius 854b4852ed perf(frontend): coalesce persistence writes off input paths (#1546)
Defer and coalesce omnivoice.app and omni_ui persistence behind a 250 ms
quiet window with a 1,000 ms hard maximum, preserving storage schemas,
synchronous pending reads, legacy formats, Factory Reset semantics,
widget read-only ownership, and lifecycle (pagehide/visibilitychange)
durability. Adds scheduler, restore, reset, StrictMode, concurrent-render,
role-ownership, and migration regression tests plus an opt-in
production-bundle responsiveness harness.

Lands #1541 by @bultodepapas (maintainer landing branch; the out-of-scope
attribution-policy commit was dropped).

Co-authored-by: Gius <bultodepapas@gmail.com>
2026-08-14 13:25:36 +00:00
Eman-Yousaf 579f2e0a2e fix(paths): treat / as a separator on Windows so stored sub-paths resolve
resolve_within split candidate paths on os.sep alone. Windows accepts /
as a real separator but os.sep is \ there, so a persisted sub-path such
as "job_123/out.mp4" stayed a single component, failed the
basename-equality check, and raised UnsafePath — while the identical
value split cleanly and resolved on POSIX. A data directory written on
Linux or by the Docker deployment and then opened by the Windows desktop
app hit exactly that.

Split on both separator families instead, which is what the comment
above the split already states the code intends. This is not a
loosening: every component still goes through the same basename / "." /
".." / empty rejection, and the commonpath containment check and symlink
resolution below are unchanged. POSIX behaviour is unchanged too — a
backslash is already rejected there as a foreign separator before the
split runs.

This also restores real coverage of the symlink-escape guard on Windows.
test_resolve_within_rejects_symlink_escape asserts through
"link/secret.wav", which previously raised at component validation
before reaching the containment check it exists to cover, so it passed
for the wrong reason. It now matches on the reason.
2026-08-14 17:39:09 +05:00
Palash DebnathandClaude Fable 5 420bc73e78 fix(release): harden the AppImage repair — pinned tooling, final-writer manifest (#1545)
* fix(release): harden the AppImage repair step

All three review findings on #1544, fixed before the tag re-runs it:
- appimagetool pinned to the immutable 1.9.1 release with a verified
  SHA-256 — a mutable 'continuous' binary must not execute with the
  updater signing key and a release-write token in its environment
- the release tag reaches the script as env data, never interpolated
  into shell source (zizmor template-injection)
- a failed latest.json download now fails the step unless the asset is
  confirmed absent, and the patch refuses to upload unless at least one
  linux signature was actually replaced — a repacked AppImage can never
  ship paired with stale updater metadata

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(release): the updater manifest gets one final writer, after the matrix

CodeRabbit + Greptile on #1545: every tauri-action leg re-uploads the
shared latest.json, so patching it inside the Linux leg races the other
platforms — a later leg's upload could resurrect the stale pre-repack
signature. The manifest patch moves to a post-matrix job that runs once
after all legs: it aligns the manifest's linux entries with the .sig
asset that actually shipped (self-verifying — no cross-job state), and
no-ops when they already agree. The leg keeps asset repack/re-sign only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(release): a failed .sig download fails the manifest-align job

Same fail-closed rule as the manifest itself: absence is decided by the
asset list; any other download failure must not exit 0 with a stale
signature left in latest.json.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 01:55:10 +00:00
Palash DebnathandClaude Fable 5 fb46fa4788 fix(release): repack the AppImage with a real .DirIcon file, re-sign, re-upload (#1544)
The v0.5.0 tag build — the first real release since #1518's guard — proved
the files-map fix loses: linuxdeploy re-links .DirIcon to an ABSOLUTE
build-machine path after tauri places the real bytes, and the guard
correctly refused to publish. tauri-action's atomic build+sign+upload
leaves only a post-upload seam, so the Linux job now repairs the packed
artifact: extract, replace .DirIcon with the icon bytes as a regular file
(nothing left to dangle), repack with appimagetool, re-sign with the
updater key, clobber the draft release's asset and patch the linux
signature inside latest.json. The existing smoke then validates the
repaired AppImage. No-ops cleanly when .DirIcon already resolves.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 00:59:14 +00:00
Palash DebnathandClaude Fable 5 f83b7371c3 docs: remote-workers guide speaks VoiceStudio (#1543)
CodeRabbit's last-cycle Minor on #1540, applied as the immediate
follow-up the docs-sync rule prescribes: three OmniVoice mentions in
docs/remote-workers.md now carry the product's name.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 00:07:05 +00:00
Palash DebnathandClaude Fable 5 9184d7d625 chore: bump to 0.5.0 (#1540)
* chore: bump to 0.5.0

Owner-requested minor bump. package.json is the source of truth; the
three mirrors (Cargo.toml, pyproject.toml, _FALLBACK_VERSION), the three
lockfiles and the branding pin move in lockstep, and the accumulated
Unreleased section becomes the curated 0.5.0 release notes — quiet
Highlights first, one-liner subsections after, duplicates folded.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: fresh README for the VoiceStudio era; 0.5.0 notes wear the release

README: 551 lines from 656 — a What's-new-in-0.5.0 section with real
captures, the engine tables corrected to the actual 16 TTS registrations,
a stale Settings path and a broken Colab link fixed, roadmap/FAQ/credits
trimmed to what earns its place.

Release notes: the quick-switch GIF and catalogue/gallery screenshots,
captured from the running app during the pre-bump test pass, embedded
after the Highlights; #1542's gallery work and the ffmpeg CI fallback
recorded in their subsections.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: the feature inventory's Remote Model Downloads mention survives the README trim

check-docs-drift requires every docs/features.yaml name verbatim in the
README; the overhaul folded the phrase away. It now lives in the remote
workers feature line.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: one blockquote, and an architecture claim that survives the opt-ins

CodeRabbit on #1540: MD028 blank line inside adjacent blockquotes, and
'every layer is on your machine' contradicted the opt-in remote paths
documented two sections away — it now states local-by-default with the
opt-ins named.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 23:50:23 +00:00
Palash DebnathandClaude Fable 5 0ee62b2261 feat(gallery): save gallery voices as profiles, with validated audio references (#1542)
* feat(gallery): save gallery voices as profiles, with validated audio references

Work-in-progress lifted from the concurrent gallery session at the
owner's request (its uncommitted working tree, preserved verbatim from
base 92b1ee5d; safety snapshot remains at rescue/gallery-wip):

- gallery voices can be saved as local profiles: audio is copied into
  the profile store with content-addressed filenames, existing profiles
  are detected and refreshed only when the source clip changed
- backend/core/audio_validation.py: symlink-rejecting, root-contained
  resolution for persisted profile WAV references, with tests
- archetype/community routers and the Voice Gallery UI updated for the
  save-as-profile handoff (spec: docs/specs/longform/26-gallery-use-handoff.md)
- locale updates for the new gallery strings across all 21 files

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: drop a stray local screenshot script that rode in with the tree copy

* fix(community): explain the tolerated Content-Length parse failure; drop an unused import

CodeQL on #1542: the empty except now says why it is safe (the streamed
byte counter enforces the same cap regardless), and the test file loses
an unused Path import.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(gallery): review findings — copy outside the write lock, no stale completions

CodeRabbit on #1542, all findings addressed:
- the profile-audio copy stages to a .part temp BEFORE BEGIN IMMEDIATE
  and publishes via atomic os.replace inside it — other backend writers
  no longer block for the duration of an audio copy; a mid-copy failure
  leaves no temp droppings and no profile row (both pinned by tests)
- VoiceGallery async ops carry per-operation generation tokens: a
  preview or save-as-profile that resolves after unmount (or after a
  newer operation) can no longer play audio, redirect into a workspace,
  or touch state — three fail-before regression tests
- VoiceGalleryActions imports the page at test runtime; the e2e locator
  uses a stable data-testid instead of a translated string; symlink
  tests skip cleanly where the OS can't create symlinks; the changelog
  line carries its PR ref

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci: static ffmpeg fallback when the chocolatey feed is down

Third feed outage to break a PR run (2026-07-20, 2026-07-28, today —
three attempts, three 'installed 0/1'). Chocolatey is a distribution
channel, not the dependency: after the retry loop exhausts, fetch the
static gyan.dev build from its GitHub release mirror and put it on
PATH — same binary, no feed in the path. URL verified live (HTTP 200).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 22:48:05 +00:00
Palash DebnathandClaude Fable 5 77ae194f9c fix(asr): ROCm torch is not CUDA — keep CTranslate2 off the HIP GPU (#1539)
An RX 7900 XTX in the :rocm image crashed ASR init with 'CUDA driver
version is insufficient for CUDA runtime version' (#1529): ROCm torch
answers torch.cuda.is_available() and hands out 'cuda' device strings,
but whisperx/faster-whisper run on CTranslate2, whose CUDA runtime is
NVIDIA-only. Same class as the Apple/#1127 lesson, on the AMD axis.

- _ctranslate2_cuda_ok(): 'cuda' for CTranslate2 only when torch is a
  real CUDA build (torch.version.hip is the honest tell); ROCm hosts
  take CPU int8 instead of a native crash.
- _auto_detect(): on a ROCm-GPU host prefer pytorch-whisper — a pure
  transformers pipeline riding torch itself, so it actually uses the
  HIP GPU while CTranslate2 engines would idle on the CPU.

Fail-before/pass-after: 4 new tests fail on the old device pick/order.

Fixes #1529

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 21:05:53 +00:00
Palash DebnathandClaude Fable 5 d3822c4976 feat(engines): open the door between the LLM family and its providers (#1538)
* feat(engines): open the door between the LLM family and its providers

The openai-compat family entry and the LLM Providers panel are one
system — llm_backend resolves every call through the active provider —
but the UI presented them as unrelated (council coherence finding). Now:

- the catalogue's openai-compat row carries a 'Provider · model' hint
  naming the endpoint that actually answers (decorative: a provider
  registry hiccup degrades to no hint, never a failed listing)
- the row offers 'Configure providers' straight into Settings → LLM
  Providers; the panel gains the backlink into catalogue → LLM family
- three new strings in all 21 locales, matching each file's provider
  terminology

Also: bugReport's encoded-ceiling test is hermetic now — it was the one
test in its file trusting ambient fetch, and hung on any machine where a
local backend holds the port without answering.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(i18n): the catalogue note names the ACTIVE provider, not the edited one

CodeRabbit on #1538: the panel can be editing a provider that is not
active, and 'this provider answers…' then points at the wrong one. The
note now says the provider MARKED ACTIVE answers, which is true under
any selection — no state-dependent copy needed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 20:46:36 +00:00
19ae20111a fix(security): replace persistent admin keys with scoped sessions (#1528)
* fix(security): replace persistent admin keys with sessions

Exchange the remote administrator key once for bounded, revocable credentials. Canonicalize backend principals, enforce cookie CSRF and exact origins, and use path-bound one-use WebSocket tickets.

Migrate the bundled UI away from durable master-key storage and credential-bearing URLs. Add unit, integration, static-hygiene, and production-browser regressions plus synchronized operator documentation.

* docs: link session hardening to PR 1528

* fix(security): key session indexes with process pepper

Use HMAC-SHA-256 instead of an unkeyed digest for in-memory session and WebSocket-ticket indexes. This preserves constant-size lookup identifiers, makes copied records unusable without the process pepper, and resolves CodeQL's weak sensitive-data hash finding.

* fix(auth): align empty bearer migration precedence

Centralize the Authorization-channel presence decision with canonical principal parsing. Bearer followed only by spaces now remains an empty channel during legacy cookie migration, while unsupported or invalid explicit credentials stay authoritative and fail closed.

* fix(security): harden admin session review boundaries

* fix(security): derive key generations with HKDF

* fix(auth): anchor the admin-session store so module reloads cannot fork it

test_master_exchange_does_not_bypass_pin_on_normal_routes failed in full-suite
runs: test_mcp_bindings' client fixture purges the services.* tree from
sys.modules and reloads main, so api.routers.auth re-imported a fresh
services.admin_sessions (new AdminSessionStore) while core.auth kept its
import-time reference to the old one — the exchange issued the cookie into
one store and the middleware resolved it against another, turning the
expected "PIN required" into "API key required".

Root cause is the class of bug, not the one test: a process-global auth
store defined as a bare module-level singleton forks under importlib.reload
or purge-and-reimport. Fix at the source: admin_session_store now resolves
through a synthetic sys.modules anchor (_omnivoice_admin_session_store_anchor)
that reloads never re-execute and package-prefix purges never match, so every
copy of the module shares the one per-process store. No consumer or behavior
changes.

Regression test reproduces both fork vectors (in-place reload and
sys.modules purge + fresh import) and asserts previously issued sessions
still resolve and the store identity is preserved; it fails before this fix
and passes after.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(auth): honor X-Forwarded-Proto for CSRF origin and Secure cookies behind TLS proxies

Behind Tailscale Serve (docs/remote-gpu.md) or any TLS-terminating proxy,
the browser talks https while the backend hop stays http, so exact-origin
CSRF compared an https Origin against an http expectation and rejected
every legitimate request, and the session cookie shipped without Secure.
uvicorn's ProxyHeadersMiddleware only rewrites the scope for loopback
peers, which misses Docker and any non-loopback proxy topology.

New core.csrf.effective_scheme derives the client-facing scheme: resolved
scope first (uvicorn's trusted-proxy rewrite wins), then an upgrade-only
read of X-Forwarded-Proto's first value — https/wss promotes http to
https, everything else is ignored, and a genuine TLS hop can never be
downgraded. Used by both the destination-origin comparison and
auth._secure_cookie so the WS-ticket/logout CSRF paths and the cookie
Secure flag agree. Spoofing gains nothing: the host:port half of the
origin tuple is untouched, browsers cannot attach the header cross-site
without a preflight this API never grants, and forging it on plain http
only adds Secure (the browser then drops the cookie — self-harm only).

Regression tests: proxied https origin accepted (origin check, Secure
flag, logout), comma-separated chains, scope-fallback path, spoofed
header still rejects cross-origin, cannot downgrade real https, junk
values ignored.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(auth): consume the stored admin key only after a successful exchange

A remote-backend user upgrading with their backend unreachable lost the
only stored copy of OMNIVOICE_API_KEY: every migration path deleted the
durable ov_api_key BEFORE the session exchange settled, stranding them
until they recovered the key from the server box. Close the whole class:

- client.ts bootstrap: read the legacy key, exchange first, and remove
  the durable copy only after the exchange succeeds; on failure the key
  stays so the next launch retries the migration (auth gate still rises).
- authSession.ts exchangeApiKey: move removeLegacyMaster from before the
  fetch to the cookie/bearer success paths — the key never coexists with
  a live session, but a rejected or hung exchange no longer consumes it.
- remoteBackendProbe.ts configuredRemoteBackend: stop wiping the key on
  every app mount.
- RemoteBackendPanel: a connection test or an aborted save no longer
  wipes the pending key; only disabling the remote backend discards it.
- prefKeys.js: ov_api_key moves from PREF_KEYS to PRESERVED_KEYS —
  factory reset preserves the pending connection credential exactly like
  ov_backend_url; the successful migration is what deletes it.

Tighten the credential-hygiene static guard to match: it accepted
sessionStorage.setItem('ov_api_key', …) — the exact class it exists to
close. The guard now flags .setItem(<master key>) on any storage
receiver, quote style, or injected-store alias, with a self-test pinning
what it catches and what stays legal.

Fail-before/pass-after regression tests: backend unreachable retains the
key and the next bootstrap retries it; a successful exchange removes it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* perf(auth): make session validation occupancy-independent

* test(auth): catch optional master-key storage calls

* feat(docs): add PR control document for bultodepapas in VoiceStudio

* docs: keep the PR tracking board in the fork; credit the changelog line

The pr-control document is excellent process discipline, but it is the
contributor's own operational board (their inventory, their update
commands) — it lives naturally in their fork, and docs/agents/ here is
context every repo agent loads. Removed with appreciation; the changelog
line gains its contributor credit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: debpalash <4178343+debpalash@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 20:46:24 +00:00
Paolo Antinori e4c1ef0de6 Merge remote-tracking branch 'upstream/main'
# Conflicts:
#	.gitignore
2026-08-13 22:06:19 +02:00
Palash DebnathandClaude Fable 5 b982192011 fix(worker): reserve the capacity slot before announcing the accept (#1537)
* fix(worker): reserve the capacity slot before announcing the accept

Two changes for #1536 (flaky test_worker_at_capacity_rejects_without_penalty):

The client now inserts into _running BEFORE awaiting the accept send.
Today _send enqueues synchronously so the old order could not actually
interleave — but the reserve-then-announce order is the invariant that
stays correct if _send ever gains backpressure (a bounded outbox is the
natural evolution), instead of silently reopening an over-accept window.
If the accept send fails, the reserved task is cancelled: work the
scheduler never saw accepted must not run to double-execution.

The test now pins the real invariant — no over-concurrency — rather than
the scheduler's bookkeeping timing: on a loaded CI runner the first
attempt can die environmentally (a stream hiccup fails _run, whose
finally frees the slot), after which accepting the second task is the
CORRECT behaviour the old flat assertion punished as a failure. The
assertion now applies only while the first attempt is still running, and
names the over-accept explicitly when it fires.

Fixes #1536

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(worker): release the reserved slot on a cancelled accept-send

CodeRabbit on #1537:
- except BaseException, not Exception: a handler cancelled while the
  accept send is in flight must release the reserved slot too, or the
  unaccepted task keeps running and double-executes after reassignment.
  Fail-before/pass-after regression test included.
- the capacity test now polls the second task out of its dispatch states
  instead of sleeping 0.5s, and asserts the penalty-free invariant
  (excluded_workers empty) unconditionally — capacity rejections never
  exclude the worker regardless of the first attempt's health.
- TaskAccepted.envelope finding skipped: the field is read nowhere
  server-side (registration is the only envelope consumer) — pre-existing
  unused-field design, not introduced here.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 19:19:22 +00:00
Palash DebnathandClaude Fable 5 81b146f53b fix(i18n): 53 default-value-only strings now speak all 21 languages (#1534)
* fix(i18n): 53 default-value-only strings now speak all 21 languages

Every string used via t(key, { defaultValue }) without a locale entry
rendered English for every non-English user — the worker/compute chrome
(quick settings, join panel, QR enrolment), clone/design labels, workspace
voice strip, and two crash explainers. All 53 keys now exist in en.json
and carry reviewed translations in the 20 other locales, matching each
file's established terminology (existing worker/token/engine vocabulary,
catalogue tab names for the in-text path references, registers preserved,
{{placeholders}} byte-identical, the ovw_ token prefix untranslated).

The two crash explainers are translated from their FULL concatenated
source text — the extraction initially captured only the first string
segment, which src/test/streamDropError.test.ts caught by failing on the
missing proxy/buffering guidance.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(i18n): reference localized UI labels inside diagnostic strings

CodeRabbit on #1534, fixed as the class: every locale's crash_broken_env
quoted the English "Clean & Retry" although the button itself is
localized — all 19 now quote each file's own clean_retry label. Plus the
flagged singles: es unload verb disambiguated from downloading, hi unload
verb aligned with crash_oom_kill, sv kontrollplan gender agreement, de
crash_broken_env moved to the file's Sie register, ru seed_reroll_hint
mistranslation, zh-TW path label matched to the real 系統日誌 section name.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 18:27:27 +00:00
Palash DebnathandClaude Fable 5 72cabb3daf feat(launchpad): wear the signal-field waveform in the hero (#1533)
The same cover artwork the project uses on the web, bleeding in from the
hero's right — where the layout holds only air — behind a radial feather
plus right-edge fade so no box edge survives, screen-blended so its dark
field vanishes into the chrome. Decorative: aria-hidden, empty alt,
pointer-inert; 22 KB webp bundled via vite.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 17:48:38 +00:00
Palash DebnathandClaude Fable 5 4a228a00b4 fix(crash): bound crash evidence to the run that produced it (#1532)
* fix(crash): bound crash evidence to the run that produced it

backend_err.log is one file shared by every backend run, and it was
TRUNCATED on each spawn. Both properties destroyed the evidence a crash
marker exists to carry: a respawn wiped the dead process's final words,
and an unbounded tail read afterwards attached the replacement's healthy
startup to the old run's crash marker — the undiagnosable report in #1510
(startup lines, no traceback, timestamps after the recorded crash).

The file is append-only now with a run-start header; each spawn records
the byte offset where its run begins; and every death path (crash markers,
venv-heal detection, restart-budget message, the 300s startup timeout)
reads through read_error_log_tail_for_run(), which cannot see another
run's output. The log rotates to backend_err.log.1 past 1 MiB so
append-only cannot grow unbounded. Bootstrap-phase reads (uv sync) keep
the whole-file reader — no backend run exists yet there.

Fail-before/pass-after: the new tests fail under the old File::create
truncation and unbounded tail.

Fixes #1510

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(crash): flush the dying run's stderr before the next offset; redact home paths

CodeRabbit on #1532:
- the stderr drainer is tracked now and joined (2s bound) before a new
  spawn records its offset, so a dead run's buffered tail cannot be
  appended after the new run's start and misattributed. Full per-child
  offset binding is unnecessary: spawns are serialized by the #1223
  spawn-once flow; the buffered tail was the only remaining window.
- the spawn-failure diagnostic redacts the home-directory prefix — it is
  retained across runs now and lands verbatim in bug reports.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 17:41:22 +00:00
Palash DebnathandClaude Fable 5 f81ace68d1 feat(engines): frame uninstalled engines as headroom, not failures (#1531)
The catalogue's group captions read "Available" / "Not installed", which
renders a fresh install (3 of 16 engines ready) as a mostly-broken app.
The sections now say "Ready to use" / "Add more engines", and the
unavailable-row details toggle asks "What it needs" instead of "Why
unavailable?" — same information, framed as headroom to unlock.

Council outcome (first-run seat): unavailable engines must read as more
you could install, never as brokenness. Keys added to all 21 locales.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 15:38:35 +00:00
Palash DebnathandClaude Fable 5 9832fbd693 feat(engines): switch engines from anywhere — footer quick switch, workspace chips, shortcuts (#1530)
* feat(engines): add quick switching controls

* docs(changelog): note engine quick switching (#1530)

* fix(support): theme amount cards

* fix(support): restore themed amount cards

* fix(engines): address quick switch review findings

* fix(engines): green the full frontend suite around the quick switch

Three failure classes the targeted runs missed:
- the popover referenced --chrome-radius, which does not exist; it now
  wears the footer's shared MENU_SURFACE like the compute popover
- LogsFooter tests hand-wrote their api/system and api/hooks mocks, which
  drop every export the footer gains next; they are partial mocks now
- DubHeader/AudiobookHero tests rendered without a QueryClientProvider,
  which useEngines needs

Also: workspace-header chips open the popover downward (dropUp stays on
the footer instance) so it cannot clip off the top of the viewport.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(engines): one QueryClient per test module, not per render

CodeRabbit: the inline client made every wrapper render a fresh cache,
so rerender() restarted the /engines query mid-test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 15:21:05 +00:00
Giuseppe Rojas cd54113173 fix(security): close server-mode admin bypasses (#1525)
* fix(security): require keys for remote admin actions

* fix(frontend): guard unavailable scrollIntoView

* docs: link changelog to PR 1525

* fix(security): align PIN-only discovery policy

* fix(security): preserve strict sidecar boundary

* fix(security): normalize remote API keys

* fix(auth): normalize credential fallback order
2026-08-13 05:04:29 +00:00
Palash DebnathandClaude Opus 5 6948399e61 fix(wayland): rewrite a stale portal identity instead of trusting it (#1526)
* fix(wayland): rewrite a stale portal identity instead of trusting it

Found live: with the desktop entry's Exec pointing at a binary that had
been moved, GLib resolves the entry to NULL, the host portal answers
"Could not register app ID: App info not found", CreateSession then
fails with "An app id is required" — and the dictation shortcut is
silently dead for the entire session. Only the focused-window fallback
keeps working, which reads as "the shortcut randomly stopped".

ensure_desktop_identity() trusted any existing entry. It now validates
the USER-LOCAL entry's Exec target and rewrites the entry when the
program is gone (parsing both the current quoted spelling and the
unquoted one older builds wrote). System-dir entries stay untouched —
deb installs manage their own.

The stale-entry class is easy to hit in the wild: a dev entry pinned to
target/debug survives cargo clean; an AppImage entry survives the file
being moved or renamed.

(Rebuilt from the first push, whose `git add -A` had swept in another
working session's unrelated in-progress files; this commit carries only
the wayland fix and its changelog line.)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(wayland): read Exec from the Desktop Entry group only

CodeRabbit, #1526: find_map over every line accepted an Exec= from a
[Desktop Action …] group, so an entry with no main-group Exec — which
GLib resolves to NULL — could be retained as healthy, keeping exactly
the stale identity the rewrite exists to replace. Parsing is scoped to
[Desktop Entry] now, with an action-only regression case.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 05:00:23 +00:00
Palash DebnathandClaude Opus 5 92b1ee5d1b test(asr): parse the selector guard instead of grepping it (#1524)
* test(asr): parse the guard, don't grep it

Two Majors CodeRabbit raised on #1523 — which I merged before reading
them, so this is the follow-up rather than a fix on the branch.

- Approved files were matched by BASENAME, so any future
  `<anything>/asr_backend.py` was exempt from the guard it exists to
  enforce. Matching is by relative path now; a decoy
  `backend/engines/asr_backend.py` calling the selector is caught.
- Detection was a line regex, wrong in both directions: it missed
  `import get_active_asr_backend as pick` and fired on the name inside
  docstrings and comments. It walks the AST now, alias-aware, so only
  real calls count.

Both verified by planting the exact bypasses: an aliased call in
services/tts_backend.py and the decoy module above. Neither was caught
before this change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test(asr): resolve the selector's bindings before flagging a call

CodeRabbit, #1524: matching any call named get_active_asr_backend also
reported a local helper or an unrelated object's method that happens to
share the name. False positives are how a guard stops being believed —
people add allowlist entries for code that was never the bug.

Bindings are resolved first now: a bare call counts only if the name was
imported FROM services.asr_backend, an attribute call only if it hangs
off a module alias for it. Six shapes are pinned in the suite — direct,
aliased and module-attribute calls flagged; a same-named local function,
an unrelated method, and the name inside a docstring not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 01:40:01 +00:00
Palash DebnathandClaude Opus 5 9760f0c501 feat(ui): one support page, and tabs where there were toggles (#1522)
* feat(ui): one support page, and tabs where there were toggles

Sponsor, commercial licence and contact were three destinations for one
question — how do I support this / how do I reach these people — and each
one made you leave to find the others. They are now three sections of a
single page: support, licence, contact, in that order, separated by a
hairline rather than more chrome. Every existing entry point still works;
`initialView` scrolls to the right section instead of hiding the other
two, so the footer heart, the dub/export commercial-licence links and
Contact all land where they meant to.

ContactPage becomes `ContactSections` — the body without the shell — and
its "Support the project" CTA now scrolls up to the support section
rather than navigating, because that surface is on the same page.

Model Catalogue: the Engines/Models switch and the matrix's TTS/ASR/LLM
switch are tabs, not Segmented. These pick between workspaces, not
between the two states of one setting, and Tabs carries roving tabindex
and role="tab" from the primitive. The matrix tabs keep their active
engine chip and now keep their hover title too — Tabs passes `title`
through.

Tests: the pane/family switches are driven by pointer down, not click —
Radix activates on pointer down, so a bare fireEvent.click leaves the
pane unchanged and reads as a switcher that ignores itself. The contact
suite now covers the section (its host owns the header), and asserts the
support CTA scrolls without ever reaching for Ko-fi.

Full frontend suite: 2022 passed. The one unhandled `window is not
defined` rejection in the parallel run predates this change — same error,
same count, on the base commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(engines): available engines first, and unavailable ones recede

Two things the matrix got wrong for a list you pick FROM: it rendered in
payload order, so a usable engine could sit under four you cannot select,
and an unavailable row was faded WHOLE — which took its status badge and
GPU chips down with it, the two things that say why it is unavailable.

Available rows now sort to the top, preserving registration order inside
each group (that order is meaningful — it puts the defaults first). The
name of an unavailable engine recedes instead, and its mark dims with it;
the evidence stays at full contrast.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(support): the footer heart returns you to the support section

CodeRabbit, #1522: App.jsx renders SupportPage in the same tree position
for donate / enterprise / contact, so React keeps ONE instance and only
swaps props. The scroll effect treated 'support' as "already at the top"
and returned early — correct for a fresh mount, wrong for the only way
this page is actually reached. Clicking the footer heart from the contact
section left you sitting on contact.

Every view scrolls now. The regression test drives the prop change the
way the router does and fails without the fix.

Also adds the (#NNN) refs the Unreleased entries were missing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test(support): cover the enterprise route's licence section

CodeRabbit, #1522: the suite drove support and contact but not the third
destination — and every section renders regardless, so only the scroll
target proves the mapping. Uses the exact initialView App.jsx passes for
mode === 'enterprise'.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 01:37:09 +00:00
Palash DebnathandClaude Opus 5 41a8a7b644 test(asr): guard the raw selector across the whole backend (#1523)
The recurrence guard from #1512 scanned api/routers only. A service or
engine module that transcribes on a request's behalf skips ensure_loaded()
just as thoroughly, so the guard could be sidestepped by moving the call
one module down the stack — verified: adding a get_active_asr_backend()
call to services/tts_backend.py passes the router scan and fails this one.

The broader scan is the one thing #1519 did better than the fix that
landed in #1515; absorbing it here rather than leaving it in a PR that
now conflicts. Thanks @ahov520.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 01:03:40 +00:00
Palash DebnathandClaude Opus 5 41722afe3b refactor(launchpad): quieter, borderless design refresh (#1515)
* refactor(launchpad): quieter, borderless design refresh

The launchpad carried decoration from an earlier direction: icon chips,
corner-hung count badges, a permanently visible filled arrow, uppercase
mono card titles, and a dotted stipple divider — plus a frame that had
been invisible since the app-wide border tokens were zeroed.

Rework it around what the borderless direction actually implies:

- Feature tiles get a whisper-faint surface instead of a dead frame, and
  read as three bands (bare glyph + count / title + arrow / description).
  `--card-hue` is spent sparingly — the glyph at rest, the surface, count
  and arrow only once raised. Titles move to sans sentence case; counts
  are plain tabular numerals. Lift softened 4px -> 2px, coloured glow ->
  neutral shadow, plus an explicit focus ring and a staggered entrance.
- Hero drops the boxed "646" pill and the filled A/B-Compare button for
  quiet type, with a hairline standing in for the separation.
- Section labels trade the dotted stipple for a single fading hairline;
  rows are transparent until hover and reveal "Open" on hover/focus (it
  stays in the DOM, so AT and keyboard always reach it).
- Hero, tiles, recent files, callout and project lists now share one
  1180px column — previously only the top half was capped, so lists ran
  edge-to-edge on a wide display while the deck stayed centred.

Two bugs found and fixed while doing it:

- Buttons that had `border border-solid border-transparent` removed fell
  back to the UA default border and rendered a visible 1px outline. They
  now carry `border-0` explicitly.
- `.lp-animate` used `animation-fill-mode: both`, so after the entrance
  it kept owning `transform` — and animation-origin declarations outrank
  normal ones, which silently killed the card hover lift. Now `backwards`,
  which still holds the from-state through the stagger delay.

Also drops CSS the page has not rendered since #904: the cursor-spotlight
layer, the breath ring, and the per-card waveform strip.

Verified with headless renders at 1600/1280/940 and the empty state.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(dictation): decode Wayland portal signals and show the capture pill

The GlobalShortcuts portal declares Activated/Deactivated as
(o session, s shortcut_id, t timestamp, a{sv} options). We decoded the
timestamp as u32, so zbus rejected every signal with

  Signature mismatch: got `(osta{sv})`, expected `(osua{sv})`

and the press was dropped as an invalid signal. Registration succeeded
and the desktop even reported the bound chord back, so the hotkey looked
wired up while doing nothing at all — on every Wayland compositor, for
the whole life of the feature (#1490). Decode the 64-bit timestamp, and
keep the 32-bit spelling as a fallback so a non-conforming portal
degrades to working rather than to silence.

With presses arriving, the second half of the failure showed: nothing
had shown the widget window since it became a hidden recorder host, so a
capture ran with no pill on screen — and a mic or Accessibility failure
rendered into a window nobody could see. Add show_dictation_pill, which
bottom-centres the capsule on the monitor under the pointer and shows it
without taking focus (Windows keeps SW_SHOWNOACTIVATE so paste still
lands in the user's document), and call it from the widget for every
state but idle. Wayland denies clients their own placement, so the
compositor picks the spot there; the pill still appears.

dispatch_dictation_capture now logs whether a press was emitted or
queued — a press that reaches Rust and produces nothing was otherwise
indistinguishable from one the compositor never delivered.

Tests: portal signals decode at both timestamp widths (the 64-bit case
fails before this change with the exact production error); pill
placement centres, respects a second monitor's origin, and clamps rather
than going off-screen; the widget shows for a state needing the user,
stays hidden while idle, and never shows for a press that arrives while
dictation is disabled.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: sync in-progress workspace changes

Uncommitted work already in the tree, checkpointed so the branch matches
the local machine:

- Remote GPU workers: join-from-the-app flow, one-time secrets, QR join
  codes, a Compute control in the status bar, and the device-list
  Workers panel (#1516)
- Model Catalogue workspace, with Settings pointing at it
- Settings sidebar search and keyboard navigation
- Demo assets for dubbing, dictation and voice design, plus the scripts
  that render them
- Backend: validation-error handling, ASR request-path degradation, and
  the accompanying tests
- CHANGELOG entries for the above and for the Wayland dictation fix

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(tests): follow Engines to the Model Catalogue, and green the sweep

- test_supertonic3 asserted the license gate points at "Settings" while
  the engine now names Model Catalogue → Engines, which is where the
  accept button actually lives. The assertion follows the move; what it
  pins is unchanged — the hint must name a place the user can reach it.
- Carries the CJK allowlist entries for the rendered dub bundle (#1517)
  and the regenerated route snapshot for /workers/agent (#1516), both of
  which this branch inherits from the workspace sync.
- docs/install/linux.md: the dictation capsule is bottom-anchored
  everywhere except Wayland, where the protocol gives applications no
  say in their placement. Documented rather than left as a surprise
  (CodeRabbit).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* ci: stop a flaky dependency fetch from failing green runs

en-core-web-sm resolves to a direct GitHub release URL, and github.com
intermittently answers `http2 error: refused stream before processing
any application logic`. uv's own three retries all land within the same
few seconds and fail together, so the whole job dies on a dependency
that has nothing to do with the change under test — it cost #1518 and
#1517 an otherwise-green run tonight.

Two changes: back off between whole `uv sync` attempts, which is what
actually clears it, and pass --no-sync to the pytest steps. `uv run`
re-resolves the environment before running, so every test step was a
fresh chance to hit the same fetch even though the install step had
already synced — that is exactly how #1518 failed, in the isolated
backend/tests step, with all 5467 tests already passed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* ci: one retry seam for every uv sync, not just the job that failed last

en-core-web-sm resolves to a direct GitHub *release* URL rather than a
package index, and github.com intermittently answers `http2 error:
refused stream before processing any application logic`. uv's own
retries all land inside the same ~10 seconds and fail together, so a job
dies on a dependency unrelated to the change under test. Tonight that
cost four otherwise-green runs across #1515, #1517 and #1518 — and the
first fix only covered the Tests job, so the next failure simply moved
to Smoke (Linux), which syncs separately.

The fetch is per-job, so the fix has to be per-job: scripts/uv-sync-retry.sh
backs off between whole attempts (15s, 45s, 90s) and every workflow that
syncs now goes through it — ci.yml (tests + the platform matrix),
release.yml, security.yml, evals.yml. It still fails loudly after four
attempts, so a genuinely broken lockfile is not disguised as a flake.

The Tests job also lacked the UV_HTTP_TIMEOUT / UV_HTTP_RETRIES the smoke
matrix has always set, which is part of why it was the one that kept
dying; it has them now.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test(ci): pin the Intel-Mac contract by intent, not by command spelling

test_ci_verifies_intel_mac_as_the_documented_remote_only_host asserted
the literal line `run: uv sync --extra pockettts`, so routing every sync
through scripts/uv-sync-retry.sh read as a broken Intel-Mac contract. The
contract it exists to protect is that the pockettts extra installs ONLY
on backend_supported legs — which the regex now pins, while leaving how
the sync is invoked free to change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* ci: keep every uv run out of the resolver, and bound the retry budget

CodeRabbit, #1517:

- `uv run` re-resolves before running, so the smoke suite, the
  worker-artifact tests, the release test run and the eval run were each
  a fresh chance to hit the flaky direct-URL fetch outside the retry
  loop. All of them pass --no-sync now; the environment is already
  synced by the step that owns the retries. security.yml's
  `uv run --with pip-audit` is deliberately left alone — it layers an
  ephemeral package rather than running the project's own tests.
- The retry count multiplied uv's own budget (UV_HTTP_RETRIES=5 with a
  120 s timeout on the smoke matrix). Three attempts and 60 s of total
  backoff outlast the refusals actually observed while staying well
  inside the jobs' timeout-minutes.
- The Intel-Mac contract test pinned the smoke command literally too, so
  --no-sync tripped it exactly like the sync line did. Same fix: assert
  the contract (smoke runs only on backend_supported legs), not its
  spelling.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 23:46:07 +00:00
Palash DebnathandClaude Opus 5 41c098e009 feat(demos): ship the demo audio and video the app already advertises (#1517)
* feat(demos): ship the demo audio and video the app already advertises

Every demo asset in the app was a dead link on anything but a Mac.

`personalities.py` has carried a `preview_url` for each of the seven
voice-design presets since they were added; DictationDemo.jsx posts three
bundled WAVs to /transcribe so the feature can be shown without microphone
permission; the Dub workspace reads a manifest and plays a source video plus
four dubbed languages. None of those files were committed, because the tooling
that renders them (scripts/build_demos.sh, scripts/build_dub_demo.sh) hard-
requires macOS `say` — it even carries a `TODO: add espeak-ng path for Linux
contributors`. So the presets returned 404, the replay buttons did nothing, and
the dubbing demo never loaded.

Rendered with VoiceStudio's own engine, which runs wherever the app does:

- 7 voice-design previews (2.2 MB)
- 3 dictation replay clips (1.1 MB) — verified by transcribing them back:
  the conversational and French clips round-trip exactly
- dubbing demo: source + 4 dubbed videos with subtitles and manifest (9.6 MB)

Tooling fixes this turned up:

- build_dub_demo.sh wrote to backend/assets/demo/dubbing, but main.py mounts
  backend/assets/samples at /demo_audio — so the frontend's
  /demo_audio/demo/dubbing/manifest.json could never have resolved even after
  a successful Mac build. Output moved under the mount.
- `say` is now the fallback rather than the requirement: the new
  scripts/render_dub_demo_audio.py renders the five tracks with the engine and
  the shell script picks them up.
- The five demo paragraphs lived in two files. They are now one JSON both read
  — two copies is one edit away from a video whose subtitles disagree with it.
- render_demos_omnivoice.py peak-normalized, which a single-sample transient
  defeats: the Helpdesk preset landed at -30 dB RMS against -17 dB for its
  neighbours, so the preview row played at wildly different volumes. Now EBU
  R128 at -18 LUFS with a -1.5 dBTP ceiling.
- …and pinning the output rate, because loudnorm resamples to 192 kHz
  internally and writes there unless told otherwise, which turned 2.1 MB of
  previews into 17.5 MB of identical-sounding audio.
- update_manifest() looked for a manifest at a path nothing writes, so it
  always printed "not found" and did nothing.
- Dictation is rendered here now too. It was excluded on the grounds that
  `say` was good enough and engine TTS was overkill — true only on macOS.

tests/test_demo_assets_exist.py resolves every advertised URL against the
directory main.py actually mounts, and checks each dubbing subtitle matches the
script its manifest entry claims. A missing static file is not an import error
and not a failing request; nothing would have caught this otherwise.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(changelog): stamp the demo-asset entries with their PR ref

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(demos): watermark rendered demo audio, and harden the render scripts

Review findings on #1517:

- Greptile P1: the renderers wrote engine output straight to disk, so a
  re-render shipped demo audio with no provenance mark. These clips play
  back to users as VoiceStudio output — they are synthetic audio leaving
  the app like any other, and now go through mark_synthetic (#1169), the
  one chokepoint every producing route uses. It runs on the file AFTER
  loudnorm, since loudnorm re-encodes what it is handed, and says so
  loudly when marking is unavailable rather than committing an unmarked
  asset. The dubbing renderer shares the same helper.
- CodeRabbit: build_dub_demo.sh checked only source.src.wav before
  deciding it could run without macOS `say`, so a Linux or Windows run
  with four of five tracks present reached a missing one, called `say`,
  and left a half-built bundle. It now requires all five.
- CodeRabbit: shutil.move over an existing path delegates to os.rename,
  which raises FileExistsError on Windows — os.replace overwrites
  atomically everywhere.
- CodeRabbit: the preview test discovered presets in a parametrize
  argument, importing app code at collection time and leaving
  core.personalities in sys.modules for later tests. Discovery moved into
  the test body.

CI: the rendered dub bundle's zh/ja subtitles, its manifest and the
script source are dubbing CONTENT, not UI strings — allowlisted in
test_no_hardcoded_cjk.py with that justification.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(demos): a render that cannot be watermarked fails instead of warning

CodeRabbit and Greptile, #1517: mark_synthetic degrades rather than
raising — correct for generation, wrong for a render script, whose whole
job is to produce files a human then commits. A printed warning on a
scrolling console is not a gate, so both scripts exited 0 with unmarked
assets sitting on disk ready to commit. They now raise, with the reason
and the fix; OMNIVOICE_DEMO_ALLOW_UNMARKED=1 stays for a local listen.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* ci: stop a flaky dependency fetch from failing green runs

en-core-web-sm resolves to a direct GitHub release URL, and github.com
intermittently answers `http2 error: refused stream before processing
any application logic`. uv's own three retries all land within the same
few seconds and fail together, so the whole job dies on a dependency
that has nothing to do with the change under test — it cost #1518 and
#1517 an otherwise-green run tonight.

Two changes: back off between whole `uv sync` attempts, which is what
actually clears it, and pass --no-sync to the pytest steps. `uv run`
re-resolves the environment before running, so every test step was a
fresh chance to hit the same fetch even though the install step had
already synced — that is exactly how #1518 failed, in the isolated
backend/tests step, with all 5467 tests already passed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* ci: one retry seam for every uv sync, not just the job that failed last

en-core-web-sm resolves to a direct GitHub *release* URL rather than a
package index, and github.com intermittently answers `http2 error:
refused stream before processing any application logic`. uv's own
retries all land inside the same ~10 seconds and fail together, so a job
dies on a dependency unrelated to the change under test. Tonight that
cost four otherwise-green runs across #1515, #1517 and #1518 — and the
first fix only covered the Tests job, so the next failure simply moved
to Smoke (Linux), which syncs separately.

The fetch is per-job, so the fix has to be per-job: scripts/uv-sync-retry.sh
backs off between whole attempts (15s, 45s, 90s) and every workflow that
syncs now goes through it — ci.yml (tests + the platform matrix),
release.yml, security.yml, evals.yml. It still fails loudly after four
attempts, so a genuinely broken lockfile is not disguised as a flake.

The Tests job also lacked the UV_HTTP_TIMEOUT / UV_HTTP_RETRIES the smoke
matrix has always set, which is part of why it was the one that kept
dying; it has them now.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test(ci): pin the Intel-Mac contract by intent, not by command spelling

test_ci_verifies_intel_mac_as_the_documented_remote_only_host asserted
the literal line `run: uv sync --extra pockettts`, so routing every sync
through scripts/uv-sync-retry.sh read as a broken Intel-Mac contract. The
contract it exists to protect is that the pockettts extra installs ONLY
on backend_supported legs — which the regex now pins, while leaving how
the sync is invoked free to change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* ci: keep every uv run out of the resolver, and bound the retry budget

CodeRabbit, #1517:

- `uv run` re-resolves before running, so the smoke suite, the
  worker-artifact tests, the release test run and the eval run were each
  a fresh chance to hit the flaky direct-URL fetch outside the retry
  loop. All of them pass --no-sync now; the environment is already
  synced by the step that owns the retries. security.yml's
  `uv run --with pip-audit` is deliberately left alone — it layers an
  ephemeral package rather than running the project's own tests.
- The retry count multiplied uv's own budget (UV_HTTP_RETRIES=5 with a
  120 s timeout on the smoke matrix). Three attempts and 60 s of total
  backoff outlast the refusals actually observed while staying well
  inside the jobs' timeout-minutes.
- The Intel-Mac contract test pinned the smoke command literally too, so
  --no-sync tripped it exactly like the sync line did. Same fix: assert
  the contract (smoke runs only on backend_supported legs), not its
  spelling.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 23:29:19 +00:00
Palash DebnathandClaude Opus 5 a310141114 feat(workers): join from the app, share by QR, and a status-bar Compute control (#1516)
* feat(workers): join from the app, share by QR, and a status-bar Compute control

Remote workers shipped with a hole in the middle: the control plane could
mint join codes, and on the other machine there was nothing to paste them
into. Becoming a worker meant launching with OMNIVOICE_WORKER_MODE and
OMNIVOICE_WORKER_TOKEN in the environment and relaunching — on the machine
that is usually the least convenient one to configure by hand.

Backend

- GET /workers/agent, POST /workers/agent/join, POST /workers/agent/enabled.
  Join redeems a code and starts the agent live; no restart.
- Worker mode now persists in settings as well as the environment (env still
  wins, and the panel is told so it can disable a switch it cannot honour),
  and it is written only after a join that actually worked — a failed
  enrolment must not have the app retrying on every launch.
- The endpoint carried by the redeemed code is remembered. Without that a
  machine that joined from the UI came back up enrolled but with nowhere to
  dial, and the only fix was OMNIVOICE_WORKER_ENDPOINT.

UI

- "Lend this machine's GPU": paste the code, Join. Once joined it offers a
  switch rather than another code, because the pinned certificate survives.
- <OneTimeSecret/> renders join codes and connection strings as a QR next to
  the text, with a live expiry countdown, and is used by both halves. QR
  generation is best-effort: a string past the format's capacity still shows
  the code and Copy, because losing the QR is a degraded share and losing the
  only copy of a one-time secret is data loss.
- Status-bar Compute control: pick local or a machine, flip the feature, mint
  a join code — without opening Settings. Absent entirely until the user has
  opted in or enrolled something.
- Remote workers now reads as a device list: status dot, address, latency,
  live task meter, resident models, last seen; housekeeping actions revealed
  on hover; a three-step empty state.
- Approve is on the row. A worker could connect, sit there labelled "Not
  approved" and never be usable, with no way out of it in the UI.

Fixes found on the way

- Status dots and menu surfaces in the GPU picker were painted from fixed
  Tailwind palette classes (bg-emerald-400, text-amber-400, hover:bg-white/5),
  so on Midnight or Catppuccin they showed Gruvbox colours next to the
  theme's own. Both controls now paint from themed --color-* tokens, shared
  in computeTarget.jsx along with the JSON wrapper all three copies duplicated.
- Button funnels every child into one <span>, so an icon passed as a child
  renders glued to its label — the flex gap only applies to the `leading`
  slot. Six buttons across these panels were affected.
- InboundNodePanel passed `variant="warning"` to Badge, which takes `tone`;
  the "on your network" warning rendered as an ordinary neutral pill.

Docs updated in the same change (docs/remote-workers.md): the join flow, the
QR, the status-bar control, and the new environment variable.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(changelog): stamp the remote-workers entries with their PR ref

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(workers): a join is not done until the control plane accepts it

Review findings on #1516:

- Greptile P1: `start()` only SCHEDULES the dial-out loop, so a control
  plane that rejected this worker — expired token, wrong address, a
  server that never answers — looked identical to a successful join. The
  route persisted worker mode, reported success, and the machine retried
  forever on every launch. The agent now signals first registration, and
  join waits for it before persisting anything.
- CodeRabbit: a failed REJOIN left the machine unable to reconnect to the
  control plane it was already serving, because pinning the new
  certificate overwrites the old one on disk. Snapshot the pinned
  certificate, endpoint and setting up front, and restore them (and the
  running agent) when the join fails.
- CodeRabbit: join and the enable toggle awaited stop()/start() with no
  exclusion, so two concurrent requests could interleave their pairs and
  have `start()` return early — reporting success for a control plane it
  never dialled. Both now hold one lifecycle lock.
- CodeRabbit: with OMNIVOICE_WORKER_MODE set, the toggle still started or
  stopped the agent and wrote a setting the rest of the app ignores,
  contradicting the env_pinned status it reports. It now answers 409 and
  says which variable is in charge.
- CodeRabbit: the QR code kept encoding the previous secret until the new
  one finished encoding, so the code on screen could disagree with the
  text beside it.

CI: regenerated tests/fixtures/api_routes.txt for the three
/workers/agent routes.

Tests: a join the control plane never accepts is a 409 that persists
nothing and leaves no agent dialling; a failed rejoin restores the
previous certificate, endpoint and setting; an env-pinned machine
refuses the toggle.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(workers): the environment pin governs joining too, not just the toggle

CodeRabbit, #1516:

- join_control_plane skipped the env_pinned guard set_agent_enabled
  enforces, and joining is precisely what ENABLES worker mode: under
  OMNIVOICE_WORKER_MODE it wrote a setting nothing consults, and with the
  variable pinned off it handed back a machine that reported a successful
  join and lent nothing. One shared guard now covers both routes.
- Two of the three rollback assertions could not fail before the fix
  (nothing wrote those settings on the failure path). The test now pins
  the behaviour only the rollback produces: the previous enrollment is
  dialling again, rather than left stopped until someone notices.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 22:20:41 +00:00
Palash DebnathandClaude Opus 5 4fc07c21fd fix(appimage): stop shipping a dangling .DirIcon, and prove it in CI (#1518)
* fix(appimage): stop shipping a dangling .DirIcon, and prove it in CI

The Linux icon is blank because the AppImage's .DirIcon is an absolute
symlink into the machine that built it. From the published v0.4.2:

  .DirIcon -> /home/runner/work/OmniVoice-Studio/OmniVoice-Studio/frontend/
              src-tauri/target/x86_64-unknown-linux-gnu/release/bundle/
              appimage/OmniVoice Studio.AppDir/OmniVoice Studio.png

That path exists on nobody's computer. The link dangles the moment the
AppImage leaves CI, so file managers have no icon for the file, and the
integration tools that read .DirIcon install nothing. A dangling symlink is
not a build error — the bundle packs, runs, and passes every check we had —
which is how it shipped for a whole release without anyone noticing.

Locally built AppDirs are worse: both .DirIcon AND the root .desktop symlink
come out absolute, so a from-source bundle has no readable desktop entry
either, which is why the icon is missing in the menu and the dock too.

- `.DirIcon` is now a real file, copied in through `appimage.files` — the
  same seam that already places the WebKitGTK marker.
- `bundle.category` is set, so the generated desktop entry stops emitting an
  empty `Categories=`. That is not the same as omitting the key:
  desktop-file-validate rejects the entry and menu builders skip it.
- verify-apprun-bundle.sh — already run against the extracted AppImage in the
  release job — now fails when .DirIcon is missing or resolves outside the
  bundle, when the .desktop entry does not resolve inside it, when Icon=
  names a file that is not at the AppImage root, or when Categories= is
  present but empty. Its unit test covers each of those, including the exact
  shape v0.4.2 shipped.

The `.DirIcon` copy cannot be verified without a full release build, so the
guard is the load-bearing part: the next release either passes it or fails
loudly. It can no longer ship blank in silence.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(changelog): stamp the AppImage icon entries with their PR ref

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(changelog): fold the AppImage icon fix into the existing Fixed section

CodeRabbit (#1518): the Unreleased block must carry one `### Fixed`
section of one-line entries. Merge the two entries in and drop the
narrative and the version reference.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 22:08:26 +00:00
Palash Debnath 8fdc5292e4 Merge pull request #1508 from anyingiit/fix/i18n-zh-cn-polish
fix(i18n): polish Simplified Chinese translations
2026-08-12 15:33:46 +00:00
debpalash a4fd62f85f fix(i18n): finish zh-CN review fixes 2026-08-12 15:15:49 +00:00
debpalash 6eaeb23403 Merge remote-tracking branch 'origin/main' into fix/i18n-zh-cn-polish 2026-08-12 15:15:06 +00:00
Palash Debnath fa8fddc456 Merge pull request #1506 from debpalash/fix/appimage-custom-apprun
fix(release): validate wrapped AppImage launcher
2026-08-12 14:57:23 +00:00
MyAnyAgent[bot] 7a7ce47b64 fix(i18n): address CodeRabbit zh-CN review findings
Accept: active-engine term, GPU-accel chip, persona export states,
slot-term consistency, preserve Cinematic token in readiness labels,
hf-token toast wording, hold-mode term. Decline: gallery.saved_as_profile
(faithful to en source) and HuggingFace spacing (matches en source).
2026-08-12 14:15:35 +00:00
MyAnyAgent[bot] 7810581beb docs: record zh-CN translation fixes (#1507) 2026-08-12 14:02:56 +00:00
debpalash a089359b02 docs: place release smoke under changes 2026-08-12 13:52:43 +00:00
debpalash 4227cbd07d docs: record AppImage smoke fix 2026-08-12 13:48:34 +00:00
MyAnyAgent[bot] 4b86692653 fix(i18n): polish Simplified Chinese translations
Correct mistranslations of brand names and technical terms (Discord →
不和谐, Tailscale → 尾鳞, LLM (Cinematic) → 法学硕士(电影), IPA →
异丙醇, Hugging Face → 拥抱脸部, ports → 港口, cast → 施法, generations
→ 各代人) and ~250 more awkward machine-translation strings across
zh-CN.json. All keys and {{placeholder}}/<n> tokens preserved; locale
parity test green.
2026-08-12 13:15:33 +00:00
debpalash b99eadfafc Merge remote-tracking branch 'origin/main' into fix/appimage-custom-apprun 2026-08-12 12:42:52 +00:00
Palash Debnath 3f32f97e09 Merge pull request #1505 from debpalash/fix/worker-id-restart-race
fix(tests): synchronize worker identity persistence
2026-08-12 12:26:03 +00:00
debpalash 45eda7db01 test(worker): await identity persistence signal 2026-08-12 12:04:44 +00:00
debpalash fce5075e77 docs: record worker identity CI fix 2026-08-12 11:36:08 +00:00
debpalash 0fb9b38bfa fix(tests): synchronize worker identity persistence 2026-08-12 11:35:29 +00:00
debpalash 64973d1829 fix(release): validate wrapped AppImage launcher 2026-08-12 11:24:01 +00:00
Palash Debnath 57e435d3c0 Merge pull request #1504 from debpalash/fix/remote-port-and-ui-schema
fix: restore dub tracks and identify worker ports
2026-08-12 11:22:41 +00:00
debpalash 20d9cb5f1b docs: record remote recovery fixes 2026-08-12 11:00:33 +00:00
debpalash d9a939dff8 fix(desktop): restore dub tracks and identify worker ports 2026-08-12 02:54:50 +00:00
Palash Debnath 6f1e26a950 Merge pull request #1503 from debpalash/fix/remote-backend-port
fix: let remote backend bypass local setup
2026-08-12 02:01:08 +00:00
debpalash 06ca67650e docs: record remote startup recovery 2026-08-12 01:43:03 +00:00
debpalash 2e8a08973d fix(desktop): let remote backend bypass local setup 2026-08-12 01:42:32 +00:00
debpalash f47cec164f Merge pull request #1495 from velixio/main 2026-08-12 01:08:55 +00:00
debpalash 6c58294d56 fix(worker): close remaining public error flows 2026-08-12 00:28:35 +00:00
debpalash b548a7ab9d fix(security): close worker transport disclosure flows 2026-08-12 00:22:59 +00:00
debpalash b74703d506 style(worker): format integrated UI changes 2026-08-11 23:56:03 +00:00
debpalash 76d8c11553 fix(worker): persist pinned reconnect credentials 2026-08-11 23:54:28 +00:00
Codex 5818e19137 fix(worker): secure inbound node connections 2026-08-11 23:51:29 +00:00
Codex 80eca8cc6f fix(workers): harden inbound panel boundaries 2026-08-11 23:50:30 +00:00
debpalash 037a5689de fix(worker): address legacy transport review findings 2026-08-11 23:49:46 +00:00
debpalash 5b806124c2 Merge remote-tracking branch 'origin/main' into fix/pr1495-final
# Conflicts:
#	CHANGELOG.md
2026-08-11 22:29:56 +00:00
Palash Debnath 50dd851bf8 Merge pull request #1502 from debpalash/feat/ui-scale-onboarding
feat: add first-run interface scaling
2026-08-11 22:13:26 +00:00
velixio 2165339a36 docs(changelog): record the cuBLAS workspace fix and the reserved-VRAM readout 2026-08-12 03:32:34 +05:30
velixio b0a6fdfdc2 Merge remote-tracking branch 'upstream/main'
# Conflicts:
#	CHANGELOG.md
#	frontend/src/components/settings/ModelStoreTab.jsx
#	frontend/src/i18n/locales/ar.json
#	frontend/src/i18n/locales/de.json
#	frontend/src/i18n/locales/en.json
#	frontend/src/i18n/locales/es.json
#	frontend/src/i18n/locales/fr.json
#	frontend/src/i18n/locales/hi.json
#	frontend/src/i18n/locales/id.json
#	frontend/src/i18n/locales/it.json
#	frontend/src/i18n/locales/ja.json
#	frontend/src/i18n/locales/ko.json
#	frontend/src/i18n/locales/nl.json
#	frontend/src/i18n/locales/pl.json
#	frontend/src/i18n/locales/pt.json
#	frontend/src/i18n/locales/ru.json
#	frontend/src/i18n/locales/sv.json
#	frontend/src/i18n/locales/th.json
#	frontend/src/i18n/locales/tr.json
#	frontend/src/i18n/locales/uk.json
#	frontend/src/i18n/locales/vi.json
#	frontend/src/i18n/locales/zh-CN.json
#	frontend/src/i18n/locales/zh-TW.json
2026-08-12 03:27:08 +05:30
debpalash ffc6fade50 fix(ui): preview first-run scale choices 2026-08-11 21:45:52 +00:00
velixio 5bb50f7832 fix(memory): clear cuBLAS workspaces so an unloaded model's segment can go
After the ordering fix, unloading the model on a 4090 still left the GPU at
1238 MiB with torch reporting 8.5 MB allocated and 803 MB reserved -- and no
number of Flush Memory presses moved it. A segment dump said why: ONE 803 MB
segment, 794.7 MB of it inactive-but-split, pinned by a single live block of
8,519,680 bytes.

That is cuBLAS's default workspace. It is taken from the caching allocator on
first use, so it lands inside whatever segment the model load had just grown,
and it is held for the life of the cuBLAS handle. empty_cache() can only
return segments that are entirely free, so one 8.5 MB block kept three
quarters of a gigabyte from ever reaching the driver again. On a machine
lending its GPU that is the difference between an idle node costing 470 MiB
and costing 1.2 GB.

free_vram() now clears the workspaces before emptying the cache, on the
unload paths only -- the next cuBLAS call re-takes one, which is cheap but
not something to pay per generate. The binding is private
(torch._C._cuda_clearCublasWorkspaces), so it is optional by construction: a
build without it keeps today's behaviour rather than failing an unload.

Found by adding reserved-vs-allocated to /system/flush-memory in 642513d2.
Allocated alone reads near zero after an unload, which is exactly why this
hid for so long -- every diagnostic we had agreed the memory was free.
2026-08-12 03:08:46 +05:30
velixio 642513d205 fix(system): report reserved VRAM alongside allocated in flush-memory
memory_allocated counts live tensors only, so after an unload it reads
near zero while nvidia-smi still shows gigabytes. That gap is the whole
substance of every "flush says it worked, the GPU says it didn't" report,
and the endpoint was reporting only the half that looks good.

memory_reserved is what the caching allocator holds from the driver; the
remainder between that and the driver's own figure is the CUDA context and
kernel workspaces, which nothing in-process can hand back.
2026-08-12 02:58:42 +05:30
velixio 090cc37144 fix(memory): release the model before emptying the cache, not after
The shared voice model's unload emptied the allocator caches and *then*
dropped the reference. That frees nothing: the weights are still reachable
when gc.collect() runs, empty_cache() only returns blocks the allocator
already considered free, and the reference drops a moment later into a cache
nothing will flush again. The unload logs success, the engine leaves the
registry, and nvidia-smi does not move.

Six modules open-coded the same two lines. Exactly one had them inverted --
OmniVoiceBackend.unload, which is the path the engine-registry idle sweep
reaches, which is the sweep a headless worker node runs. So every unload a
user could trigger from the UI worked, and the one that runs unattended on a
machine lending its GPU held 3.6 GB indefinitely. Found on hardware: the
sweep fired on schedule, logged "Released 1 idle engine(s)", and VRAM stayed
flat at 3656 MiB for the next two minutes.

Replace all six with model_manager.unload_shared_model(), which clears the
reference, drops the clone-prompt side cache, then frees -- in that order,
in one place. Two callers gain the side-cache drop they were missing
(/system/flush-memory and the shutdown path), which is the same defect one
step down: an unload that kept the encoded reference tensors belonging to the
model it had just released.

A source guard asserts nothing outside model_manager assigns the shared
reference, so the next caller cannot reintroduce the ordering. It caught the
sixth site while being written.

Also give the AudioSeal watermark models the bargain every other model in the
app already makes: they loaded on the first embed and stayed resident for the
life of the process. CPU-resident, so this is system RAM rather than VRAM,
and the machines that notice are the ones running batches.

The error text on a failing unload changes with the ordering. "Could not be
unloaded, retry after the current generation finishes" was accurate when the
cache flush ran first and aborted before the release; now the release has
already happened and only the flush can fail, so it says that instead of
sending the user to repeat work that is done.
2026-08-12 02:33:04 +05:30
debpalash 0f69b4d3ca docs: record interface scale setup 2026-08-11 21:02:08 +00:00
debpalash 232784cb5a Merge remote-tracking branch 'origin/main' into feat/ui-scale-onboarding 2026-08-11 21:01:41 +00:00
Palash Debnath 32bd5cfedb Merge pull request #1491 from debpalash/feat/workspace-design-refresh
feat(ui): refresh core workspaces and settings
2026-08-11 20:39:56 +00:00
debpalash 6ba2e2a914 feat(ui): add first-run interface scaling 2026-08-11 20:35:11 +00:00
debpalash d5a496a8ad fix(ui): complete workspace review follow-ups 2026-08-11 20:24:49 +00:00
velixio 6a6f3fbc29 fix(models): do not preload a model on a machine with no local user
The startup preload exists so the first generate feels instant for the person
sitting in front of the app. A machine lending its GPU has nobody sitting
there, so it was several GB of VRAM held from boot against a request that may
never arrive — and the idle sweep could not reclaim it, because the sweep owns
the worker executor's engines while this is the default local model.

Measured on gpu2: a node that had run nothing still sat at 2.4 GB, and an idle
unload after a real job returned it to exactly that floor rather than below it.

Worker-mode processes now load on first request and release when idle, which is
what a node should do. A machine that is both a desktop app and a worker keeps
the warm-up — there is a real user there and the point stands.
2026-08-12 01:40:22 +05:30
debpalash 8a8059b9c6 fix(ui): address workspace review findings 2026-08-11 20:03:15 +00:00
velixio 5ebf21166d Merge branch 'main' of github.com:velixio/VoiceStudio 2026-08-12 01:31:18 +05:30
velixio b8fb5a14c2 feat(workers): let the idle-unload timings be shortened for testing
Watching a ten-minute rule take effect means waiting ten minutes, so it tends
not to get watched. Both numbers are now env-tunable:
OMNIVOICE_ENGINE_IDLE_UNLOAD_SECONDS and OMNIVOICE_IDLE_SWEEP_SECONDS.

They are documented as a pair, because shortening only the threshold still
means waiting a full sweep interval to see it fire — which reads as a broken
sweep and sends you looking for a bug that is not there.

Unparseable values and anything below the floor are ignored with a warning
rather than honoured. A zero threshold would hand back a model the instant it
went idle and reload it for the very next request, which is worse than the
behaviour being tuned.
2026-08-12 01:30:53 +05:30
velixio 17a364c476 Merge branch 'debpalash:main' into main 2026-08-12 01:25:10 +05:30
debpalash ce5f051252 fix(ui): remove framed structural borders 2026-08-11 19:49:10 +00:00
velixio 1ac3dcf3fe fix(workers): unload idle models on an inbound-only node
The ten-minute idle sweep lived inside the dial-out agent. A node that only
accepts inbound connections never starts that agent — on gpu2 it fails outright
with 'Set OMNIVOICE_WORKER_ENDPOINT' — so a machine lending its GPU to panels
that dial IN held several GB of weights forever. That is precisely the cost the
sweep exists to avoid, and it was silently missing in the mode most likely to
be a shared box.

The loop moves to module scope and both transports use it. Inbound starts it
when the listener starts and cancels it when the listener stops, and passes a
callback that re-advertises capabilities to every attached panel, so a control
plane's view of what is resident does not go stale the moment it becomes
useful. Local behaviour is unchanged: nothing sweeps unless a worker role runs.
2026-08-12 01:14:16 +05:30
debpalash 6535ed3284 Merge commit '99e865600b2ce730ee7ac9860f97deea75ff57a1' into feat/workspace-design-refresh-consolidated 2026-08-11 19:38:59 +00:00
debpalash 5ef5d9b73a style: format consolidated workspace changes 2026-08-11 19:35:41 +00:00
debpalash 735cdd6b6f docs: record workspace design refresh 2026-08-11 19:34:17 +00:00
debpalash 6c679bfd75 feat(launchpad): simplify creative entry screen 2026-08-11 19:34:08 +00:00
debpalash 16a1ee63ee feat(profile): simplify the voice inspector 2026-08-11 19:30:52 +00:00
debpalash 2b0cd04599 docs: align workspace design specifications 2026-08-11 19:28:52 +00:00
debpalash f0382e0290 feat(workspaces): refine voice story and audiobook flows 2026-08-11 19:28:43 +00:00
Palash Debnath 99e865600b Merge pull request #1501 from debpalash/fix/remote-backend-recovery-1496
fix: recover from unreachable remote backends
2026-08-11 19:22:57 +00:00
debpalash 680fa8fdef feat(settings): refresh responsive preferences workspace 2026-08-11 19:16:21 +00:00
velixio 40569d0657 Merge remote-tracking branch 'upstream/main'
# Conflicts:
#	CHANGELOG.md
2026-08-12 00:34:31 +05:30
debpalash cf316b18bc test(remote): exercise streamed health responses 2026-08-11 19:04:05 +00:00
debpalash 5832a81bb6 feat(onboarding): refresh the bundled demo voice 2026-08-11 19:02:38 +00:00
velixio 9ef0f4a61b Merge pull request #2 from velixio/feat/inbound-node-mode
Share one GPU machine between several people
2026-08-12 00:31:34 +05:30
debpalash 8e5a023058 fix(remote): bound startup health responses 2026-08-11 18:52:17 +00:00
velixio b718b2be46 fix(workers): accept the nested input ids that staging actually produces
Found by clicking Synthesize in the desktop UI — the one path nothing had
exercised.

task_store.stage_input mints inputs/<digest><ext>, a path rather than a bare
name. The node ran safe_filename over it, which rejects anything nested, so
every real clone input was refused, the dispatch failed, and the scheduler
retried about eighteen times a second while the 4090 sat idle and the user
watched a spinner.

The wire id is now hashed into a directory name rather than used as one. That
accepts any id the protocol allows while leaving placement entirely ours to
decide, which is the property the check was really buying. The declared
filename is still required to be a bare name, and a hostile one is still
refused outright — covered by its own test so the containment cannot be traded
away later to fix some future rejection.

Every earlier test used a flat id like 'ref-1' and so never met the shape
production emits.
2026-08-12 00:14:08 +05:30
debpalash 8d8765315f docs: link remote recovery changelog 2026-08-11 18:04:36 +00:00
debpalash dc5c9cf43e fix(remote): recover from unreachable backends
Closes #1496
2026-08-11 18:03:51 +00:00
velixio fd7f06d62e fix(workers): give each attach a fresh outbox
Found on hardware. The queue was built once per connection and reused across
reconnects, so a frame a dying session left behind became the FIRST frame of
the next attach. The node requires a registration there, aborted the call, and
the two span at full speed — session epoch 2445 inside one second, the node
logging 'Locally aborted' on repeat, and the panel reporting the machine
offline while the connection list showed it connected.
2026-08-11 23:28:27 +05:30
velixio 33714b2fe0 fix(workers): make Disconnect hold, and stop a bad paste from replacing a good key
Two more found on hardware.

Disconnect ended the session and the panel redialled two seconds later, so the
log read disconnected and connected in the same breath and the button appeared
to do nothing. A kicked key now sits out for a minute — long enough that the
disconnect is real and the person notices, short enough that it is plainly not
a revocation, which stays a separate and permanent action. The docs now say
which of the two buttons does which.

Re-pasting a connection string for an already-connected machine saved the new
string and then short-circuited on the existing session, so a wrong key
reported success, kept running on the old connection, and only failed after a
restart — by which point nothing pointed back at the paste that caused it. The
live session is now torn down before the new one is dialled.
2026-08-11 23:22:47 +05:30
Palash Debnath 008c8a70a6 Merge pull request #1500 from debpalash/fix/wavesurfer-abort-report-1498
fix(ui): ignore expected aborted audio streams
2026-08-11 17:47:10 +00:00
velixio 4f2dea97b8 fix(workers): read a result ref's size as a size, not an offset
Found on hardware. FetchResult seeked to request.size_bytes as though it were
a resume point, but that field is the artifact's total size — so every fetch
started at end-of-file, yielded no chunks, and failed with 'the result ended
before its final chunk' while the finished render sat on the node's disk.

ArtifactRef carries no resume field, so resumption is a protocol addition
rather than a reinterpreted one, and the fetch now always starts at zero.

Every earlier test drove publish and stage directly and never called
FetchResult with a populated ref, which is exactly why this survived them.
2026-08-11 23:13:29 +05:30
velixio 3dbae35feb fix(workers): actually move artifacts on an inbound session
Found on hardware. The job ran on the GPU machine and the audio never arrived:
'gpu2 finished the job but its audio did not arrive.'

Both artifact directions were built and neither was wired. A result reported by
a dialled node is only staged on that node's disk — nothing pushes it, because
the node cannot call us — so the commit recorded an artifact path that had
never been written. Inputs had the mirror problem: nothing sent them, so a
clone would have failed on a reference file that was never delivered.

Results are now pulled when the frame naming them arrives, and inputs are
pushed before the assignment rather than alongside it, because the executor
asks for them as soon as it starts and an assignment that overtakes its own
reference audio fails on a file that is merely late.

A fetch that fails is not a silent loss: no artifact is recorded, the task
fails naming the machine, and the node keeps its copy because nothing
acknowledges a result we could not fetch.
2026-08-11 23:11:08 +05:30
velixio 569517e5d8 fix(workers): send heartbeats on an inbound session
Found on hardware. The Attach handler started the read pump and the outbound
loop but never the heartbeat loop that the outbound path starts inside
_connect_once. So a node registered, went silent, was declared dead about
ninety seconds later, reconnected, and flapped forever — and in between, work
aimed at it fell back to the local machine with 'gpu2 is offline', while the
panel had shown it ready at 3.4 ms moments earlier.

Every end-to-end test in this file finished inside three seconds, comfortably
within the grace window that hid it. The regression test therefore asserts on
the emitted heartbeat frames themselves rather than on liveness, and shortens
the advertised interval so it does that in two seconds instead of twenty.
2026-08-11 23:05:20 +05:30
debpalash eaad1017df Merge commit '9930a0b41ab4a97560ddc30afdadc9728293d796' into fix/wavesurfer-abort-report-1498 2026-08-11 17:28:14 +00:00
velixio e121e69d0f fix(workers): put a dialable address in the connection string
Found on hardware. With the listener bound to 0.0.0.0 — which is what sharing
a GPU across a network requires — the issued string came out as
ovnode://...@0.0.0.0:7444. That is a legal bind and a meaningless destination,
so it would have failed on the far end with a connection error naming nothing,
and the person who pasted it had no way to tell a bad string from a firewall.

The string is now built from an advertised address rather than the bind: for a
wildcard bind, the source address the routing table would use to leave this
machine, found with a connected UDP socket that sends no packets and needs no
DNS. An explicitly typed bind is advertised verbatim, because someone who
entered a specific address meant it.
2026-08-11 22:57:44 +05:30
velixio 0988a48caa feat(workers): Settings UI for sharing a GPU, in all 21 languages
Adds the panel that makes inbound mode usable: a toggle to accept connections,
a bind field that says which side of "only this machine" you are on, per-person
connection strings with a copy button, the live list of who is connected with a
disconnect button, and a paste box for joining someone else's GPU.

Placed behind the existing Remote workers toggle rather than beside it. "Off
means off" is this feature's stated contract, and a second switch that stayed
live underneath would be exactly the surprise that promise exists to prevent.
Headless machines that only lend a GPU set OMNIVOICE_INBOUND_NODE and never see
this panel.

The unencrypted warning appears where it becomes true, not buried in a doc:
next to the bind field once it points beyond this machine, naming the address,
and again under every freshly issued connection string. The remove-access
confirm says the others stay connected, since that is the only place a user
learns keys are per person rather than one switch for everybody.

All 36 strings are translated into all 20 non-English locales in this change,
with the {{address}}, {{label}}, {{count}} and {{when}} placeholders verified
programmatically against en.json before writing — a dropped token is the exact
bug the parity test was built for, and en-only keys would have passed CI
silently while every other language read English.

Two existing WorkersPanel rename tests queried the only textbox on the page.
That was incidental, not intentional; they now name the field they mean.
2026-08-11 22:46:06 +05:30
Palash Debnath 9930a0b41a Merge pull request #1499 from Marc-oss-hub/feat/orcarouter-provider
feat(llm-providers): add OrcaRouter as a named LLM provider
2026-08-11 17:11:12 +00:00
velixio 2b6f49c596 feat(workers): make inbound mode reachable — settings, endpoints, docs
Wires the two transport halves into something a user can actually turn on.

Two independent switches, deliberately not one. "Accept connections" makes this
machine a node others dial; "saved connections" are the nodes this panel dials
out to. A workstation with a GPU that also drives jobs on a second box does
both, so neither implies the other.

Binding stays on 127.0.0.1 until someone explicitly widens it, and widening is
its own field rather than a flag riding along with the enable toggle. With no
encryption that boundary is the difference between a credential on one machine
and a credential on a network, so it is never crossed as a side effect. The
API reports `exposed` so the UI can say which side of it the user is on.

Saved nodes are redialled only after the control plane is up, since the
connector hands frames to its servicer. Failing to listen records the reason
rather than leaving the feature looking enabled while it quietly accepts
nothing.

Docs say plainly that this mode is unencrypted, that the connection string is a
password crossing the network in the clear, and that dial-out remains the
better choice when one machine is enough. The Security section no longer
implies its TLS guarantees cover both modes.
2026-08-11 22:28:06 +05:30
debpalash 29ccf8ee52 docs: normalize OrcaRouter changelog credit 2026-08-11 16:53:58 +00:00
velixio 53cb316854 feat(workers): dial a node from the panel and run work on it
Completes the inbound path. The panel opens NodeService.Attach with its key in
call metadata, answers the node's register frame, and then runs the ordinary
control-plane loops against the dialled stream — the same _read_loop and
_ping_loop the outbound path uses, so assignments, cancels, results and
reconciliation all behave identically. Only who opened the socket changed.

Registration is shared rather than copied: the body of Register is now
establish_session, reached from both roads. A second copy of session issue,
capability application and in-flight reconciliation is a second thing to keep
in step forever, and the half that gets forgotten is always reconciliation.
The version and feature gates run on the inbound road too — skipping them would
let an out-of-date node register cleanly and then ignore task inputs, which is
how a clone with no reference audio once came back reported as success.

Artifacts invert with the transport: the panel pushes inputs before it assigns,
and pulls results after. Both directions verify the declared sha256 and refuse
a stream that ends without its final chunk, because a truncated file renamed
into place and called done is the failure the upload path was already hardened
against.

Two things the end-to-end tests found, neither visible from unit tests:

  * Every Attach built a fresh client with an empty worker id, so the challenge
    signature could never match after first enrollment — inbound could connect
    once and never reconnect. The id is now kept per panel key, because each
    panel keeps its own registry and the same machine is a different worker id
    to each of them.
  * A node that has lost the id a panel gave it could prove possession of its
    key and still be refused forever, with no way back except deleting it from
    both sides. It is now re-adopted on proof of key possession, narrowly: the
    public key must already be the one enrolled, so this can never admit a new
    key. Covered by a test that forges a valid self-signature from a different
    keypair and asserts it is refused.
2026-08-11 22:18:26 +05:30
debpalash e6096a4eab test(llm): complete OrcaRouter provider contract 2026-08-11 16:47:43 +00:00
debpalash eb97a54105 Merge commit '1fc0b89778d056ca664723e4959621dc5fed9153' into audit/pr1499-fixes 2026-08-11 16:47:22 +00:00
debpalash 9687611e6f fix(ui): ignore cancelled waveform reports 2026-08-11 16:36:19 +00:00
velixio ef671de36e feat(workers): let a panel dial the GPU machine, so more than one person can use it
Remote workers connect outbound: the node dials the control plane, spends an
enrollment token, pins a certificate. That stays the default and is unchanged.

It is also structurally 1:1 — a worker process holds one endpoint, one pinned
certificate and one worker id — so a second person wanting the same GPU box has
to get shell access to it, repoint the start script at their own address and
restart, which disconnects whoever was using it. Sharing a GPU requires root on
it and evicts the incumbent, and no amount of UI work fixes that, because the
constraint is the shape of the connection.

This adds the other arrangement: the node listens, and any panel holding a key
connects to it, concurrently, with no shell access to the machine.

  * NodeService mirrors WorkerService. Transport roles invert; message roles do
    not — the node still sends WorkerMessage and the panel still sends
    ServerMessage, so every state machine on both sides is untouched. Register
    folds into the stream as the first exchange and reuses the existing
    request/response messages rather than growing parallel ones.
  * Keys are per panel, not per node. Revoking one person leaves everyone else
    connected; a shared key would be revoked by nobody and leave no record of
    who used it. Stored hashed, compared in constant time against every key so
    the reply time is not an oracle, and the plaintext exists exactly once.
  * Failed authentication is throttled per source address, so one stale
    bookmark cannot lock out a different panel.
  * A connection log records every attach, refusal and disconnect, and any
    session can be kicked. That is what replaces per-job approval, which would
    make a shared GPU unusable and train people to click yes.
  * Artifacts invert too: the panel pushes inputs before assigning, and fetches
    results after. The node stages both under one contained directory and
    trusts no id or filename off the wire.

Runs in plaintext by deliberate decision, recorded with its accepted risk in
docs/adr/inbound-node-mode.md, and scoped there to LAN and self-hosted use —
never a fleet transport, which goal_v2 B2/B5.2 still require to dial out.

Off by default, and bound to 127.0.0.1 until someone explicitly widens it.
2026-08-11 22:06:12 +05:30
Palash Debnath 1fc0b89778 Merge pull request #1490 from debpalash/fix/wayland-capture-shortcut
fix(dictation): support global shortcuts on Wayland
2026-08-11 16:23:16 +00:00
debpalash a95afc28ea Merge commit '19e352560e41feaf37b65d5007a57d93b9c0e1d4' into fix/wayland-capture-shortcut 2026-08-11 15:55:32 +00:00
Marc-oss-hubandClaude 13c342c238 feat(llm-providers): add OrcaRouter as a named LLM provider
Add OrcaRouter to the Settings → LLM Providers registry (OpenAI-compatible
gateway, base_url https://api.orcarouter.ai/v1, default openai/gpt-5.5).
Env surface follows the existing provider pattern: ORCAROUTER_API_KEY /
ORCAROUTER_BASE_URL / ORCAROUTER_MODEL.

- registry: Provider entry after OpenRouter
- llm_backend: include OrcaRouter in the not-configured hint
- settings search: 'orcarouter' keyword on the LLM Providers category
- docs: list OrcaRouter in the supported-provider docs (docs-sync)
- test: registry test covers the new id

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-11 23:44:44 +08:00
Palash Debnath 19e352560e Merge pull request #1493 from debpalash/feat/dub-footer-action-polish
Polish dubbing workflow actions
2026-08-11 15:34:15 +00:00
debpalash 606d02a3ea Merge remote-tracking branch 'origin/main' into feat/dub-footer-action-polish
# Conflicts:
#	CHANGELOG.md
#	frontend/src/components/dub/DubHeader.jsx
2026-08-11 15:18:04 +00:00
Palash Debnath b28d0f5f08 Merge pull request #1489 from debpalash/feat/dub-workspace-polish
Polish Dub workspace controls and media history
2026-08-11 14:56:33 +00:00
debpalash 539a8bb571 fix(i18n): cover Arabic dub history plurals 2026-08-11 14:32:05 +00:00
debpalash e369179efb Merge commit '283642340b60054dfd67c3390831a94f45f34063' into fix/wayland-capture-shortcut 2026-08-11 13:59:55 +00:00
debpalash 918fca3ead Merge commit '283642340b60054dfd67c3390831a94f45f34063' into feat/dub-workspace-polish 2026-08-11 13:58:48 +00:00
debpalash 465b08cb9c fix(dub): pluralize history metadata 2026-08-11 13:58:34 +00:00
debpalash b52165d2a7 fix(dub): dismiss stale QC progress 2026-08-11 13:58:29 +00:00
debpalash 441099bf68 fix(dictation): close capture startup edge cases 2026-08-11 13:58:29 +00:00
debpalash c960dcb7e2 Merge remote-tracking branch 'origin/main' into feat/dub-footer-action-polish 2026-08-11 13:56:02 +00:00
velixio ed48861008 Merge pull request #1 from velixio/feat/worker-protocol-v1
Remote GPU workers: run every GPU operation on the machine you pick
2026-08-11 18:50:49 +05:30
velixio 673e544812 Merge origin/main into feat/worker-protocol-v1
Three conflicts, all additive on both sides — resolved by keeping both
rather than choosing, since either side's entries were real shipped work:

  * CHANGELOG.md — remote-GPU entries against branding, IndexTTS 2.5 and
    the recording-input work
  * setup/download.py — the per-target progress reset against main's
    active-install tracking; both belong in the same finally block
  * docs/features.yaml — the remote-worker and model docs against
    docs/branding.md

Backend 5349 passed, frontend 1871 passed. `bun install --frozen-lockfile`
reports no changes, so the Docker build sees the same tree CI does.
2026-08-11 18:36:51 +05:30
velixio aa1d739843 feat(workers): dubbing goes remote, and the protocol stops lying to old workers
The remote-GPU line, verified on hardware rather than asserted.

**Dubbing renders on the worker.** dub_generate.py dispatches the coarse
`dub_segments` operation through the gateway, following the audiobook
pattern: per-unit local fallback after consecutive remote failures, one
aggregated notice rather than one per segment. A 40-minute dub that loses
its worker at segment 200 degrades instead of producing 200 error rows.

**An out-of-date worker is now refused by name.** This was the worst
defect in the plan and it was silent: an un-upgraded worker registered
cleanly, then ignored `inputs` and rendered a clone with NO reference
audio — returned as success. A plausible wrong result with nothing
anywhere to surface it. Workers now declare features, and one missing
them is turned away with the features named and `no task was run`.
Verified live: a worker one commit behind was correctly refused.

**"Offline" and "cannot run this" are different facts.** Asking a live
worker for an engine it lacks answered "is offline or cannot be reached.
Wake the selected worker" — while that worker reported ready, one free
slot and 3.6 ms latency. The user was sent to wake a machine that was
already awake. The scheduler now distinguishes absent from present-but-
incapable, and names the engine rather than the operation, because the
engine is the thing a user can install.

**An engine with no catalog entry is no longer hidden.** A `repo_ids`
non-emptiness check had been implemented as a runtime filter, so a worker
silently refused to advertise any engine lacking a models.yaml entry —
which is four registered engines, including CosyVoice. Users with those
already installed would have lost remote support with only a log line.
Empty `repo_ids` now means "not downloadable here", never "not runnable".

**And a script so this stops being done by hand.**
scripts/verify-remote-worker.sh runs the per-phase acceptance checks
against a live worker, non-destructively. Its preconditions are the
mistakes that cost the most time: exactly one listener on the control
port (two instances silently shared it), and never detecting the worker
with a pgrep pattern that matches the ssh shell running it.

Its first real run found the dubbing picker claiming remote placement.
That turned out to be the CHECK being stale, not the picker — the port
had landed since it was written. It now asserts self-consistency instead:
the picker may claim remote only for an operation the control plane
actually advertises as remotely producible, which cannot rot the next
time an op is ported.

Backend 5291 passed, frontend 1812 passed. Acceptance script: no
automated failures across Phases 4-8 on an RTX 4090. Four checks remain
MANUAL by design — true airplane mode, concurrent downloads, killing a
worker mid-audiobook, and the model-list UI — and are reported as
unverified rather than passed.
2026-08-11 17:39:46 +05:30
debpalash 7a3367be53 fix(dub): serialize workflow actions 2026-08-11 11:50:02 +00:00
debpalash cedd038e3e fix(dictation): make capture reliable across desktops 2026-08-11 11:48:01 +00:00
velixio b7caa494eb feat(workers): remote downloads, audiobook chapters, and one port that stays honest
Five workstreams that finish the remote-GPU line, plus the test hole that
let a broken signature reach a commit.

**Downloads go through the normal path** (Phase 5). Rather than a second
remote-only route, the existing Models install flow became target-aware,
so a model landing on a worker uses the same code, the same progress
events and the same UI as a local one. Progress rows key on
(target, repo_id) — the aggregator keyed on bare repo_id, so the same
model downloading here and on a worker at once collapsed into one row
that told the user nothing true about either.

**Audiobooks render chapter by chapter on the worker** (Phase 8), with
per-chapter local fallback and ONE aggregated notice. The failure that
shape exists to prevent: a remote GPU that sleeps at chapter 40 of 200
must not turn a working book into 160 rows of PROGRESS_LEASE_EXPIRED.
Dictation is deliberately NOT ported — it runs ASR per utterance inside a
live WebSocket loop, and paying queue admission plus a round trip there
would spend the one thing that route is for.

**Dubbing stays local, and says so** (Phase 7). The coarse worker
operation is not finished, so the picker still reports dubbing as local
rather than showing a green remote chip over work this machine is doing.
What could not wait is the in-loop OOM retry: it sniffed the error string
and flushed the *local* CUDA cache, which under remote execution is the
wrong machine's GPU entirely. That is fixed now, before the path that
would have exercised it exists.

**Two instances can no longer share the control plane.** A second
VoiceStudio silently bound the same worker port and coexisted, so remote
workers landed on whichever process won the race — a session that
registers with one instance and appears dead to the other. This produced
hours of misdiagnosis during hardware testing and would hit any user with
the app open twice. The second instance now keeps running locally and
explains the conflict instead of quietly competing.

**And the hole that allowed all this to be missable.** gpu_gateway called
Scheduler.submit(pinned_worker_id=...) one commit before that parameter
existed. Every remote generation raised TypeError; 5236 tests passed
anyway, because nothing exercised the gateway against the real scheduler.
tests/test_gpu_gateway_scheduler_contract.py now runs that path for real
and binds every gateway→dependency call signature. Verified by renaming
the parameter away and watching both tests fail with the original error.

Gallery previews also fall back to a local render when a downloaded clip
cannot be decoded, rather than yielding silence.

Backend 5274 passed, frontend 1812 passed.

Not yet verified on hardware: Phases 4, 5, 6, 7, 8. Only the TTS path and
its artifact transport have been proven on a real GPU.
2026-08-11 16:53:33 +05:30
velixio bda169c900 feat(workers): pin work to the chosen GPU, and say when its model is missing
Three phases that only make sense together: a job that names a worker,
a worker that reports honestly what it can actually run, and the small
defects that made both lie.

**Pinning** (Phase 1). `pinned_worker_id` is now honoured in both places
that choose a worker — `eligible_workers` and `select_worker` build
independent lists, so applying it to one silently leaked work onto
whichever machine was least busy. The pin persists across a restart via
an additive column, deliberately not alembic (justified in the code, per
the precedent already in db.py): quitting mid-render used to drop it
without a word. `max_attempts=1` was rejected as the mechanism — it makes
the FIRST failure terminal, including the penalty-free ones a stale
advisory view produces routinely.

Cancel now actually reaches the worker. `WorkerServicer.cancel` had zero
callers, so cancelling released the slot while the GPU thread kept
running, and a late result could resurrect the task as COMPLETED —
`commit_result` assigned that state directly, bypassing the transition
table where CANCELLED is terminal by construction.

**Honest capabilities** (Phase 4). A worker now probes whether weights
are actually present, and a job stops BEFORE dispatch with a typed 409
naming the model and the machine, instead of failing mid-task. The probe
fails OPEN: `is_cached`/`cache_is_complete` cannot see a user-managed
clone outside the HF layout, so only a positive "absent" refuses.
Refusing an engine that works today would break the compatibility
promise. `pool.supports` deliberately still ignores `downloaded` — had it
not, the scheduler would drop the worker and answer with a terminal
NO_CAPABLE_WORKER, which tells the user to check their install when the
truth is one download away. The frontend no longer offers "Report this
bug" for that state; it offers the download.

Catalog tags resolve against the TARGET's OS/arch/backend, not this
machine's. From a Mac control plane, a CUDA worker's model list was
showing the mlx-community repos it cannot run and hiding the ones it
needs.

**And the quiet ones** (Phase 0 leftovers): a model's human label rides
its own proto field so renaming it cannot orphan breaker history; an
empty model_id no longer forks the capacity slot key into two slots for
one model; the idle sweep cannot evict an engine out from under a live
LOCAL render.

Verified on real hardware, which is the only verification that has ever
caught anything here: 2025 characters, default settings, routed to an
RTX 4090 over the wire — 100% GPU utilisation on the remote box, 119.6 s
of 24 kHz audio returned in 16.6 s, 5.7 MB delivered out of band through
the artifact path rather than the control stream.

Backend 5259 passed, frontend 1808 passed.
2026-08-11 15:24:11 +05:30
velixio fe467d15e7 fix(workers): stop evicting workers for pinging at the interval we set
Every enrolled worker sat at connected=False against a healthy control
plane, and the control-plane log showed no Register call arriving at all.
The worker's own log said only "connecting", then nothing.

The cause was on our side of the handshake. The client sends an HTTP/2
ping every 25 s to keep its long-lived Control RPC alive through NAT —
an interval the control plane itself configures. But the server kept
gRPC's default enforcement policy, which permits two idle pings and then
answers ENHANCE_YOUR_CALM:

    GOAWAY received; Error code: 11; Debug Text: too_many_pings

So the control plane hung up on every worker for obeying the keepalive
the control plane asked for. Idle workers were hit hardest, because a
session with no traffic is exactly the case the ping exists to protect.

Fixed by accepting the interval this protocol configures: a 20 s minimum
still rate-limits an abusive peer, while removing the idle-ping count
ceiling stops a healthy session dying of its own liveness mechanism.
This is a whole-fleet fix, not a per-enrollment one.

Worth recording what this was NOT, because it looked exactly like it:
TLS pin-on-first-use was the obvious suspect, since a control plane that
regenerated its certificate on restart would strand every enrolled
worker with no useful error. Disproved — the live certificate
fingerprint and the pinned copy on the remote worker match exactly, and
the certificate survives restarts. Enrollment was never involved.

Verified live against a remote worker: the session now establishes where
previously nothing reached the server. It is not yet stable — it drops
after ~17 s and advertises zero engines — but that is a separate defect
being tracked on its own, and this fix is a prerequisite for reaching it.
2026-08-11 14:30:03 +05:30
velixio b54cd28403 feat(workers): one gateway for GPU calls, and results too big for the wire
Two phases of the remote-GPU plan, landing together because neither is
useful alone: on a 4090 any render long enough to exercise the progress
lease also outgrows the 8 MiB message cap, so a gateway that routes work
remotely without an artifact transport just moves where the failure
happens.

**The gateway** (`services/gpu_gateway.py`) is the single owner of GPU
calling, model status, downloads and model load, for both targets —
`prewarm`, `run`, `status`, `download`. prewarm and run stay separate
because collapsing them loses the two-phase load/generate budget split
(#1033/#1037) that the worker protocol already mirrors. Admission moves
in here too: the old `check_gpu_admission` call read *local* pool stats,
so under Remote it would 429 on local saturation while the remote GPU
sat idle.

**Artifacts** now move out of band above a negotiated threshold. Bytes
land in an attempt-scoped `.part` file, are verified against a declared
sha256, and are renamed into place only on an explicit last chunk — a
transfer that arrives short, reordered, or simply stops commits nothing.
A resume rehashes what is already on disk, or the digest would attest
only to the tail, which is the exact case a resume exists to protect.

Two failure modes found while verifying this, both fixed with
mutation-checked regressions:

  * an oversized payload with no session (mid-reconnect, or a control
    plane too old to serve UploadResult) has nowhere to go. It must not
    enter `_pending` — an over-cap frame is re-sent on every reconnect,
    killing the session each time and stranding every other task — but
    it must stay retryable, unlike the size gate's TERMINAL verdict:
    nothing about the render is wrong, only the route to it.

  * the upload resume loop was bounded by "did the offset change", which
    a receiver alternating between two byte counts satisfies forever.
    The worker is single-slot by default, so that is not one lost upload
    but the machine, doing nothing else, until someone restarts it.
    Bounded by a round count instead.

The control stream is split into control and bulk queues so the
heartbeat this whole liveness model rests on cannot queue behind a
payload — `result_json` has no size cliff to catch it, and the next bulk
message added to the protocol would have reintroduced the stall
silently.

Live streaming stays on the control plane and now says so once per
socket: that route exists to put audio in the user's ear before the
sentence finishes, and paying queue admission plus a round trip per
utterance would spend the one thing it is for. Silence would have been
worse than the limit — the header badge would read "gpu2" while this
machine did all the work.

Backend 5236 passed, frontend 1807 passed. End-to-end verification on
real hardware has NOT been re-run since these changes; the CHANGELOG
claim for the Synthesize button waits on that.
2026-08-11 13:30:26 +05:30
debpalash c88955fb37 Harden dubbing action semantics 2026-08-11 06:23:31 +00:00
debpalash 69a6867e5c Document polished dubbing actions 2026-08-11 06:22:35 +00:00
debpalash 935deba22b Polish dubbing workflow actions 2026-08-11 06:21:36 +00:00
debpalash 084c1d1ebb Merge remote-tracking branch 'origin/main' into feat/dub-workspace-polish
# Conflicts:
#	frontend/src/components/MultiLangPicker.jsx
2026-08-11 06:18:17 +00:00
debpalash 137c76abb9 Merge commit '8af10ea0b960f3715dad5c99cba221b3f3a15b82' into fix/wayland-capture-shortcut
# Conflicts:
#	CHANGELOG.md
2026-08-11 06:15:25 +00:00
debpalash 7eec1d7d11 fix(dictation): make shortcuts truthful across desktops 2026-08-11 05:49:51 +00:00
debpalash aa14c9ade3 fix(dub): address workspace review findings 2026-08-11 05:43:56 +00:00
debpalash 29232f4591 Merge remote-tracking branch 'origin/main' into feat/dub-workspace-polish
# Conflicts:
#	CHANGELOG.md
2026-08-11 05:40:22 +00:00
debpalash 5ae23f37d1 Merge commit '7053f1d1c01b064266a32f227ea365100a68ae21' into fix/wayland-capture-shortcut
# Conflicts:
#	CHANGELOG.md
2026-08-11 05:22:09 +00:00
debpalash 94af4c9f3a docs(changelog): note Wayland dictation shortcut 2026-08-11 04:32:13 +00:00
debpalash aaa5bfa0c6 docs(dictation): record Wayland shortcut path 2026-08-11 04:31:32 +00:00
debpalash e1964dfbb3 feat(dub): polish workspace controls 2026-08-11 04:29:19 +00:00
debpalash 27d76955da fix(dictation): support capture shortcuts on Wayland 2026-08-11 04:27:36 +00:00
velixio c643706d07 feat(workers): make a remote GPU actually run a task, end to end
Selecting a remote worker repainted a badge and nothing else. The cause was
not subtle: `scheduler.submit` had no production caller, and `routing.decide()`
was read only by the status endpoint that paints the header. Remote execution
was a complete, tested pipeline with no producer at its head.

This adds the producer and fixes the defects that made the pipeline unable to
carry a real job:

- Nothing routed to the scheduler. Adds `POST /workers/tasks` (loopback-gated,
  **development-only** until the gateway lands) and `Scheduler.wait`, backed by
  per-task futures rather than the unregisterable `on_change` listener list.
- Every task over two minutes died. No worker ever sent `TaskProgress`, so the
  120s progress lease expired mid-render — including during the cold model
  load, which happens after `TaskStarted`. Workers now report progress and
  emit a keepalive, bounded by the phase's absolute budget so it renews the
  lease without deleting the only enforced bound in the system.
- The executor rebuilt its engine per task (`return cls()`), so every job paid
  a cold load. Engines now share one instance cache with the router, resolved
  by the assignment's engine — never `get_active_tts_backend()`, which returns
  the worker machine's own Settings preference and would silently run the
  wrong engine.
- One lease expiry took a worker offline permanently: parked slots were never
  reclaimed. Parks now expire on a TTL, and are deliberately NOT reconciled
  against the worker's own load report — at a ceiling of one the only task such
  a worker can report is the wedged one, so "busy" would drop the park and the
  next idle heartbeat would hand out a slot with a live GPU thread (#730/#1190).
- A worker that dropped and reconnected mid-render had every liveness frame
  discarded: task frames were fenced on the live session epoch, which bumps on
  every reconnect, while the worker echoes the ref stamped at dispatch. The
  control plane then expired a task whose GPU was still rendering, and swallowed
  the failure report when it went wrong. Fenced per attempt instead.
- A result from one worker could commit another's task, after which the owner's
  real delivery arrived as a duplicate and its audio was discarded. "Unknown
  attempt" and "another worker's attempt" are no longer the same answer.
- An oversized result was a poison pill, re-sent identically on every reconnect
  and permanently disconnecting the worker. It is now a terminal
  `RESULT_TOO_LARGE`, which is also classified — it was falling through to
  TRANSIENT and retrying a re-render that could never fit.
- `_store_inline` joined the artifact directory with worker-supplied ids, and
  `os.path.join` discards its prefix on an absolute component. Paths are now
  minted control-plane-side and resolved through `core.path_security`.
- Remote synthesis bypassed `mark_synthetic`, and the guard that exists to
  catch exactly that walked only `backend/api` and `backend/services` — so it
  stayed green while a fourth unmarked producer shipped. Marking moved to the
  worker's tensor stage; the guard now walks `backend/worker` too.

Also adds pre-rendered voice previews (`services/gallery.py`), so browsing the
gallery no longer needs a GPU or a downloaded model. The manifest is verified
against the updater's release key already baked into the binary; a fresh
install hears voices without downloading 2.4GB first, and everything falls back
to local rendering when the gallery is unreachable.

Verified on hardware, not just in CI: 1728 characters submitted to an RTX 4090
returned 105.94s of 24kHz audio in 23.9s, committed and served from the
artifact store.

Not yet done, and deliberately not claimed: the keepalive fix cannot be
exercised end-to-end on fast hardware, because any job long enough to reach the
120s lease produces audio past the 8MiB inline cap. Chunked `UploadResult` has
to land first. Pinning to the worker the user chose is also still absent, so
"Remote" reaches a remote GPU but not necessarily the one on the badge.
2026-08-11 07:16:04 +05:30
velixio 7924b35f8d Merge remote-tracking branch 'origin/main' into feat/worker-protocol-v1 2026-08-10 21:40:44 +05:30
velixio 9eb1ec7591 feat(workers): choose where jobs run, and show whether that machine is well
Adds a GPU target picker to the header: Local, or one of the machines you
enrolled. Exactly one is active at a time; other connected workers are
standby and receive nothing.

The selection is the user's, not the scheduler's. The engine underneath can
rank many workers and the hosted platform will need that, but a desktop app
is better served by a choice you can predict and explain: "your worker is
offline, this ran locally" is a sentence, "least-busy ranking preferred the
laptop" is not. Picking an offline machine is allowed on purpose — you
choose your desktop, then go and switch it on.

`routing.decide()` is the single answer to "where does the next job run",
shared by the badge and (soon) the generation path, so the badge cannot
claim something the router will not do. It shows the RESOLVED answer rather
than the stored choice: pick your desktop, let it sleep, and the chip reads
Local with the reason, while the menu still shows your desktop selected.

Connection latency is now real. `latency_ms` existed but nothing measured
it — the protocol had Ping with no reply — so it was always zero. Adds Pong
(additive, field 12) and times the round trip on the control plane's
MONOTONIC clock, so an NTP step or a sleep/wake cannot produce a nonsense
reading, and no worker timestamp is trusted. Reported as a median of five
samples and withheld until a second sample exists: the first round trip
after connect lands while the worker is still importing torch, which
measured 139 ms on loopback and, averaged, carried that for a minute.

This is CONNECTION latency, not time-to-result. It is shown as information,
never as a routing input — RTT is milliseconds where inference is seconds,
so ranking on it would optimise noise.

Also fixes a bug the picker exposed: worker config was read from the pool,
which caches the row handed to it at connect time. Renaming a CONNECTED
worker updated the database and the API kept serving the old name until it
reconnected — same for priority and enable/disable. Config now comes from
the database and liveness from the pool, never the reverse, and writers
refresh the live copy so the scheduler's logs do not use a stale name.

Adds worker rename (the backend already supported it; no UI called it),
worker address as seen by the control plane rather than self-reported, and
ready/busy/offline status behind the header dot.
2026-08-10 21:40:07 +05:30
velixio 02ec8e3675 fix(workers): send real JSON from the panel, and cover the endpoints that hid it
The Settings panel posted a JSON *string* with no content type, so FastAPI
refused every write with a 422 ("Input should be a valid dictionary"). It
also read `.enabled` straight off apiFetch's return value — but apiFetch
resolves to a raw Response, not parsed JSON, and does not throw on 4xx. So
the panel could never have shown a worker even once the 422 was fixed, and
no HTTP error ever reached a catch block.

All three now go through one request() helper: it sets the content type,
checks res.ok, parses, and raises FastAPI's `detail` so the user reads
"Remote workers are turned off." rather than a status code.

Why the tests missed it: they mocked apiFetch as if it returned parsed data,
so they agreed with the mock instead of the client. The mock now returns a
Response-shaped object, and the assertions check the wire shape — method,
Content-Type, parsed body — because a was-it-called assertion cannot see a
missing header.

Three endpoints had no test at all (/enabled, /resume, /tasks/{id}/cancel);
/enabled is the one that broke. All nine are covered now, including the
string-body 422 itself.
2026-08-10 15:27:03 +05:30
velixio b8f44e089d fix(workers): advertise the port the control plane actually bound
An enrollment token carries the endpoint a worker will dial, but
default_endpoint() read the CONFIGURED port rather than the bound one. Start
on any other port and every token points somewhere nothing is listening —
the worker retries forever against a dead address with backoff, so it looks
like a network problem rather than a wrong number.

Found by running the feature end to end on a non-default port, which is also
the second bug in this seam: the first was advertising a .local hostname
gRPC's resolver cannot resolve. Both were about what the token tells a
worker to dial, so both now have regression tests.
2026-08-10 14:34:57 +05:30
velixio 4f4d9c6e3e refactor(workers): give Remote workers its own System entry; ignore remote/
Remote workers was nested under Sharing, which reads backwards: everything
in Sharing is about letting something else reach THIS machine (a remote
backend, an MCP client, a share PIN), while remote workers sends work OUT
to machines you own. It is now its own System entry.

Docs-sync: every "Settings → Sharing → Remote workers" reference is
updated — the guide, the changelog, the two API error messages that tell a
user where to generate a token, and the agent's not-enrolled error.

Also ignores remote/ (local goal docs, review briefs, council reports) and
repoints the code comments that cited remote/goal_v2.md at the shipped
docs/remote-workers.md, so no committed file references a path that is not
in the repo.
2026-08-10 14:28:16 +05:30
velixio e5bd11bf85 Merge remote-tracking branch 'origin/main' into feat/worker-protocol-v1 2026-08-10 14:18:48 +05:30
velixio 43de1c794c feat(workers): remote GPU workers over a versioned gRPC protocol
Send individual jobs to GPUs on your other machines while everything else
stays local. Opt-in, off by default: with the toggle off there is no
listening socket, no certificate and no background loop.

Design follows remote/goal_v2.md, the council-revised goal doc. The
decisions that shaped the code, and why:

* A disconnect is an unknown outcome, not a failure. The original design
  reassigned on disconnect while also describing the case where the worker
  had already finished — following both guarantees duplicate execution. An
  attempt now holds a grace window; a worker returning inside it commits
  its result and no second attempt is ever made.
* At-least-once execution, exactly-once result commit. The result is
  persisted BEFORE it is acknowledged, so a crash between the two cannot
  silently lose a finished render.
* Deadlines are phased (accept -> model load -> execute -> deliver) and
  liveness is a progress lease. The old fixed 30s execution budget was two
  orders of magnitude below what this product actually does; silence is
  the failure signal, not slowness.
* Capacity is derived from free VRAM, never configured: a static value
  corrupts output under torch.compile thread affinity (#315) and aborts
  the process on small cards (#567).
* A circuit breaker replaces the reliability-score/quarantine machinery,
  which had no recovery path (no probation workload exists in a TTS
  product) and penalised consumer networks for existing.
* Identity is a keypair the worker generates and never sends. A
  server-assigned id is a name, not an authenticator, so revocation of one
  would be theatre. Enrollment tokens are single-use and carry the control
  plane's certificate fingerprint for pin-on-first-use.

Adds the domain core, scheduler, durable task store, gRPC transport,
worker agent, management API, Settings panel, and docs. Protobuf reserves
the tenant/trace/usage fields a hosted control plane would need, since
adding them later means upgrading a whole fleet.

Includes tests for the failure paths that matter: duplicate delivery,
stale-session fencing, reconnect reconciliation, grace expiry, breaker
attribution, and a real end-to-end TLS round trip.
2026-08-10 14:18:42 +05:30
Paolo Antinori 5229a9504c chore: untrack local backlog/ task tracker (gitignored) 2026-07-30 16:10:36 +02:00
Paolo Antinori 214a859344 Merge remote-tracking branch 'upstream/main' 2026-07-30 06:34:28 +02:00
Paolo Antinori b72436a4e5 Merge remote-tracking branch 'upstream/main' 2026-07-29 15:49:36 +02:00
Paolo Antinori c2955dbe92 chore(backlog): initialize Backlog.md project structure 2026-07-28 17:58:56 +02:00
Paolo Antinori a6f008ec38 docs(backlog): investigate VRAM/lifecycle bug + durable-fix exploration
TASK-1: stuck model loads (>1200s), unkillable abandoned workers holding device
TASK-2: exploration of durable fixes (flush caches, CPU engine, timeout, shorter text)
2026-07-28 17:46:16 +02:00
882 changed files with 151815 additions and 13557 deletions
+60
View File
@@ -0,0 +1,60 @@
---
name: fastapi-python
description: Expert in FastAPI Python development with best practices for APIs and async operations
---
# FastAPI Python
You are an expert in FastAPI and Python backend development.
## Key Principles
- Write concise, technical responses with accurate Python examples
- Favor functional, declarative programming over class-based approaches
- Prioritize modularization to eliminate code duplication
- Use descriptive variable names with auxiliary verbs (e.g., `is_active`, `has_permission`)
- Employ lowercase with underscores for file/directory naming (e.g., `routers/user_routes.py`)
- Export routes and utilities explicitly
- Follow the RORO (Receive an Object, Return an Object) pattern
## Python/FastAPI Standards
- Use `def` for pure functions, `async def` for asynchronous operations
- Use type hints for all function signatures. Prefer Pydantic models over raw dictionaries
- Structure: exported router, sub-routes, utilities, static content, types (models, schemas)
- Use ordinary Python control flow; prefer readability over compressed one-line conditionals
## Error Handling
- Handle edge cases at function entry points
- Employ early returns for error conditions
- Place happy path logic last
- Avoid unnecessary else statements; use if-return patterns
- Implement guard clauses for preconditions
- Provide proper error logging and user-friendly messaging
## FastAPI-Specific Guidelines
- Use functional components (plain functions) and Pydantic models for input validation
- Declare routes with clear return type annotations
- Prefer lifespan context managers for managing startup and shutdown events
- Leverage middleware for logging, error monitoring, and optimization
- Use HTTPException for expected errors and model them as specific HTTP responses
- Apply Pydantic's BaseModel consistently for validation
## Performance Optimization
- Minimize blocking I/O. In `async def` handlers, use awaitable database/API clients; put synchronous SQLite or other blocking work in synchronous routes or explicitly offload it
- Implement caching with Redis or in-memory stores
- Optimize Pydantic serialization/deserialization
- Use lazy loading for large datasets
## Key Conventions
1. Rely on FastAPI's dependency injection system
2. Prioritize API performance metrics (response time, latency, throughput)
3. Structure routes and dependencies for readability and maintainability
## Dependencies
FastAPI, Pydantic v2, asyncpg/aiomysql, SQLAlchemy 2.0
+357
View File
@@ -0,0 +1,357 @@
---
name: vite
description: Expert guidance for Vite development with modern build tooling, HMR, framework integrations, and performance optimization
---
# Vite Development
You are an expert in Vite, modern JavaScript/TypeScript build tooling, and frontend development.
## Key Principles
- Leverage native ES modules for fast development
- Use Vite's opinionated defaults when possible
- Configure only what needs customization
- Understand the dev/build differences
- Optimize for both development speed and production performance
## Project Setup
### Basic Configuration
```typescript
// vite.config.ts
import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react';
export default defineConfig({
plugins: [react()],
server: {
port: 3000,
open: true,
},
build: {
outDir: 'dist',
sourcemap: true,
},
});
```
### Path Aliases
```typescript
import { defineConfig } from 'vite';
export default defineConfig({
resolve: {
alias: {
'@': new URL('./src', import.meta.url).pathname,
'@components': new URL('./src/components', import.meta.url).pathname,
'@utils': new URL('./src/utils', import.meta.url).pathname,
},
},
});
```
## Environment Variables
### Usage
```typescript
// .env
VITE_API_URL=https://api.example.com
VITE_APP_TITLE=My App
// In code
const apiUrl = import.meta.env.VITE_API_URL;
const isDev = import.meta.env.DEV;
const isProd = import.meta.env.PROD;
const mode = import.meta.env.MODE;
```
### Type Definitions
```typescript
// src/vite-env.d.ts
/// <reference types="vite/client" />
interface ImportMetaEnv {
readonly VITE_API_URL: string;
readonly VITE_APP_TITLE: string;
}
interface ImportMeta {
readonly env: ImportMetaEnv;
}
```
## Hot Module Replacement
### Manual HMR
```typescript
// For libraries without HMR support
if (import.meta.hot) {
import.meta.hot.accept('./module.ts', (newModule) => {
// Handle the updated module
console.log('Module updated:', newModule);
});
import.meta.hot.dispose(() => {
// Cleanup before module is replaced
});
}
```
## Asset Handling
### Static Assets
```typescript
// Import as URL
import imageUrl from './image.png';
// <img src={imageUrl} />
// Import as string (raw)
import shaderCode from './shader.glsl?raw';
// Import as worker
import Worker from './worker.ts?worker';
const worker = new Worker();
```
### Public Directory
```
public/
├── favicon.ico # Served at /favicon.ico
├── robots.txt # Served at /robots.txt
└── images/ # Served at /images/
```
## Framework Integrations
### React
```typescript
import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react';
export default defineConfig({
plugins: [
react({
// Babel plugins
babel: {
plugins: ['@emotion/babel-plugin'],
},
}),
],
});
```
### Vue
```typescript
import { defineConfig } from 'vite';
import vue from '@vitejs/plugin-vue';
export default defineConfig({
plugins: [vue()],
});
```
### Svelte
```typescript
import { defineConfig } from 'vite';
import { svelte } from '@sveltejs/vite-plugin-svelte';
export default defineConfig({
plugins: [svelte()],
});
```
## Build Optimization
### Code Splitting
```typescript
// Dynamic imports create separate chunks
const AdminPanel = lazy(() => import('./AdminPanel'));
// Manual chunks
export default defineConfig({
build: {
rollupOptions: {
output: {
manualChunks: {
vendor: ['react', 'react-dom'],
utils: ['lodash', 'date-fns'],
},
},
},
},
});
```
### Chunk Size Optimization
```typescript
export default defineConfig({
build: {
chunkSizeWarningLimit: 500,
rollupOptions: {
output: {
manualChunks(id) {
if (id.includes('node_modules')) {
return id.split('node_modules/')[1].split('/')[0];
}
},
},
},
},
});
```
## CSS Handling
### CSS Modules
```typescript
// styles.module.css is auto-detected
import styles from './styles.module.css';
// <div className={styles.container}>
```
### PostCSS
```javascript
// postcss.config.js
export default {
plugins: {
tailwindcss: {},
autoprefixer: {},
},
};
```
### Preprocessors
```typescript
// Automatically handled with package installed
// npm install -D sass
import './styles.scss';
```
## Proxy Configuration
```typescript
export default defineConfig({
server: {
proxy: {
'/api': {
target: 'http://localhost:4000',
changeOrigin: true,
rewrite: (path) => path.replace(/^\/api/, ''),
},
'/socket.io': {
target: 'ws://localhost:4000',
ws: true,
},
},
},
});
```
## Plugin Development
```typescript
// my-vite-plugin.ts
import type { Plugin } from 'vite';
export function myPlugin(): Plugin {
return {
name: 'my-plugin',
// Hook: modify config
config(config, { mode }) {
return {
define: {
__BUILD_TIME__: JSON.stringify(new Date().toISOString()),
},
};
},
// Hook: transform code
transform(code, id) {
if (id.endsWith('.md')) {
return {
code: `export default ${JSON.stringify(code)}`,
map: null,
};
}
},
// Hook: configure dev server
configureServer(server) {
server.middlewares.use((req, res, next) => {
// Custom middleware
next();
});
},
};
}
```
## Testing with Vitest
```typescript
// vitest.config.ts
import { defineConfig } from 'vitest/config';
export default defineConfig({
test: {
globals: true,
environment: 'jsdom',
setupFiles: './src/test/setup.ts',
coverage: {
provider: 'v8',
reporter: ['text', 'json', 'html'],
},
},
});
```
## SSR Configuration
```typescript
export default defineConfig({
build: {
ssr: true,
rollupOptions: {
input: './src/entry-server.ts',
},
},
ssr: {
external: ['express'],
noExternal: ['my-ui-library'],
},
});
```
## Library Mode
```typescript
export default defineConfig({
build: {
lib: {
entry: './src/index.ts',
name: 'MyLib',
fileName: (format) => `my-lib.${format}.js`,
},
rollupOptions: {
external: ['react', 'react-dom'],
output: {
globals: {
react: 'React',
'react-dom': 'ReactDOM',
},
},
},
},
});
```
## Best Practices
- Use `vite preview` to test production builds locally
- Keep dependencies that support ESM in regular deps
- Use `optimizeDeps.include` for CommonJS dependencies
- Enable `build.sourcemap` for debugging production
- Use `server.warmup` for faster dev server starts
+4 -1
View File
@@ -5,6 +5,9 @@ description: "Local TTS, voice cloning, voice design, and video dubbing via the
# VoiceStudio
The canonical cross-agent package lives at `skills/omnivoice/SKILL.md`. This
Claude-specific package retains the MCP lifecycle helpers and references.
## Overview
Generate audio locally via the VoiceStudio MCP server. Tools: `generate_speech`, `list_voices`, `list_personalities`, `list_languages`, `check_health`. Resources: `voice://{id}`, `history://recent`.
@@ -166,4 +169,4 @@ The MCP server does not expose the dubbing endpoint. The full transcribe → tra
Backend Swagger / OpenAPI: `http://127.0.0.1:3900/docs` (when backend is up).
Upstream: github.com/debpalash/VoiceStudio — FSL-1.1-ALv2 (free for personal/internal/non-commercial; auto-converts to Apache-2.0 two years after each release).
Upstream: github.com/debpalash/VoiceStudio. The app uses AGPL-3.0-only; optional engines and downloaded models retain their own licenses. See `LICENSE-NOTICE.md` in the repository.
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# Start the OmniVoice FastAPI backend on 127.0.0.1:3900, detached, idempotent.
# Honors $OMNIVOICE_HOME (default ~/OmniVoice-Studio).
# Honors $OMNIVOICE_HOME (default ~/VoiceStudio).
#
# Exit codes:
# 0 success (already running, or freshly started + healthy within 60s)
@@ -11,7 +11,7 @@
set -euo pipefail
HOME_DIR="${OMNIVOICE_HOME:-$HOME/OmniVoice-Studio}"
HOME_DIR="${OMNIVOICE_HOME:-$HOME/VoiceStudio}"
URL="${OMNIVOICE_API_URL:-http://127.0.0.1:3900}"
LOG="$HOME_DIR/backend.log"
+1
View File
@@ -40,6 +40,7 @@ sudo apt-get install -y \
libwebkit2gtk-4.1-dev libgtk-3-dev libpango1.0-dev libcairo2-dev \
libsoup-3.0-dev libgdk-pixbuf-2.0-dev \
libayatana-appindicator3-dev librsvg2-dev libssl-dev libxdo-dev \
gstreamer1.0-plugins-good \
libasound2-dev build-essential curl wget file
```
+1 -1
View File
@@ -4,7 +4,7 @@
| Version | Supported |
|---------|-----------|
| 0.3.x (latest release + `main` previews) | ✅ Current — all fixes land here |
| 0.5.x (latest release + `main` previews) | ✅ Current — all fixes land here |
| 0.2.7 | ⚠️ Legacy stable — security fixes only, upgrade recommended |
| < 0.2.7 | ❌ No longer supported |
+15 -1
View File
@@ -51,6 +51,13 @@ jobs:
- os: ubuntu-latest
platform: linux-x86_64
experimental: false
- os: ubuntu-24.04-arm
platform: linux-aarch64
# Apple Silicon under Asahi Linux. Experimental: the Vulkan
# (Honeykrisp GPU) build path is new and the hosted arm64
# runner has no GPU — it validates that the binary builds;
# on-host Vulkan acceleration is exercised by users.
experimental: true
- os: windows-latest
platform: windows-x86_64
experimental: false
@@ -80,11 +87,18 @@ jobs:
# Linux-only: upstream `buildcpu.sh` enables `-DGGML_BLAS=ON` which
# requires a system BLAS implementation at cmake configure time.
- name: Linux system deps (BLAS for ggml-blas backend)
if: matrix.platform == 'linux-x86_64'
if: startsWith(matrix.platform, 'linux')
run: |
sudo apt-get update
sudo apt-get install -y libopenblas-dev pkg-config
# linux-aarch64: let the build script's Vulkan path (Honeykrisp GPU
# on Asahi) engage instead of silently falling back to CPU.
- name: Vulkan dev deps (linux-aarch64 GPU backend)
if: matrix.platform == 'linux-aarch64'
run: |
sudo apt-get install -y glslc libvulkan-dev spirv-headers
- name: Build omnivoice-tts
shell: bash
# Pass values through env (quoted) rather than ${{ }} interpolation
+81 -5
View File
@@ -11,6 +11,11 @@ on:
push:
branches: [main]
workflow_dispatch:
inputs:
windows_wix_diagnostic:
description: Run only the tiny nonpublishing Windows MSI authoring diagnostic
type: boolean
default: false
permissions:
contents: read
@@ -21,8 +26,15 @@ env:
jobs:
test:
if: ${{ !inputs.windows_wix_diagnostic }}
name: Tests (backend + frontend)
runs-on: ubuntu-22.04
env:
# Same restricted-network resilience the smoke matrix already sets. This
# job resolves the same direct-URL dependency and had none of it, which
# is why it was the one that kept dying (see scripts/uv-sync-retry.sh).
UV_HTTP_TIMEOUT: "120"
UV_HTTP_RETRIES: "5"
steps:
- uses: actions/checkout@v4
@@ -61,7 +73,14 @@ jobs:
# so their tests can exercise the real import path, not the
# "package not installed" fallback. Smoke job below stays on bare
# `uv sync` because smoke only hits /health + fixture profiles.
run: uv sync --all-extras
#
# Retried because one dependency — en-core-web-sm — resolves to a
# direct GitHub release URL, and github.com intermittently answers
# `http2 error: refused stream before processing any application
# logic`. uv's own 3 retries all land inside the same few seconds and
# fail together, which has cost otherwise-green runs (#1517, #1518).
# Backing off between whole attempts is what actually clears it.
run: bash scripts/uv-sync-retry.sh --all-extras
# HF_HUB_OFFLINE=1 is a recurrence guard, not an optimization: a test
# that reaches huggingface.co fails fast and loud instead of silently
@@ -71,7 +90,7 @@ jobs:
# interactions in tests are stubbed; anything that trips this is a
# test-isolation bug.
- name: Run pytest
run: uv run pytest tests/ -q --tb=short
run: uv run --no-sync pytest tests/ -q --tb=short
env:
HF_HUB_OFFLINE: "1"
@@ -92,13 +111,14 @@ jobs:
run: |
bash frontend/src-tauri/appimage/AppRun.test.sh
bash scripts/inject-apprun.test.sh
bash scripts/verify-apprun-bundle.test.sh
# `backend/tests/` mounts routers on bare FastAPI apps (no heavy main
# import chain) with a hermetic data dir from its conftest.py. It no
# longer stubs sys.modules, so mixed sessions with tests/ are safe;
# the separate session is kept for cheaper, clearer CI output.
- name: Run pytest (backend/tests, isolated)
run: uv run pytest backend/tests/ -q --tb=short
run: uv run --no-sync pytest backend/tests/ -q --tb=short
env:
HF_HUB_OFFLINE: "1" # same no-silent-downloads guard as tests/
@@ -175,6 +195,7 @@ jobs:
# and `cargo test --lib` runs the shell's unit tests natively on each OS.
# Full bundling stays in release.yml on tag push.
tauri-cross-platform:
if: ${{ !inputs.windows_wix_diagnostic }}
name: Tauri shell check (${{ matrix.label }})
needs: test
strategy:
@@ -257,6 +278,17 @@ jobs:
working-directory: frontend/src-tauri
run: cargo test --lib --target ${{ matrix.rust_target }} --message-format=short
# Backend-lifecycle fault-injection harness: real child processes die
# scripted deaths through the OMNIVOICE_BACKEND_CMD seam, and each
# scenario asserts the user-visible diagnosis names the actual cause
# (port conflict / traceback root cause / spawn failure / timeout /
# crash-loop exhaustion / signal 9 / deliberate replace / deferred-
# startup step). Serial: the scenarios share process-global state
# (env vars, crash store, kill-intended flag) by design.
- name: Cargo test (backend lifecycle harness)
working-directory: frontend/src-tauri
run: cargo test --test backend_lifecycle --target ${{ matrix.rust_target }} --message-format=short -- --test-threads=1
# ── Cross-platform Python runtime smoke (Phase 0 GATE-02) ───────────────
# Loads the frozen tests/fixtures/omnivoice_data/ fixture and boots the
# FastAPI app in-process via TestClient on macOS/Windows/Linux. Catches
@@ -264,6 +296,7 @@ jobs:
# job above misses. Narrow scope (tests/smoke/ only) — full pytest stays
# on Linux until Phase 1's INST-01 lands setuptools for WhisperX.
smoke-matrix:
if: ${{ !inputs.windows_wix_diagnostic }}
name: Smoke (${{ matrix.label }})
needs: test
strategy:
@@ -351,6 +384,19 @@ jobs:
echo "choco attempt $i did not produce ffmpeg — retrying in $((i * 30))s"
sleep $((i * 30))
done
# Chocolatey is one distribution channel, not the dependency. When
# its feed is down across every retry (2026-08-13: three attempts,
# three 'installed 0/1'), fall back to the static gyan.dev release
# build GitHub mirror — the same binary, no feed in the path.
if ! command -v ffmpeg >/dev/null 2>&1; then
echo "::warning::choco feed down — falling back to static ffmpeg build"
curl -fsSL --retry 3 -o /tmp/ffmpeg.zip \
https://github.com/GyanD/codexffmpeg/releases/download/7.1/ffmpeg-7.1-essentials_build.zip
unzip -q /tmp/ffmpeg.zip -d /tmp/ffmpeg
bindir=$(dirname "$(find /tmp/ffmpeg -name ffmpeg.exe | head -1)")
echo "$bindir" >> "$GITHUB_PATH"
export PATH="$bindir:$PATH"
fi
ffmpeg -version
- name: System deps (Linux)
@@ -366,7 +412,7 @@ jobs:
# backend host. The Intel-Mac leg separately pins the documented
# unsupported contract: its UI is a remote-backend client only (#889).
if: matrix.backend_supported
run: uv sync --extra pockettts
run: bash scripts/uv-sync-retry.sh --extra pockettts
- name: Verify the documented Intel Mac contract
if: ${{ !matrix.backend_supported }}
@@ -391,7 +437,37 @@ jobs:
- name: Run smoke tests
if: matrix.backend_supported
run: uv run pytest tests/smoke/ -q --tb=short
run: uv run --no-sync pytest tests/smoke/ -q --tb=short
env:
HF_HUB_OFFLINE: "1" # same no-silent-downloads guard as the main pytest job
HF_HUB_CACHE: ${{ runner.temp }}/pockettts-empty-hf-cache
# Artifact commits depend on native Windows rename/replace semantics;
# Linux emulation cannot exercise sharing rules or path parsing.
- name: Remote-worker artifact paths (Windows)
if: runner.os == 'Windows' && matrix.backend_supported
run: uv run --no-sync pytest tests/test_worker_upload_server.py tests/test_worker_server_integrity.py -q --tb=short
env:
HF_HUB_OFFLINE: "1"
HF_HUB_CACHE: ${{ runner.temp }}/worker-artifact-empty-hf-cache
windows-wix-diagnostic:
name: Windows MSI authoring (no publishing)
needs: test
if: ${{ !cancelled() && (inputs.windows_wix_diagnostic || needs.test.result == 'success') }}
runs-on: windows-2022
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
- uses: oven-sh/setup-bun@v1
- name: Bundle canonical system and per-user templates with a tiny payload
shell: pwsh
run: ./scripts/diagnose-windows-wix.ps1
- name: Preserve verbose linker output and rendered authoring
if: always()
uses: actions/upload-artifact@v4
with:
name: windows-wix-diagnostic
path: wix-diagnostic-artifacts/
if-no-files-found: warn
retention-days: 3
+2 -2
View File
@@ -38,14 +38,14 @@ jobs:
cache-dependency-glob: "uv.lock"
- name: Install deps
run: uv sync
run: bash scripts/uv-sync-retry.sh
- name: Run eval suites (non-gating)
continue-on-error: true
env:
TRANSLATE_BASE_URL: ${{ secrets.EVALS_LLM_BASE_URL }}
TRANSLATE_API_KEY: ${{ secrets.EVALS_LLM_API_KEY }}
run: uv run python tests/evals/run_evals.py --output eval-report.json
run: uv run --no-sync python tests/evals/run_evals.py --output eval-report.json
- name: Upload report artifact
uses: actions/upload-artifact@v4
+127
View File
@@ -0,0 +1,127 @@
# Installer smoke — runs scripts/install.sh / scripts/install.ps1 end-to-end
# on all three desktop platforms so the one-liner installers can't rot.
#
# Gated by `paths` because a cold run downloads multi-GB wheels (torch) and
# takes ~15-30 min per OS; it only needs to fire when an installer or this
# workflow changes. The heavy Tauri bundles stay in release.yml (tag push).
name: Install smoke
on:
pull_request:
paths:
- "scripts/install.sh"
- "scripts/install.ps1"
- ".github/workflows/install-smoke.yml"
push:
branches: [main]
paths:
- "scripts/install.sh"
- "scripts/install.ps1"
- ".github/workflows/install-smoke.yml"
workflow_dispatch:
permissions:
contents: read
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
jobs:
install:
name: Install (${{ matrix.os }})
runs-on: ${{ matrix.os }}
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
os: [ubuntu-22.04, macos-latest, windows-latest]
steps:
- uses: actions/checkout@v4
# Running `sh scripts/install.sh` from the repo root exercises the
# repo-root resolution (script dir is scripts/, project root one level
# up) — the exact bug that made a local run clone a duplicate repo.
# Binary mode is the default: prebuilt release asset, checksum verified.
- name: Run installer — binary (macOS/Linux)
if: runner.os != 'Windows'
run: sh scripts/install.sh
- name: Verify install — binary (macOS/Linux)
if: runner.os != 'Windows'
run: |
if [ "$(uname)" = "Darwin" ]; then
test -d "/Applications/VoiceStudio.app" || { echo "::error::VoiceStudio.app missing from /Applications"; exit 1; }
echo "✓ VoiceStudio.app installed in /Applications"
else
test -x "$HOME/.local/bin/VoiceStudio" || { echo "::error::AppImage missing from ~/.local/bin"; exit 1; }
"$HOME/.local/bin/VoiceStudio" --appimage-help >/dev/null 2>&1 || true
echo "✓ AppImage installed and executable"
fi
# Source mode stays covered end-to-end behind --source.
- name: Run installer — source (macOS/Linux)
if: runner.os != 'Windows'
run: sh scripts/install.sh --source
- name: Verify install — source (macOS/Linux)
if: runner.os != 'Windows'
working-directory: ${{ github.workspace }}
run: |
test -d .venv || { echo "::error::.venv missing"; exit 1; }
test -f frontend/dist/index.html || { echo "::error::frontend build missing"; exit 1; }
echo "✓ venv + frontend bundle present"
# Binary mode is the default; CI runs msiexec silently.
- name: Run installer — binary (Windows)
if: runner.os == 'Windows'
env:
CI: true
shell: pwsh
run: '& { $ErrorActionPreference = "Stop"; & "${{ github.workspace }}\scripts\install.ps1" }'
- name: Verify install — binary (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: |
$paths = @(
"HKLM:\Software\Microsoft\Windows\CurrentVersion\Uninstall\*",
"HKLM:\Software\WOW6432Node\Microsoft\Windows\CurrentVersion\Uninstall\*",
"HKCU:\Software\Microsoft\Windows\CurrentVersion\Uninstall\*"
)
$key = Get-ItemProperty $paths -ErrorAction SilentlyContinue |
Where-Object { $_.DisplayName -match "VoiceStudio|OmniVoice" } |
Select-Object -First 1
if (-not $key) {
Get-ItemProperty $paths -ErrorAction SilentlyContinue |
Where-Object DisplayName | ForEach-Object { Write-Host " installed: $($_.DisplayName)" }
Write-Host "::error::MSI product not registered"; exit 1
}
Write-Host "✓ MSI product registered: $($key.DisplayName)"
# Source mode stays covered end-to-end behind -Source.
- name: Run installer — source (Windows)
if: runner.os == 'Windows'
env:
VOICESTUDIO_INSTALL_MODE: source
shell: pwsh
run: '& { $ErrorActionPreference = "Stop"; & "${{ github.workspace }}\scripts\install.ps1" }'
- name: Verify install — source (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: |
if (-not (Test-Path .venv)) { Write-Host "::error::.venv missing"; exit 1 }
if (-not (Test-Path frontend\dist\index.html)) { Write-Host "::error::frontend build missing"; exit 1 }
Write-Host "✓ venv + frontend bundle present"
- name: Upload install log on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: install-log-${{ matrix.os }}
path: |
/Users/runner/Library/Application Support/OmniVoice/*.log
/home/runner/.local/share/VoiceStudio/*.log
${{ runner.temp }}/VoiceStudio/**/*.log
if-no-files-found: ignore
+185 -38
View File
@@ -109,10 +109,10 @@ jobs:
version: 1.0
- name: Install Python deps
run: uv sync
run: bash scripts/uv-sync-retry.sh
- name: Run pytest
run: uv run pytest tests/ -q --tb=short
run: uv run --no-sync pytest tests/ -q --tb=short
- name: Cache bun deps
uses: actions/cache@v4
@@ -148,15 +148,20 @@ jobs:
preview-gate:
name: Preview gate
runs-on: ubuntu-22.04
permissions:
contents: read
outputs:
is_preview: ${{ steps.decide.outputs.is_preview }}
proceed: ${{ steps.decide.outputs.proceed }}
stable_tag: ${{ steps.decide.outputs.stable_tag }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 50
- id: decide
shell: bash
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
event="${{ github.event_name }}"
@@ -171,6 +176,13 @@ jobs:
exit 1
fi
echo "is_preview=true" >> "$GITHUB_OUTPUT"
# Resolve once before the matrix starts so every platform stamps
# against the same immutable Stable-channel snapshot.
STABLE_TAG=$(gh release view --repo "$GITHUB_REPOSITORY" --json tagName --jq .tagName)
[[ "$STABLE_TAG" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]] || {
echo "::error::latest stable release has an invalid tag"; exit 1;
}
echo "stable_tag=$STABLE_TAG" >> "$GITHUB_OUTPUT"
else
echo "is_preview=false" >> "$GITHUB_OUTPUT"
fi
@@ -497,35 +509,22 @@ jobs:
echo "APPLE_TEAM_ID=$TID"
} >> "$GITHUB_ENV"
# Stamp each preview build with a unique, monotonically increasing semver
# PRERELEASE so the updater actually offers it (a rolling preview that
# always reported the static 0.3.0 never looked "newer", so no update was
# ever delivered). Ephemeral, CI-only — never committed. Tauri reads the
# bundle + updater version from tauri.conf.json, so rewriting it here
# stamps the artifacts + latest.json. Under the versioning hard rule
# (owner-set 2026-06-11) main is always last-release + 1, so BASE-N is a
# prerelease of the NEXT version and semver-sorts ABOVE the last stable
# (0.3.6-N > 0.3.5) — preview users naturally upgrade past stable, and
# the Windows MSI ProductVersion (which strips the prerelease → 0.3.6)
# is also correctly above the last stable.
# Stamp each preview with a numeric prerelease that is strictly above the
# latest stable release. Main may intentionally retain the released
# version while AUTO_VERSION_BUMP is disabled; in that case the helper
# advances the preview base by one patch so stable users can still opt in
# and receive it. The edit is ephemeral and never committed.
- name: Stamp preview version
if: needs.preview-gate.outputs.is_preview == 'true'
shell: bash
env:
STABLE_TAG: ${{ needs.preview-gate.outputs.stable_tag }}
run: |
set -euo pipefail
# package.json is the single source of truth; tauri.conf.json reads its
# version from it ("version": "../package.json"), so stamping
# package.json restamps the whole bundle.
CONF=frontend/package.json
BASE=$(jq -r .version "$CONF")
# MSI/WiX requires the semver pre-release identifier to be numeric-only
# (and <= 65535). "preview.N" hard-fails the Windows bundler, so the
# preview stamp is BASE-N — still sorts below the stable BASE for the
# updater, still unique per run.
PREVIEW_VERSION="${BASE}-${{ github.run_number }}"
tmp=$(mktemp)
jq --arg v "$PREVIEW_VERSION" '.version = $v' "$CONF" > "$tmp"
mv "$tmp" "$CONF"
PREVIEW_VERSION=$(python scripts/stamp-preview-version.py \
--package-json frontend/package.json \
--stable-tag "$STABLE_TAG" \
--run-number "${{ github.run_number }}")
echo "Stamped preview version: $PREVIEW_VERSION"
# The rolling `preview` release is REUSED every night, and macOS updater
@@ -649,6 +648,47 @@ jobs:
updaterJsonPreferNsis: false
includeUpdaterJson: true
- name: Build per-user Windows MSI
if: runner.os == 'Windows'
shell: bash
working-directory: frontend
env:
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
run: |
set -euo pipefail
python ../scripts/render-per-user-wix.py \
--source src-tauri/wix/main.wxs \
--system-wxs src-tauri/target/${{ matrix.rust_target }}/release/wix/x64/main.wxs \
--output src-tauri/target/wix-per-user/main.wxs
bunx tauri build --target ${{ matrix.rust_target }} --bundles msi \
--config src-tauri/tauri.per-user.conf.json
DIR="src-tauri/target/${{ matrix.rust_target }}/release/bundle/msi"
while IFS= read -r artifact; do
safe=${artifact// (Current User)/_Current_User}
[ "$safe" = "$artifact" ] || mv "$artifact" "$safe"
done < <(find "$DIR" -maxdepth 1 -type f -name '*Current*User*.msi*')
- name: Publish per-user Windows updater channel
if: runner.os == 'Windows'
shell: bash
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
RELEASE_TAG: ${{ (needs.preview-gate.outputs.is_preview == 'true') && 'preview' || github.ref_name }}
run: |
set -euo pipefail
DIR="frontend/src-tauri/target/${{ matrix.rust_target }}/release/bundle/msi"
MSI=$(find "$DIR" -name '*Current*User*.msi' -type f | head -1)
[ -n "$MSI" ] || { echo "per-user MSI missing"; find "$DIR" -type f; exit 1; }
[ -f "$MSI.sig" ] || { echo "per-user MSI signature missing"; exit 1; }
VERSION=$(jq -r .version frontend/package.json)
python scripts/build_windows_user_manifest.py \
--repo "$GITHUB_REPOSITORY" --tag "$RELEASE_TAG" --version "$VERSION" \
--asset "$(basename "$MSI")" --signature-file "$MSI.sig" \
--output latest-user.json
gh release upload "$RELEASE_TAG" "$MSI" "$MSI.sig" latest-user.json \
--clobber --repo "$GITHUB_REPOSITORY"
# ── Installer smoke (Phase 0 GATE-03) ─────────────────────────────
# Structural verification of the installed/extracted bundle. The thin
# uv-venv installer ships NO frozen backend binary (the venv is built on
@@ -717,8 +757,9 @@ jobs:
shell: bash
run: |
set -euo pipefail
MSI=$(find frontend/src-tauri/target/${{ matrix.rust_target }}/release/bundle/msi -name "*.msi" | head -1)
MSI=$(find frontend/src-tauri/target/${{ matrix.rust_target }}/release/bundle/msi -name "*.msi" ! -name '*Current*User*' | head -1)
echo "Smoke-testing MSI: $MSI"
powershell.exe -NoProfile -ExecutionPolicy Bypass -File scripts/verify-windows-msi.ps1 -MsiPath "$(cygpath -w "$MSI")"
# /quiet = no UI, /norestart = don't reboot the runner if a dep asks
msiexec.exe //i "$(cygpath -w "$MSI")" //quiet //norestart
INSTALL="/c/Program Files/VoiceStudio"
@@ -731,6 +772,72 @@ jobs:
find "$INSTALL" -type f -path '*backend*main.py' | grep -q . || fail "backend source main.py missing"
echo "OK — MSI installed shell + uv + backend resources"
- name: Per-user installer smoke (Windows, non-admin account)
if: runner.os == 'Windows'
timeout-minutes: 8
shell: bash
run: |
set -euo pipefail
MSI=$(find frontend/src-tauri/target/${{ matrix.rust_target }}/release/bundle/msi -name '*Current*User*.msi' | head -1)
powershell.exe -NoProfile -ExecutionPolicy Bypass \
-File scripts/smoke-per-user-msi.ps1 -MsiPath "$(cygpath -w "$MSI")"
# linuxdeploy re-links .DirIcon as an ABSOLUTE symlink into the build
# machine AFTER tauri's files-map has placed the real icon bytes — the
# exact bug #1518 guarded against, resurfacing on the first real tag
# build (v0.5.0). The seam tauri-action leaves us is post-upload: repack
# the AppImage with the icon as a REGULAR FILE, re-sign it (the updater
# signature covered the old bytes), and clobber the draft release's
# asset + the linux signature inside latest.json. The smoke below then
# validates the repaired artifact, not the broken one.
- name: Repair AppImage .DirIcon, re-sign, re-upload
if: runner.os == 'Linux'
timeout-minutes: 10
shell: bash
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
# Data, not shell source (zizmor template-injection): a crafted ref
# must never expand inside a script that holds the signing key.
TAG: ${{ (needs.preview-gate.outputs.is_preview == 'true') && 'preview' || github.ref_name }}
run: |
set -euo pipefail
APPIMAGE=$(find frontend/src-tauri/target/${{ matrix.rust_target }}/release/bundle/appimage -name "*.AppImage" | head -1)
APPIMAGE=$(realpath "$APPIMAGE")
WORK="$(mktemp -d)"; cd "$WORK"
"$APPIMAGE" --appimage-extract >/dev/null
ROOT="$WORK/squashfs-root"
ICON=$(readlink -f "$ROOT/.DirIcon" 2>/dev/null || true)
if [ -n "$ICON" ] && [ -f "$ICON" ] && case "$ICON" in "$ROOT"/*) true;; *) false;; esac; then
echo ".DirIcon already resolves inside the bundle — no repair needed"
exit 0
fi
# The real bytes are at the AppDir root (linuxdeploy put them there
# before mislinking). Ship a regular file: nothing left to dangle.
SRC=$(find "$ROOT" -maxdepth 1 -name "*.png" | head -1)
[ -n "$SRC" ] || SRC=$(find "$ROOT/usr/share/icons" -name "*.png" | head -1)
[ -n "$SRC" ] || { echo "no icon bytes found in bundle"; exit 1; }
rm -f "$ROOT/.DirIcon"
cp "$SRC" "$ROOT/.DirIcon"
# Pinned immutable release + checksum: this binary runs with the
# updater signing key and a release-write token in its environment,
# so a mutable 'continuous' asset is not acceptable supply chain.
AIT_URL="https://github.com/AppImage/appimagetool/releases/download/1.9.1/appimagetool-x86_64.AppImage"
AIT_SHA256="ed4ce84f0d9caff66f50bcca6ff6f35aae54ce8135408b3fa33abfc3cb384eb0"
curl -fsSL --retry 3 -o "$WORK/appimagetool" "$AIT_URL"
echo "$AIT_SHA256 $WORK/appimagetool" | sha256sum -c - || { echo "appimagetool checksum mismatch"; exit 1; }
chmod +x "$WORK/appimagetool"
# Same FUSE-less trick the build itself uses.
APPIMAGE_EXTRACT_AND_RUN=1 ARCH=x86_64 "$WORK/appimagetool" --no-appstream "$ROOT" "$APPIMAGE"
cd "$GITHUB_WORKSPACE/frontend"
bunx tauri signer sign "$APPIMAGE"
gh release upload "$TAG" "$APPIMAGE" "$APPIMAGE.sig" --clobber --repo "$GITHUB_REPOSITORY"
# latest.json is NOT patched here: every tauri-action leg re-uploads
# the shared manifest, so an in-leg patch races the other platforms —
# the repair-updater-manifest job below is the single final writer.
echo "repacked, re-signed, re-uploaded"
- name: Installer smoke (Linux)
if: runner.os == 'Linux'
timeout-minutes: 5
@@ -751,17 +858,13 @@ jobs:
"$APPIMAGE" --appimage-extract >/dev/null
ROOT="$EXTRACT_DIR/squashfs-root"
fail() { echo "FAIL — $1"; find "$ROOT" -maxdepth 5 -type f 2>/dev/null | head -40; exit 1; }
# Regression gate: beforeBundleCommand runs before Tauri creates the
# AppDir. The v0.4.2 artifact therefore silently shipped Tauri's
# stock AppRun and bypassed every WebKit/Mesa compatibility fix.
cmp -s "$ROOT/AppRun" "$GITHUB_WORKSPACE/frontend/src-tauri/appimage/AppRun" \
|| fail "custom AppRun missing from final AppImage"
[ -s "$ROOT/usr/lib/.bundled-webkitgtk-version" ] \
|| fail "bundled WebKitGTK version marker missing"
cmp -s \
"$ROOT/usr/lib/.bundled-webkitgtk-version" \
"$GITHUB_WORKSPACE/frontend/src-tauri/target/.tauri/bundled-webkitgtk-version" \
|| fail "bundled WebKitGTK version marker is stale or mismatched"
# linuxdeploy's GTK/GStreamer hooks wrap the seeded launcher as
# AppRun.wrapped. Verify the complete launcher chain, not only the
# small hook runner installed at the AppImage root.
bash "$GITHUB_WORKSPACE/scripts/verify-apprun-bundle.sh" \
"$ROOT" \
"$GITHUB_WORKSPACE/frontend/src-tauri/appimage/AppRun" \
"$GITHUB_WORKSPACE/frontend/src-tauri/target/.tauri/bundled-webkitgtk-version"
# Thin uv-venv installer: verify the AppImage carries the shell binary,
# the bundled uv sidecar, and the backend source resources.
{ [ -f "$ROOT/AppRun" ] || find "$ROOT" -type f \( -name "VoiceStudio" -o -name "omnivoice-studio" \) | grep -q .; } || fail "shell binary / AppRun missing"
@@ -848,6 +951,50 @@ jobs:
# the tag (v0.3.20 shipped with only the Linux AppImage that way). `needs:
# [build]` guarantees the release already exists; `--clobber` makes a re-run
# idempotent. This can never create a second release.
# The Linux leg may repack + re-sign its AppImage (see the repair step in
# the build matrix); every tauri-action leg also re-uploads the SHARED
# latest.json, so patching the manifest inside any leg races the others.
# This job runs once after the whole matrix as the single final writer:
# it makes the manifest's linux signature agree with the .sig asset that
# actually shipped, and refuses to leave a mismatch behind.
repair-updater-manifest:
needs: [build, preview-gate]
runs-on: ubuntu-latest
timeout-minutes: 10
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Data, not shell source — same zizmor rule as the leg step.
TAG: ${{ (needs.preview-gate.outputs.is_preview == 'true') && 'preview' || github.ref_name }}
steps:
- name: Align latest.json's linux signature with the shipped .sig asset
shell: bash
run: |
set -euo pipefail
WORK="$(mktemp -d)"
HAS_MANIFEST=$(gh release view "$TAG" --repo "$GITHUB_REPOSITORY" --json assets --jq '[.assets[].name]|contains(["latest.json"])')
if [ "$HAS_MANIFEST" != "true" ]; then
echo "no latest.json on the release — nothing to align"; exit 0
fi
gh release download "$TAG" --pattern latest.json --output "$WORK/latest.json" --repo "$GITHUB_REPOSITORY"
# Same fail-closed rule as the manifest: absence is checked against
# the asset LIST; an actual download failure must fail the job, or
# the manifest keeps a signature nobody shipped.
HAS_SIG=$(gh release view "$TAG" --repo "$GITHUB_REPOSITORY" --json assets --jq '[.assets[].name|select(endswith(".AppImage.sig"))]|length > 0')
if [ "$HAS_SIG" != "true" ]; then
echo "no AppImage .sig asset on the release — nothing to align"; exit 0
fi
gh release download "$TAG" --pattern "*.AppImage.sig" --dir "$WORK" --repo "$GITHUB_REPOSITORY"
SIG_FILE=$(find "$WORK" -name "*.AppImage.sig" | head -1)
[ -n "$SIG_FILE" ] || { echo "sig asset listed but download produced nothing"; exit 1; }
NEW_SIG=$(cat "$SIG_FILE")
CHANGED=$(python3 -c 'import json,sys; p,sig=sys.argv[1],sys.argv[2]; d=json.load(open(p)); n=sum(1 for k,v in d.get("platforms",{}).items() if k.startswith("linux") and v.get("signature")!=sig and not v.update({"signature":sig})); json.dump(d,open(p,"w"),indent=2); print(n)' "$WORK/latest.json" "$NEW_SIG")
if [ "$CHANGED" -ge 1 ]; then
gh release upload "$TAG" "$WORK/latest.json" --clobber --repo "$GITHUB_REPOSITORY"
echo "aligned $CHANGED linux signature(s) with the shipped .sig"
else
echo "manifest already agrees with the shipped .sig — no write"
fi
uninstall-scripts:
needs: [build]
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
+1 -1
View File
@@ -174,7 +174,7 @@ jobs:
- name: pip-audit (Python)
continue-on-error: true
run: |
uv sync
bash scripts/uv-sync-retry.sh
uv run --with pip-audit pip-audit
# Pin a floor: `bun audit` was added in bun 1.2.x, so guarantee it exists.
+16
View File
@@ -154,3 +154,19 @@ playwright-report/
# probe — generated HTML reports
tests/probe/reports/
# Local architecture/planning scratch (goal docs, review briefs, council
# reports). Working notes for whoever is driving a change, not a repo artifact.
/remote/
# OmniVoice GGUF runtime build artifacts (scripts/build-omnivoice-tts.sh).
# Only 0-byte placeholders of omnivoice-tts-* are tracked; real binaries,
# the checksums manifest and the copied libggml shared libs ship via CI.
bin/libggml*
bin/checksums.sha256
bin/omnivoice-tts-linux-aarch64
# Dubbing-demo intermediates. The .mp4/.srt/manifest.json in this directory ARE
# committed (they ship with the app); the per-language source WAVs are just the
# inputs scripts/render_dub_demo_audio.py hands to scripts/build_dub_demo.sh.
backend/assets/samples/demo/dubbing/*.src.wav
+2
View File
@@ -25,4 +25,6 @@ regexes = [
'''^hf_QWERTYUIOPasdfghjklZXCVBNM0123456789xyzAB$''',
# NLLB generation length argument, not the value of a credential.
'''^max_length=400$''',
# cryptography's Ed25519 private-key type name, not key material.
'''^Ed25519PrivateKey$''',
]
+4
View File
@@ -35,6 +35,10 @@ Binding for every AI agent (Claude, Codex, Cursor, review bots, …). CLAUDE.md
## Agent skills
Project development skills are pinned in `skills-lock.json` and installed under
`.agents/skills/`: Vite and FastAPI.
Repository rules and tracker mappings override generic skill guidance.
### Issue tracker
GitHub Issues on `debpalash/VoiceStudio`, via the `gh` CLI. See `docs/agents/issue-tracker.md`.
+273 -22
View File
@@ -10,21 +10,230 @@ the frozen-backend fallback mirror it for their toolchains.
**Highlights**
- VoiceStudio now gives the app, desktop chrome, documentation, and package metadata one clear identity
- A local-first creative studio: voice cloning, design, dubbing, dictation, stories, audiobooks, and transcription without a subscription meter
- Reliability first: automatic cache repair, truthful hardware routing, safer sidecars, and actionable recovery instead of mystery failures
- Security boundaries now match the product: native file access stays native, untrusted network destinations fail closed, and public errors keep private diagnostics local
- RTX 40-series GPUs are used again instead of being sent to the CPU
- A warning before a slow generation, rather than after a five-minute wait
- The watermark can be turned off in Settings, as the docs always said
- Workspace tabs in the title bar, if you prefer them to the icon rail (#1412)
- macOS support now matches what the app actually delivers
- Linux AppImage: a blank white window on rolling distros (Mesa 26.1+) now starts normally
- Apple Silicon: transcription no longer needs a system ffmpeg, as the docs always said — thanks @gambletan! (#1436)
- A failed audiobook chapter says why, instead of turning red and saying nothing
- Fix current-user Windows installer validation and nested resource cleanup (#730)
- Voice cloning now starts with a clear upload-or-record choice, reveals recording and reference details only when needed, and keeps sampling controls under Production Overrides (#1817)
### Changed
### Added
### Docs
### Fixed
## [0.5.2] — 2026-09-02
**Highlights**
- Show estimated and measured model, dependency, cache, and temporary disk costs in the engine catalogue (#1718)
- Preview builds now stay newer than Stable even when automatic post-release version bumps are disabled (#1762)
- CosyVoice setup guidance now separates downloaded model files from the runtime that makes the engine available (#1761)
- MCP tools can now keep audio out of agent context by returning files and accepting base-path-confined file inputs (#1760) — thanks @agudmund!
- Hear a dub line as you type it — an opt-in live preview streams TTS for the edited segment (#1769) — thanks @mvanhorn!
- Studio gains a Convert method: re-say any clip in one of your saved voices, speech to speech, fully local (#1765) — thanks @mvanhorn!
- Hardsub video export gains an opt-in karaoke word-highlight caption style (#1764) — thanks @mvanhorn!
- The batch queue can now watch a folder: new videos dropped into it are dubbed automatically (#1768) — thanks @mvanhorn!
- The audiobook player now shows the chapter text and highlights the word being narrated (#1766) — thanks @mvanhorn!
- The dub editor gains a casting board: drag voice chips onto speakers, dropdowns stay in sync (#1767) — thanks @mvanhorn!
### Changed
- Voice Design simplified: the 12-row fine-grained block collapses to one summary line with a five-field editor, English accent and Chinese dialect merge into a single field, and the starting-point chips now show 5 with an overflow toggle (#1793)
### Added
- The audiobook result is now a synced-lyrics player: chapter text follows playback with the current word highlighted and click-to-seek, timed from the render's own chapter durations with a karaoke-style even split — no ASR pass, fully local (#1766) — thanks @mvanhorn!
- The dub CAST strip expands into a project-level casting board: drag voice chips (clone profiles, design presets, Default) onto speaker rows — or pick from a keyboard listbox — writing the same per-speaker cast fields as the existing dropdowns (#1767) — thanks @mvanhorn!
- Studio's new Convert method turns a dropped or recorded clip into an existing voice profile's voice, with optional source-duration matching (#1765) — thanks @mvanhorn!
- Opt-in watch folder on the batch queue: pick a directory once and new videos are auto-enqueued with your last Add-to-queue settings, with pause/stop controls and copy-in-progress protection — files upload as bytes, paths never leave the app (#1768) — thanks @mvanhorn!
- Hardsub export can now burn karaoke word-highlight captions: an opt-in Line | Karaoke control renders a word-timed ASS sweep from timings persisted at transcription, with an even-split fallback for older jobs and translated tracks, plus a `GET /dub/ass/{job_id}` sidecar (#1764) — thanks @mvanhorn!
- Windows releases now include an independently updatable per-user MSI that installs and uninstalls without elevation (#1713)
- Dub segments can now stream live TTS while you edit a translated line — opt-in toggle, existing `/ws/tts` socket, shared generation admission, exports still render at full quality (#1769) — thanks @mvanhorn!
- Engine status and diagnostic bundles now record loaded execution provider, device, precision, fallback stage, accelerator identity, runtime versions, and parent-process memory visibility (#1717)
### Docs
- Local gigastt is now documented as a supported OpenAI-compatible ASR endpoint, with loopback privacy distinguished from remote servers (#1736) — thanks @ekhodzitsky!
- The CosyVoice guide now states that packaged builds have no one-click runtime installer and records the exact readiness checks exposed by [Discussion 1631](https://github.com/debpalash/VoiceStudio/discussions/1631) (#1761)
- A production private-API guide now covers pinned containers, root credentials, network isolation, streaming proxies, health checks, upgrades, and benchmark evidence (#1720)
- RX 6700 XT/gfx1031 over WSL2 ROCDXG is now explicitly unverified until a published end-to-end GPU workload proves the mapped path (#1716)
### Fixed
- The generation compute-time budget is now a Settings control (Performance & Device) instead of an env-var-only setting the timeout error recommended with no UI path — the error copy points there too, and long CPU/MPS renders get an upfront heads-up before they start (#1787)
- Windows: the backend can now start when the install path contains non-English characters (e.g. a CJK username) on a non-UTF-8 system code page — a new or broken Python environment now builds at an ASCII-safe path automatically (a healthy existing one is never relocated), and a specific error message names the cause and a working fix if the interpreter still crashes in `site` (#1783)
- Exports and other native-picker actions no longer 403 with "Invalid or expired desktop authorization" when the desktop app and backend resolve different data directories, e.g. dev mode or a custom data folder (#1781)
- Voice Design no longer lets you pick a Chinese dialect and an English accent together — the picker keeps them mutually exclusive instead of round-tripping a 400 (#1771)
- The desktop app no longer attaches to an already-running backend on version string alone: it now verifies the backend's actual code fingerprint too, so an orphaned or manually started backend reporting the current version but running older code (e.g. a stale `destination_path` export 422) gets replaced instead of adopted (#1770)
- Korean locale overhauled: 231 mistranslations corrected and all 493 missing keys translated (#1776) — thanks @j30231!
- Japanese "Cleaning…" clone status now reads as denoising instead of housekeeping (#1775) — thanks @j30231!
- The batch dubbing queue now has a UI entry point — a quiet link on the Dub landing (it was previously unreachable: the app switched on a mode nothing ever set) (#1768) — thanks @mvanhorn!
- OpenAI-compatible ASR now requires HTTPS outside loopback and refuses redirects so audio stays on the configured origin (#1736)
- Windows isolated engines now retain direct Job ownership without an extra Python supervisor process that can deadlock the child loader (#1734)
- The setup splash now waits through the backend's full startup budget instead of reporting slow Windows CUDA initialization as stuck after two minutes (#1749)
- Dubbing jobs can now reuse every source-language code produced by automatic ASR detection without a 400 error on the next upload (#1737)
- Incomplete Sherpa-ONNX model snapshots now self-repair before recognizer startup instead of failing on a missing ONNX file (#1733)
- OmniVoice subprocess startup now allows slow packaged Windows Python runtimes to signal readiness before termination (#1711)
- SRT files selected during source analysis now wait for speaker cloning, then replace transcript text without losing voices (#1709)
- Windows MSI deployments can now prohibit WebView2 bootstrap with `DISABLEWEBVIEW2BOOTSTRAP=1`, and `AUTOLAUNCHAPP=0` reliably suppresses first launch (#1714)
- Subtitle rows now provide 100 ms timing steppers and flag adjacent overlaps without requiring precise timeline dragging (#1710)
- Repair-sync failures now retain uv's final dependency error instead of reporting only an opaque exit status (#1705)
- YouTube ingest now retries yt-dlp's transient “page needs to be reloaded” response (#1706)
- Dictation model readiness now follows the live Hugging Face cache selected in Settings (#1707)
- Dictation capture now queues native events whenever its webview listener unmounts or reloads instead of emitting them to nobody (#1707)
- Desktop-contained backends now exit when their owning app disappears instead of surviving as stale port-3900 processes (#1707)
## [0.5.1] — 2026-08-28
**Highlights**
- OmniVoice generation on Apple Silicon now runs in a crash-isolated child, so fatal MPS memory exits no longer take down the local backend (#1697, #1698) — thanks @ndntran14!
- Model-load GPU exhaustion now returns a sanitized, actionable dubbing error, and readiness correctly attributes the shared model status to TTS (#1695)
- Source-mode development now restarts an isolated backend crash without tearing down the UI, while repeated crash loops still stop loudly with diagnostics (#1690)
- Dubbing playback now keeps an audible companion source when a WebView can render the preview picture but cannot decode its audio (#1692)
- Model Catalogue engine rows now use the available desktop width and keep identity, runtime state, and actions from crowding one another (#1689)
- VoiceStudio now acts as a local speech platform: other apps can trigger its native dictation or connect through versioned HTTP, WebSocket, JSON-RPC, CLI, and MCP transports (#1646)
- A timed-out in-process dub transcription no longer starts a second WhisperX/CTranslate2 call over the abandoned native worker, preventing the overlapping access that preceded Windows `0xC0000005` exits (#1669)
- Windows debugger termination code `0x40010004` is no longer misreported as a backend crash or charged against automatic restart recovery (#1663)
- Studio now keeps one generation reservation across page changes, preventing a remount from stacking native jobs until the backend reports capacity busy or is killed under memory pressure (#1670)
- Uploaded dubbing videos are normalized to browser-safe H.264/AAC before preview, preventing valid VP9, AV1, or Opus media from failing with “no supported sources” (#1644)
- Dubbing now separates spoken and target languages, preserves translations through segment cleanup, and lets failed translations be retried or skipped without restarting the batch (#1654) — thanks @Number16BusShelter!
- Importing replacement SRT subtitles now keeps each cue bound to the best-overlapping source speaker and clone instead of resetting every line to a random default voice (#1660) — thanks @invio-a11y!
- Uploading a Dub preview no longer blocks every backend request while ffmpeg extracts its audio (#1667) — thanks @tfreyd!
- Docker quick starts now require the administrator key needed through container NAT instead of starting a UI whose protected actions return 403 (#1651) — thanks @wd357dui!
- WSL2 AMD containers now use the `/dev/dxg` ROCDXG bridge with actionable GPU diagnostics instead of silently falling back to CPU (#1655) — thanks @wd357dui!
- Ad-hoc voice-clone references now stay alive until cancelled or timed-out GPU work actually stops reading them, so prompt caching can finish instead of failing on a deleted temp file (#1668) — thanks @tfreyd!
- Dictation now stays bound to the app where it started and recovers locally from silent recognizer output (#1175)
- The backend now answers within a second of launch and narrates its startup step by step (#1550)
- Reporting a bug from an outdated build now offers the latest release first (#1547)
- The backend is only announced ready once it can actually serve, and crash-loop restarts now pace themselves (#1548)
- Invisible watermarking no longer stalls — or silently skips — the first take of a session (#1615)
- Dub subtitles can be retimed, inserted, and merged in either direction from the segment table (#1612) — thanks @invio-a11y!
### Changed
- Model Catalogue now uses one breathable workspace canvas with simpler pane and engine-family navigation instead of nested cards and scroll regions (#1685)
- Linux source launchers now catch missing libxdo and GStreamer audio plugins before they can cause a linker error or an aborted, blank WebKit renderer (#1680, #1682)
- Dictation now carries one native output session from shortcut-down through final delivery, restores text, HTML, image, or file-list clipboards only when untouched, keeps Wayland copy-safe unless current-focus insertion is explicitly enabled, and retries silent Sherpa speech only through an already-installed local ASR model (#1175)
- The backend binds its port immediately and reports startup progress live — `/health` answers 503-with-step and a new `/startup/progress` endpoint lists every step while PyTorch, API routes, and database migrations load in the background, so "starting at step X" is never mistakable for "dead"; the desktop splash narrates each step (#1550)
### Added
- A bundled Rust loopback sidecar exposes dictation start/stop/toggle, focused-output sessions, discovery, and JSON-RPC; the backend adds versioned streaming events and a dependency-free CLI bridge for Herdr, coding agents, editors, desktop apps, and TUIs (#1646)
- Headless NVIDIA and ROCm machines can now join as worker-only Docker Compose services with no published UI and durable protocol-v2 enrollment; update both machines together before reconnecting (#1638) — thanks @jkrogers9862!
- Linux ARM64 (Asahi Apple Silicon) support for the OmniVoice GGUF engine — a `linux-aarch64` binary built with GGML Vulkan where the toolchain allows it, so Apple GPUs accelerate generation through the open-source Honeykrisp driver instead of falling back to CPU-only (#1641)
- One-command install on every desktop OS: `curl -fsSL https://voicestudio.sh/install | sh` (macOS/Linux/WSL) or `irm https://voicestudio.sh/install | iex` (Windows) — the URL serves the right script per platform, and Windows gains a source installer (`scripts/install.ps1`) with a 3-OS CI smoke (#1626)
- Per-line subtitle management in the dub table: a line's end time is editable alongside its start (typing a time and dragging its timeline edge now take the same path), lines merge with the previous row as well as the next (`Ctrl/Cmd+Shift+M`), and a new line can be inserted into the gap after any row (#1612) — thanks @invio-a11y!
- CI now enforces performance regression budgets on the hot paths — operation-count tests pin streaming TTS to one synthesis per sentence and cached dub re-mixes to zero re-synthesis; fast-path guards cover zero re-decoding and ⌈N/W⌉ native batch calls when enabled (#1594)
- Default-engine dubbing now synthesizes several segments per forward pass instead of one call per line — the width follows the host's device headroom (1 on CPU and low-VRAM cards, up to 8), `OMNIVOICE_DUB_BATCH_WIDTH` overrides it, and engines without native batching keep the single-segment path (#1594)
- `/ws/tts` now reports real time-to-first-audio, and its RTF measures synthesis alone so a slow client can't inflate it (#1594)
- The locally cached AudioSeal watermark generator warms on a background thread ~35s after boot (`OMNIVOICE_PRELOAD_WATERMARK=0` opts out; explicitly setting `=1` may download it), so the first synthesis no longer serializes the audioseal import + model load inline — measured at ~42s on a cold filesystem, 3s short of a 90s client timeout (#1576) — thanks @paoloantinori!
- Voices you've cloned stay "warm" across restarts — encoded references now persist to disk (~10 KB each), so the first generation of a session skips the re-encode and any transcription pass; `OMNIVOICE_PROMPT_DISK_CACHE=0` opts out (#1565)
- Optional FlashInfer acceleration for the default engine on CUDA (`OMNIVOICE_FLASHINFER=1`, ~2.2x measured) — needs the optional `flashinfer-python` package; missing package or kernel failure logs why and falls back to the standard path (#1565)
- The bug reporter notices when you're on an outdated build and offers the latest release before filing — with a "File anyway" escape hatch — and stamps a `Build status` line into every report so up-to-date reports are tellable from stale ones (#1547)
- Settings → Performance & Device gains a compute-device override (Auto / CUDA / ROCm / XPU / MPS / CPU, or `OMNIVOICE_DEVICE`) — pin the device when auto-detect picks wrong; only devices your machine actually has are offered (#1557)
- Opt-in 24-layer PocketTTS checkpoints via `OMNIVOICE_POCKETTTS_24L` — better prosody for it/de/es/pt at roughly 2x render time (still faster than real-time); the fast 6-layer model stays the default (#1613) — thanks @paoloantinori!
### Docs
- Supported-version and install guidance now identifies 0.5.1 as the stable desktop and container release (#1687)
- The Docker Hub overview now shows the current engine-switching demo, Model Catalogue, and gallery voice workflow (#1593)
- The Docker Hub overview and install guide now show the v0.5 tags and the built-in API-key/share-PIN security model instead of obsolete v0.4 and no-authentication guidance (#1592)
- The READMEs now lead with download buttons and a three-step first-clone walkthrough, and a new benchmarks page anchors measured per-engine/per-device numbers on the in-repo harness (#1555)
- Every engine now has its own guide — 21 new pages under docs/engines plus an index covering all 16 TTS and 11 ASR engines, linked from both READMEs (#1556)
- The OmniVoice guide now covers combining style attributes with a reference clip (consistent instruct stabilizes cloning; the reference wins conflicts), inline pronunciation control (pinyin / CMU phonemes), and corrects the claim that the default engine can't do voice design — it can, from attributes (#1565)
### Fixed
- Workspaces now measure their responsive width when the post-bootstrap shell actually mounts, so native UI scaling reflows Projects and History instead of crushing the Dubbing demo into unreadable columns (#1683)
- Dubbing keeps the source-language selector visible after a local file is chosen, so ASR can be pinned before transcription starts (#1678) — thanks @Lonki-lomki-cloud!
- First-run media-engine downloads become available to TTS immediately without a restart, and missing media-process failures now point to repair controls (#1677) — thanks @farhataligpt-dev!
- Source installs on AMD GPUs honour `OMNIVOICE_TORCH_VARIANT=rocm`: `bun run desktop` now swaps in the ROCm torch wheel after `uv sync` and launches the backend without re-syncing, instead of silently reverting to the CPU-only CUDA build on every start (#1665) — thanks @uberclokr!
- `bun run desktop` on a fresh clone no longer fails with "resource path `../../frontend/dist` doesn't exist" — the dev launcher creates the placeholder Tauri resource directory before compiling (#1664) — thanks @uberclokr!
- macOS no longer loses TTS after the first request when Python lacks `os.waitid`; subprocess ownership now uses a safe `waitpid` fallback without risking reused process groups (#1656) — thanks @paoloantinori!
- Desktop startup, Retry, reset, uninstall, shutdown, and crash recovery now share one backend lifecycle owner; quitting interrupts first-run installers and gracefully drains then force-cleans the full backend process tree, so overlaps cannot duplicate or orphan it (#1635) — thanks @Xohaibxobi!
- Large Stories and Audiobook projects now persist in IndexedDB instead of overflowing the `omnivoice.app` localStorage envelope, with quota-safe migration and orderly exit/reload flushing (#1636) — thanks @leodzai!
- OmniVoice and its crash-isolated subprocess now route to AMD ROCm GPUs instead of warning and falling back to CPU (#1629) — thanks @j4r3kb!
- Dictation now cancels pending startup work, capture resources, sockets, and timers when the capture widget closes, preventing late work against a destroyed webview (#1645)
- Streaming generation failures now show recognized recovery guidance and appear in Diagnostics instead of only returning a generic error (#1607)
- The worker-capacity transport test no longer races its own setup: the 1-slot limit now goes through the enrollment handshake instead of mutating client config after connect, where the server's stream-open ConfigUpdate (carrying the registered capacity of 2) could overwrite it and fake an over-accept; failed CI twice on 2026-08-21 (#1630)
- Moving words across a speaker boundary in a dub — merging two lines and splitting them again — no longer dubs the second half in the first speaker's voice; each half now keeps the speaker, voice, direction, gain, and language of whoever actually says it (#1612) — thanks @invio-a11y!
- Dictation on a WebView that refuses a 16 kHz audio context (WKWebView) now low-passes before downsampling, so frequencies above 8 kHz stop folding into the speech the recognizer is fed (#1610)
- A microphone context that cannot be resumed now reports a mic error instead of leaving the dictation pill on "Listening" while capturing nothing (#1610)
- Dictation no longer retains a whole session's audio for silent-model recovery — an open mic grew that buffer by ~115 MB an hour; the recent two minutes are kept instead (#1610)
- The clipboard-delivery status is now translated in all 21 languages, so Wayland users — where clipboard delivery is the default — no longer see an English string (#1610)
- A native sherpa-onnx load failure of any exception type now degrades to "engine unavailable" instead of taking the dictation WebSocket down (#1610)
- Dictation now ships Whisper Tiny as its one cross-platform default, avoiding Parakeet's measured empty decoding on Windows while keeping Parakeet selectable behind runtime fallback (#1175)
- Re-mixing a dub no longer decodes, rewrites, and re-reads every cached segment — same-rate cached audio is reused directly (and rejected if truncated), switching timing modes can't reuse slot-truncated audio as natural-rate, and RVC respects natural-rate modes (#1594)
- PocketTTS French works again — pocket-tts only ships a 24-layer French model and rejected the name the sidecar asked for, so every French request failed at model load; French now always loads `french_24l` (#1613) — thanks @paoloantinori!
- Installing IndexTTS 2.5 no longer fails claiming an interrupted download — the weights repo ships `config.yaml` and VoiceStudio demanded a `config_v2_5.yaml` that exists in no upstream release; both names are accepted, so a hand-renamed checkout keeps working (#1611) — thanks @zuiaiyutu!
- IndexTTS 2.5 no longer has long-text generation killed at 60 seconds — the sidecar now proves it is alive every 5 seconds while `infer()` runs, and its deadline rises to 900s (`OMNIVOICE_INDEXTTS_RECV_TIMEOUT_S`) (#1611) — thanks @zuiaiyutu!
- The OpenAI-compatible `/v1/audio/speech` route now reuses the shared cached engine for explicit `model` ids instead of constructing a fresh engine — and its sidecar/model load, a ~28s floor per call for subprocess engines — on every request, with the same single-engine-resident discipline `/generate` applies (#1614) — thanks @paoloantinori!
- The setup wizard's RAM check no longer blocks 8 GB machines whose OS reports ~7.8 GB usable — the thresholds now tolerate reserved memory, and `OMNIVOICE_RAM_PREFLIGHT=0` turns a genuine block into a warning for those who accept the OOM risk (#1618)
- Invisible watermarking now runs eagerly instead of through `torch.compile` — AudioSeal's lazy compile sent the first embed of every session into Inductor's C++ codegen, which failed outright on macOS hosts whose toolchain couldn't serve it and shipped the audio unmarked after a 30-40s wait; first embed drops from 9.70s to 0.26s (#1615) — thanks @paoloantinori!
- The macOS Accessibility blocker now rechecks while visible and closes as soon as the grant is enabled instead of keeping a stale permission prompt on screen (#1609)
- The dubbing editor's video and transcript columns can now be resized by pointer or keyboard, and the chosen split persists across launches (#1571) — thanks @invio-a11y!
- CPU-only synthesis now gets a bounded ten-minute execution budget, and a render that exhausts it is reported as a compute timeout instead of misleading "generation capacity is busy" queue pressure (#1588) — thanks @ChienNguyen1111!
- Rapid Launchpad ↔ Dub navigation now replaces the workspace DOM owner cleanly, so late media/waveform cleanup cannot trigger React's `insertBefore` crash (#1590) — thanks @nicolas-jacques!
- Watermark embedding failures now log the full traceback instead of just the exception message, so a silently-unmarked-audio incident (audio passes through unmarked by design) is diagnosable from the log alone (#1576) — thanks @paoloantinori!
- Dubbing now recovers rapid two-speaker exchanges when diarization collapses them, defaults new projects to lip sync without overwriting saved timing choices, and keeps the editor usable on narrow screens (#1584) — thanks @victordonat0!
- `OMNIVOICE_ASR_BACKEND=omnivoice` now selects the PyTorch-native Whisper path, so the documented ROCm escape hatch no longer fails as an unknown engine (#1582) — thanks @patmansk!
- Network Sharing from Windows MSI/portable installs now serves the bundled web interface to LAN devices instead of redirecting them to their own `localhost` (#1589) — thanks @TWIISTED-STUDIOS!
- Exported dubbed videos now mark the dubbed language as the default audio stream while keeping Original available as an explicit choice (#1575) — thanks @invio-a11y!
- Cloning references can no longer exhaust system memory: transcript-free clips up to 75 seconds are searched in five bounded passages, longer clips ask to be trimmed, and supplied transcripts remain capped at 20 seconds to preserve alignment (#1578) — thanks @ACKAPOB!
- Stored artifact subpaths now resolve after moving a data directory between Windows, macOS, Linux, and Docker, while traversal and symlink escapes remain blocked (#1559) — thanks @Eman-Yousaf!
- A remote browser hitting an API-key-configured server's admin 403 now gets the API-key login form instead of endless console 403s, while desktop and PIN-only/no-key servers keep the plain loopback error so guests are never offered a login no key can satisfy (#1568) — thanks @paoloantinori!
- The crash-isolated ASR sidecar and its download preflight now agree on which model to load — setting the shared faster-whisper model variable applies to both variants instead of the sidecar quietly using a different one (#1556)
- "Ready" now requires the deep health probe (a working database-backed route), not just the identity probe — a backend whose install broke underneath can no longer be announced up while every real request fails (#1548)
- Supervisor restarts after repeat crashes now back off (immediate, then 5s, then 15s) instead of respawning back-to-back, so a tight crash loop can't burn the whole restart budget in seconds (#1548)
- The Linux desktop cleanup regression test now isolates build artifacts, so an existing developer build can no longer change its result (#1566)
- Renaming, deleting, or revoking consent on a voice (and starring/clearing history, recording exports) now live-updates every open tab again — the sync routes' WebSocket events were silently dropped, which could look like "all my voices are gone" (#1561) — thanks @paoloantinori!
### CI
- Project agents now share pinned Vite and FastAPI skills from skills.sh (#1594)
- Weekly full-history secret scans no longer mistake the Ed25519 private-key type name for committed key material (#1591)
## [0.5.0] — 2026-08-13
**Highlights**
- The app is now **VoiceStudio** (previously OmniVoice-Studio) — one waveform-and-spark identity across the app, docs and installers. Your data folder, settings and Docker image paths stay put.
- **Model Catalogue** — engines and models in one workspace: every TTS, transcription and LLM engine with its device routing and install state, defaults picked there.
- Switch TTS, ASR and LLM engines from the status bar or any workspace — ready-only choices, memory status, environment-pin protection, `Ctrl/Cmd+E`. (#1530)
- Lend another machine's GPU with a join code and a QR scan — a Compute control in the status bar picks where jobs run, and several people can share one GPU box with revocable, certificate-pinned connections. (#1516, #1496)
- Server mode is locked down: admin actions require an API key (#1525), and the remote UI exchanges it for short-lived sessions that never sit in browser storage or WebSocket URLs (#1528) — thanks @bultodepapas!
- A faster, cleaner Dub workspace for multilingual production, with a production command bar and per-language cards. (#1489)
- The demo audio and video the app always advertised now actually ship, rendered by VoiceStudio's own engine. (#1517)
- Dictation works on Wayland now — the portal shortcut actually fires (#1490, #1526) — and the recording pill is back on every desktop.
- The Launchpad wears the project's signal-field waveform artwork over a quieter, borderless layout. (#1533)
- The catalogue reads as headroom, not breakage: available engines sort first, uninstalled ones say what they need (#1531), and the LLM row names the provider that actually answers (#1538).
- Gallery voices can be saved as local profiles — audio lands in your profile store with validated, content-addressed references. (#1542)
<img src="https://raw.githubusercontent.com/debpalash/VoiceStudio/main/docs/media/0.5.0/quick-switch.gif" alt="Switching TTS engines from the status bar" width="820" />
| The Model Catalogue | The Voice Gallery |
| --- | --- |
| <img src="https://raw.githubusercontent.com/debpalash/VoiceStudio/main/docs/media/0.5.0/catalogue.png" alt="Model Catalogue — engines pane" width="420" /> | <img src="https://raw.githubusercontent.com/debpalash/VoiceStudio/main/docs/media/0.5.0/gallery-save.png" alt="Voice Gallery — save a voice as a profile" width="420" /> |
### Changed
- Gallery personas now preview through the local backend, retain their complete voice-design recipe, and open directly in Voice, Stories, or Audiobook. (#1542)
- Typing and large workspace edits no longer serialize and rewrite persisted documents on every input; writes are coalesced off the interaction path — thanks @bultodepapas! (#1541)
- Support amount choices now use every theme's shared card, accent and focus tokens. (#1530)
- Sponsoring, commercial licensing and getting in touch are one page now. They answered the same question between them and each used to live somewhere else, so they are three sections on a single scroll — the footer heart, the commercial-licence links and Contact all land on it, at the section you asked for. (#1522)
- Model Catalogue switches panes with tabs instead of a two-state toggle, and the Engine Compatibility Matrix's TTS / ASR / LLM switcher is now tabs too — arrow-key navigable, and each tab still shows the engine it would use. (#1522)
- Engines you can actually use sort to the top of the compatibility matrix, and an unavailable engine's name recedes instead of the whole row fading — the status badge and GPU chips that say *why* it is unavailable stay legible. (#1522)
- Remote workers reads as a device list: status dot, address, latency, a live task meter, resident models and last-seen per machine, with housekeeping actions revealed on hover and a three-step empty state. (#1516)
- The GPU picker and the new status-bar control paint their status dots and menu surfaces from themed tokens instead of fixed palette classes, so they stop showing Gruvbox colours on Midnight and Catppuccin. (#1516)
- Dictation shows the pill again: a capture puts a small always-on-top capsule near the bottom of the screen you are working on — listening, transcribing, the result, and any error — and takes it away when the session ends. It never takes focus, so the text still lands in the app you were typing into. On Wayland the compositor decides where it sits; everywhere else it is bottom-centred.
- Engines and models moved out of Settings into a new Model Catalogue workspace, reachable from the icon rail (or the title-bar tabs); Settings → Engines and Settings → Models now point there, and Settings keeps the models directory and Hugging Face mirror.
- The Settings sidebar is keyboard-navigable: ⌘K / Ctrl+K jumps to the filter, ↑/↓ and Home/End move between categories, and Enter or ↓ from the filter drops into the list. Matching text in a filtered category name is highlighted, and group headers stay pinned while the list scrolls.
- The Launchpad has a quieter, more spacious look: borderless feature tiles that light up on hover or keyboard focus, plain-numeral counts, hairline section rules, and one shared page column for the hero, tiles, recent files and project lists.
- Linux release smoke now validates linuxdeploy's wrapped custom launcher instead of rejecting a healthy AppImage. (#1506)
- Remote GPU workers render audiobooks chapter by chapter, with automatic per-chapter local fallback and one combined notice if the worker drops out. (#1478)
- Remote GPU workers can now run a job to completion: long renders no longer die at two minutes, a worker that drops and reconnects mid-render keeps its work, and a timed-out job no longer takes the worker offline for good. Placing a job still needs the development-only `POST /workers/tasks`; wiring the app's own Synthesize button to it comes next.
- Voice, Stories, Audiobook, Gallery, Settings, profiles, and Launchpad now use compact, responsive layouts with accessible controls. (#1491)
- Dubbing's Generate Dub, Verify, and Export actions now use a compact hierarchy with visible labels, responsive reflow, and motion-safe feedback. (#1493)
- The Dub workspace now has a compact production command bar, responsive flag-based language cards, media previews in Dub History, and a narrower Projects rail. (#1489)
- VoiceStudio now uses one waveform-and-spark mark across the title bar, About screen, README, browser favicon, and every desktop/platform icon. (#1487)
- PocketTTS now asks you to review its code license, model license and gated-access conditions before first use, and explains how to unlock the model instead of showing a raw download failure — thanks @paoloantinori! (#1442)
- The repository moved to github.com/debpalash/VoiceStudio. Every link in the app, docs and scripts now points there; GitHub redirects the old URLs, and the Docker image paths, the app bundle identifier and your data folder are all deliberately unchanged. (#1394)
@@ -35,6 +244,19 @@ the frozen-backend fallback mirror it for their toolchains.
### Added
- Gallery personas preview through the local backend, keep their full voice-design recipe, and open directly in Voice, Stories, or Audiobook — and can be saved as local profiles with validated audio references. (#1542)
- The demo audio the app has always advertised now actually ships: previews for all seven voice-design presets, the three dictation replay clips, and the dubbing demo's source video plus four dubbed languages with subtitles. Every one of those was a dead link before — the tooling that renders them required macOS, so on Windows and Linux the files were never built. (#1517)
- Demo assets are rendered by VoiceStudio's own engine, so the tooling runs wherever the app does, and the demos are made by the thing they demonstrate. (#1517)
- A machine can now join a control plane from the app: Settings → System → Remote workers → **Lend this machine's GPU**, paste the join code, done — no environment variables and no restart. The address travels with the code, so the machine reconnects on its own afterwards. (#1516)
- Join codes and connection strings are shown as a **QR code** alongside the text, with a live expiry countdown — scan it from the other machine instead of retyping forty characters. (#1516)
- A **Compute** control in the status bar: pick local or a remote machine, turn remote workers on or off, and mint a join code without opening Settings. It appears only once you have opted in or enrolled a machine. (#1516)
- A worker waiting for approval can be approved from its row. The panel labelled that state before but offered no way out of it. (#1516)
- **Model Catalogue** — a workspace of its own for engines and models: browse every TTS, transcription and LLM engine with its device routing and install state, pick the default for each, and install or remove model weights, all from one screen instead of two Settings categories.
- Remote GPU machines can now accept connections instead of dialling out, so several people can use the same box at once — each gets their own revocable connection string, with certificate-pinned TLS, a live list of who is connected, and a disconnect button. (#1496)
- Remote GPU model downloads now use the normal Models install flow and show per-worker progress. (#1478)
- Settings → System → **Remote workers** sends individual jobs to GPUs on your other machines while everything else stays here. Off by default; each machine is added with a single-use token and approved before any audio reaches it. See [docs/remote-workers.md](docs/remote-workers.md).
- First-run setup now recommends a screen-aware interface scale, with compact controls available throughout setup. (#1502)
- OrcaRouter is now available as a named OpenAI-compatible LLM provider — thanks @Marc-oss-hub! (#1499)
- IndexTTS 2.5 is available as a pinned one-click sidecar with five-language dubbing, expressive cloning, and backward-compatible IndexTTS-2 support. (#1482) — thanks @marwanlhabti5-coder!
- Voice recording now offers microphone and channel selection with a live input-level meter on every desktop platform. (#1481)
- Settings → Appearance → **Navigation style** switches the workspace switcher between the icon rail down the window edge and browser-style tabs across the title bar. Both offer the same workspaces; the choice sticks across launches, and the rail stays the default. Tab labels fold down to icons when the title bar runs out of room — the workspace you're in keeps its name. (#1412)
@@ -42,16 +264,47 @@ the frozen-backend fallback mirror it for their toolchains.
- Settings → Privacy now has an **Invisible watermark** toggle. On by default, available to everyone, and it only affects audio generated after the change. (#1308)
- A new opt-in crash-isolated TTS engine, so a native crash takes down the sidecar instead of the whole backend — thanks @paoloantinori! (#1292, #1298, #1304)
- **PocketTTS** (Kyutai), an opt-in CPU-only engine for fast, low-latency renders in six languages (en/fr/de/pt/it/es) with zero-shot cloning from a reference clip. Enable in Settings → Engines — thanks @paoloantinori! (#1306, #1328)
- A warning before a slow generation, rather than after a five-minute wait. (#1280)
### CI
### Docs
- The stdio wire protocol every engine sidecar speaks is now tested once across all nine of them, instead of against a single engine — a bug in any one sidecar's copy gets caught — thanks @paoloantinori! (#1408)
- Engine acceptance: new `docs/engine-acceptance.md` documents the job map, the bar a new engine must clear, and the out-of-tree path (#1306)
- macOS install notes and the README support table now state the real floor (#1268)
- Contact: the project X account is listed alongside Discord (#1313)
- `OMNIVOICE_ALLOWED_ORIGINS` is finally documented: a browser loading the UI from another machine's origin needs the backend's CORS allow-list, which neither server mode nor trusted networks touches — thanks @vanderlpp! (#1348)
### Fixed
- AMD/ROCm hosts no longer crash ASR with "CUDA driver version is insufficient": ROCm torch reports itself as CUDA, but whisperx/faster-whisper run on CTranslate2, which is NVIDIA-only — they now take the CPU path there, and auto-detect prefers pytorch-whisper, which genuinely uses the HIP GPU. (#1529)
- Crash reports now carry the crashed run's own stderr: the shared error log is append-only with per-run offsets, so a restart can no longer overwrite the dying process's final output with the replacement's healthy startup. (#1510)
- Wayland: a stale portal identity no longer kills the dictation shortcut for the whole session. The desktop entry the app writes for the GlobalShortcuts portal could point at a binary that has since moved (a `cargo clean`, a relocated AppImage) — GNOME then refuses the bind with "App info not found" and the hotkey silently dies. The entry is validated and rewritten at startup now. (#1526)
- The guard that keeps transcription on the degrading ASR loader now scans the whole backend, not just the routers — a service that transcribes on a request's behalf skipped `ensure_loaded()` just as thoroughly. (#1519) — thanks @ahov520!
- The Linux app icon is no longer blank. Every AppImage since v0.4.2 shipped `.DirIcon` as an absolute symlink into the machine that built it (`/home/runner/work/…`), so the link dangled on every user's computer and file managers, app menus and desktop integration all drew nothing. The release build now verifies the icon resolves inside the bundle before publishing. (#1518)
- The Linux desktop entry no longer ships an empty `Categories=`, which `desktop-file-validate` rejects and menu builders skip. (#1518)
- Wayland: the dictation shortcut now actually starts dictation. The desktop portal registered the key correctly — GNOME and KDE even showed it back — but every press was discarded while decoding the compositor's signal, so the hotkey did nothing on any Wayland session. (#1490)
- The first-run "Choose a comfortable UI size" screen no longer stutters while you sit there. Applying a scale resizes the window's own viewport, which the screen was reading back to re-pick a size — so it flipped between two sizes forever without anyone touching it. (#1514)
- Transcription now moves to the next working engine when the auto-picked one passes its availability check but breaks on first real use, instead of returning an internal error — the recovery dubbing already had. Affected accurate-mode transcription, the OpenAI-compatible API, batch, dub verify, and voice-clone reference text. (#1512)
- A malformed request now gets a clear 422 instead of an internal error, and uploading a file to an endpoint that expects JSON no longer copies the whole upload into the app log — a 145 KB clip wrote roughly 500 KB of log, recording your audio in the file people paste into bug reports. (#1513)
- The Simplified Chinese (zh-CN) translation no longer mistranslates brand names and technical terms — Discord, Tailscale, Hugging Face, IPA, and LLM (Cinematic) were rendered as nonsensical literal translations, and ~250 more awkward machine-translation strings are now natural Chinese. (#1508) — thanks @anyingiit!
- Worker restart coverage now waits for the registration response to persist its identity instead of racing the client callback in CI. (#1505)
- Dub language and export selections now restore without false schema warnings, and remote-worker port 7443 is identified instead of reported as a generic timeout. (#1504)
- A configured remote backend now bypasses local first-run setup, verifies itself before app requests begin, and shows recovery instead of leaving the desktop stuck on Setup. (#1503)
- An idle voice model now actually hands its memory back. The unload emptied the GPU cache a moment before releasing the model, so it freed nothing while reporting success — a GPU machine lending its card sat on 3.6 GB indefinitely. (#1495)
- Unloading a model on an NVIDIA GPU now returns the last ~770 MB too. A single 8.5 MB cuBLAS workspace sat inside the model's memory block and kept the whole block reserved, so an idle machine held 1.2 GB instead of 470 MB no matter how often you pressed Flush Memory. (#1495)
- Flush Memory reports reserved GPU memory alongside allocated. Allocated alone reads near zero right after an unload while the GPU still shows gigabytes, which is exactly the case people were reporting. (#1495)
- The AudioSeal watermark models are released after the same idle period as everything else, instead of staying in memory for the life of the app once anything was watermarked. (#1495)
- Remote GPU workers now synthesize a dub's fresh segments as one coarse job with live progress and cancellation; fitting, assembly and RVC remain local. (#1478)
- Gallery voice previews now fall back to a local render when a downloaded clip cannot be decoded, instead of failing silently. (#1478)
- A second VoiceStudio instance can no longer silently share the remote-worker port; it keeps running locally and explains how to resolve the conflict. (#1478)
- Remote GPU jobs stay pinned to the selected worker across retries and restarts, stop when their caller leaves, and cannot return from cancellation as completed. (#1478)
- Remote GPU model labels now survive registration, legacy blank model IDs share one capacity slot, long jobs retain bounded leases, and idle cleanup cannot evict a live local render. (#1478)
- Remote GPU jobs now stop before dispatch when that worker lacks the model, offer the download there, and refresh scheduling as soon as it finishes. (#1478)
- Leaving a screen while its waveform is still loading no longer opens a bug-report prompt for a normal cancelled request. (#1498)
- An unreachable remote backend now opens a retryable recovery screen instead of sending the app into local model setup, with clear TLS, CORS, network, HTTP, and wrong-port guidance — thanks @debpalash! (#1501)
- Linux production test launches now stop their own extracted AppImage before resetting SQLite and logs. (#1494)
- Restored the pre-release version to 0.4.2 while the next release remains in preparation. (#1488)
- Large multi-language dubbing batches now use compact searchable language and track managers instead of overflowing the editor. (#1492)
- Dictation shortcuts now register and rebind through the desktop portal on Wayland, honor custom keys in focused app views, and show the effective platform keys. (#1490)
- Multi-language dubbing now translates, edits, generates, retains, and exports every selected language, and its language picker stays visible at viewport edges. (#1486)
- Dubbing's **From video** cast now uses available source-audio samples for every speaker and short line, including jobs without a pooled diarization clone. (#1484)
- Basic Dubbing translation remains available without an LLM; Cinematic and Autofit now degrade through the existing Fast translation path instead of blocking the quality choice. (#1481)
@@ -140,19 +393,17 @@ the frozen-backend fallback mirror it for their toolchains.
- Translation through LM Studio works. The built-in model name was the placeholder `local-model`, which LM Studio rejects because it serves whatever you have loaded — VoiceStudio now asks it, and a 404 from a local server names the models that ARE loaded instead of telling you to check a URL that was fine — thanks @biga73! (#1332)
- Generation that silently dropped the end of the input now says so. When an engine returns no audio for part of the text the result sounds clean and is simply short, so the only way to notice was to read along; the backend log now names the sentences that produced nothing. (#1330)
- Dubbing: a re-rendered line that quietly came back in a default voice instead of the cloned one now says why in the backend log — the clone clips are extracted per job and a saved dub outlives them, so regenerating after cleanup loses the reference with no error. (#1331)
### Docs
- Engine acceptance: new `docs/engine-acceptance.md` documents the job map, the bar a new engine must clear, and the out-of-tree path (#1306)
- macOS install notes and the README support table now state the real floor (#1268)
- Contact: the project X account is listed alongside Discord (#1313)
- `OMNIVOICE_ALLOWED_ORIGINS` is finally documented: a browser loading the UI from another machine's origin needs the backend's CORS allow-list, which neither server mode nor trusted networks touches — thanks @vanderlpp! (#1348)
- RTX 40-series GPUs are used again instead of being sent to the CPU. (#1289)
- Apple Silicon: transcription no longer needs a system ffmpeg, as the docs always said — thanks @gambletan! (#1436)
- A failed audiobook chapter says why, instead of turning red and saying nothing. (#1325)
### CI
- Windows CI falls back to a static ffmpeg build when the Chocolatey feed is down, instead of failing the run. (#1542)
- The stdio wire protocol every engine sidecar speaks is now tested once across all nine of them, instead of against a single engine — a bug in any one sidecar's copy gets caught — thanks @paoloantinori! (#1408)
- Windows smoke tests stopped silently passing a broken ffmpeg install, and every smoke leg is now budgeted for a cold dependency install. (#1290)
- Test suites no longer leak config paths or model-manager shutdown state into one another, which had been failing unrelated pull requests. (#1269)
- The nightly preview build stopped refusing to publish its own healthy updater manifest when the macOS legs finished a few minutes ahead of the slowest one — Preview-channel users were silently left without new builds.
- The nightly preview build stopped refusing to publish its own healthy updater manifest when the macOS legs finished a few minutes ahead of the slowest one — Preview-channel users were silently left without new builds.
## [0.4.2] — 2026-07-28
+4 -1
View File
@@ -66,7 +66,10 @@ Architecture not yet mapped. Follow existing patterns found in the codebase.
<!-- GSD:skills-start source:skills/ -->
## Project Skills
No project skills found. Add skills to any of: `.claude/skills/`, `.agents/skills/`, `.cursor/skills/`, `.github/skills/`, or `.codex/skills/` with a `SKILL.md` index file.
- `vite` — Vite configuration, assets, HMR, builds, and Vitest guidance.
- `fastapi-python` — FastAPI and Pydantic implementation patterns.
Canonical copies live under `.agents/skills/`; `skills-lock.json` pins their sources and hashes. Claude should follow these paths directly, avoiding cross-platform symlinks.
<!-- GSD:skills-end -->
<!-- GSD:workflow-start source:GSD defaults -->
+10 -4
View File
@@ -10,10 +10,10 @@ Copyright 2024-present Palash Debnath and VoiceStudio contributors.
VoiceStudio is **free and open-source software, licensed under the GNU
Affero General Public License, Version 3 (AGPL-3.0)**. You are free to use,
copy, modify, and redistribute it — and that **includes commercial and internal
business use**: run the app, use its outputs commercially, sell the audio you
produce with it, provide professional/client services with it, and deploy it
within your organization.
copy, modify, and redistribute it. That **includes commercial and internal
business use** of the application itself. Model weights, tokenizers, and other
third-party assets retain their own terms; this application license does not
grant or summarize rights under those separate terms.
Because this is the **Affero** GPL, one additional obligation applies: if you
modify VoiceStudio and make that modified version available to others over
@@ -41,6 +41,12 @@ is **separately licensed under Apache License 2.0** by its upstream authors and
is not relicensed here. Apache License 2.0 is compatible with, and may be
combined under, the GNU AGPL-3.0. See `pyproject.toml`.
Downloaded model weights are not relicensed by VoiceStudio. The default
`k2-fsa/OmniVoice` model card identifies its code as Apache-2.0 and pretrained
weights as CC-BY-NC. Its `audio_tokenizer/LICENSE` contains separate Boson
Higgs Audio 2 and Meta Llama community terms. A commercial license for
VoiceStudio-owned code does not replace any of those terms.
Third-party dependencies retain their own licenses. See `Cargo.lock`,
`bun.lock`, and `uv.lock` for the resolved set.
+370 -540
View File
File diff suppressed because it is too large Load Diff
+126 -55
View File
@@ -20,6 +20,7 @@
</p>
<p>
<a href="https://github.com/debpalash/VoiceStudio/actions/workflows/ci.yml"><img src="https://img.shields.io/github/actions/workflow/status/debpalash/VoiceStudio/ci.yml?branch=main&style=flat-square&label=CI" alt="CI 状态" /></a>
<a href="https://github.com/debpalash/VoiceStudio/stargazers"><img src="https://img.shields.io/github/stars/debpalash/VoiceStudio?style=flat-square&color=f59e0b" alt="Star 数" /></a>
<a href="https://github.com/debpalash/VoiceStudio/releases/latest"><img src="https://img.shields.io/github/v/release/debpalash/VoiceStudio?style=flat-square&color=10b981" alt="版本" /></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/license-AGPL--3.0-blue?style=flat-square" alt="许可证" /></a>
@@ -37,7 +38,7 @@
<br/>
<div align="center">
<img src="docs/screenshot-launchpad.png" alt="VoiceStudio — 启动台" width="100%"/>
<img src="docs/media/0.5.0/quick-switch.gif" alt="VoiceStudio — 从状态栏快速切换 TTS 引擎" width="100%"/>
</div>
> **声音很私人,创作空间也应该真正属于你。** VoiceStudio 的核心流程运行在你的硬件上:克隆、设计、配音、听写,并以 646 种语言创作,不需要订阅,也没有用量计费。联网引擎和服务始终是清晰可见的可选项,而不是隐藏依赖。
@@ -45,6 +46,72 @@
> [!WARNING]
> **活跃 Beta 阶段。** 各版本之间可能出现故障——如需最新修复,请从源码运行。非常欢迎 Bug 报告和 PR:[提交 Issue](https://github.com/debpalash/VoiceStudio/issues) 或 [加入 Discord](https://discord.gg/bzQavDfVV9)。
<a id="quickstart"></a>
## ⚡ 快速开始
<div align="center">
<a href="https://github.com/debpalash/VoiceStudio/releases/latest"><img src="https://img.shields.io/badge/macOS-DMG_(Apple_Silicon)-000?style=for-the-badge&logo=apple&logoColor=white" alt="下载 macOS DMG" /></a>
<a href="https://github.com/debpalash/VoiceStudio/releases/latest"><img src="https://img.shields.io/badge/Windows-MSI_(x64)-0078D4?style=for-the-badge&logo=windows&logoColor=white" alt="下载 Windows MSI" /></a>
<a href="https://github.com/debpalash/VoiceStudio/releases/latest"><img src="https://img.shields.io/badge/Linux-AppImage_(x64)-FCC624?style=for-the-badge&logo=linux&logoColor=black" alt="下载 Linux AppImage" /></a>
<br/>
<sub>三个按钮都会打开最新发布页——在资源列表中下载对应你系统的安装包。</sub><br/>
<sub><b>macOS</b>首次启动需要一次性批准——右键点击 → <b>打开</b>macOS 15 上为 系统设置 → 隐私与安全性 → <b>“仍要打开”</b>)。无需终端。<a href="docs/install/macos.md#gatekeeper-quarantine">为什么?</a> · <b>Intel Mac</b>不支持本地后端(<a href="https://github.com/debpalash/VoiceStudio/issues/889">#889</a>)——<a href="docs/install/macos.md">详情</a>。</sub>
</div>
选择你的操作系统,按指南从头到尾操作:
- 🍎 **macOS** — [docs/install/macos.md](docs/install/macos.md)
- 🪟 **Windows** — [docs/install/windows.md](docs/install/windows.md)
- 🐧 **Linux** — [docs/install/linux.md](docs/install/linux.md)
- 🐳 **Docker** — [docs/install/docker.md](docs/install/docker.md) · [Docker Hub: `palashdeb/omnivoice-studio`](https://hub.docker.com/r/palashdeb/omnivoice-studio)
```bash
# Docker 快速运行 (CPU / 本地环回模式)
docker run -d -p 127.0.0.1:3900:3900 -v omnivoice-data:/app/omnivoice_data --name voicestudio palashdeb/omnivoice-studio:stable
```
**三步克隆出你的第一个声音:**
1. **安装并启动。** 首次启动会自动搭建 Python 运行环境并下载模型权重——启动画面会逐步显示进度(仅首次,需要几分钟;之后即开即用)。
2. 从启动台打开**语音克隆**,拖入任意声音的 **3 秒音频**
3. **输入一句话,点击生成。** 音频在你的设备上生成并保存,支持 646 种语言(商业使用前请审阅所选模型与分词器的许可条款)。
### 🎧 音频示例
在线试听 VoiceStudio 本地生成的实际音频样例:
| 工作流 | 提示词 / 参考音频 | 生成音频 |
|---|---|---|
| **声音克隆** | [demo_voice.wav](backend/assets/samples/demo_voice.wav) | [demo_clone_output.wav](backend/assets/samples/demo_clone_output.wav) |
| **声音设计** (美语新闻主播) | *"清晰、权威的美国广播级音色"* | [demo_voice_design_us_news_anchor.wav](backend/assets/samples/voice_design/demo_voice_design_us_news_anchor.wav) |
| **声音设计** (英式有声书) | *"温暖生动的英式故事讲述音色"* | [demo_voice_design_audiobook_uk_narrator.wav](backend/assets/samples/voice_design/demo_voice_design_audiobook_uk_narrator.wav) |
| **视频配音** (多语种) | [source.src.wav](backend/assets/samples/demo/dubbing/source.src.wav) | [西班牙语](backend/assets/samples/demo/dubbing/dubbed_es.src.wav) · [法语](backend/assets/samples/demo/dubbing/dubbed_fr.src.wav) · [日语](backend/assets/samples/demo/dubbing/dubbed_ja.src.wav) · [中文](backend/assets/samples/demo/dubbing/dubbed_zh.src.wav) |
觉得慢?[docs/performance.md](docs/performance.md) 讲清了生成时间到底花在哪里、有哪些调优开关,以及“它变慢了”的三个经典原因。各引擎/设备的实测数据见 [docs/benchmarks.md](docs/benchmarks.md)。
> 正在从 **[CorentinJ/Real-Time-Voice-Cloning](https://github.com/CorentinJ/Real-Time-Voice-Cloning)**(现已归档)迁移过来?我们有专门的迁移指南:[docs/migration/real-time-voice-cloning.md](docs/migration/real-time-voice-cloning.md)。
<details>
<summary><b>🧰 卡住了?自检、Token 与受限网络</b></summary>
<br/>
先运行内置自检——在应用中打开 **设置 → 关于 → “运行自检”**,或在源码检出目录中执行
`uv run python backend/main.py --diagnose`(加 `--deep` 还会实际加载当前引擎进行测试)。然后查看
[docs/install/troubleshooting.md](docs/install/troubleshooting.md) 中排名前
10 的安装错误。运行时出错时,应用内的错误界面会直接深链到对应条目;**设置 → 关于 →
“保存诊断包”** 会把脱敏日志与自检报告打包,方便附在 Bug 报告里。
Hugging Face Token 的配置见
[docs/setup/huggingface-token.md](docs/setup/huggingface-token.md)。说话人分离相关的模型访问门槛见
[docs/features/diarization.md](docs/features/diarization.md)。下载速度、⚡ 快速下载(Xet)状态,以及受限网络 / 镜像选项见
[docs/downloading-models.md](docs/downloading-models.md)。
</details>
---
<a id="features"></a>
## ✨ 功能
@@ -112,49 +179,6 @@
---
<a id="quickstart"></a>
## ⚡ 快速开始
<div align="center">
<a href="https://github.com/debpalash/VoiceStudio/releases/latest"><img src="https://img.shields.io/badge/macOS-DMG_(Apple_Silicon)-000?style=for-the-badge&logo=apple&logoColor=white" alt="下载 macOS DMG" /></a>
<a href="https://github.com/debpalash/VoiceStudio/releases/latest"><img src="https://img.shields.io/badge/Windows-MSI_(x64)-0078D4?style=for-the-badge&logo=windows&logoColor=white" alt="下载 Windows MSI" /></a>
<a href="https://github.com/debpalash/VoiceStudio/releases/latest"><img src="https://img.shields.io/badge/Linux-AppImage_(x64)-FCC624?style=for-the-badge&logo=linux&logoColor=black" alt="下载 Linux AppImage" /></a>
<br/>
<sub><b>macOS</b>首次启动需要一次性批准——右键点击 → <b>打开</b>macOS 15 上为 系统设置 → 隐私与安全性 → <b>“仍要打开”</b>)。无需终端。<a href="docs/install/macos.md#gatekeeper-quarantine">为什么?</a> · <b>Intel Mac</b>不支持本地后端(<a href="https://github.com/debpalash/VoiceStudio/issues/889">#889</a>)——<a href="docs/install/macos.md">详情</a>。</sub>
</div>
选择你的操作系统,按指南从头到尾操作:
- 🍎 **macOS** — [docs/install/macos.md](docs/install/macos.md)
- 🪟 **Windows** — [docs/install/windows.md](docs/install/windows.md)
- 🐧 **Linux** — [docs/install/linux.md](docs/install/linux.md)
- 🐳 **Docker** — [docs/install/docker.md](docs/install/docker.md) · [Docker Hub: `palashdeb/omnivoice-studio`](https://hub.docker.com/r/palashdeb/omnivoice-studio)
觉得慢?[docs/performance.md](docs/performance.md) 讲清了生成时间到底花在哪里、有哪些调优开关,以及“它变慢了”的三个经典原因。
> 正在从 **[CorentinJ/Real-Time-Voice-Cloning](https://github.com/CorentinJ/Real-Time-Voice-Cloning)**(现已归档)迁移过来?我们有专门的迁移指南:[docs/migration/real-time-voice-cloning.md](docs/migration/real-time-voice-cloning.md)。
<details>
<summary><b>🧰 卡住了?自检、Token 与受限网络</b></summary>
<br/>
先运行内置自检——在应用中打开 **设置 → 关于 → “运行自检”**,或在源码检出目录中执行
`uv run python backend/main.py --diagnose`(加 `--deep` 还会实际加载当前引擎进行测试)。然后查看
[docs/install/troubleshooting.md](docs/install/troubleshooting.md) 中排名前
10 的安装错误。运行时出错时,应用内的错误界面会直接深链到对应条目;**设置 → 关于 →
“保存诊断包”** 会把脱敏日志与自检报告打包,方便附在 Bug 报告里。
Hugging Face Token 的配置见
[docs/setup/huggingface-token.md](docs/setup/huggingface-token.md)。说话人分离相关的模型访问门槛见
[docs/features/diarization.md](docs/features/diarization.md)。下载速度、⚡ 快速下载(Xet)状态,以及受限网络 / 镜像选项见
[docs/downloading-models.md](docs/downloading-models.md)。
</details>
---
<a id="why-voicestudio"></a>
## 💡 为什么选择 VoiceStudio
@@ -173,8 +197,8 @@ Hugging Face Token 的配置见
| **API 密钥** | 需要账号 | 本地流程不需要 |
| **GPU 支持** | 不适用(云端) | CUDA · Apple Silicon · ROCmLinux)· CPU |
| **桌面应用** | ❌ | ✅ macOS · Windows · Linux |
| **TTS 引擎** | 1 | **14** — [完整矩阵](#tts-engines) |
| **ASR 引擎** | 1 | **10** — [完整阵容](#asr-engines) |
| **TTS 引擎** | 1 | **16** — [完整矩阵](#tts-engines) |
| **ASR 引擎** | 1 | **11** — [完整阵容](#asr-engines) |
| **MCP 服务器** | ❌ | ✅ 可从 Claude、Cursor 及任何 MCP 客户端使用 |
| **自检** | ❌ | ✅ 诊断套件、错误日志、脱敏调试包 |
| **可定制** | ❌ 闭源 | ✅ 随你 Fork、扩展、发布 |
@@ -210,14 +234,24 @@ Hugging Face Token 的配置见
> [!IMPORTANT]
> **macOS Intelx86_64)不支持本地后端:** 应用 UI 可以安装,但 Python 后端无法运行,因为 PyTorch 已不再发布 Intel Mac 轮子([#889](https://github.com/debpalash/VoiceStudio/issues/889))。Intel Mac 用户仍可让 UI 指向另一台机器上的远程后端——参见 [docs/install/macos.md](docs/install/macos.md)。
<a id="hardware-recommendations"></a>
### 💡 按硬件推荐引擎配置
| 硬件配置 | 推荐 TTS 引擎 | 推荐 ASR 语音识别 | 优势 |
|---|---|---|---|
| **Apple Silicon (M1M4)** | [MLX-Audio](docs/engines/mlx-audio.md) · [OmniVoice](docs/engines/omnivoice.md) (MPS) | [MLX Whisper](docs/engines/mlx-whisper.md) · [Parakeet MLX](docs/engines/parakeet-mlx.md) | 原生统一内存,macOS 上延迟最低、性能最强 |
| **NVIDIA 显卡 (8 GB+ 显存)** | [OmniVoice](docs/engines/omnivoice.md) · [CosyVoice 3](docs/engines/cosyvoice.md) | [WhisperX](docs/engines/whisperx.md) | 极致零样本克隆品质、字级时间戳对齐与说话人分离 |
| **低显存 / 仅 CPU 设备** | [PocketTTS](docs/engines/pockettts.md) · [Sherpa-ONNX](docs/engines/sherpa-onnx.md) · [KittenTTS](docs/engines/kittentts.md) | [Moonshine](docs/engines/moonshine.md) · [Faster-Whisper](docs/engines/faster-whisper.md) (`int8`) | 超低内存占用,针对 CPU 指令集深度优化 |
<a id="tts-engines"></a>
### 🗣️ TTS 引擎
**14 个引擎,一个选择器。** VoiceStudio(默认,支持 600+ 语言)始终可用;另有七个引擎可选装并自动检测(CosyVoice 3、GPT-SoVITS、VoxCPM2、MOSS-TTS-Nano、KittenTTS、MLX-Audio、Sherpa-ONNX),外加个按需延迟安装的重量级引擎(IndexTTS 2.5、OmniVoice GGUF、Supertonic 3、MOSS-TTS-v1.5、dots.tts、Confucius4-TTS)。在 **设置 → TTS 引擎** 中切换;所选引擎将应用于所有语音合成场景。
**16 个引擎,一个选择器。** VoiceStudio(默认,支持 600+ 语言)始终可用;另有七个引擎可选装并自动检测(CosyVoice 3、GPT-SoVITS、VoxCPM2、MOSS-TTS-Nano、KittenTTS、MLX-Audio、Sherpa-ONNX),外加个按需延迟安装的引擎(IndexTTS 2.5、OmniVoice GGUF、OmniVoice 子进程版、PocketTTS、Supertonic 3、MOSS-TTS-v1.5、dots.tts、Confucius4-TTS)。在 **设置 → TTS 引擎** 中切换;所选引擎将应用于所有语音合成场景。**每个引擎都有独立指南:[docs/engines](docs/engines/README.md)(英文)。**
<details>
<summary><b>📊 完整矩阵</b>——14 个引擎 × 平台 × 克隆/指令 × 许可证</summary>
<summary><b>📊 完整矩阵</b>——16 个引擎 × 平台 × 克隆/指令 × 许可证</summary>
<br/>
@@ -254,10 +288,10 @@ Hugging Face Token 的配置见
### 🎧 ASR 引擎
**10 个引擎**——它们驱动听写、视频配音和字幕。**WhisperX** 是跨平台的默认引擎(约 100 种语言,词级时间对齐);其余引擎均为可选装并自动检测。在 **设置 → 引擎** 中切换。个完全在本地设备上运行;第十个(OpenAI 兼容)是可选的远程客户端,可用于 Qwen3-ASR 或任何兼容的服务器。
**11 个引擎**——它们驱动听写、视频配音和字幕。**WhisperX** 是跨平台的默认引擎(约 100 种语言,词级时间对齐);其余引擎均为可选装并自动检测。在 **设置 → 引擎** 中切换。个完全在本地设备上运行;第十个(OpenAI 兼容)是可选的远程客户端,可用于 Qwen3-ASR 或任何兼容的服务器。
<details>
<summary><b>📊 完整阵容</b>——10 个引擎、各自的强项与计算类型说明</summary>
<summary><b>📊 完整阵容</b>——11 个引擎、各自的强项与计算类型说明</summary>
<br/>
@@ -274,7 +308,7 @@ Hugging Face Token 的配置见
| **sherpa-onnx**(实时听写) | `sherpa-onnx-asr` | 25 种欧洲语言 + 90+ | 实时、快于实时的听写——小体积流式/离线 ONNX 模型(Parakeet TDT v3/v2、流式 Zipformer 与 Paraformer、Whisper Tiny),CPU 运行,macOS / Windows / Linux 表现完全一致。在 **设置 → 语音** 中按模型选择。 |
| **OpenAI 兼容** ⚠️ 远程 | `openai-compat-asr` | 取决于服务器 | 当下通往 **Qwen3-ASR** 的路径(自托管服务器,无需等 transformers 支持)、任何 OpenAI 兼容的转录端点,或 OpenAI 官方 API——无需安装,在 **设置 → 引擎**(ASR 标签页)中配置并测试连接。音频会离开你的设备,发送到你指定的任何服务器;参见 [docs/engines/openai-compatible-asr.md](docs/engines/openai-compatible-asr.md)。 |
> Whisper 系列引擎覆盖约 100 种语言;**FunASR / SenseVoice** 额外提供一条多语言一体化路径,内置语音活动检测与行内说话人分离。**sherpa-onnx** 驱动实时听写的模型选择器——你边说,文字边出现。每个引擎都在本地设备上运行——无需 API 密钥,无需云端。
> Whisper 系列引擎覆盖约 100 种语言;**FunASR / SenseVoice** 额外提供一条多语言一体化路径,内置语音活动检测与行内说话人分离。**sherpa-onnx** 驱动实时听写的模型选择器——你边说,文字边出现。除可选的 OpenAI 兼容远程客户端外,所有引擎都在本地设备上运行——无需 API 密钥,无需云端。
> **GPU 不支持高效 float16** 在较老的 NVIDIA GPUMaxwell/Pascal、GTX 16xx)上,或在 CTranslate2/cuDNN 版本不匹配之后,CTranslate2 系 ASR 引擎(WhisperX、Faster-Whisper)无法运行 `float16`VoiceStudio 会自动改用 `int8` 重试——无需配置。如果转录仍然失败,可用 `ASR_COMPUTE_TYPE` 环境变量固定计算类型(逃生舱口):`ASR_COMPUTE_TYPE=int8`CPU 用 `float32`)。将其设为 `int8` 并重启后端。
@@ -331,9 +365,9 @@ print(result.text)
### 📓 在 Google Colab 上运行
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/debpalash/VoiceStudio/blob/main/notebooks/VoiceStudio_Studio_Colab.ipynb)
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/debpalash/VoiceStudio/blob/main/notebooks/OmniVoice_Studio_Colab.ipynb)
没有本地 GPU?官方笔记本([notebooks/VoiceStudio_Studio_Colab.ipynb](notebooks/VoiceStudio_Studio_Colab.ipynb))可在免费的 Colab T4 上启动完整应用(包含 Web 界面):在笔记本内直接构建前端,用 uv 安装后端(复用 Colab 预装的 CUDA PyTorch),并通过 Colab 内置端口代理打开界面。无需第三方隧道,也无需任何 API 密钥。随后还有一套覆盖全部主要功能的 API 导览,全部可在笔记本内直接播放:多语言 TTS、声音克隆与声音设计、已保存的声音档案、语音转写、AI 水印检测、OpenAI 兼容 API、多角色故事、带章节的 m4b 有声书,以及一个附带人声分离音轨的迷你视频配音。
没有本地 GPU?官方笔记本([notebooks/OmniVoice_Studio_Colab.ipynb](notebooks/OmniVoice_Studio_Colab.ipynb))可在免费的 Colab T4 上启动完整应用(包含 Web 界面):在笔记本内直接构建前端,用 uv 安装后端(复用 Colab 预装的 CUDA PyTorch),并通过 Colab 内置端口代理打开界面。无需第三方隧道,也无需任何 API 密钥。随后还有一套覆盖全部主要功能的 API 导览,全部可在笔记本内直接播放:多语言 TTS、声音克隆与声音设计、已保存的声音档案、语音转写、AI 水印检测、OpenAI 兼容 API、多角色故事、带章节的 m4b 有声书,以及一个附带人声分离音轨的迷你视频配音。
### 🤝 智能体技能(Agent Skills
@@ -345,6 +379,36 @@ npx skills add debpalash/omnivoice-studio
内含两个 [skills](https://skills.sh)**`omnivoice`**——让任何智能体通过你的本地安装进行语音合成与转录(包括你克隆的声音),免费且离线;以及 **`oss-maintainer`**——本项目所遵循的维护者方法论,适合任何用智能体运营自己开源项目的人。
### 🔌 模型上下文协议(MCP 服务器)
VoiceStudio 在 `http://localhost:3900/mcp` 挂载了 MCP 服务,可供 Claude Desktop、Cursor 与自主智能体调用:
```json
{
"mcpServers": {
"voicestudio": {
"url": "http://localhost:3900/mcp"
}
}
}
```
对于需要 stdio 管道传输的客户端,请使用内置的本地桥接脚本(`docs/mcp.json`):
```json
{
"mcpServers": {
"voicestudio": {
"command": "python",
"args": ["-m", "backend.mcp_shim"],
"cwd": "/path/to/VoiceStudio"
}
}
}
```
支持 `generate_speech``clone_voice``transcribe` 等工具与流式文件输出模式,详见 [docs/mcp.md](docs/mcp.md)。
---
## 🗺️ 路线图
@@ -373,7 +437,7 @@ npx skills add debpalash/omnivoice-studio
| **TTS** | 14 个引擎(VoiceStudio、CosyVoice 3、GPT-SoVITS、VoxCPM2、MOSS-TTS-Nano、KittenTTS、MLX-Audio、Sherpa-ONNX+ 延迟安装:IndexTTS 2.5、OmniVoice GGUF、Supertonic 3、MOSS-TTS-v1.5、dots.tts、Confucius4-TTS)、带 GPU 预检的引擎路由 |
| **基础设施** | Docker 部署、CUDA/MPS/ROCm 自动检测、cuDNN 8 兼容、显存感知模型卸载、引擎路由(绝不静默回退 CPU)、诊断套件与错误日志、受限网络镜像支持 |
| **AI 溯源** | AudioSeal 不可见水印(类似 SynthID)、视频徽标叠加、水印检测 API |
| **用户体验** | 撤销/重做、键盘快捷键、拖放、会话持久化、毛玻璃设计系统、Linux/WebKitGTK 的 UI 缩放修复 |
| **用户体验** | 撤销/重做、键盘快捷键、拖放、会话持久化、首次启动按屏幕推荐界面缩放,以及原生 WebKitGTK 缩放 |
| **实时事件** | WebSocket 事件总线——数据变更时即时刷新侧边栏、指数退避重连 |
| **状态管理** | Zustand 状态迁移——`uiSlice``pillSlice``dubSlice``generateSlice``prefsSlice``glossarySlice` |
| **桌面** | 跨平台 Tauri 安装程序(macOS DMG——Apple SiliconIntel 不支持本地后端,#889——Windows MSI、Linux deb/AppImage)、自动更新基础设施、单实例约束、关闭最小化到托盘、macOS Gatekeeper 修复 |
@@ -535,6 +599,13 @@ VoiceStudio **免费**且采用 **AGPL-3.0** 许可——没有付费版,没
VoiceStudio 完全本地运行——卸载就是删除应用及其写入的文件夹(模型缓存、Python 环境、你的声音/项目、配置)。运行 <code>scripts/uninstall.sh</code>macOS/Linux)或 <code>scripts\uninstall.ps1</code>Windows)——它会先以干跑方式列出每个文件夹及其大小,加 <code>--yes</code> 才会真正删除。完整的各平台路径列表和应用移除步骤见 <a href="docs/install/uninstall.md"><b>docs/install/uninstall.md</b></a>。
</details>
## 🛡️ 负责任使用与安全
VoiceStudio 在个人硬件上提供零样本语音克隆与语音创作能力。我们提倡负责任的技术使用:
- **明确授权:** 严禁在未经说话人本人知情并明确授权的情况下克隆其声音。
- **AI 溯源:** VoiceStudio 默认集成 [AudioSeal](https://github.com/facebookresearch/audioseal) 不可见神经音频水印,在完全不影响听感音质的前提下精准标记合成语音。
- **本地隐私:** 默认本地工作流下,所有音频、声音档案、项目与转录文本始终保存在你的本地设备上;仅当你主动配置远程工作节点或第三方 ASR 端点时,相应数据才会传输到对应服务。
---
<a id="license"></a>
@@ -574,7 +645,7 @@ VoiceStudio 站在这些杰出开源工作的肩膀上:
## 🧰 来自同一作者的更多本地开源项目
喜欢这种本地优先的理念?它是一脉相承的——同一位作者,同一条准则:**你的数据只留在你的设备上。**
喜欢这种本地优先的理念?它是一脉相承的——同一位作者,同一条准则:**你的数据只留在你的设备上。** 全部项目见 [palash.dev](https://palash.dev)。
<table>
<tr>
+10
View File
@@ -41,6 +41,16 @@ hiddenimports = [
# even though pyproject.toml ships the package. Guarded by
# tests/test_socks_proxy.py.
'socksio',
# Remote GPU workers (backend/worker/). The feature is opt-in, so every
# import of it is deliberately deferred to the moment it is switched on —
# inside `lifespan` and inside `ControlPlane.start()`. That keeps the cost
# off users who never enable it, but it also means a frozen build has no
# static import chain to follow, so the modules must be named here or the
# feature raises ModuleNotFoundError only in the installers.
'grpc', 'grpc.aio',
'worker.service', 'worker.agent',
'worker.transport.server', 'worker.transport.client',
'worker.protocol.gen.worker_v1_pb2', 'worker.protocol.gen.worker_v1_pb2_grpc',
# Core
'uuid', 'asyncio',
+121 -119
View File
@@ -6,7 +6,10 @@ composed at the route or router level without surprises.
Currently exposed:
- `require_loopback`: 403 unless the request came from a loopback origin
(bypassed in explicit server mode see `_server_mode`).
(read-only bootstrap is allowed in explicit server mode; mutations still
require the admin API key see `_server_mode`).
- `require_admin`: method-aware admin gate for privileged routers.
- `require_admin_action`: strict admin gate for side-effectful GET actions.
- `require_native_access`: true-loopback-only access to the host filesystem;
unlike `require_loopback`, it is never bypassed by server mode.
- `ws_remote_authorized`: whether a WebSocket handshake from a non-loopback
@@ -14,64 +17,19 @@ Currently exposed:
keep their own inline loopback guards.
"""
import ipaddress
import os
import secrets
from fastapi import HTTPException, Request
# IPv4 + IPv6 loopback literals + the conventional `localhost` hostname.
# `request.client.host` carries an address, not a hostname, so the literal
# "localhost" entry is defensive — some upstream wrappers (TestClient with
# a custom client tuple, certain reverse-proxy headers) may pass strings
# rather than parsed addresses. We accept the broader set without weakening
# the guard: nothing here matches a non-loopback origin.
_LOOPBACK_HOSTS = frozenset({"127.0.0.1", "::1", "localhost"})
def _trusted_networks():
"""CIDR networks from OMNIVOICE_TRUSTED_NETWORKS (comma-separated) treated as
loopback-trusted e.g. a reverse proxy or self-hosted LAN, so the API-key /
PIN gates don't block LAN clients that can't present the credential (a proxy
that strips the Authorization header). Read at call time (matching
`_server_mode` / `remote_api_key`) so tests can monkeypatch the env; restart
to apply changes in production."""
nets = []
for cidr in os.environ.get("OMNIVOICE_TRUSTED_NETWORKS", "").split(","):
cidr = cidr.strip()
if cidr:
try:
nets.append(ipaddress.ip_network(cidr, strict=False))
except ValueError:
pass # malformed entry ignored — never wedge the auth gate
return nets
def is_loopback(host):
"""True loopback address only (127.0.0.1, ::1, localhost) — NOT a trusted
network. Admin gates (``require_loopback`` ``/system/set-env``,
``/api/settings/*``) use this so a trusted-network CIDR exempts consumption
(TTS / dictation) but never the RCE-class admin surface."""
return host in _LOOPBACK_HOSTS
def is_local_host(host):
"""Loopback address, OR on a configured trusted network. The consumption
gates (PIN/API-key middleware, WS guard) call this so a trusted LAN/proxy is
exempted. Admin gates use :func:`is_loopback` NOT this to preserve the
two-tier privilege model: consumption trust admin trust."""
if is_loopback(host):
return True
try:
ip = ipaddress.ip_address(host)
except (ValueError, TypeError):
return False
# Unwrap IPv4-mapped IPv6 (::ffff:192.168.1.5) so it matches IPv4 CIDRs —
# dual-stack proxies (Caddy, Node.js) frequently pass the mapped form.
if getattr(ip, "ipv4_mapped", None):
ip = ip.ipv4_mapped
return any(ip in net for net in _trusted_networks())
from core.auth import (
CredentialTransport,
PrincipalKind,
is_local_host,
is_loopback,
principal_for,
remote_api_key,
)
from core.csrf import SAFE_HTTP_METHODS, cookie_csrf_allowed
_TRUTHY = frozenset({"1", "true", "yes", "on"})
@@ -96,6 +54,15 @@ def _server_mode() -> bool:
return os.environ.get("OMNIVOICE_SERVER_MODE", "").strip().lower() in _TRUTHY
def validate_server_admin_key() -> None:
"""Reject an explicitly blank key before a server-mode app starts."""
raw_key = os.environ.get("OMNIVOICE_API_KEY")
if _server_mode() and raw_key is not None and not raw_key.strip():
raise RuntimeError(
"OMNIVOICE_API_KEY is blank; configure a non-whitespace administrator key"
)
def _configured_pin(request) -> str | None:
"""The active share PIN (``app.state.network_share.pin``) or None. Read via
getattr so a bare Request stub (or a request that hit before lifespan set
@@ -107,40 +74,45 @@ def _configured_pin(request) -> str | None:
def _admin_credential_configured(request) -> bool:
"""Whether the operator has set ANY credential gate — the remote API key or
a share PIN. When neither is set, server mode leaves admin open (the Docker
issue #261 flow the image depends on)."""
if os.environ.get("OMNIVOICE_API_KEY"):
"""Whether an API key or share PIN is configured.
The PIN cannot authorize admin access, but its presence means the operator
opted out of bare-server discovery. Remote admin then remains closed until
they configure and present the long API key.
"""
if remote_api_key():
return True
return bool(_configured_pin(request))
def _request_presents_admin_credential(request) -> bool:
"""Whether the request carries a valid **API key** via the channels the
middleware accepts (``Authorization: Bearer`` / ``?api_key`` / ``ov_key``
cookie).
def _request_presents_admin_credential(
request,
*,
side_effectful_get: bool = False,
) -> bool:
"""Whether the canonical principal carries remote admin capability.
Admin is RCE-class (``/system/set-env`` + ``/api/settings/*``), so only the
API key a long operator-chosen secret unlocks it. The 6-digit share PIN
is deliberately NOT accepted here: it is a *consumption* credential for LAN
playback and is short enough to brute-force (10^6, no lockout), so it must
never gate the admin surface (CodeRabbit #1213). A trusted-network CIDR
(``is_local_host`` also a consumption exemption) likewise never unlocks
admin. Net: remote admin in server mode requires the API key; a PIN-only
deployment keeps admin loopback-only. getattr-defensive so a minimal Request
stub never raises."""
api_key = os.environ.get("OMNIVOICE_API_KEY") or ""
if not api_key:
API-key and short-lived session principals may unlock server-mode admin.
PIN and trusted-network principals remain consumption-only.
"""
principal = principal_for(request)
if principal.kind not in {
PrincipalKind.API_KEY,
PrincipalKind.ADMIN_SESSION,
}:
return False
headers = getattr(request, "headers", None) or {}
query = getattr(request, "query_params", None) or {}
cookies = getattr(request, "cookies", None) or {}
auth = headers.get("authorization", "")
supplied = auth[7:].strip() if auth.lower().startswith("bearer ") else ""
if not supplied:
supplied = query.get("api_key") or cookies.get("ov_key") or ""
return bool(supplied and secrets.compare_digest(supplied, api_key))
if principal.transport not in {
CredentialTransport.COOKIE,
CredentialTransport.LEGACY_COOKIE,
}:
return True
method = str(getattr(request, "method", "GET")).upper()
if side_effectful_get or method not in SAFE_HTTP_METHODS:
return cookie_csrf_allowed(
request,
side_effectful_get=side_effectful_get,
)
return True
def require_loopback(request: Request) -> None:
@@ -163,9 +135,9 @@ def require_loopback(request: Request) -> None:
unenforceable, so the gate can't require true loopback. It then applies the
admin-credential rule instead:
- No credential configured (no API key, no PIN) open, matching the #261
Docker flow where the operator reaches ``/system/*`` off the bridge
gateway with nothing set.
- No credential configured (no API key, no PIN) read-only requests are
open, matching the #261 Docker bootstrap flow. State-changing requests
fail closed even if a route accidentally kept this legacy dependency.
- A credential IS configured the request must present the **API key**.
This keeps the two-tier privilege model intact under server mode:
``OMNIVOICE_TRUSTED_NETWORKS`` is a *consumption* exemption
@@ -173,14 +145,20 @@ def require_loopback(request: Request) -> None:
NEVER by itself unlock the admin surface (``/system/set-env`` RCE-class
and ``/api/settings/*``). The 6-digit share PIN is a consumption credential
too and does not gate admin, so a PIN-only deployment keeps admin
loopback-only; remote admin requires the (long) API key. A LAN client in a
trusted CIDR or one holding only the PIN gets 403 here even though it
sails through the consumption gates. See docs/api-auth.md (#1213).
loopback-only; remote admin requires the long API key. See
docs/api-auth.md (#1213).
"""
host = request.client.host if request.client else None
if is_loopback(host):
return
if _server_mode():
method = str(getattr(request, "method", "GET")).upper()
if method not in SAFE_HTTP_METHODS:
# Defense in depth. Privileged routers should declare
# ``require_admin`` directly, but a missed migration must not turn
# into an unauthenticated Docker write primitive.
require_admin(request)
return
if not _admin_credential_configured(request):
return
if _request_presents_admin_credential(request):
@@ -188,6 +166,31 @@ def require_loopback(request: Request) -> None:
raise HTTPException(status_code=403, detail="loopback origin required")
def _admin_gate_403() -> None:
"""Raise the admin-gate 403 with a detail that states what would ACTUALLY
satisfy the gate. The bundled UI routes any 403 whose detail mentions
"admin api key" to the API-key login form (frontend ``client.ts``; the
literal contract is locked by ``tests/test_auth_gate_detail_lockstep.py``),
so the wording must not name a key where presenting one cannot help.
The detail names the key only when the gate would accept one: server mode
WITH an API key configured. Every other rejection desktop mode (the
credential checks in the callers only run under server mode) and a
server-mode deployment with only a share PIN or nothing configured keeps
the plain loopback detail, because only loopback can use admin there.
Naming the key in those cases would trap a LAN-share guest in a login
form that can never succeed (#1213, #1525; PR #1569 review).
"""
raise HTTPException(
status_code=403,
detail=(
"loopback origin or admin API key required"
if _server_mode() and remote_api_key()
else "loopback origin required"
),
)
def require_admin(request: Request) -> None:
"""Gate RCE/filesystem-capable admin routers.
@@ -206,12 +209,30 @@ def require_admin(request: Request) -> None:
return
if _server_mode():
method = str(getattr(request, "method", "GET")).upper()
read_only = method in {"GET", "HEAD", "OPTIONS"}
if read_only and not os.environ.get("OMNIVOICE_API_KEY", "").strip():
read_only = method in SAFE_HTTP_METHODS
if read_only and not _admin_credential_configured(request):
return
if _request_presents_admin_credential(request):
return
raise HTTPException(status_code=403, detail="loopback origin or admin API key required")
_admin_gate_403()
def require_admin_action(request: Request) -> None:
"""Gate an administrative action even when its HTTP method is read-only.
A small number of legacy GET endpoints have real side effects. For example,
an engine health check may spawn a sidecar process. Such routes cannot use
:func:`require_admin`'s bare-server discovery exception.
"""
host = request.client.host if request.client else None
if is_loopback(host):
return
if _server_mode() and _request_presents_admin_credential(
request,
side_effectful_get=True,
):
return
_admin_gate_403()
def require_desktop(request: Request) -> None:
@@ -232,9 +253,10 @@ def require_local(request: Request) -> None:
trusted network. The consumption-tier companion to :func:`require_loopback`:
use on routes a trusted-network client (LAN/proxy) should reach without a PIN
or API key e.g. the dictation model/prefs endpoints that pair with the
dictation WebSocket. Admin routes stay on :func:`require_loopback`.
dictation WebSocket. Admin routes stay on :func:`require_admin`.
In server mode the gate is a no-op (same as :func:`require_loopback`)."""
In server mode this consumption gate is a no-op. Admin dependencies remain
method-aware and independent from this exemption."""
host = request.client.host if request.client else None
if is_local_host(host):
return
@@ -256,29 +278,9 @@ def require_native_access(request: Request) -> None:
raise HTTPException(status_code=403, detail="native filesystem access requires loopback origin")
def remote_api_key() -> str | None:
"""The remote-backend bearer key (Wave 2.3), or None when remote mode is
off. Read at call time so tests can monkeypatch the env."""
return os.environ.get("OMNIVOICE_API_KEY") or None
def ws_remote_authorized(websocket) -> bool:
"""Whether a WebSocket handshake presents the remote API key.
Browser WebSockets cannot set an Authorization header, so the key may
arrive as ``?api_key=`` or via the ``ov_key`` cookie that the bearer
middleware sets on the first authenticated HTTP request. Returns False
when remote mode is off callers keep their loopback-only behavior.
"""
key = remote_api_key()
if not key:
return False
auth = websocket.headers.get("authorization", "")
supplied = auth[7:].strip() if auth.lower().startswith("bearer ") else ""
if not supplied:
supplied = (
websocket.query_params.get("api_key")
or websocket.cookies.get("ov_key")
or ""
)
return secrets.compare_digest(supplied, key)
"""Whether the canonical WS principal has a remote admin credential."""
return principal_for(websocket).kind in {
PrincipalKind.API_KEY,
PrincipalKind.ADMIN_SESSION,
}
+7
View File
@@ -49,6 +49,13 @@ def public_backends(entries: list[dict]) -> list[dict]:
item["routing_reason"] = _public_routing_reason(
item.get("routing_status"), item["routing_reason"]
)
evidence = item.get("execution_evidence")
if isinstance(evidence, dict) and evidence.get("cpu_fallback_reason") is not None:
evidence = dict(evidence)
evidence["cpu_fallback_reason"] = _public_routing_reason(
"cpu_fallback", evidence["cpu_fallback_reason"]
)
item["execution_evidence"] = evidence
safe.append(item)
return safe
+400 -65
View File
@@ -15,22 +15,33 @@ Design notes
* Previews are cached on disk keyed by a hash of (instruct, language), so two
archetypes that resolve to the same voice share a cache file and the cold
render only happens once per distinct voice.
* That same key names the pre-rendered clips in the opt-in voice gallery
(``services.gallery``), which is consulted BEFORE the engine so a fresh
install can hear voices before the 2.4 GB checkpoint finishes downloading.
Gallery files win over a local render of the same key but only for
``/preview``. ``/use`` always renders locally: the WAV it keeps in
``VOICES_DIR`` is the reference audio a cloned voice is built from, and a
downloaded MP3 must never become that.
"""
from __future__ import annotations
import hashlib
import json
import logging
import os
import re
import time
import uuid
from pathlib import Path
from typing import Optional
from fastapi import APIRouter, HTTPException, Query
from fastapi import APIRouter, Body, HTTPException, Query
from fastapi.responses import FileResponse
from core import archetypes
from core.audio_validation import is_playable_wav, resolve_regular_file
from core.config import OUTPUTS_DIR, VOICES_DIR
from services import gallery
logger = logging.getLogger("omnivoice.archetypes")
@@ -61,6 +72,153 @@ def _preview_key(a: dict) -> str:
).hexdigest()[:16]
def _design_profile_values(a: dict) -> tuple[str, str]:
"""Canonical instruct + complete picker state for a designed archetype."""
return a["instruct"], json.dumps(a["attrs"], sort_keys=True)
def _profile_audio_path(ref_audio_path: object) -> Optional[Path]:
"""Resolve only a regular, non-symlinked file inside ``VOICES_DIR``."""
return resolve_regular_file(VOICES_DIR, ref_audio_path)
def _materialized_audio_is_current(row, a: dict) -> bool:
"""Whether an existing row still has the sample described by its metadata."""
expected_filename = _profile_audio_filename(row["id"])
path = _profile_audio_path(row["ref_audio_path"])
return bool(
row["ref_audio_path"] == expected_filename
and is_playable_wav(path)
and row["instruct"] == a["instruct"]
and row["language"] == a["language"]
and row["ref_text"] == a["sample_script"]
and row["seed"] == _PREVIEW_SEED
)
def _profile_audio_filename(profile_id: str) -> str:
safe_id = (
profile_id if re.fullmatch(r"[A-Za-z0-9_-]{1,64}", profile_id or "")
else hashlib.sha256(str(profile_id).encode("utf-8")).hexdigest()[:16]
)
return f"{safe_id}.wav"
def _archetype_personality(a: dict) -> str:
return f"archetype:{a['id']}"
def _legacy_archetype_profile(conn, a: dict):
"""Adopt only a row that an older archetype materializer could have made."""
row = conn.execute(
"SELECT * FROM voice_profiles WHERE personality=? LIMIT 1",
(a["id"],),
).fetchone()
if row is None:
return None
expected_audio = _profile_audio_filename(row["id"])
try:
states_match = (
not row["vd_states"] or json.loads(row["vd_states"]) == a["attrs"]
)
except (TypeError, ValueError):
states_match = False
if (
row["ref_audio_path"] == expected_audio
and row["instruct"] == a["instruct"]
and row["language"] == a["language"]
and row["ref_text"] == a["sample_script"]
and row["seed"] == _PREVIEW_SEED
and row["kind"] in (None, "", "clone", "design")
and not row["is_locked"]
and not row["verified_own_voice"]
and states_match
):
return row
return None
def _is_materialized_archetype_row(row, a: dict) -> bool:
"""Recognize rows owned by this materializer without trusting identity text alone."""
try:
states_match = json.loads(row["vd_states"]) == a["attrs"]
except (TypeError, ValueError):
return False
return bool(
row["personality"] == _archetype_personality(a)
and row["kind"] == "design"
and row["seed"] == _PREVIEW_SEED
and row["ref_audio_path"] == _profile_audio_filename(row["id"])
and row["instruct"] == a["instruct"]
and row["language"] == a["language"]
and row["ref_text"] == a["sample_script"]
and states_match
and not row["is_locked"]
and not row["verified_own_voice"]
)
def _existing_archetype_profile(conn, a: dict):
rows = conn.execute(
"SELECT * FROM voice_profiles WHERE personality=? ORDER BY created_at, id",
(_archetype_personality(a),),
).fetchall()
owned = next((row for row in rows if _is_materialized_archetype_row(row, a)), None)
return owned if owned is not None else _legacy_archetype_profile(conn, a)
async def _render_profile_audio(
a: dict, profile_id: str, *, publish: bool = True,
) -> tuple[str, Path]:
"""Render one validated sample, optionally staging it for a later CAS."""
audio_filename = _profile_audio_filename(profile_id)
safe_id = Path(audio_filename).stem
audio_path = Path(VOICES_DIR) / audio_filename
if publish:
await _render_wav_atomic(a, audio_path, prefix=f".{safe_id}-")
else:
audio_path.parent.mkdir(parents=True, exist_ok=True)
audio_path = audio_path.parent / f".{safe_id}-{uuid.uuid4().hex}.staged.wav"
try:
await _render_archetype_wav(a, audio_path)
if not is_playable_wav(audio_path):
raise RuntimeError("the voice engine produced an invalid WAV")
except BaseException:
with __import__("contextlib").suppress(OSError):
audio_path.unlink()
raise
return audio_filename, audio_path
async def _render_wav_atomic(a: dict, out_path: Path, *, prefix: str = ".render-") -> Path:
"""Render and validate a WAV before atomically replacing *out_path*."""
audio_path = Path(out_path)
audio_path.parent.mkdir(parents=True, exist_ok=True)
tmp_path = audio_path.parent / f"{prefix}{uuid.uuid4().hex}.wav"
try:
await _render_archetype_wav(a, tmp_path)
if not is_playable_wav(tmp_path):
raise RuntimeError("the voice engine produced an invalid WAV")
os.replace(tmp_path, audio_path)
finally:
with __import__("contextlib").suppress(OSError):
tmp_path.unlink()
return audio_path
def _heal_materialized_profile(conn, row, a: dict, audio_filename: str) -> None:
"""Repair profiles created before archetype `/use` persisted design kind."""
instruct, vd_states = _design_profile_values(a)
conn.execute(
"UPDATE voice_profiles SET kind='design', instruct=?, vd_states=?, language=?, "
"ref_text=?, seed=?, ref_audio_path=?, personality=? WHERE id=?",
(
instruct, vd_states, a["language"], a["sample_script"], _PREVIEW_SEED,
audio_filename, _archetype_personality(a), row["id"],
),
)
# A non-empty script is always required — synthesizing empty text yields
# silence. Every archetype carries a use-case script, but guard the render path
# too so a malformed archetype can never drive a blank render.
@@ -172,7 +330,7 @@ async def _render_archetype_wav(a: dict, out_path: Path) -> None:
# GPU pool and brick the backend (#730 class). Budget comes from the shared
# length-scaled helper (#1190) instead of the flat 300s default.
from services.model_manager import generate_timeout_s
_budget = generate_timeout_s(text)
_budget = generate_timeout_s(text, engine=model)
audio_tensor = await run_on_gpu_pool_guarded(
lambda: _infer(_PREVIEW_SEED), what="Archetype preview generate",
timeout=_budget)
@@ -199,21 +357,60 @@ async def _render_archetype_wav(a: dict, out_path: Path) -> None:
# Runs on the dedicated watermark pool (#1190): AudioSeal embedding is CPU
# work that holds no VRAM, so it must not occupy a GPU worker ahead of the
# next generate on 1-worker hosts.
from services.watermark import mark_synthetic
from services.model_manager import get_watermark_pool
import functools
audio_tensor = await run_on_gpu_pool_guarded(
functools.partial(mark_synthetic, audio_tensor, model.sampling_rate,
context="archetypes.render"),
what="Archetype watermark",
from services.watermark import mark_synthetic_async
audio_tensor = await mark_synthetic_async(
audio_tensor, model.sampling_rate,
context="archetypes.render",
timeout=generate_timeout_s(""),
executor=get_watermark_pool(),
)
out_path.parent.mkdir(parents=True, exist_ok=True)
_safe_torchaudio_save(str(out_path), audio_tensor, model.sampling_rate)
def _no_voice_model_downloaded() -> bool:
"""True only on a *positive* "no TTS weights on this machine" answer.
Fails open on purpose: the cache probes are best-effort (a user-managed
clone outside the HF layout is invisible to them), and telling someone with
a working engine to go download a model is worse than saying nothing. Only
a catalog we could read, with not one TTS repo cached, earns the offline
message.
"""
try:
from api.routers.setup.models import get_model_catalog, is_cached
tts = [m for m in get_model_catalog().all if m.get("role") == "TTS"]
return bool(tts) and not any(is_cached(m["repo_id"]) for m in tts)
except Exception:
return False
def _preview_source(a: dict) -> tuple[str, str]:
"""Which path ``/preview`` will take for *a*, and what to tell the user.
Replaces the old "see Settings → Logs → Backend" advice, which asked a user
who wanted to hear a voice to go read a log file. The three states that
actually differ are: we already have the audio (gallery), we can make it
(render say so, it takes a moment), and we can neither fetch nor make it
(no model the one state with an action attached).
"""
key = _preview_key(a)
if gallery.cached_preview(key) is not None:
return "gallery", (
"Pre-rendered preview from the voice gallery — a fixed reference "
"rendering, not a render from your current engine."
)
if is_playable_wav(_PREVIEW_DIR / f"{key}.wav"):
return "cached", ""
if _no_voice_model_downloaded():
return "no_model", (
"You're offline and no voice model is downloaded yet — "
"Model Catalogue → Models → Download."
)
return "rendering", "Rendering this preview on your machine — it may take a moment."
# ── Read endpoints (no model) ─────────────────────────────────────────────────
# NOTE: declare the literal `/archetypes/categories` before `/archetypes/{id}`
# so it isn't swallowed by the path-parameter route.
@@ -223,6 +420,37 @@ def list_categories():
return archetypes.categories()
# ── Voice-gallery (pre-rendered previews) ─────────────────────────────────────
# Declared above `/archetypes/{archetype_id}` for the same reason as
# `/categories`: keep literal paths out of the path-parameter route's reach.
@router.get("/archetypes/previews/status")
def preview_gallery_status():
"""Consent state, coverage and freshness for the Settings line."""
return gallery.status()
@router.put("/archetypes/previews")
async def set_preview_gallery(enabled: bool = Body(..., embed=True)):
"""Turn pre-rendered previews on or off.
Turning it ON is the user's explicit yes to an outbound call, and is the
only thing that ever starts one there is no on-install background fetch.
The featured set is pulled right here so the yes has a visible effect;
failures are silent by design (``fetch_featured`` swallows them) and leave
previews rendering locally.
"""
state = gallery.set_enabled(enabled)
if enabled:
state = await gallery.fetch_featured()
return state
@router.post("/archetypes/previews/check")
async def check_preview_gallery():
"""Manual "check now" — bypasses the 24 h throttle, never the signature."""
return await gallery.check_for_updates(force=True)
@router.get("/archetypes")
def list_archetypes_endpoint(
q: Optional[str] = None,
@@ -262,33 +490,77 @@ def get_archetype_endpoint(archetype_id: str):
# ── Render endpoints (model-gated) ────────────────────────────────────────────
@router.get("/archetypes/{archetype_id}/preview/state")
def preview_archetype_state(archetype_id: str):
"""Where the next ``/preview`` for this archetype would come from.
Touches neither the model nor the network, so a picker can label a voice
("may take a moment", "download a model first") *before* it commits to a
request that may take 40 seconds or fail.
"""
a = archetypes.get_archetype(archetype_id)
if a is None:
raise HTTPException(status_code=404, detail="Archetype not found")
source, message = _preview_source(a)
return {"source": source, "message": message}
@router.get("/archetypes/{archetype_id}/preview")
async def preview_archetype(archetype_id: str):
"""Serve a short preview clip — pre-rendered if cached, else render once."""
async def preview_archetype(
archetype_id: str,
local: bool = Query(False, description="Bypass gallery audio after a client decode failure"),
):
"""Serve a short preview clip — from the gallery, the cache, or the engine."""
a = archetypes.get_archetype(archetype_id)
if a is None:
raise HTTPException(status_code=404, detail="Archetype not found")
cache_path = _PREVIEW_DIR / f"{_preview_key(a)}.wav"
if not cache_path.exists():
key = _preview_key(a)
# Gallery first, and only for /preview: these bytes are audio we can prove
# the provenance of, so they beat a local render of the same key. A miss
# (offline, disabled, key not published) is silent — we just render.
gallery_path = None if local else gallery.cached_preview(key)
if gallery_path is None and not local:
gallery_path = await gallery.fetch_preview(key)
if gallery_path is not None:
# Nothing else in the app polls, so the daily refresh hangs off the
# request that proves previews are being used. Fire-and-forget.
gallery.maybe_refresh_in_background()
return FileResponse(
str(gallery_path),
media_type="audio/mpeg",
headers={"Cache-Control": "no-cache",
"X-OmniVoice-Preview-Source": "gallery"},
)
cache_path = _PREVIEW_DIR / f"{key}.wav"
if not is_playable_wav(cache_path):
try:
await _render_archetype_wav(a, cache_path)
await _render_wav_atomic(a, cache_path, prefix=".preview-")
except Exception as e: # model missing / OOM / inference failure
logger.error("Archetype preview render failed", exc_info=True)
raise HTTPException(
status_code=503,
detail=(
"Couldn't render a preview right now — the voice engine is "
f"unavailable. See Settings → Logs → Backend. Error: {e}"
),
)
# Two different failures, two different answers. Without a model
# there is nothing to read in a log — there is something to do.
if _no_voice_model_downloaded():
detail = (
"You're offline and no voice model is downloaded yet — "
"Model Catalogue → Models → Download. (Or turn on pre-rendered "
"voice previews in Model Catalogue → Models.)"
)
else:
detail = (
"Couldn't render a preview right now — the voice engine "
f"reported: {e}"
)
raise HTTPException(status_code=503, detail=detail)
# no-cache (not no-store): the URL is stable but its bytes change when an
# archetype's preview is re-rendered, so force the client to revalidate
# against the ETag instead of serving a stale cached clip indefinitely.
return FileResponse(
str(cache_path),
media_type="audio/wav",
headers={"Cache-Control": "no-cache"},
headers={"Cache-Control": "no-cache",
"X-OmniVoice-Preview-Source": "local"},
)
@@ -300,6 +572,11 @@ async def use_archetype(archetype_id: str, name: Optional[str] = Query(None)):
preview) and inserts a ``voice_profiles`` row carrying the archetype's
instruct + language. The profile then shows up everywhere voices are
picked (Dub / Generate / Clone).
Never sourced from the voice gallery, no matter how cheap that would be:
this WAV lands in ``VOICES_DIR`` as the profile's reference audio, so a
downloaded, lossily-encoded MP3 would silently become the sample every
future clone of this voice is built from. It renders locally or it fails.
"""
a = archetypes.get_archetype(archetype_id)
if a is None:
@@ -311,64 +588,122 @@ async def use_archetype(archetype_id: str, name: Optional[str] = Query(None)):
# Idempotent (dedup): an archetype materializes to exactly ONE voice profile.
# Picking the same gallery voice again — from any picker (Gallery grid,
# VoiceSelector, …) — must reuse that one row instead of rendering + inserting
# a fresh duplicate every time. The `personality` column already carries the
# source archetype id (stamped by the INSERT below), so it's the natural
# dedup key; the expensive render + INSERT only run on first use.
# a fresh duplicate every time. Use a namespaced personality identity so an
# imported persona cannot collide with and be rewritten by an archetype id.
with db_conn() as conn:
existing = conn.execute(
"SELECT id, name FROM voice_profiles WHERE personality = ? LIMIT 1",
(a["id"],),
).fetchone()
existing = _existing_archetype_profile(conn, a)
profile_id = existing["id"] if existing is not None else str(uuid.uuid4())[:8]
audio_path: Optional[Path] = None
if existing is not None and _materialized_audio_is_current(existing, a):
audio_filename = existing["ref_audio_path"]
else:
try:
audio_filename, audio_path = await _render_profile_audio(
a, profile_id, publish=existing is None,
)
except Exception as e:
logger.error("Archetype 'use' render failed", exc_info=True)
# Same actionable/diagnostic split as /preview — minus the gallery
# suggestion, which cannot help here.
if _no_voice_model_downloaded():
detail = (
"Creating a voice needs the voice model — no voice model is "
"downloaded yet. Model Catalogue → Models → Download."
)
else:
detail = (
"Couldn't create a voice from this archetype — the voice engine "
f"reported: {e}"
)
raise HTTPException(status_code=503, detail=detail) from e
if existing is not None:
return {"profile_id": existing["id"], "name": existing["name"]}
with db_conn() as conn:
conn.execute("BEGIN IMMEDIATE")
current = conn.execute(
"SELECT * FROM voice_profiles WHERE id=?", (existing["id"],),
).fetchone()
owned = _existing_archetype_profile(conn, a)
still_owned = current is not None and (
owned is not None and owned["id"] == current["id"]
)
if still_owned:
if audio_path is not None:
destination = Path(VOICES_DIR) / audio_filename
os.replace(audio_path, destination)
audio_path = None
_heal_materialized_profile(conn, current, a, audio_filename)
existing_result = {"profile_id": current["id"], "name": current["name"]}
else:
existing_result = None
if existing_result is not None:
event_bus.emit("profiles", {"action": "updated", "id": existing_result["profile_id"]})
return existing_result
# The row was edited/deleted while rendering. Preserve it and use the
# validated staged sample for a fresh canonical materialization.
profile_id = str(uuid.uuid4())[:8]
audio_filename = _profile_audio_filename(profile_id)
destination = Path(VOICES_DIR) / audio_filename
if audio_path is None:
try:
audio_filename, audio_path = await _render_profile_audio(a, profile_id)
except Exception as e:
raise HTTPException(
status_code=503, detail="Couldn't create a voice from this archetype.",
) from e
else:
os.replace(audio_path, destination)
audio_path = destination
profile_id = str(uuid.uuid4())[:8]
audio_filename = f"{profile_id}.wav"
audio_path = Path(VOICES_DIR) / audio_filename
try:
await _render_archetype_wav(a, audio_path)
except Exception as e:
logger.error("Archetype 'use' render failed", exc_info=True)
raise HTTPException(
status_code=503,
detail=(
"Couldn't create a voice from this archetype — the voice engine "
f"is unavailable. See Settings → Logs → Backend. Error: {e}"
),
)
if audio_path is None: # defensive: a new profile always rendered above
raise RuntimeError("new archetype profile has no rendered audio")
profile_name = (name or a["name"]).strip() or a["name"]
try:
with db_conn() as conn:
conn.execute("BEGIN IMMEDIATE")
# Re-check under the write connection right before inserting: a
# concurrent /use for the same archetype may have inserted while we
# were rendering (the pre-render SELECT above raced). Reuse that row
# and drop our just-rendered sample instead of creating a duplicate.
# (personality is NOT globally unique — marketplace/persona imports
# reuse the column — so a UNIQUE index isn't an option; this closes
# the realistic window for the single-user desktop app.)
dup = conn.execute(
"SELECT id, name FROM voice_profiles WHERE personality = ? LIMIT 1",
(a["id"],),
).fetchone()
# `personality` is not globally UNIQUE, so serialize and re-check.
dup = _existing_archetype_profile(conn, a)
if dup is not None:
duplicate_audio = dup["ref_audio_path"]
if not _materialized_audio_is_current(dup, a):
duplicate_audio = _profile_audio_filename(dup["id"])
_duplicate_path = Path(VOICES_DIR) / duplicate_audio
_duplicate_path.parent.mkdir(parents=True, exist_ok=True)
os.replace(audio_path, _duplicate_path)
audio_path = None
_heal_materialized_profile(conn, dup, a, duplicate_audio)
with __import__("contextlib").suppress(OSError):
os.remove(audio_path)
return {"profile_id": dup["id"], "name": dup["name"]}
conn.execute(
"INSERT INTO voice_profiles "
"(id, name, ref_audio_path, ref_text, instruct, language, seed, personality, created_at) "
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)",
(
profile_id, profile_name, audio_filename, a["sample_script"],
a["instruct"], a["language"], _PREVIEW_SEED, a["id"], time.time(),
),
)
if audio_path is not None:
os.remove(audio_path)
duplicate_result = {"profile_id": dup["id"], "name": dup["name"]}
else:
duplicate_result = None
if duplicate_result is None:
instruct, vd_states = _design_profile_values(a)
conn.execute(
"INSERT INTO voice_profiles "
"(id, name, ref_audio_path, ref_text, instruct, language, seed, personality, "
"created_at, kind, vd_states) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, 'design', ?)",
(
profile_id, profile_name, audio_filename, a["sample_script"],
instruct, a["language"], _PREVIEW_SEED,
_archetype_personality(a), time.time(), vd_states,
),
)
except Exception:
with __import__("contextlib").suppress(OSError):
os.remove(audio_path)
if audio_path is not None:
os.remove(audio_path)
raise
if duplicate_result is not None:
event_bus.emit("profiles", {"action": "updated", "id": duplicate_result["profile_id"]})
return duplicate_result
event_bus.emit("profiles", {"action": "created", "id": profile_id})
return {"profile_id": profile_id, "name": profile_name}
+137 -30
View File
@@ -25,6 +25,7 @@ import json
import logging
import os
import re
import shutil
import uuid
from collections.abc import Awaitable, Callable
@@ -360,7 +361,7 @@ LONGFORM_NUM_STEP = 32
LONGFORM_GUIDANCE_SCALE = 2.0
def _seed_segment_rng(base_seed, text: str, nonce: int = 0) -> None:
def _seed_segment_rng(base_seed, text: str, nonce: int = 0) -> int | None:
"""Apply a profile's pinned seed to this synth call (#1139).
``_resolve_voice`` has always fetched the profile ``seed`` but only the
@@ -379,11 +380,13 @@ def _seed_segment_rng(base_seed, text: str, nonce: int = 0) -> None:
must cover /generate and here together, not one path.
"""
if base_seed is None:
return
return None
import torch
from services.audiobook import segment_seed
torch.manual_seed(segment_seed(base_seed, text, nonce))
seed = segment_seed(base_seed, text, nonce)
torch.manual_seed(seed)
return seed
def _base_seed(opts: ExpressiveOptions, voice: dict):
@@ -507,16 +510,21 @@ def _build_synth(
"get_model": get_model, "language": language, "opts": opts}
backend = cls()
extra = _generic_extra_kwargs(opts)
native_proxy = bool(getattr(cls, "supports_native_omnivoice_controls", False))
extra = (_omnivoice_sampling_kwargs(opts) if native_proxy
else _generic_extra_kwargs(opts))
next_nonce = _make_occ_counter(opts)
def synth(text, voice_id, speed=None):
v = resolve(voice_id)
_seed_segment_rng(_base_seed(opts, v), text, next_nonce())
seed = _seed_segment_rng(_base_seed(opts, v), text, next_nonce())
call_extra = dict(extra)
if native_proxy and seed is not None:
call_extra["seed"] = seed
return backend.generate(
text, language=language, ref_audio=v["ref_audio"],
ref_text=v["ref_text"], instruct=v["instruct"], duration=None,
speed=float(speed) if speed else 1.0, **extra,
speed=float(speed) if speed else 1.0, **call_extra,
)
return {"mode": "generic", "resolve": resolve, "engine_id": engine_id,
"synth": synth, "sample_rate": backend.sample_rate}
@@ -681,6 +689,87 @@ def _render_chapter_cached(chapter, synth, sr, engine_id, resolve, cache_dir, le
"cached": seg_cache.hits}
def _remote_chapter_call(chapter, *, engine_id, default_voice, voice_map,
language, lexicon, opts, cache_dir):
"""Build one opaque remote chapter task without loading a local TTS model."""
import hashlib
from services import gpu_gateway
from services.text_normalization import normalize_for_tts
from services.watermark import is_enabled as watermark_enabled
rows, voices, refs = [], [], []
for span in chapter.spans:
profile_id = _map_span_voice(span.voice_id, default_voice, voice_map)
voice = _resolve_voice(profile_id)
rows.append({
"text": normalize_for_tts(span.text, language),
"pause_ms_after": span.pause_ms_after,
"speed": getattr(span, "speed", None),
})
refs.append(voice.get("ref_audio"))
voices.append({
"ref_text": voice.get("ref_text"), "instruct": voice.get("instruct"),
"seed": voice.get("seed"),
})
params = {
"spans": rows, "voices": voices, "ref_audio": refs,
"language": language, "lexicon": lexicon,
"expressive": opts.to_manifest(), "watermark": bool(watermark_enabled()),
}
signature = hashlib.sha256(json.dumps(params, sort_keys=True, default=str).encode()).hexdigest()
wav_path = os.path.join(cache_dir, f"remote-{signature}.wav")
def decode(result):
import soundfile as sf
if not os.path.exists(wav_path):
partial = f"{wav_path}.part"
shutil.copyfile(result.path, partial)
os.replace(partial, wav_path)
info = sf.info(wav_path)
return wav_path, float(info.duration), False, None
return gpu_gateway.RemoteCall(
engine=engine_id, operation="audiobook", params=params,
idempotency_key=f"audiobook:{signature}", decode=decode,
), wav_path
async def _run_chapter(chapter, *, operation="audiobook", decision, job, default_voice, language, opts,
voice_map, lexicon, cache_dir):
"""Run one chapter through the gateway; local preparation stays lazy."""
from services import gpu_gateway
from services.tts_backend import active_backend_id
engine_id = active_backend_id()
remote, remote_cache = _remote_chapter_call(
chapter, engine_id=engine_id, default_voice=default_voice,
voice_map=voice_map, language=language, lexicon=lexicon,
opts=opts, cache_dir=cache_dir,
)
if decision.remote and os.path.exists(remote_cache):
import soundfile as sf
info = sf.info(remote_cache)
return remote_cache, float(info.duration), True, None
async def prepare_local():
synth, sr, resolve, local_engine = await _prepare_synth(
default_voice, language=language, opts=opts, voice_map=voice_map
)
return gpu_gateway.LocalCall(
fn=lambda: _render_chapter_cached(
chapter, synth, sr, local_engine, resolve, cache_dir, lexicon,
language, opts, voice_map,
),
what="Audiobook chapter",
)
return await gpu_gateway.run(
operation, local=gpu_gateway.LocalCall(prepare=prepare_local),
remote=remote, decision=decision, job=job,
)
class AudiobookPreviewRequest(ExpressiveMixin):
text: str
chapter_index: int = 0
@@ -700,7 +789,7 @@ async def audiobook_preview(req: AudiobookPreviewRequest) -> dict:
cache (the later full render reuses it) and a re-preview is instant.
"""
from core.config import OUTPUTS_DIR
from services.model_manager import _gpu_pool
from services import gpu_gateway
plan = parse_audiobook_script(req.text, default_voice=req.default_voice)
if not plan.chapters:
@@ -714,16 +803,11 @@ async def audiobook_preview(req: AudiobookPreviewRequest) -> dict:
os.makedirs(cache_dir, exist_ok=True)
resolved_lang = _resolve_default_language(req.language, req.default_voice)
opts = _expressive_opts(req)
synth, sr, resolve, engine_id = await _prepare_synth(
req.default_voice,
language=resolved_lang,
opts=opts,
voice_map=req.voice_map,
)
loop = asyncio.get_running_loop()
wav_path, dur, was_cached, _seg_stats = await loop.run_in_executor(
_gpu_pool, _render_chapter_cached, chapter, synth, sr, engine_id, resolve, cache_dir,
req.lexicon, resolved_lang, opts, req.voice_map,
decision = gpu_gateway.decide("audiobook")
wav_path, dur, was_cached, _seg_stats = await _run_chapter(
chapter, decision=decision, job=None, default_voice=req.default_voice,
language=resolved_lang, opts=opts, voice_map=req.voice_map,
lexicon=req.lexicon, cache_dir=cache_dir,
)
return {
"output": os.path.relpath(wav_path, OUTPUTS_DIR), # served via /audio
@@ -762,7 +846,7 @@ async def _render_longform_sse(
from core.config import OUTPUTS_DIR
from core.failure import build_failure, build_failure_event
from services.ffmpeg_utils import find_ffmpeg, run_ffmpeg
from services.model_manager import _gpu_pool
from services import gpu_gateway
opts = opts or ExpressiveOptions()
@@ -837,13 +921,11 @@ async def _render_longform_sse(
cache_dir = os.path.join(OUTPUTS_DIR, "longform_cache")
os.makedirs(cache_dir, exist_ok=True)
prune_cache_dir(cache_dir) # bound disk before this job adds its chapters
loop = asyncio.get_running_loop()
try:
resolved_lang = _resolve_default_language(language, default_voice)
synth, sr, resolve, engine_id = await _prepare_synth(
default_voice, language=resolved_lang, opts=opts, voice_map=voice_map
)
operation = "audiobook" if job_type == "audiobook" else "longform"
decision = gpu_gateway.decide(operation)
chapter_run = gpu_gateway.JobRun(operation)
total = len(plan.chapters)
chapter_files: list[str] = []
@@ -877,10 +959,11 @@ async def _render_longform_sse(
interrupted = True
break
try:
wav_path, dur, was_cached, seg_stats = await loop.run_in_executor(
_gpu_pool, _render_chapter_cached,
chapter, synth, sr, engine_id, resolve, cache_dir, lexicon,
resolved_lang, opts, voice_map,
wav_path, dur, was_cached, seg_stats = await _run_chapter(
chapter, operation=operation, decision=decision, job=chapter_run,
default_voice=default_voice, language=resolved_lang,
opts=opts, voice_map=voice_map, lexicon=lexicon,
cache_dir=cache_dir,
)
except Exception as e: # isolate a bad chapter — keep going
logger.warning("[%s] chapter %d (%s) failed to render",
@@ -918,6 +1001,11 @@ async def _render_longform_sse(
ev["cached_segments"] = seg_stats["cached"]
yield _emit(ev)
route_notice = chapter_run.notice()
if route_notice is not None:
yield _emit({"type": "routing_notice", "status": route_notice[0],
"reason": route_notice[1]})
if interrupted:
logger.info("[%s] client disconnected — stopped after %d/%d chapters",
job_id, len(chapter_files), total)
@@ -1033,6 +1121,25 @@ async def _render_longform_sse(
yield _emit({"type": "error", "error": "render failed (see backend log)"})
async def _public_longform_stream(plan, **render_kwargs):
"""Keep generator diagnostics local if setup fails before its own guard."""
try:
async for event in _render_longform_sse(plan, **render_kwargs):
yield event
except asyncio.CancelledError:
raise
except Exception as exc:
from core.public_errors import public_failure
error = public_failure(
logger,
"Longform response stream failed",
exc,
response="Render failed; check the backend log for details.",
)
yield f"data: {json.dumps({'type': 'error', 'error': error})}\n\n"
@router.post("/audiobook")
async def audiobook_synthesize(req: AudiobookRequest, request: Request = None):
"""Synthesize a chapterized audiobook from a script, streaming SSE progress."""
@@ -1041,7 +1148,7 @@ async def audiobook_synthesize(req: AudiobookRequest, request: Request = None):
# to a direct in-process call, e.g. a unit test); its disconnect poll is what
# lets Stop cancel the render mid-book (#1216).
return StreamingResponse(
_render_longform_sse(
_public_longform_stream(
plan, default_voice=req.default_voice, language=req.language,
fmt=req.format, bitrate=req.bitrate,
loudness=req.loudness, cover_path=req.cover_path, metadata=req.metadata,
@@ -1102,7 +1209,7 @@ async def longform_render(req: LongformRenderRequest, request: Request = None):
chapters.append(Chapter(title=c.title or f"Chapter {i + 1}", spans=spans))
plan = AudiobookPlan(chapters=chapters)
return StreamingResponse(
_render_longform_sse(
_public_longform_stream(
plan, default_voice=req.default_voice, language=req.language,
fmt=req.format, bitrate=req.bitrate,
loudness=req.loudness, cover_path=req.cover_path, metadata=req.metadata,
@@ -1195,7 +1302,7 @@ async def resume_longform(job_id: str, request: Request = None):
# unrendered ones synthesize. Using a fresh id means the request's job_id
# never names a work dir / output file (defence-in-depth path-injection).
return StreamingResponse(
_render_longform_sse(
_public_longform_stream(
plan, default_voice=p.get("default_voice"), language=p.get("language"),
fmt=p.get("fmt", "m4b"), bitrate=p.get("bitrate", "128k"),
loudness=p.get("loudness"), cover_path=p.get("cover_path"),
+231
View File
@@ -0,0 +1,231 @@
"""Short-lived credentials for the first-party remote administration UI."""
from __future__ import annotations
import math
import threading
import time
from collections import OrderedDict, deque
from collections.abc import Callable
from datetime import UTC, datetime
from typing import Literal
from fastapi import APIRouter, HTTPException, Request, Response
from fastapi.responses import JSONResponse
from pydantic import BaseModel
from core.auth import (
CredentialTransport,
PrincipalKind,
authorization_credential_present,
legacy_master_cookie_valid,
master_header_valid,
principal_for,
remote_api_key,
)
from core.csrf import cookie_csrf_allowed, effective_scheme
from services.admin_sessions import (
SESSION_TTL_SECONDS,
WS_TICKET_TTL_SECONDS,
admin_session_store,
)
router = APIRouter(prefix="/api/auth", tags=["auth"])
_FAILED_EXCHANGE_LIMIT = 10
_FAILED_EXCHANGE_WINDOW_SECONDS = 60
_MAX_TRACKED_CLIENTS = 1024
class _ExchangeAttemptLimiter:
"""Bounded per-client sliding window for failed pre-auth exchanges."""
def __init__(
self,
*,
monotonic: Callable[[], float] = time.monotonic,
limit: int = _FAILED_EXCHANGE_LIMIT,
window_seconds: int = _FAILED_EXCHANGE_WINDOW_SECONDS,
max_clients: int = _MAX_TRACKED_CLIENTS,
) -> None:
if limit <= 0 or window_seconds <= 0 or max_clients <= 0:
raise ValueError("rate-limit bounds must be positive")
self._monotonic = monotonic
self._limit = limit
self._window_seconds = window_seconds
self._max_clients = max_clients
self._attempts: OrderedDict[str, deque[float]] = OrderedDict()
self._lock = threading.Lock()
def register_failure(self, client_id: str) -> int | None:
now = self._monotonic()
cutoff = now - self._window_seconds
with self._lock:
failures = self._attempts.setdefault(client_id, deque())
while failures and failures[0] <= cutoff:
failures.popleft()
self._attempts.move_to_end(client_id)
while len(self._attempts) > self._max_clients:
self._attempts.popitem(last=False)
if len(failures) >= self._limit:
return max(
1,
math.ceil(self._window_seconds - (now - failures[0])),
)
failures.append(now)
return None
def clear(self, client_id: str) -> None:
with self._lock:
self._attempts.pop(client_id, None)
def reset(self) -> None:
with self._lock:
self._attempts.clear()
_exchange_attempt_limiter = _ExchangeAttemptLimiter()
class SessionRequest(BaseModel):
transport: Literal["cookie", "bearer"]
class WebSocketTicketRequest(BaseModel):
path: str
def _secure_cookie(request: Request) -> bool:
# Same effective-scheme logic as the exact-origin CSRF check: the resolved
# scope first (uvicorn's trusted-proxy rewrite), upgraded — never
# downgraded — by X-Forwarded-Proto for TLS-terminating proxies uvicorn
# doesn't trust (Tailscale Serve into Docker, etc.). Spoofing the header on
# a plain-http hop can only ADD the Secure flag, which fails safe: the
# browser drops such a cookie, so the spoofer only breaks their own
# session. See core.csrf.effective_scheme for the full analysis.
return effective_scheme(request) == "https"
def _set_session_cookie(response: Response, request: Request, token: str, expires_at: float) -> None:
response.set_cookie(
"ov_session",
token,
max_age=SESSION_TTL_SECONDS,
expires=datetime.fromtimestamp(expires_at, tz=UTC),
path="/",
secure=_secure_cookie(request),
httponly=True,
samesite="strict",
)
def _expire_cookie(response: Response, request: Request, name: str) -> None:
response.delete_cookie(
name,
path="/",
secure=_secure_cookie(request),
httponly=name == "ov_session",
samesite="strict",
)
def _client_id(request: Request) -> str:
host = request.client.host if request.client else "unknown"
return str(host).strip().lower()[:255] or "unknown"
def _reject_master_exchange(request: Request) -> None:
retry_after = _exchange_attempt_limiter.register_failure(_client_id(request))
if retry_after is not None:
raise HTTPException(
status_code=429,
detail="Too many authentication attempts",
headers={"Retry-After": str(retry_after)},
)
raise HTTPException(status_code=401, detail="API key required")
@router.post("/session")
def create_session(payload: SessionRequest, request: Request) -> Response:
configured = remote_api_key()
if not configured:
raise HTTPException(status_code=401, detail="API key required")
authorization_present = authorization_credential_present(request)
header_authorized = master_header_valid(request)
legacy_authorized = legacy_master_cookie_valid(request)
migrating_legacy = False
if authorization_present:
if not header_authorized:
_reject_master_exchange(request)
elif legacy_authorized:
if payload.transport != "cookie" or not cookie_csrf_allowed(request):
raise HTTPException(status_code=403, detail="browser origin rejected")
migrating_legacy = True
else:
_reject_master_exchange(request)
_exchange_attempt_limiter.clear(_client_id(request))
issued = admin_session_store.issue(configured)
if payload.transport == "bearer":
return JSONResponse(
{
"token": issued.token,
"expires_at": issued.expires_at,
"expires_in": SESSION_TTL_SECONDS,
},
status_code=201,
)
response = Response(status_code=204)
_set_session_cookie(response, request, issued.token, issued.expires_at)
if migrating_legacy or request.cookies.get("ov_key"):
_expire_cookie(response, request, "ov_key")
return response
@router.delete("/session", status_code=204)
def delete_session(request: Request) -> Response:
principal = principal_for(request)
if principal.kind is PrincipalKind.ADMIN_SESSION:
if (
principal.transport is CredentialTransport.COOKIE
and not cookie_csrf_allowed(request)
):
raise HTTPException(status_code=403, detail="browser origin rejected")
admin_session_store.revoke_by_credential(principal.credential_id)
response = Response(status_code=204)
_expire_cookie(response, request, "ov_session")
return response
@router.post("/ws-ticket")
def create_ws_ticket(payload: WebSocketTicketRequest, request: Request) -> JSONResponse:
principal = principal_for(request)
if principal.kind is not PrincipalKind.ADMIN_SESSION:
raise HTTPException(status_code=403, detail="admin session required")
if (
principal.transport is CredentialTransport.COOKIE
and not cookie_csrf_allowed(request)
):
raise HTTPException(status_code=403, detail="browser origin rejected")
try:
ticket = admin_session_store.issue_ws_ticket_for_credential(
principal.credential_id,
payload.path,
remote_api_key(),
)
except ValueError as exc:
raise HTTPException(status_code=422, detail=str(exc)) from None
except PermissionError:
raise HTTPException(status_code=401, detail="admin session required") from None
return JSONResponse(
{
"ticket": ticket.token,
"expires_at": ticket.expires_at,
"expires_in": WS_TICKET_TTL_SECONDS,
},
status_code=201,
)
+213 -19
View File
@@ -103,6 +103,93 @@ def _set_progress(job, stage, percent=0, **extra):
job["progress"] = {"stage": stage, "percent": percent, **extra}
#: Override for the native dub batch width. Set to 1 to disable batching.
BATCH_WIDTH_ENV = "OMNIVOICE_DUB_BATCH_WIDTH"
#: Hard ceiling on the override — a batch this wide is already amortizing
#: almost all of the per-call setup, and beyond it the failure mode is an OOM
#: that costs more than the saving.
_MAX_BATCH_WIDTH = 16
# Bound each allocation while persisting multipart uploads. Video inputs can
# be many gigabytes; `await UploadFile.read()` with no size used to mirror the
# entire file in process memory before writing it back out.
_UPLOAD_CHUNK_BYTES = 1024 * 1024
async def _save_upload(upload: UploadFile, destination: str) -> None:
try:
with open(destination, "wb") as output:
while chunk := await upload.read(_UPLOAD_CHUNK_BYTES):
output.write(chunk)
except BaseException:
try:
unlink_if_present(destination)
except FileCleanupError:
logger.warning("Could not remove incomplete batch upload", exc_info=True)
raise
def _native_batch_width(backend) -> int:
"""How many segments to render in one native batch on THIS host.
A native batch widens the forward pass, so the width cannot be a constant.
The default engine declares ``min_vram_gb = 6.0`` for a SINGLE job; an
unconditional 8-wide batch would OOM the 4-8 GB CUDA cards and the MPS
Macs where the per-segment path succeeds today turning a throughput
optimization into a regression on exactly the hardware that already
struggles (#1616 is a 4 GB card reporting capacity failures). Default
behaviour must not get riskier on a host, so the width is derived from
measured headroom and falls back to 1 (no batching) when unknown.
CPU hosts get 1: batching there buys no kernel amortization and only
multiplies peak RAM.
"""
override = os.environ.get(BATCH_WIDTH_ENV, "").strip()
if override:
try:
return max(1, min(_MAX_BATCH_WIDTH, int(override)))
except (TypeError, ValueError):
logger.warning(
"%s=%r is not an integer — deriving the batch width from the host instead.",
BATCH_WIDTH_ENV, override,
)
try:
from core.device_caps import detect_host_caps
caps = detect_host_caps()
except Exception: # noqa: BLE001 — an unprobeable host takes the safe path
return 1
if caps.family == "cpu" or not caps.vram_gb:
return 1
headroom = caps.vram_gb - float(getattr(backend, "min_vram_gb", 0.0) or 0.0)
if headroom < 2.0:
return 1
if headroom < 6.0:
return 2
if headroom < 12.0:
return 4
return 8
def _batch_timeout_s(texts: list[str], backend) -> float:
"""Execution budget for one native batch.
Not the sum of the per-item budgets: ``generate_timeout_s`` returns a
floor (300s GPU / 600s CPU) plus per-length overage, so summing it across
eight items yields a ~2400s budget and a wedged batch would hold a
GPU-pool worker for forty minutes before the reset this file depends on
(#730). One floor covers wedge detection for the whole call; only the
length-driven overage is genuinely additive.
"""
from services.model_manager import generate_timeout_s
floor = generate_timeout_s("", engine=backend)
overage = sum(
max(0.0, generate_timeout_s(text, engine=backend) - floor) for text in texts
)
return floor + overage
async def _run_batch_pipeline(job_id: str, job: dict):
"""Full batch dub pipeline: extract → transcribe → translate → generate → mix → export."""
import subprocess
@@ -151,14 +238,17 @@ async def _run_batch_pipeline(job_id: str, job: dict):
# ── 2. Transcribe ─────────────────────────────────────────────────
_set_progress(job, "transcribe", 0)
from services.asr_backend import get_active_asr_backend
from services.asr_backend import load_active_asr_backend
from services.model_manager import _gpu_pool, _cpu_pool, run_on_gpu_pool_guarded
from services.segmentation import (
segment_transcript, assign_speakers_heuristic,
)
def _transcribe():
backend = get_active_asr_backend()
# `load_*`, not `get_*`: the plain selector returns engines whose
# shallow probe passed but whose deep import chain is broken, failing
# the whole batch job at `.transcribe()` instead of degrading (#1185).
backend = load_active_asr_backend()
result = backend.transcribe(audio_path, word_timestamps=True)
detected_lang = result.get("language", "en")
segments = segment_transcript(result, duration=duration)
@@ -191,7 +281,7 @@ async def _run_batch_pipeline(job_id: str, job: dict):
# ── Engine resolution (issue #312 class) ────────────────────────────
# Batch used to hardcode VoiceStudio via get_model() regardless of the
# engine selected in Settings → Engines. require_cloning only when a
# engine selected in Model Catalogue → Engines. require_cloning only when a
# specific voice is pinned (job["voice_id"]) — an unpinned job is fine on
# any active engine. Resolved ONCE for the whole job (every language
# below shares the same active engine); an uncaught ValueError here
@@ -276,6 +366,111 @@ async def _run_batch_pipeline(job_id: str, job: dict):
full_audio = torch.zeros(1, total_samples)
total_segs = len(translated_segments)
# Native engines can amortize encoder/decoder setup across a small
# batch. Keep the adapter seam optional: engines without a real batch
# implementation inherit TTSBackend.generate_batch(), which preserves
# the established one-segment behavior below.
from services.tts_backend import TTSBackend
batched_audio: dict[int, torch.Tensor] = {}
has_native_batch = type(backend).generate_batch is not TTSBackend.generate_batch
if has_native_batch:
from services.text_normalization import normalize_for_tts
batch_ref_audio = None
batch_ref_text = None
if job.get("voice_id"):
from core.db import db_conn
from core.config import VOICES_DIR as _VD
with db_conn() as conn:
row = conn.execute(
"SELECT * FROM voice_profiles WHERE id=?",
(job["voice_id"],),
).fetchone()
if row:
if row["is_locked"] and row["locked_audio_path"]:
batch_ref_audio = os.path.join(_VD, row["locked_audio_path"])
elif row["ref_audio_path"]:
batch_ref_audio = os.path.join(_VD, row["ref_audio_path"])
batch_ref_text = row["ref_text"]
batch_width = _native_batch_width(backend)
async def _prefetch_batch(first_index: int) -> None:
"""Render the batch beginning at ``first_index`` into
``batched_audio``.
Rendered on demand rather than prerendering the whole track:
the tensors are popped as they are placed, so peak host memory
is one batch instead of every segment of the language and
the progress bar tracks placement instead of running to the
end and restarting at segment 1.
"""
if job["status"] == "cancelled":
return
batch_rows = []
index = first_index
while index < total_segs and len(batch_rows) < batch_width:
seg = translated_segments[index]
if (seg.get("end", 0) - seg.get("start", 0) > 0.05
and seg.get("text", "").strip()):
batch_rows.append((index, seg))
index += 1
if len(batch_rows) < 2:
return # nothing to amortize — the per-segment path is equal
batch_indices = [index for index, _ in batch_rows]
batch_texts = [
normalize_for_tts(row.get("text", "").strip(), target_lang)
for _, row in batch_rows
]
batch_durations = [
row.get("end", 0) - row.get("start", 0)
for _, row in batch_rows
]
def _render_native_batch():
generated = backend.generate_batch(
batch_texts,
language=target_lang,
ref_audio=batch_ref_audio,
ref_text=batch_ref_text,
duration=batch_durations,
num_step=16,
guidance_scale=2.0,
speed=1.0,
denoise=True,
postprocess_output=True,
)
if len(generated) != len(batch_indices):
raise RuntimeError(
f"native batch returned {len(generated)} outputs for "
f"{len(batch_indices)} segments"
)
rendered = []
for audio_out in generated:
if not getattr(backend, "applies_own_mastering", False):
audio_out = apply_mastering(audio_out, sample_rate=sr)
rendered.append(normalize_audio(audio_out, target_dBFS=-2.0))
return rendered
try:
rendered = await run_on_gpu_pool_guarded(
_render_native_batch,
what="Batch generate",
timeout=_batch_timeout_s(batch_texts, backend),
)
batched_audio.update(zip(batch_indices, rendered))
except TimeoutError:
# Do not immediately queue the same expensive work again:
# the timed-out pool task may still be holding the device.
raise
except Exception as e:
logger.warning(
"Native TTS batch failed for segments %s-%s; falling back per segment: %s",
batch_indices[0] + 1,
batch_indices[-1] + 1,
e,
)
for i, seg in enumerate(translated_segments):
if job["status"] == "cancelled":
return
@@ -353,10 +548,15 @@ async def _run_batch_pipeline(job_id: str, job: dict):
# Budget is the shared length-scaled one (#1190): a long segment
# on CPU-class hardware no longer dies on the flat 300s.
from services.model_manager import generate_timeout_s
audio_tensor = await run_on_gpu_pool_guarded(
_gen, what="Batch generate",
timeout=generate_timeout_s(seg_text),
)
if has_native_batch and i not in batched_audio:
await _prefetch_batch(i)
if i in batched_audio:
audio_tensor = batched_audio.pop(i)
else:
audio_tensor = await run_on_gpu_pool_guarded(
_gen, what="Batch generate",
timeout=generate_timeout_s(seg_text, engine=backend),
)
# Fit to slot
target_samples_seg = int(seg_duration * sr)
@@ -410,19 +610,15 @@ async def _run_batch_pipeline(job_id: str, job: dict):
# unmarked while the interactive dub pipeline marked every segment.
# One whole-track embed (chunked internally, #1045) is equivalent to
# dub_generate's per-segment marks: the 16-bit message repeats
# throughout. Runs in the GPU pool like generate's finalize; never
# raises (degrades to unmarked on failure, same as every producer).
# throughout. Never raises (degrades to unmarked on failure, same as
# every producer).
# Dispatched to the dedicated watermark pool, not the GPU pool (#1190):
# AudioSeal embedding is CPU work that holds no VRAM, and a whole-track
# embed is long enough that occupying a GPU worker with it stalled the
# next language's segments on 1-worker hosts.
from services.watermark import mark_synthetic
from services.model_manager import get_watermark_pool
import functools
full_audio = await loop.run_in_executor(
get_watermark_pool(),
functools.partial(mark_synthetic, full_audio, sr,
context="batch.dub_track"),
from services.watermark import mark_synthetic_async
full_audio = await mark_synthetic_async(
full_audio, sr, context="batch.dub_track",
)
# Same assembly pattern as dub_generate.py:390 — `full_audio` is a
@@ -512,9 +708,7 @@ async def enqueue_batch_job(
ext = os.path.splitext(video.filename or "video.mp4")[1] or ".mp4"
video_path = os.path.join(batch_dir, f"{job_id}{ext}")
with open(video_path, "wb") as f:
content = await video.read()
f.write(content)
await _save_upload(video, video_path)
job = {
"id": job_id,
+24 -5
View File
@@ -8,8 +8,11 @@ raw audio bytes and get back transcribed text immediately. Used by:
The MCP server's future `transcribe_audio` tool
CLI consumers that just want speech-to-text
The ASR engine is whatever `get_active_asr_backend()` returns WhisperX
by default, or MLX Whisper on Apple Silicon when configured.
The ASR engine is whatever `load_active_asr_backend()` returns WhisperX
by default, or MLX Whisper on Apple Silicon when configured. The *loader*,
not the bare selector: it also runs `ensure_loaded()` and falls through to
the next healthy engine when the selected one has a broken deep import chain
(#1185), which the shallow `is_available()` probe cannot see.
"""
from __future__ import annotations
@@ -97,8 +100,12 @@ async def transcribe_audio(
if use_accurate:
# Accurate mode: full WhisperX with forced alignment —
# for when the user explicitly wants word-level timing.
from services.asr_backend import get_active_asr_backend
backend = get_active_asr_backend()
# `load_*`, not `get_*`: the selector alone hands back an
# engine whose shallow probe passed but whose deep import
# chain is broken, which then 500s at `.transcribe()`. The
# loader degrades to the next healthy engine (#1185).
from services.asr_backend import load_active_asr_backend
backend = load_active_asr_backend()
result = backend.transcribe(tmp.name, word_timestamps=True)
else:
# Fast mode (default): use the fastest available engine
@@ -110,7 +117,11 @@ async def transcribe_audio(
return result, backend.id
from services.model_manager import _gpu_pool
from services.asr_backend import ASRTimeoutError, run_transcribe_guarded
from services.asr_backend import (
ASRModelMissingError,
ASRTimeoutError,
run_transcribe_guarded,
)
t0 = time.perf_counter()
try:
result, engine_id = await run_transcribe_guarded(
@@ -121,6 +132,14 @@ async def transcribe_audio(
# silent hang the UI reads as "can't reach the local backend".
logger.warning("Capture transcription timed out: %s", e)
raise HTTPException(status_code=504, detail=str(e))
except ASRModelMissingError as e:
# Degraded past the broken engine onto one with no weights on
# disk — same typed 409 (+ download CTA) as the preflight above,
# never a 500 and never a silent multi-GB auto-download.
raise HTTPException(
status_code=409,
detail={**e.payload, "message": asr_model_missing_detail(e.payload)},
)
elapsed = round(time.perf_counter() - t0, 2)
# Normalize result shape
+322 -75
View File
@@ -27,6 +27,10 @@ Protocol:
"detail": "..."} error ("detail"
kept for legacy)
Sherpa ``final`` frames additionally carry
``"final_kind": "utterance"|"summary"``. Utterances are mid-session
commits; the summary is the authoritative whole-session result at EOF.
Every ``final`` text is normalised by services.text_polish (leading
capital for Latin scripts, terminal punctuation, single-spaced) so the
pasted result reads like typed text. Partials are raw.
@@ -34,10 +38,14 @@ Protocol:
from __future__ import annotations
import asyncio
import json
import logging
import math
import os
import tempfile
import time
import uuid
from typing import Any
from fastapi import APIRouter, WebSocket, WebSocketDisconnect
@@ -47,6 +55,9 @@ from services.text_polish import polish_text
router = APIRouter()
logger = logging.getLogger("omnivoice.capture_ws")
SPEECH_PROTOCOL = "voicestudio.speech.v1"
PLATFORM_STREAM_PATH = "/v1/audio/transcriptions/stream"
# How often (seconds) to run transcription on the accumulated buffer.
# Shorter = more responsive but more GPU load.
PARTIAL_INTERVAL_S = float(os.environ.get("OMNIVOICE_STREAM_INTERVAL", "2.0"))
@@ -70,17 +81,79 @@ _AEC_NEAR = 0x00 # microphone frame (clean it, then buffer for ASR)
_AEC_FAR = 0x01 # playback reference frame (feed the echo model only)
def _requested_pcm_sample_rate(query_params) -> int | None:
"""Return a bounded PCM rate for ``?pcm=1``/``?aec=1`` sessions."""
raw_pcm = query_params.get("pcm") in ("1", "true", "on")
aec = query_params.get("aec") in ("1", "true", "on")
if not raw_pcm and not aec:
return None
# Client-supplied ``?sr=`` values outside the range real capture devices use
# are replaced with 16 kHz. The rate sizes server-side state — RecoveryTail
# multiplies it by RECOVERY_TAIL_SECONDS to compute its byte ceiling — so an
# absurd rate must never be believed: it would re-open the unbounded-memory
# path the recovery-tail cap closed.
SR_MIN, SR_MAX = 8000, 96000
def _is_end_control(text: str | None) -> bool:
"""Accept the versioned JSON control frame and the legacy ``EOF`` frame."""
if text == "EOF":
return True
if not text:
return False
try:
message = json.loads(text)
except (TypeError, json.JSONDecodeError):
return False
return isinstance(message, dict) and message.get("type") == "input_audio.end"
class _PlatformWebSocket:
"""Add v1 session metadata without changing the legacy WebSocket contract."""
def __init__(self, websocket: WebSocket):
self._websocket = websocket
self.session_id = uuid.uuid4().hex
def __getattr__(self, name: str) -> Any:
return getattr(self._websocket, name)
async def send_json(self, data: Any, mode: str = "text") -> None:
if isinstance(data, dict):
data = dict(data)
data.setdefault("protocol", SPEECH_PROTOCOL)
data.setdefault("session_id", self.session_id)
if data.get("type") == "final":
data.setdefault("final_kind", "summary")
await self._websocket.send_json(data, mode=mode)
def _bounded_sample_rate(query_params) -> int:
try:
sample_rate = int(query_params.get("sr", "16000"))
except (TypeError, ValueError):
return 16000
return sample_rate if 8000 <= sample_rate <= 96000 else 16000
return sample_rate if SR_MIN <= sample_rate <= SR_MAX else 16000
def _requested_pcm_sample_rate(query_params) -> int | None:
"""Return the bounded rate when the client transport is raw PCM.
Sherpa clients omit ``pcm=1`` because the selected model already defines
that transport. If the model is demoted or its runtime is unavailable, the
legacy recognizer fallback must still decode those same bytes as PCM.
"""
raw_pcm = query_params.get("pcm") in ("1", "true", "on")
aec = query_params.get("aec") in ("1", "true", "on")
sherpa_pcm = False
requested_model = query_params.get("model")
if requested_model:
try:
from services.sherpa_dictation import is_sherpa_model
sherpa_pcm = is_sherpa_model(requested_model)
except Exception: # noqa: BLE001
# A broken sherpa install must not decide the framing question —
# sherpa_pcm stays False and the session negotiates the
# MediaRecorder path; availability is re-probed (and reported)
# when the model is actually selected.
sherpa_pcm = False
if not raw_pcm and not aec and not sherpa_pcm:
return None
return _bounded_sample_rate(query_params)
def _demux_aec_frame(data: bytes) -> tuple[str, bytes]:
@@ -137,24 +210,50 @@ def _select_sherpa_spec(websocket: WebSocket):
from services import sherpa_dictation as sd
except Exception:
return None
def _usable_spec(model_id):
spec = sd.get_spec(model_id)
if spec is not None and sd.is_demoted(spec.id):
logger.warning(
"dictation model %s is demoted — using the capture ASR fallback",
spec.id,
)
return None
return spec
requested = websocket.query_params.get("model")
if requested:
return sd.get_spec(requested) # explicit selection (may be None if bad)
return _usable_spec(requested) # explicit selection (may be unavailable)
# Fall back to the persisted dictation pref.
try:
from services.asr_backend import dictation_model_id
mid = dictation_model_id()
except Exception:
mid = None
return sd.get_spec(mid) if mid else None
return _usable_spec(mid) if mid else None
@router.websocket(PLATFORM_STREAM_PATH)
@router.websocket("/ws/transcribe")
async def ws_transcribe(websocket: WebSocket):
"""Stream audio in, get partial + final transcription out."""
is_platform_stream = websocket.url.path == PLATFORM_STREAM_PATH
if is_platform_stream:
websocket = _PlatformWebSocket(websocket)
# A browser can reach localhost regardless of the page's own origin.
# Reject ambient cross-site WebSocket handshakes before the loopback-host
# shortcut or accept(), while keeping native clients (no Origin header)
# and configured/same-origin browser UIs working (#1646 review).
origin = websocket.headers.get("origin")
if origin:
from core.csrf import origin_allowed
if not origin_allowed(websocket):
await websocket.close(code=1008, reason="browser origin not allowed")
return
# Loopback origin guard — refuse anything not from 127.0.0.1, ::1, or
# localhost. HTTP routers use Depends(require_loopback) at router level;
# WebSocket dependency injection differs across FastAPI versions, so we
# localhost. Privileged HTTP routers use Depends(require_admin) at router
# level; WebSocket dependency injection differs across FastAPI versions, so we
# inline the check before accept(). Without it, any local process could
# stream the user's microphone over this endpoint.
# Wave 2.3 (remote backend): a non-loopback client that presents the
@@ -166,6 +265,16 @@ async def ws_transcribe(websocket: WebSocket):
return
await websocket.accept()
if is_platform_stream:
await websocket.send_json({
"type": "session.started",
"input_format": (
"audio/pcm;encoding=s16le;channels=1"
if _requested_pcm_sample_rate(websocket.query_params) is not None
else "audio/webm;codecs=opus"
),
"sample_rate": _bounded_sample_rate(websocket.query_params),
})
# Live-dictation engine selection. When a sherpa-onnx model is selected
# (via ?model= or the dictation.model_id pref) AND sherpa is installed,
@@ -288,7 +397,7 @@ async def ws_transcribe(websocket: WebSocket):
total_bytes += len(data)
last_audio_time = time.monotonic()
continue
if msg.get("text") == "EOF":
if _is_end_control(msg.get("text")):
# Client signals end-of-audio but stays connected for `final`.
running = False
break
@@ -422,6 +531,64 @@ SHERPA_OFFLINE_SILENCE_S = float(os.environ.get("OMNIVOICE_SHERPA_OFFLINE_SILENC
SHERPA_OFFLINE_RMS_FLOOR = float(os.environ.get("OMNIVOICE_SHERPA_OFFLINE_RMS", "0.01"))
#: Seconds of audio retained for silent-model recovery. Recovery only needs
#: enough speech to prove the model is broken and to re-transcribe what was
#: said; retaining the whole session grew ~115 MB/hour at 16 kHz on an open
#: mic, unbounded, and only ever got read when the fallback fired.
RECOVERY_TAIL_DEFAULT_SECONDS = 120.0
RECOVERY_TAIL_MAX_SECONDS = 300.0
def _bounded_recovery_tail_seconds(value: str | None) -> float:
"""Parse the recovery tail override without allowing unbounded buffers."""
try:
seconds = float(value) if value is not None else RECOVERY_TAIL_DEFAULT_SECONDS
except (TypeError, ValueError):
return RECOVERY_TAIL_DEFAULT_SECONDS
if not math.isfinite(seconds) or seconds <= 0:
return RECOVERY_TAIL_DEFAULT_SECONDS
return min(seconds, RECOVERY_TAIL_MAX_SECONDS)
RECOVERY_TAIL_SECONDS = _bounded_recovery_tail_seconds(
os.environ.get("OMNIVOICE_DICTATION_RECOVERY_TAIL_S")
)
class RecoveryTail:
"""The most recent ``RECOVERY_TAIL_SECONDS`` of session audio.
Keeps the *tail* rather than the head: a long dictation's useful speech is
what the user just said, and the silent-model check cares about how much
audio the session carried overall which ``total_bytes`` still reports
truthfully after trimming.
"""
__slots__ = ("_buf", "_max", "total_bytes")
def __init__(self, sample_rate: int, seconds: float = RECOVERY_TAIL_SECONDS):
# int16 mono → 2 bytes/sample. Floor of one frame so a nonsense rate
# or seconds value can't produce a zero-length buffer.
self._max = max(2, int(seconds * max(1, sample_rate)) * 2)
self._buf = bytearray()
self.total_bytes = 0
def extend(self, pcm: bytes) -> None:
self._buf.extend(pcm)
self.total_bytes += len(pcm)
excess = len(self._buf) - self._max
if excess > 0:
# int16 mono: trim whole samples only. A split frame can carry an
# odd byte count, and an odd trim would leave the tail starting
# mid-sample — every later sample byte-shifted, and the recovery
# transcription fed noise.
excess += excess % 2
del self._buf[:excess]
def tail(self) -> bytes:
return bytes(self._buf)
def is_model_silent(text: str, heard_speech: bool, pcm_bytes: int) -> bool:
"""True when the dictation model produced NO text despite real speech.
@@ -448,19 +615,74 @@ def _pcm16_to_f32(pcm: bytes):
return np.frombuffer(pcm, dtype=np.int16).astype(np.float32) / 32768.0
async def _sherpa_session(websocket: WebSocket):
"""Shared WS receive setup for the sherpa handlers.
def _pcm16_rms(pcm: bytes) -> float:
samples = _pcm16_to_f32(pcm)
if not len(samples):
return 0.0
return float((samples * samples).mean() ** 0.5)
Returns ``(get_frame, state)`` where ``get_frame`` is an async callable
that yields the next near-end (mic) PCM bytes, ``b""`` for a keepalive/ref
frame, or ``None`` on EOF/disconnect. ``state`` carries sample rate, AEC,
and the disconnect flag for the caller's finaliser.
"""
pcm_sr = 16000
async def _recover_silent_sherpa(
spec, pcm: bytes, pcm_sr: int,
) -> tuple[str, list[dict]]:
"""Retry a token-silent Sherpa session through an installed local ASR."""
logger.warning(
"dictation model %s decoded NOTHING from %.1fs of speech-level audio "
"— falling back to the capture ASR engine for this session",
spec.id, len(pcm) / float(max(1, pcm_sr) * 2),
)
try:
pcm_sr = int(websocket.query_params.get("sr", "16000"))
except (TypeError, ValueError):
pcm_sr = 16000
from services.asr_backend import asr_model_missing_error
fallback_missing = await asyncio.to_thread(
asr_model_missing_error,
purpose="dictation",
skip_sherpa=True,
require_installed=True,
)
if fallback_missing is not None:
logger.warning(
"dictation silent-model fallback is not installed (%s); "
"skipping recovery to avoid an automatic download",
fallback_missing.get("missing_repo_id", "unknown"),
)
return "", []
result = await _transcribe_buffer_full(
[pcm], pcm_sr=pcm_sr, skip_sherpa=True,
)
text = polish_text(_result_text(result))
if not text:
return "", []
# The RMS gate can fire on fan/keyboard noise. Only another recognizer
# producing words proves the audio held speech and makes persistent
# demotion safe.
try:
from services.sherpa_dictation import demote_model
if await asyncio.to_thread(demote_model, spec.id):
logger.error(
"dictation model %s demoted on this machine — it will no longer be "
"auto-selected. Pick it again in Settings to give it another chance.",
spec.id,
)
except Exception:
logger.exception("silent-model demotion failed")
segments = (result or {}).get("segments") or [
{"start": 0.0, "end": None, "text": text}
]
return text, segments
except Exception:
logger.exception("dictation silent-model fallback failed")
return "", []
async def _sherpa_session(websocket: WebSocket):
"""Shared WS setup for the sherpa handlers.
Returns ``(pcm_sr, aec)``: the bounded PCM sample rate for the session
and the echo canceller when ``?aec=1`` requested one (``None`` otherwise
or when AEC setup fails).
"""
pcm_sr = _bounded_sample_rate(websocket.query_params)
aec = None
if websocket.query_params.get("aec") in ("1", "true", "on"):
try:
@@ -498,7 +720,7 @@ async def _recv_pcm_frame(websocket: WebSocket, aec):
return "skip", b""
return "near", aec.process_near_end(payload)
return "near", data
if msg.get("text") == "EOF":
if _is_end_control(msg.get("text")):
return "eof", b""
return "skip", b""
@@ -569,6 +791,8 @@ async def _run_sherpa_streaming(websocket: WebSocket, spec):
last_partial = ""
committed: list[str] = [] # finalized utterances this session
session_pcm = RecoveryTail(pcm_sr) # bounded audio for silent-model recovery
heard_speech = False
client_disconnected = False
async def _send(payload) -> bool:
@@ -610,6 +834,9 @@ async def _run_sherpa_streaming(websocket: WebSocket, spec):
break
if kind == "skip":
continue
session_pcm.extend(pcm)
if not heard_speech and _pcm16_rms(pcm) >= SHERPA_OFFLINE_RMS_FLOOR:
heard_speech = True
text, endpoint = await asyncio.to_thread(_decode_after_feed, pcm)
if endpoint:
# Commit this utterance (polished — it gets pasted); reset
@@ -618,6 +845,7 @@ async def _run_sherpa_streaming(websocket: WebSocket, spec):
if text:
committed.append(text)
await _send({"type": "final", "text": text,
"final_kind": "utterance",
"segments": [{"start": 0.0, "end": None, "text": text}],
"language": "auto", "engine": backend.id})
rec.reset(stream)
@@ -644,7 +872,28 @@ async def _run_sherpa_streaming(websocket: WebSocket, spec):
# Pieces are already polished; the join is too (polish is idempotent).
full = " ".join(t for t in committed if t).strip()
segments = [{"start": 0.0, "end": None, "text": t} for t in committed if t]
model_silent = is_model_silent(full, heard_speech, session_pcm.total_bytes)
if model_silent:
recovered, recovered_segments = await _recover_silent_sherpa(
spec, session_pcm.tail(), pcm_sr,
)
if recovered:
full = recovered
segments = recovered_segments
if not client_disconnected:
payload = {"type": "final", "text": full, "final_kind": "summary",
"segments": segments,
"language": "auto", "engine": backend.id}
if model_silent:
payload["engine"] = "capture-asr-fallback" if full else backend.id
payload["model_silent"] = spec.id
payload["warning"] = (
f"The selected dictation model ({spec.id}) produced no text from your "
"speech. Switched to the fallback engine for this session — pick a "
"different model in Settings → Dictation."
)
if full:
# Hard-bounded refinement (~4s): never delays this summary `final`
# beyond OMNIVOICE_REFINE_TIMEOUT_S even with a dead LLM endpoint.
@@ -653,14 +902,9 @@ async def _run_sherpa_streaming(websocket: WebSocket, spec):
refined = await maybe_refine_async(full)
except Exception:
refined = None
payload = {"type": "final", "text": full, "segments": segments,
"language": "auto", "engine": backend.id}
if refined and refined != full:
payload["refined_text"] = refined
await _send(payload)
else:
await _send({"type": "final", "text": "", "segments": [],
"language": "auto", "engine": backend.id})
await _send(payload)
try:
await websocket.close()
except Exception:
@@ -697,7 +941,7 @@ async def _run_sherpa_offline(websocket: WebSocket, spec):
# whisper/zipformer transcribe the same bytes). Keep the whole session's
# audio and whether any of it was speech-level, so the finaliser can tell
# "user said nothing" (fine) from "model produced nothing" (broken).
session_pcm = bytearray()
session_pcm = RecoveryTail(pcm_sr)
heard_speech = False
running = True
client_disconnected = False
@@ -716,12 +960,6 @@ async def _run_sherpa_offline(websocket: WebSocket, spec):
client_disconnected = True
return False
def _rms(pcm: bytes) -> float:
samples = _pcm16_to_f32(pcm)
if not len(samples):
return 0.0
return float((samples * samples).mean() ** 0.5)
def _decode_window(pcm: bytes) -> str:
samples = _pcm16_to_f32(pcm)
if not len(samples):
@@ -740,7 +978,7 @@ async def _run_sherpa_offline(websocket: WebSocket, spec):
continue
buf.extend(pcm)
session_pcm.extend(pcm)
if not heard_speech and _rms(pcm) >= SHERPA_OFFLINE_RMS_FLOOR:
if not heard_speech and _pcm16_rms(pcm) >= SHERPA_OFFLINE_RMS_FLOOR:
heard_speech = True
last_audio = time.monotonic()
except WebSocketDisconnect:
@@ -766,6 +1004,7 @@ async def _run_sherpa_offline(websocket: WebSocket, spec):
if text:
committed.append(text)
await _send({"type": "final", "text": text,
"final_kind": "utterance",
"segments": [{"start": 0.0, "end": None, "text": text}],
"language": "auto", "engine": backend.id})
@@ -777,8 +1016,8 @@ async def _run_sherpa_offline(websocket: WebSocket, spec):
continue
snapshot = bytes(buf)
if len(snapshot) > sil_bytes and \
_rms(snapshot[-sil_bytes:]) < SHERPA_OFFLINE_RMS_FLOOR:
if _rms(snapshot[:-sil_bytes]) >= SHERPA_OFFLINE_RMS_FLOOR:
_pcm16_rms(snapshot[-sil_bytes:]) < SHERPA_OFFLINE_RMS_FLOOR:
if _pcm16_rms(snapshot[:-sil_bytes]) >= SHERPA_OFFLINE_RMS_FLOOR:
await _commit(snapshot)
else:
# Pure silence — drop it (keep the gate window for
@@ -824,39 +1063,18 @@ async def _run_sherpa_offline(websocket: WebSocket, spec):
# quiet user — hand the session to the capture ASR backend so the user
# still gets their words, and say which model let them down. Bounded to
# this session; the pref is left alone so the user stays in control.
model_silent = is_model_silent(full, heard_speech, len(session_pcm))
model_silent = is_model_silent(full, heard_speech, session_pcm.total_bytes)
if model_silent:
logger.warning(
"dictation model %s decoded NOTHING from %.1fs of speech-level audio "
"— falling back to the capture ASR engine for this session",
spec.id, len(session_pcm) / float(max(1, pcm_sr) * 2),
recovered, recovered_segments = await _recover_silent_sherpa(
spec, session_pcm.tail(), pcm_sr,
)
# Demote it so the NEXT session doesn't repeat this round trip. The
# curated default can be broken on a platform we never tested (the
# NeMo-TDT decoder is, on Windows), and observing it beats guessing.
try:
from services.sherpa_dictation import demote_model
if demote_model(spec.id):
logger.error(
"dictation model %s demoted on this machine — it will no longer be "
"auto-selected. Pick it again in Settings to give it another chance.",
spec.id,
)
except Exception:
logger.exception("silent-model demotion failed")
try:
result = await _transcribe_buffer_full([bytes(session_pcm)], pcm_sr=pcm_sr)
fb_text = polish_text((result or {}).get("text", "") or "")
if fb_text:
full = fb_text
segments = (result or {}).get("segments") or [
{"start": 0.0, "end": None, "text": fb_text}
]
except Exception:
logger.exception("dictation silent-model fallback failed")
if recovered:
full = recovered
segments = recovered_segments
if not client_disconnected:
payload = {"type": "final", "text": full, "segments": segments,
payload = {"type": "final", "text": full, "final_kind": "summary",
"segments": segments,
"language": "auto", "engine": backend.id}
if model_silent:
# The client surfaces this so a silently-broken model can't look
@@ -884,6 +1102,35 @@ async def _run_sherpa_offline(websocket: WebSocket, spec):
pass
def _result_text(result: dict | None) -> str:
"""Normalize text from every ASR backend result shape.
Some backends return a top-level ``text`` value, while WhisperX, Faster
Whisper, Moonshine, and OpenAI-compatible ASR expose only ``segments`` and
``chunks``. Dictation partials and finals must interpret both contracts the
same way.
"""
if not isinstance(result, dict):
return ""
text = result.get("text")
if isinstance(text, str) and text.strip():
return text.strip()
for key in ("segments", "chunks"):
items = result.get(key)
if not isinstance(items, (list, tuple)):
continue
text = " ".join(
str(item.get("text", "")).strip()
for item in items
if isinstance(item, dict) and item.get("text")
).strip()
if text:
return text
return ""
async def _transcribe_buffer(chunks: list[bytes], *, pcm_sr: int | None = None) -> str:
"""Quick partial transcription of the current audio buffer."""
@@ -898,7 +1145,7 @@ async def _transcribe_buffer(chunks: list[bytes], *, pcm_sr: int | None = None)
def _run():
backend = get_capture_asr_backend()
result = backend.transcribe(tmp, word_timestamps=False)
return result.get("text", "")
return _result_text(result)
# Bound dictation transcribes (#730): a wedged whisperx/CTranslate2 call
# must not hold its GPU-pool worker forever and starve TTS / other ASR
@@ -912,7 +1159,9 @@ async def _transcribe_buffer(chunks: list[bytes], *, pcm_sr: int | None = None)
pass
async def _transcribe_buffer_full(chunks: list[bytes], *, pcm_sr: int | None = None) -> dict:
async def _transcribe_buffer_full(
chunks: list[bytes], *, pcm_sr: int | None = None, skip_sherpa: bool = False,
) -> dict:
"""Full transcription with timing info for the final result."""
tmp = _pcm16_to_wav(b"".join(chunks), pcm_sr) if pcm_sr else _chunks_to_wav(chunks)
if tmp is None:
@@ -924,15 +1173,13 @@ async def _transcribe_buffer_full(chunks: list[bytes], *, pcm_sr: int | None = N
from services.asr_backend import get_capture_asr_backend, run_transcribe_guarded
def _run():
backend = get_capture_asr_backend()
backend = get_capture_asr_backend(skip_sherpa=skip_sherpa)
t0 = time.perf_counter()
result = backend.transcribe(tmp, word_timestamps=False)
elapsed = round(time.perf_counter() - t0, 2)
segments = result.get("segments", [])
full_text = result.get("text", "")
if not full_text and segments:
full_text = " ".join(s.get("text", "") for s in segments).strip()
full_text = _result_text(result)
# Wave 1.1: strip Whisper hallucination loops from the final
# text (the string that gets auto-pasted). Segments keep the
+728 -99
View File
@@ -20,18 +20,26 @@ Design / safety
from __future__ import annotations
import asyncio
import contextlib
import hashlib
import json
import logging
import os
import re
import shutil
import tempfile
import time
import uuid
from pathlib import Path
from typing import Optional
from urllib.parse import urlparse
from urllib.parse import urljoin, urlparse
from fastapi import APIRouter, HTTPException, Query
from fastapi.responses import FileResponse
from core import archetypes
from core.config import DATA_DIR
from core.audio_validation import is_playable_wav, resolve_regular_file
from core.config import DATA_DIR, VOICES_DIR
logger = logging.getLogger("omnivoice.community")
router = APIRouter()
@@ -42,9 +50,32 @@ _ALLOWED_AUDIO_HOSTS = {
"cdn.jsdelivr.net", "github.com", "raw.githubusercontent.com",
"objects.githubusercontent.com", "release-assets.githubusercontent.com",
}
_ALLOWED_MANIFEST_HOSTS = {"cdn.jsdelivr.net"}
_VALID_TOKENS = set(archetypes._VD._INSTRUCT_ALL_VALID)
_USE_CASE_IDS = {c["id"] for c in archetypes.USE_CASES}
_SOURCE_RE = re.compile(r"^[A-Za-z0-9._-]+/[A-Za-z0-9._-]+$") # owner/repo only
_SOURCE_RE = re.compile(
r"^[A-Za-z0-9._-]{1,100}/[A-Za-z0-9._-]{1,100}$",
) # owner/repo only
_ITEM_ID_RE = re.compile(r"^[A-Za-z0-9_-]{1,128}$")
_SHA256_RE = re.compile(r"^[0-9a-f]{64}$")
# A gallery open may touch this loader several times (grid, preview, use). Keep
# a successful response for six hours, then revalidate it once. On a network
# failure the readable stale copy remains usable and its check time advances,
# preventing every offline gallery open from waiting through the same timeout.
_MANIFEST_MAX_AGE_S = 6 * 60 * 60
_MAX_MANIFEST_BYTES = 4 << 20
_MAX_SAMPLE_SCRIPT_CHARS = 2_000
_MAX_REF_TEXT_CHARS = 4_000
# Community voice submissions are documented as short clean WAV clips. The cap
# comfortably covers 15 s of uncompressed 96 kHz stereo PCM while preventing a
# remote manifest from turning Preview into an unbounded disk/memory download.
_MAX_VOICE_AUDIO_BYTES = 32 << 20
_ATTR_NAMES = (
"Gender", "Age", "Pitch", "Style", "EnglishAccent", "ChineseDialect",
)
# ── Config: which content repos to load ───────────────────────────────────────
@@ -52,14 +83,18 @@ def configured_sources() -> list[str]:
"""Gallery sources, in priority order. Env var > config file > default."""
env = os.environ.get("OMNIVOICE_GALLERY_SOURCES")
if env:
return [s.strip() for s in env.split(",") if s.strip()]
sources = [s.strip() for s in env.split(",")]
valid = [s for s in sources if _SOURCE_RE.fullmatch(s)]
return valid or list(_DEFAULT_SOURCES)
cfg = Path(DATA_DIR) / "gallery_sources.json"
if cfg.exists():
try:
data = json.loads(cfg.read_text(encoding="utf-8"))
srcs = data.get("sources")
if isinstance(srcs, list) and srcs:
return [str(s) for s in srcs]
valid = [s for s in srcs if isinstance(s, str) and _SOURCE_RE.fullmatch(s)]
if valid:
return valid
except Exception:
logger.warning("gallery_sources.json unreadable; using default")
return list(_DEFAULT_SOURCES)
@@ -81,9 +116,51 @@ def _safe_audio_url(url: str) -> bool:
return False
def _safe_manifest_url(url: str) -> bool:
try:
parsed = urlparse(url or "")
return parsed.scheme == "https" and parsed.hostname in _ALLOWED_MANIFEST_HOSTS
except Exception:
return False
def normalize_preset_instruct(instruct: str) -> Optional[tuple[str, dict]]:
"""Normalize one validator-safe tag per design category.
Membership in the vocabulary is not enough: ``male, female`` contains two
individually valid tokens but the engine rejects the pair as conflicting.
Build the frontend's full ``vd_states`` shape at this trust boundary too,
so Magic Wand never inherits stale sliders from the previous voice.
"""
attrs = {name: "Auto" for name in _ATTR_NAMES}
normalized: list[str] = []
seen_categories: set[int] = set()
for raw in re.split("[," + chr(0xFF0C) + "]", str(instruct or "")):
token = raw.strip().lower()
if not token or token not in _VALID_TOKENS:
return None
category = archetypes._VD._instruct_category_index(token)
if category < 0 or category in seen_categories:
return None
seen_categories.add(category)
# The picker represents the universal gender/age/pitch/style axes in
# English even for Chinese speech; dialect remains Chinese-only.
canonical = archetypes._VD._INSTRUCT_ZH_TO_EN.get(token, token)
attrs[_ATTR_NAMES[category]] = canonical
normalized.append(canonical)
if not normalized:
return None
# Accent and Chinese dialect are separate taxonomy buckets but the engine
# deliberately forbids mixing them in a single design.
if 4 in seen_categories and 5 in seen_categories:
return None
return ", ".join(normalized), attrs
def is_valid_instruct(instruct: str) -> bool:
toks = [t.strip() for t in (instruct or "").split(",") if t.strip()]
return bool(toks) and all(t in _VALID_TOKENS for t in toks)
return normalize_preset_instruct(instruct) is not None
def validate_item(raw: dict) -> Optional[dict]:
@@ -93,62 +170,203 @@ def validate_item(raw: dict) -> Optional[dict]:
it = dict(raw)
if it.get("type") not in ("preset", "voice"):
return None
if not it.get("id") or not it.get("name"):
if not isinstance(it.get("id"), str) or not _ITEM_ID_RE.fullmatch(it["id"]):
return None
if not isinstance(it.get("name"), str) or not it["name"].strip():
return None
it["name"] = it["name"].strip()[:80]
if it.get("use_case") not in _USE_CASE_IDS:
return None
if it["type"] == "preset" and not is_valid_instruct(it.get("instruct", "")):
return None # would crash synthesis — drop it
if it["type"] == "voice" and not _safe_audio_url((it.get("audio") or {}).get("url", "")):
return None
it.setdefault("facets", {})
raw_facets = it.get("facets")
if not isinstance(raw_facets, dict):
raw_facets = {}
language = it.get("language")
if not isinstance(language, str) or not language.strip():
language = raw_facets.get("lang", "English")
it["language"] = language.strip() if isinstance(language, str) and language.strip() else "English"
facets = dict(raw_facets)
if it["type"] == "preset":
normalized = normalize_preset_instruct(it.get("instruct", ""))
if normalized is None:
return None # unknown/conflicting tokens would crash synthesis
it["instruct"], it["attrs"] = normalized
attrs = it["attrs"]
facets.update({
"gender": None if attrs["Gender"] == "Auto" else attrs["Gender"],
"age": None if attrs["Age"] == "Auto" else attrs["Age"],
"pitch": None if attrs["Pitch"] == "Auto" else attrs["Pitch"],
"accent": None if attrs["EnglishAccent"] == "Auto" else attrs["EnglishAccent"],
"whisper": attrs["Style"] == "whisper",
"lang": it["language"],
})
sample_script = it.get("sample_script")
it["sample_script"] = (
sample_script.strip()[:_MAX_SAMPLE_SCRIPT_CHARS]
if isinstance(sample_script, str) else ""
)
else:
audio = it.get("audio")
if not isinstance(audio, dict) or not _safe_audio_url(audio.get("url", "")):
return None
expected = audio.get("sha256")
if expected is not None:
expected = str(expected).lower()
if not _SHA256_RE.fullmatch(expected):
return None
audio = {**audio, "sha256": expected}
ref_text = audio.get("ref_text")
audio = {
**audio,
"ref_text": (
ref_text.strip()[:_MAX_REF_TEXT_CHARS]
if isinstance(ref_text, str) else ""
),
}
it["audio"] = audio
facets.setdefault("gender", None)
facets.setdefault("age", None)
facets.setdefault("pitch", None)
facets.setdefault("accent", None)
facets.setdefault("whisper", False)
facets.setdefault("lang", it["language"])
it["facets"] = facets
it.setdefault("icon", archetypes._USE_ICON.get(it["use_case"], "Sparkles"))
it.setdefault("language", it.get("facets", {}).get("lang", "English"))
it["is_community"] = it.get("source") != "starter"
it["preview_url"] = f"/community/items/{it['id']}/preview"
return it
def _merge(manifests: list[tuple[str, Optional[dict]]]) -> tuple[list, list]:
items, packs, seen = [], [], set()
for src, m in manifests:
if not m:
if not isinstance(m, dict):
continue
for raw in (m.get("items") or []):
raw_items = m.get("items")
for raw in raw_items if isinstance(raw_items, list) else []:
v = validate_item(raw)
if v and v["id"] not in seen:
v["_source_repo"] = src
seen.add(v["id"])
items.append(v)
for p in (m.get("packs") or []):
raw_packs = m.get("packs")
for p in raw_packs if isinstance(raw_packs, list) else []:
if isinstance(p, dict):
packs.append({**p, "_source_repo": src})
return items, packs
def _fetch_manifest(source: str, refresh: bool) -> Optional[dict]:
"""Return a source's manifest from cache, or fetch + cache it. None if both fail."""
cache = _cache_path(source)
if not refresh and cache.exists():
try:
return json.loads(cache.read_text(encoding="utf-8"))
except Exception:
pass
def _read_manifest_cache(cache: Path) -> Optional[dict]:
try:
import httpx
with httpx.Client(timeout=15.0, follow_redirects=True) as client:
resp = client.get(_manifest_url(source))
resp.raise_for_status()
data = resp.json()
cache.parent.mkdir(parents=True, exist_ok=True)
cache.write_text(json.dumps(data), encoding="utf-8")
if cache.stat().st_size > _MAX_MANIFEST_BYTES:
return None
data = json.loads(cache.read_text(encoding="utf-8"))
return data if isinstance(data, dict) else None
except (OSError, ValueError, TypeError):
return None
def _write_bytes_atomic(path: Path, data: bytes) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
fd, tmp = tempfile.mkstemp(dir=str(path.parent), prefix=f".{path.name}-", suffix=".part")
try:
with os.fdopen(fd, "wb") as handle:
handle.write(data)
handle.flush()
os.fsync(handle.fileno())
os.replace(tmp, path)
except BaseException:
with contextlib.suppress(OSError):
os.unlink(tmp)
raise
def _fetch_remote_manifest(source: str, *, client=None) -> dict:
"""Fetch one bounded manifest, validating every redirect before request."""
import httpx
if not _SOURCE_RE.fullmatch(source or ""):
raise ValueError("invalid gallery source")
owned_client = client is None
http = client or httpx.Client(timeout=15.0, follow_redirects=False)
current_url = _manifest_url(source)
payload = bytearray()
try:
fetched = False
for _redirect in range(6):
if not _safe_manifest_url(current_url):
raise ValueError("gallery manifest URL is not from an allowed host")
with http.stream("GET", current_url, follow_redirects=False) as response:
if response.status_code in (301, 302, 303, 307, 308):
location = response.headers.get("location")
next_url = urljoin(current_url, location or "")
if not location or not _safe_manifest_url(next_url):
raise ValueError("gallery manifest redirected to a disallowed host")
current_url = next_url
continue
response.raise_for_status()
length = response.headers.get("content-length")
if length:
try:
declared_length = int(length)
except ValueError:
declared_length = None
if declared_length is not None and declared_length > _MAX_MANIFEST_BYTES:
raise ValueError("gallery manifest exceeded the size limit")
for chunk in response.iter_bytes():
if not chunk:
continue
if len(payload) + len(chunk) > _MAX_MANIFEST_BYTES:
raise ValueError("gallery manifest exceeded the size limit")
payload.extend(chunk)
fetched = True
break
if not fetched:
raise ValueError("gallery manifest followed too many redirects")
finally:
if owned_client:
http.close()
if not payload:
raise ValueError("gallery manifest was empty")
data = json.loads(payload)
if not isinstance(data, dict):
raise ValueError("gallery manifest is not a JSON object")
return data
def _fetch_manifest(
source: str, refresh: bool, *, now: Optional[float] = None,
) -> Optional[dict]:
"""Return a fresh manifest, with a throttled stale-cache offline fallback."""
cache = _cache_path(source)
cached = _read_manifest_cache(cache)
checked_at = time.time() if now is None else float(now)
if not refresh and cached is not None:
try:
if checked_at - cache.stat().st_mtime < _MANIFEST_MAX_AGE_S:
return cached
except OSError:
pass # treat a stat race as stale and try the source once
try:
data = _fetch_remote_manifest(source)
encoded = json.dumps(
data, ensure_ascii=False, separators=(",", ":"),
).encode("utf-8")
if len(encoded) > _MAX_MANIFEST_BYTES:
raise ValueError("gallery manifest exceeded the cache size limit")
_write_bytes_atomic(cache, encoded)
# Tests inject their own clock; production's value equals wall time.
os.utime(cache, (checked_at, checked_at))
return data
except Exception as e: # offline / 404 / bad json
logger.warning("manifest fetch failed for %s: %s", source, e)
if cache.exists():
try:
return json.loads(cache.read_text(encoding="utf-8"))
except Exception:
pass
if cached is not None:
# This mtime is a last-*check* marker. Advancing it on failure keeps
# an offline app responsive while guaranteeing another check after
# the bounded freshness interval.
with contextlib.suppress(OSError):
os.utime(cache, (checked_at, checked_at))
return cached
return None
@@ -214,6 +432,385 @@ def community_submit_url(item_type: str = Query("preset", alias="type"), source:
return {"url": f"https://github.com/{src}/issues/new?template={template}"}
def _find_item(items: list[dict], item_id: str) -> dict:
if not _ITEM_ID_RE.fullmatch(item_id or ""):
raise HTTPException(status_code=404, detail="Item not found in the gallery.")
item = next((it for it in items if it["id"] == item_id), None)
if item is None:
raise HTTPException(status_code=404, detail="Item not found in the gallery.")
return item
def _canonical_archetype(item: dict) -> Optional[dict]:
"""The built-in archetype represented exactly by a marketplace preset."""
if item.get("type") != "preset":
return None
canonical = archetypes.get_archetype(item["id"])
if canonical is None:
return None
if (canonical.get("instruct") != item.get("instruct")
or canonical.get("language") != item.get("language")):
return None
remote_script = (item.get("sample_script") or "").strip()
if remote_script and remote_script != (canonical.get("sample_script") or "").strip():
return None
return canonical
def _preset_preview_path(item: dict) -> Path:
fingerprint = hashlib.sha256(
json.dumps({
"instruct": item.get("instruct"),
"language": item.get("language"),
"sample_script": item.get("sample_script"),
}, sort_keys=True).encode("utf-8")
).hexdigest()[:16]
return _CACHE_DIR / "previews" / f"{item['id']}-{fingerprint}.wav"
def _voice_audio_fingerprint(item: dict) -> str:
audio = item.get("audio") or {}
return hashlib.sha256(
f"{audio.get('url', '')}|{audio.get('sha256', '')}".encode("utf-8")
).hexdigest()[:16]
def _voice_audio_path(item: dict) -> Path:
return _CACHE_DIR / "audio" / f"{item['id']}-{_voice_audio_fingerprint(item)}.wav"
async def _render_preset_atomic(item: dict, out_path: Path) -> Path:
if is_playable_wav(out_path):
return out_path
from api.routers.archetypes import _render_archetype_wav
out_path.parent.mkdir(parents=True, exist_ok=True)
fd, tmp_name = tempfile.mkstemp(dir=str(out_path.parent), prefix=".preview-", suffix=".wav")
os.close(fd)
tmp = Path(tmp_name)
try:
await _render_archetype_wav({
"instruct": item["instruct"],
"language": item.get("language", "English"),
"sample_script": (
(item.get("sample_script") or "").strip()
or "Hello — this is a preview of this voice."
),
}, tmp)
if not is_playable_wav(tmp):
raise RuntimeError("the voice engine produced an invalid preview WAV")
os.replace(tmp, out_path)
return out_path
finally:
with contextlib.suppress(OSError):
tmp.unlink()
def _download_voice_audio(item: dict, out_path: Path, *, client=None) -> None:
"""Stream one allow-listed voice clip into an atomic, size-bounded file."""
audio = item.get("audio") or {}
url = audio.get("url", "")
if not _safe_audio_url(url):
raise HTTPException(status_code=400, detail="Voice audio URL is not from an allowed host.")
import httpx
owned_client = client is None
http = client or httpx.Client(timeout=30.0, follow_redirects=False)
out_path.parent.mkdir(parents=True, exist_ok=True)
fd, tmp_name = tempfile.mkstemp(dir=str(out_path.parent), prefix=".voice-", suffix=".part")
total = 0
digest = hashlib.sha256()
try:
with os.fdopen(fd, "wb") as handle:
current_url = url
downloaded = False
for _redirect in range(6):
with http.stream("GET", current_url, follow_redirects=False) as response:
if response.status_code in (301, 302, 303, 307, 308):
location = response.headers.get("location")
next_url = urljoin(current_url, location or "")
if not location or not _safe_audio_url(next_url):
raise HTTPException(
status_code=502,
detail="Community voice audio redirected to a disallowed host.",
)
current_url = next_url
continue
response.raise_for_status()
length = response.headers.get("content-length")
if length:
try:
if int(length) > _MAX_VOICE_AUDIO_BYTES:
raise HTTPException(
status_code=502,
detail="Community voice audio exceeded the download size limit.",
)
except ValueError:
# A non-numeric Content-Length header is the
# server's problem, not a reason to refuse the
# download — the streamed byte counter below
# still enforces the same cap on what actually
# arrives.
pass
for chunk in response.iter_bytes():
if not chunk:
continue
total += len(chunk)
if total > _MAX_VOICE_AUDIO_BYTES:
raise HTTPException(
status_code=502,
detail="Community voice audio exceeded the download size limit.",
)
digest.update(chunk)
handle.write(chunk)
downloaded = True
break
if not downloaded:
raise HTTPException(
status_code=502,
detail="Community voice audio followed too many redirects.",
)
if total == 0:
raise HTTPException(status_code=502, detail="Community voice audio was empty.")
expected = audio.get("sha256")
if expected and digest.hexdigest() != expected:
raise HTTPException(
status_code=502,
detail="Downloaded voice failed its integrity check.",
)
handle.flush()
os.fsync(handle.fileno())
if not is_playable_wav(Path(tmp_name)):
raise HTTPException(
status_code=502, detail="Community voice audio was not a valid WAV.",
)
os.replace(tmp_name, out_path)
except BaseException:
with contextlib.suppress(OSError):
os.unlink(tmp_name)
raise
finally:
if owned_client:
http.close()
def _cached_voice_audio(item: dict) -> Path:
path = _voice_audio_path(item)
if is_playable_wav(path):
return path
with contextlib.suppress(OSError):
path.unlink()
_download_voice_audio(item, path)
return path
def _copy_atomic(source: Path, destination: Path) -> None:
destination.parent.mkdir(parents=True, exist_ok=True)
fd, tmp_name = tempfile.mkstemp(
dir=str(destination.parent), prefix=f".{destination.name}-", suffix=".part",
)
try:
with os.fdopen(fd, "wb") as out, source.open("rb") as src:
shutil.copyfileobj(src, out)
out.flush()
os.fsync(out.fileno())
os.replace(tmp_name, destination)
except BaseException:
with contextlib.suppress(OSError):
os.unlink(tmp_name)
raise
@router.get("/community/items/{item_id}/preview")
async def community_preview(
item_id: str,
local: bool = Query(False, description="Bypass canonical gallery audio after decode failure"),
):
"""Serve every community preview through the authenticated same-origin API."""
_, items, _, _ = await asyncio.to_thread(_load, False)
item = _find_item(items, item_id)
canonical = _canonical_archetype(item)
if canonical is not None:
# Reuse the signed-gallery/local-render fallback and cache owned by the
# canonical endpoint rather than synthesizing the same preset twice.
# Delegate in-process: a root-relative HTTP redirect drops supported
# reverse-proxy path prefixes such as ``https://host/api``.
from api.routers.archetypes import preview_archetype
return await preview_archetype(canonical["id"], local=local)
try:
if item["type"] == "preset":
path = await _render_preset_atomic(item, _preset_preview_path(item))
else:
path = await asyncio.to_thread(_cached_voice_audio, item)
except HTTPException:
raise
except Exception as exc:
logger.warning("Community preview unavailable (%s)", type(exc).__name__)
raise HTTPException(
status_code=503, detail="This community voice preview is unavailable right now.",
) from exc
return FileResponse(
path, media_type="audio/wav",
headers={"Cache-Control": "no-cache", "X-OmniVoice-Preview-Source": "community"},
)
def _profile_fields(item: dict) -> tuple[str, str, Optional[str], Optional[int]]:
if item["type"] == "preset":
return "design", item["instruct"], json.dumps(item["attrs"]), 42
return "clone", "", None, None
def _community_profile_audio_filename(profile_id: str, item: dict) -> str:
safe_id = (
profile_id if re.fullmatch(r"[A-Za-z0-9_-]{1,64}", profile_id or "")
else hashlib.sha256(str(profile_id).encode("utf-8")).hexdigest()[:16]
)
if item["type"] == "voice":
# The manifest URL/checksum fingerprint makes a changed submission
# invalidate its already-materialized clone without a schema change.
return f"{safe_id}-community-{_voice_audio_fingerprint(item)}.wav"
return f"{safe_id}.wav"
def _stored_profile_audio(ref_audio_path: object) -> Optional[Path]:
return resolve_regular_file(VOICES_DIR, ref_audio_path)
def _community_audio_is_current(row, item: dict, ref_text: str) -> bool:
path = _stored_profile_audio(row["ref_audio_path"])
expected_filename = _community_profile_audio_filename(row["id"], item)
if row["ref_audio_path"] != expected_filename or not is_playable_wav(path):
return False
kind, instruct, _vd_states, seed = _profile_fields(item)
inputs_match = (
row["instruct"] == instruct
and row["language"] == item.get("language", "Auto")
and row["ref_text"] == ref_text
and row["seed"] == seed
)
if not inputs_match:
return False
return True
async def _materialize_item_audio(
item: dict, profile_id: str, *, publish: bool = True,
) -> tuple[str, Path]:
"""Copy the current manifest audio, optionally staging it for a later CAS."""
audio_filename = _community_profile_audio_filename(profile_id, item)
destination = Path(VOICES_DIR) / audio_filename
audio_path = destination
if not publish:
destination.parent.mkdir(parents=True, exist_ok=True)
audio_path = destination.parent / f".{Path(audio_filename).stem}-{uuid.uuid4().hex}.staged.wav"
if item["type"] == "preset":
cached = await _render_preset_atomic(item, _preset_preview_path(item))
else:
cached = await asyncio.to_thread(_cached_voice_audio, item)
await asyncio.to_thread(_copy_atomic, cached, audio_path)
return audio_filename, audio_path
def _community_personality(item: dict) -> str:
source = item.get("_source_repo")
if not isinstance(source, str) or not _SOURCE_RE.fullmatch(source):
source = _DEFAULT_SOURCES[0]
return f"community:{source}:{item['id']}"
def _is_materialized_community_row(row, item: dict) -> bool:
if (
row["personality"] != _community_personality(item)
or row["is_locked"] or row["verified_own_voice"]
):
return False
if item["type"] == "voice":
safe_id = Path(_community_profile_audio_filename(row["id"], item)).name.split(
"-community-", 1,
)[0]
return bool(
row["kind"] == "clone"
and row["seed"] is None
and not row["vd_states"]
and row["instruct"] == ""
and row["language"] == item.get("language", "Auto")
and row["ref_text"] == (item.get("audio") or {}).get("ref_text", "")
and re.fullmatch(
rf"{re.escape(safe_id)}-community-[0-9a-f]{{16}}\.wav",
row["ref_audio_path"] or "",
)
)
try:
states = json.loads(row["vd_states"])
except (TypeError, ValueError):
return False
return bool(
row["kind"] == "design"
and row["seed"] == 42
and row["ref_audio_path"] == _community_profile_audio_filename(row["id"], item)
and row["instruct"] == item["instruct"]
and row["language"] == item.get("language", "Auto")
and row["ref_text"] == (item.get("sample_script") or "")
and states == item["attrs"]
)
def _existing_community_profile(conn, item: dict, personality: str):
candidates = conn.execute(
"SELECT * FROM voice_profiles WHERE personality=? ORDER BY created_at, id",
(personality,),
).fetchall()
existing = next(
(row for row in candidates if _is_materialized_community_row(row, item)), None,
)
if existing is not None:
return existing
# Old builds stored the bare item id. Import formats preserve arbitrary
# personality text too, so adopt only the exact shape the old materializer
# wrote; otherwise a remote item id could rewrite a user's imported voice.
if archetypes.get_archetype(item["id"]) is None:
legacy = conn.execute(
"SELECT * FROM voice_profiles WHERE personality=? LIMIT 1",
(item["id"],),
).fetchone()
if legacy is not None:
kind, instruct, _vd_states, _seed = _profile_fields(item)
ref_text = item.get("sample_script") or (item.get("audio") or {}).get(
"ref_text", "",
)
if (
legacy["ref_audio_path"] == f"{legacy['id']}.wav"
and legacy["kind"] == kind
and legacy["instruct"] == instruct
and legacy["language"] == item.get("language", "Auto")
and legacy["ref_text"] == ref_text
and legacy["seed"] is None
and not legacy["vd_states"]
and not legacy["is_locked"]
and not legacy["verified_own_voice"]
):
return legacy
return None
def _heal_existing_profile(
conn, row, item: dict, ref_text: str, personality: str, audio_filename: str,
) -> None:
kind, instruct, vd_states, seed = _profile_fields(item)
conn.execute(
"UPDATE voice_profiles SET kind=?, instruct=?, vd_states=?, language=?, "
"ref_text=?, seed=?, personality=?, ref_audio_path=? WHERE id=?",
(
kind, instruct, vd_states, item.get("language", "Auto"), ref_text,
seed, personality, audio_filename, row["id"],
),
)
@router.post("/community/items/{item_id}/use")
async def community_use(item_id: str, name: Optional[str] = Query(None)):
"""Materialize a community item into a reusable voice profile.
@@ -223,76 +820,108 @@ async def community_use(item_id: str, name: Optional[str] = Query(None)):
``voice_profiles`` row usable everywhere voices are picked.
"""
_, items, _, _ = await asyncio.to_thread(_load, False)
item = next((it for it in items if it["id"] == item_id), None)
if item is None:
raise HTTPException(status_code=404, detail="Item not found in the gallery.")
item = _find_item(items, item_id)
canonical = _canonical_archetype(item)
if canonical is not None:
from api.routers.archetypes import use_archetype
return await use_archetype(canonical["id"], name)
import time
import uuid
from core import event_bus
from core.db import db_conn
from core.config import VOICES_DIR
profile_id = str(uuid.uuid4())[:8]
audio_filename = f"{profile_id}.wav"
audio_path = Path(VOICES_DIR) / audio_filename
profile_name = (name or item["name"]).strip() or item["name"]
instruct = item.get("instruct", "") if item["type"] == "preset" else ""
ref_text = item.get("sample_script") or (item.get("audio") or {}).get("ref_text", "")
personality = _community_personality(item)
with db_conn() as conn:
existing = _existing_community_profile(conn, item, personality)
try:
if item["type"] == "preset":
from api.routers.archetypes import _render_archetype_wav
pseudo = {
"instruct": instruct,
"language": item.get("language", "English"),
"sample_script": ref_text or "Hello — this is a preview of this voice.",
}
await _render_archetype_wav(pseudo, audio_path)
else: # voice — download the reference clip (off the event loop)
await asyncio.to_thread(_download_voice_audio, item, audio_path)
except HTTPException:
raise
except Exception as e:
logger.error("Community 'use' failed", exc_info=True)
raise HTTPException(status_code=503, detail=f"Couldn't add this voice right now. Error: {e}")
try:
# A community "preset" is a synthetic designed voice (rendered from an
# instruct string) → kind='design'; a "voice" carries a real reference
# clip → kind='clone'. Setting kind makes the persona-gallery
# synthetic-only gating work (§R3) instead of defaulting all imports to
# 'clone'.
kind = "design" if item["type"] == "preset" else "clone"
with db_conn() as conn:
conn.execute(
"INSERT INTO voice_profiles "
"(id, name, ref_audio_path, ref_text, instruct, language, seed, personality, created_at, kind) "
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
(profile_id, profile_name, audio_filename, ref_text, instruct,
item.get("language", "Auto"), None, item["id"], time.time(), kind),
profile_id = existing["id"] if existing is not None else str(uuid.uuid4())[:8]
audio_path: Optional[Path] = None
if existing is not None and _community_audio_is_current(existing, item, ref_text):
audio_filename = existing["ref_audio_path"]
else:
try:
audio_filename, audio_path = await _materialize_item_audio(
item, profile_id, publish=existing is None,
)
except HTTPException:
raise
except Exception as e:
logger.error("Community 'use' failed", exc_info=True)
raise HTTPException(
status_code=503, detail="Couldn't add this voice right now.",
) from e
if existing is not None:
with db_conn() as conn:
conn.execute("BEGIN IMMEDIATE")
current = conn.execute(
"SELECT * FROM voice_profiles WHERE id=?", (existing["id"],),
).fetchone()
owned = _existing_community_profile(conn, item, personality)
still_owned = current is not None and (
_is_materialized_community_row(current, item)
or (owned is not None and owned["id"] == current["id"])
)
if still_owned:
if audio_path is not None:
destination = Path(VOICES_DIR) / audio_filename
os.replace(audio_path, destination)
audio_path = None
_heal_existing_profile(
conn, current, item, ref_text, personality, audio_filename,
)
existing_result = {"profile_id": current["id"], "name": current["name"]}
else:
existing_result = None
if existing_result is not None:
event_bus.emit("profiles", {"action": "updated", "id": existing_result["profile_id"]})
return existing_result
profile_id = str(uuid.uuid4())[:8]
audio_filename = _community_profile_audio_filename(profile_id, item)
destination = Path(VOICES_DIR) / audio_filename
if audio_path is None:
audio_filename, audio_path = await _materialize_item_audio(item, profile_id)
else:
os.replace(audio_path, destination)
audio_path = destination
if audio_path is None: # defensive: a new profile always materialized above
raise RuntimeError("new community profile has no materialized audio")
profile_name = (name or item["name"]).strip() or item["name"]
kind, instruct, vd_states, seed = _profile_fields(item)
try:
with db_conn() as conn:
conn.execute("BEGIN IMMEDIATE")
duplicate = _existing_community_profile(conn, item, personality)
if duplicate is not None:
duplicate_audio = duplicate["ref_audio_path"]
if not _community_audio_is_current(duplicate, item, ref_text):
duplicate_audio = _community_profile_audio_filename(duplicate["id"], item)
duplicate_path = Path(VOICES_DIR) / duplicate_audio
_copy_atomic(audio_path, duplicate_path)
_heal_existing_profile(
conn, duplicate, item, ref_text, personality, duplicate_audio,
)
with contextlib.suppress(OSError):
audio_path.unlink()
duplicate_result = {"profile_id": duplicate["id"], "name": duplicate["name"]}
else:
duplicate_result = None
if duplicate_result is None:
conn.execute(
"INSERT INTO voice_profiles "
"(id, name, ref_audio_path, ref_text, instruct, language, seed, personality, "
"created_at, kind, vd_states) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
(profile_id, profile_name, audio_filename, ref_text, instruct,
item.get("language", "Auto"), seed, personality, time.time(), kind, vd_states),
)
except Exception:
with __import__("contextlib").suppress(OSError):
os.remove(audio_path)
with contextlib.suppress(OSError):
audio_path.unlink()
raise
if duplicate_result is not None:
event_bus.emit("profiles", {"action": "updated", "id": duplicate_result["profile_id"]})
return duplicate_result
event_bus.emit("profiles", {"action": "created", "id": profile_id})
return {"profile_id": profile_id, "name": profile_name}
def _download_voice_audio(item: dict, out_path: Path) -> None:
import hashlib
audio = item.get("audio") or {}
url = audio.get("url", "")
if not _safe_audio_url(url):
raise HTTPException(status_code=400, detail="Voice audio URL is not from an allowed host.")
import httpx
with httpx.Client(timeout=30.0, follow_redirects=True) as client:
resp = client.get(url)
resp.raise_for_status()
data = resp.content
expected = audio.get("sha256")
if expected and hashlib.sha256(data).hexdigest() != expected:
raise HTTPException(status_code=502, detail="Downloaded voice failed its integrity check.")
out_path.parent.mkdir(parents=True, exist_ok=True)
out_path.write_bytes(data)
+347 -38
View File
@@ -134,6 +134,82 @@ _save_job = dub_pipeline.save_job
# paste (or a mis-aimed binary) burn CPU in the parser.
_MAX_SUBTITLE_PASTE_CHARS = 2_000_000
_SRT_REPLACED_FIELDS = {
"id",
"start",
"end",
"text",
"text_original",
"translations",
"translate_error",
"translate_degraded",
}
def _best_overlapping_segment(cue: dict, existing: list[dict]) -> dict | None:
"""Return the prior segment with the strongest temporal overlap."""
cue_start = float(cue.get("start") or 0.0)
cue_end = float(cue.get("end") or cue_start)
cue_mid = (cue_start + cue_end) / 2.0
best = None
best_key = None
for index, segment in enumerate(existing):
start = float(segment.get("start") or 0.0)
end = float(segment.get("end") or start)
overlap = min(cue_end, end) - max(cue_start, start)
if overlap <= 0:
continue
midpoint_distance = abs(cue_mid - ((start + end) / 2.0))
key = (overlap, -midpoint_distance, -index)
if best_key is None or key > best_key:
best = segment
best_key = key
return best
def _carry_srt_voice_metadata(
cues: list[dict],
existing: list[dict],
segment_clones: dict | None,
speaker_clones: dict | None = None,
) -> tuple[list[dict], dict]:
"""Replace subtitle content while retaining the source cast assignment."""
source_clones = dict(segment_clones or {})
source_speaker_clones = dict(speaker_clones or {})
# Replacement cues get new positional ids. Starting from the old map would
# let an unmatched cue whose new id happens to equal an old id inherit an
# unrelated reference. Only explicitly overlap-matched references survive.
clones = {}
merged_segments = []
for new_id, cue in enumerate(cues):
prior = _best_overlapping_segment(cue, existing)
metadata = {
key: value
for key, value in (prior or {}).items()
if key not in _SRT_REPLACED_FIELDS
}
merged = {
**metadata,
"id": new_id,
"start": cue.get("start", 0.0),
"end": cue.get("end", 0.0),
"text": cue.get("text", ""),
"text_original": cue.get("text", ""),
}
if not merged.get("speaker_id"):
merged["speaker_id"] = cue.get("speaker_id") or "Speaker 1"
if prior is not None:
prior_id = str(prior.get("id", ""))
clone = source_clones.get(prior_id)
if clone is None:
clone = source_speaker_clones.get(prior.get("speaker_id"))
if clone is not None:
clones[str(new_id)] = clone
if merged.get("profile_id") == f"auto-seg:{prior_id}":
merged["profile_id"] = f"auto-seg:{new_id}"
merged_segments.append(merged)
return merged_segments, clones
@router.post("/dub/parse-subtitle-text")
def dub_parse_subtitle_text(req: ParseSubtitleTextRequest):
@@ -234,7 +310,32 @@ async def dub_import_srt(job_id: str, file: UploadFile = File(...)):
else:
segments = result.segments
prior_segments = [
segment for segment in (job.get("segments") or []) if isinstance(segment, dict)
]
segments, segment_clones = _carry_srt_voice_metadata(
segments,
prior_segments,
job.get("segment_clones"),
job.get("speaker_clones"),
)
job["segments"] = segments
job["segment_clones"] = segment_clones
# A pooled speaker clone is keyed only by a display label. Replacement
# cues can reuse that label without overlapping the original speaker, so
# retain matched pooled references as segment-specific clones above and
# drop the global map before rebuilding the cast.
job["speaker_clones"] = {}
if segment_clones:
from services.speaker_clone import build_cast_sources
job["cast_sources"] = build_cast_sources(
segments,
None,
segment_clones,
)
else:
job.pop("cast_sources", None)
# `source_lang` stays whatever the user (or the upload step) set; we
# don't try to language-detect off the cue text — that's noisy and the
# user usually knows what their .srt is.
@@ -351,12 +452,13 @@ async def preview_upload(video: UploadFile = File(...)):
safe_name = f"{uuid.uuid4().hex[:12]}"
vid_path = os.path.join(PREVIEW_DIR, f"{safe_name}{ext}")
wav_path = os.path.join(PREVIEW_DIR, f"{safe_name}.wav")
with open(vid_path, "wb") as f:
f.write(await video.read())
has_audio = False
if ext not in [".wav", ".mp3", ".m4a", ".aac"]:
payload = await video.read()
def _write_and_extract() -> bool:
with open(vid_path, "wb") as f:
f.write(payload)
if ext in {".wav", ".mp3", ".m4a", ".aac"}:
return False
try:
ffmpeg_cmd = [
find_ffmpeg(), "-y", "-i", vid_path,
@@ -368,10 +470,16 @@ async def preview_upload(video: UploadFile = File(...)):
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
timeout=300,
)
has_audio = True
return True
except Exception as e:
logger.warning("FFmpeg extraction failed: %s", log_safe(e))
pass
return False
# File writes and ffmpeg are blocking operations. Keep them on the bounded
# CPU pool so a large preview cannot stall unrelated API requests (#1667).
has_audio = await asyncio.get_running_loop().run_in_executor(
_cpu_pool, _write_and_extract
)
return {
"url": f"/preview/{safe_name}{ext}",
@@ -410,12 +518,52 @@ _ingest_gen = dub_pipeline.ingest_pipeline
#: container so a mislabelled video can't slip past the video-skipping branch.
_AUDIO_EXTS = {".wav", ".mp3", ".m4a", ".aac", ".flac", ".ogg", ".opus", ".wma"}
# Source-language choices exposed by the first-party dub UI, plus every
# language code Whisper can write back after auto-detection. A restored job
# may reuse that detected value as the next upload's override, so rejecting our
# own persisted codes strands otherwise valid dubbing sessions (#1737).
# Keeping this an allow-list still rejects language names and private-use
# BCP-47 tags. Values are normalized to lowercase below.
_DUB_SOURCE_LANG_CODES = frozenset({
"af", "sq", "am", "ar", "hy", "az", "eu", "be", "bn", "bs", "bg",
"my", "ca", "cmn-hans", "cmn-hant", "hr", "cs", "da", "nl", "en",
"et", "fi", "fr", "gl", "ka", "de", "el", "gu", "ht", "ha", "haw",
"he", "hi", "hu", "is", "id", "it", "ja", "jw", "kn", "kk", "km",
"ko", "ku", "ky", "lo", "la", "lv", "lt", "mk", "ms", "ml", "mt",
"mi", "mr", "mn", "ne", "no", "ps", "fa", "pl", "pt", "pa", "ro",
"ru", "sm", "gd", "sr", "sn", "sd", "si", "sk", "sl", "so", "es",
"su", "sw", "sv", "tg", "ta", "te", "th", "tr", "uk", "ur", "uz",
"vi", "cy", "xh", "yi", "yo", "zu",
"as", "ba", "bo", "br", "fo", "lb", "ln", "mg", "nn", "oc", "sa",
"tk", "tl", "tt", "yue", "zh",
})
def _source_lang_override(value: str | None) -> str | None:
"""Normalize a user-selected source language; auto/und means detect."""
code = (value or "").strip().lower()
if code in {"", "auto", "und"}:
return None
if code not in _DUB_SOURCE_LANG_CODES:
raise HTTPException(status_code=400, detail="Invalid source language code")
return code
def _detected_source_lang(value: str | None) -> str:
"""Normalize an ASR language without truncating valid three-letter codes."""
code = (value or "en").split("_", 1)[0].strip().lower()
if code in _DUB_SOURCE_LANG_CODES:
return code
short = code[:2]
return short if short in _DUB_SOURCE_LANG_CODES else "en"
@router.post("/dub/upload")
async def dub_upload(
video: UploadFile = File(...),
job_id: Optional[str] = Form(None),
input_type: str = Form("video"),
source_lang: Optional[str] = Form(None),
):
"""Accept a media upload, write to disk, queue background prep task.
@@ -445,6 +593,7 @@ async def dub_upload(
detail=f"Audio-only dubbing needs an audio file ({', '.join(sorted(_AUDIO_EXTS))}); got '{ext or 'no extension'}'.",
)
source_lang_override = _source_lang_override(source_lang)
os.makedirs(job_dir, exist_ok=True)
video_path = os.path.join(job_dir, f"original{ext}")
@@ -456,7 +605,13 @@ async def dub_upload(
await task_manager.add_task(
task_id, "prep",
_ingest_gen, job_id, job_dir,
{"kind": "file", "path": video_path, "input_type": input_type}, filename,
{
"kind": "file",
"path": video_path,
"input_type": input_type,
"source_lang": source_lang_override,
},
filename,
)
return JSONResponse(
status_code=202,
@@ -478,6 +633,7 @@ async def dub_ingest_url(req: DubIngestUrlRequest, request: Request):
status_code=400,
detail="URL must start with http:// or https://. Paste a full video link (e.g. https://youtube.com/watch?v=…) or drop a local file instead.",
)
source_lang_override = _source_lang_override(req.source_lang)
try:
import yt_dlp # noqa: F401
@@ -513,6 +669,7 @@ async def dub_ingest_url(req: DubIngestUrlRequest, request: Request):
"fetch_subs": bool(req.fetch_subs),
"sub_langs": req.sub_langs or None,
"cookie_file": cookie_path,
"source_lang": source_lang_override,
}
try:
await task_manager.add_task(
@@ -557,7 +714,7 @@ _prep_event_helper = dub_pipeline.prep_event # alias; we keep the module-local
#: into one reference, which is how "made up" clone voices happen).
CLONE_SKIP_HEURISTIC_MSG = (
"auto voice cloning skipped: speaker labels are gap-based estimates — "
"set up diarization (Settings → Models → pyannote) for per-speaker clones"
"set up diarization (Model Catalogue → Models → pyannote) for per-speaker clones"
)
@@ -577,6 +734,118 @@ def _clamp_num_speakers(value) -> Optional[int]:
return value if 1 <= value <= 20 else None
def _recover_from_phrase_embeddings(
diar_pipe,
diarized_segments: list[dict],
*,
phrases: list[dict],
requested_speakers: int | None,
audio_target: str,
segments: list[dict],
words: list,
):
"""Recover rapid turns when pyannote collapses a two-speaker exchange.
Uses ASR phrase boundaries and the embedding/audio components already
loaded by speaker-diarization-3.1. Weak or imbalanced clusters are rejected
so ordinary single-speaker recordings remain untouched. Returns
``(segments, separation)`` or ``None``.
"""
present = {
str(seg.get("speaker_id")) for seg in diarized_segments
if seg.get("speaker_id")
}
if len(present) > 1:
return None
usable_phrases = [
phrase for phrase in phrases
if phrase.get("text")
and float(phrase.get("end", 0.0)) - float(phrase.get("start", 0.0)) >= 0.75
]
if len(usable_phrases) < 4:
return None
requested = int(requested_speakers) if requested_speakers else 2
if requested != 2:
return None
embedding = getattr(diar_pipe, "_embedding", None)
audio = getattr(diar_pipe, "_audio", None)
if embedding is None or audio is None:
return None
try:
import numpy as np
from pyannote.core import Segment as _PyannoteSegment
from sklearn.cluster import AgglomerativeClustering
vectors = []
durations = []
for phrase in usable_phrases:
start, end = float(phrase["start"]), float(phrase["end"])
duration = end - start
waveform, _ = audio.crop(
audio_target, _PyannoteSegment(start, end),
duration=duration, mode="pad",
)
vector = np.asarray(embedding(waveform[None])).reshape(-1)
if not np.isfinite(vector).all():
return None
vectors.append(vector)
durations.append(duration)
matrix = np.vstack(vectors)
labels = np.asarray(AgglomerativeClustering(
n_clusters=2, metric="cosine", linkage="average",
).fit_predict(matrix))
if len(set(labels.tolist())) != 2:
return None
counts = [int(np.sum(labels == cluster)) for cluster in (0, 1)]
cluster_durations = [
float(sum(duration for duration, label in zip(durations, labels) if label == cluster))
for cluster in (0, 1)
]
if min(counts) < 2 or min(cluster_durations) < 1.5:
return None
normalized = matrix / np.maximum(np.linalg.norm(matrix, axis=1, keepdims=True), 1e-8)
similarities = normalized @ normalized.T
within, cross = [], []
for left in range(len(labels)):
for right in range(left + 1, len(labels)):
target = within if labels[left] == labels[right] else cross
target.append(float(similarities[left, right]))
if not within or not cross:
return None
separation = float(np.mean(within) - np.mean(cross))
min_separation = 0.12 if requested_speakers == 2 else 0.18
if separation < min_separation:
logger.info(
"phrase-embedding speaker recovery rejected (separation=%.3f < %.3f)",
separation, min_separation,
)
return None
speaker_map = {}
turns = []
for phrase, label in zip(usable_phrases, labels.tolist()):
if label not in speaker_map:
speaker_map[label] = f"Speaker {len(speaker_map) + 1}"
turns.append({
"start": float(phrase["start"]),
"end": float(phrase["end"]),
"speaker": speaker_map[label],
})
# Assignment mutates segment dictionaries. Work on copies so a recovery
# rejected by the final two-speaker check cannot leak partial labels
# into the ordinary pyannote result.
assigned = assign_speakers_from_turns([dict(item) for item in segments], turns)
recovered = resplit_segments_by_turns(assigned, words, turns)
if len({item.get("speaker_id") for item in recovered if item.get("speaker_id")}) < 2:
return None
return recovered, separation
except Exception:
logger.exception("phrase-embedding speaker recovery failed")
return None
@router.get("/dub/transcribe-stream/{job_id}")
async def dub_transcribe_stream(
job_id: str,
@@ -912,6 +1181,12 @@ async def dub_transcribe_stream(
# Words (global-timeline) retained so diarization can re-split a segment
# that spans two speakers' turns at the word boundary (#486).
all_words: list = []
# Preserve the ASR backend's natural phrase boundaries before
# segment_transcript merges short neighboring phrases. Pyannote 3.1
# occasionally collapses rapid exchanges into one dominant speaker; in
# that narrow case these phrase spans give its own WeSpeaker embedding
# model clean candidate utterances for a conservative recovery pass.
asr_phrase_segments: list[dict] = []
detected_lang = None
next_seg_id = 0
chunk_errors: list[str] = []
@@ -981,21 +1256,13 @@ async def dub_transcribe_stream(
"error_code": failure["code"],
}
# Retry a failed/timed-out chunk once on a fresh pool before giving
# up. Otherwise a transient wedge on the FIRST chunk (whisperx often
# cold-loads its model there, the #730 hang) drops that whole window
# and the transcript is "missing the beginning, only middle+end".
# The retry reuses the same audio window, so a recovered chunk fills
# the hole instead of leaving silent gaps.
# Retry an ordinary completed failure once. A timed-out native call
# is different: its thread is still executing and must not overlap
# a retry against the same backend (#1669).
part = None
timed_out = False
for _attempt in range(1, _CHUNK_TRANSCRIBE_ATTEMPTS + 1):
# A wedged chunk gets the SAME guarded-timeout + pool-reset
# semantics as the whole-file paths (#730/#851):
# run_transcribe_guarded bounds the call, abandons the poisoned
# pool so the retry (and any concurrent TTS work) gets a fresh
# worker, and raises the actionable ASRTimeoutError. Run it as
# a task and poll so we can keep yielding pings — the
# EventSource connection drops without them.
# Run as a task and poll so pings keep the EventSource alive.
task = asyncio.ensure_future(run_transcribe_guarded(
_gpu_pool, _transcribe_chunk,
what=f"Dub chunk {i + 1}/{chunks_n}",
@@ -1010,9 +1277,12 @@ async def dub_transcribe_stream(
try:
part = task.result()
except ASRTimeoutError:
# The guard already reset the pool; keep the actionable
# message (it names the durable fixes, and — after repeated
# timeouts — the crash-isolated engine escape hatch).
# Python cannot kill an in-process native transcribe. Do
# not swap pools and retry over the still-running call:
# concurrent whisperx/CTranslate2 access caused the native
# Windows access violation in #1669. Stop this transcript;
# the worker remains honestly occupied until it exits.
timed_out = True
logger.error(
"Transcribe chunk %d/%d timed out after %.0fs (attempt %d/%d, job=%s)",
i + 1, chunks_n, transcribe_timeout_s, _attempt,
@@ -1031,23 +1301,36 @@ async def dub_transcribe_stream(
# error-part; the timeout path already reset the pool).
if part is not None and not part.get("error"):
break
if _attempt < _CHUNK_TRANSCRIBE_ATTEMPTS:
if timed_out:
break
if not timed_out and _attempt < _CHUNK_TRANSCRIBE_ATTEMPTS:
logger.warning(
"Retrying transcribe chunk %d/%d after failure/timeout (next attempt %d/%d, job=%s)",
i + 1, chunks_n, _attempt + 1, _CHUNK_TRANSCRIBE_ATTEMPTS, log_safe(job_id),
)
# A completed exception did not wedge the worker. Resetting
# the pool here leaked a healthy executor on every ordinary
# decode failure; run_transcribe_guarded already resets the
# pool on the only case that needs it: a real timeout.
# A completed exception did not leave native work behind,
# so retrying this same audio window is safe.
if part.get("error"):
chunk_errors.append(part["error"])
if part.get("error_code"):
chunk_error_codes.append(part["error_code"])
logger.warning("Chunk %d/%d error: %s", i + 1, chunks_n, log_safe(part["error"]))
if timed_out:
break
if detected_lang is None and part.get("language"):
detected_lang = part["language"]
asr_speaker_turns.extend(part.get("speaker_turns") or [])
for _phrase in part.get("chunks", []) or []:
_pts = _phrase.get("timestamp") or (None, None)
_ptext = (_phrase.get("text") or "").strip()
try:
_ps, _pe = float(_pts[0]), float(_pts[1])
except (TypeError, ValueError, IndexError):
continue
if _ptext and _pe > _ps:
asr_phrase_segments.append({
"start": _ps, "end": _pe, "text": _ptext,
})
chunk_segs = segment_transcript(part, duration=t1, scene_cuts=scene_cuts)
# Same word source segment_transcript used (already global-timeline),
# kept for the post-diarization speaker re-split (#486).
@@ -1203,7 +1486,7 @@ async def dub_transcribe_stream(
f"unavailable, so the ASR engine's built-in speaker "
f"turns were used and the detected count may differ "
f"from the {num_speakers} you set. Set up diarization "
f"(Settings → Models → pyannote) to enforce an exact "
f"(Model Catalogue → Models → pyannote) to enforce an exact "
f"speaker count."
)
return resplit, {
@@ -1313,7 +1596,25 @@ async def dub_transcribe_stream(
assigned = assign_speakers_from_diarization(all_segments, diar)
# #486: split any segment that spans two speakers' turns at the
# word boundary (single-speaker segments pass through unchanged).
return resplit_segments_by_diarization(assigned, all_words, diar), None, "pyannote"
resplit = resplit_segments_by_diarization(assigned, all_words, diar)
recovered = _recover_from_phrase_embeddings(
diar_pipe,
resplit,
phrases=asr_phrase_segments,
requested_speakers=num_speakers,
audio_target=asr_audio_target,
segments=all_segments,
words=all_words,
)
if recovered is not None:
recovered_segments, separation = recovered
logger.info(
"Recovered rapid two-speaker exchange from ASR phrase embeddings "
"(phrases=%d, separation=%.3f).",
len(asr_phrase_segments), separation,
)
return recovered_segments, None, "phrase_embeddings"
return resplit, None, "pyannote"
except Exception as e:
logger.exception("Diarization failed")
# Inline ASR turns beat the silence-gap heuristic as a crash
@@ -1522,7 +1823,9 @@ async def dub_transcribe_stream(
except Exception as e:
logger.warning("speaker_clone extraction skipped: %s", e)
job["source_lang"] = ((detected_lang or "en").split("_")[0][:2] or "en").lower()
job["source_lang"] = job.get("source_lang_override") or _detected_source_lang(
detected_lang
)
job["full_transcript"] = " ".join(s.get("text", "") for s in final_segs)
_save_job(job_id, job)
@@ -1687,8 +1990,11 @@ async def dub_transcribe(job_id: str, num_speakers: Optional[int] = None):
# / mlx / pytorch based on what's installed + user preference. Works
# identically on all platforms; the older mlx-vs-pytorch branching
# here duplicated the logic in asr_backend.py and skipped WhisperX.
from services.asr_backend import get_active_asr_backend
_asr = get_active_asr_backend(asr_pipe=getattr(_model, "_asr_pipe", None))
# `load_*`, not `get_*`: the plain selector hands back engines whose
# shallow probe passed but whose deep import chain is broken, which
# then dies at `.transcribe()`. The loader degrades (#1185).
from services.asr_backend import load_active_asr_backend
_asr = load_active_asr_backend(asr_pipe=getattr(_model, "_asr_pipe", None))
try:
try:
logger.info("Transcribing full audio via %s ...", _asr.id)
@@ -1716,7 +2022,9 @@ async def dub_transcribe(job_id: str, num_speakers: Optional[int] = None):
except Exception as e:
logger.warning("Failed to unload ASR backend: %s", e)
job["source_lang"] = (detected_lang or "en").split("_")[0][:2].lower()
job["source_lang"] = job.get("source_lang_override") or _detected_source_lang(
detected_lang
)
scene_cuts = job.get("scene_cuts") or []
segments = segment_transcript(result, duration=job.get("duration", 0.0), scene_cuts=scene_cuts)
@@ -1772,7 +2080,8 @@ async def dub_transcribe(job_id: str, num_speakers: Optional[int] = None):
# Bound the whole-file transcribe (#730): a wedged whisperx/CTranslate2
# call would otherwise hold its GPU-pool worker forever and starve
# every other request into a "can't reach backend". run_transcribe_guarded
# also resets the pool on timeout so capacity is restored.
# leaves an unkillable native worker accounted for on timeout so a
# retry cannot overlap it (#1669).
segments_result = await run_transcribe_guarded(_gpu_pool, _transcribe, what="Dub")
except asyncio.CancelledError:
job["aborted"] = True
+164 -20
View File
@@ -23,6 +23,7 @@ from services.ffmpeg_utils import (
find_ffmpeg,
run_ffmpeg,
)
from services.karaoke_ass import build_ass, scale_words
from services.video_retime import (
DRIFT_TOLERANCE_S,
RetimeError,
@@ -403,6 +404,27 @@ def _write_burn_srt(job: dict, exports_dir: str, stamp: str, dual: bool,
return sub_path
def _write_burn_ass(job: dict, exports_dir: str, stamp: str,
fitted_segments: "list[dict] | None" = None,
lang: "str | None" = None) -> str | None:
"""Karaoke variant of ``_write_burn_srt``: word-timed ASS via ``build_ass``.
Same text/timing resolution (``_segments_for_lang`` + fitted-cue overlay,
which also scales per-word times onto the fitted timeline); the basename
is plain ASCII under exports_dir so it is ffmpeg-filter-safe. Returns
None if there are no segments to render.
"""
segments = _segments_for_lang(job, lang)
if not segments:
return None
if fitted_segments:
segments = _apply_fitted_times(segments, fitted_segments)
sub_path = os.path.join(exports_dir, f"burn_subs_{stamp}.ass")
with open(sub_path, "w", encoding="utf-8") as f:
f.write(build_ass(segments))
return sub_path
def _ffmpeg_filter_escape(path: str) -> str:
"""Escape a path for use inside an ffmpeg filter value (subtitles=...).
@@ -515,6 +537,20 @@ def _apply_fitted_times(segments: list[dict], fitted: list[dict]) -> list[dict]:
patched = dict(seg)
patched["start"] = float(cue["start"])
patched["end"] = float(cue["end"])
# Karaoke burn-in: persisted word times live on the original timeline;
# scale them linearly onto the fitted cue span so the highlight sweep
# follows the retimed audio. Degenerate spans drop the words — export
# then falls back to an even split over the fitted span. Inert for
# SRT/VTT, which never read ``words``.
if isinstance(seg.get("words"), list) and seg.get("words"):
scaled = scale_words(
seg["words"], seg.get("start", 0.0), seg.get("end", 0.0),
patched["start"], patched["end"],
)
if scaled is not None:
patched["words"] = scaled
else:
patched.pop("words", None)
out.append(patched)
return out
@@ -572,11 +608,12 @@ def _build_audio_export_cmd(
async def dub_download(
job_id: str,
preserve_bg: bool = Query(True, description="Mix background noise into dubbed tracks"),
default_track: str = Query("original"),
default_track: str = Query("", description="Default audio track; omitted selects the first dubbed track"),
include_tracks: str = Query("", description="Comma-separated list of tracks to include (e.g. 'original,de,es'). Empty = include all."),
save_authorization: str = Header("", alias="X-VoiceStudio-Path-Authorization"),
burn_subs: bool = Query(False, description="Burn subtitles into the video stream (forces re-encode). Uses dual-subtitle layout when dual=1."),
dual: bool = Query(False, description="When burn_subs=1, render translated on top of italicised original."),
karaoke: bool = Query(False, description="When burn_subs=1, burn a word-timed karaoke highlight (ASS) instead of line subtitles. Ignored when dual=1 (dual karaoke is unsupported — the line burn renders instead)."),
out_format: str = Query("m4a", description="Audio-only jobs (#119): output container — wav, m4a, mp3, or flac. Ignored for video jobs."),
):
# Strict allowlist on the path param BEFORE it reaches any filesystem
@@ -607,6 +644,18 @@ async def dub_download(
for key, value in filtered_tracks.items()
}
# A dub export should play the dub without requiring player-specific track
# selection. Keep ``original`` as an explicit opt-in, but when callers omit
# the preference choose the first generated dub consistently (#1575).
if (
filtered_tracks
and not (default_track == "original" and include_original)
and default_track not in filtered_tracks
):
default_track = next(iter(filtered_tracks))
elif not filtered_tracks and include_original:
default_track = "original"
if not filtered_tracks and not include_original:
raise HTTPException(status_code=400, detail="No tracks selected for export")
@@ -631,12 +680,17 @@ async def dub_download(
fmt = (out_format or "m4a").lower()
if fmt not in _AUDIO_FORMAT_CODECS:
fmt = "m4a"
# lang_code is already constrained to an existing track key, but
# allowlist-sanitize it before it reaches the output path so a path
# component can never carry separators/traversal (same pattern as
# safe_name below).
safe_lang = "".join(c for c in lang_code if c.isalnum() or c in "-_") or "track"
out_path = os.path.join(exports_dir, f"dubbed_audio_{safe_lang}_{stamp}.{fmt}")
# Keep route/job data out of the filesystem and logging trust boundary.
# The selected format reaches the path only through literal branches.
if fmt == "wav":
output_name = f"dubbed_audio_{stamp}.wav"
elif fmt == "mp3":
output_name = f"dubbed_audio_{stamp}.mp3"
elif fmt == "flac":
output_name = f"dubbed_audio_{stamp}.flac"
else:
output_name = f"dubbed_audio_{stamp}.m4a"
out_path = os.path.join(exports_dir, output_name)
bg = _optional_dub_artifact(job.get("no_vocals_path"), job_id) if preserve_bg else None
cmd = _build_audio_export_cmd(ffmpeg, track_info["path"], bg, out_path, fmt)
try:
@@ -654,15 +708,28 @@ async def dub_download(
)
if not os.path.exists(out_path) or os.path.getsize(out_path) == 0:
raise HTTPException(status_code=500, detail="ffmpeg audio export produced no output file")
logger.info("Dub audio export wrote %s (%d bytes)", out_path, os.path.getsize(out_path))
logger.info("Dub audio export completed (%d bytes)", os.path.getsize(out_path))
base_name = os.path.splitext(job.get("filename", "output"))[0]
safe_name = "".join(c for c in base_name if c.isalnum() or c in "-_ ").strip() or "output"
dl_name = f"dubbed_{safe_name}_{safe_lang}_{stamp}.{fmt}"
# Response metadata must not become a second path-like sink for job or
# request data. Keep the user-selected format through explicit literal
# branches; source names and language keys never enter the label.
if fmt == "wav":
dl_name = f"dubbed_audio_{stamp}.wav"
elif fmt == "mp3":
dl_name = f"dubbed_audio_{stamp}.mp3"
elif fmt == "flac":
dl_name = f"dubbed_audio_{stamp}.flac"
else:
dl_name = f"dubbed_audio_{stamp}.m4a"
media_type = _MEDIA_TYPES.get(f".{fmt}", "audio/mp4")
save_path = _consume_native_save(save_authorization)
if save_path:
return _native_save(out_path, save_path, dl_name, media_type=media_type)
# Keep the request-derived download label out of the filesystem
# trust boundary. It is response metadata, not a source or
# destination path (CodeQL, #1575).
result = _native_save(out_path, save_path, "dubbed_audio", media_type=media_type)
result["display_name"] = dl_name
return result
return FileResponse(
out_path, media_type=media_type,
headers={"Content-Disposition": content_disposition(dl_name)},
@@ -699,7 +766,18 @@ async def dub_download(
fitted_segments = _fitted_segments_for(job, default_track) if default_track and default_track != "original" else None
# Burn the DEFAULT track's text (P1.2) — it's the audio the viewer hears.
_burn_lang = default_track if default_track and default_track != "original" else None
sub_path = _write_burn_srt(job, exports_dir, stamp, dual, fitted_segments=fitted_segments, lang=_burn_lang) if burn_subs else None
# Karaoke (word-highlight) burn writes an ASS instead of the line SRT.
# Dual layout keeps the line burn — dual karaoke is out of scope, matching
# the disabled control in the Export drawer. The default (karaoke off)
# takes exactly the legacy SRT path.
sub_path = None
sub_is_ass = False
if burn_subs:
if karaoke and not dual:
sub_path = _write_burn_ass(job, exports_dir, stamp, fitted_segments=fitted_segments, lang=_burn_lang)
sub_is_ass = sub_path is not None
if sub_path is None:
sub_path = _write_burn_srt(job, exports_dir, stamp, dual, fitted_segments=fitted_segments, lang=_burn_lang)
# ── Smart Fit video retime (two-tier) ─────────────────────────────────
# Tier 1 (≤48 chunks): single filter_complex graph inlined into the mux
@@ -799,14 +877,16 @@ async def dub_download(
esc = _ffmpeg_filter_escape(sub_path)
# Burn AFTER any retime so cues (already on the fitted timeline for
# Smart Fit) land on the retimed video. Without retime this reduces
# to the legacy `[0:v]subtitles=…[vsub]` graph.
# to the legacy `[0:v]subtitles=…[vsub]` graph. Karaoke burns the
# word-timed ASS through the ass filter at the same graph position.
if video_map.startswith("["):
sub_src = video_map
elif retimed_idx is not None:
sub_src = f"[{retimed_idx}:v]"
else:
sub_src = "[0:v]"
filter_parts.append(f"{sub_src}subtitles='{esc}'[vsub]")
_sub_filter = "ass" if sub_is_ass else "subtitles"
filter_parts.append(f"{sub_src}{_sub_filter}='{esc}'[vsub]")
video_map = "[vsub]"
if stretch_entry:
orig_dur = float(stretch_entry.get("orig_duration") or job.get("duration") or 0.0)
@@ -887,7 +967,10 @@ async def dub_download(
if default_track == "original" and include_original:
cmd += ["-disposition:a:0", "default"]
else:
target_idx = 0
# A stale/missing language preference still means "play a dub", not
# "silently fall back to the source". The first processed dub is the
# deterministic fallback; ``original`` above remains explicit.
target_idx = tracks_to_process[0]["stream_idx"] if tracks_to_process else 0
for t in tracks_to_process:
if t['lang_code'] == default_track:
target_idx = t["stream_idx"]
@@ -1434,13 +1517,20 @@ async def dub_qc_pass(job_id: str, lang: str = Query(None), drift_threshold: flo
)
def _recognize():
from services.asr_backend import get_active_asr_backend
backend = get_active_asr_backend()
# `load_*`, not `get_*`: the plain selector returns engines whose
# shallow probe passed but whose deep import chain is broken, which
# then 500s at `.transcribe()`. The loader degrades (#1185).
from services.asr_backend import load_active_asr_backend
backend = load_active_asr_backend()
result = backend.transcribe(wav_path, word_timestamps=False)
return result.get("segments", []), backend.id
try:
from services.asr_backend import ASRTimeoutError, run_transcribe_guarded
from services.asr_backend import (
ASRModelMissingError,
ASRTimeoutError,
run_transcribe_guarded,
)
from services.model_manager import _get_gpu_pool
recognized, engine_id = await run_transcribe_guarded(
_get_gpu_pool(), _recognize, what="QC",
@@ -1449,6 +1539,13 @@ async def dub_qc_pass(job_id: str, lang: str = Query(None), drift_threshold: flo
# Backend is alive; ASR just couldn't finish in time. 504, not 500/connection.
logger.warning("dub QC ASR pass timed out")
raise HTTPException(status_code=504, detail=str(e))
except ASRModelMissingError as e:
# Degraded onto an engine with no weights on disk — typed 409 with the
# download CTA, matching the preflight above.
raise HTTPException(
status_code=409,
detail={**e.payload, "message": asr_model_missing_detail(e.payload)},
)
except Exception as e:
logger.exception("dub QC ASR pass failed")
raise HTTPException(status_code=500, detail=f"QC transcription failed: {e}")
@@ -1552,7 +1649,10 @@ async def dub_download_audio(
return _native_save(wav_path, save_path, dl_name, media_type="audio/wav")
return FileResponse(
wav_path, media_type="audio/wav",
headers={"Content-Disposition": content_disposition(dl_name)},
headers={
"Cache-Control": "no-store",
"Content-Disposition": content_disposition(dl_name),
},
)
@@ -1694,6 +1794,50 @@ async def dub_export_vtt(
)
@router.get("/dub/ass/{job_id}")
@router.get("/dub/ass/{job_id}/{filename}")
async def dub_export_ass(
job_id: str,
lang: str = Query(None, description="Track language code. Same text/timing resolution as /dub/srt, rendered as a karaoke (word-highlight) ASS sidecar."),
):
"""Karaoke ASS sidecar — the same script the karaoke burn-in renders.
Raw text body like /dub/srt and /dub/vtt (the Tauri side writes the file
itself; no ?save_path= variant see the comment above /dub/srt).
"""
_job_dir_or_400(job_id)
lang = _safe_lang_or_400(lang)
job = _get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
segments = _segments_for_lang(job, lang)
if not segments:
raise HTTPException(status_code=400, detail="No transcript segments available")
# Same strategy-aware cue timing as /dub/srt. The fitted overlay also
# scales word times; the stretch_video cue path has no per-word record,
# so words are dropped and build_ass even-splits over the new spans.
fitted = _fitted_segments_for(job, lang)
if fitted:
segments = _apply_fitted_times(segments, fitted)
else:
cues = _fitted_cue_times(job, lang)
if cues:
segments = [
{**{k: v for k, v in seg.items() if k != "words"}, "start": s, "end": e}
for seg, (s, e) in zip(segments, cues)
]
base_name = os.path.splitext(job.get('filename', 'video'))[0]
dl_name = f"subtitles_{base_name}_karaoke.ass"
return Response(
content=build_ass(segments),
media_type="text/plain",
headers={"Content-Disposition": content_disposition(dl_name)},
)
@router.get("/dub/export-segments/{job_id}")
async def dub_export_segments_zip(job_id: str, lang: str = Query(None)):
import zipfile
+350 -47
View File
@@ -1,9 +1,12 @@
import os
import re
import json
import struct
import logging
import time
import asyncio
import shutil
import zipfile
import torch
import torchaudio
from fastapi import APIRouter, HTTPException
@@ -13,7 +16,8 @@ from core.config import DUB_DIR, VOICES_DIR, dub_seg_path
from core.tasks import task_manager
from schemas.requests import DubRequest
from services.model_manager import _gpu_pool, run_on_gpu_pool_guarded
from services.tts_backend import resolve_generation_backend
from services.tts_backend import resolve_generation_backend, active_backend_id
from services import gpu_gateway
from services.audio_dsp import apply_mastering, normalize_audio, apply_effects_chain, get_effect_chain
from services.audio_io import atomic_save_wav, _safe_torchaudio_save
from services.ffmpeg_utils import (
@@ -45,6 +49,94 @@ logger = logging.getLogger("omnivoice.dub")
MAX_STRETCH_RATIO = 1.8
def _prepare_oom_retry(error: Exception, *, execution_target: str) -> bool:
"""Prepare one *local* low-step retry after a genuine device OOM.
The cache being flushed must belong to the device that raised the error.
A remote worker owns its own recovery policy; flushing this process's CUDA
cache after a remote failure both stalls the wrong GPU and can evict an
unrelated local job. Keep this guard at the retry chokepoint so a future
``dub_segments`` producer cannot accidentally inherit the old behaviour.
Returns ``False`` for non-OOM errors. Remote OOMs are deliberately raised
unchanged: the worker may classify/retry them, but this process must not.
"""
is_oom = (
isinstance(error, torch.cuda.OutOfMemoryError)
or "out of memory" in str(error).lower()
or "CUDA error" in str(error)
)
if not is_oom:
return False
if execution_target != "local":
raise error
import gc
gc.collect()
if torch.cuda.is_available():
torch.cuda.empty_cache()
elif hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
torch.mps.empty_cache()
return True
def _cached_payload_intact(path: str, info) -> bool:
"""Cheap truth check on a cached WAV whose header we are about to trust.
The natural-rate fast path hands the mixer a PATH instead of decoded
audio, so a cache whose header reads fine but whose payload is truncated
would only fail later, during assembly after the timing plan (Smart Fit,
video stretch) had been computed from the header's frame count. The plan
would then describe audio that no longer exists and the segment would be
replaced by slot-length silence, leaving the persisted video plan and the
rendered track disagreeing.
Comparing the declared frame count against the physical ``data`` chunk
catches that without decoding: a truncated file cannot hold the samples
its header claims. Anything failing here falls through to the decoding path, which
already degrades to a warning plus silence. Formats with no fixed
bits-per-sample (compressed caches) are left to the decoder as before.
"""
try:
bits = int(getattr(info, "bits_per_sample", 0) or 0)
frames = int(getattr(info, "num_frames", 0) or 0)
channels = int(getattr(info, "num_channels", 0) or 0)
if bits <= 0 or frames <= 0 or channels <= 0:
# Undecidable metadata fails CLOSED (review on #1620): these caches
# are PCM WAVs this module wrote itself, so anything else is
# unexpected — and the decode path this falls through to handles
# every format the fast path would have.
return False
payload = frames * channels * (bits // 8)
if payload <= 0:
return False
# A WAV may carry JUNK/LIST metadata before data, so its header is not
# necessarily 44 bytes. Locate the data chunk instead of counting
# metadata as audio; otherwise an extended header can mask truncation.
file_size = os.path.getsize(path)
with open(path, "rb") as wav:
header = wav.read(12)
if len(header) != 12 or header[:4] != b"RIFF" or header[8:12] != b"WAVE":
return False
offset = 12
while offset + 8 <= file_size:
wav.seek(offset)
chunk_id = wav.read(4)
chunk_size_raw = wav.read(4)
if len(chunk_id) != 4 or len(chunk_size_raw) != 4:
return False
chunk_size = struct.unpack("<I", chunk_size_raw)[0]
data_offset = offset + 8
if chunk_id == b"data":
return chunk_size >= payload and file_size >= data_offset + payload
offset = data_offset + chunk_size + (chunk_size % 2)
return False
except Exception: # noqa: BLE001 — an unstattable cache is the decoder's problem
return False
def _underrun_min_rate() -> float:
"""Floor for the underrun fill (audio slowed toward its slot, never below
this rate). Default 0.85 stays natural-sounding; OMNIVOICE_UNDERRUN_MIN_RATE=1.0
@@ -317,6 +409,75 @@ def resolve_consistent_ref(job: dict, speaker_key: str, memo: dict | None = None
return ref
def _remote_voice(job: dict, profile_id: str | None, seg_id, voice_match: str,
memo: dict) -> tuple[str | None, str | None, bool, str | None, int | None]:
"""Resolve a dub binding without touching the TTS model."""
ref_audio = ref_text = instruct = None
seed = None
single_use = False
if profile_id and profile_id.startswith("auto-seg:"):
sid = profile_id[len("auto-seg:"):]
info = (job.get("segment_clones") or {}).get(sid)
shared = False
if voice_match == "consistent" and sid == str(seg_id):
key = _speaker_key_for_segment(job, sid)
alternate = resolve_consistent_ref(job, key, memo) if key else None
if alternate:
info = alternate
shared = True
if info:
ref_audio, ref_text = info.get("ref_audio"), info.get("ref_text")
single_use = not shared
elif profile_id and profile_id.startswith("auto:"):
key = profile_id[len("auto:"):]
if voice_match == "consistent":
info = resolve_consistent_ref(job, key, memo)
else:
info = ((job.get("segment_clones") or {}).get(str(seg_id))
or _find_speaker_clone(job.get("speaker_clones") or {}, key))
single_use = str(seg_id) in (job.get("segment_clones") or {})
if info:
ref_audio, ref_text = info.get("ref_audio"), info.get("ref_text")
elif profile_id:
with db_conn() as conn:
row = conn.execute("SELECT * FROM voice_profiles WHERE id=?", (profile_id,)).fetchone()
if row:
seed = row["seed"]
if row["is_locked"] and row["locked_audio_path"]:
ref_audio = os.path.join(VOICES_DIR, row["locked_audio_path"])
ref_text = row["ref_text"]
elif row["instruct"] and not row["is_locked"]:
try:
vd_states = row["vd_states"]
except (KeyError, IndexError):
vd_states = None
instruct = heal_design_instruct(row["instruct"], vd_states)
else:
ref_audio = os.path.join(VOICES_DIR, row["ref_audio_path"])
ref_text = row["ref_text"]
return ref_audio, ref_text, single_use, instruct, seed
def _decode_remote_dub(result: gpu_gateway.RemoteResult) -> dict[int, str]:
"""Extract the worker bundle into a task-scoped directory, path-safely."""
target = os.path.join(DUB_DIR, ".remote", result.task_id)
os.makedirs(target, exist_ok=True)
paths: dict[int, str] = {}
with zipfile.ZipFile(result.path) as archive:
for member in archive.infolist():
match = re.fullmatch(r"segments/(\d+)\.wav", member.filename)
if not match:
raise ValueError(f"unexpected dub artifact member: {member.filename}")
index = int(match.group(1))
destination = os.path.join(target, f"{index}.wav")
partial = f"{destination}.part"
with archive.open(member) as source, open(partial, "wb") as output:
shutil.copyfileobj(source, output)
os.replace(partial, destination)
paths[index] = destination
return paths
router = APIRouter()
@router.post("/dub/generate/{job_id}")
@@ -331,7 +492,7 @@ async def dub_generate(job_id: str, req: DubRequest):
# ── Engine resolution (issue #312 class) ────────────────────────────────
# Dub used to hardcode VoiceStudio via get_model() regardless of the engine
# selected in Settings → Engines — a SILENT fallback. Every real dub
# selected in Model Catalogue → Engines — a SILENT fallback. Every real dub
# segment's ref_audio resolves to either an auto:<speaker>/auto-seg:<id>
# clone cut from the source video or a saved voice-profile row (see
# `_gen` below), so require_cloning=True: an engine that can't clone
@@ -342,6 +503,18 @@ async def dub_generate(job_id: str, req: DubRequest):
backend = await resolve_generation_backend(require_cloning=True)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
except Exception as e:
from core.failure import is_gpu_oom
if not is_gpu_oom(e):
raise
from core.public_errors import public_exception_response
payload = public_exception_response(
e,
fallback="The TTS model could not be loaded.",
)
raise HTTPException(status_code=503, detail=payload["detail"]) from e
async def _stream(task_id):
total = len(req.segments)
@@ -488,11 +661,12 @@ async def dub_generate(job_id: str, req: DubRequest):
# every segment of that speaker for the whole run.
voice_match = (req.voice_match or "per_line").lower()
_consistent_ref_memo: dict = {}
# Strategy-transition guard: smart_fit re-mixes the *natural-rate*
# per-segment WAVs from disk. If the previous run used strict_slot,
# the on-disk WAVs are slot-squeezed ("slotted") — reusing them would
# double-compress. Force one full regen; afterwards seg_wav_kind is
# "natural" and partial regen / fit-only re-mix (regen_only=[]) work.
remote_audio: dict[int, str] = {}
# Strategy-transition guard: concise, stretch_video and smart_fit all
# re-mix *natural-rate* per-segment WAVs. If the previous run used
# strict_slot, the on-disk WAVs are slot-squeezed ("slotted") — the
# missing tails cannot be recovered by a re-mix. Force one full regen;
# afterwards partial regen / fit-only re-mix (regen_only=[]) is safe.
# Jobs predating this field have unknown kind → also regen once.
# P1.3: the kind is per-track now (each language renders under its own
# strategy); the flat job["seg_wav_kind"] is only consulted for jobs
@@ -503,7 +677,7 @@ async def dub_generate(job_id: str, req: DubRequest):
_wav_kind = (
_kind_map.get(lang_code) if isinstance(_kind_map, dict) else job.get("seg_wav_kind")
)
if strategy == "smart_fit" and regen_only is not None and _wav_kind != "natural":
if strategy != "strict_slot" and regen_only is not None and _wav_kind != "natural":
regen_only = None
# Manifest: stable segment id per current index. Per-segment WAVs are
# named by stable id (dub_seg_path) so regen reuses the right audio after
@@ -528,6 +702,86 @@ async def dub_generate(job_id: str, req: DubRequest):
_t_cache = 0.0
_t_tts = 0.0
# One coarse remote lease for every segment that actually needs fresh
# synthesis. Assembly, fitting and the separately-pooled RVC pass stay
# here; the worker returns a single verified bundle of segment WAVs.
decision = gpu_gateway.decide("dub_segments")
if decision.remote:
remote_rows: list[dict] = []
remote_refs: list[str | None] = []
for i, seg in enumerate(req.segments):
seg_id = seg_ids[i] if i < len(seg_ids) else f"seg_{i}"
if (regen_only is not None and seg_id not in regen_only) or not seg.text.strip():
continue
ref_audio, ref_text, ref_single_use, profile_instruct, seed = _remote_voice(
job, seg.profile_id or None, seg_id, voice_match, _consistent_ref_memo
)
ref_audio = warn_if_ref_missing(
ref_audio, job_id=job_id, seg_id=seg_id, where="remote dub render"
)
seg_instruct = seg.instruct or req.instruct or profile_instruct
seg_speed = seg.speed if seg.speed is not None else req.speed
if seg.direction and seg.direction.strip():
try:
from services.director import parse as _parse_direction
direction = _parse_direction(seg.direction)
extra = direction.instruct_prompt()
if extra:
seg_instruct = f"{seg_instruct}, {extra}" if seg_instruct else extra
bias = direction.rate_bias()
if bias and abs(bias - 1.0) > 0.01 and strategy == "strict_slot":
seg_speed = (seg_speed or 1.0) * bias
except Exception:
logger.debug("direction parse skipped for remote segment %s", seg_id,
exc_info=True)
remote_rows.append({
"index": i, "text": seg.text,
"language": seg.target_lang or req.language,
"ref_text": ref_text, "ref_single_use": ref_single_use,
"instruct": seg_instruct,
"duration": (seg.end - seg.start) if strategy == "strict_slot" else None,
"num_step": 8 if req.preview else req.num_step,
"guidance_scale": req.guidance_scale, "speed": seg_speed,
"effect_preset": seg.effect_preset or "broadcast",
"seed": seed,
# RVC changes the waveform locally after TTS, so that path
# is marked at the existing post-RVC chokepoint below.
"watermark": not rvc_is_enabled(),
})
remote_refs.append(ref_audio)
if remote_rows:
states: asyncio.Queue = asyncio.Queue()
call = gpu_gateway.RemoteCall(
engine=active_backend_id(), operation="dub_segments",
params={"segments": remote_rows, "ref_audio": remote_refs},
decode=_decode_remote_dub,
)
dub_run = gpu_gateway.JobRun("dub_segments")
run = asyncio.create_task(gpu_gateway.run(
"dub_segments", local=gpu_gateway.LocalCall(fn=lambda: {}),
remote=call, decision=decision, job=dub_run,
on_state=states.put_nowait,
))
while not run.done():
if task_manager.is_cancelled(task_id):
run.cancel()
try:
await run
except asyncio.CancelledError:
pass
yield f"data: {json.dumps({'type': 'cancelled', 'segments_processed': 0})}\n\n"
return
try:
state = await asyncio.wait_for(states.get(), timeout=0.25)
except asyncio.TimeoutError:
continue
fraction = float(state.get("progress") or 0.0)
yield f"data: {json.dumps({'type': 'progress', 'current': round(fraction * total, 2), 'total': total, 'text': state.get('stage') or state.get('phase')})}\n\n"
remote_audio = await run
notice = dub_run.notice()
if notice is not None:
yield f"data: {json.dumps({'type': 'routing_notice', 'status': notice[0], 'reason': notice[1]})}\n\n"
for i, seg in enumerate(req.segments):
seg_id = seg_ids[i] if i < len(seg_ids) else f"seg_{i}"
@@ -536,7 +790,8 @@ async def dub_generate(job_id: str, req: DubRequest):
yield f"data: {json.dumps({'type': 'cancelled', 'segments_processed': i})}\n\n"
return
yield f"data: {json.dumps({'type': 'progress', 'current': i, 'total': total, 'text': seg.text[:50]})}\n\n"
if not remote_audio:
yield f"data: {json.dumps({'type': 'progress', 'current': i, 'total': total, 'text': seg.text[:50]})}\n\n"
seg_duration = seg.end - seg.start
if seg_duration <= 0.05 or not seg.text.strip():
@@ -573,15 +828,38 @@ async def dub_generate(job_id: str, req: DubRequest):
if os.path.exists(seg_wav_path):
try:
_t_cache_0 = time.perf_counter()
# Natural-rate caches are already the exact assembly
# input. Keep the durable path in the manifest so the
# mixer decodes it once; the old path decoded here,
# wrote an identical mix_<id> scratch WAV, then decoded
# that copy again. Header-only inspection preserves
# the resample fallback for caches made by an engine
# with a different sample rate.
if strategy != "strict_slot":
try:
cached_info = torchaudio.info(seg_wav_path)
except Exception:
cached_info = None
if (
cached_info is not None
and int(cached_info.sample_rate) == int(backend.sample_rate)
and _cached_payload_intact(seg_wav_path, cached_info)
):
all_segment_wavs.append(
(seg.start, seg.end, seg_wav_path, backend.sample_rate)
)
sync_scores.append(getattr(seg, 'sync_ratio', None) or 1.0)
_t_cache += time.perf_counter() - _t_cache_0
continue
cached_wav, cached_sr = torchaudio.load(seg_wav_path)
if cached_sr != backend.sample_rate:
import torchaudio.functional as AF
cached_wav = AF.resample(cached_wav, cached_sr, backend.sample_rate)
# Pad/trim to slot — except smart_fit, whose mix
# loop needs the natural-rate length to compute the
# audio/video split (the seg_wav_kind guard above
# guarantees these cached WAVs are natural-rate).
if strategy != "smart_fit":
# strict_slot persists slot-sized buffers. Every other
# strategy consumes natural-rate audio and lets the mix
# loop fit it to the current timeline.
if strategy == "strict_slot":
target_samples = int(seg_duration * backend.sample_rate)
current_samples = cached_wav.shape[-1]
if target_samples > current_samples:
@@ -612,7 +890,8 @@ async def dub_generate(job_id: str, req: DubRequest):
sync_scores.append(1.0)
continue
def _gen(text, lang, instruct_str, dur_s, nstep, cfg, spd, profile_id, effect_preset):
def _gen(text, lang, instruct_str, dur_s, nstep, cfg, spd, profile_id, effect_preset,
*, execution_target="local"):
# Normalize once at the segment's text→engine choke point
# (covers the OOM-retry generate below too, which reuses this
# closure's `text`). Pref-gated, idempotent, never raises.
@@ -782,19 +1061,7 @@ async def dub_generate(job_id: str, req: DubRequest):
)
return normalize_audio(mastered_audio, target_dBFS=-2.0)
except Exception as e:
is_oom = (
isinstance(e, torch.cuda.OutOfMemoryError)
or "out of memory" in str(e).lower()
or "CUDA error" in str(e)
)
import gc
gc.collect()
if torch.cuda.is_available():
torch.cuda.empty_cache()
elif hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
torch.mps.empty_cache()
if not is_oom:
if not _prepare_oom_retry(e, execution_target=execution_target):
raise
retry_steps = min(nstep, 8)
@@ -900,14 +1167,24 @@ async def dub_generate(job_id: str, req: DubRequest):
# Budget from the shared length-scaled helper (#1190): a long
# dub segment used to die on the flat 300s even after v0.3.22.
from services.model_manager import generate_timeout_s
audio_tensor = await run_on_gpu_pool_guarded(
lambda: _gen(
seg.text, seg_lang, seg_instruct, _dur_for_tts,
_num_step, req.guidance_scale, seg_speed, seg_profile, seg_effect_preset,
),
what="Dub generate",
timeout=generate_timeout_s(seg.text),
)
if i in remote_audio:
audio_tensor, remote_sr = torchaudio.load(remote_audio[i])
try:
os.unlink(remote_audio[i])
except OSError:
pass
if remote_sr != backend.sample_rate:
import torchaudio.functional as AF
audio_tensor = AF.resample(audio_tensor, remote_sr, backend.sample_rate)
else:
audio_tensor = await run_on_gpu_pool_guarded(
lambda: _gen(
seg.text, seg_lang, seg_instruct, _dur_for_tts,
_num_step, req.guidance_scale, seg_speed, seg_profile, seg_effect_preset,
),
what="Dub generate",
timeout=generate_timeout_s(seg.text, engine=backend),
)
_t_tts += time.perf_counter() - _t_tts_0
# Check abort immediately after GPU work completes
@@ -979,12 +1256,15 @@ async def dub_generate(job_id: str, req: DubRequest):
if rvc_sr == backend.sample_rate:
audio_tensor = rvc_wav
target_samples = int(seg_duration * backend.sample_rate)
current_samples = audio_tensor.shape[-1]
if target_samples > current_samples:
audio_tensor = torch.nn.functional.pad(audio_tensor, (0, target_samples - current_samples))
elif current_samples > target_samples:
audio_tensor = audio_tensor[..., :target_samples]
if strategy == "strict_slot":
target_samples = int(seg_duration * backend.sample_rate)
current_samples = audio_tensor.shape[-1]
if target_samples > current_samples:
audio_tensor = torch.nn.functional.pad(
audio_tensor, (0, target_samples - current_samples)
)
elif current_samples > target_samples:
audio_tensor = audio_tensor[..., :target_samples]
except Exception as e:
yield f"data: {json.dumps({'type': 'warning', 'segment': i, 'message': f'RVC skipped: {str(e)[:120]}'})}\n\n"
@@ -996,8 +1276,9 @@ async def dub_generate(job_id: str, req: DubRequest):
# no double-mark. Cached-reuse audio is already marked;
# silence/zero slots carry no speech to mark, so neither is
# re-watermarked.
audio_tensor = mark_synthetic(audio_tensor, backend.sample_rate,
context="dub_generate.segment")
if i not in remote_audio or rvc_is_enabled():
audio_tensor = mark_synthetic(audio_tensor, backend.sample_rate,
context="dub_generate.segment")
seg_wav_path = _seg_lang_path(seg_id)
try:
@@ -1022,7 +1303,15 @@ async def dub_generate(job_id: str, req: DubRequest):
pass
_release_audio_tensors()
except Exception as e:
yield f"data: {json.dumps({'type': 'error', 'segment': i, 'error': str(e)})}\n\n"
# A task-stream error bypasses the global exception handler.
# Never publish engine exception text here: allocator errors
# carry process tables and arbitrary failures can carry paths,
# tokens, or source text. The shared helper enriches recognized
# classes using VoiceStudio-owned constants only.
from core.public_errors import stream_generation_failure
error_detail = stream_generation_failure(e)["detail"]
yield f"data: {json.dumps({'type': 'error', 'segment': i, 'error': error_detail})}\n\n"
sr = backend.sample_rate
all_segment_wavs.append(_store_mix_wav(seg.start, seg.end, torch.zeros(1, max(0, int(seg_duration * sr))), sr, f"mix_{seg_id}"))
sync_scores.append(1.0)
@@ -1170,7 +1459,21 @@ async def dub_generate(job_id: str, req: DubRequest):
seg_gain = getattr(seg_ref, "gain", None) if seg_ref is not None else None
seg_gain = seg_gain if seg_gain is not None else 1.0
seg_gain = max(0.0, min(2.0, seg_gain))
wav = _load_entry_wav((start, end, wav_path, sr), sr)
try:
wav = _load_entry_wav((start, end, wav_path, sr), sr)
except Exception as e:
# A WAV header can be readable while its payload is
# truncated. Direct cache reuse deliberately defers the
# decode to assembly, so preserve the old recovery contract
# here: warn and fill this slot with silence instead of
# aborting the entire dub.
warning = {
"type": "warning",
"segment": i,
"message": f"cached seg lost, padding silence: {str(e)[:120]}",
}
yield f"data: {json.dumps(warning)}\n\n"
wav = torch.zeros(1, max(0, int((end - start) * sr)))
adjusted = wav * seg_gain
if adjusted.ndim == 2 and adjusted.shape[0] > 1:
adjusted = adjusted.mean(dim=0, keepdim=True)
@@ -1612,7 +1915,7 @@ async def preview_segment(job_id: str, req: SegmentPreviewRequest):
from services.model_manager import generate_timeout_s
audio_tensor = await run_on_gpu_pool_guarded(
_gen, what="Dub preview generate",
timeout=generate_timeout_s(req.text),
timeout=generate_timeout_s(req.text, engine=backend),
)
sr = backend.sample_rate
+67 -48
View File
@@ -25,7 +25,7 @@ from huggingface_hub import utils as hf_utils
from huggingface_hub.errors import HFValidationError
from pydantic import BaseModel
from api.dependencies import require_loopback
from api.dependencies import require_admin, require_admin_action, require_desktop
from core import prefs
from services import tts_backend, asr_backend, llm_backend, translation_engines
from services.audio_dsp import list_effect_presets
@@ -41,6 +41,15 @@ _FAMILIES = {
"llm": (llm_backend, "llm_backend"),
}
def _family_payload(family: str, module):
"""Public inventory plus whether an environment pin owns this family."""
return {
"active": module.active_backend_id(),
"env_override": bool(os.environ.get(f"OMNIVOICE_{family.upper()}_BACKEND")),
"backends": public_backends(module.list_backends()),
}
def _is_hf_repo_id(value: str) -> bool:
"""Validate the route's ``owner/repo`` contract in bounded time."""
if not isinstance(value, str) or len(value) > 96 or value.count("/") != 1:
@@ -55,34 +64,40 @@ def _is_hf_repo_id(value: str) -> bool:
@router.get("/engines")
def list_all_engines():
return {
"tts": {
"active": tts_backend.active_backend_id(),
"backends": public_backends(tts_backend.list_backends()),
},
"asr": {
"active": asr_backend.active_backend_id(),
"backends": public_backends(asr_backend.list_backends()),
},
"llm": {
"active": llm_backend.active_backend_id(),
"backends": public_backends(llm_backend.list_backends()),
},
"tts": _family_payload("tts", tts_backend),
"asr": _family_payload("asr", asr_backend),
"llm": _family_payload("llm", llm_backend),
}
@router.get("/engines/tts")
def list_tts_backends():
return {"active": tts_backend.active_backend_id(), "backends": public_backends(tts_backend.list_backends())}
return _family_payload("tts", tts_backend)
@router.get(
"/engines/{engine_id}/disk-usage",
dependencies=[Depends(require_admin_action)],
)
def engine_disk_usage(engine_id: str):
"""Measure owned engine bytes only when a catalogue row is opened."""
try:
tts_backend.get_backend_class(engine_id)
except ValueError:
raise HTTPException(status_code=404, detail="Unknown TTS engine")
from services.engine_disk_usage import disk_usage_for
return disk_usage_for(engine_id)
@router.get("/engines/asr")
def list_asr_backends():
return {"active": asr_backend.active_backend_id(), "backends": public_backends(asr_backend.list_backends())}
return _family_payload("asr", asr_backend)
@router.get("/engines/llm")
def list_llm_backends():
return {"active": llm_backend.active_backend_id(), "backends": public_backends(llm_backend.list_backends())}
return _family_payload("llm", llm_backend)
@router.get("/engines/effects/presets", response_model=EffectPresetsResponse)
@@ -113,7 +128,10 @@ def list_translation_engines():
}
@router.post("/engines/translation/{engine_id}/install")
@router.post(
"/engines/translation/{engine_id}/install",
dependencies=[Depends(require_admin)],
)
async def install_translation_engine(engine_id: str):
entry = translation_engines.get_engine(engine_id)
if not entry:
@@ -149,7 +167,10 @@ async def install_translation_engine(engine_id: str):
}
@router.delete("/engines/translation/{engine_id}")
@router.delete(
"/engines/translation/{engine_id}",
dependencies=[Depends(require_admin)],
)
async def uninstall_translation_engine(engine_id: str):
entry = translation_engines.get_engine(engine_id)
if not entry:
@@ -178,7 +199,7 @@ async def uninstall_translation_engine(engine_id: str):
# Sidecar engines (dedicated venv + source checkout + weights, isolated from
# the parent's transformers>=5.3) used to require four manual terminal steps.
# These routes drive services.sidecar_install: POST starts a resumable
# background job, GET polls its step-by-step status (the Settings → Engines
# background job, GET polls its step-by-step status (the Model Catalogue → Engines
# Install button polls this), DELETE removes an app-managed install.
#
# Path namespace: /engines/sidecar/{engine_id}/… — NOT /engines/{engine_id}/…
@@ -188,15 +209,16 @@ async def uninstall_translation_engine(engine_id: str):
# POST /engines/sonitranslate/install). Mirrors the
# /engines/translation/{engine_id}/install namespace pattern.
#
# Loopback-gated: installing spawns subprocesses (git/uv) and writes to the
# data directory — only the local desktop frontend may trigger it. The job
# runs fine in packaged builds: the venv lives under the user data dir, not
# inside the signed app bundle, and uv resolves via OMNIVOICE_BUNDLED_UV/PATH.
# Desktop-only: installing spawns git/uv against mutable source and writes an
# editable environment. An API key does not make that supply-chain path safe to
# trigger remotely. The job runs fine in packaged builds: the venv lives under
# the user data dir, not inside the signed app bundle, and uv resolves via
# OMNIVOICE_BUNDLED_UV/PATH.
@router.post(
"/engines/sidecar/{engine_id}/install",
dependencies=[Depends(require_loopback)],
dependencies=[Depends(require_admin), Depends(require_desktop)],
)
def install_sidecar_engine(engine_id: str):
"""Start (or report) the one-click install for a sidecar engine.
@@ -222,7 +244,7 @@ def install_sidecar_engine(engine_id: str):
@router.get(
"/engines/sidecar/{engine_id}/install/status",
dependencies=[Depends(require_loopback)],
dependencies=[Depends(require_admin)],
)
def sidecar_install_status(engine_id: str):
"""Step-by-step status of the sidecar install job (poll while running).
@@ -243,7 +265,7 @@ def sidecar_install_status(engine_id: str):
@router.delete(
"/engines/sidecar/{engine_id}/install",
dependencies=[Depends(require_loopback)],
dependencies=[Depends(require_admin)],
)
def uninstall_sidecar_engine(engine_id: str):
"""Remove an app-managed sidecar install (checkout + venv + weights) and
@@ -274,29 +296,22 @@ def uninstall_sidecar_engine(engine_id: str):
# frame. Result includes wall-clock latency so the UI can render
# "1234 ms — pong" inline next to the button.
#
# Loopback-gated (T-02-13): only the local desktop frontend may trigger
# a sidecar spawn through this endpoint.
# Admin-gated (T-02-13): only the local desktop frontend or an authenticated
# server-mode administrator may trigger a sidecar spawn through this endpoint.
# Engine instances cached for the lifetime of the FastAPI process so that
# repeated health checks don't spawn a new SubprocessBackend (each spawn
# allocates a sidecar venv probe + atexit hook). The cache is keyed by
# class to survive registry-sandbox tests that rebind ids transiently.
_ENGINE_INSTANCES: dict[type, object] = {}
#
# It now lives in services.tts_backend — the worker executor needs the same
# warm instances and cannot import an API router without inverting the
# layering. This name is the SAME dict object, kept so the existing consumers
# (engine_memory eviction, model_lifecycle inventory/unload) go on working
# unchanged; rebinding it here would fork the cache in two.
_ENGINE_INSTANCES: dict[type, object] = tts_backend._ENGINE_INSTANCES
def _get_engine_instance(cls):
"""Return a cached singleton instance of ``cls``.
SubprocessBackend's ``__init__`` registers an atexit shutdown hook,
so re-instantiating per request would leak handler entries (and on
real engines, additional sidecar processes the first time the lock
is acquired). One instance per process is the right move.
"""
inst = _ENGINE_INSTANCES.get(cls)
if inst is None:
inst = cls()
_ENGINE_INSTANCES[cls] = inst
return inst
_get_engine_instance = tts_backend.get_engine_instance
def _resolve_engine_class(engine_id: str):
@@ -318,7 +333,7 @@ def _resolve_engine_class(engine_id: str):
@router.get(
"/engines/{engine_id}/health",
dependencies=[Depends(require_loopback)],
dependencies=[Depends(require_admin_action)],
)
def engine_health(engine_id: str):
"""Spawn-and-ping a SubprocessBackend; ``is_available()`` for the rest.
@@ -392,7 +407,7 @@ def engine_health(engine_id: str):
# hanging the Settings panel. The orphaned worker is best-effort daemon.
# * A process-wide lock serialises self-tests so a click-storm can't stack
# concurrent model loads.
# * Only ever on user click (POST) — never on Settings load. Loopback-gated.
# * Only ever on user click (POST) — never on Settings load. Admin-gated.
# Deliberately short + ASCII so the synth stays CPU-cheap and the phrase never
# trips the no-hardcoded-CJK guard.
@@ -459,7 +474,7 @@ class SelfTestResponse(BaseModel):
@router.post(
"/engines/{engine_id}/selftest",
response_model=SelfTestResponse,
dependencies=[Depends(require_loopback)],
dependencies=[Depends(require_admin)],
)
def engine_selftest(engine_id: str):
"""Run a bounded, real synthesis on an available in-process TTS engine.
@@ -558,7 +573,11 @@ class SelectEngineResponse(BaseModel):
routing_reason: str | None = None
@router.post("/engines/select", response_model=SelectEngineResponse)
@router.post(
"/engines/select",
response_model=SelectEngineResponse,
dependencies=[Depends(require_admin)],
)
def select_engine(req: SelectEngineRequest):
"""Persist a family's engine pick to prefs.json. Refuses unknown backends,
backends whose deps aren't installed, AND backends that cannot run on THIS
@@ -589,7 +608,7 @@ def select_engine(req: SelectEngineRequest):
# #981: mlx-audio multiplexes 7+ curated models behind one backend id —
# persist the model pick alongside the backend id so the UI can actually
# select which curated model gets loaded (previously it always defaulted
# to Kokoro no matter what the user downloaded in Settings → Models).
# to Kokoro no matter what the user downloaded in Model Catalogue → Models).
if req.family == "tts" and req.backend_id == "mlx-audio" and req.model_id is not None:
known_keys = tts_backend.MLXAudioBackend.CURATED_MODELS
# Accept a curated key OR a raw HF repo id ("owner/name") — the same
+229 -86
View File
@@ -1,18 +1,24 @@
import os
import json
import uuid
import time
import asyncio
import contextlib
import json
import logging
from typing import Optional, List
import os
import re
import shutil
import tempfile
import time
import uuid
from pathlib import Path
from typing import List, Optional
from fastapi import APIRouter, File, Form, UploadFile, HTTPException, Query
from fastapi.responses import FileResponse, RedirectResponse
from fastapi.responses import FileResponse
from pydantic import BaseModel
from core.db import db_conn
from core.config import VOICES_DIR, OUTPUTS_DIR
from core import event_bus
from core.audio_validation import resolve_regular_file
from core.file_cleanup import FileCleanupError, unlink_if_present
from services.ffmpeg_utils import spawn_subprocess
@@ -360,46 +366,223 @@ async def upload_voice_clip(
}
def _stage_profile_audio(source: Path, directory: Path) -> Path:
"""Copy an imported clip to a hidden temp file inside ``directory``.
The temp lives in the destination directory itself so a later
``os.replace`` to the final name is an atomic same-filesystem rename
cheap enough to run while holding a DB write lock, unlike the copy.
Callers own cleanup of the returned path if they never publish it.
"""
directory.mkdir(parents=True, exist_ok=True)
fd, tmp_name = tempfile.mkstemp(
dir=str(directory), prefix=".gallery-import-", suffix=".part",
)
os.close(fd)
try:
shutil.copy2(source, tmp_name)
except BaseException:
with contextlib.suppress(OSError):
os.unlink(tmp_name)
raise
return Path(tmp_name)
def _copy_profile_audio(source: Path, destination: Path) -> None:
"""Copy an imported clip without exposing a partial profile audio file."""
staged = _stage_profile_audio(source, destination.parent)
try:
os.replace(staged, destination)
except BaseException:
with contextlib.suppress(OSError):
os.unlink(staged)
raise
def _gallery_profile_audio_filename(profile_id: str, source: Path) -> str:
"""Return the canonical, portable filename for a My Imports profile."""
safe_id = (
profile_id if re.fullmatch(r"[A-Za-z0-9_-]{1,64}", profile_id or "")
else uuid.uuid5(uuid.NAMESPACE_URL, str(profile_id)).hex[:16]
)
suffix = source.suffix.lower()
if not re.fullmatch(r"\.[a-z0-9]{1,8}", suffix):
suffix = ".wav"
return f"{safe_id}_gallery{suffix}"
def _is_materialized_gallery_profile(row, voice: dict, audio_filename: str) -> bool:
"""Recognize only rows created by this materializer, not identity collisions."""
return bool(
row["personality"] == f"gallery:{voice['id']}"
and row["ref_audio_path"] == audio_filename
and row["ref_text"] == ""
and row["instruct"] == ""
and row["language"] == "Auto"
and row["seed"] is None
and row["kind"] == "clone"
and not row["vd_states"]
and row["description"] == (voice.get("description") or "")
and not row["is_locked"]
and not row["verified_own_voice"]
and not row["locked_audio_path"]
)
def _existing_gallery_profile(conn, voice: dict, source: Path):
personality = f"gallery:{voice['id']}"
rows = conn.execute(
"SELECT * FROM voice_profiles WHERE personality=? ORDER BY created_at, id",
(personality,),
).fetchall()
for row in rows:
expected = _gallery_profile_audio_filename(row["id"], source)
if _is_materialized_gallery_profile(row, voice, expected):
return row
return None
def _gallery_profile_audio_is_current(row, source: Path) -> bool:
"""Detect missing/replaced copies without re-hashing unchanged imports."""
destination = resolve_regular_file(VOICES_DIR, row["ref_audio_path"])
if destination is None:
return False
try:
source_stat = source.stat()
destination_stat = destination.stat()
# copy2 preserves mtime; size + nanosecond mtime catches ordinary edits
# and partial writes while keeping repeated Use clicks inexpensive.
return (
source_stat.st_size == destination_stat.st_size
and source_stat.st_mtime_ns == destination_stat.st_mtime_ns
)
except OSError:
return False
def _materialize_gallery_profile(
voice_id: str, requested_name: Optional[str] = None,
) -> dict:
"""Idempotently materialize/heal one My Imports clip as a clone profile."""
personality = f"gallery:{voice_id}"
copied_path: Optional[Path] = None
created = False
staged_path: Optional[Path] = None
staged_source: Optional[Path] = None
try:
# Stage the (potentially large) audio copy BEFORE taking SQLite's
# write lock: copying inside BEGIN IMMEDIATE would stall every other
# backend writer for the whole copy. The staged temp lives in
# VOICES_DIR itself, so publishing it inside the transaction is an
# atomic same-filesystem os.replace. This pre-read is advisory only —
# the locked transaction below re-reads and re-decides everything.
copy_needed = False
with db_conn() as conn:
pre_row = conn.execute(
"SELECT * FROM voice_gallery WHERE id = ?", (voice_id,),
).fetchone()
if pre_row is not None:
pre_source = Path(pre_row["audio_path"])
if pre_source.is_file():
pre_existing = _existing_gallery_profile(conn, dict(pre_row), pre_source)
copy_needed = pre_existing is None or not _gallery_profile_audio_is_current(
pre_existing, pre_source,
)
if copy_needed:
staged_path = _stage_profile_audio(pre_source, Path(VOICES_DIR))
staged_source = pre_source
with db_conn() as conn:
# The identity is not globally UNIQUE because personality is shared
# with other import mechanisms. Serialize this check+insert in
# SQLite so simultaneous Use clicks cannot both create a row.
conn.execute("BEGIN IMMEDIATE")
row = conn.execute(
"SELECT * FROM voice_gallery WHERE id = ?", (voice_id,),
).fetchone()
if row is None:
raise HTTPException(status_code=404, detail="Voice not found")
voice = dict(row)
source = Path(voice["audio_path"])
if not source.is_file():
raise HTTPException(status_code=404, detail="Audio file not found on disk")
def _install_audio(destination: Path) -> None:
"""Publish the staged copy under the lock via atomic rename."""
nonlocal staged_path
if staged_path is not None and staged_source == source:
os.replace(staged_path, destination)
staged_path = None
else:
# Rare race: the gallery row changed between the advisory
# pre-read and taking the lock, so any staged bytes may be
# from the wrong source. Fall back to the blocking copy
# rather than publish stale audio.
_copy_profile_audio(source, destination)
existing = _existing_gallery_profile(conn, voice, source)
if existing is not None:
ref_filename = _gallery_profile_audio_filename(existing["id"], source)
if not _gallery_profile_audio_is_current(existing, source):
ref_path = Path(VOICES_DIR) / ref_filename
_install_audio(ref_path)
copied_path = ref_path
conn.execute(
"UPDATE voice_profiles SET ref_audio_path=?, ref_text='', instruct='', "
"language='Auto', seed=NULL, description=?, kind='clone', vd_states=NULL, "
"personality=? WHERE id=?",
(
ref_filename, voice["description"] or "", personality,
existing["id"],
),
)
result = {"profile_id": existing["id"], "name": existing["name"]}
else:
profile_id = str(uuid.uuid4())[:8]
profile_name = (requested_name or voice["name"]).strip() or voice["name"]
ref_filename = _gallery_profile_audio_filename(profile_id, source)
copied_path = Path(VOICES_DIR) / ref_filename
_install_audio(copied_path)
conn.execute(
"""INSERT INTO voice_profiles
(id, name, ref_audio_path, ref_text, instruct, language, seed,
personality, is_locked, locked_audio_path, description, kind,
vd_states, created_at)
VALUES (?, ?, ?, '', '', 'Auto', NULL, ?, 0, '', ?, 'clone', NULL, ?)""",
(
profile_id, profile_name, ref_filename, personality,
voice["description"] or "", time.time(),
),
)
created = True
result = {"profile_id": profile_id, "name": profile_name}
except BaseException:
if copied_path is not None:
with contextlib.suppress(OSError):
copied_path.unlink()
raise
finally:
# Staged but never published (failure, or a concurrent request healed
# the profile first) — never leave .part droppings in VOICES_DIR.
if staged_path is not None:
with contextlib.suppress(OSError):
os.unlink(staged_path)
event_bus.emit(
"profiles", {"action": "created" if created else "updated", "id": result["profile_id"]},
)
return result
@router.post("/gallery/voices/{voice_id}/save-as-profile")
async def save_voice_as_profile(
voice_id: str,
profile_name: str = Query(..., description="Name for the voice profile"),
):
"""Save a gallery voice as a voice profile for cloning."""
with db_conn() as conn:
row = conn.execute(
"SELECT * FROM voice_gallery WHERE id = ?", (voice_id,)
).fetchone()
if not row:
raise HTTPException(status_code=404, detail="Voice not found")
profile_id = str(uuid.uuid4())[:8]
import shutil
ext = os.path.splitext(row["audio_path"])[1]
new_audio_path = os.path.join(VOICES_DIR, f"{profile_id}{ext}")
shutil.copy(row["audio_path"], new_audio_path)
conn.execute(
"""
INSERT INTO voice_profiles (id, name, ref_audio_path, ref_text, instruct, language, seed, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
""",
(
profile_id,
profile_name,
f"{profile_id}{ext}",
row["description"] or "",
row["character"] or "",
"Auto",
None,
time.time(),
),
)
event_bus.emit("profiles", {"action": "created", "id": profile_id})
return {"profile_id": profile_id, "name": profile_name}
result = await asyncio.to_thread(_materialize_gallery_profile, voice_id, profile_name)
return {"profile_id": result["profile_id"], "name": result["name"]}
@router.get("/gallery/voices/{voice_id}/preview")
@@ -415,22 +598,10 @@ def preview_voice(voice_id: str):
audio_path = row["audio_path"]
# Debug logging
is_absolute = os.path.isabs(audio_path)
path_exists = os.path.exists(audio_path) if audio_path else False
# If absolute path, serve directly or redirect
if is_absolute and path_exists:
# Get just the relative path from outputs dir
outputs_path = str(OUTPUTS_DIR)
if audio_path.startswith(outputs_path):
# Remove outputs_dir prefix to get relative path within outputs
rel_path = os.path.relpath(audio_path, outputs_path)
# The audio_path is like: /Users/user4/.../outputs/voice_gallery/file.wav
# rel_path becomes: voice_gallery/file.wav
# We want to serve from /audio/ so: /audio/voice_gallery/file.wav
return RedirectResponse(f"/audio/{rel_path}")
return FileResponse(audio_path, media_type="audio/wav")
if os.path.isabs(audio_path) and os.path.exists(audio_path):
# Serve the file from this API route so deployments mounted below a
# path prefix do not lose that prefix while following a redirect.
return FileResponse(audio_path)
raise HTTPException(
status_code=404,
@@ -503,33 +674,5 @@ def batch_delete_voices(body: dict):
@router.post("/gallery/voices/{voice_id}/to-profile")
def voice_to_profile(voice_id: str):
"""Create a voice profile from a gallery clip."""
with db_conn() as conn:
row = conn.execute("SELECT * FROM voice_gallery WHERE id = ?", (voice_id,)).fetchone()
if not row:
raise HTTPException(status_code=404, detail="Voice not found")
voice = dict(row)
audio_path = voice["audio_path"]
if not os.path.exists(audio_path):
raise HTTPException(status_code=404, detail="Audio file not found on disk")
import shutil
import uuid
profile_id = str(uuid.uuid4())[:8]
# Copy audio to voices dir
dest_filename = f"{profile_id}_gallery.wav"
dest_path = os.path.join(VOICES_DIR, dest_filename)
shutil.copy2(audio_path, dest_path)
import time
now = time.time()
conn.execute(
"""INSERT INTO voice_profiles
(id, name, ref_audio_path, ref_text, instruct, seed, is_locked, locked_audio_path, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)""",
(profile_id, voice["name"], dest_filename, "", None, None, 0, None, now, now),
)
event_bus.emit("profiles", {"action": "created", "id": profile_id})
return {"success": True, "profile_id": profile_id, "name": voice["name"]}
result = _materialize_gallery_profile(voice_id)
return {"success": True, "profile_id": result["profile_id"], "name": result["name"]}
File diff suppressed because it is too large Load Diff
+2 -2
View File
@@ -9,13 +9,13 @@ from __future__ import annotations
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel, Field
from api.dependencies import require_loopback
from api.dependencies import require_admin
from services import mcp_bindings
router = APIRouter(
prefix="/api/mcp",
tags=["mcp"],
dependencies=[Depends(require_loopback)],
dependencies=[Depends(require_admin)],
)
+2 -2
View File
@@ -12,10 +12,10 @@ import logging
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from api.dependencies import require_loopback
from api.dependencies import require_admin
logger = logging.getLogger("omnivoice.api")
router = APIRouter(dependencies=[Depends(require_loopback)])
router = APIRouter(dependencies=[Depends(require_admin)])
class CustomPathRequest(BaseModel):
+49 -12
View File
@@ -160,7 +160,9 @@ _OPENAI_VOICE_ALIASES = {
def _resolve_engine(model_id: str):
"""Map an OpenAI model name to a VoiceStudio backend."""
from services.tts_backend import get_backend_class, get_active_tts_backend
from services.tts_backend import (
get_backend_class, get_active_tts_backend, get_engine_instance_for,
)
# Accept OpenAI model names as pass-through to the active engine.
if model_id in ("tts-1", "tts-1-hd"):
@@ -177,8 +179,18 @@ def _resolve_engine(model_id: str):
)
from services.tts_backend import OmniVoiceBackend
if cls is OmniVoiceBackend:
# OmniVoice only ever runs as the shared active engine — the
# explicit-omnivoice request is the active-engine request.
return get_active_tts_backend()
return cls()
# Cached singleton, not a fresh cls(): SubprocessBackend engines would
# spawn a sidecar process and reload their model on EVERY request, and
# register a new atexit hook each time (get_engine_instance's contract).
# No router-local cache on top of it: the shared cache is keyed by
# CLASS precisely so id rebinds/evictions can't serve a stale instance,
# and cross-engine memory discipline is create_speech's
# evict_other_tts_engines call (the same seam /generate uses) — not a
# bespoke unload here.
return get_engine_instance_for(model_id)
except ValueError:
raise HTTPException(
status_code=400,
@@ -388,6 +400,15 @@ async def create_speech(req: SpeechRequest):
# VRAM eviction runs in get_model()'s warm-return path now, covering every
# native TTS generate (this route, WS TTS, dub, batch, audiobook).
# Single-active-engine memory discipline (MM2-01), the same call /generate
# makes before its load: hand back every OTHER resident TTS engine's model
# before this one warms up, so switching `model` ids across requests —
# explicit id → explicit id, or explicit id → the tts-1/omnivoice aliases —
# can't stack multi-GB engines/sidecars. No-op when nothing else is
# resident; opt out with OMNIVOICE_SINGLE_ENGINE_RESIDENT=0.
from services.engine_memory import evict_other_tts_engines
await evict_other_tts_engines(backend.id)
# ── #1033/#1037/#1014: warm the engine under the LOAD budget before the
# generate clock starts. The T4 verification (#1014) measured a fresh
# install's first /v1/audio/speech burning its whole 300s generate budget
@@ -415,7 +436,7 @@ async def create_speech(req: SpeechRequest):
detail=(
f"TTS engine '{backend.id}' did not finish loading within its "
f"model-load budget — on a first run this usually means the weight "
f"download is slow or stalled (check Settings → Models for "
f"download is slow or stalled (check Model Catalogue → Models for "
f"progress), not that generation failed. Retry once the model "
f"shows as installed."
),
@@ -454,7 +475,7 @@ async def create_speech(req: SpeechRequest):
from services.model_manager import generate_timeout_s
wav, sr = await run_on_gpu_pool_guarded(
lambda: _run_tts(backend, text, kw), what="OpenAI TTS generate",
timeout=generate_timeout_s(text))
timeout=generate_timeout_s(text, engine=backend))
except Exception as e:
# #1172/#1173: typed failures get their real status + actionable
# message (400 bad input / 503 broken engine binary) instead of a
@@ -517,9 +538,10 @@ async def create_transcription(
):
"""Transcribe audio to text. Compatible with OpenAI's POST /v1/audio/transcriptions."""
from services.asr_backend import (
ASRModelMissingError,
asr_model_missing_detail,
asr_model_missing_error,
get_active_asr_backend,
load_active_asr_backend,
)
# TTS-only install: no ASR model on disk → actionable 409, BEFORE any
@@ -546,18 +568,25 @@ async def create_transcription(
raise HTTPException(status_code=400, detail=f"Could not read audio file: {e}")
try:
backend = get_active_asr_backend()
# Run transcription in the thread pool to avoid blocking the event loop,
# bounded so a stuck/starved ASR returns a 504 with guidance instead of
# hanging the request forever (see run_transcribe_guarded).
from services.asr_backend import run_transcribe_guarded
word_ts = response_format == "verbose_json"
result = await run_transcribe_guarded(
_gpu_pool,
lambda: backend.transcribe(tmp_path, word_timestamps=word_ts),
what="OpenAI",
)
# `load_active_asr_backend`, not `get_active_asr_backend`: the latter is
# a pure selector, so a backend whose shallow `is_available()` probe
# passes but whose deep import chain is broken (whisperx →
# ctranslate2 failing to dlopen on a hardened kernel) reached
# `.transcribe()` and 500'd, even with a healthy engine next in line.
# The loader does select + ensure_loaded + degrade (#1185). It loads
# weights, so it belongs inside the pool with the transcribe call —
# never on the event loop.
def _run():
backend = load_active_asr_backend()
return backend.transcribe(tmp_path, word_timestamps=word_ts)
result = await run_transcribe_guarded(_gpu_pool, _run, what="OpenAI")
# Extract the full text from segments
segments = result.get("segments", [])
@@ -625,6 +654,14 @@ async def create_transcription(
except HTTPException:
raise
except ASRModelMissingError as e:
# A degraded-to candidate has no weights on disk. Same typed 409 the
# preflight above raises — never a 500, and never a silent multi-GB
# auto-download.
raise HTTPException(
status_code=409,
detail={**e.payload, "message": asr_model_missing_detail(e.payload)},
)
except TimeoutError as e:
# ASRTimeoutError (subclass): backend alive, ASR too heavy for compute.
logger.warning("OpenAI transcription timed out: %s", e)
+10 -10
View File
@@ -7,7 +7,7 @@ CRUD for the DB-backed, per-language pronunciation dictionary the
before synthesis (see ``services/pronunciation.apply_pronunciation`` and the
generate path), so a saved entry actually changes the audio on every engine.
Endpoints (loopback-only, like the dictation router):
Endpoints (admin-gated; loopback or authenticated server mode):
GET /pronunciation list every entry
POST /pronunciation create one entry
PUT /pronunciation/{entry_id} update an entry (partial)
@@ -30,12 +30,12 @@ from typing import List, Optional
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from api.dependencies import require_loopback
from api.dependencies import require_admin
from core.db import db_conn
from services.pronunciation import apply_pronunciation, entries_for_language
logger = logging.getLogger("omnivoice.pronunciation")
router = APIRouter()
router = APIRouter(dependencies=[Depends(require_admin)])
_VALID_TYPES = ("respelling", "ipa", "cmu")
_ALL_LANG = "*"
@@ -133,7 +133,7 @@ class PronImportRequest(BaseModel):
# ── CRUD ─────────────────────────────────────────────────────────────────────
@router.get("/pronunciation", dependencies=[Depends(require_loopback)])
@router.get("/pronunciation")
def list_entries():
with db_conn() as conn:
rows = conn.execute(
@@ -143,7 +143,7 @@ def list_entries():
return [_row_to_dict(r) for r in rows]
@router.post("/pronunciation", dependencies=[Depends(require_loopback)])
@router.post("/pronunciation")
def create_entry(entry: PronEntry):
term = entry.term.strip()
if not term:
@@ -171,7 +171,7 @@ def create_entry(entry: PronEntry):
return _row_to_dict(row)
@router.put("/pronunciation/{entry_id}", dependencies=[Depends(require_loopback)])
@router.put("/pronunciation/{entry_id}")
def update_entry(entry_id: str, patch: PronEntryUpdate):
with db_conn() as conn:
existing = conn.execute(
@@ -226,7 +226,7 @@ def update_entry(entry_id: str, patch: PronEntryUpdate):
return _row_to_dict(row)
@router.delete("/pronunciation/{entry_id}", dependencies=[Depends(require_loopback)])
@router.delete("/pronunciation/{entry_id}")
def delete_entry(entry_id: str):
with db_conn() as conn:
cur = conn.execute("DELETE FROM pronunciation_entries WHERE id = ?", (entry_id,))
@@ -236,7 +236,7 @@ def delete_entry(entry_id: str):
# ── Dry-run + import/export ───────────────────────────────────────────────────
@router.post("/pronunciation/test", dependencies=[Depends(require_loopback)])
@router.post("/pronunciation/test")
def test_substitution(req: PronTestRequest):
"""Show the post-substitution text for ``req.text`` — no model call.
@@ -258,7 +258,7 @@ def test_substitution(req: PronTestRequest):
}
@router.get("/pronunciation/export", dependencies=[Depends(require_loopback)])
@router.get("/pronunciation/export")
def export_entries():
"""Every entry as a JSON-serializable list (round-trips ``/import``)."""
with db_conn() as conn:
@@ -273,7 +273,7 @@ def export_entries():
]}
@router.post("/pronunciation/import", dependencies=[Depends(require_loopback)])
@router.post("/pronunciation/import")
def import_entries(req: PronImportRequest):
"""Bulk-add entries. ``replace=true`` clears the table first.
+88 -7
View File
@@ -20,7 +20,7 @@ from fastapi import APIRouter, Depends, HTTPException, Query
from pydantic import BaseModel, Field
from core.logging_utils import log_safe
from api.dependencies import require_admin
from api.dependencies import require_admin, require_admin_action
logger = logging.getLogger("omnivoice.api.settings")
@@ -92,8 +92,8 @@ def get_hf_token_state(fresh: bool = Query(False)):
# ── Performance settings (INST-12) ────────────────────────────────────────
# Threat T-02-04: same loopback guard as the hf-token endpoints via the
# router-level `require_loopback` dep.
# Threat T-02-04: same admin guard as the hf-token endpoints via the
# router-level `require_admin` dep.
_TORCH_COMPILE_KEY = "perf.torch_compile_disabled"
@@ -133,6 +133,83 @@ def set_torch_compile_disabled(body: _TorchCompileBody):
return _torch_compile_state()
# ── Compute-device override (Settings → Performance) ──────────────────────
class _ComputeDeviceBody(BaseModel):
value: str = Field(..., description="auto | cuda | rocm | xpu | mps | cpu")
def _compute_device_state() -> dict:
"""Everything the Performance panel needs to render the device control:
the resolved pick (env > prefs > auto), what this process actually applied
at probe time (differs after a change until restart caps are immutable
per process), what auto would pick, and which families exist here."""
from core import device_caps
caps = device_caps.detect_host_caps()
env_pin = (os.environ.get("OMNIVOICE_DEVICE") or "").strip().lower()
auto_family = next(
(f for f in ("cuda", "rocm", "xpu", "mps") if f in caps.available_families),
"cpu",
)
value = device_caps.requested_device_override()
return {
"value": value,
"applied": caps.requested_family,
"restart_required": value != caps.requested_family,
# The running process asked for a family it doesn't have (env pin on
# the wrong machine, hardware removed): auto is in effect, and a
# restart would not change that — the panel says so instead of
# pretending the pick took.
"override_ignored": (
caps.requested_family not in ("auto", caps.family)
),
"effective_family": caps.family,
"auto_family": auto_family,
"available_families": list(caps.available_families),
"env_pinned": env_pin in device_caps.DEVICE_OVERRIDE_CHOICES and env_pin != "",
"choices": list(device_caps.DEVICE_OVERRIDE_CHOICES),
}
@router.get("/compute-device")
def get_compute_device():
"""Current compute-device override state (Settings → Performance)."""
return _compute_device_state()
@router.put("/compute-device")
def set_compute_device(body: _ComputeDeviceBody):
"""Persist the compute-device pick. Applied by the capability probe at
the next backend start (host caps are immutable per process same
restart contract as the rest of the Performance tab). ``OMNIVOICE_DEVICE``
always wins over this pick; the UI shows the pin instead of pretending."""
from core import device_caps, prefs
value = (body.value or "").strip().lower()
if value not in device_caps.DEVICE_OVERRIDE_CHOICES:
raise HTTPException(
status_code=400,
detail=f"Unknown device '{value}'. Valid: {', '.join(device_caps.DEVICE_OVERRIDE_CHOICES)}",
)
caps = device_caps.detect_host_caps()
if value not in ("auto", "cpu") and value not in caps.available_families:
raise HTTPException(
status_code=400,
detail=(
f"'{value}' is not available on this host "
f"(have: {', '.join(caps.available_families)})"
),
)
try:
prefs.set_("compute_device", value)
except Exception:
logger.exception("set_compute_device failed")
raise HTTPException(status_code=500, detail="Failed to persist setting")
return _compute_device_state()
# ── Generation-history retention (Studio takes rail) ──────────────────────
@@ -481,7 +558,10 @@ def _local_models(base_url: str, api_key: str):
return None
@router.get("/llm-providers/{provider_id}/models")
@router.get(
"/llm-providers/{provider_id}/models",
dependencies=[Depends(require_admin_action)],
)
def list_llm_provider_models(provider_id: str):
"""List model ids the provider's key can access (OpenAI-compat /models).
@@ -955,9 +1035,10 @@ def set_asr_openai_compat(body: _ASROpenAICompatBody):
from services import asr_backend, settings_store
if body.base_url is not None:
url = body.base_url.strip().rstrip("/")
if url and not url.startswith(("http://", "https://")):
raise HTTPException(status_code=400, detail="Base URL must start with http(s)://")
try:
url = asr_backend.normalize_openai_compat_asr_base_url(body.base_url)
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
settings_store.set_text(asr_backend._ASR_OPENAI_COMPAT_BASE_URL_KEY, url)
if body.model is not None:
settings_store.set_text(
+70 -17
View File
@@ -76,6 +76,7 @@ _cancelled: set[str] = set()
_active_installs: set[str] = set()
_active_installs_lock = threading.Lock()
_install_tasks: set[asyncio.Task] = set()
_install_tasks_by_repo: dict[str, asyncio.Task] = {}
def _download_max_workers() -> int:
@@ -296,17 +297,19 @@ def _validate_snapshot_has_weights(repo_id: str, snapshot_path: str) -> None:
f"{repo_id}: download finished but no model weights were found in the "
"snapshot (largest file "
f"{biggest} bytes). The download was likely interrupted — delete the "
"model in Settings → Models and install it again."
"model in Model Catalogue → Models and install it again."
)
@router.get("/setup/download-stream")
async def setup_download_stream():
async def setup_download_stream(target: str | None = None):
"""SSE: forward every HuggingFace download tqdm update as a JSON event."""
queue: asyncio.Queue = asyncio.Queue(maxsize=512)
loop = asyncio.get_running_loop()
def listener(event):
if target and event.get("target", "local") != target:
return
try:
loop.call_soon_threadsafe(_safe_put, queue, event)
except RuntimeError:
@@ -340,6 +343,7 @@ async def setup_download_stream():
class InstallModelRequest(BaseModel):
repo_id: str
target: str | None = None
@@ -389,6 +393,21 @@ async def install_model(req: InstallModelRequest):
+ ", ".join(m["repo_id"] for m in KNOWN_MODELS)
),
)
target = (req.target or "").strip()
if target != "local":
from services import gpu_gateway # noqa: PLC0415
from worker import routing # noqa: PLC0415
decision = routing.decide()
if target and target != "local" and (
not decision.remote or decision.worker_id != target
):
raise HTTPException(status_code=409, detail="The selected GPU target changed; try again.")
if decision.remote:
try:
return await gpu_gateway.download(req.repo_id, decision=decision)
except gpu_gateway.GatewayError as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
# Cooldown guard — don't retry if the same model just failed.
import time as _time_check
_sweep_cooldowns(_time_check.time()) # bound the dict (MM2-06)
@@ -402,15 +421,11 @@ async def install_model(req: InstallModelRequest):
f"Retry in {remaining}s or check your network."
),
)
with _active_installs_lock:
if req.repo_id in _active_installs:
return {"status": "already_running", "repo_id": req.repo_id}
_active_installs.add(req.repo_id)
loop = asyncio.get_running_loop()
def _do():
token = hf_progress.current_repo_id.set(req.repo_id)
_cancelled.discard(req.repo_id) # clear any stale cancel from a prior run
target_token = hf_progress.current_target.set("local")
hf_progress.emit({
"repo_id": req.repo_id,
"filename": req.repo_id,
@@ -505,6 +520,7 @@ async def install_model(req: InstallModelRequest):
return
download_aggregator.start(
req.repo_id,
target=target or "local",
total_bytes=_summary["to_download_bytes"],
files_total=max(0, _summary["n_files"] - _summary["n_cached"]),
)
@@ -518,7 +534,7 @@ async def install_model(req: InstallModelRequest):
# No preflight (older/gated repo, mirror without dry-run, etc.):
# fall back to today's fill-in-as-files-appear behaviour.
logger.info("model install %s: preflight unavailable (%s)", req.repo_id, _pf_err)
download_aggregator.start(req.repo_id)
download_aggregator.start(req.repo_id, target=target or "local")
hf_progress.emit({
"repo_id": req.repo_id,
"filename": req.repo_id,
@@ -618,7 +634,7 @@ async def install_model(req: InstallModelRequest):
# Flush the overall bar to 100% with the true byte total (FDL-06):
# under Xet the per-file byte bars don't surface completion, so the
# aggregator can sit below 100% even though every file landed.
download_aggregator.complete(req.repo_id)
download_aggregator.complete(req.repo_id, target=target or "local")
logger.info("model install done: %s", req.repo_id)
hf_progress.emit({
"repo_id": req.repo_id,
@@ -662,22 +678,59 @@ async def install_model(req: InstallModelRequest):
})
finally:
_cancelled.discard(req.repo_id)
download_aggregator.finish(req.repo_id)
download_aggregator.finish(req.repo_id, target=target or "local")
hf_progress.current_repo_id.reset(token)
hf_progress.current_target.reset(target_token)
with _active_installs_lock:
_active_installs.discard(req.repo_id)
try:
task = loop.create_task(asyncio.to_thread(_do))
_install_tasks.add(task)
task.add_done_callback(_install_tasks.discard)
except Exception:
with _active_installs_lock:
with _active_installs_lock:
if req.repo_id in _active_installs:
return {"status": "already_running", "repo_id": req.repo_id}
_active_installs.add(req.repo_id)
# Admission and task publication are one atomic generation boundary:
# cancellation can never observe an admitted install without its task.
_cancelled.discard(req.repo_id)
try:
task = loop.create_task(asyncio.to_thread(_do))
_install_tasks.add(task)
_install_tasks_by_repo[req.repo_id] = task
except Exception:
_active_installs.discard(req.repo_id)
raise
raise
def install_finished(completed: asyncio.Task) -> None:
with _active_installs_lock:
_install_tasks.discard(completed)
if _install_tasks_by_repo.get(req.repo_id) is completed:
_install_tasks_by_repo.pop(req.repo_id, None)
task.add_done_callback(install_finished)
return {"status": "install_started", "repo_id": req.repo_id}
async def cancel_install_and_wait(repo_id: str) -> None:
"""Request cancellation and retain authority until its thread exits."""
from worker.async_utils import drain_task # noqa: PLC0415
with _active_installs_lock:
_cancelled.add(repo_id)
_install_cooldowns.pop(repo_id, None)
task = _install_tasks_by_repo.get(repo_id)
if task is None:
return
try:
# asyncio.to_thread cannot stop snapshot_download mid-file. Cancelling
# its wrapper would only detach the thread, so wait until the blocking
# call observes the flag or naturally returns.
await drain_task(task)
finally:
with _active_installs_lock:
current = _install_tasks_by_repo.get(repo_id)
if current is None or current is task:
_cancelled.discard(repo_id)
@router.post("/models/install/cancel")
async def cancel_install(req: InstallModelRequest):
"""Request cancellation of an in-flight install (FDL-11).
+115 -42
View File
@@ -90,6 +90,36 @@ def get_model_catalog() -> ModelCatalog:
# ── Platform Detection ─────────────────────────────────────────────────────
def _target_worker():
"""Selected live remote worker, or None when the catalog targets local."""
try:
from worker import routing, service # noqa: PLC0415
decision = routing.decide()
plane = service.control_plane
return plane.pool.get(decision.worker_id) if decision.remote and plane.pool else None
except Exception:
return None
def _target_host() -> dict | None:
"""Selected remote worker host, or None when the catalog targets local."""
live = _target_worker()
return dict(live.record.host or {}) if live is not None else None
def _target_repo_inventory() -> tuple[str, set[str]] | None:
"""Selected worker id and the catalog repositories it reports on disk."""
live = _target_worker()
if live is None:
return None
downloaded: set[str] = set()
for capability in live.record.capabilities or []:
if capability.get("downloaded"):
downloaded.update(str(repo) for repo in capability.get("repo_ids") or [])
return live.id, downloaded
def _current_platform_tags() -> list[str]:
"""Return platform tags that the current host supports.
@@ -100,6 +130,25 @@ def _current_platform_tags() -> list[str]:
``rocm`` (AMD HIP builds), and ``cpu`` (no GPU acceleration at all
Apple Silicon is NOT tagged cpu; it curates via ``darwin-arm64``).
"""
target = _target_host()
if target is not None:
target_os = {"windows": "win32", "darwin": "darwin"}.get(
str(target.get("os") or "").lower(), "linux"
)
arch = str(target.get("arch") or "").lower()
arch = {"amd64": "x86_64", "aarch64": "arm64"}.get(arch, arch)
tags = [target_os, f"{target_os}-{arch}"]
backend = ""
if target.get("gpus"):
backend = str(target["gpus"][0].get("backend") or "").lower()
if backend:
tags.append(backend)
if backend == "rocm":
tags.append("cuda")
if not backend and not (target_os == "darwin" and arch == "arm64"):
tags.append("cpu")
return tags
tags = [sys.platform]
arch = _platform.machine()
tags.append(f"{sys.platform}-{arch}")
@@ -455,35 +504,52 @@ def list_models():
Uses a 10 s response cache to avoid repeated ``scan_cache_dir()`` disk
walks when the frontend polls.
"""
cached_response = _cached("models")
platform_tags = _current_platform_tags()
remote_inventory = _target_repo_inventory()
target_key = remote_inventory[0] if remote_inventory else "local"
cache_key = "models:" + target_key + ":" + ",".join(sorted(platform_tags))
cached_response = _cached(cache_key)
if cached_response is not None:
return cached_response
cached_by_repo: dict[str, dict] = {}
try:
from huggingface_hub import scan_cache_dir
info = scan_cache_dir()
for entry in info.repos:
cached_by_repo[entry.repo_id] = {
"size_on_disk": entry.size_on_disk,
"last_accessed": entry.last_accessed,
"nb_files": entry.nb_files,
}
except Exception as e:
# WinError-448 fallback (#117/#118): use a direct disk scan so installed
# models still show as installed instead of offering a re-download.
logger.warning("scan_cache_dir failed (%s); using disk fallback", e)
cached_by_repo = _scan_cache_on_disk()
if remote_inventory is not None:
for model in KNOWN_MODELS:
if model["repo_id"] in remote_inventory[1]:
cached_by_repo[model["repo_id"]] = {
"size_on_disk": int(float(model.get("size_gb") or 0) * _GIB),
"last_accessed": None,
"nb_files": 0,
}
else:
try:
from huggingface_hub import scan_cache_dir
info = scan_cache_dir()
for entry in info.repos:
cached_by_repo[entry.repo_id] = {
"size_on_disk": entry.size_on_disk,
"last_accessed": entry.last_accessed,
"nb_files": entry.nb_files,
}
except Exception as e:
# WinError-448 fallback (#117/#118): use a direct disk scan so installed
# models still show as installed instead of offering a re-download.
logger.warning("scan_cache_dir failed (%s); using disk fallback", e)
cached_by_repo = _scan_cache_on_disk()
out = []
host_tags = set(_current_platform_tags())
host_tags = set(platform_tags)
for m in KNOWN_MODELS:
cached = cached_by_repo.get(m["repo_id"])
on_disk = cached is not None and cached["size_on_disk"] > 0
on_disk = (
m["repo_id"] in remote_inventory[1]
if remote_inventory is not None
else cached is not None and cached["size_on_disk"] > 0
)
# A size-positive cache can still be a truncated download (config landed,
# weight shard didn't). Treat that as not-installed + incomplete so the
# wizard re-offers the download instead of stranding the user (#622).
incomplete = on_disk and not cache_is_complete(m)
incomplete = on_disk and remote_inventory is None and not cache_is_complete(m)
out.append({
**m,
"installed": on_disk and not incomplete,
@@ -498,14 +564,14 @@ def list_models():
response = {
"models": out,
"total_installed_bytes": sum(m["size_on_disk_bytes"] for m in out),
"hf_cache_dir": hf_cache_dir(),
"hf_cache_dir": "" if remote_inventory is not None else hf_cache_dir(),
# Free space on the cache volume, so the Model Store header can warn
# BEFORE an "Install all" overruns the disk (pairs with the per-install
# disk_space_error guard in setup/download.py).
"disk_free_gb": round(disk_free_bytes() / _GIB, 1),
"platform_tags": _current_platform_tags(),
"disk_free_gb": None if remote_inventory is not None else round(disk_free_bytes() / _GIB, 1),
"platform_tags": platform_tags,
}
_set_cache("models", response)
_set_cache(cache_key, response)
return response
@@ -518,18 +584,19 @@ def recommendations():
TTS model is required; the ASR picks here are the optional "best for your
system" set the wizard and Settings surface for on-demand install.
"""
is_mac_arm = sys.platform == "darwin" and _platform.machine() == "arm64"
is_mac_intel = sys.platform == "darwin" and _platform.machine() == "x86_64"
is_linux = sys.platform.startswith("linux")
is_windows = sys.platform == "win32"
tags = set(_current_platform_tags())
target_os = "darwin" if "darwin" in tags else "win32" if "win32" in tags else "linux"
target_arch = next((tag.split("-", 1)[1] for tag in tags if tag.startswith(target_os + "-")), _platform.machine())
is_mac_arm = target_os == "darwin" and target_arch == "arm64"
is_mac_intel = target_os == "darwin" and target_arch == "x86_64"
is_linux = target_os == "linux"
is_windows = target_os == "win32"
has_cuda = "cuda" in tags and "rocm" not in tags
has_rocm = "rocm" in tags
# Device label — used as the card title.
if is_mac_arm:
device_label = f"Apple Silicon ({_platform.machine()})"
device_label = f"Apple Silicon ({target_arch})"
elif is_mac_intel:
device_label = "macOS Intel (x86_64)"
elif is_windows:
@@ -537,7 +604,7 @@ def recommendations():
elif is_linux:
device_label = "Linux x64" + (" + CUDA" if has_cuda else " + ROCm" if has_rocm else "")
else:
device_label = f"{sys.platform} / {_platform.machine()}"
device_label = f"{target_os} / {target_arch}"
# Curated preset for this host, in catalog order (required entries lead).
curated = [
@@ -574,24 +641,30 @@ def recommendations():
"instant English TTS."
)
remote_inventory = _target_repo_inventory()
cached_ids: set[str] = set()
try:
from huggingface_hub import scan_cache_dir
info = scan_cache_dir()
cached_ids = {
entry.repo_id for entry in info.repos if entry.size_on_disk > 0
}
except Exception as e:
# WinError-448 fallback (#117/#118): recommend based on the disk scan.
logger.debug("scan_cache_dir failed (%s); using disk fallback", e)
cached_ids = set(_scan_cache_on_disk().keys())
if remote_inventory is not None:
cached_ids = remote_inventory[1]
else:
try:
from huggingface_hub import scan_cache_dir
info = scan_cache_dir()
cached_ids = {
entry.repo_id for entry in info.repos if entry.size_on_disk > 0
}
except Exception as e:
# WinError-448 fallback (#117/#118): recommend based on the disk scan.
logger.debug("scan_cache_dir failed (%s); using disk fallback", e)
cached_ids = set(_scan_cache_on_disk().keys())
entries = []
for meta in curated:
rid = meta["repo_id"]
# Mirror /models: a truncated cache (weights missing) is not installed, so
# the wizard counts it toward the remaining download instead of "all set".
installed = rid in cached_ids and cache_is_complete(meta)
installed = rid in cached_ids and (
remote_inventory is not None or cache_is_complete(meta)
)
entries.append({
"repo_id": rid,
"label": meta.get("label", rid),
@@ -607,8 +680,8 @@ def recommendations():
return {
"device": {
"os": sys.platform,
"arch": _platform.machine(),
"os": target_os,
"arch": target_arch,
"is_mac_arm": is_mac_arm,
"is_mac_intel": is_mac_intel,
"is_linux": is_linux,
+24 -8
View File
@@ -62,6 +62,11 @@ def setup_status():
_MIN_NVIDIA_DRIVER = 555
_RAM_FAIL_GB = 8
_RAM_WARN_GB = 12
# Installed DIMMs never fully reach the OS: firmware, integrated graphics and
# kernel reservations shave off up to ~7% (an "8 GB" Windows laptop reports
# ~7.8 GB usable). Thresholds are compared with this allowance applied so the
# machines a threshold is meant to admit aren't blocked by that gap (#1618).
_RAM_RESERVED_ALLOWANCE = 0.93
def _run_cmd(args: list[str], timeout: float = 2.0) -> tuple[int, str]:
@@ -183,7 +188,7 @@ def _hf_endpoint_host() -> tuple[str, int]:
"""Host/port of the Hugging Face endpoint actually in effect.
Mirror-aware: restricted-network users (e.g. behind the Great Firewall)
point HF_ENDPOINT at a mirror via Settings Models Hugging Face
point HF_ENDPOINT at a mirror via Model Catalogue Models Hugging Face
mirror. Probing hardcoded huggingface.co would fail them even when their
configured mirror works fine.
"""
@@ -281,7 +286,7 @@ def _network_check() -> dict:
"id": "network", "label": "Network (configured endpoint)",
"status": "warn",
"detail": "The configured Hugging Face endpoint could not be validated.",
"fix": "Review the endpoint in Settings → Models, then re-check.",
"fix": "Review the endpoint in Model Catalogue → Models, then re-check.",
"mirror_reachable": False,
}
net_ok = _probe_network(net_host, net_port)
@@ -352,17 +357,28 @@ def preflight():
# ── RAM
ram = _ram_gb()
# Escape hatch (#1618): a preflight should inform, not brick setup —
# OMNIVOICE_RAM_PREFLIGHT=0 downgrades the hard block to a warning for
# users who accept the OOM risk. Same opt-out shape as
# OMNIVOICE_ASR_VRAM_PREFLIGHT.
ram_gate = os.environ.get(
"OMNIVOICE_RAM_PREFLIGHT", "1"
).strip().lower() not in ("0", "false", "no")
if ram == 0:
ram_status, ram_detail, ram_fix = (
"warn", "Could not detect system RAM.",
"Install psutil in the backend environment or ignore this warning.",
)
elif ram < _RAM_FAIL_GB:
elif ram < _RAM_FAIL_GB * _RAM_RESERVED_ALLOWANCE:
ram_status, ram_detail, ram_fix = (
"fail", f"{ram:.1f} GB total (need ≥ {_RAM_FAIL_GB} GB)",
"The app will OOM on first dub. Close other apps or upgrade RAM.",
"fail" if ram_gate else "warn",
f"{ram:.1f} GB total (need ≥ {_RAM_FAIL_GB} GB)",
"The app will OOM on first dub. Close other apps or upgrade RAM."
if ram_gate else
"RAM check disabled via OMNIVOICE_RAM_PREFLIGHT=0 — dubbing may "
"OOM on this machine.",
)
elif ram < _RAM_WARN_GB:
elif ram < _RAM_WARN_GB * _RAM_RESERVED_ALLOWANCE:
ram_status, ram_detail, ram_fix = (
"warn", f"{ram:.1f} GB total ({_RAM_WARN_GB}+ GB recommended)",
"Long videos may hit swap. Keep other apps closed during dubbing.",
@@ -496,10 +512,10 @@ def preflight():
elif _rs == "unavailable":
r_status, r_detail, r_fix = "fail", (
f"{_eng} can't run on this host: {_why or 'needs a GPU this machine lacks'}"), (
"Select an engine with a CPU path in Settings → Engines.")
"Select an engine with a CPU path in Model Catalogue → Engines.")
else: # "none" / unknown
r_status, r_detail, r_fix = "warn", "No active TTS engine resolved for routing.", (
"Pick an engine in Settings → Engines.")
"Pick an engine in Model Catalogue → Engines.")
checks.append({
"id": "gpu_routing", "label": "Active engine routing",
"status": r_status, "detail": r_detail, "fix": r_fix,
+160
View File
@@ -0,0 +1,160 @@
"""Discovery contract for VoiceStudio's local speech platform.
Interfaces should discover this document instead of hard-coding whichever
dictation route the desktop happens to use. Endpoint URLs are relative so the
same response works on loopback, a tailnet GPU host, and a reverse proxy.
"""
from __future__ import annotations
import os
from typing import Literal
from fastapi import APIRouter
from pydantic import BaseModel, Field
from core.version import APP_VERSION
router = APIRouter(tags=["Speech Platform"])
SPEECH_PROTOCOL = "voicestudio.speech.v1"
STREAM_PATH = "/v1/audio/transcriptions/stream"
class EndpointCapability(BaseModel):
path: str
transport: Literal["http", "websocket", "mcp-streamable-http", "mcp-stdio"]
method: str | None = None
protocol: str | None = None
class StreamInputCapability(BaseModel):
framing: Literal["binary"] = "binary"
formats: list[str]
default_format: str
sample_rate_query: str = "sr"
end_control: dict[str, str]
class StreamOutputCapability(BaseModel):
framing: Literal["json"] = "json"
events: list[str]
final_kinds: list[str]
class SpeechFeatureCapabilities(BaseModel):
batch_transcription: bool = True
streaming_transcription: bool = True
partial_transcripts: bool = True
utterance_finals: bool = True
session_summary: bool = True
word_timestamps: bool = True
local_refinement: bool = True
acoustic_echo_cancellation: bool = True
native_dictation_control: bool = False
class SpeechAuthCapabilities(BaseModel):
loopback: Literal["none"] = "none"
remote: Literal["bearer"] = "bearer"
header: str = "Authorization: Bearer <OMNIVOICE_API_KEY>"
browser_session_endpoint: str = "/api/auth/session"
websocket_ticket_endpoint: str = "/api/auth/ws-ticket"
websocket_ticket_query_parameter: Literal["ws_ticket"] = "ws_ticket"
class SpeechCapabilities(BaseModel):
schema_: Literal["voicestudio.speech-capabilities"] = Field(
default="voicestudio.speech-capabilities",
serialization_alias="schema",
)
protocol: Literal["voicestudio.speech.v1"] = SPEECH_PROTOCOL
protocol_version: Literal["1.0"] = "1.0"
service: str = "VoiceStudio"
service_version: str = APP_VERSION
local_first: bool = True
endpoints: dict[str, EndpointCapability]
stream_input: StreamInputCapability
stream_output: StreamOutputCapability
features: SpeechFeatureCapabilities
authentication: SpeechAuthCapabilities
def speech_capabilities() -> SpeechCapabilities:
"""Return the stable, side-effect-free integration contract."""
endpoints = {
"capabilities": EndpointCapability(
path="/.well-known/voicestudio-speech",
transport="http",
method="GET",
),
"batch_transcription": EndpointCapability(
path="/v1/audio/transcriptions",
transport="http",
method="POST",
protocol="openai.audio.transcriptions",
),
"streaming_transcription": EndpointCapability(
path=STREAM_PATH,
transport="websocket",
protocol=SPEECH_PROTOCOL,
),
"mcp": EndpointCapability(
path="/mcp",
transport="mcp-streamable-http",
method="POST",
protocol="mcp",
),
"mcp_stdio": EndpointCapability(
path="python -m backend.mcp_shim",
transport="mcp-stdio",
protocol="mcp",
),
}
native_control = False
try:
control_port = int(os.environ.get("VOICESTUDIO_SPEECH_CONTROL_PORT", ""))
except (TypeError, ValueError):
control_port = 0
if 0 < control_port <= 65535:
native_control = True
endpoints["native_dictation_control"] = EndpointCapability(
path=f"http://127.0.0.1:{control_port}/v1/capabilities",
transport="http",
method="GET",
protocol=SPEECH_PROTOCOL,
)
return SpeechCapabilities(
endpoints=endpoints,
stream_input=StreamInputCapability(
formats=[
"audio/pcm;encoding=s16le;channels=1",
"audio/webm;codecs=opus",
],
default_format="audio/webm;codecs=opus",
end_control={"type": "input_audio.end"},
),
stream_output=StreamOutputCapability(
events=["session.started", "status", "partial", "final", "error"],
final_kinds=["utterance", "summary"],
),
features=SpeechFeatureCapabilities(
native_dictation_control=native_control,
),
authentication=SpeechAuthCapabilities(),
)
@router.get(
"/.well-known/voicestudio-speech",
response_model=SpeechCapabilities,
response_model_by_alias=True,
)
@router.get(
"/v1/audio/capabilities",
response_model=SpeechCapabilities,
response_model_by_alias=True,
)
async def get_speech_capabilities() -> SpeechCapabilities:
"""Advertise batch, streaming, and agent-facing speech transports."""
return speech_capabilities()
+81 -7
View File
@@ -11,7 +11,7 @@ from core.prefs import set_ as prefs_set, delete as prefs_delete
from services import network_share
from services import tailscale as _tailscale
from api.schemas import SysinfoResponse, SystemInfoResponse, ModelStatusResponse
from api.dependencies import is_loopback, require_admin
from api.dependencies import is_loopback, require_admin, require_admin_action
from fastapi.responses import FileResponse, StreamingResponse
import torch
import shutil
@@ -203,8 +203,22 @@ def system_info():
"""
try:
_ffmpeg = find_ffmpeg()
from services import model_manager as _mm
from core import prefs as _prefs_mod
return {
"app_version": APP_VERSION,
"generate_timeout_s": _mm.GPU_JOB_TIMEOUT_S,
"cpu_generate_timeout_s": _mm.CPU_JOB_TIMEOUT_S,
# #1787 review fix: a saved prefs.json value for either key can be
# silently shadowed by an external env var (os.environ.setdefault
# in core.prefs.restore_env is a no-op when one is already
# present) — the Settings panel must say so rather than promise a
# restart will apply a value that never will.
"generate_timeout_shadowed": _prefs_mod.is_env_shadowed(
"OMNIVOICE_GENERATE_TIMEOUT_S"),
"cpu_generate_timeout_shadowed": _prefs_mod.is_env_shadowed(
"OMNIVOICE_CPU_GENERATE_TIMEOUT_S"),
"code_fingerprint": os.environ.get("OMNIVOICE_BUILD_FINGERPRINT", ""),
"data_dir": DATA_DIR,
"outputs_dir": OUTPUTS_DIR,
"crash_log_path": CRASH_LOG_PATH,
@@ -240,6 +254,11 @@ def system_info():
logger.exception("system_info failed — returning safe defaults")
return {
"app_version": APP_VERSION,
"generate_timeout_s": 300.0,
"cpu_generate_timeout_s": 600.0,
"generate_timeout_shadowed": False,
"cpu_generate_timeout_shadowed": False,
"code_fingerprint": os.environ.get("OMNIVOICE_BUILD_FINGERPRINT", ""),
"data_dir": DATA_DIR,
"outputs_dir": OUTPUTS_DIR,
"crash_log_path": str(CRASH_LOG_PATH),
@@ -544,9 +563,10 @@ async def flush_memory(unload_model: bool = False):
if unload_model:
import services.model_manager as mm
async with mm._model_lock:
if mm.model is not None:
mm.model = None
freed_model = True
# Also drops the clone-prompt side cache, which this path used to
# leave resident — an "unload" that kept the encoded reference
# tensors belonging to the model it just released (#1495).
freed_model = mm.unload_shared_model()
# Multi-pass GC to break reference cycles
gc.collect(generation=2)
@@ -555,15 +575,25 @@ async def flush_memory(unload_model: bool = False):
free_vram()
# Snapshot after flush
# Snapshot after flush. Two numbers, because one of them is a lie by
# omission: `memory_allocated` counts live tensors only, so it reads ~0
# after an unload while nvidia-smi still shows gigabytes — which is exactly
# the report we keep getting ("flush says it worked, the GPU says it
# didn't"). `memory_reserved` is what the caching allocator holds from the
# driver, and the gap between reserved and the driver's own figure is the
# CUDA context plus kernel workspaces, which no in-process call can return.
vram_after = 0.0
vram_reserved = 0.0
try:
if hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
driver = getattr(torch.mps, "driver_allocated_memory", None)
if driver:
vram_after = driver() / (1024**3)
current = getattr(torch.mps, "current_allocated_memory", None)
vram_reserved = (current() / (1024**3)) if current else vram_after
elif torch.cuda.is_available():
vram_after = torch.cuda.memory_allocated() / (1024**3)
vram_reserved = torch.cuda.memory_reserved() / (1024**3)
except Exception:
pass
@@ -574,6 +604,7 @@ async def flush_memory(unload_model: bool = False):
"unloaded_model": freed_model,
"ram_after": round(ram_after, 2),
"vram_after": round(vram_after, 2),
"vram_reserved": round(vram_reserved, 2),
}
@@ -836,6 +867,14 @@ PERSISTENT_KEYS = {
# the Rust sidecar reads OMNIVOICE_PORT at startup and the backend derives
# the LAN-share/UI ports from the others.
"OMNIVOICE_PORT", "OMNIVOICE_SHARE_PORT", "OMNIVOICE_UI_PORT",
# Per-job compute-time budgets (#1787). Both are captured at import time
# by services/model_manager.py (GPU_JOB_TIMEOUT_S / CPU_JOB_TIMEOUT_S), so
# a value saved here takes effect on the NEXT backend restart — same
# contract as OMNIVOICE_PORT above. Restored into os.environ during the
# "env_prefs" startup step (main.py), which runs before model_manager is
# first imported ("ml_imports"), so the restored value is what the module
# captures. The Settings UI must say so (RestartBadge).
"OMNIVOICE_GENERATE_TIMEOUT_S", "OMNIVOICE_CPU_GENERATE_TIMEOUT_S",
}
# Sidecar-engine install dirs (OMNIVOICE_INDEXTTS_DIR, …). The one-click
@@ -853,6 +892,16 @@ except Exception: # pragma: no cover — defensive: env panel > installer wirin
# being set so a bad value never reaches uvicorn / the share listener.
_PORT_KEYS = {"OMNIVOICE_PORT", "OMNIVOICE_SHARE_PORT", "OMNIVOICE_UI_PORT"}
# Keys whose value is a wall-clock compute-time budget in seconds (#1787).
# Validated the same way as _PORT_KEYS: reject anything that isn't a
# positive number before it reaches services/model_manager.py. Upper bound is
# generous — long enough that a legitimate multi-hour, audiobook-length CPU
# render is never blocked — but still bounded, so a fat-fingered extra digit
# (300 -> 3000000) can't turn a wedged job into one that silently occupies a
# worker for days before the guard ever fires.
_TIMEOUT_KEYS = {"OMNIVOICE_GENERATE_TIMEOUT_S", "OMNIVOICE_CPU_GENERATE_TIMEOUT_S"}
_MAX_GENERATE_TIMEOUT_S = 21600.0 # 6 hours
@router.post("/system/set-env")
async def set_env_var(body: dict):
@@ -896,6 +945,22 @@ async def set_env_var(body: dict):
status_code=400,
detail=f"Invalid port for {key}: must be between 1024 and 65535.",
)
if key in _TIMEOUT_KEYS:
try:
timeout_n = float(value)
except (TypeError, ValueError):
raise HTTPException(
status_code=400,
detail=f"Invalid timeout for {key}: '{value}' is not a number.",
)
if not (0 < timeout_n <= _MAX_GENERATE_TIMEOUT_S):
raise HTTPException(
status_code=400,
detail=(
f"Invalid timeout for {key}: must be greater than 0 "
f"and at most {_MAX_GENERATE_TIMEOUT_S:.0f} seconds."
),
)
os.environ[key] = value
logger.info("Environment variable set (length=%d)", len(value))
@@ -938,7 +1003,13 @@ async def set_env_var(body: dict):
else:
prefs_delete(prefs_key)
return {"key": key, "set": bool(value)}
# #1787 review fix: tell the caller up front when the value just saved is
# being shadowed by an external env var — set at THIS process's startup,
# before our own prefs restore ran, so it predicts the next restart too.
# A response that just said {"set": True} let the Settings panel promise
# a restart would apply a value that never will.
from core.prefs import is_env_shadowed
return {"key": key, "set": bool(value), "shadowed": is_env_shadowed(key)}
@router.post("/clean-audio")
@@ -1077,7 +1148,10 @@ async def diagnostic_bundle(network: bool = Query(False, description="Include th
# ── Self-check diagnostics ────────────────────────────────────────────────
@router.get("/system/diagnose")
@router.get(
"/system/diagnose",
dependencies=[Depends(require_admin_action)],
)
async def system_diagnose(
network: bool = Query(True, description="Include the HuggingFace hub reachability probe"),
deep: bool = Query(False, description="Also load the active engine and synthesize a short utterance (may cold-load the model — minutes on first run)"),
+113 -19
View File
@@ -10,7 +10,8 @@ as they're generated. This unlocks:
Protocol:
Client sends JSON: {"text": "...", "voice": "profile_id", ...}
Server sends binary audio chunks (PCM16 @ 24kHz mono) as generated
Server sends JSON: {"type": "done", "duration_s": 4.2, "gen_time_s": 1.1}
Server sends JSON: {"type": "done", "duration_s": 4.2,
"gen_time_s": 1.1, "ttfa_ms": 180.0, "rtf": 0.262}
Server sends JSON: {"type": "error", "detail": "..."}
The chunked delivery targets <100ms time-to-first-audio (TTFA) on warm models.
@@ -33,6 +34,30 @@ logger = logging.getLogger("omnivoice.tts_stream")
# Smaller chunks = lower latency but more WebSocket overhead.
CHUNK_SAMPLES = int(os.environ.get("OMNIVOICE_STREAM_CHUNK", "4800"))
# Module seam for deterministic latency-contract tests. Keep every timing
# sample on the same monotonic clock.
_perf_counter = time.perf_counter
async def _resolve_stream_backend(engine_id: str | None):
"""Resolve the live-stream engine without bypassing host isolation."""
from services.tts_backend import (
OmniVoiceBackend,
active_backend_id,
get_active_tts_backend,
get_backend_class,
)
if engine_id:
return get_backend_class(engine_id)()
cls = get_backend_class(active_backend_id())
if cls is OmniVoiceBackend:
from services.model_manager import get_model
return get_active_tts_backend(model=await get_model())
return get_active_tts_backend()
class StreamTTSRequest(BaseModel):
"""Client request for streaming TTS."""
@@ -62,6 +87,11 @@ async def ws_tts(websocket: WebSocket):
await websocket.accept()
logger.info("TTS streaming WebSocket connected")
# Said once per socket, not once per utterance: a conversational client
# sends many requests down one connection and a repeated notice would be
# noise. See `_announce_local_only`.
announced_local_only = False
try:
while True:
# Wait for a text request from the client
@@ -80,15 +110,48 @@ async def ws_tts(websocket: WebSocket):
})
continue
t0 = time.perf_counter()
t0 = _perf_counter()
text = data["text"]
# Remote GPU: this socket stays on this machine, and says so.
#
# /generate's port trades progressive playback for the remote
# render — the classic path was always a single wait, so spending
# it on a faster GPU is a straight win. This route is the opposite
# shape: it exists to put audio in the user's ear before the
# sentence has finished synthesizing, and sending each utterance to
# a worker would pay queue admission, a round trip and cold-load
# risk per utterance, for the one surface where latency IS the
# feature.
#
# Silence would be worse than the limitation: the header badge
# would read "gpu2" while this machine does 100% of the work, the
# same class of lie the op-aware picker exists to stop. Said once
# per socket — a conversational client sends many requests down one
# connection — and BEFORE engine resolution, so an engine that
# cannot load still tells the user where it would have run.
if not announced_local_only:
announced_local_only = True
try:
from worker import routing as worker_routing
target = worker_routing.decide(op="tts")
except Exception: # noqa: BLE001 — advisory; never break audio
target = None
if target is not None and target.remote:
from core.scrub import scrub_text as _scrub
await websocket.send_json({
"type": "routing",
"status": "local_stream",
"reason": _scrub(
f"{target.label} is your GPU target, but live "
f"streaming runs on this machine"
),
})
try:
# Resolve engine
from services.tts_backend import (
get_active_tts_backend,
get_backend_class,
)
engine_id = data.get("engine")
# #1224: leave a breadcrumb when memory is already tight before
# a heavy load. /generate has done this since the 16 GB-Mac
@@ -104,13 +167,7 @@ async def ws_tts(websocket: WebSocket):
log_if_low(f"TTS stream load ({engine_id or 'active engine'})")
except Exception:
pass
if engine_id:
cls = get_backend_class(engine_id)
backend = cls()
else:
from services.model_manager import get_model
model = await get_model()
backend = get_active_tts_backend(model=model)
backend = await _resolve_stream_backend(engine_id)
# ── Routing gate (#21 — no silent CPU fallback). WebSockets have
# no response headers, so this uses frames: an error frame +
@@ -216,6 +273,11 @@ async def ws_tts(websocket: WebSocket):
from services.model_manager import run_on_gpu_pool_guarded
def _generate(sentence_text):
# Timed INSIDE the pool worker: the guarded dispatch below
# can queue behind other jobs, and queue wait is not
# synthesis (review on #1620) — under contention it would
# inflate rtf without the engine slowing at all.
_synth_t0 = _perf_counter()
from services.audio_dsp import apply_mastering, normalize_audio
from services.watermark import mark_synthetic
wav = backend.generate(sentence_text, **kw)
@@ -237,12 +299,19 @@ async def ws_tts(websocket: WebSocket):
# watermark._iter_chunks), which is inherent to marking
# ultra-short clips, not a coverage gap.
wav = mark_synthetic(wav, sr_actual, context="tts_stream.sentence")
return wav, sr_actual
return wav, sr_actual, _perf_counter() - _synth_t0
import torch
total_samples = 0
sr = backend.sample_rate
started = False
first_audio_at: float | None = None
# Synthesis time only. The wall clock below also carries socket
# delivery and the per-chunk event-loop yields, so deriving RTF
# from it reports "how slow was the client" as if it were engine
# throughput — on a slow consumer that inflates RTF without the
# engine having changed at all.
synth_time = 0.0
for sentence in sentences:
# Bounded + pool-reset on hang so a wedged generate can't
@@ -252,11 +321,12 @@ async def ws_tts(websocket: WebSocket):
# Length-scaled budget per sentence (#1190) — the flat 300s
# default is gone from every dispatch.
from services.model_manager import generate_timeout_s
wav_tensor, sr = await run_on_gpu_pool_guarded(
wav_tensor, sr, sentence_synth_s = await run_on_gpu_pool_guarded(
functools.partial(_generate, sentence),
what="TTS generate",
timeout=generate_timeout_s(sentence),
timeout=generate_timeout_s(sentence, engine=backend),
)
synth_time += sentence_synth_s
if not started:
# Send metadata after the first generation so
@@ -283,25 +353,49 @@ async def ws_tts(websocket: WebSocket):
end = min(sent_samples + CHUNK_SAMPLES, n_samples)
chunk = pcm_bytes[sent_samples * 2: end * 2]
await websocket.send_bytes(chunk)
if first_audio_at is None:
# TTFA ends when the first audio bytes have been
# handed to the socket. The previous log used the
# whole-render duration and called it TTFA.
first_audio_at = _perf_counter()
sent_samples = end
# Yield to event loop between chunks for responsiveness
await asyncio.sleep(0)
total_samples += n_samples
gen_time = round(time.perf_counter() - t0, 3)
finished_at = _perf_counter()
wall_time_raw = max(0.0, finished_at - t0)
synth_time_raw = max(0.0, synth_time)
gen_time = round(wall_time_raw, 3)
duration = round(total_samples / sr, 3)
ttfa_ms = (
round(max(0.0, first_audio_at - t0) * 1000.0, 1)
if first_audio_at is not None
else None
)
# RTF is a render metric: synthesis seconds per audio second.
rtf = (
round(synth_time_raw / (total_samples / sr), 3)
if total_samples > 0
else None
)
await websocket.send_json({
"type": "done",
"duration_s": duration,
"gen_time_s": gen_time,
"ttfa_ms": ttfa_ms,
"rtf": rtf,
"samples": total_samples,
"sample_rate": sr,
"engine": backend.id,
})
logger.info(
"TTS stream: %.1fs audio in %.1fs (TTFA=%.0fms)",
duration, gen_time, gen_time * 1000,
"TTS stream: %.1fs audio in %.1fs (TTFA=%s, RTF=%s)",
duration,
gen_time,
f"{ttfa_ms:.0f}ms" if ttfa_ms is not None else "n/a",
f"{rtf:.3f}" if rtf is not None else "n/a",
)
except Exception as e:
+378
View File
@@ -0,0 +1,378 @@
"""Speech-to-speech voice changer — Studio's Convert method (POST /convert).
The user drops (or records) a source clip, picks an existing voice profile,
and gets the same words back in that profile's voice: the active ASR backend
transcribes the clip (no word timestamps the text is all we need), the
active TTS engine re-synthesizes it conditioned on the profile's reference
audio, and by default the take is pitch-preservingly time-stretched
(ffmpeg atempo, clamped to one well-behaved 0.52.0 stage) so it lands near
the source clip's duration.
Deliberately reuses the /generate choke points instead of re-deriving them:
* profile row conditioning via ``generation._resolve_profile_conditioning``
(lock wins, ``kind`` authoritative, #533 language fill),
* engine resolution via ``services.tts_backend.resolve_generation_backend``
(never a silent OmniVoice fallback; ``require_cloning=True`` refuses
clone-less engines with the actionable switch-engine message),
* synthesis via ``generation._run_backend_inference`` on the guarded GPU
pool (#730 bound + reset; busy/timeout → retryable 503),
* provenance + persistence via ``services.watermark.mark_synthetic_async``
and ``generation._finalize_generation`` (watermark WAV in OUTPUTS_DIR
history row retention prune), marked AFTER the stretch so the take users
keep carries exactly one whole-take mark.
Local-first: no network calls; ASR-model-less installs get the same typed
409 download CTA as /transcribe; a backend mid-shutdown surfaces the global
503 ``[shutting_down]`` (ModelLoadInterruptedByShutdown main.py handler).
Reachability matches /generate: loopback bind by default, with the shared
network-share PIN / API-key middleware gating any non-loopback exposure.
"""
from __future__ import annotations
import asyncio
import functools
import logging
import os
import tempfile
import time
from fastapi import APIRouter, File, Form, HTTPException, UploadFile
router = APIRouter()
logger = logging.getLogger("omnivoice.convert")
#: ffmpeg's atempo filter is well-behaved in [0.5, 2.0] per stage. Convert
#: clamps to ONE stage by design: needing more than 2× either way means the
#: synthesized speech differs so much from the source that "matching" it
#: would produce chipmunk/slow-motion artifacts worse than the mismatch.
ATEMPO_MIN = 0.5
ATEMPO_MAX = 2.0
#: Within this relative tolerance the durations already match — stretching
#: would resample the whole take for an inaudible gain.
_MATCH_TOLERANCE = 0.02
#: Convert clips are short conversational inputs, not long-form media. Stream
#: them to disk in bounded chunks so a network-share client cannot make the
#: backend materialize an arbitrarily large multipart upload in memory.
_MAX_SOURCE_AUDIO_BYTES = 64 * 1024 * 1024
_UPLOAD_CHUNK_BYTES = 1024 * 1024
async def _copy_source_upload(audio: UploadFile, destination) -> int:
"""Stream ``audio`` into ``destination`` with the Convert upload cap."""
total = 0
while True:
chunk = await audio.read(_UPLOAD_CHUNK_BYTES)
if not chunk:
return total
total += len(chunk)
if total > _MAX_SOURCE_AUDIO_BYTES:
raise HTTPException(
status_code=413,
detail="Source audio is too large (maximum 64 MB).",
)
destination.write(chunk)
def _clamped_tempo_ratio(tts_duration_s: float, source_duration_s: float) -> "float | None":
"""The atempo ratio that fits the take into the source duration, or None.
ratio > 1 speeds the take up (it came out longer than the source),
ratio < 1 slows it down. Clamped to a single atempo stage's [0.5, 2.0];
None when either duration is unusable or they already match.
"""
if not source_duration_s or source_duration_s <= 0:
return None
if not tts_duration_s or tts_duration_s <= 0:
return None
ratio = tts_duration_s / source_duration_s
if abs(ratio - 1.0) <= _MATCH_TOLERANCE:
return None
return min(ATEMPO_MAX, max(ATEMPO_MIN, ratio))
async def _match_source_duration(audio_tensor, sample_rate: int, source_duration_s: float):
"""Best-effort pitch-preserving stretch of the take toward the source
clip's duration. Returns the input unchanged when no stretch is needed
or ffmpeg fails a duration mismatch is better than a failed convert."""
n_samples = int(audio_tensor.shape[-1])
ratio = _clamped_tempo_ratio(n_samples / sample_rate, source_duration_s)
if ratio is None:
return audio_tensor
target_samples = max(1, int(round(n_samples / ratio)))
from services.ffmpeg_utils import _pitch_preserving_stretch
try:
return await _pitch_preserving_stretch(audio_tensor, target_samples, sample_rate)
except Exception as e: # noqa: BLE001 — stretch is opt-in polish, never fatal
logger.warning("duration match skipped — atempo stretch failed: %s", e)
return audio_tensor
async def _transcribe_source(tmp_path: str, *, source_lease=None) -> dict:
"""Active-ASR transcription of the uploaded clip (no word timestamps).
Mirrors POST /transcribe: typed 409 + download CTA before any backend
is constructed (never a silent multi-GB auto-download), the guarded GPU
pool dispatch (#730), 504 on timeout, and the same 409 when the loader
degrades onto an engine with no weights on disk (#1185).
"""
from services.asr_backend import (
ASRModelMissingError,
ASRTimeoutError,
asr_model_missing_detail,
asr_model_missing_error,
run_transcribe_guarded,
)
missing = await asyncio.to_thread(asr_model_missing_error, purpose="transcribe")
if missing is not None:
raise HTTPException(
status_code=409,
detail={**missing, "message": asr_model_missing_detail(missing)},
)
def _run():
# `load_*`, not `get_*`: the loader runs ensure_loaded() and degrades
# past an engine whose deep import chain is broken (#1185).
from services.asr_backend import load_active_asr_backend
backend = load_active_asr_backend()
return backend.transcribe(tmp_path, word_timestamps=False)
from services.model_manager import _gpu_pool
release = source_lease.acquire() if source_lease is not None else None
abandoned = False
try:
return await run_transcribe_guarded(
_gpu_pool,
_run,
what="Voice convert",
on_abandon=release,
)
except asyncio.CancelledError:
# The guard now owns the lease token until the native worker drains.
abandoned = True
raise
except ASRTimeoutError as e:
abandoned = True
logger.warning("Convert transcription timed out: %s", e)
raise HTTPException(status_code=504, detail=str(e))
except ASRModelMissingError as e:
raise HTTPException(
status_code=409,
detail={**e.payload, "message": asr_model_missing_detail(e.payload)},
)
finally:
if release is not None and not abandoned:
release()
@router.post("/convert")
async def convert_speech(
audio: UploadFile = File(...),
profile_id: str = Form(...),
match_duration: bool = Form(True),
):
"""Convert a spoken clip into an existing voice profile's voice.
Multipart form: ``audio`` (the source clip), ``profile_id`` (an existing
voice profile), optional ``match_duration`` (default on atempo the take
toward the source clip's length, clamped to 0.52.0×).
Returns JSON ``{audio_url, text, duration_s, id}`` the take is saved to
OUTPUTS_DIR and served from the ``/audio`` mount like every other take.
"""
from core.db import db_conn
from api.routers.generation import _resolve_profile_conditioning, _TempReferenceLease
# ── Profile first: strict 404, unlike /generate's silent skip — Convert
# has no meaning without a target voice.
with db_conn() as conn:
row = conn.execute(
"SELECT * FROM voice_profiles WHERE id=?", (profile_id,)
).fetchone()
if not row:
raise HTTPException(
status_code=404,
detail="That voice profile doesn't exist. It may have been deleted from another tab.",
)
cond = _resolve_profile_conditioning(row)
# ── Save the upload before loading an engine. Every ASR backend (and
# ffprobe) needs a file path; the bounded streaming copy rejects oversized
# network-share requests without materializing them in process memory or
# starting heavyweight model work.
ext = os.path.splitext(audio.filename or "audio.wav")[1] or ".wav"
tmp = tempfile.NamedTemporaryFile(delete=False, suffix=ext)
source_lease = None
try:
try:
await _copy_source_upload(audio, tmp)
finally:
tmp.close()
source_lease = _TempReferenceLease(tmp.name)
# ── Engine gate before ASR/TTS work: the shared resolver refuses a
# clone-less engine with the actionable switch-engine message (→ 400),
# and a backend mid-shutdown raises ModelLoadInterruptedByShutdown out
# of the model load → the global 503 [shutting_down] handler.
from services.tts_backend import resolve_generation_backend
try:
backend = await resolve_generation_backend(
require_cloning=True, cloning_purpose="voice conversion",
)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
result = await _transcribe_source(tmp.name, source_lease=source_lease)
segments = result.get("segments", [])
text = result.get("text", "")
if not text and segments:
text = " ".join(s.get("text", "") for s in segments).strip()
# Same final-text hygiene as /transcribe: strip Whisper hallucination
# loops, then deterministic polish (leading capital + terminal
# punctuation) so the TTS input reads as typed text.
from services.refinement import collapse_repetitive_artifacts
from services.text_polish import polish_text
text = polish_text(collapse_repetitive_artifacts(text))
if not text or not text.strip():
raise HTTPException(
status_code=422,
detail=(
"No speech was recognized in the source clip, so there is "
"nothing to convert. Record or drop a clip with clear, "
"audible speech and try again."
),
)
# #308/#1032 parity with /generate: a clone profile saved without a
# transcript conditions better when its reference clip is transcribed,
# and that transcript is cached onto the row so it happens ONCE, not
# per convert. Best-effort exactly like /generate — a timeout/failure
# degrades to ref_text=None and the engine's own fallback. The ASR
# model is already warm here (the source transcribe above just used it).
if cond["ref_audio_path"] and not cond["ref_text"]:
from api.routers.generation import (
_generate_timeout_s,
_persist_profile_ref_text,
)
from services.asr_backend import transcribe_reference
from services.model_manager import run_on_gpu_pool_guarded
try:
cond["ref_text"] = await run_on_gpu_pool_guarded(
functools.partial(transcribe_reference, cond["ref_audio_path"]),
what="Reference transcribe",
timeout=_generate_timeout_s(""),
)
except TimeoutError as e:
logger.warning(
"reference transcribe hung (%s); using engine ASR fallback", e,
)
cond["ref_text"] = None
if cond["ref_text"] and cond["persist_ref_text"]:
_persist_profile_ref_text(profile_id, cond["ref_text"])
# Source duration for the optional match: the container's own length
# (ffprobe), falling back to the last ASR segment end. Best-effort —
# None just skips the stretch.
source_duration_s = None
if match_duration:
from services.ffmpeg_utils import probe_duration
source_duration_s = await probe_duration(
tmp.name, allowed_root=os.path.dirname(tmp.name),
)
if not source_duration_s and segments:
source_duration_s = max((s.get("end", 0) or 0) for s in segments) or None
# ── Same text choke point as /generate: engine-agnostic normalization
# (numbers→words, junk strip) on the fully resolved language.
from services.text_normalization import normalize_for_tts
language = cond["language"]
text = normalize_for_tts(text, language)
used_seed = cond["seed"]
if used_seed is None:
import random
used_seed = random.randint(0, 2**31 - 1)
from api.routers.generation import (
_finalize_generation,
_generate_timeout_s,
_run_backend_inference,
)
from services.model_manager import (
GpuJobTimeoutError,
GpuPoolBusyError,
run_on_gpu_pool_guarded,
)
start_time = time.time()
_render = functools.partial(
_run_backend_inference,
backend, text, language, cond["ref_audio_path"], cond["ref_text"],
cond["instruct"],
None, # duration — the model picks; match_duration owns pacing
16, 2.0, # num_step / guidance_scale (the /generate defaults)
1.0, # speed
True, True, # denoise / postprocess_output
used_seed,
)
try:
audio_tensor = await run_on_gpu_pool_guarded(
_render,
what="Voice convert",
timeout=_generate_timeout_s(text),
min_vram_gb=getattr(type(backend), "min_vram_gb", 0.0),
)
except GpuPoolBusyError as e:
raise HTTPException(
status_code=503, detail=str(e),
headers={"Retry-After": str(e.retry_after),
"X-OmniVoice-Retryable": "true"},
) from e
except GpuJobTimeoutError as e:
raise HTTPException(
status_code=503, detail=str(e),
headers={"Retry-After": "30", "X-OmniVoice-Retryable": "true"},
) from e
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e)) from e
sample_rate = backend.sample_rate
if match_duration and source_duration_s:
audio_tensor = await _match_source_duration(
audio_tensor, sample_rate, source_duration_s,
)
# Provenance mark AFTER the stretch (one whole-take mark on the audio
# the user actually keeps), then the shared finalize tail — WAV in
# OUTPUTS_DIR, self-healing history row, retention prune, event emit.
from services.watermark import mark_synthetic_async
audio_tensor = await mark_synthetic_async(
audio_tensor, sample_rate, context="convert.finalize",
)
_, meta = await _finalize_generation(
audio_tensor, sample_rate, text=text, history_mode="convert",
ref_audio_path=cond["ref_audio_path"], language=language,
instruct=cond["instruct"], resolved_profile_id=profile_id,
used_seed=used_seed, start_time=start_time,
already_marked=True,
)
return {
"id": meta["id"],
"audio_url": f"/audio/{meta['filename']}",
"text": text,
"duration_s": meta["duration"],
"gen_time_s": meta["gen_time"],
}
finally:
if source_lease is not None:
source_lease.finish_request()
else:
try:
os.unlink(tmp.name)
except OSError:
pass
+819
View File
@@ -0,0 +1,819 @@
"""Remote worker management API.
Deliberately small. The council's warning about the original design was that
seven strategies times three execution modes times priorities times weights
times per-model concurrency is a configuration surface nobody can test and
every knob is a compatibility promise forever. So this exposes what a user
actually needs to run their other GPU: see workers, add one, name it, prefer
one, pause one, remove one.
Two things here are not conveniences and must not be softened:
* **Consent is explicit and per worker.** Audio, reference voices, and text
leave the machine for a worker, so each one is approved individually. There
is no global "trust all workers".
* **A token is shown exactly once.** Only its hash is stored, so it cannot be
re-displayed which is the point.
One endpoint here is not part of that surface: `POST /workers/tasks` submits a
single task and waits for it, and exists only because the scheduler otherwise
has no caller at all outside the tests. It is marked dev-only everywhere it
appears and is replaced by the GPU gateway.
"""
from __future__ import annotations
import asyncio
import contextlib
import logging
from fastapi import APIRouter, Depends, HTTPException, Request
from fastapi.responses import JSONResponse
from pydantic import BaseModel, Field
from api.dependencies import require_admin
from worker import registry, routing, service
from worker.async_utils import drain_task, to_thread_and_defer_cancellation
logger = logging.getLogger("omnivoice.worker")
# How often an awaiting request checks whether its caller is still there.
# Starlette does not cancel a handler when the client hangs up, so polling is
# the only way the "cancel what nobody is waiting for" rule can fire before
# the task's own deadline does.
_DISCONNECT_POLL_SECONDS = 1.0
# Management is admin-gated: these endpoints mint join tokens and revoke
# machines, so Docker writes require the API key while desktop stays loopback.
router = APIRouter(prefix="/workers", tags=["workers"], dependencies=[Depends(require_admin)])
class EnableRequest(BaseModel):
enabled: bool
class EnrollRequest(BaseModel):
label: str = Field("", max_length=120)
endpoint: str = Field("", max_length=256)
ttl_seconds: int = Field(900, ge=60, le=24 * 3600)
class JoinRequest(BaseModel):
"""A join code, as pasted (or scanned) from the control plane."""
token: str = Field(..., max_length=4096)
class TargetRequest(BaseModel):
"""`local`, or the id of an enrolled worker."""
target: str = Field(..., max_length=64)
class WorkerUpdate(BaseModel):
name: str | None = Field(None, max_length=120)
enabled: bool | None = None
priority: int | None = Field(None, ge=0, le=100)
class SubmitTaskRequest(BaseModel):
"""One unit of work for a remote worker. **Dev only** — see `submit_task`."""
engine: str = Field(..., max_length=64)
operation: str = Field("tts", max_length=32)
model_id: str = Field("", max_length=128)
params: dict = Field(default_factory=dict)
# Mandatory, and deliberately without a default: the sweeper fails a task
# on its deadline only while it is QUEUED, so one submitted without a
# deadline while no worker is online waits forever with nothing left in
# the system that would ever time it out.
deadline_seconds: float = Field(..., gt=0, le=6 * 3600)
idempotency_key: str | None = Field(None, max_length=128)
class _ClientGone(Exception):
"""The caller hung up while its task was still running."""
class _WaitExpired(Exception):
"""The task did not reach a terminal state inside its deadline."""
@router.get("")
def list_workers() -> dict:
"""Everything the workers panel renders, in one call."""
return service.control_plane.snapshot()
@router.get("/target")
def get_target(op: str = "") -> dict:
"""What the GPU picker shows: the choice, the resolved answer, the options.
`active` is the same answer the generation path uses, so the badge cannot
claim work goes somewhere the router will not send it. Pass `op` for the
surface being rendered omitting it answers for the target as a whole,
which is what the picker's own menu asks.
"""
return routing.status(op=op.strip() or None)
@router.post("/target")
def set_target(request: TargetRequest) -> dict:
"""Choose where work runs. Exactly one target is active at a time."""
chosen = request.target.strip() or routing.LOCAL
if chosen != routing.LOCAL:
worker = registry.get(chosen)
if worker is None or worker.revoked:
raise HTTPException(status_code=404, detail="No such worker.")
routing.set_target_id(chosen)
return routing.status()
@router.post("/enabled")
async def set_enabled(request: EnableRequest) -> dict:
"""Turn the feature on or off.
Off means off: the control plane stops, the listening socket closes, and
the app is exactly what it was before the toggle existed.
"""
service.set_remote_workers_enabled(request.enabled)
if request.enabled:
try:
await service.control_plane.start()
except Exception as exc:
service.control_plane.startup_error = str(exc)
raise HTTPException(status_code=409, detail=str(exc)) from exc
else:
await service.control_plane.stop()
return service.control_plane.snapshot()
@router.get("/agent")
def agent_status() -> dict:
"""The other side of the same feature: is THIS machine lending its GPU?
Separate from `GET /workers`, which answers for the control plane. A
machine can legitimately be both a desktop that borrows a laptop's GPU
and lends its own to a colleague so neither status can stand in for the
other.
"""
from worker import agent as worker_agent # noqa: PLC0415
return worker_agent.agent.status()
@router.get("/agent/readiness", include_in_schema=False, response_model=None)
def agent_readiness() -> JSONResponse:
"""Container readiness: 200 only after this process registered as a worker."""
from worker import agent as worker_agent # noqa: PLC0415
readiness = worker_agent.agent.readiness()
return JSONResponse(
status_code=200 if readiness["ready"] else 503,
content=readiness,
headers={} if readiness["ready"] else {"Retry-After": "2"},
)
def _refuse_when_env_pinned(worker_agent) -> None:
"""OMNIVOICE_WORKER_MODE wins over the setting everywhere else.
`worker_mode_enabled()` reads the variable first and `status()` reports the
machine as env-pinned, so a route that changed worker mode anyway would
contradict both: it writes a setting nothing consults, and the next restart
undoes whatever the user just saw happen.
"""
if worker_agent.agent.status()["env_pinned"]:
raise HTTPException(
status_code=409,
detail=(
"OMNIVOICE_WORKER_MODE controls this machine's worker mode. Unset it "
"and restart VoiceStudio to manage it from here."
),
)
async def _finish_cleanup(awaitable):
"""Run rollback to completion even if its HTTP task was cancelled."""
task = asyncio.create_task(awaitable)
await drain_task(task)
return task.result()
async def _set_worker_mode(worker_agent, enabled: bool) -> None:
_result, cancelled = await to_thread_and_defer_cancellation(
worker_agent.set_worker_mode_enabled, enabled
)
if cancelled:
raise asyncio.CancelledError
async def _restore_agent_transaction(
worker_agent, previous: dict, *, was_running: bool
) -> None:
"""Restore durable enrollment/settings and the exact prior live state."""
try:
await _finish_cleanup(worker_agent.agent.stop())
await _finish_cleanup(worker_agent.restore_enrollment(previous))
if was_running and not worker_agent.agent.running:
await _finish_cleanup(worker_agent.agent.start())
elif not was_running and worker_agent.agent.running:
await _finish_cleanup(worker_agent.agent.stop())
except worker_agent.EnrollmentRollbackError:
raise
except BaseException as exc:
message = (
"The previous worker state could not be restored safely. "
"Worker mode remains stopped; fix its enrollment/settings storage, then retry."
)
with contextlib.suppress(BaseException):
await _finish_cleanup(worker_agent.agent.stop())
worker_agent.agent.last_error = message
raise worker_agent.EnrollmentRollbackError(message) from exc
def _raise_agent_transaction_failure(
worker_agent, operation: BaseException, rollback: BaseException | None
) -> None:
if isinstance(operation, asyncio.CancelledError):
if rollback is not None:
logger.error(
"Worker rollback failed during request cancellation",
exc_info=(type(rollback), rollback, rollback.__traceback__),
)
raise operation
if rollback is not None:
raise HTTPException(status_code=409, detail=str(rollback)) from rollback
if isinstance(operation, Exception):
worker_agent.agent.last_error = str(operation)
raise HTTPException(status_code=409, detail=str(operation)) from operation
raise operation
@router.post("/agent/join")
async def join_control_plane(request: JoinRequest) -> dict:
"""Redeem a join code and start working for that control plane.
This is the endpoint that makes the feature reachable. Joining used to mean
setting OMNIVOICE_WORKER_MODE and OMNIVOICE_WORKER_TOKEN in the environment
and relaunching the app a step most users will never take, on the machine
that is usually the least convenient to configure by hand.
The code is single-use and short-lived, so a failure here is nearly always
"expired" or "wrong address"; it is returned verbatim rather than as a bare
409, because the user's next action depends on which one it was.
"""
from worker import agent as worker_agent # noqa: PLC0415
token = request.token.strip()
if not token:
raise HTTPException(status_code=422, detail="Paste the join code first.")
# Same rule as the toggle below: joining ENABLES worker mode, so under
# OMNIVOICE_WORKER_MODE it would write a setting the rest of the app
# ignores — and with the variable set to 0, hand the user a machine that
# says it joined and never lends anything (CodeRabbit).
_refuse_when_env_pinned(worker_agent)
async with worker_agent.agent.lifecycle:
try:
previous, cancelled = await to_thread_and_defer_cancellation(
worker_agent.snapshot_enrollment
)
except worker_agent.EnrollmentStateError as exc:
worker_agent.agent.last_error = str(exc)
raise HTTPException(status_code=409, detail=str(exc)) from exc
if cancelled:
raise asyncio.CancelledError
was_running = worker_agent.agent.running
# A rejoin stops a working agent before the replacement is accepted.
# Stop, acceptance and the durable setting are one transaction: every
# failure, including cancellation, restores both trust and live state.
try:
await worker_agent.agent.stop()
await worker_agent.agent.start(token_text=token)
# Success is the control plane ACCEPTING this worker, not the
# connection being scheduled — see wait_until_registered.
await worker_agent.agent.wait_until_registered()
await _set_worker_mode(worker_agent, True)
except BaseException as exc:
rollback_exc = None
try:
await _restore_agent_transaction(
worker_agent, previous, was_running=was_running
)
except BaseException as rollback_error:
rollback_exc = rollback_error
_raise_agent_transaction_failure(worker_agent, exc, rollback_exc)
worker_agent.agent.last_error = ""
return worker_agent.agent.status()
@router.post("/agent/enabled")
async def set_agent_enabled(request: EnableRequest) -> dict:
"""Start or stop lending this machine, without forgetting the enrollment.
Off stops the agent and clears the setting, so nothing dials out; the
pinned certificate stays, which is what lets "on" resume without asking for
another code.
"""
from worker import agent as worker_agent # noqa: PLC0415
_refuse_when_env_pinned(worker_agent)
async with worker_agent.agent.lifecycle:
try:
previous, cancelled = await to_thread_and_defer_cancellation(
worker_agent.snapshot_enrollment
)
except worker_agent.EnrollmentStateError as exc:
worker_agent.agent.last_error = str(exc)
raise HTTPException(status_code=409, detail=str(exc)) from exc
if cancelled:
raise asyncio.CancelledError
was_running = worker_agent.agent.running
try:
if request.enabled:
await worker_agent.agent.start()
await worker_agent.agent.wait_until_registered()
await _set_worker_mode(worker_agent, True)
else:
await worker_agent.agent.stop()
await _set_worker_mode(worker_agent, False)
except BaseException as exc:
rollback_exc = None
try:
await _restore_agent_transaction(
worker_agent, previous, was_running=was_running
)
except BaseException as rollback_error:
rollback_exc = rollback_error
_raise_agent_transaction_failure(worker_agent, exc, rollback_exc)
worker_agent.agent.last_error = ""
return worker_agent.agent.status()
@router.post("/enrollments")
def create_enrollment(request: EnrollRequest) -> dict:
"""Mint a single-use join token.
The plaintext is returned once and never stored the response is the only
time it exists outside the worker that redeems it.
"""
if not service.control_plane.running:
raise HTTPException(
status_code=409,
detail="Remote workers are turned off. Enable them in Settings → System → Remote workers first.",
)
try:
token = service.control_plane.create_enrollment(
endpoint=request.endpoint,
label=request.label,
ttl_seconds=request.ttl_seconds,
)
except service.EndpointCertificateError as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
return {
"token": token.encode(),
"endpoint": token.endpoint,
"fingerprint": token.cert_fingerprint,
"expires_at": token.expires_at,
"shown_once": True,
}
def _persist_worker_update(
worker_id: str, request: WorkerUpdate
):
"""Write policy on a worker thread; live publication stays loop-owned."""
return registry.update_policy(
worker_id,
name=request.name,
enabled=request.enabled,
priority=request.priority,
)
@router.patch("/{worker_id}")
async def update_worker(worker_id: str, request: WorkerUpdate) -> dict:
pool = service.control_plane.pool if service.control_plane.running else None
live = None
was_pending = False
if pool is not None:
# Quiesce dispatch before releasing authority for the SQLite write.
# The publication after the await restores the exact prior state, so a
# concurrent registration handoff remains quiesced for its own reason.
with registry.authority_guard():
live = pool.get(worker_id)
if live is not None:
was_pending = live.registration_pending
live.registration_pending = True
updated = None
cancelled = False
try:
updated, cancelled = await to_thread_and_defer_cancellation(
_persist_worker_update, worker_id, request
)
finally:
if pool is not None:
with registry.authority_guard():
if updated is not None:
# Pool state, including the cached record the scheduler
# reads, belongs to the app's event loop.
pool.refresh_record(updated)
current = pool.get(worker_id)
if current is live:
current.registration_pending = was_pending
if updated is None:
if cancelled:
raise asyncio.CancelledError
raise HTTPException(status_code=404, detail="No such worker.")
if cancelled:
raise asyncio.CancelledError
return updated.to_dict()
@router.post("/{worker_id}/consent")
def grant_consent(worker_id: str) -> dict:
"""Record the user's explicit yes to sending their audio to this machine."""
if registry.get(worker_id) is None:
raise HTTPException(status_code=404, detail="No such worker.")
registry.grant_consent(worker_id)
worker = registry.get(worker_id)
return worker.to_dict() if worker else {}
@router.post("/{worker_id}/resume")
async def clear_breaker(worker_id: str) -> dict:
"""Clear a paused worker's circuit breakers.
The user fixed the machine and knows it a breaker with no manual clear is
the quarantine trap the reputation system had.
"""
if not service.control_plane.running:
raise HTTPException(status_code=409, detail="Remote workers are turned off.")
breakers = service.control_plane.pool.breakers
for breaker in breakers.open_breakers(worker_id):
breaker.force_close()
return {"ok": True}
@router.delete("/{worker_id}")
async def revoke_worker(worker_id: str) -> dict:
"""Remove a worker — which means revoke its key, not hide the row.
Its in-flight work is released so it can be retried elsewhere rather than
waiting out a lease on a machine that will never answer again.
"""
pool = service.control_plane.pool if service.control_plane.running else None
live = None
was_pending = False
if pool is not None:
with registry.authority_guard():
live = pool.get(worker_id)
if live is not None:
was_pending = live.registration_pending
live.registration_pending = True
try:
revoked, cancelled = await to_thread_and_defer_cancellation(
registry.revoke, worker_id
)
except BaseException:
if pool is not None:
with registry.authority_guard():
current = pool.get(worker_id)
if current is live:
current.registration_pending = was_pending
raise
if not revoked:
if pool is not None:
with registry.authority_guard():
current = pool.get(worker_id)
if current is live:
current.registration_pending = was_pending
if cancelled:
raise asyncio.CancelledError
raise HTTPException(status_code=404, detail="No such worker.")
# The tombstone committed before any egress/session mutation. Everything
# below is loop-owned and published under the same scheduler authority read
# used by next_assignment(), so no task can bind in the handoff window.
with registry.authority_guard():
if service.control_plane.running:
if service.control_plane.servicer is not None:
service.control_plane.servicer.revoke_worker_sessions(worker_id)
service.control_plane.scheduler.on_disconnected(worker_id)
service.control_plane.pool.breakers.forget_worker(worker_id)
if cancelled:
raise asyncio.CancelledError
return {"ok": True, "revoked": worker_id}
@router.get("/tasks")
def list_tasks(limit: int = 50) -> dict:
"""Recent remote tasks, for the queue view."""
if not service.control_plane.running:
return {"tasks": [], "queue_depth": 0}
from worker import task_store # noqa: PLC0415
return {
"queue_depth": service.control_plane.scheduler.queue_depth,
"tasks": [t.to_dict() for t in task_store.list_tasks(limit=min(200, max(1, limit)))],
}
@router.post("/tasks")
async def submit_task(request: Request, body: SubmitTaskRequest) -> dict:
"""Run one task on a remote worker and wait for it. **DEV ONLY.**
This is the producer the remote pipeline never had: until it existed the
scheduler had no caller outside the test suite, so picking a remote GPU
changed the badge and nothing else every job still ran locally. It is
the smallest thing that makes remote execution observable end to end, not
the shipping surface: the GPU gateway takes over routing real generation
and this endpoint goes with it.
Loopback-only and behind the same opt-in as the rest of the feature, so a
user who never enabled remote workers cannot reach it at all.
"""
from worker.lifecycle import TaskState # noqa: PLC0415
from worker.scheduler import QueueFull, SchedulerStopped # noqa: PLC0415
if not service.remote_workers_enabled() or not service.control_plane.running:
raise HTTPException(status_code=409, detail="Remote workers are turned off.")
if not routing.supports_operation(body.operation):
raise HTTPException(
status_code=400,
detail=f"'{body.operation}' does not run on a remote worker yet.",
)
scheduler = service.control_plane.scheduler
try:
submit = getattr(scheduler, "submit_async", None)
submit = submit if callable(submit) else scheduler.submit
submitted = submit(
operation=body.operation,
engine=body.engine,
model_id=body.model_id,
params=body.params,
idempotency_key=body.idempotency_key or None,
deadline_seconds=body.deadline_seconds,
pinned_worker_id=routing.decide().worker_id or None,
)
task = await submitted if asyncio.iscoroutine(submitted) else submitted
except QueueFull as exc:
raise HTTPException(status_code=429, detail=str(exc)) from exc
settled = None
reason = "the request was interrupted"
try:
settled = await _await_terminal(
request, scheduler, task.task_id, timeout=body.deadline_seconds
)
except _ClientGone:
reason = "the client disconnected"
raise HTTPException(status_code=499, detail="The client stopped waiting.") from None
except _WaitExpired:
reason = "the task passed its deadline"
raise HTTPException(
status_code=504,
detail=f"The task did not finish within {body.deadline_seconds:g}s.",
) from None
except SchedulerStopped as exc:
# Deliberately no cancel: the worker was never told to stop and may
# still be rendering, so claiming the task is cancelled would be a
# statement about someone else's GPU that we cannot make.
reason = None
raise HTTPException(status_code=503, detail=str(exc)) from None
finally:
# Nothing else will stop it: a worker holds its slot — often its only
# one — until the control plane says otherwise, and the sweeper only
# enforces deadlines on tasks that are still queued. Swallowed because
# a failure here would replace the caller's real error with a 500.
if settled is None and reason is not None:
try:
await service.control_plane.cancel(task.task_id, reason=reason)
except Exception:
logger.exception("Could not cancel abandoned remote task %s", task.task_id)
payload = settled.to_dict()
if settled.state is TaskState.COMPLETED:
return payload
# A failure that answered 200 would be indistinguishable from success to
# anything that does not read `state` — which is the whole point of this
# endpoint existing before the gateway does.
raise HTTPException(
status_code=409 if settled.state is TaskState.CANCELLED else 502, detail=payload
)
async def _await_terminal(request: Request, scheduler, task_id: str, *, timeout: float):
"""Wait for a terminal task, giving up if the caller does first."""
waiter = asyncio.ensure_future(scheduler.wait(task_id, timeout=timeout))
while True:
done, _pending = await asyncio.wait({waiter}, timeout=_DISCONNECT_POLL_SECONDS)
if done:
try:
settled = waiter.result()
except (asyncio.TimeoutError, TimeoutError) as exc:
raise _WaitExpired() from exc
if settled is None or not settled.state.terminal:
raise _WaitExpired()
return settled
if await request.is_disconnected():
waiter.cancel()
raise _ClientGone()
@router.post("/tasks/{task_id}/cancel")
async def cancel_task(task_id: str) -> dict:
if not service.control_plane.running:
raise HTTPException(status_code=409, detail="Remote workers are turned off.")
cancelled = await service.control_plane.cancel(task_id, reason="cancelled by user")
if not cancelled:
raise HTTPException(status_code=404, detail="No such active task.")
return {"ok": True}
# ── Inbound mode ───────────────────────────────────────────────────────────
#
# The other direction: this machine accepts connections from panels, or dials
# out to nodes that do. Outbound enrollment above is unchanged and remains the
# default — see docs/adr/inbound-node-mode.md for why this exists alongside it
# rather than replacing it.
class InboundEnableRequest(BaseModel):
enabled: bool
# Widening the bind is a separate decision from turning the feature on,
# so it is a separate field with a safe default rather than a flag that
# rides along with `enabled`.
bind: str = ""
port: int = 0
class IssueKeyRequest(BaseModel):
label: str = Field(default="", max_length=64)
class ConnectRequest(BaseModel):
connection_string: str = Field(min_length=1, max_length=512)
@router.get("/inbound")
def inbound_status() -> dict:
from worker.inbound import service as inbound_service # noqa: PLC0415
return {
**inbound_service.node.snapshot(),
"connections": inbound_service.outbound.snapshot(),
}
@router.post("/inbound/enabled")
async def set_inbound_enabled(request: InboundEnableRequest) -> dict:
from worker.inbound import service as inbound_service # noqa: PLC0415
if inbound_service.enabled_override() is not None:
raise HTTPException(
status_code=409,
detail=(
"Accept connections is controlled by OMNIVOICE_INBOUND_NODE on this "
"machine. Change that environment setting and restart VoiceStudio."
),
)
requested_bind = (
inbound_service.normalise_bind_host(request.bind)
if request.bind
else inbound_service.bind_host()
)
requested_port = request.port or inbound_service.bind_port()
if (
request.enabled
and inbound_service.node.running
and (
requested_bind != inbound_service.bind_host()
or requested_port != inbound_service.node.port
)
):
# start() is intentionally idempotent while a listener owns its
# socket. Persisting a new endpoint here would make the UI report a
# narrower/different bind while the original socket stayed live.
raise HTTPException(
status_code=409,
detail=(
"Turn off Accept connections before changing its bind address "
"or port."
),
)
if request.bind:
inbound_service.set_bind_host(requested_bind)
if request.port:
inbound_service.set_bind_port(request.port)
inbound_service.set_enabled(request.enabled)
if inbound_service.enabled():
await inbound_service.node.start()
if inbound_service.node.startup_error:
logger.error("Inbound worker listener failed to start; details withheld.")
raise HTTPException(
status_code=409,
detail=(
"The inbound worker listener could not start; "
"check the backend log for details."
),
)
else:
await inbound_service.node.stop()
return inbound_service.node.snapshot()
@router.post("/inbound/keys")
def issue_inbound_key(request: IssueKeyRequest) -> dict:
"""Mint one panel's key and return the string it pastes.
The secret is in this response and nowhere else afterwards only its hash
is stored, so it cannot be shown again, only replaced.
"""
from worker.inbound import service as inbound_service # noqa: PLC0415
from worker.inbound.keys import KeyLimitExceeded # noqa: PLC0415
if not inbound_service.node.running:
raise HTTPException(
status_code=409,
detail=(
"This machine is not accepting connections yet. Turn on "
"Settings → System → Remote workers → Accept connections first."
),
)
try:
issued = inbound_service.node.keys.issue(request.label)
except KeyLimitExceeded as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
return {
"key_id": issued.key.key_id,
"label": issued.key.label,
"connection_string": inbound_service.node.connection_string(issued.secret),
"exposed": inbound_service.is_exposed(),
"shown_once": True,
}
@router.delete("/inbound/keys/{key_id}")
async def revoke_inbound_key(key_id: str) -> dict:
"""Revoke one panel. Everyone else stays connected — the whole reason keys
are per panel rather than one shared node key."""
from worker.inbound import service as inbound_service # noqa: PLC0415
if not await inbound_service.node.revoke_key(key_id):
raise HTTPException(status_code=404, detail="No such key.")
return inbound_service.node.snapshot()
@router.post("/inbound/sessions/{session_id}/disconnect")
def disconnect_inbound_session(session_id: str) -> dict:
from worker.inbound import service as inbound_service # noqa: PLC0415
if not inbound_service.node.log.kick(session_id):
raise HTTPException(status_code=404, detail="That connection has already ended.")
return inbound_service.node.snapshot()
@router.post("/inbound/connections")
async def add_inbound_connection(request: ConnectRequest) -> dict:
"""Paste a connection string from a GPU machine and dial it."""
from worker.inbound import service as inbound_service # noqa: PLC0415
from worker.inbound.connection_string import InvalidConnectionString # noqa: PLC0415
from worker.inbound.connector import InboundConnectionError # noqa: PLC0415
if not service.control_plane.running:
raise HTTPException(
status_code=409,
detail=(
"Remote workers are turned off. Enable them in "
"Settings → System → Remote workers first."
),
)
try:
connection = await inbound_service.outbound.add(
request.connection_string, service.control_plane.servicer
)
except InvalidConnectionString as exc:
# 400 with the parser's own words: every one of these otherwise
# surfaces as "cannot connect", which is what a firewall, a wrong port
# and a dead node all say too.
raise HTTPException(status_code=400, detail=str(exc)) from exc
except InboundConnectionError as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
return {"endpoint": connection.endpoint, "connections": inbound_service.outbound.snapshot()}
@router.delete("/inbound/connections/{endpoint}")
async def remove_inbound_connection(endpoint: str) -> dict:
from worker.inbound import service as inbound_service # noqa: PLC0415
from worker.inbound.connector import InboundConnectionError # noqa: PLC0415
try:
await inbound_service.outbound.remove(endpoint)
except InboundConnectionError as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
return {"connections": inbound_service.outbound.snapshot()}
+15
View File
@@ -26,6 +26,21 @@ class SystemInfoResponse(BaseModel):
model_config = ConfigDict(extra="allow")
app_version: str = ""
# Effective compute-time budgets (seconds) for one synthesis job — the
# values services/model_manager.py's GPU_JOB_TIMEOUT_S / CPU_JOB_TIMEOUT_S
# captured at backend import time (#1787). A value just saved via
# /system/set-env is NOT reflected here until the next restart.
generate_timeout_s: float = 300.0
cpu_generate_timeout_s: float = 600.0
# True when an external env var (shell, `.env`, Docker, …) is currently
# shadowing a prefs.json save for this key — see core.prefs.is_env_shadowed.
generate_timeout_shadowed: bool = False
cpu_generate_timeout_shadowed: bool = False
# #1770: the desktop attach handshake's code fingerprint — whatever
# Tauri set OMNIVOICE_BUILD_FINGERPRINT to when it spawned this process,
# echoed back verbatim. Blank when unset (dev mode, a manually started
# backend). See frontend/src-tauri/src/backend.rs::code_fingerprint_is_current.
code_fingerprint: str = ""
data_dir: str
outputs_dir: str
crash_log_path: str
Binary file not shown.
@@ -0,0 +1,3 @@
1
00:00:00,000 --> 00:00:13,720
VoiceStudio es una aplicación de escritorio para clonación de voz, doblaje de vídeo y diseño de voz. Funciona completamente en tu máquina. Sin cuentas, sin nube, sin claves de API. Solo abre la aplicación y comienza a crear.
Binary file not shown.
@@ -0,0 +1,3 @@
1
00:00:00,000 --> 00:00:15,000
VoiceStudio est une application de bureau pour le clonage de voix, le doublage vidéo et la conception vocale. Elle fonctionne entièrement sur votre machine. Pas de compte, pas de cloud, pas de clé d'API. Ouvrez l'application et commencez à créer.
Binary file not shown.
@@ -0,0 +1,3 @@
1
00:00:00,000 --> 00:00:16,560
VoiceStudioは、ボイスクローン、ビデオ吹き替え、ボイスデザインのためのデスクトップアプリです。すべてお使いのコンピュータ上で動作します。アカウント、クラウド、APIキーは不要です。アプリを開けば、すぐに制作を始められます。
Binary file not shown.
@@ -0,0 +1,3 @@
1
00:00:00,000 --> 00:00:13,200
VoiceStudio 是一款桌面应用,用于语音克隆、视频配音和声音设计。它完全在你的电脑上运行。无需账户,无需云端,无需 API 密钥。打开应用即可开始创作。
@@ -0,0 +1,47 @@
{
"version": "0.3.0",
"rendered_by": "omnivoice engine + ffmpeg showwaves",
"rendered_at": "2026-08-12T19:47:29Z",
"license": "MIT (synthetic, no third-party IP)",
"source": {
"code": "en",
"label": "English",
"video": "source.mp4",
"srt": "source.srt",
"script": "VoiceStudio is a desktop app for voice cloning, video dubbing, and voice design. It runs entirely on your machine. No accounts, no cloud, no API keys. Just open the app and start creating."
},
"dubbed": [
{
"code": "es",
"label": "Español",
"video": "dubbed_es.mp4",
"srt": "dubbed_es.srt",
"dir": "ltr",
"script": "VoiceStudio es una aplicación de escritorio para clonación de voz, doblaje de vídeo y diseño de voz. Funciona completamente en tu máquina. Sin cuentas, sin nube, sin claves de API. Solo abre la aplicación y comienza a crear."
},
{
"code": "fr",
"label": "Français",
"video": "dubbed_fr.mp4",
"srt": "dubbed_fr.srt",
"dir": "ltr",
"script": "VoiceStudio est une application de bureau pour le clonage de voix, le doublage vidéo et la conception vocale. Elle fonctionne entièrement sur votre machine. Pas de compte, pas de cloud, pas de clé d'API. Ouvrez l'application et commencez à créer."
},
{
"code": "zh",
"label": "中文",
"video": "dubbed_zh.mp4",
"srt": "dubbed_zh.srt",
"dir": "ltr",
"script": "VoiceStudio 是一款桌面应用,用于语音克隆、视频配音和声音设计。它完全在你的电脑上运行。无需账户,无需云端,无需 API 密钥。打开应用即可开始创作。"
},
{
"code": "ja",
"label": "日本語",
"video": "dubbed_ja.mp4",
"srt": "dubbed_ja.srt",
"dir": "ltr",
"script": "VoiceStudioは、ボイスクローン、ビデオ吹き替え、ボイスデザインのためのデスクトップアプリです。すべてお使いのコンピュータ上で動作します。アカウント、クラウド、APIキーは不要です。アプリを開けば、すぐに制作を始められます。"
}
]
}
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,3 @@
1
00:00:00,000 --> 00:00:11,400
VoiceStudio is a desktop app for voice cloning, video dubbing, and voice design. It runs entirely on your machine. No accounts, no cloud, no API keys. Just open the app and start creating.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+28 -10
View File
@@ -159,17 +159,16 @@ models:
- repo_id: "csukuangfj/sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8"
label: "Parakeet TDT v3 (sherpa-onnx — dictation, 25 EU langs)"
role: ASR
size_gb: 0.18
size_gb: 0.67
engine: sherpa-onnx
dictation_id: sherpa-parakeet-tdt-v3
tag: offline
curated_on: [all]
note: "Recommended live-dictation default. CPU, int8 ONNX. Requires sherpa-onnx."
note: "Multilingual European-language dictation. CPU, int8 ONNX. Requires sherpa-onnx."
- repo_id: "csukuangfj/sherpa-onnx-nemo-parakeet-tdt-0.6b-v2-int8"
label: "Parakeet TDT v2 (sherpa-onnx — dictation, English)"
role: ASR
size_gb: 0.17
size_gb: 0.66
engine: sherpa-onnx
dictation_id: sherpa-parakeet-tdt-v2
tag: offline
@@ -178,7 +177,7 @@ models:
- repo_id: "csukuangfj/sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20"
label: "Zipformer Bilingual (sherpa-onnx — streaming, zh+en)"
role: ASR
size_gb: 0.13
size_gb: 0.2
engine: sherpa-onnx
dictation_id: sherpa-zipformer-bilingual-zh-en
tag: streaming
@@ -187,7 +186,7 @@ models:
- repo_id: "csukuangfj/sherpa-onnx-streaming-paraformer-bilingual-zh-en"
label: "Paraformer Bilingual (sherpa-onnx — streaming, zh+en)"
role: ASR
size_gb: 0.115
size_gb: 0.24
engine: sherpa-onnx
dictation_id: sherpa-paraformer-bilingual-zh-en
tag: streaming
@@ -196,7 +195,7 @@ models:
- repo_id: "csukuangfj/sherpa-onnx-streaming-zipformer-en-20M-2023-02-17"
label: "Zipformer Streaming EN 20M (sherpa-onnx — streaming, English)"
role: ASR
size_gb: 0.128
size_gb: 0.044
engine: sherpa-onnx
dictation_id: sherpa-zipformer-en-20m
tag: streaming
@@ -205,7 +204,7 @@ models:
- repo_id: "csukuangfj/sherpa-onnx-streaming-zipformer-zh-14M-2023-02-23"
label: "Zipformer Streaming ZH 14M (sherpa-onnx — streaming, Chinese)"
role: ASR
size_gb: 0.074
size_gb: 0.025
engine: sherpa-onnx
dictation_id: sherpa-zipformer-zh-14m
tag: streaming
@@ -214,11 +213,12 @@ models:
- repo_id: "csukuangfj/sherpa-onnx-whisper-tiny"
label: "Whisper Tiny (sherpa-onnx — dictation, 90+ langs)"
role: ASR
size_gb: 0.116
size_gb: 0.104
engine: sherpa-onnx
dictation_id: sherpa-whisper-tiny
tag: offline
note: "Multilingual offline dictation (auto-detect). CPU, int8 ONNX. Requires sherpa-onnx."
curated_on: [all]
note: "Recommended cross-platform dictation default (auto-detect). CPU, int8 ONNX. Requires sherpa-onnx."
# ── Diarisation ───────────────────────────────────────────────────────
@@ -242,6 +242,24 @@ models:
size_gb: 0.08
curated_on: [all]
- repo_id: "openbmb/VoxCPM2"
label: "VoxCPM2 (30 languages, voice cloning and design)"
role: TTS
size_gb: 5.0
curated_on: [cuda]
- repo_id: "FunAudioLLM/Fun-CosyVoice3-0.5B-2512"
label: "CosyVoice 3 0.5B (multilingual zero-shot)"
role: TTS
size_gb: 9.8
curated_on: [cuda]
- repo_id: "lj1995/GPT-SoVITS"
label: "GPT-SoVITS pretrained weights"
role: TTS
size_gb: 2.0
curated_on: [cuda]
# ── mlx-audio engines (Apple Silicon only) ────────────────────────────
- repo_id: "mlx-community/Kokoro-82M-bf16"
+106
View File
@@ -0,0 +1,106 @@
"""Lightweight validation for persisted profile WAV references.
This module deliberately uses only the standard library. Gallery routers import
it during startup, so pulling in torch/torchaudio merely to validate a cached
file would make every Gallery open pay the model stack's import cost.
"""
from __future__ import annotations
import os
import wave
from pathlib import Path
from typing import Optional
from core.path_security import UnsafePath, resolve_within, safe_filename
_READ_CHUNK_BYTES = 1 << 20
_MAX_CHANNELS = 64
_MAX_SAMPLE_RATE = 768_000
_MAX_SAMPLE_WIDTH = 8
def resolve_regular_file(root: os.PathLike[str] | str, value: object) -> Optional[Path]:
"""Resolve a portable bare filename inside *root*, rejecting symlinks."""
try:
name = safe_filename(value)
unresolved = Path(root).resolve(strict=False) / name
if unresolved.is_symlink():
return None
return resolve_within(root, name)
except (OSError, UnsafePath):
return None
def is_playable_wav(path: Optional[Path]) -> bool:
"""Return true only for a regular, decodable WAV with audio frames."""
if path is None:
return False
try:
if not path.is_file() or path.is_symlink():
return False
file_size = path.stat().st_size
with wave.open(str(path), "rb") as wav:
channels = wav.getnchannels()
sample_rate = wav.getframerate()
sample_width = wav.getsampwidth()
frame_count = wav.getnframes()
if (
not 0 < channels <= _MAX_CHANNELS
or not 0 < sample_rate <= _MAX_SAMPLE_RATE
or not 0 < sample_width <= _MAX_SAMPLE_WIDTH
or frame_count <= 0
):
return False
# ``wave.getnframes`` trusts the header. Read through the declared
# payload so an interrupted write with a complete header but a
# truncated data chunk cannot masquerade as playable audio.
frame_size = channels * sample_width
expected_bytes = frame_count * frame_size
# A PCM payload cannot be larger than the containing file. Check
# before calling ``readframes`` so hostile header values cannot
# turn a tiny file into a multi-gigabyte allocation request.
if expected_bytes > file_size:
return False
read_bytes = 0
chunk_frames = max(1, min(frame_count, _READ_CHUNK_BYTES // frame_size))
while read_bytes < expected_bytes:
chunk = wav.readframes(chunk_frames)
if not chunk or len(chunk) % frame_size:
return False
read_bytes += len(chunk)
return read_bytes == expected_bytes
except (MemoryError, OSError, EOFError, OverflowError, wave.Error):
# Python 3.11's wave module rejects valid IEEE-float/WAVE_EXTENSIBLE
# files. SoundFile is already a runtime dependency and recognizes those
# containers; import it only on the uncommon fallback path.
try:
import soundfile as sf
with sf.SoundFile(str(path)) as audio:
if (
audio.format != "WAV"
or not 0 < audio.channels <= _MAX_CHANNELS
or not 0 < audio.samplerate <= _MAX_SAMPLE_RATE
or len(audio) <= 0
):
return False
remaining = len(audio)
# Decode through the declared payload in byte-bounded chunks;
# ``sf.info`` alone also trusts a truncated file's header.
chunk_frames = max(
1, _READ_CHUNK_BYTES // (audio.channels * 4),
)
while remaining:
frames = audio.read(
min(remaining, chunk_frames), dtype="float32", always_2d=True,
)
count = len(frames)
if count <= 0:
return False
remaining -= count
return True
except Exception:
return False
__all__ = ["is_playable_wav", "resolve_regular_file"]
+421
View File
@@ -0,0 +1,421 @@
"""Canonical authentication identity for HTTP and WebSocket connections.
Transport parsing belongs here; authorization remains in FastAPI dependencies.
Each ASGI scope receives exactly one secret-free :class:`AuthPrincipal` so
middleware and route guards cannot disagree about credential precedence.
"""
from __future__ import annotations
import ipaddress
import importlib
import os
import secrets
from collections.abc import Mapping
from dataclasses import dataclass, field
from enum import Enum
from services.admin_sessions import (
AdminSessionStore,
)
_AUTH_STATE_KEY = "auth_principal"
_LOOPBACK_HOSTS = frozenset({"127.0.0.1", "::1", "localhost"})
CONSUME_CAPABILITIES = frozenset({"consume"})
ADMIN_CAPABILITIES = frozenset({"consume", "admin"})
LOOPBACK_CAPABILITIES = frozenset({"consume", "admin", "native"})
class PrincipalKind(str, Enum):
ANONYMOUS = "anonymous"
LOOPBACK = "loopback"
TRUSTED_NETWORK = "trusted_network"
PIN = "pin"
API_KEY = "api_key"
ADMIN_SESSION = "admin_session"
class CredentialTransport(str, Enum):
NONE = "none"
HEADER = "header"
QUERY = "query"
COOKIE = "cookie"
LEGACY_COOKIE = "legacy_cookie"
WS_TICKET = "ws_ticket"
@dataclass(frozen=True)
class AuthPrincipal:
kind: PrincipalKind
capabilities: frozenset[str]
credential_id: str | None = None
transport: CredentialTransport = CredentialTransport.NONE
def allows(self, capability: str) -> bool:
return capability in self.capabilities
@dataclass(frozen=True)
class _CredentialCandidate:
value: str = field(repr=False)
transport: CredentialTransport
allow_master: bool = False
allow_session: bool = False
allow_ticket: bool = False
def remote_api_key() -> str | None:
"""Normalized remote operator key, read dynamically for rotation support."""
return os.environ.get("OMNIVOICE_API_KEY", "").strip() or None
def credential_matches(supplied: str | None, configured: str | None) -> bool:
"""Constant-time credential comparison that accepts the full Unicode range."""
if not supplied or not configured:
return False
return secrets.compare_digest(
supplied.encode("utf-8", errors="surrogatepass"),
configured.encode("utf-8", errors="surrogatepass"),
)
def _active_admin_session_store() -> AdminSessionStore:
"""Resolve mutable process state at call time so app reloads cannot split it."""
module = importlib.import_module("services.admin_sessions")
return module.admin_session_store
def _trusted_networks() -> tuple[ipaddress.IPv4Network | ipaddress.IPv6Network, ...]:
networks = []
for value in os.environ.get("OMNIVOICE_TRUSTED_NETWORKS", "").split(","):
value = value.strip()
if not value:
continue
try:
networks.append(ipaddress.ip_network(value, strict=False))
except ValueError:
# Invalid configuration never makes the gate fail open or wedge the
# backend. It simply contributes no trusted range.
continue
return tuple(networks)
def is_loopback(host: str | None) -> bool:
return host in _LOOPBACK_HOSTS
def is_local_host(host: str | None) -> bool:
if is_loopback(host):
return True
try:
address = ipaddress.ip_address(host)
except (TypeError, ValueError):
return False
if getattr(address, "ipv4_mapped", None):
address = address.ipv4_mapped
return any(address in network for network in _trusted_networks())
def _mapping_get(mapping: Mapping[str, str] | object, name: str) -> str:
if not mapping:
return ""
getter = getattr(mapping, "get", None)
if callable(getter):
value = getter(name, "")
if value:
return str(value)
# Real Starlette Headers are case-insensitive. This small fallback keeps
# minimal request stubs and non-Starlette callers correct too.
items = getattr(mapping, "items", None)
if callable(items):
for key, value in items():
if str(key).lower() == name.lower():
return str(value or "")
return ""
def _scope_type(connection) -> str:
scope = getattr(connection, "scope", None)
return str(scope.get("type", "http")) if isinstance(scope, dict) else "http"
def _path(connection) -> str:
scope = getattr(connection, "scope", None)
if isinstance(scope, dict):
return str(scope.get("path", ""))
return str(getattr(connection, "url", "") or "")
def _canonical_websocket_path(connection) -> str:
"""Remove only the ASGI-configured deployment prefix from a WS path."""
path = _path(connection)
scope = getattr(connection, "scope", None)
if not isinstance(scope, dict):
return path
root_path = str(scope.get("root_path", "") or "").rstrip("/")
if not root_path or root_path == "/":
return path
root_path = "/" + root_path.lstrip("/")
if path.startswith(root_path + "/"):
return path[len(root_path) :]
return path
def _client_host(connection) -> str | None:
client = getattr(connection, "client", None)
if client is not None:
return getattr(client, "host", None)
scope = getattr(connection, "scope", None)
if isinstance(scope, dict) and scope.get("client"):
return scope["client"][0]
return None
def _credential_candidate(connection) -> _CredentialCandidate | None:
query = getattr(connection, "query_params", None) or {}
cookies = getattr(connection, "cookies", None) or {}
raw_authorization = authorization_header(connection)
authorization = raw_authorization.strip()
if raw_authorization.lower().startswith("bearer "):
value = raw_authorization[7:].strip()
if value:
return _CredentialCandidate(
value=value,
transport=CredentialTransport.HEADER,
allow_master=True,
allow_session=True,
)
# Preserve the legacy normalization contract: ``Bearer`` followed
# only by whitespace is equivalent to an empty credential channel.
elif authorization:
# Any non-empty explicit Authorization value is authoritative, even
# when its scheme is unsupported or its Bearer payload is missing.
# It must never fall through to a stale ambient cookie.
return _CredentialCandidate(
value=authorization,
transport=CredentialTransport.HEADER,
)
if _scope_type(connection) == "websocket":
ticket = _mapping_get(query, "ws_ticket").strip()
if ticket:
return _CredentialCandidate(
value=ticket,
transport=CredentialTransport.WS_TICKET,
allow_ticket=True,
)
query_key = _mapping_get(query, "api_key").strip()
if query_key:
return _CredentialCandidate(
value=query_key,
transport=CredentialTransport.QUERY,
allow_master=True,
)
session = _mapping_get(cookies, "ov_session").strip()
if session:
return _CredentialCandidate(
value=session,
transport=CredentialTransport.COOKIE,
allow_session=True,
)
legacy_key = _mapping_get(cookies, "ov_key").strip()
if legacy_key:
return _CredentialCandidate(
value=legacy_key,
transport=CredentialTransport.LEGACY_COOKIE,
allow_master=True,
)
return None
def presented_api_key(connection) -> str:
"""Compatibility extractor for the durable API-key transports only."""
candidate = _credential_candidate(connection)
if candidate is None or not candidate.allow_master:
return ""
return candidate.value
def authorization_header(connection) -> str:
headers = getattr(connection, "headers", None) or {}
return _mapping_get(headers, "authorization")
def authorization_credential_present(connection) -> bool:
"""Whether Authorization contains an authoritative credential channel.
This deliberately mirrors :func:`_credential_candidate`: whitespace and
``Bearer`` followed only by spaces are empty channels that may fall back to
legacy migration state. Unsupported schemes and ``Bearer`` without the
required separating space remain explicit invalid credentials.
"""
authorization = authorization_header(connection)
if authorization.lower().startswith("bearer ") and not authorization[7:].strip():
return False
return bool(authorization.strip())
def bearer_header_value(connection) -> str:
authorization = authorization_header(connection)
if not authorization.lower().startswith("bearer "):
return ""
return authorization[7:].strip()
def legacy_master_cookie_valid(connection) -> bool:
configured = remote_api_key()
cookies = getattr(connection, "cookies", None) or {}
supplied = _mapping_get(cookies, "ov_key").strip()
return credential_matches(supplied, configured)
def master_header_valid(connection) -> bool:
configured = remote_api_key()
supplied = bearer_header_value(connection)
return credential_matches(supplied, configured)
def _configured_pin(connection) -> str | None:
app = getattr(connection, "app", None)
state = getattr(app, "state", None) if app is not None else None
network_share = getattr(state, "network_share", None) if state is not None else None
pin = getattr(network_share, "pin", None) if network_share is not None else None
return str(pin) if pin else None
def _valid_pin(connection) -> bool:
configured = _configured_pin(connection)
if not configured:
return False
headers = getattr(connection, "headers", None) or {}
query = getattr(connection, "query_params", None) or {}
cookies = getattr(connection, "cookies", None) or {}
supplied = (
_mapping_get(headers, "x-omnivoice-pin").strip()
or _mapping_get(query, "pin").strip()
or _mapping_get(cookies, "ov_pin").strip()
)
return credential_matches(supplied, configured)
def _attached_principal(connection) -> AuthPrincipal | None:
scope = getattr(connection, "scope", None)
if not isinstance(scope, dict):
return None
state = scope.get("state")
if isinstance(state, dict):
principal = state.get(_AUTH_STATE_KEY)
return principal if isinstance(principal, AuthPrincipal) else None
return None
def _attach_principal(connection, principal: AuthPrincipal) -> AuthPrincipal:
scope = getattr(connection, "scope", None)
if isinstance(scope, dict):
state = scope.setdefault("state", {})
if isinstance(state, dict):
state[_AUTH_STATE_KEY] = principal
return principal
def resolve_principal(
connection,
*,
store: AdminSessionStore | None = None,
) -> AuthPrincipal:
"""Resolve and attach the single authentication decision for one scope."""
attached = _attached_principal(connection)
if attached is not None:
return attached
if store is None:
store = _active_admin_session_store()
host = _client_host(connection)
if is_loopback(host):
return _attach_principal(
connection,
AuthPrincipal(PrincipalKind.LOOPBACK, LOOPBACK_CAPABILITIES),
)
candidate = _credential_candidate(connection)
configured_key = remote_api_key()
if candidate is not None:
principal: AuthPrincipal | None = None
if (
candidate.allow_master
and credential_matches(candidate.value, configured_key)
):
principal = AuthPrincipal(
PrincipalKind.API_KEY,
ADMIN_CAPABILITIES,
credential_id="api-key",
transport=candidate.transport,
)
elif candidate.allow_session:
session = store.resolve(candidate.value, configured_key)
if session is not None:
principal = AuthPrincipal(
PrincipalKind.ADMIN_SESSION,
session.capabilities,
credential_id=session.credential_id,
transport=candidate.transport,
)
elif candidate.allow_ticket:
session = store.consume_ws_ticket(
candidate.value,
_canonical_websocket_path(connection),
configured_key,
)
if session is not None:
principal = AuthPrincipal(
PrincipalKind.ADMIN_SESSION,
session.capabilities,
credential_id=session.credential_id,
transport=candidate.transport,
)
if principal is not None:
return _attach_principal(connection, principal)
# An explicit, non-empty credential is authoritative. Do not silently
# fall back to network or PIN trust after an invalid higher-priority
# credential was presented.
return _attach_principal(
connection,
AuthPrincipal(
PrincipalKind.ANONYMOUS,
frozenset(),
transport=candidate.transport,
),
)
if is_local_host(host):
return _attach_principal(
connection,
AuthPrincipal(PrincipalKind.TRUSTED_NETWORK, CONSUME_CAPABILITIES),
)
if _valid_pin(connection):
return _attach_principal(
connection,
AuthPrincipal(
PrincipalKind.PIN,
CONSUME_CAPABILITIES,
transport=CredentialTransport.HEADER,
),
)
return _attach_principal(
connection,
AuthPrincipal(PrincipalKind.ANONYMOUS, frozenset()),
)
def principal_for(
connection,
*,
store: AdminSessionStore | None = None,
) -> AuthPrincipal:
return _attached_principal(connection) or resolve_principal(connection, store=store)
+716
View File
@@ -0,0 +1,716 @@
"""Nested subprocess ownership for desktop-managed backend operations.
The desktop owns the backend with an OS process group/Job. Engine and
installer operations also need an independently terminable subtree: killing
only their direct child on a timeout leaves uv/git/model workers holding pipes
and mutating files.
On POSIX a small supervisor is the unreaped leader of a nested process group.
A control-pipe EOF (including kernel EOF when the backend dies) kills that
group; the parent also drains the group before reaping its stable leader. On
Windows the backend retains a nested kill-on-close Job directly and assigns
the suspended operation before resuming it. The outer desktop Job remains the
terminal fallback.
Standalone/server launches use the same nested owner, preserving their
independently terminable subtree without relying on ``taskkill`` or discovery.
"""
from __future__ import annotations
import os
import signal
import struct
import subprocess
import sys
import threading
import time
from pathlib import Path
from typing import Any, Optional
_RESULT = struct.Struct("!i")
_DESKTOP_MARKER = "OMNIVOICE_DESKTOP_CONTAINED"
_DRAIN_FD_ENV = "OMNIVOICE_DESKTOP_DRAIN_FD"
def backend_drain_fd(*, required: bool = False) -> Optional[int]:
"""Validated Rust-owned drain writer inherited by the desktop backend."""
if os.name != "posix" or os.environ.get(_DESKTOP_MARKER) != "1":
return None
try:
fd = int(os.environ[_DRAIN_FD_ENV])
os.fstat(fd)
except (KeyError, ValueError, OSError) as exc:
if required:
raise RuntimeError(
"desktop backend is missing its live nested-operation drain descriptor"
) from exc
return None
return fd
def secure_backend_drain_fd() -> None:
"""Restore CLOEXEC after Rust's one intentional backend inheritance."""
fd = backend_drain_fd(required=True)
if fd is not None:
os.set_inheritable(fd, False)
class OwnedPopen:
"""Popen-compatible handle for a desktop-owned nested operation."""
def __init__(
self,
proc: subprocess.Popen,
control_fd: int,
result_fd: int,
) -> None:
self._proc = proc
self._control_fd: Optional[int] = control_fd
self._result_fd: Optional[int] = result_fd
self._returncode: Optional[int] = None
self._lock = threading.RLock()
# Popen callers use these directly (protocol pipes and log drains).
self.stdin = proc.stdin
self.stdout = proc.stdout
self.stderr = proc.stderr
@property
def pid(self) -> int:
return self._proc.pid
@property
def args(self) -> Any:
return self._proc.args
@property
def returncode(self) -> Optional[int]:
return self._returncode
def _close_control(self) -> None:
fd, self._control_fd = self._control_fd, None
if fd is not None:
try:
os.close(fd)
except OSError:
# Cleanup is idempotent; another teardown path already closed it.
pass
def _read_result(self, fallback: int) -> int:
fd, self._result_fd = self._result_fd, None
if fd is None:
return fallback
try:
payload = b""
while len(payload) < _RESULT.size:
chunk = os.read(fd, _RESULT.size - len(payload))
if not chunk:
break
payload += chunk
return _RESULT.unpack(payload)[0] if len(payload) == _RESULT.size else fallback
except OSError:
return fallback
finally:
try:
os.close(fd)
except OSError:
# The descriptor may have been closed by cancellation cleanup.
pass
def _posix_exited_unreaped(self) -> bool:
flags = os.WEXITED | os.WNOHANG | os.WNOWAIT
info = os.waitid(os.P_PID, self.pid, flags)
return info is not None and info.si_pid != 0
def _posix_exited_reaping(self) -> Optional[int]:
"""macOS fallback for :meth:`_posix_exited_unreaped` (#1656).
CPython on macOS does not expose ``os.waitid`` (HAVE_WAITID is not set
in its build), so the WNOWAIT probe is unavailable there. This
fallback *reaps* the wrapper with ``waitpid(WNOHANG)``: it returns
the wrapper's exit code once it has exited, None while it is still
running, and raises ``ChildProcessError`` when another owner already
reaped it (the same refusal the waitid probe gives).
Reaping earlier than the WNOWAIT dance loses the pre-reap group kill
in :meth:`poll`; that is safe because the supervisor's control-pipe
EOF already terminates the whole nested group (#1635 design).
"""
pid, status = os.waitpid(self.pid, os.WNOHANG)
if pid != self.pid:
return None
rc = os.waitstatus_to_exitcode(status)
# Publish on the underlying Popen so its own wait()/poll() no-op.
self._proc.returncode = rc
return rc
def _posix_exit_state_reaping(self) -> Optional[int]:
""":meth:`_posix_exited_reaping` plus one concession: if the leader
was already reaped through *this* Popen (``_proc.returncode`` known),
report that code rather than refusing reaping by our own handle is
not the foreign reaper the ECHILD refusal exists for."""
try:
return self._posix_exited_reaping()
except ChildProcessError:
return self._proc.returncode
def _signal_owned_group(self, sig: int) -> None:
# The numeric group is safe only while its direct-child leader remains
# ours and unreaped. ECHILD therefore refuses rather than guessing.
try:
os.waitid(os.P_PID, self.pid, os.WEXITED | os.WNOHANG | os.WNOWAIT)
except ChildProcessError:
return
except AttributeError:
# macOS CPython has no os.waitid (#1656). waitpid still proves
# that this exact numeric pid is our live child: ECHILD refuses a
# foreign-reaped/reused pid, while pid == self.pid records an exit
# without ever signalling the now-unowned process-group number.
try:
pid, status = os.waitpid(self.pid, os.WNOHANG)
except ChildProcessError:
return
if pid == self.pid:
self._proc.returncode = os.waitstatus_to_exitcode(status)
return
try:
os.killpg(self.pid, sig)
except ProcessLookupError:
# The owned group exited between the waitid probe and the signal.
pass
def poll(self) -> Optional[int]:
with self._lock:
if self._returncode is not None:
return self._returncode
if os.name == "posix":
try:
if hasattr(os, "waitid"):
if not self._posix_exited_unreaped():
return None
self._signal_owned_group(signal.SIGKILL)
wrapper_rc = self._proc.wait()
else:
# macOS CPython: no os.waitid (#1656) — the reaping
# probe already terminated/killed nothing; the group
# is torn down by the control-pipe EOF in _close_control.
wrapper_rc = self._posix_exit_state_reaping()
if wrapper_rc is None:
return None
except ChildProcessError:
# Never signal a potentially reused group after another
# owner reaped the stable leader.
return None
else:
wrapper_rc = self._proc.poll()
if wrapper_rc is None:
return None
self._close_control()
self._returncode = self._read_result(wrapper_rc)
return self._returncode
def wait(self, timeout: Optional[float] = None) -> int:
deadline = None if timeout is None else time.monotonic() + timeout
while True:
rc = self.poll()
if rc is not None:
return rc
if deadline is not None and time.monotonic() >= deadline:
raise subprocess.TimeoutExpired(self.args, timeout)
time.sleep(0.01)
def terminate(self) -> None:
with self._lock:
if self._returncode is not None:
return
self._close_control()
if os.name == "posix":
self._signal_owned_group(signal.SIGTERM)
else:
# Closing the control pipe asks the supervisor to terminate
# its nested Job. The stable wrapper handle is a fallback.
try:
self._proc.terminate()
except OSError:
# The wrapper exited after the return-code check.
pass
def kill(self) -> None:
with self._lock:
if self._returncode is not None:
return
self._close_control()
if os.name == "posix":
self._signal_owned_group(signal.SIGKILL)
else:
try:
self._proc.kill()
except OSError:
# The wrapper exited after the return-code check.
pass
def __getattr__(self, name: str) -> Any:
return getattr(self._proc, name)
def __del__(self) -> None:
self._close_control()
fd, self._result_fd = self._result_fd, None
if fd is not None:
try:
os.close(fd)
except OSError:
# Finalization may race explicit wait or cancellation cleanup.
pass
class WindowsJobPopen:
"""Popen-compatible handle whose child tree lives in a retained Job.
Windows Job handles already provide the stable ownership that POSIX needs
a supervisor process group for. Keeping the handle in the backend means an
abrupt backend exit closes it in the kernel and kills the whole operation
tree, without inserting a second Python process in the sidecar loader path
(#1734).
"""
def __init__(self, proc: subprocess.Popen, job: Any, kernel32: Any) -> None:
self._proc = proc
self._job = job
self._kernel32 = kernel32
self._lock = threading.RLock()
self.stdin = proc.stdin
self.stdout = proc.stdout
self.stderr = proc.stderr
@property
def pid(self) -> int:
return self._proc.pid
@property
def args(self) -> Any:
return self._proc.args
@property
def returncode(self) -> Optional[int]:
return self._proc.returncode
def _close_job(self, *, terminate: bool) -> None:
job, self._job = self._job, None
if job is None:
return
try:
if terminate:
self._kernel32.TerminateJobObject(job, 1)
finally:
self._kernel32.CloseHandle(job)
def poll(self) -> Optional[int]:
with self._lock:
rc = self._proc.poll()
if rc is None:
return None
# A successful direct child may leave helpers behind. Match the
# supervisor contract by draining the retained Job before return.
self._close_job(terminate=True)
return rc
def wait(self, timeout: Optional[float] = None) -> int:
try:
rc = self._proc.wait(timeout=timeout)
except subprocess.TimeoutExpired:
raise
with self._lock:
self._close_job(terminate=True)
return rc
def terminate(self) -> None:
with self._lock:
self._close_job(terminate=True)
def kill(self) -> None:
self.terminate()
def __getattr__(self, name: str) -> Any:
return getattr(self._proc, name)
def __del__(self) -> None:
try:
self._close_job(terminate=True)
except Exception:
pass # interpreter shutdown; closing the OS handle is best-effort
def _spawn_windows_owned(argv: list[str], kwargs: dict[str, Any]) -> WindowsJobPopen:
"""Start *argv* suspended, assign its tree to a Job, then resume it."""
import ctypes
job, kernel32, wintypes = _windows_job()
child: Optional[subprocess.Popen] = None
popen_kwargs = dict(kwargs)
supplied_env = popen_kwargs.get("env")
operation_env = dict(os.environ if supplied_env is None else supplied_env)
operation_env.pop(_DRAIN_FD_ENV, None)
operation_env.pop(_DESKTOP_MARKER, None)
popen_kwargs["env"] = operation_env
supplied_flags = int(popen_kwargs.pop("creationflags", 0))
popen_kwargs["creationflags"] = supplied_flags | 0x08000000 | 0x00000004
try:
child = subprocess.Popen(argv, **popen_kwargs)
assign = kernel32.AssignProcessToJobObject
assign.argtypes = (wintypes.HANDLE, wintypes.HANDLE)
assign.restype = wintypes.BOOL
if not assign(job, wintypes.HANDLE(child._handle)):
raise OSError(ctypes.get_last_error(), "AssignProcessToJobObject")
_resume_windows_process(kernel32, wintypes, child.pid)
return WindowsJobPopen(child, job, kernel32)
except BaseException:
kernel32.TerminateJobObject(job, 1)
if child is not None:
try:
child.kill()
except OSError:
pass # the suspended child may already have exited
try:
child.wait(timeout=5)
except (OSError, subprocess.TimeoutExpired):
pass # Job termination remains the authoritative cleanup
kernel32.CloseHandle(job)
raise
def spawn_owned(
argv: list[str], **kwargs: Any
) -> "subprocess.Popen | OwnedPopen | WindowsJobPopen":
"""Spawn an operation with a stable, independently terminable owner."""
if os.name == "nt":
return _spawn_windows_owned(argv, kwargs)
drain_fd = backend_drain_fd(required=True)
control_read, control_write = os.pipe()
result_read, result_write = os.pipe()
wrapper_argv = _supervisor_argv(
control_read,
result_write,
argv,
)
wrapper_kwargs = dict(kwargs)
wrapper_kwargs["start_new_session"] = True
pass_fds = [control_read, result_write]
if drain_fd is not None:
pass_fds.append(drain_fd)
if wrapper_kwargs.get("env") is not None:
wrapper_env = dict(wrapper_kwargs["env"])
wrapper_env[_DESKTOP_MARKER] = "1"
wrapper_env[_DRAIN_FD_ENV] = str(drain_fd)
wrapper_kwargs["env"] = wrapper_env
wrapper_kwargs["pass_fds"] = tuple(pass_fds)
try:
proc = subprocess.Popen(wrapper_argv, **wrapper_kwargs)
except BaseException:
# The finally block exclusively owns the child-side endpoints. Closing
# them here as well risks closing a reused descriptor in another thread.
for fd in (control_write, result_read):
try:
os.close(fd)
except OSError:
# A partial spawn may already have closed a parent-side endpoint.
pass
raise
finally:
for fd in (control_read, result_write):
try:
os.close(fd)
except OSError:
# Popen may have consumed an inherited child-side endpoint.
pass
return OwnedPopen(proc, control_write, result_read)
def _supervisor_argv(
control_token: int,
result_token: int,
argv: list[str],
) -> list[str]:
prefix = [sys.executable]
if not getattr(sys, "frozen", False):
prefix.append(str(Path(__file__).resolve().parents[1] / "main.py"))
return [
*prefix,
"--supervise",
str(control_token),
str(result_token),
"--",
*map(str, argv),
]
def _write_result(fd: int, returncode: int) -> None:
try:
os.write(fd, _RESULT.pack(int(returncode)))
except OSError:
# The caller may have cancelled and closed its result reader.
pass
finally:
try:
os.close(fd)
except OSError:
# Writing or cancellation may already have closed the descriptor.
pass
def _operation_env() -> dict[str, str]:
env = os.environ.copy()
# The operation intentionally does not own the Rust drain writer. Avoid
# exposing a stale numeric token which nested code could mistake as valid.
env.pop(_DRAIN_FD_ENV, None)
env.pop(_DESKTOP_MARKER, None)
return env
def _supervise_posix(control_fd: int, result_fd: int, argv: list[str]) -> int:
def cancel_on_eof() -> None:
try:
while os.read(control_fd, 1):
pass
except OSError:
# Closing the control descriptor is itself a cancellation signal.
pass
os.killpg(os.getpgrp(), signal.SIGKILL)
threading.Thread(target=cancel_on_eof, daemon=True).start()
try:
child = subprocess.Popen(argv, close_fds=True, env=_operation_env())
rc = child.wait()
except OSError:
rc = 127
_write_result(result_fd, rc)
# Drain children which outlived the operation before the stable group
# leader exits. SIGKILL intentionally includes this supervisor.
os.killpg(os.getpgrp(), signal.SIGKILL)
return rc # unreachable
def _windows_job() -> tuple[Any, Any, Any]:
import ctypes
import ctypes.wintypes as wintypes
kernel32 = ctypes.WinDLL("kernel32", use_last_error=True)
kernel32.CloseHandle.argtypes = (wintypes.HANDLE,)
kernel32.CloseHandle.restype = wintypes.BOOL
kernel32.TerminateJobObject.argtypes = (wintypes.HANDLE, wintypes.UINT)
kernel32.TerminateJobObject.restype = wintypes.BOOL
kernel32.ReadFile.argtypes = (
wintypes.HANDLE,
ctypes.c_void_p,
wintypes.DWORD,
ctypes.POINTER(wintypes.DWORD),
ctypes.c_void_p,
)
kernel32.ReadFile.restype = wintypes.BOOL
kernel32.WriteFile.argtypes = (
wintypes.HANDLE,
ctypes.c_void_p,
wintypes.DWORD,
ctypes.POINTER(wintypes.DWORD),
ctypes.c_void_p,
)
kernel32.WriteFile.restype = wintypes.BOOL
create = kernel32.CreateJobObjectW
create.argtypes = (ctypes.c_void_p, wintypes.LPCWSTR)
create.restype = wintypes.HANDLE
job = create(None, None)
if not job:
raise OSError(ctypes.get_last_error(), "CreateJobObjectW")
class BasicLimits(ctypes.Structure):
_fields_ = [
("PerProcessUserTimeLimit", ctypes.c_longlong),
("PerJobUserTimeLimit", ctypes.c_longlong),
("LimitFlags", wintypes.DWORD),
("MinimumWorkingSetSize", ctypes.c_size_t),
("MaximumWorkingSetSize", ctypes.c_size_t),
("ActiveProcessLimit", wintypes.DWORD),
("Affinity", ctypes.c_size_t),
("PriorityClass", wintypes.DWORD),
("SchedulingClass", wintypes.DWORD),
]
class IoCounters(ctypes.Structure):
_fields_ = [(name, ctypes.c_ulonglong) for name in (
"ReadOperationCount", "WriteOperationCount", "OtherOperationCount",
"ReadTransferCount", "WriteTransferCount", "OtherTransferCount",
)]
class ExtendedLimits(ctypes.Structure):
_fields_ = [
("BasicLimitInformation", BasicLimits),
("IoInfo", IoCounters),
("ProcessMemoryLimit", ctypes.c_size_t),
("JobMemoryLimit", ctypes.c_size_t),
("PeakProcessMemoryUsed", ctypes.c_size_t),
("PeakJobMemoryUsed", ctypes.c_size_t),
]
info = ExtendedLimits()
info.BasicLimitInformation.LimitFlags = 0x00002000 # KILL_ON_JOB_CLOSE
set_info = kernel32.SetInformationJobObject
set_info.argtypes = (wintypes.HANDLE, ctypes.c_int, ctypes.c_void_p, wintypes.DWORD)
set_info.restype = wintypes.BOOL
if not set_info(job, 9, ctypes.byref(info), ctypes.sizeof(info)):
error = ctypes.get_last_error()
kernel32.CloseHandle(job)
raise OSError(error, "SetInformationJobObject")
return job, kernel32, wintypes
def _resume_windows_process(kernel32: Any, wintypes: Any, pid: int) -> None:
import ctypes
class ThreadEntry(ctypes.Structure):
_fields_ = [
("dwSize", wintypes.DWORD),
("cntUsage", wintypes.DWORD),
("th32ThreadID", wintypes.DWORD),
("th32OwnerProcessID", wintypes.DWORD),
("tpBasePri", wintypes.LONG),
("tpDeltaPri", wintypes.LONG),
("dwFlags", wintypes.DWORD),
]
kernel32.CreateToolhelp32Snapshot.argtypes = (wintypes.DWORD, wintypes.DWORD)
kernel32.CreateToolhelp32Snapshot.restype = wintypes.HANDLE
kernel32.Thread32First.argtypes = (wintypes.HANDLE, ctypes.POINTER(ThreadEntry))
kernel32.Thread32First.restype = wintypes.BOOL
kernel32.Thread32Next.argtypes = (wintypes.HANDLE, ctypes.POINTER(ThreadEntry))
kernel32.Thread32Next.restype = wintypes.BOOL
kernel32.OpenThread.argtypes = (wintypes.DWORD, wintypes.BOOL, wintypes.DWORD)
kernel32.OpenThread.restype = wintypes.HANDLE
kernel32.ResumeThread.argtypes = (wintypes.HANDLE,)
kernel32.ResumeThread.restype = wintypes.DWORD
snapshot = kernel32.CreateToolhelp32Snapshot(0x00000004, 0)
invalid = ctypes.c_void_p(-1).value
if snapshot == invalid:
raise OSError(ctypes.get_last_error(), "CreateToolhelp32Snapshot")
try:
entry = ThreadEntry(dwSize=ctypes.sizeof(ThreadEntry))
found = kernel32.Thread32First(snapshot, ctypes.byref(entry))
while found:
if entry.th32OwnerProcessID == pid:
thread = kernel32.OpenThread(0x0002, False, entry.th32ThreadID)
if not thread:
raise OSError(ctypes.get_last_error(), "OpenThread")
try:
if kernel32.ResumeThread(thread) == 0xFFFFFFFF:
raise OSError(ctypes.get_last_error(), "ResumeThread")
return
finally:
kernel32.CloseHandle(thread)
found = kernel32.Thread32Next(snapshot, ctypes.byref(entry))
finally:
kernel32.CloseHandle(snapshot)
raise OSError("suspended operation thread was not found")
def _supervise_windows(control_fd: int, result_fd: int, argv: list[str]) -> int:
import ctypes
job, kernel32, wintypes = _windows_job()
cancelled = threading.Event()
job_lock = threading.Lock()
job_open = True
def terminate_job() -> None:
with job_lock:
if job_open:
kernel32.TerminateJobObject(job, 1)
def cancel_on_eof() -> None:
byte = ctypes.create_string_buffer(1)
count = wintypes.DWORD()
while kernel32.ReadFile(
wintypes.HANDLE(control_fd), byte, 1, ctypes.byref(count), None
) and count.value:
pass
kernel32.CloseHandle(wintypes.HANDLE(control_fd))
cancelled.set()
terminate_job()
threading.Thread(target=cancel_on_eof, daemon=True).start()
child: Optional[subprocess.Popen] = None
rc = 127
try:
child = subprocess.Popen(
argv,
close_fds=True,
env=_operation_env(),
creationflags=0x08000000 | 0x00000004, # NO_WINDOW | SUSPENDED
)
assign = kernel32.AssignProcessToJobObject
assign.argtypes = (wintypes.HANDLE, wintypes.HANDLE)
assign.restype = wintypes.BOOL
if not assign(job, wintypes.HANDLE(child._handle)):
raise OSError(ctypes.get_last_error(), "AssignProcessToJobObject")
if cancelled.is_set():
terminate_job()
else:
_resume_windows_process(kernel32, wintypes, child.pid)
rc = child.wait()
# A successful direct child may leave helpers behind; terminate the
# nested stable Job before reporting completion.
terminate_job()
except OSError:
terminate_job()
if child is not None:
try:
# Assignment itself may have failed, leaving this suspended
# process outside the nested Job. Terminate it through its
# stable process handle before waiting; never strand an
# unassigned operation or rely on the outer desktop Job.
child.kill()
except OSError:
# The suspended child may have exited during Job teardown.
pass
try:
child.wait(timeout=5)
except (OSError, subprocess.TimeoutExpired):
# The outer desktop Job remains the terminal containment fallback.
pass
finally:
payload = _RESULT.pack(int(rc))
payload_buffer = ctypes.create_string_buffer(payload)
written = wintypes.DWORD()
kernel32.WriteFile(
wintypes.HANDLE(result_fd),
payload_buffer,
len(payload),
ctypes.byref(written),
None,
)
kernel32.CloseHandle(wintypes.HANDLE(result_fd))
with job_lock:
job_open = False
kernel32.CloseHandle(job)
return rc
def supervisor_main(args: list[str]) -> int:
if len(args) < 5 or args[0] != "--supervise" or args[3] != "--":
return 2
control_fd = int(args[1])
result_fd = int(args[2])
argv = args[4:]
secure_backend_drain_fd()
if os.name == "posix":
return _supervise_posix(control_fd, result_fd, argv)
return _supervise_windows(control_fd, result_fd, argv)
def _main() -> int:
return supervisor_main(sys.argv[1:])
if __name__ == "__main__":
raise SystemExit(_main())
+140
View File
@@ -0,0 +1,140 @@
"""Exact-origin CSRF checks for ambient browser authentication."""
from __future__ import annotations
import os
from urllib.parse import SplitResult, urlsplit
CSRF_HEADER = "x-voicestudio-csrf"
CSRF_VALUE = "1"
SAFE_HTTP_METHODS = frozenset({"GET", "HEAD", "OPTIONS"})
_FORWARDED_PROTO_HEADER = "x-forwarded-proto"
def effective_scheme(connection) -> str:
"""Scheme of the client-facing hop: the resolved scope, TLS-upgraded by proxy evidence.
Behind a TLS-terminating proxy (Tailscale Serve the flagship remote-GPU
deployment in docs/remote-gpu.md nginx, Caddy, ...) the browser talks
``https`` while the backend hop is plain ``http``. uvicorn's
ProxyHeadersMiddleware (on by default in both launch paths: ``uvicorn.run``
in backend/main.py and the Docker ``python -m uvicorn`` entrypoint) already
rewrites the ASGI scope from ``X-Forwarded-Proto``, but only when the peer
is in ``--forwarded-allow-ips`` (default: loopback). That covers Serve on
bare metal, and we prefer that signal the scope is consulted first but
it misses Docker (the proxy connects from the bridge gateway) and any other
non-loopback proxy topology, so the header is honored here as well.
Spoofing analysis why honoring it never weakens a check: the upgrade is
one-way. ``https``/``wss`` as the first forwarded value promotes ``http``
to ``https``; every other value is ignored, so a forged header can never
downgrade a genuine TLS hop. For the exact-origin comparison the host:port
half of the tuple is untouched, a browser cannot attach X-Forwarded-Proto
cross-site without a CORS preflight this API never grants, and a
non-browser client able to forge the header can already forge Origin
itself it gains nothing. For cookies the upgrade can only ADD the Secure
flag (a Secure cookie set over plain http is simply dropped by the
browser the spoofer only breaks their own session), never strip it.
"""
url = getattr(connection, "url", None)
scheme = getattr(url, "scheme", None)
if not scheme:
scope = getattr(connection, "scope", None)
scheme = scope.get("scheme", "http") if isinstance(scope, dict) else "http"
scheme = {"ws": "http", "wss": "https"}.get(scheme, scheme)
if scheme != "https":
headers = getattr(connection, "headers", None) or {}
forwarded = (
headers.get(_FORWARDED_PROTO_HEADER, "") if hasattr(headers, "get") else ""
)
if forwarded.split(",")[0].strip().lower() in {"https", "wss"}:
scheme = "https"
return scheme
def _origin_tuple(value: str | None) -> tuple[str, str, int | None] | None:
if not value or value == "null":
return None
try:
parsed: SplitResult = urlsplit(value)
port = parsed.port
except (TypeError, ValueError):
return None
if (
not parsed.scheme
or not parsed.hostname
or parsed.username is not None
or parsed.password is not None
or parsed.path not in ("", "/")
or parsed.query
or parsed.fragment
):
return None
scheme = parsed.scheme.lower()
if scheme not in {"http", "https", "tauri"}:
return None
if port is None:
if scheme == "http":
port = 80
elif scheme == "https":
port = 443
return scheme, parsed.hostname.lower(), port
def configured_allowed_origins() -> frozenset[tuple[str, str, int | None]]:
raw_port = os.environ.get("OMNIVOICE_UI_PORT", "3901")
try:
ui_port = int(raw_port)
except (TypeError, ValueError):
ui_port = 3901
values = os.environ.get(
"OMNIVOICE_ALLOWED_ORIGINS",
f"http://localhost:{ui_port},http://127.0.0.1:{ui_port},"
"tauri://localhost,http://tauri.localhost",
).split(",")
return frozenset(
origin
for value in values
if (origin := _origin_tuple(value.strip())) is not None
)
def _destination_origin(connection) -> tuple[str, str, int | None] | None:
scheme = effective_scheme(connection)
url = getattr(connection, "url", None)
netloc = getattr(url, "netloc", None)
if netloc:
return _origin_tuple(f"{scheme}://{netloc}")
scope = getattr(connection, "scope", None)
headers = getattr(connection, "headers", None) or {}
if not isinstance(scope, dict):
return None
host = headers.get("host", "") if hasattr(headers, "get") else ""
return _origin_tuple(f"{scheme}://{host}")
def origin_allowed(connection) -> bool:
headers = getattr(connection, "headers", None) or {}
origin_value = headers.get("origin", "") if hasattr(headers, "get") else ""
presented = _origin_tuple(origin_value)
if presented is None:
return False
return presented == _destination_origin(connection) or presented in configured_allowed_origins()
def cookie_csrf_allowed(connection, *, side_effectful_get: bool = False) -> bool:
headers = getattr(connection, "headers", None) or {}
marker = headers.get(CSRF_HEADER, "") if hasattr(headers, "get") else ""
if marker != CSRF_VALUE or not origin_allowed(connection):
return False
method = getattr(connection, "method", None)
if method is None:
scope = getattr(connection, "scope", None)
method = scope.get("method", "GET") if isinstance(scope, dict) else "GET"
method = str(method).upper()
if side_effectful_get or method in SAFE_HTTP_METHODS:
fetch_site = headers.get("sec-fetch-site", "") if hasattr(headers, "get") else ""
return fetch_site == "same-origin"
return True
+102
View File
@@ -176,6 +176,108 @@ _BASE_SCHEMA = """
created_at REAL
);
CREATE INDEX IF NOT EXISTS idx_pron_lang ON pronunciation_entries(language);
-- Remote GPU workers (docs/remote-workers.md). Opt-in: an install with no
-- remote workers never writes a row here and behaves exactly as before.
--
-- `public_key` is the worker's identity — a server-assigned id is a name,
-- not proof, so every reconnect is verified against this key. Revocation
-- is a persisted fact (not in-memory state) precisely so a restart of the
-- control plane cannot silently readmit a worker the user removed.
CREATE TABLE IF NOT EXISTS remote_workers (
id TEXT PRIMARY KEY,
name TEXT NOT NULL DEFAULT '',
key_id TEXT NOT NULL,
public_key BLOB NOT NULL,
enabled INTEGER NOT NULL DEFAULT 1,
revoked INTEGER NOT NULL DEFAULT 0,
revoked_at REAL,
priority INTEGER NOT NULL DEFAULT 50,
endpoint TEXT NOT NULL DEFAULT '',
host_json TEXT NOT NULL DEFAULT '{}',
capabilities_json TEXT NOT NULL DEFAULT '[]',
max_concurrent_tasks INTEGER NOT NULL DEFAULT 1,
-- Bumped on every successful (re)connect. Messages stamped with an
-- older epoch are from a session we have already replaced.
session_epoch INTEGER NOT NULL DEFAULT 0,
consent_granted_at REAL,
created_at REAL NOT NULL,
last_seen_at REAL
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_remote_workers_key ON remote_workers(key_id);
-- Single-use join tokens. Only the hash is stored: the plaintext exists
-- once, in the dialog that shows it.
CREATE TABLE IF NOT EXISTS remote_worker_enrollments (
token_id TEXT PRIMARY KEY,
secret_hash TEXT NOT NULL,
endpoint TEXT NOT NULL DEFAULT '',
cert_fingerprint TEXT NOT NULL DEFAULT '',
label TEXT NOT NULL DEFAULT '',
created_at REAL NOT NULL,
expires_at REAL NOT NULL,
used_at REAL,
used_by_worker TEXT
);
-- Tasks dispatched to remote workers. Unlike the local `jobs` table (whose
-- startup sweep marks anything in-flight as failed), these must SURVIVE a
-- control-plane restart: the desktop app quits while a remote GPU keeps
-- rendering, and the worker is the source of truth for what is still
-- running. Reconciliation on reconnect rebuilds live state from here.
CREATE TABLE IF NOT EXISTS remote_tasks (
id TEXT PRIMARY KEY,
-- Client-supplied; deduplicates client retries before the worker
-- protocol is involved at all.
idempotency_key TEXT,
operation TEXT NOT NULL,
engine TEXT NOT NULL DEFAULT '',
model_id TEXT NOT NULL DEFAULT '',
params_json TEXT NOT NULL DEFAULT '{}',
priority INTEGER NOT NULL DEFAULT 0,
state TEXT NOT NULL DEFAULT 'queued',
max_attempts INTEGER NOT NULL DEFAULT 3,
excluded_json TEXT NOT NULL DEFAULT '[]',
error_json TEXT,
-- Written BEFORE RESULT_ACK is sent. If the server dies between
-- receiving a result and acknowledging it, the worker redelivers and
-- this row is what makes the second delivery a no-op instead of a
-- silently lost multi-minute render.
result_ref TEXT,
result_json TEXT,
project_id TEXT,
created_at REAL NOT NULL,
updated_at REAL NOT NULL,
deadline_at REAL,
-- Deliberate additive-reconcile exception to the alembic rule: remote
-- task recovery must work in bundled installs where alembic may be
-- unavailable, and this nullable affinity column is additive-only.
pinned_worker_id TEXT,
finished_at REAL
);
CREATE INDEX IF NOT EXISTS idx_remote_tasks_state ON remote_tasks(state, priority, created_at);
CREATE UNIQUE INDEX IF NOT EXISTS idx_remote_tasks_idem ON remote_tasks(idempotency_key)
WHERE idempotency_key IS NOT NULL;
CREATE TABLE IF NOT EXISTS remote_task_attempts (
id TEXT PRIMARY KEY,
task_id TEXT NOT NULL,
worker_id TEXT NOT NULL,
session_epoch INTEGER NOT NULL DEFAULT 0,
attempt_number INTEGER NOT NULL DEFAULT 1,
state TEXT NOT NULL DEFAULT 'assigned',
progress REAL NOT NULL DEFAULT 0,
stage TEXT NOT NULL DEFAULT '',
error_json TEXT,
created_at REAL NOT NULL,
accepted_at REAL,
started_at REAL,
finished_at REAL,
lease_expires_at REAL,
grace_expires_at REAL
);
CREATE INDEX IF NOT EXISTS idx_remote_attempts_task ON remote_task_attempts(task_id);
CREATE INDEX IF NOT EXISTS idx_remote_attempts_worker ON remote_task_attempts(worker_id, state);
"""
# Only tables/columns this module is allowed to ALTER. Prevents SQL injection via
+100
View File
@@ -177,6 +177,23 @@ def gfx_for_hsa_override(value: str) -> str | None:
#: The ROCm kernel driver interface. Its absence, or its presence without
#: permission, are the two commonest reasons a ROCm host silently runs on CPU.
_KFD_DEVICE = "/dev/kfd"
_DXG_DEVICE = "/dev/dxg"
_DXG_RUNTIME_PATHS = (
"/usr/lib/libdxcore.so",
"/usr/lib/librocdxg.so",
"/usr/share/rocdxg/dids.conf",
)
def _rocm_requires_dxg_detection(version: object) -> bool:
"""Whether WSL's ROCDXG bridge still needs its explicit opt-in."""
try:
parts = str(version).split(".")
return (int(parts[0]), int(parts[1])) < (7, 13)
except (IndexError, TypeError, ValueError):
# Unknown versions get the conservative advice. The variable is
# harmless on newer runtimes and necessary on every older one.
return True
def why_no_gpu(torch) -> tuple[str, ...]:
@@ -230,6 +247,40 @@ def why_no_gpu(torch) -> tuple[str, ...]:
# /dev/kfd only exists on Linux; on any other platform its absence
# says nothing, so don't invent a reason.
if sys.platform.startswith("linux"):
if not os.path.exists(_KFD_DEVICE) and os.path.exists(_DXG_DEVICE):
if not os.access(_DXG_DEVICE, os.R_OK | os.W_OK):
return (
f"ROCm {hip} is installed and {_DXG_DEVICE} exists, "
"but this process cannot open it — pass "
"--device /dev/dxg to the WSL container",
)
dxg_detection = os.environ.get("HSA_ENABLE_DXG_DETECTION", "").strip()
if dxg_detection == "0":
return (
f"ROCm {hip} is installed and {_DXG_DEVICE} is reachable, "
"but HSA_ENABLE_DXG_DETECTION=0 explicitly disables the "
"WSL GPU bridge; remove it or set it to 1",
)
if _rocm_requires_dxg_detection(hip) and dxg_detection != "1":
return (
f"ROCm {hip} is installed and {_DXG_DEVICE} is "
"reachable, but this pre-7.13 runtime requires "
"HSA_ENABLE_DXG_DETECTION=1 inside WSL containers",
)
missing = [
path for path in _DXG_RUNTIME_PATHS if not os.path.exists(path)
]
if missing:
return (
f"ROCm {hip} can reach {_DXG_DEVICE}, but the WSL "
"ROCDXG runtime mounts are incomplete; missing: "
f"{', '.join(missing)}",
)
return (
f"ROCm {hip} and the WSL ROCDXG bridge are reachable, "
"but no GPU was enumerated — verify the AMD Windows "
"driver, librocdxg/ROCm compatibility, and host `rocminfo`",
)
if not os.path.exists(_KFD_DEVICE):
return (
f"ROCm {hip} is installed but {_KFD_DEVICE} is not "
@@ -374,6 +425,33 @@ class HostCaps:
probe_ok: bool = True
"""``False`` only when torch could not be imported (degraded CPU-only)."""
requested_family: str = "auto"
"""The user's compute-device override as requested — ``"auto"`` when none.
``family`` reflects what was actually honored: an override that names a
family this host doesn't have is noted and ignored, never obeyed blindly."""
#: Every value the compute-device override accepts. "auto" = today's
#: priority pick; "cpu" is always honorable (invariant: cpu is always
#: available); accelerator names are honored only when detected.
DEVICE_OVERRIDE_CHOICES: tuple[str, ...] = ("auto", "cuda", "rocm", "xpu", "mps", "cpu")
def requested_device_override() -> str:
"""The user's compute-device pick: ``OMNIVOICE_DEVICE`` env > the Settings
choice (``compute_device`` in prefs.json) > ``"auto"``. Env wins so
power-users can pin a device without the UI silently undoing it (same
resolution order as engine selection, #981). Unknown values normalize to
``"auto"`` the probe must never raise."""
try:
from core import prefs
raw = prefs.resolve("compute_device", env="OMNIVOICE_DEVICE", default="auto")
except Exception:
raw = os.environ.get("OMNIVOICE_DEVICE", "auto")
val = str(raw or "auto").strip().lower()
return val if val in DEVICE_OVERRIDE_CHOICES else "auto"
def _probe() -> HostCaps:
"""Run the probe once. Enumerates every failure branch from the spec's
@@ -386,6 +464,7 @@ def _probe() -> HostCaps:
available_families=("cpu",),
notes=("torch not importable; treating host as CPU-only",),
probe_ok=False,
requested_family=requested_device_override(),
)
notes: list[str] = []
@@ -507,6 +586,26 @@ def _probe() -> HostCaps:
# available_families: every detected accelerator + cpu, deduped, cpu last.
available: tuple[DeviceFamily, ...] = tuple(dict.fromkeys([*detected, "cpu"]))
# User override (Settings → Performance, or OMNIVOICE_DEVICE): honored
# only when the named family actually exists on this host — an override
# can steer, it cannot invent hardware. Applied here, at the single
# choke point, so routing, model loads (get_best_device delegates its
# family decision here), and every badge inherit it for free.
requested = requested_device_override()
if requested != "auto":
if requested in available:
if requested != family:
notes.append(
f"compute device pinned to '{requested}' by user override "
f"(auto would pick '{family}')"
)
family = requested # type: ignore[assignment]
else:
notes.append(
f"requested compute device '{requested}' is not available on "
f"this host (have: {', '.join(available)}) — using '{family}'"
)
return HostCaps(
family=family,
available_families=available,
@@ -515,6 +614,7 @@ def _probe() -> HostCaps:
driver=driver,
notes=tuple(notes),
probe_ok=True,
requested_family=requested,
)
+66 -2
View File
@@ -21,6 +21,7 @@ Check shape:
"""
from __future__ import annotations
import importlib
import os
import platform
import shutil
@@ -200,12 +201,12 @@ def _check_engines() -> dict:
return _check(
"engines", "TTS engines", FAIL,
f"{detail} - active engine '{active}' is unavailable: {reason}",
active_row.get("install_hint") or "Pick a different engine in Settings > Engines.",
active_row.get("install_hint") or "Pick a different engine in Model Catalogue > Engines.",
)
if not available:
return _check(
"engines", "TTS engines", FAIL, detail,
"No usable TTS engine. Install one from Settings > Engines.",
"No usable TTS engine. Install one from Model Catalogue > Engines.",
)
return _check("engines", "TTS engines", OK, detail)
@@ -367,10 +368,44 @@ def run_diagnostics(include_network: bool = True, deep: bool = False) -> dict:
counts = {OK: 0, WARN: 0, FAIL: 0}
for c in checks:
counts[c["status"]] += 1
engine_execution = []
for family in ("tts", "asr"):
active = "unknown"
try:
module = importlib.import_module(f"services.{family}_backend")
active = module.active_backend_id()
row = next((item for item in module.list_backends() if item.get("id") == active), None)
if row is not None:
engine_execution.append({
"family": family,
"engine_id": active,
**row["execution_evidence"],
})
except Exception: # noqa: BLE001 - evidence must not break diagnostics
# Preserve the other family's successful evidence and make this
# collection failure explicit without exposing exception text.
engine_execution.append({
"family": family,
"engine_id": active,
"implementation_variant": None,
"declared_device_families": [],
"evidence_state": "collection_failed",
"actual_execution_provider": None,
"actual_execution_device": None,
"gpu_name": None,
"gpu_architecture": None,
"precision_or_quantization": None,
"cpu_fallback_reason": None,
"cpu_fallback_stage": None,
"parent_memory_observable": None,
"runtime_versions": {},
})
return {
"app_version": APP_VERSION,
"platform": scrub_text(platform.platform()),
"checks": checks,
"engine_execution": engine_execution,
"summary": {
"ok": counts[FAIL] == 0,
"passed": counts[OK],
@@ -395,6 +430,35 @@ def format_text(report: dict) -> str:
lines.append(f"{tag[c['status']]} {c['label']}: {c['detail']}")
if c.get("hint"):
lines.append(f" hint: {c['hint']}")
if report.get("engine_execution"):
lines.append("")
lines.append("Engine execution evidence:")
for item in report["engine_execution"]:
if item.get("actual_execution_provider"):
provider = item["actual_execution_provider"]
elif item.get("evidence_state") == "subprocess_loaded_provider_unreported":
provider = "loaded child; provider not reported"
else:
provider = "not loaded"
precision = item.get("precision_or_quantization") or "unknown"
device = item.get("actual_execution_device") or "unknown"
gpu = item.get("gpu_name") or "none"
architecture = item.get("gpu_architecture") or "unknown"
fallback_stage = item.get("cpu_fallback_stage") or "none"
fallback_reason = item.get("cpu_fallback_reason") or "none"
versions = ",".join(
f"{name}={version}"
for name, version in sorted(item.get("runtime_versions", {}).items())
) or "none"
visible = "yes" if item.get("parent_memory_observable") else "no"
lines.append(
f" {item['family']}:{item['engine_id']} provider={provider}; "
f"device={device}; gpu={gpu}; architecture={architecture}; "
f"precision={precision}; fallback-stage={fallback_stage}; "
f"fallback-reason={fallback_reason}; runtimes={versions}; "
f"evidence-state={item.get('evidence_state', 'unknown')}; "
f"parent-memory-visible={visible}"
)
s = report["summary"]
lines.append("")
lines.append(
+34 -8
View File
@@ -23,9 +23,17 @@ logger = logging.getLogger("omnivoice.events")
_listeners: list[asyncio.Queue] = []
_lock = asyncio.Lock()
# The loop that serves /ws/events, captured on first use. Sync FastAPI
# endpoints (rename/delete profile, revoke consent) run in threadpool workers
# where `asyncio.get_running_loop()` raises, which used to silently drop their
# events — the UI then never refetched the voice list (#1158 class).
_serving_loop: asyncio.AbstractEventLoop | None = None
async def subscribe() -> asyncio.Queue:
"""Register a new listener. Returns a Queue that receives event dicts."""
global _serving_loop
_serving_loop = asyncio.get_running_loop()
q: asyncio.Queue = asyncio.Queue(maxsize=64)
async with _lock:
_listeners.append(q)
@@ -57,11 +65,29 @@ def emit(kind: str, payload: dict[str, Any] | None = None) -> None:
}
event_str = json.dumps(event)
try:
loop = asyncio.get_running_loop()
loop.create_task(_broadcast(event_str))
caller_loop = asyncio.get_running_loop()
except RuntimeError:
# No event loop running (unlikely in FastAPI context but safe)
caller_loop = None
target_loop = _serving_loop or caller_loop
if target_loop is None:
# No serving loop yet — nobody to notify; dropping is correct.
logger.debug("No event loop — event dropped: %s", kind)
return
try:
if caller_loop is target_loop:
target_loop.create_task(_broadcast(event_str))
else:
# Sync endpoints and async producers on a foreign loop must both
# hand off: the lock and listener queues belong to serving_loop.
target_loop.call_soon_threadsafe(_schedule_broadcast, event_str)
except RuntimeError:
# The serving loop closed between capture and use (app shutdown).
logger.debug("Event loop closed — event dropped: %s", kind)
def _schedule_broadcast(event_str: str) -> None:
"""Run `_broadcast` on the serving loop; called via call_soon_threadsafe."""
asyncio.get_running_loop().create_task(_broadcast(event_str))
async def _broadcast(event_str: str) -> None:
@@ -73,11 +99,11 @@ async def _broadcast(event_str: str) -> None:
q.put_nowait(event_str)
except asyncio.QueueFull:
# Slow consumer — drop oldest, then push. Not a race (#1163):
# every queue op runs on the single event loop, and there is
# no await between the QueueFull and this get_nowait/put_nowait
# pair — no consumer can interleave, so get_nowait cannot raise
# QueueEmpty here. emit() from a foreign thread drops the event
# before ever touching a queue (see the RuntimeError branch).
# every queue op runs on the single event loop (a foreign
# thread's emit() hands off via call_soon_threadsafe first),
# and there is no await between the QueueFull and this
# get_nowait/put_nowait pair — no consumer can interleave, so
# get_nowait cannot raise QueueEmpty here.
try:
q.get_nowait()
q.put_nowait(event_str)
+69 -2
View File
@@ -52,6 +52,35 @@ _REDACTED_VALUE = "***REDACTED***"
# One-line "what to do" per docs-taxonomy key. Keys mirror error_docs_map's
# taxonomy; the docs URL itself stays owned by error_docs_map.
_HINTS: dict[str, str] = {
"GPU_OOM": "Close other GPU-heavy apps or unload models, then retry. You can also choose CPU in Settings → Performance & Device or select a smaller TTS engine.",
"WORKER_AT_CAPACITY": "Wait for a running job on that worker to finish, or choose another available worker and retry.",
"MODEL_NOT_INSTALLED": "Install or enable this engine on the worker machine, then refresh its capabilities and retry.",
"MODEL_NOT_DOWNLOADED": "Open Models, install this model on the selected worker, then retry when the download completes.",
"INSUFFICIENT_MEMORY": "Choose a worker with more free GPU memory, unload another model there, or use a smaller model and retry.",
"OPERATION_UNSUPPORTED": "Choose a worker whose capability list includes this operation, or run the job locally.",
"ACCEPT_TIMEOUT": "Check that the worker is responsive and not overloaded, then reconnect it and retry.",
"MODEL_LOAD_TIMEOUT": "Check the worker's model download and load status, then retry after the model is ready.",
"EXECUTION_TIMEOUT": "Check the worker for a stalled engine or GPU error, restart that engine if needed, then retry.",
"PROGRESS_LEASE_EXPIRED": "Check the worker connection and engine log, reconnect or restart the worker, then retry the job.",
"RESULT_DELIVERY_TIMEOUT": "Check the connection and free disk space on both machines, then reconnect the worker and retry.",
"INPUT_FETCH_TIMEOUT": "Check the connection to the worker and retry; keep both machines awake until the reference file finishes transferring.",
"INPUT_FETCH_FAILED": "Check that the source file still exists and both machines are connected, then submit the job again.",
"RESULT_UPLOAD_FAILED": "Check the worker connection and free disk space on this machine, reconnect the worker, then retry.",
"WORKER_FAILED": "Open the selected worker's log for the underlying error, fix it there, then reconnect and retry.",
"SESSION_EXPIRED": "Reconnect the worker; if it cannot renew its session, remove it and enroll it again.",
"STALE_EPOCH": "Reconnect the worker so it receives the current session, then retry the job.",
"STALE_ATTEMPT": "Refresh the job state and retry only if the current attempt has not already completed elsewhere.",
"UPGRADE_REQUIRED": "Update VoiceStudio on the machine named in the error, then reconnect the worker.",
"WORKER_REVOKED": "Add the worker again from Settings → System → Remote workers to create a new trusted enrollment.",
"AUTH_FAILED": "Remove this worker, generate a new enrollment token, and add it again.",
"INVALID_TASK_PARAMS": "Review the job inputs, correct the invalid or missing value named in the error, and submit it again.",
"MODEL_REF_REJECTED": "Select a model from VoiceStudio's catalog on that worker instead of a path or custom model reference.",
"RESULT_TOO_LARGE": "Shorten or split the job so each result is smaller, then render the parts separately.",
"ARTIFACT_TOO_LARGE": "Shorten or split the job so each uploaded artifact is smaller, then render the parts separately.",
"OFFSET_MISMATCH": "Reconnect the worker and retry the upload from the byte count reported by the control plane.",
"SIZE_MISMATCH": "Reconnect the worker and retry the result upload; if it repeats, restart the worker before rerendering.",
"DIGEST_MISMATCH": "Retry the result upload; if it repeats, check the worker's disk and network for corruption, then rerender.",
"UPLOAD_INCOMPLETE": "Reconnect the worker and resume the result upload from the byte count reported by the control plane.",
"PKG_RESOURCES_MISSING": "Run `uv pip install --reinstall 'setuptools>=75,<80'` in the backend venv (a plain install is skipped when setuptools' metadata is present but its pkg_resources files were removed by antivirus). Restart after.",
"GATEKEEPER_QUARANTINE": "Clear the macOS quarantine flag (xattr -cr the app), then reopen.",
"APPIMAGE_WEBKIT_WHITESCREEN": "Launch with WEBKIT_DISABLE_DMABUF_RENDERER=1 set.",
@@ -65,7 +94,7 @@ _HINTS: dict[str, str] = {
# fail with "file not found" for exactly the users most likely to need it
# (greptile on #1377). tests/test_failure_classify.py pins these literals
# to the constraint file so they cannot drift when the pins bump.
"TRANSFORMERS_IMPORT": "Your transformers install is incomplete, or a package it loads models through (torchaudio, torchvision) is missing or mismatched with your torch — a torch/torchvision version mismatch fails with exactly this wording. Reinstall them together at the pinned versions (`uv pip install --python .venv --reinstall torch==2.8.0 torchaudio==2.8.0 torchvision==0.23.0 transformers` in the project folder), then restart the backend. If only transcription is affected, switching ASR to faster-whisper (Settings → Models) also works around it.",
"TRANSFORMERS_IMPORT": "Your transformers install is incomplete, or a package it loads models through (torchaudio, torchvision) is missing or mismatched with your torch — a torch/torchvision version mismatch fails with exactly this wording. Reinstall them together at the pinned versions (`uv pip install --python .venv --reinstall torch==2.8.0 torchaudio==2.8.0 torchvision==0.23.0 transformers` in the project folder), then restart the backend. If only transcription is affected, switching ASR to faster-whisper (Model Catalogue → Models) also works around it.",
"WINDOWS_APP_CONTROL_BLOCKED": "Windows refused to load a file VoiceStudio needs — an Application Control policy (Smart App Control, WDAC, or AppLocker) blocked it. On a personal PC: Windows Security → App & browser control → Smart App Control → Off (Windows only lets you turn it off once — re-enabling requires a Windows reset), then restart VoiceStudio. On a managed/work PC, ask IT to allow the VoiceStudio install folder.",
"WINDOWS_PAGING_FILE_TOO_SMALL": "Windows ran out of virtual memory while mapping the model into memory — its paging file is smaller than the model needs. This is not the same as your RAM being full, and closing other apps usually won't fix it: Windows has to be allowed to back the mapping. Set a bigger paging file — Settings → System → About → Advanced system settings → Performance → Settings → Advanced → Virtual memory → Change: untick \"Automatically manage\", pick your system drive, choose \"Custom size\" and set both Initial and Maximum to at least 32768 MB (more than the model's size), then OK and restart Windows. A smaller/quantized engine (OmniVoice GGUF, Supertonic-3) also avoids the large mapping entirely.",
"MEDIA_TOOL_MISSING": "VoiceStudio's media engine (ffmpeg/ffprobe) wasn't on the system path when a component went looking for it. Open Settings → Audio tools and use Download/Repair to fetch the bundled copy, then retry — a restart picks it up for everything. If you'd rather use a system install, install ffmpeg (macOS: `brew install ffmpeg`; Windows: `winget install Gyan.FFmpeg`; Linux: your package manager) and restart VoiceStudio, or point FFMPEG_PATH / OMNIVOICE_FFPROBE_PATH at the binaries in Settings.",
@@ -89,7 +118,7 @@ _HINTS: dict[str, str] = {
# told the reporter to reinstall transformers — advice that cannot work,
# because nothing is wrong with their install. Checked first so the cause
# wins over the symptom.
"MODEL_DOWNLOAD_INTERRUPTED": "A model download was cut off mid-request, and the component it was fetching then failed to load. Nothing is wrong with your install — reinstalling won't help, and the partial download is resumed rather than restarted. Just retry. If it keeps happening, check your connection (and any VPN, proxy or HF mirror setting); if only transcription is affected, switching ASR to faster-whisper in Settings → Models avoids the pipeline that downloads this component.",
"MODEL_DOWNLOAD_INTERRUPTED": "A model download was cut off mid-request, and the component it was fetching then failed to load. Nothing is wrong with your install — reinstalling won't help, and the partial download is resumed rather than restarted. Just retry. If it keeps happening, check your connection (and any VPN, proxy or HF mirror setting); if only transcription is affected, switching ASR to faster-whisper in Model Catalogue → Models avoids the pipeline that downloads this component.",
"BROKEN_VENV": "The Python backend environment was moved or damaged. VoiceStudio rebuilds it automatically on the next launch; if it keeps failing, use Clean & Retry on the setup screen.",
"MODEL_CACHE_CORRUPT": "A model file is missing or damaged — a download that stopped part-way, a broken link to downloaded data, or a file changed on disk after it arrived (interrupted renames and antivirus interference both cause this). VoiceStudio repairs it automatically and retries the load once, re-downloading the damaged file where a resume would not have replaced it. If the error persists, quit VoiceStudio, delete the model's models--<org>--<name> folder inside the Hugging Face cache, and restart — the model re-downloads automatically.",
# HF_MIRROR_UNREACHABLE has a DYNAMIC hint (it names the configured mirror)
@@ -262,6 +291,9 @@ def append_hf_mirror_hint(text: str) -> str:
# must NOT be added: its bare "timed out" trigger would stamp a "video server"
# hint on a model-load timeout that leaks through the 500 handler.
_CONTEXT_FREE_HINT_CLASSES = frozenset({
# Device allocator signatures are specific enough to attach the shared
# recovery without exposing CUDA's process table or filesystem paths.
"GPU_OOM",
"SOCKS_PROXY_SUPPORT_MISSING",
"SSL_HANDSHAKE_FAILURE",
# Its trigger is an exact OpenSSL string, so it cannot be confused with
@@ -295,6 +327,38 @@ def append_hint(text: str) -> str:
return f"{text}{hint}" if hint else text
_GPU_OOM_SIGNATURES = (
"cuda out of memory",
"cuda error: out of memory",
"cuda_error_out_of_memory",
"mps backend out of memory",
"hip out of memory",
"out of memory on device",
)
def is_gpu_oom(error: BaseException | str) -> bool:
"""Recognize device OOMs through wrappers without importing torch."""
pending: list[BaseException] = [error] if isinstance(error, BaseException) else []
seen: set[int] = set()
while pending:
current = pending.pop()
if id(current) in seen:
continue
seen.add(id(current))
if type(current).__name__ == "OutOfMemoryError":
return True
if any(signature in str(current).lower() for signature in _GPU_OOM_SIGNATURES):
return True
if current.__cause__ is not None:
pending.append(current.__cause__)
if current.__context__ is not None:
pending.append(current.__context__)
if isinstance(error, str):
return any(signature in error.lower() for signature in _GPU_OOM_SIGNATURES)
return False
def classify(reason: str) -> str:
"""Map a failure reason to a docs-taxonomy key, or "" when unknown.
@@ -302,6 +366,8 @@ def classify(reason: str) -> str:
backend log / diagnostic names the same class the UI deeplink will use.
"""
low = (reason or "").lower()
if is_gpu_oom(low):
return "GPU_OOM"
if "pkg_resources" in low:
return "PKG_RESOURCES_MISSING"
if "quarantine" in low or "is damaged" in low or "gatekeeper" in low:
@@ -491,6 +557,7 @@ def classify(reason: str) -> str:
or "unable to download video" in low
or "remote end closed" in low
or "timed out" in low
or "the page needs to be reloaded" in low
):
return "VIDEO_DOWNLOAD_NETWORK"
# #1227: Windows Smart App Control / WDAC / AppLocker refused to load a
+38 -9
View File
@@ -3,10 +3,11 @@ isn't empty on initial launch. Runs once; skips silently if any
profiles already exist.
"""
import filecmp
import logging
import os
import shutil
import time
import logging
from core.db import get_db
from core.config import VOICES_DIR
@@ -23,16 +24,15 @@ DEMO_PROFILE_NAME = "VoiceStudio Demo Voice"
# Must match the actual spoken content of backend/assets/samples/demo_voice.wav.
# Regenerated by scripts/build_demos.sh — update both files in lockstep.
DEMO_REF_TEXT = (
"Hi, I'm the VoiceStudio demo voice. Everything you hear me say from now on "
"was synthesized on your own machine. No cloud, no account, just you and "
"the model."
"Hey. I'm the VoiceStudio demo voice. I was made right here, on your "
"machine: private, local, and ready whenever you are."
)
_DEMO_DESCRIPTION = (
"A neutral reference voice bundled with VoiceStudio. Clone it to hear how "
"the engine sounds on your machine, then replace it with your own "
"recording when you're ready."
"An original warm, low cinematic voice bundled with VoiceStudio. Clone it "
"to hear how the engine sounds on your machine, then replace it with your "
"own recording when you're ready."
)
@@ -43,8 +43,14 @@ def _backfill_demo_metadata(conn):
try:
conn.execute(
"UPDATE voice_profiles SET description=?, is_demo=1, ref_text=? "
"WHERE id=? AND (is_demo=0 OR description='' OR ref_text!=?)",
(_DEMO_DESCRIPTION, DEMO_REF_TEXT, DEMO_PROFILE_ID, DEMO_REF_TEXT),
"WHERE id=? AND (is_demo=0 OR description!=? OR ref_text!=?)",
(
_DEMO_DESCRIPTION,
DEMO_REF_TEXT,
DEMO_PROFILE_ID,
_DEMO_DESCRIPTION,
DEMO_REF_TEXT,
),
)
conn.commit()
except Exception as e:
@@ -52,11 +58,34 @@ def _backfill_demo_metadata(conn):
logger.debug("Demo backfill skipped: %s", e)
def _refresh_demo_audio(conn):
"""Keep the canonical demo profile in sync with the bundled render."""
try:
row = conn.execute(
"SELECT 1 FROM voice_profiles WHERE id=? AND is_demo=1",
(DEMO_PROFILE_ID,),
).fetchone()
if not row or not os.path.isfile(_DEMO_AUDIO):
return
os.makedirs(VOICES_DIR, exist_ok=True)
dest = os.path.join(VOICES_DIR, f"{DEMO_PROFILE_ID}.wav")
if not os.path.isfile(dest) or not filecmp.cmp(
_DEMO_AUDIO, dest, shallow=False
):
shutil.copy2(_DEMO_AUDIO, dest)
logger.info("Refreshed bundled demo voice audio")
except Exception as e:
# The demo must never make startup fail; a fresh seed below can still
# repair it once the schema and data directory are available.
logger.debug("Demo audio refresh skipped: %s", e)
def seed_sample_project():
"""Create the demo voice profile if no profiles exist yet."""
conn = get_db()
try:
_backfill_demo_metadata(conn)
_refresh_demo_audio(conn)
count = conn.execute("SELECT COUNT(*) FROM voice_profiles").fetchone()[0]
if count > 0:
return # Not first run — skip
+34
View File
@@ -0,0 +1,34 @@
"""Terminate a desktop-contained backend when its owning shell disappears."""
from __future__ import annotations
import os
import sys
import threading
from typing import BinaryIO, Callable
def _watch_parent_pipe(reader: BinaryIO, exit_process: Callable[[int], None]) -> None:
"""Block until the desktop-owned stdin pipe closes, then exit immediately."""
try:
while reader.read(1):
pass
except (OSError, ValueError):
# A broken or already-closed parent-owned pipe is equivalent to EOF.
pass
exit_process(0)
def arm_desktop_parent_watchdog() -> bool:
"""Use stdin EOF as an unforgeable parent-liveness signal for desktop runs."""
if os.environ.get("OMNIVOICE_DESKTOP_CONTAINED") != "1":
return False
reader = getattr(sys.stdin, "buffer", None)
if reader is None:
return False
threading.Thread(
target=_watch_parent_pipe,
args=(reader, os._exit),
name="desktop-parent-watchdog",
daemon=True,
).start()
return True
+43 -16
View File
@@ -7,6 +7,7 @@ only the unguessable capability token crosses loopback HTTP.
from __future__ import annotations
import json
import logging
import os
import re
import secrets
@@ -14,6 +15,8 @@ import stat
from core.config import DATA_DIR
logger = logging.getLogger("omnivoice.path_authorization")
_TOKEN_RE = re.compile(r"[0-9a-f]{64}\Z")
_KINDS = {
"models_dir",
@@ -40,25 +43,49 @@ def consume(token: str, expected_kind: str) -> str:
if expected_kind not in _KINDS or not _TOKEN_RE.fullmatch(token or ""):
raise PathAuthorizationError("Invalid or expired desktop authorization")
root = _AUTH_DIR
# Distinguish "the store exists but this token isn't in it" (expired /
# already consumed / never issued — normal, no server-side signal) from
# "the store doesn't exist at all" (the desktop app and this backend are
# very likely pointed at different data directories, e.g. a dev backend
# started without OMNIVOICE_DATA_DIR, or a stale custom data folder — see
# #1781). The client-facing message is byte-identical either way (never
# leak local filesystem paths, or even which case occurred, over HTTP —
# CWE-200); the mismatch case additionally gets a server log line so it's
# diagnosable instead of a silent 403. That log line is deliberately
# path-free too (CWE-532: per-user filesystem paths, e.g. a home
# directory username, are sensitive and don't belong in application
# logs) — it names the failure mode, not the directory.
try:
entries = os.scandir(root)
except FileNotFoundError as exc:
logger.warning(
"path authorization store does not exist; the desktop app and "
"this backend likely resolved different data directories "
"(see #1781)"
)
raise PathAuthorizationError("Invalid or expired desktop authorization") from exc
except OSError as exc:
raise PathAuthorizationError("Invalid or expired desktop authorization") from exc
candidate = None
try:
for entry in os.scandir(root):
if not _TOKEN_RE.fullmatch(entry.name.removesuffix(".json")):
continue
if not entry.is_file(follow_symlinks=False):
continue
try:
with open(entry.path, "r", encoding="utf-8") as handle:
probe = json.load(handle)
except (OSError, UnicodeError, json.JSONDecodeError):
continue # Ignore corrupt/stale capabilities; they authorize nothing.
if isinstance(probe, dict) and secrets.compare_digest(
str(probe.get("token", "")), token
):
candidate = entry.path
break
with entries:
for entry in entries:
if not _TOKEN_RE.fullmatch(entry.name.removesuffix(".json")):
continue
if not entry.is_file(follow_symlinks=False):
continue
try:
with open(entry.path, "r", encoding="utf-8") as handle:
probe = json.load(handle)
except (OSError, UnicodeError, json.JSONDecodeError):
continue # Ignore corrupt/stale capabilities; they authorize nothing.
if isinstance(probe, dict) and secrets.compare_digest(
str(probe.get("token", "")), token
):
candidate = entry.path
break
if candidate is None:
raise OSError("capability not found")
raise PathAuthorizationError("Invalid or expired desktop authorization")
claimed = os.path.join(root, f".consuming-{os.getpid()}-{secrets.token_hex(16)}")
os.replace(candidate, claimed)
except OSError as exc:
+21 -6
View File
@@ -10,8 +10,20 @@ from __future__ import annotations
import ntpath
import os
import re
from pathlib import Path
_WINDOWS_RESERVED_NAMES = frozenset({"CON", "PRN", "AUX", "NUL"}) | frozenset(
f"{prefix}{number}" for prefix in ("COM", "LPT") for number in range(1, 10)
)
# Both separator families, so a stored sub-path splits into the same components
# on every host. Windows accepts ``/`` as a real separator, so splitting on
# ``os.sep`` alone left ``"job/out.mp4"`` as a single component there while the
# identical value split cleanly on POSIX. POSIX input never reaches this with a
# backslash — it is rejected as a foreign separator before the split.
_PATH_SEPARATORS = re.compile(r"[\\/]")
class UnsafePath(ValueError):
"""Raised when a path crosses its allowed filesystem boundary."""
@@ -28,6 +40,10 @@ def safe_filename(value: object) -> str:
or os.path.isabs(name)
or ntpath.isabs(name)
or ntpath.basename(name) != name
or name.endswith((" ", "."))
or re.search(r"[\x00-\x1f]", name)
or name.split(".", 1)[0].upper() in _WINDOWS_RESERVED_NAMES
or len(name.encode("utf-8")) > 240
):
raise UnsafePath("expected a bare filename")
return name
@@ -43,11 +59,10 @@ def resolve_within(root: os.PathLike[str] | str, value: os.PathLike[str] | str)
raw = os.fspath(value) if value is not None else ""
if not isinstance(raw, str) or not raw:
raise UnsafePath("path is empty")
# Treat both separator families as structural on every host. Otherwise a
# Windows traversal string is an innocent-looking filename when validated
# on Linux (and can become dangerous after persisted data is moved).
if os.sep != "\\" and ("\\" in raw or bool(ntpath.splitdrive(raw)[0])):
raise UnsafePath("path uses a foreign separator or drive")
# Treat both separator families as structural on every host while still
# rejecting Windows drive paths before rebuilding relative components.
if os.sep != "\\" and bool(ntpath.splitdrive(raw)[0]):
raise UnsafePath("path uses a drive")
root_path = Path(root).expanduser().resolve(strict=False)
root_text = str(root_path)
if os.path.isabs(raw):
@@ -60,7 +75,7 @@ def resolve_within(root: os.PathLike[str] | str, value: os.PathLike[str] | str)
# containment proof explicit to static analysis, this rejects empty,
# dot, parent, drive, and separator-bearing components before Path sees
# any persisted/request-derived string.
parts = raw.split(os.sep)
parts = _PATH_SEPARATORS.split(raw)
clean_parts: list[str] = []
for part in parts:
clean = os.path.basename(part)
+50
View File
@@ -91,3 +91,53 @@ def resolve(key: str, *, env: Optional[str] = None, default: Any = None) -> Any:
if v:
return v
return get(key, default)
# ── external-override detection (#1787 review fix) ──────────────────────────
# restore_env() below uses os.environ.setdefault(), so a value already present
# in the process's environment (shell profile, `.env`, Docker `-e`, systemd
# unit, …) silently wins over anything saved in prefs.json — the setdefault
# call is a no-op. That is the right behavior (env stays authoritative,
# matching resolve()'s contract above), but a Settings control that persists a
# value to prefs.json must not tell the user it "took effect after restart"
# when an external source will keep shadowing it on every future restart too.
#
# _EXTERNALLY_PROVIDED records, once per process start, every bare key that
# was ALREADY present in os.environ the moment restore_env() ran — i.e.
# before our own setdefault() calls could have put it there, and before any
# value our Settings UI ever wrote (Settings only ever writes prefs.json plus
# the CURRENT process's os.environ; it never touches a shell profile or `.env`
# file). Snapshotting unconditionally — not only for keys prefs.json already
# has an entry for — means is_env_shadowed() also answers correctly for a key
# a user is about to save for the FIRST time. Membership is stable for the
# life of the process (nothing removes an inherited env var), and since a
# plain restart re-inherits the same shell / container environment, it is
# also a reliable predictor for the NEXT start: if the external source is
# still exporting the key, the next restart will be shadowed again the same
# way.
_EXTERNALLY_PROVIDED: frozenset[str] = frozenset()
def restore_env(data: dict) -> None:
"""Restore ``env.*`` prefs into ``os.environ`` (startup only).
Called once from main.py's ``env_prefs`` step, before any user code reads
``os.environ``. Snapshots which keys were already externally provided
see :func:`is_env_shadowed` then applies every saved ``env.*`` pref via
``setdefault`` (never overriding an explicitly-set env var).
"""
global _EXTERNALLY_PROVIDED
_EXTERNALLY_PROVIDED = frozenset(os.environ.keys())
for k, v in data.items():
if not k.startswith("env.") or not v:
continue
os.environ.setdefault(k[len("env."):], str(v))
def is_env_shadowed(key: str) -> bool:
"""Whether *key* was already present in the environment from a source
other than our own prefs restore, as of the last time :func:`restore_env`
ran. If prefs.json holds (or will hold) a saved value for *key*, that
value is being silently ignored and will be again on the next restart
unless the external source is removed."""
return key in _EXTERNALLY_PROVIDED
+51
View File
@@ -28,6 +28,15 @@ def stream_failure(code: str) -> dict[str, object]:
"detail": "Generation capacity is busy. Try again shortly.",
"retryable": True,
},
"generation_timeout": {
"code": "generation_timeout",
"detail": (
"Generation exceeded the compute-time limit. The backend is "
"still running; try a shorter passage, or raise the "
"compute-time budget in Settings → Performance & Device."
),
"retryable": True,
},
"invalid_request": {
"code": "invalid_request",
"detail": "The generation request could not be processed.",
@@ -65,6 +74,48 @@ def stream_failure(code: str) -> dict[str, object]:
return dict(failures.get(code, failures["generation_failed"]))
def stream_generation_failure(error: BaseException | object) -> dict[str, object]:
"""``generation_failed`` stream metadata, enriched with the actual cause.
The bare "Generation failed. Check the selected engine and try again." is
the floor for an *unrecognized* failure. When the private exception DOES
classify to a known failure class a corrupt model cache, an unreachable
Hugging Face mirror, a missing ffmpeg/ffprobe, a Windows paging-file limit,
a SOCKS/TLS proxy problem, the stable VoiceStudio-owned remediation for
that class is appended so the user can self-diagnose instead of guessing
which engine or which failure. This is the same enrichment the classic
(non-streaming) ``/generate`` 500 already gets via
:func:`public_exception_response`; the in-band streaming error frame
replaces the global 500 handler for a streaming request and used to bypass
it entirely (#1607).
Only VoiceStudio-owned constants are copied never a substring of
``error`` (Constitution I). Never raises: a diagnosis failure must not
replace the failure being diagnosed.
"""
payload = stream_failure("generation_failed")
try:
enriched = public_exception_response(error, fallback=str(payload["detail"]))
except Exception:
return payload
hint = enriched.get("hint")
if hint:
payload["detail"] = enriched["detail"]
payload["hint"] = hint
topic = enriched.get("docs_topic")
if topic:
payload["docs_topic"] = topic
try:
from core import error_docs_map
url = error_docs_map.ERROR_DOCS.get(topic, "")
except Exception:
url = ""
if url:
payload["docs_url"] = url
return payload
def public_failure(
logger: logging.Logger,
log_message: str,
+125
View File
@@ -0,0 +1,125 @@
"""Startup progress ledger — what the backend is doing before it can serve.
Why this exists: the project's #1 lifetime failure class is "can't reach the
local backend", and a large slice of it was never a dead backend at all —
just one that couldn't say "I'm starting, currently loading PyTorch" because
nothing listened until every heavy import and migration finished. main.py now
binds the socket early and defers the heavy work; this module is the shared
state the early `/health` + `/startup/progress` endpoints report from while
that work runs.
Thread-safety: the deferred init runs Phase A in an executor thread while the
event loop serves probes, so every mutation and snapshot takes the lock.
"""
from __future__ import annotations
import threading
import time
# Execution order matters only for display; the ledger records whatever order
# steps actually begin in. Keep ids stable — the desktop shell field-sniffs
# them and tests pin them.
STEPS: "dict[str, str]" = {
"env_prefs": "Restoring settings…",
"native_preload": "Preparing GPU libraries…",
"ml_imports": "Loading ML runtime (PyTorch)…",
"api_routes": "Loading API routes…",
"db_migrate": "Preparing database…",
"services_start": "Starting background services…",
}
_lock = threading.Lock()
_t0 = time.monotonic()
_current: "str | None" = None
_done: "list[tuple[str, float]]" = [] # (step_id, seconds it took)
_started_at: float = 0.0
_ready = False
_error: "dict | None" = None
def begin_step(step_id: str) -> None:
global _current, _started_at
with _lock:
_finish_current_locked()
_current = step_id
_started_at = time.monotonic()
def _finish_current_locked() -> None:
global _current
if _current is not None:
_done.append((_current, round(time.monotonic() - _started_at, 2)))
_current = None
def mark_ready() -> None:
global _ready
with _lock:
_finish_current_locked()
_ready = True
def fail(message: str) -> None:
"""Record a startup failure against the step that was running."""
global _error
with _lock:
_error = {"step": _current, "message": str(message)[:500]}
def is_ready() -> bool:
with _lock:
return _ready
def current_step() -> "tuple[str | None, str | None]":
"""(step_id, human label) of the active step, or (None, None)."""
with _lock:
if _current is None:
return None, None
return _current, STEPS.get(_current, _current)
def snapshot() -> dict:
"""The `/startup/progress` body. Always safe to call, never raises."""
with _lock:
if _error is not None:
status = "failed"
elif _ready:
status = "ready"
else:
status = "starting"
states = {sid: "pending" for sid in STEPS}
for sid, _t in _done:
states[sid] = "done"
if _current is not None:
states[_current] = "active"
if _error is not None and _error.get("step"):
states[_error["step"]] = "failed"
durations = dict(_done)
return {
"status": status,
"step": _current,
"label": STEPS.get(_current, _current) if _current else None,
"steps": [
{
"id": sid,
"label": label,
"state": states.get(sid, "pending"),
**({"t": durations[sid]} if sid in durations else {}),
}
for sid, label in STEPS.items()
],
"elapsed_s": round(time.monotonic() - _t0, 2),
"error": _error,
}
def _reset_for_tests() -> None:
global _current, _ready, _error, _started_at
with _lock:
_current = None
_done.clear()
_ready = False
_error = None
_started_at = 0.0
+1 -1
View File
@@ -144,6 +144,6 @@ def _drop_invalid_path_keys() -> None:
logger.warning(
"%s from the saved env file points at an unusable path (%s) — "
"ignoring it for this run and falling back to the default "
"location. Fix or clear it in Settings → Models.", key, val,
"location. Fix or clear it in Model Catalogue → Models.", key, val,
)
os.environ.pop(key, None)
+1 -1
View File
@@ -24,7 +24,7 @@ from pathlib import Path
# tests/test_app_version.py::test_all_version_files_in_lockstep and bumped by
# release.yml's version-bump job, so it stays equal to
# pyproject/tauri.conf/Cargo/package.json.
_FALLBACK_VERSION = "0.4.2"
_FALLBACK_VERSION = "0.5.2"
def _fallback_version() -> str:
+19 -3
View File
@@ -85,11 +85,27 @@ def _get_model():
global _model
if _model is None:
from faster_whisper import WhisperModel
name = os.environ.get("ASR_MODEL_FW", "large-v3")
# Same weights as in-process faster-whisper: ASR_MODEL_FASTER selects
# for BOTH variants, ASR_MODEL_FW stays as a sidecar-only override.
# Before this, the sidecar read only ASR_MODEL_FW while the download
# preflight read ASR_MODEL_FASTER — set one and the other variant (or
# the preflight) quietly used a different model.
name = (
os.environ.get("ASR_MODEL_FW")
or os.environ.get("ASR_MODEL_FASTER")
or "large-v3"
)
try:
import torch
device = "cuda" if torch.cuda.is_available() else "cpu"
# The probe honors the user compute-device override and the
# ROCm/CT2 incompatibility (#1529) — the child must agree with
# the parent's device decision, not re-derive its own.
from core.device_caps import detect_host_caps
device = "cuda" if detect_host_caps().family == "cuda" else "cpu"
except Exception:
# Fail SAFE: guessing "cuda" from torch here would bypass a cpu
# override and hand CTranslate2 HIP-flavoured cuda on ROCm
# (#1529). CPU always works; say why in the sidecar log.
print("asr-sidecar: device probe failed — using cpu", file=sys.stderr, flush=True)
device = "cpu"
# Degrade fp16 → int8 rather than crash on GPUs without efficient fp16
# (older Maxwell/Pascal, GTX 16xx, CTranslate2/cuDNN mismatch) (#551).
+18
View File
@@ -28,6 +28,7 @@ packages. The parent only ever spawns it as a subprocess.
from __future__ import annotations
import logging
import math
import os
import re
from typing import TYPE_CHECKING
@@ -164,6 +165,23 @@ class IndexTTS2Backend(SubprocessBackend):
from engines.indextts.bootstrap import resolve_indextts_venv
return resolve_indextts_venv()
@property
def recv_timeout_s(self) -> float:
# IndexTTS was the only sidecar left on the 60s class default while
# pockettts and omnivoice-subprocess both raised theirs. infer() is one
# blocking upstream call, so a long passage legitimately outruns 60s and
# the parent's watchdog killed a healthy synthesis (#1611). main.py also
# heartbeats during infer(), which is what actually proves liveness —
# this deadline is the ceiling for a sidecar that has gone genuinely
# silent. OMNIVOICE_INDEXTTS_RECV_TIMEOUT_S tunes it.
try:
v = float(os.environ.get("OMNIVOICE_INDEXTTS_RECV_TIMEOUT_S", "900"))
except (ValueError, TypeError):
return 900.0
if not math.isfinite(v): # reject inf/nan so the deadline can't be disabled
return 900.0
return max(30.0, v)
@classmethod
def sidecar_script(cls):
from engines.indextts.bootstrap import INDEXTTS_SIDECAR_SCRIPT
+87 -7
View File
@@ -63,11 +63,13 @@ Restrictions:
from __future__ import annotations
import base64
import contextlib
import json
import os
import struct
import sys
import tempfile
import threading
import traceback
@@ -117,11 +119,59 @@ EMOTION_KWARGS_ALLOWLIST = frozenset({
# ── wire protocol ─────────────────────────────────────────────────────────
#: Seconds between keep-alive progress frames during a long blocking call.
_HEARTBEAT_S = 5.0
#: Serializes _send across threads (the heartbeat below + the main loop) so
#: concurrent length+body writes can't interleave and corrupt the framing.
_send_lock = threading.Lock()
def _send(stream, obj: dict) -> None:
body = json.dumps(obj, separators=(",", ":")).encode("utf-8")
stream.write(struct.pack("!I", len(body)))
stream.write(body)
stream.flush()
with _send_lock:
stream.write(struct.pack("!I", len(body)))
stream.write(body)
stream.flush()
@contextlib.contextmanager
def _heartbeat(stdout, stage: str):
"""Emit a progress frame every ~5s for the duration of the block.
IndexTTS spends the whole of a cold load and the whole of ``infer()``
inside one blocking upstream call, saying nothing on the wire. The parent
reads that silence two ways, and BOTH kill a perfectly healthy synthesis
of a long passage (#1611):
* ``SubprocessBackend.generate`` re-arms its recv watchdog on every
frame, so with no frames it hard-kills the sidecar at recv_timeout_s;
* each frame also reports activity to the GPU pool's execution clock
(#1367), so with no frames the outer generate budget expires and
blames the hardware.
Raising the deadline alone therefore does not fix long-text generation
the sidecar has to prove it is alive. Percent climbs 1..99 because the
upstream call exposes no real progress; it is a liveness signal, not a
measurement.
"""
stop = threading.Event()
def _beat() -> None:
pct = 1
while not stop.wait(_HEARTBEAT_S):
pct = min(pct + 1, 99)
try:
_send(stdout, {"op": "progress", "stage": stage, "percent": pct})
except Exception:
return # pipe gone — the main loop will surface it
hb = threading.Thread(target=_beat, name=f"indextts-{stage}-heartbeat", daemon=True)
hb.start()
try:
yield
finally:
stop.set()
hb.join(timeout=_HEARTBEAT_S + 1)
def _recv(stream):
@@ -160,14 +210,40 @@ def _torch_bf16_supported() -> bool:
return False
#: Model-config filenames to look for, most-preferred first, per version.
#: IndexTeam/IndexTTS-2.5 ships ``config.yaml``; VoiceStudio used to demand
#: ``config_v2_5.yaml``, a name that exists in no upstream revision, so the
#: install failed until the user hand-renamed the file (#1611). Both names are
#: accepted now — the hand-renamed installs must keep working untouched — and
#: the renamed one wins, because a user who created it did so deliberately.
_CFG_NAMES = {
"2.5": ("config_v2_5.yaml", "config.yaml"),
"2": ("config.yaml",),
}
def _resolve_cfg_path(model_dir: str, *, version: str) -> str:
"""First accepted config that exists in ``model_dir``.
Falls back to the last candidate when none exist, so the failure surfaces
as upstream's own "no such file" naming a real expected path rather than
a name no upstream release has ever shipped.
"""
names = _CFG_NAMES.get(version, _CFG_NAMES["2"])
for name in names:
candidate = os.path.join(model_dir, name)
if os.path.isfile(candidate):
return candidate
return os.path.join(model_dir, names[-1])
def _model_init_kwargs(
repo_dir: str, *, version: str, reduced_precision: bool,
) -> dict:
"""Build version-specific constructor arguments for IndexTTS 2.5 or 2."""
model_dir = os.path.join(repo_dir, "checkpoints")
cfg_name = "config_v2_5.yaml" if version == "2.5" else "config.yaml"
kwargs = {
"cfg_path": os.path.join(model_dir, cfg_name),
"cfg_path": _resolve_cfg_path(model_dir, version=version),
"model_dir": model_dir,
"use_cuda_kernel": False,
"use_deepspeed": False,
@@ -216,7 +292,8 @@ def _load_model(stdout) -> object:
model_kw = _model_init_kwargs(
repo_dir, version=_model_version, reduced_precision=reduced_precision,
)
_model = IndexTTS2(**model_kw)
with _heartbeat(stdout, "loading_model"):
_model = IndexTTS2(**model_kw)
_send(stdout, {"op": "progress", "stage": "loading_model", "percent": 100})
return _model
@@ -276,7 +353,10 @@ def _handle_synthesize(msg: dict, stdout) -> None:
tmp_path = tmp.name
try:
infer_kw["output_path"] = tmp_path
model.infer(**infer_kw)
# A long passage keeps infer() busy for minutes with nothing on the
# wire; without this the parent kills the sidecar mid-synthesis (#1611).
with _heartbeat(stdout, "synthesizing"):
model.infer(**infer_kw)
pcm_b64, sr, n_samples = _wav_to_pcm_b64(tmp_path)
finally:
try:

Some files were not shown because too many files have changed in this diff Show More