Compare commits

..
43 Commits
Author SHA1 Message Date
254f071b45 fix(docker): ship alembic.ini in the image, add an image-level HEALTHCHECK (#1080)
Migrations in Docker fell back to the additive-column self-heal because
alembic.ini was never copied; the real migration chain now runs. The
HEALTHCHECK covers plain docker-run (compose files keep their own), with
a start period sized for first-boot schema creation. Docs example tag
freshened.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 22:17:57 +05:30
d1522cbba0 release: freeze v0.3.17 — version bump, lockfiles, changelog (#1079)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 22:05:58 +05:30
d3e88c0f3e fix(scripts): desktop-fresh kill guard referenced the wrong dry-run flag (#1078)
The kill-before-wipe block used DRY_RUN; the script's flag is dryRun —
any run with a live instance crashed with ReferenceError before wiping.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 21:40:51 +05:30
fb0508f4f6 fix(shell): deep health probe before attaching to a running backend; scripts kill before wiping (#1077)
A backend that keeps running while its install is deleted or replaced
underneath it still answers /health and /system/info from memory — the
launcher's version check passed and the UI attached to a process that
500s every DB-touching route (raw errors without CORS headers, so the
webview reports access-control failures). The attach path now requires a
DB-touching probe (/profiles) to return an actual 200 status line, and
replaces the squatter otherwise — the status line is parsed explicitly
because the raw HTTP helper previously returned 500 bodies as Ok.
desktop-prod/desktop-fresh now terminate our own running processes
(bundle, dev binary, app-scoped port-3900 listener) before wiping, which
is how the zombie was produced.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 19:32:58 +05:30
855a72038b docs(changelog): unreleased entry for #1074 (#1075)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 15:47:51 +05:30
e0a7b2202d fix(ui): default UI scale is 100% — native size out of the box (#1074)
New installs rendered at 130% zoom, which read as oversized on typical
displays. Fresh sessions now start at 100%; anyone who already picked a
scale keeps it (uiScale is persisted and wins over the default).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 15:34:56 +05:30
f7be62207e docs(changelog): unreleased entries for #1071 and #1072 (#1073)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 13:11:53 +05:30
7762bc48bd feat(settings): Engines & Models go tabbed and compact — strict two-line rows, aligned columns (#1072)
Settings → Engines collapses from three stacked full matrices (where one
engine row could sprawl to 5+ lines and fill a viewport) into ONE section
with a TTS / ASR / LLM tab strip (the matrix's Radix Segmented — roving
tabindex + arrow keys, active engine named in each tab caption). The single
mounted matrix still issues exactly one GET /engines + one GET /model/loaded
per Settings open; switching tabs re-slices the fetched payload with no
refetch. openSettingsTab('engines') deep-linking is unchanged (nothing in
the app targets a specific family — audited).

Every engine row is now strictly two lines inside a fixed h-16 shell:
line 1 = EngineMark + truncated display name (full name via title, never
wraps) + active/in-memory badges; line 2 = engine id, cloning chip,
curated-model picker, one-line truncated hints (full text via title).
Header and rows share one grid template
(minmax(0,1fr) 108px 176px 92px 232px) so the STATUS / GPU COMPAT /
ISOLATION / ACTIONS columns align on every row; actions sit right-aligned
and vertically centered across both lines. Below 880px the three meta cells
re-place onto the row's second line (same DOM nodes) instead of forcing a
horizontal scroll. Unavailable-row details (reason, install hint, last
error, setup snippet) move out of the row into an aria-expanded expansion
panel that opens BELOW it, so sibling rows never lose alignment. The
previously hardcoded column labels are i18n'd (engines.col*).

Settings → Models already has its natural grouping as the role filter
strip (All/TTS/ASR/… with counts) + search over one list, so no tabs were
invented there; its rows get the same compactness treatment — 5px vertical
padding + 52px min-height two-line rows (virtualizer estimate updated),
with title attributes carrying the full label/repo text past the ellipsis.

Tests: EnginesTab suite rewritten for the tabbed layout (tab strip renders,
switching families doesn't refetch, single /engines + /model/loaded probes,
Use-on-ASR flows through the tab); matrix suite updated for the expansion
panel (reason/last-error/setup-snippet live behind the Why-unavailable
toggle) and extended with layout regressions: fixed-height two-line shell,
name truncation + title, header/rows sharing identical grid tracks,
panel open/close as a sibling below the row. 1139 frontend tests pass;
typecheck:ci, oxlint, oxfmt and vite build are clean.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 12:58:27 +05:30
5562aa16a7 feat(setup): media tools become invisible — bundled by default, controllable in Settings (#1071)
Most users should never learn what ffmpeg is. The Setup Wizard's SYSTEM
PREFLIGHT stops listing FFmpeg / FFprobe / yt-dlp as user-installed
requirements ("brew install ffmpeg…"): they are internal dependencies the
app provisions for itself. Genuine user facts (OS, RAM, disk, GPU,
network, Python) are untouched.

Backend
- New services/media_tools.py: per-tool status {version, path, origin:
  sidecar|bundled|system|custom}; background acquisition of a pinned,
  SHA-256-verified static ffmpeg+ffprobe build (immutable-commit fetch
  from the same upstream the static-ffmpeg pip package uses — that
  package itself was audited and rejected: mutable raw/main URL, no
  checksums, writes into site-packages); binaries are `-version`-probed
  via the existing _binary_runs before being trusted, installed under
  DATA_DIR (update-surviving, frozen-build-safe), zero new Python deps.
- ffmpeg_utils resolution chain gains the acquired-bundled tier — and
  ffprobe finally has a bundled tier at all (imageio-ffmpeg ships none),
  closing the source-install gap.
- New /media-tools router (loopback-gated, same contract as
  /system/set-env): status, acquire, {tool}/custom-path | use-system |
  restore, ytdlp/update | restore. Overrides persist via the existing
  env.FFMPEG_PATH / env.FFPROBE_PATH prefs convention — one store, no
  competing controls.
- yt-dlp updates: audited in-venv pip/uv upgrade and rejected (venv is
  uv-managed with no pip; yt-dlp is a locked dep, so the updater's
  --inexact drift sync would revert it). Instead the newest wheel —
  verified against PyPI's own sha256 — lands in a DATA_DIR overlay
  prepended to sys.path at startup: survives app updates, works in
  frozen builds, and "Restore tested version" is just deleting the
  overlay. Gallery now runs yt-dlp via `python -m yt_dlp` (module, not
  PATH) so the CLI can never be a user-install task either.
- /setup/preflight drops the three tool rows, carries a media_tools
  verdict, and self-heals: kicks the bundled download in the background
  when no tier resolves (never re-fires after a failure — the wizard's
  card owns Retry). diagnose + the ffmpeg-missing notification now point
  at Settings → Audio tools instead of package managers.

Frontend
- Wizard: new MediaEngineCard — renders NOTHING when the engine is ready,
  a one-line progress while acquiring, and only on failure an actionable
  card (Retry / Use a system copy / Choose file…).
- Settings → Audio tools (new category, System group): FFmpeg + FFprobe
  rows with version, path, origin badge, Use system copy / Choose file… /
  Restore bundled, header-level "Update bundled build"; yt-dlp row with
  Update + Restore tested version (+ restart affordance). Package-manager
  commands appear only as copyable prose, never executed.
- The FFmpeg-path override moved out of Settings → Network (pointer row
  deep-links to Audio tools; no second writer of env.FFMPEG_PATH).
  Notifications gain a settings-tab action type.
- All strings i18n (en + defaultValue), a11y labels on every control.

Tests: 29 new backend (origin classification, checksum/size/probe
rejection, override persistence, overlay update/restore, router gating +
route-shadowing) + preflight contract tests (tool rows gone, verdict
present, auto-acquire fires once); 14 new frontend (wizard hide/progress/
failure-card, Audio tools rows/badges/actions). Route snapshot
regenerated. Docs (macos/linux install, troubleshooting §7b) describe the
new reality in the same commit.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 12:22:49 +05:30
34c8a33628 fix(scripts): desktop-prod builds clean, desktop-fresh emulates a brand-new machine (#1070)
desktop-prod fixes:
- `tauri build --debug` used to produce every bundle and THEN exit 1 at the
  updater-artifact signing step (no TAURI_SIGNING_PRIVATE_KEY on dev
  machines); the script papered over it with a blanket "non-fatal bundle
  error" grep that also swallowed real bundling failures. Local emulation
  builds now pass `--config '{"bundle":{"createUpdaterArtifacts":false}}'`
  and only build the bundle the script launches (--bundles app / appimage,
  --no-bundle on Windows), so the build exits 0. Any nonzero exit now FAILS
  the script — the sole tolerated case is a specifically-detected
  linuxdeploy/FUSE failure on Linux when the raw debug binary was produced.
- The HF cache wipe ran `rm -rf ~/.cache/huggingface` on macOS/Linux — the
  SHARED global cache (backend/core/config.py only relocates it on Windows),
  deleting models unrelated to OmniVoice. Non-app-scoped cache paths are now
  kept with a "models will be reused" notice; FRESH_NUKE_HF=1 opts in.
- Honest clean marks (removed ✓ / already-clean ○ instead of ✗ for success),
  `open -n` always (plain `open` focused a stale running instance instead of
  launching the freshly built one), stale-AppImage removal on Linux.

New `bun desktop-fresh` (+ desktop-fresh:run), macOS-only with explicit
refusal elsewhere: true new-user emulation.
- Blank slate: everything desktop-prod cleans PLUS the traces that survive a
  reinstall + data wipe — ~/Library/WebKit (webview localStorage), Caches,
  HTTPStorages*, Preferences plist (+ defaults delete), Saved Application
  State. Per-path found/removed/absent status with sizes; --dry-run prints
  the full plan without touching anything.
- Dev-machine camouflage: launches by direct exec of the bundle's Mach-O
  (which inherits env — `open` hands off to launchd and drops it) with PATH
  stripped of /opt/homebrew/{bin,sbin} + /usr/local/bin and HF_TOKEN /
  HUGGING_FACE_HUB_TOKEN / HF_HOME / HF_HUB_CACHE / HF_ENDPOINT /
  OMNIVOICE_* unset, and prints a banner of what is hidden.

Shared pure helpers live in scripts/desktop-common.mjs, covered by 9 node
tests (tests/frontend/desktopScripts.test.mjs): every cleanable path is
app-scoped and under $HOME, the PATH/env sanitizers strip exactly the
intended entries, and the build args carry the updater-artifacts-off config.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 11:53:47 +05:30
17e2bb5ed5 docs(changelog): unreleased entry for #1067 (#1068)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 10:03:04 +05:30
fbac0de817 fix(dub): interrupted dub sessions no longer trap the app in an eternal spinner on relaunch (#1067)
The omni_ui session persisted dubStep verbatim, including in-flight values
(uploading/transcribing/generating/stopping). Quitting or crashing mid-dub
froze that step into localStorage, and every relaunch restored a wait on
work that died with the process — a blank Dub pane with an eternal spinner
that even reinstalling couldn't clear (the webview's localStorage survives).
Restores now clamp to settled states: editing when the session has segments,
idle otherwise; unknown/corrupt values are treated as transient.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 09:50:13 +05:30
2e71cc3744 docs(changelog): unreleased entries for #1058-#1064 (#1065)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 04:35:07 +05:30
e816a2c24e fix(settings): make provider/token panels honest — real Test now, gated probes, MCP bindings i18n + confirm (#1064)
HuggingFace token (ApiKeysPanel):
- "Test now" actually re-runs whoami: GET /api/settings/hf-token/state gains
  ?fresh=1 which drops the resolver's 300s validation cache (the invalidate
  hook existed but was never wired to any endpoint), so a fixed network or
  rotated token no longer shows a stale verdict for up to 5 minutes. Plain
  panel mounts keep the cache.
- Initial load renders a "Checking token sources…" placeholder instead of
  flashing a false amber "not set" for all three sources.
- Source rows are now a valid ARIA list (the old role="table" had rows with
  no cells, hiding the status from screen readers).
- Enter in the token input respects the in-flight guard the Save button
  already had (no duplicate POSTs).

LLM Providers:
- Test / Fetch models abort when the implicit save fails, instead of probing
  the previously-stored config and pairing a green "Test ok" badge with a
  save error.
- A failed initial load now offers a Retry button instead of dead-ending
  until the panel remounts.

LLM Skills: the per-skill provider Select carries an accessible name
("Provider for <skill>") instead of announcing as an unlabeled combobox.

MCP voice bindings:
- All user-facing strings go through i18n (the panel was the only Settings
  surface with hardcoded English throughout).
- First-run guidance moved out of per-row hints (which never rendered with
  zero bindings and duplicated per row) into the section header + an
  InfoHint that links to docs/mcp.md; an empty state invites the first add.
- Delete asks for confirmation via the shared askConfirm, disables the row's
  button while in flight, and re-syncs the list even when the DELETE fails
  (a 404 row no longer lingers on screen).
- The add row exposes the optional label the API already accepted (the row
  title rendered b.label without any way to set it); default_engine stays
  MCP-side-only and is documented as such.
- First component test file for the panel (load/empty/add/delete/error/a11y).

Tests: backend fail-before/pass-after for the fresh=1 cache bust; new
frontend coverage for every behavioral change above.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 04:21:47 +05:30
fd083749c6 fix(settings): network & privacy panels — clearable proxy after reload, HF-mirror panel never vanishes, guarded remote-backend save, honest privacy claims (#1063)
Settings → Network / Models / Sharing / Privacy / OpenAPI fixes:

- NetworkTab: a proxy persisted in a previous session can now be cleared —
  the Clear button and "Set" badge derive from the backend-persisted value
  (sysInfo.proxy_url), not only from a save in the current session. Proxy row
  copy now matches its real semantics ("Applies now" badge; desc/toast no
  longer claim a restart is needed or leak yt-dlp jargon — reworded in all
  21 locales). FFmpeg path placeholder is platform-appropriate instead of
  Windows-only on every OS.
- HFMirrorPanel: the panel no longer disappears when the initial GET fails —
  the section shell always renders, with a loading state and an error +
  Retry affordance. Saving now toasts, the active preset is marked
  (aria-pressed), and the custom-URL row is labelled "Custom mirror URL"
  instead of raw HF_ENDPOINT jargon (env var moved to the row note).
- RemoteBackendPanel: full i18n (was 100% hardcoded English); Save & reload
  now validates the URL (http/https, parseable) and asks for confirmation
  before saving a URL that hasn't passed a connection test — a typo'd base
  no longer bricks every API call after reload. Dropped the contradictory
  "Restart required" badge (saving reloads the app itself; description says
  so). docs/remote-gpu.md updated to match (docs-sync).
- PrivacyTab: the "Network calls" row no longer shows the green "Offline
  translator" assurance when the backend is down or reports 'unknown' —
  green is reserved for confirmed-offline providers (nllb/argos/
  libretranslate), everything unconfirmed shows a neutral "Unknown" badge.
  The online-translator warning now deep-links to Translation settings.
- OpenApiPanel: a failed clipboard copy toasts an error instead of silence.
- a11y: all five text inputs across these panels now carry accessible names
  (aria-label), previously announced only by their vanishing placeholders.

Tests: new colocated suites for NetworkTab, HFMirrorPanel,
RemoteBackendPanel, PrivacyTab; OpenApiPanel suite extended with copy
success/failure. Frontend suite 140 files / 1061 tests green; i18n parity
probes green (new keys en-only with defaultValue, reworded keys updated in
every locale).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 04:08:41 +05:30
b603b9f78d fix(settings): factory reset covers all prefs, guarded log clearing, temp reclaim, full Performance i18n (#1061)
Settings system-group cleanup — every fix keeps existing behavior contracts
and adds a fail-before/pass-after regression test:

- Factory reset now does what it promises: clears every locally-persisted
  preference via a single registry (utils/prefKeys.js) instead of only the
  zustand blob — nav-rail side, capture live-typing, stories speed, logs
  footer state, last settings category, dismissed tips, donate prompts, and
  the legacy omni_ui blob included. User data and connection state
  (omni_transcriptions, ov_backend_url, ov_api_key) are explicitly preserved,
  and prefKeys.test.js scans the source tree so any future localStorage key
  must be categorized or CI fails. The failure toast now carries the actual
  error message.
- Disk-usage "Clear logs" is confirm-gated with the same wording as
  Settings → Logs — it truncates the crash log (the bug-report artifact), so
  it can no longer be a single stray click.
- Temporary files got a reclaim action: a confirmed "Clear temp files"
  button backed by POST /api/settings/storage/temp/clear, which deletes only
  the omnivoice* entries in the OS temp dir (symlinks unlinked, never
  followed) and invalidates the cached report.
- Performance panel goes through i18n end to end (title, row, note, hint,
  errors, aria-label) — it was the last fully hardcoded panel; the
  non-Windows subtitle now reads "Windows only — not needed on this
  platform" instead of "not applicable".
- History retention: GET failures now surface an alert and hold Save until
  a load succeeds (404 from older backends stays silent), Enter saves, the
  dead !res.ok branch is gone, and the bespoke button is the shared Button.
- Logs tab: "Open folder" reveals the log file, "Copy visible log" copies
  the tail, the viewer autoscrolls to the newest lines, and the scroll box
  is keyboard-focusable (role=log) with a labelled source switcher.
- Storage paths: the app-data row is labelled "App data stored at" (it was
  borrowing the Privacy tab's "Uploads stored at"), and all three path rows
  gained Open folder.
- i18n stragglers routed through t(): storage load/open/clear fallbacks,
  the backend-status badge, and the frontend log buffer label.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 03:55:08 +05:30
2f3e40db24 fix(settings): dictation & pronunciation panels — error states, language-scoped preview, i18n, a11y, dictionary import/export (#1060)
Settings → Dictation / Pronunciation / remote-ASR panel fixes:

- RefinementPanel no longer vanishes when the initial GET fails (backend
  down/restarting): the section shell always renders, with the error, a
  Retry button, and a loading line — matching its sibling panels.
- RefinementPanel offers the "Open LLM Providers" deep-link as soon as
  llm_ready is false, not only after the first refinement failure.
- Pronunciation test preview gains a preview-language selector and sends
  it to POST /pronunciation/test, so language-scoped entries finally show
  up in the preview instead of a misleading "No entries match"; a hint
  explains that Global previews skip language-scoped entries.
- Preview requests are debounced and sequence-guarded (a slow stale
  response can never overwrite a newer one), failures surface as a
  "preview unavailable" note instead of silently blanking, and the
  preview re-runs after add/toggle/delete/import so it never goes stale.
- Dictionary backup & restore: Export JSON / Import JSON buttons wired to
  the existing GET /pronunciation/export and POST /pronunciation/import
  endpoints (import prompts replace-vs-merge when entries exist).
- a11y: each entry's enable switch is named after its term ("Enable
  GIF"), all add-form/test inputs and selects carry aria-labels, and the
  cramped language field gets a short placeholder with the long
  explanation moved to the row hint.
- RefinementPanel + AecPanel converted to i18n (`dictation.*` keys; the
  refine-failure helper now returns a key instead of hardcoded English),
  per the all-UI-strings-through-i18n convention; "experimental" is now
  sentence-cased via its key.
- Copy: empty state says "Add one below" (the form is below the list, in
  every locale) and the test row is titled "Test a sentence" instead of
  its ellipsized placeholder.
- AsrOpenAICompatPanel: Save is disabled until a field actually differs
  from the server values and shows a "Saved" confirmation after saving —
  URL/model-only edits are no longer silently ambiguous.
- Enter submits the pronunciation add form; Add is disabled while the
  term is blank.

New pronunciation locale keys are translated in all 21 locales (keeping
that namespace fully covered); the new `dictation.*` namespace is en-only
with fallback, matching the `models.asrOpenAICompat*` precedent. No
backend changes — /pronunciation/test already accepted `language`.

Tests: RefinementPanel + AsrOpenAICompatPanel component tests added,
PronunciationPanel tests extended (language-scoped preview, stale-response
guard, preview error, re-run after add, Enter-to-add, per-entry switch
names, export/import round-trip), refine-note test updated for i18n keys.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 03:42:19 +05:30
6978f90f16 fix(settings): correct About links, wire RTL, and close core Settings UX gaps (#1059)
- About linked the wrong project: "OmniVoice on GitHub" and "Model card"
  opened k2-fsa/OmniVoice. The GitHub button now derives from a single
  REPO_URL constant in utils/bugReport.js (issue/search URLs derive from it
  too), and the Model card button is gone — a multi-engine app has no single
  model card. A test pins the button to the constant so links can't drift.
- Picking Arabic now actually flips the UI to RTL: the languageChanged
  handler sets document dir + lang from i18n.dir(), covering any future RTL
  locale as well.
- A Settings search that matches nothing now shows "No settings match" with
  a Clear action instead of a silently blank sidebar (and an option-less
  nav <select> in the narrow layout).
- Hotkey recording no longer swallows modifier-less presses in silence — it
  shows inline "add a modifier" feedback, the Record button becomes a Cancel
  toggle while listening, window blur cancels the global key listener, and
  the row copy states the modifier requirement.
- About no longer dead-ends on fixable problems: "HF token set: no" and
  failing self-checks deep-link into the owning Settings category via
  openSettingsTab.
- Sidebar search now also matches translated setting-row titles
  (keywordKeys), so localized users can find categories by localized names;
  English keywords keep working everywhere.
- Network gets its missing restart flag (the FFmpeg-path row is a
  restart-bound env write); a lockstep test keeps RestartBadge usage and
  category flags in sync.
- The theme-dot and font-tile radiogroups implement the real WAI-ARIA radio
  pattern: roving tabindex plus arrow-key movement with focus following
  selection.
- Copy cleanup: sentence-case "UI scale" / "Commercial license"; the review
  segmented control moves off the orphaned engines.review_* keys to
  settings.review_mode_on/off (renamed across all 21 locales) with plainer
  English labels.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 02:59:46 +05:30
17ae952810 feat(settings): Models & Engines pages — engine identity marks, capability badges, upgrade hints, filter, residency (#1058)
The engine list gains a scannable identity mark per engine (EngineMark),
capability badges (cloning, device routing with reasons, sidecar isolation),
and surfaces available-but-has-advice hints that list_backends previously
dropped (new additive hint field; the ready-with-advice convention). The
model store gains a filter, disk context near downloads, in-memory residency
indicators with safe unload, copyable setup snippets, and actionable
empty/error states. Registry additions are additive only (hint,
supports_cloning with the property-descriptor guard).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 02:21:21 +05:30
55c40be307 release: freeze v0.3.16 — version bump, lockfiles, changelog (#1057)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 00:56:18 +05:30
7f59d5f8fe fix(models): self-heal HF-cache snapshots with broken file links, retry load once (#1056)
A first-run breaker: all blobs download fine, but the snapshots/<rev>/
entries are dangling symlinks (0 KB) — os.path.isfile() is False on a
dangling link, so transformers reports the weights missing even though
the bytes are on disk, and the existing resume repair can't fix it.

New services/hf_cache_repair.py deletes exactly the broken snapshot
entries (dangling symlinks + zero-byte weight/config stand-ins; never
blobs, never resolving entries) and restores them via snapshot_download,
verifying afterwards — if the restore recreates broken links (hub's
memoized symlink probe passing while real links come out broken), it
forces hub into copy-mode and repairs once more with real files.
model_manager retries the load exactly once per repo per process
(rung 0 of the cache-recovery ladder); dead-end errors now name the
exact models--<org>--<name> folder to delete. failure.py classifies the
class as MODEL_CACHE_CORRUPT so the user-facing error and auto bug
report explain the automatic repair.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 00:35:10 +05:30
d3ec4ed371 fix(tts): voxcpm2 — version floor, reference-clip prep, trailing-silence guard (#1055)
Three hardenings of the voxcpm2 engine path, all backward-compatible and
platform-identical:

- Version floor: every install hint now says pip install "voxcpm>=2.0.3"
  (2.0.3 fixed an Apple-Silicon/MPS audio-quality bug). Floor only — an
  already-installed older version stays available and working; it just
  surfaces an actionable upgrade hint in the is_available reason and a
  load-time warning.
- Reference-clip prep: the voxcpm package no longer trims reference audio
  itself, so raw user clips reached the model unconditioned. The clone path
  now trims leading/trailing near-silence (-50 dBFS floor, 50 ms edge pad)
  and caps the reference at 30 s. Fail-open (any prep problem falls back to
  the raw clip) and a strict no-op for short clean clips.
- Trailing-silence guard: generated output is trimmed to the last voiced
  sample + ~0.3 s natural tail via the new audio_dsp.trim_trailing_silence.
  Silence-trim only, no content analysis; a no-op on outputs without a
  silent tail and on all-silent (dead) renders.

22 new fake-module tests in tests/test_voxcpm2_guardrails.py; existing
engine/hint tests strengthened to guard the floor.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 00:21:54 +05:30
a99f1fdff9 feat(tts): normalization covers the OpenAI-compat API, streaming, and batch paths (#1054)
The engine-agnostic text-normalization pre-pass now runs at the three
remaining text→engine choke points, applied exactly once per request:
/v1/audio/speech (req.language), /ws/tts (whole text, before the sentence
chunker fans it out), and the batch queue's per-segment _gen (target
language) — matching the /generate, dub, and audiobook wiring. Route-level
tests pin exactly-once (spy) + toggle-off-raw for each path.

Also fixes a pre-existing /ws/tts bug the new test exposed: any request
omitting emo_alpha hit a KeyError and got an error frame instead of audio.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 00:05:04 +05:30
236c727cd4 docs(changelog): unreleased entries for #1048-#1052 (#1053)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 19:06:44 +05:30
efc99be337 feat(studio): generation takes — star, replay, and restore past takes; capped history retention (#1052)
Every generate already recorded a generation_history row; now that history is
usable: a takes rail in the workspace history lists recent takes with star/
unstar, replay, and one-click restore as the active output. Alembic migration
0009 adds the starred column (the startup schema self-heal covers pre-
migration DBs), a retention cap (setting, default 200) prunes the oldest
UNstarred rows — starred takes are never pruned — and history WAVs are only
deleted when no other row references them.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 18:54:14 +05:30
5ce9d0e51d feat(dub): predict segment fit before synthesis — tight/impossible badges + opt-in shorter rewrites (#1051)
New pure planning layer (services/duration_planner.py) runs after translation,
before TTS: estimates each translated line's natural speech duration (self-
calibrating from the job's already-synthesized segments, static per-language
rates as cold-start fallback) and classifies it fits/tight/impossible against
slot + capped gap borrow, with thresholds derived from fit_planner's own caps
so "impossible" means "would be trimmed". Verdicts ride the /dub/translate
response and badge the segment table; an opt-in (default OFF) LLM pass attaches
one-click shorter-rewrite suggestions for impossible lines. Never blocks
generation — informs before GPU time is burned.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 18:40:16 +05:30
7dbb95fa15 feat(dub): LLM translations keep terms consistent and sound spoken — auto-glossary brief + reflect pass (#1050)
One up-front LLM pass over the full transcript extracts a theme summary +
terminology map, merges it under the user's manual glossary (user entries
always win), caches it on the dub job per target language (job_data blob, no
schema change), and injects the brief into every per-segment prompt. A new
reflect pass then critiques each segment's direct translation for wordiness /
stiff register and rewrites it as natural spoken dialogue — any failure or
divergence silently keeps the direct translation. Both stages have Dub-tab
toggles (default ON for the LLM engine, persisted; MT engines unaffected),
with i18n strings across all 21 locales and docs updated.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 18:26:07 +05:30
60b29b4006 feat(tts): numbers, times, and abbreviations are spoken correctly in every engine (#1049)
New conservative, idempotent pre-TTS normalization pass
(services/text_normalization.py): strips zero-width/control junk, caps
pathological repeat runs, expands digits/times/ordinals/currency via
num2words (29 locales) and per-language abbreviation maps (EN/DE/ES/FR).
Wired once at each text-to-engine choke point — /generate, dub segments
(+ preview), and longform chapters — BEFORE the pronunciation dictionary
so user respellings stay the final say. Pref-gated
(text_normalization_enabled, default ON) with OMNIVOICE_TEXT_NORMALIZATION
env override; num2words promoted to a direct dependency.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 18:13:08 +05:30
04d6d0cb7a feat(longform): segment-level render cache — edit one sentence, re-render one segment (#1048)
Adds a content-addressed segment cache (segment_cache_key + SegmentCache,
cache_dir/segments/) under the existing chapter cache: a changed chapter now
reuses every untouched span's WAV and synthesizes only the edited/missing
ones, and an interrupted chapter render resumes from the segments that already
finished (each persists the moment it renders). The chapter key derivation is
unchanged so on-disk caches from released versions keep hitting, a fully-
unchanged chapter never touches segment files, and prune_cache_dir now walks
both layers so one byte cap bounds the whole cache. Chapter SSE events gain
additive segments/cached_segments counts.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 17:40:15 +05:30
2695ef97ae docs(research): adjacent-projects notes — voicebox, RTVC, VoxCPM, ebook2audiobook, VideoLingo (#1047)
* docs(research): adjacent-projects read — RTVC, VoxCPM upstream, ebook2audiobook, VideoLingo

Owner-requested comparative research tied to the current maturity map:
voxcpm2 upstream sync items (>=2.0.3 MPS fix, ref-trim removal in 2.0.1,
trailing-audio guard), audiobook per-sentence cache playbook, dub
translation reflect-loop + glossary, RTVC migration positioning.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(research): add voicebox (jamiepine) — the direct competitor read

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 15:56:53 +05:30
800207ddb5 fix(watermark): bound AudioSeal memory — long audio embeds/detects in 30s chunks (#1045) (#1046)
A multi-minute generation pushed the whole waveform through the AudioSeal
generator in one call; its activation memory grows linearly with length, and
a reporter's 16 GB Windows box failed a single ~2.2 GB CPU allocation mid-
generate (DefaultCPUAllocator: not enough memory). Embedding and detection
now slice audio into ~30 s chunks (sub-second tails fold into the previous
chunk), so peak memory is flat regardless of audio length. Detection keeps
the best-confidence chunk, which also stops whole-file averaging from
diluting spliced audio.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 15:26:40 +05:30
2d073cfaec fix(clone): ⊕ Insert popover opens below the script input instead of climbing out of the viewport (#1043)
The popover was hard-anchored bottom-[60px] — always growing upward
from the textarea. ScriptPanel's only mount (CloneDesignTab) puts that
input at the very top of the panel, so the tag list (max-h 280px,
including the CMU phoneme chips visible in the owner's screenshot)
extended past the viewport top, unreachable and unscrollable. Anchored
top-[calc(100%+6px)] instead: below the input, where the panel's
topmost placement guarantees room in its one mount.

Regression test locks the placement (top-anchored, bottom-[60px]
banned).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 07:11:14 +05:30
b1e83658f0 feat(ui): global audio mini-player — waveform, seek, and time for every playback that had none (#1042)
playBlobAudio-path audio (generate auto-play, profile/segment previews,
story lines, gallery voices, Projects renders) played through a bare
Audio()/AudioContext with no on-screen player; the #1032 stop pill was a
stop-only band-aid with a fixed-overlay overlap quirk at 1440x900.

- playback.js: claimTrackedPlayback extends the single-playback manager
  with label + seek/pause/resume transport and a timeupdate-driven track
  snapshot (currentTime/duration/paused/peaks); claimPlayback stays as
  the thin wrapper, single-playback invariant unchanged.
- media.js: every playBlobAudio path registers tracked - element paths
  get real seek/timeupdate, the Tauri Web Audio path gets offset-based
  seek + suspend/resume pause, and peaks are computed once from the
  blob/decoded buffer already in hand (never refetched). onDone(reason)
  lets callers chain (stories) or reset card state (gallery).
- GlobalAudioPlayer.jsx: persistent bottom bar (only for source
  'output' — exact pill exclusion semantics) with peaks canvas,
  click/drag/keyboard seek, play/pause, elapsed/total, label, stop.
- Layout: the bar is a real grid row (row 3) above the LogsFooter,
  mirroring the footer's in-flow fix — content physically ends at its
  top edge, so the pill's overlay-overlap class cannot recur; fixed
  overlays anchored above the footer also clear --audio-dock-height.
  Verified headless (Chromium 1440x900 + 1000x700, isolated vite, all
  :3900 traffic intercepted): bar meets footer edge-to-edge, clears the
  nav rail, seek/pause/stop drive the owner callbacks.
- Callers pass labels: "Generated audio" (useTTS/first-sound), profile
  name / segment text (useProfiles), story line (StoriesEditor), voice
  name (VoiceGallery/CommunityZone/ImportsZone), render title
  (Projects). VoiceGallery drops its bespoke copy of the Tauri playback
  detour; StoriesEditor line previews now actually play under WebKit
  (blob: media URLs never worked there) and are stoppable mid-chain.
- PlaybackStopPill.jsx + its test deleted; intent migrated into
  GlobalAudioPlayer.test.jsx (appears on output/hidden when idle/stop
  works/excluded sources) plus transport coverage; playback.test.js
  covers the tracked API; playBlobAudioTracked.test.js covers the
  media wiring incl. onDone reasons; logsFooterInFlow.test.js now
  guards both bars' grid rows and the overlay anchor calc.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 06:07:44 +05:30
29b6f30f2b release: freeze v0.3.15 — version bump, lockfiles, changelog (#1041)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 04:59:59 +05:30
72e979f1d9 fix(tts): model-load time stops eating the generate timeout budget (#1039)
* fix(tts): model-load time stops eating the generate timeout budget (#1033, #1037)

The generate guard (OMNIVOICE_GENERATE_TIMEOUT_S, 300s) wrapped the
adapter's lazy _ensure_loaded() — weight download included — together
with the synthesis. A cold first request burned the whole window on
the download and died with the VRAM-guidance 503; #1014's T4
verification measured it (0% GPU util for the full 300s), and #1033 +
#1037 match the signature.

New public TTSBackend.ensure_ready() (dispatches to the adapter's
_ensure_loaded when present) runs FIRST under the model-load budget
(OMNIVOICE_MODEL_LOAD_TIMEOUT, 1200s) in both /generate's adapter path
and /v1/audio/speech — the same load/generate split get_model()
already gave the native engine. Warm engines no-op. A load exceeding
its own budget 503s with load-specific text pointing at Settings →
Models, never the misleading 'too heavy for compute' guidance.

Tests: end-to-end class test (load slower than a tiny generate budget
but inside the load budget → succeeds; fail-before verified), the
stalled-load error path, and the base-hook dispatch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* changelog entry for the load-budget split (#1033, #1037)

* catch the builtin TimeoutError base — reload-proof class identity (CI-only miss)

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 04:35:17 +05:30
6c6207a132 docs: link the community Colab notebook (#1038) (#1040)
@shakib30 built and tested a working Colab notebook for the project
and offered it upstream. Linking it from the README (community-
maintained, credited) makes the no-local-GPU path discoverable without
taking on notebook maintenance in-repo.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 04:19:54 +05:30
0effe485f4 fix(studio): restore Clear History, stoppable auto-play preview, + cached ref transcripts (#1032) (#1036)
Three-part fix for the v0.3.5-comparison report:

1. Perf: since v0.3.6 (#308), a clone reference without a stored
   transcript triggered a FULL ASR model load + transcribe on every
   /generate — get_active_asr_backend() builds a fresh whisper backend
   per call. Measured live: 92.7s wall vs 14.9s of actual TTS. Now the
   first auto-transcript is persisted onto the (unlocked, clone-kind)
   profile row, and transcribe_reference caches results by audio
   content hash (bounded LRU, no model/VRAM held), so the cost is paid
   once per clip, not per request. User-typed transcripts are never
   overwritten; locked/design profiles are excluded from the persist.

2. Clear History: the workspace UX overhaul (#374) moved history into
   the right-side WorkspaceHistory panels and dropped the old Sidebar's
   clear-all control (the Sidebar is now hidden in every mode). Both
   the Voice and Dub panels get a scoped Clear History button wired to
   the existing DELETE /history and /dub/history endpoints, with the
   same confirm dialog the Sidebar used.

3. Auto-play: the finished-render playback (playBlobAudio) has no
   on-screen player and the only stop lived in the Voice ActionBar's
   CTA morph — unstoppable from the Dub workspace, profile pages, or
   after navigating away. A global PlaybackStopPill now appears for any
   'output' playback on every page. The existing Settings → Appearance
   "Auto-play preview" pref (#667) now also gates the generate path,
   as its label always promised (default ON — no behavior change).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 18:43:33 +05:30
32407bc781 fix(api): /v1/audio/speech honors num_step + guidance_scale instead of silently dropping them (#1014) (#1035)
A contributor's measured Tesla T4 verification (PR #1014) caught that
POST /v1/audio/speech accepted num_step/guidance_scale in the JSON
body with a 200 OK and discarded both (pydantic's default
extra=ignore) — API callers could never reach the model's documented
quality preset (num_step=32) through the OpenAI-compatible surface,
while the native /generate exposes both as form fields.

Both are now declared as validated optional extensions (num_step 1-128,
guidance_scale 0-20) and passed through to the engine's generate()
kwargs — omitted means absent (engines that don't accept the kwargs
never see a stray None), exactly like the existing duration/seed
extensions.

Tests: passthrough reaches the engine kwargs; omitted stays absent.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 17:49:25 +05:30
df870e9ed3 docs(agents): add verified Tesla T4 (16GB) inference notes (#1014)
* docs: add AGENTS.md with verified Tesla T4 (16GB) inference notes

Documents two things found while verifying inference on a real T4:
1. Cold-cache first /v1/audio/speech call can hit the 300s
   OMNIVOICE_GENERATE_TIMEOUT_S because the checkpoint download happens
   inside that budget — workaround via existing POST /models/install or
   raising the timeout, no code change needed.
2. The OpenAI-compatible endpoint silently ignores num_step/guidance_scale
   (schema doesn't declare them) — use native /generate for those.

Also documents the T4 acceleration checklist (dtype/attention/int8/CUDA
graphs) and measured VRAM (peak 2.05GB). No code changes.

* fix(docs): make /models/install workaround command actually executable

Addresses Greptile review: the instruction omitted the required
repo_id body field (InstallModelRequest rejects an empty body).

* fix(docs): correct port in /models/install example (3900, not 8000)

The app serves on port 3900 (confirmed: /health returns 200 there,
connection refused on 8000). Verified the exact corrected curl command
returns 200 {"status":"install_started",...}.

* move T4 notes to docs/hardware-notes-tesla-t4.md — AGENTS.md is the auto-loaded agent-instructions filename

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 17:30:05 +05:30
stronghamjjiandstronghamjji 323892b36c fix(clone): bound the ref-text re-transcribe like every other ASR dispatch (#730) (#1031)
The re-transcribe added for the (ref_audio, ref_text) mismatch fix is
dispatched with a bare run_in_executor(_gpu_pool, ...), and refine_ref_text
calls asr_backend.transcribe() directly. Its try/except catches a raised
error but not a *hang* — a wedged whisperx/CTranslate2 transcribe (#730)
holds the GPU-pool worker forever. On a <=10 GB card the pool is 1 worker,
so that starves every later GPU job into the misleading "can't reach the
local backend", and there's no ping on the await so the EventSource drops.

Route both refine dispatches (per-speaker and per-segment) through the same
run_transcribe_guarded the rest of dub_core.py already uses (the chunk loop
and the whole-file "Dub" transcribe). On timeout it resets the pool and
raises ASRTimeoutError; keep the original clones, matching refine_ref_text's
own "failure is a strict no-op" fallback.

Adds a repro test: refine_ref_texts dispatched raw is unbounded on a hang;
through the guard it times out and falls back to the original ref_text.

Co-authored-by: stronghamjji <289942360+stronghamjji@users.noreply.github.com>
2026-07-09 17:10:23 +05:30
36ee06c7cc feat(skills): installable Agent Skills — npx skills add debpalash/omnivoice-studio (#1034)
Two skills in the standard skills/<name>/SKILL.md layout (vercel-labs/
skills CLI; listed on skills.sh via install telemetry):

- omnivoice — teaches any agent (Claude Code, Cursor, Codex, …) to
  speak and transcribe through the user's LOCAL install via the
  OpenAI-compatible API at localhost:3900: health preflight, TTS with
  cloned-voice-profile discovery via /v1/audio/voices, STT with
  srt/vtt subtitle formats, and the local-first rule (never silently
  fall back to a cloud API).
- oss-maintainer — the maintainer methodology this repo is actually
  run with, distilled from real sessions: absorbed-or-declined queue
  discipline, check-the-PR-queue-before-implementing, root-cause →
  fix-the-class → regression-test, structural merge gates with
  flaky-vs-real judgment, the release protocol, and
  thank-contributors-specifically.

Every endpoint/flag in the omnivoice skill verified against
backend/api/routers/openai_compat.py and the README's API section.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 17:06:14 +05:30
9bfd8f9a13 fix(update): app updates stop uninstalling user-added engines — drift sync goes --inexact (#1029) (#1030)
Every app update whose uv.lock changed ran `uv sync --frozen` to
reconcile the venv (#307 drift path) — and uv sync's exact mode
UNINSTALLS every package not in the lockfile. That silently deleted
user-pip-installed optional engines (voxcpm, kittentts — packages the
app's own Settings → Engines hints tell users to install into this
venv) on every single update. Reported as "VoxCPM2 is automatically
uninstalled after updating Studio."

Fix: the routine drift sync now carries --inexact — locked deps are
still installed/upgraded exactly per the lockfile, but extras the user
added on purpose are left alone. Deliberate asymmetry: the venv-REPAIR
sync stays exact, because repair runs when the venv is broken and a
user-installed extra is a plausible cause — healing must restore the
known-good locked state. First-run syncs are untouched (a fresh venv
has no extras; exact == inexact there).

Both sync arg sets are now named constants with contract tests pinning
the asymmetry, so neither side can silently regress.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 10:06:23 +05:30
cb6c72b409 docs(faq): honest ElevenLabs comparison — where each wins, and why dub quality varies (community question) (#1028)
Asked directly on Discord ('how it compares to something like 11 labs
in quality?'). The old answer ('yes, comparable for most use cases')
oversold — the honest version names where ElevenLabs still wins
(out-of-the-box English polish/consistency) and where OmniVoice is
genuinely competitive (cloning from clean references, 646 languages,
structural advantages), plus the dubbing-specific truth another
same-day report surfaced: a dub is a chain, and incoherent output
usually traces to transcription quality on the user's source audio —
with the check-the-original-text-first debugging step that actually
helps.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 09:13:11 +05:30
219 changed files with 19125 additions and 1669 deletions
@@ -0,0 +1,205 @@
# Adjacent open-source projects — research notes (2026-07-10)
Owner-requested research on five neighboring projects, read against OmniVoice
Studio's current feature-maturity map. Each section ends with what we should
take from it. Priorities are consolidated at the bottom.
| Project | Stars | License | Status | Why it matters to us |
|---|---|---|---|---|
| [Real-Time-Voice-Cloning](https://github.com/CorentinJ/Real-Time-Voice-Cloning) | ~60k | MIT | Retired (models frozen 2019, maintainer quit 2020) | Positioning/SEO opportunity, cautionary tales |
| [VoxCPM](https://github.com/OpenBMB/VoxCPM) | ~33k | Apache-2.0 | Very active (VoxCPM2, Apr 2026) | **Upstream of our `voxcpm2` engine** — sync items below |
| [ebook2audiobook](https://github.com/DrewThomasson/ebook2audiobook) | ~19.5k | Apache-2.0 (default XTTS weights are CPML non-commercial) | Very active, weekly releases | The playbook for our weakest shipped surface (audiobook) |
| [VideoLingo](https://github.com/Huanshere/VideoLingo) | ~17.7k | Apache-2.0 | Active, bursty | Dub-pipeline techniques (translation loop, timeline fit) |
| [voicebox](https://github.com/jamiepine/voicebox) | ~40.2k | MIT | Very active, post-viral triage debt | **Direct competitor** — same stack, same pitch, 10x the audience |
## 1. Real-Time-Voice-Cloning — the retired ancestor
The 2019 SV2TTS implementation ("clone a voice in 5 seconds") that created the
DIY voice-cloning category. Explicitly retired: the maintainer said in 2020 he
won't develop it again; the README now calls itself old and redirects users to
Chatterbox. Models are frozen 2019 checkpoints — 16 kHz, English-only, weak
similarity, Tacotron+WaveRNN. Community PRs keep the install alive (uv
one-command install landed Sept 2025), but ~163 open issues are mostly "how do
I make it sound good" — the answer is: you can't.
**Integrating it as an engine: no.** Strictly worse than everything we ship,
plus PyQt/legacy baggage.
**Take:**
- 60k stars of traffic reads a README that says "go elsewhere," and the
redirect target is a model repo, not a product. An honest
"Real-Time-Voice-Cloning alternative" comparison page is cheap, truthful,
and lands exactly our pitch (local, free, modern quality, 646 languages,
actual installer).
- Its headline copy discipline ("Clone a voice in 5 seconds, generate
arbitrary speech in real-time") is better than ours; our 3-second-reference
claim deserves the same outcome-first, time-boxed phrasing.
- Its failure modes validate our Core Value: out-of-band model links rotted
for years; a research toolbox without packaging drowned in install issues.
## 2. VoxCPM — upstream of our `voxcpm2` engine
Tokenizer-free TTS on a MiniCPM-4 backbone. Current model is **VoxCPM2**
(Apr 2026): 2B params, 30 languages + 9 Chinese dialects, 48 kHz, ~8 GB VRAM,
RTF ~0.30 (0.13 with Nano-vLLM). Latest tag v2.0.3 (May 2026); main has
unreleased seed support and timestamp alignment. Apache-2.0, healthy cadence,
~868k monthly HF downloads.
**Sync items for our integration** (we install `voxcpm` unpinned):
1. **Floor the install at `voxcpm>=2.0.3`** — it carries the MPS
audio-quality fix (low-precision dtypes promoted to float32 on Apple
Silicon). Directly relevant to our default-platform-parity rule.
2. **v2.0.1 removed reference-audio auto-trim** — if we hand raw user clips
to cloning, we now own trim/normalize. Verify our clone path; cloning
quality may have silently regressed when upstream released 2.0.1.
3. **Trailing-audio guard**: end-of-audio gibberish/hallucination is a known
open upstream bug (#352). A trailing-silence/garbage trim on our side is
cheap insurance.
4. **Later, when tagged**: seed support (reproducible generation — currently
buggy upstream, #351) and timestamp alignment (useful for dub sync);
`generate_streaming()` is a candidate for `tts_stream.py`.
5. **Risk**: unpinned dependency + active upstream = next release lands
silently in fresh installs. Consider pinning a tested range.
## 3. ebook2audiobook — the audiobook playbook
Any-format ebook (epub/pdf/docx/even scanned images via OCR) → Calibre
normalize to EPUB → TOC/spine chapters ("blocks") → per-language sentence
split → per-sentence TTS → chapterized m4b with metadata/cover. Gradio UI +
headless CLI + Docker for every accelerator. Engine roster is 2023-era Coqui
(XTTSv2 default, Bark, Piper, MMS…), with voice-conversion post-processing to
fake cloning on non-cloning engines. 19.5k stars, near-weekly releases, only
4 open issues.
This is the mature version of exactly the surface where we're weakest: our
audiobook/stories feature is a thin UI over per-chapter render caching, with
no server-side ebook parsing and no per-segment regeneration.
**Take (prioritized):**
1. **Per-sentence render cache + content-hashed blocks + missing-file
resume.** Every sentence is its own file; restart re-renders only what's
missing; editing a block invalidates only that block. This closes our
biggest audiobook gap (per-chapter cache, no crash resume) and is the same
span-level model spec 03 already calls for — dub's `incremental.py`
pattern, extended to longform.
2. **Normalize-to-EPUB ingestion** (Calibre `ebook-convert`) instead of
building N format parsers; blocks carry keep/drop flags for front matter.
3. **Engine-agnostic text-normalization pre-pass**: per-language abbreviation
maps, num2words, roman numerals, and a non-text character filter that
kills TTS hallucination triggers. Benefits every engine we ship, not just
audiobooks.
4. **Chapterized m4b output** (ffmpeg FFMETADATA chapters, cover art, VTT
sidecar) — small work, high perceived value.
5. **Inline voice/pause tags** for multi-voice narration — our cloning
quality makes this worth more to us than it is to them.
**Where we already win:** native desktop UX, modern engine quality
(CosyVoice3/IndexTTS2/VoxCPM2 vs 2023 Coqui), real zero-shot cloning without
VC hacks, no Calibre-wall install, and a commercially-clean default engine
(their default XTTS weights are CPML non-commercial).
## 4. VideoLingo — dub-pipeline techniques
"Netflix-quality subtitles + dubbing" as a 14-stage Streamlit pipeline:
yt-dlp → WhisperX word-level ASR → spaCy + LLM two-candidate semantic split →
summarize-first terminology glossary → 3-step TranslateReflectAdapt →
length-constrained subtitles → duration-aware dub-chunk planning →
per-chunk reference audio → TTS → merge. Its recommended path is
cloud-heavy (API LLM/TTS, optionally API ASR); fully-local is possible but
fragile. Single-speaker only — it explicitly gave up on diarized multi-voice
dubbing. Apache-2.0, ~17.7k stars, bursty maintenance, install pain on
Windows/CUDA.
**Take (prioritized):**
1. **TranslateReflectAdapt** — add a reflection/critique pass to our
per-segment translation prompt. Prompt-level change, meaningful quality
win on idiomatic output.
2. **Summarize-first glossary** — extract theme + terminology once per video,
inject into every segment's translation. Fixes term drift on long videos.
3. **Duration-aware chunk planning** — estimate TTS duration *before*
generating; classify each line ok / needs-speedup / impossible; borrow
inter-subtitle gap time and merge adjacent segments before resorting to
atempo; for impossible lines, LLM-trim filler from the dub text instead of
chipmunking. Our smart-fit handles the tail of this; their pre-planning
avoids generating doomed audio at all.
4. **Two-candidate split prompt** — generate two `[br]` segmentations, have
the LLM pick, instead of accepting the first.
**Where we already win:** fully local by design, per-segment regeneration +
directorial AI (they have coarse folder-state resume, no per-segment redo),
cross-platform installers, cloning stable across languages. Their
single-speaker ceiling is our opening if diarized multi-voice dubbing ever
ships.
## 5. voicebox — the direct competitor
Jamie Pine's (Spacedrive founder) "open-source AI voice studio. Clone,
dictate, create." — architecturally a near-twin: **Tauri + React/TS +
FastAPI/Python + SQLite**, MIT, local-first, explicitly pitched as
ElevenLabs-out + WisprFlow-in replacement. Launched Jan 29, 2026; the launch
post did ~17M views on X, and it sits at **~40.2k stars** with ~10 community
contributors and heavy AI co-authorship. Latest tagged release v0.5.0
(Apr 2026); main is active but untagged for ~10 weeks, with **434 open
issues / 105 open PRs** — a polished happy path with thin edges.
Engines: Qwen3-TTS 0.6B/1.7B (flagship cloner), Qwen CustomVoice, LuxTTS,
Chatterbox Multilingual (23 langs) + Turbo, HumeAI TADA, Kokoro. Features
where they lead: global-hotkey dictation overlay with LLM transcript cleanup
(macOS-verified), Pedalboard post-FX chain, generation versioning/starring,
multi-track Stories editor, **MCP per-client voice bindings** ("Claude Code
speaks in your cloned voice") used as a viral wedge, DirectML/Intel-Arc
coverage, and an agent-facing CONTRIBUTING pattern that farms drive-by
contributions.
Two strategic facts:
- **They are adding accounts.** "Log in with browser" auth for a
`voicebox.sh` cloud tier merged July 5 (their PR #812). Open-core with a
paid cloud is visibly forming — which cuts against the pitch that won them
their audience.
- **Press already flagged their missing consent/misuse policy** — we ship
watermarking by default and consent attestation in `.ovsvoice`.
**Where we're ahead:** 646 languages vs 23, video dubbing (they have none),
voice design from text descriptions (roadmap item for them, shipped for us),
engine breadth (CosyVoice3/VoxCPM2/IndexTTS2/GPT-SoVITS/sherpa-onnx), and
backward-compat/release discipline.
**Take:**
1. **Positioning: own "no accounts, ever."** Their cloud login is our
opening — state the local-first guarantee in the README as a permanent
commitment, next to the 646-language and dubbing advantages they can't
match today.
2. **Tell the MCP agent-voice story loudly.** We already ship an MCP server
and Agent Skills; per-client voice bindings + a speak-in-your-voice demo
was their single best growth hook and costs us mostly marketing effort.
3. **Generation versioning/starring and post-FX presets** — cheap,
high-perceived-value Studio features worth absorbing.
4. **Watch their triage debt** (434 open issues): our absorb-or-decline
queue discipline is a real contributor-trust differentiator — keep it.
## Consolidated priorities
Ordered by (user impact on already-shipped surfaces) × (effort):
1. **voxcpm2 upstream sync** (§2 items 13): version floor, ref-clip trim
audit, trailing-audio guard. Small, protects an engine users already run.
2. **Dub translation quality loop** (§4 items 12): reflect pass + glossary.
Prompt-level, no new deps, lifts the flagship dubbing feature.
3. **Audiobook maturity via per-sentence cache + resume** (§3 item 1): the
established pattern for the feature the maturity survey ranked weakest —
and it's the same architecture spec 03 already prescribes.
4. **Text-normalization pre-pass** (§3 item 3): engine-agnostic hallucination
reduction; pairs with the pronunciation dictionary we already shipped.
5. **Duration-aware dub planning** (§4 item 3) and **chapterized m4b export**
(§3 item 4): next tier, both self-contained.
6. **Competitive positioning vs voicebox** (§5 items 12): own "no accounts,
ever" while they onboard a cloud tier, and tell the MCP agent-voice story
we already technically ship.
7. **RTVC comparison/migration page** (§1): marketing, not engineering;
cheap and honest.
*Method note: compiled from five parallel research passes over the repos'
READMEs, releases, issues, and (for ebook2audiobook) source; figures as of
2026-07-10.*
+69 -1
View File
@@ -6,7 +6,75 @@ The format is loosely based on [Keep a Changelog](https://keepachangelog.com/).
Versions track the desktop app (`tauri.conf.json` + `frontend/src-tauri/Cargo.toml`).
The bundled TTS model package (`pyproject.toml`) is versioned independently.
## [Unreleased]
## [0.3.17] — 2026-07-11
The polish release. The dubbing workspace can no longer trap you — an interrupted dub session used to relaunch into an eternal spinner that even reinstalling couldn't clear (thank you @nanai97 for the screenshot that cracked it). A 58-finding audit of every Settings panel got fixed end to end, **FFmpeg and yt-dlp stopped being your problem** (the app provisions its own, with a new Audio tools panel when you want control), the Engines and Models pages went compact and tabbed, the launcher stopped trusting half-dead backends, and the app finally opens at 100% scale.
### Added
- **FFmpeg, FFprobe, and yt-dlp stopped being your problem.** The setup wizard no longer lists them as system requirements with "brew install" homework — the app provisions them itself: shipped installs already bundle them, and when nothing is found the backend downloads its own checksum-pinned static build in the background, showing a single actionable card only if that fails. A new **Settings → Audio tools** panel gives back the control: per-tool version and origin (App package / Bundled / System / Custom), update / use-system / choose-file / restore-bundled — and **one-click yt-dlp updates** that survive app upgrades, because video-site support changes faster than releases. Install docs updated to match. (#1071)
- **The Engines and Models pages got compact and tabbed.** Engines is now one section with TTS / ASR / LLM tabs; every engine is a strict two-line, fixed-height row with truncated text and aligned status / GPU / isolation / action columns, so the whole engine list fits one screen — details like "Why unavailable?" expand below the row instead of stretching it. Models rows tightened the same way. (#1072)
- **The Engines and Models pages got a full readability-and-features pass.** Every engine row now carries a small identity mark and honest capability badges (voice cloning, device routing with the reason on hover, sidecar isolation), and engines that are ready-but-have-advice finally say so — upgrade hints used to be dropped before reaching the UI. The model store gains a filter, disk-space context next to downloads, "in memory — safe to unload" indicators, copyable setup snippets for opt-in engines, and empty states that tell you what to do next. (#1058)
### Fixed
- **The app no longer attaches to a "zombie" backend that looks alive but fails everything.** If a backend process survived while its install was replaced or deleted underneath it, it kept answering health checks from memory — so the next launch attached to it and every real request failed with a confusing access-control error. The launcher now runs a deeper probe (an endpoint that actually touches the database) before attaching, and replaces any backend that fails it. The local dev/test scripts also now terminate running instances before wiping data, which is how this state was produced. (#1077)
- **The app opens at 100% scale by default.** New installs rendered everything at 130% zoom, which read as oversized on typical displays. Fresh sessions now start at native size; if you already picked a scale in Settings → Appearance, your choice is kept. (#1074)
- **The app no longer relaunches into a dead "generating" dub session — the blank-pane-and-spinner trap.** The saved dub session was restoring its in-flight state verbatim: quit (or crash) while a dub was generating and every subsequent launch waited forever for work that died with the process — and reinstalling couldn't clear it. Interrupted sessions now reopen on the segment editor with all your work intact (or the upload screen if nothing was transcribed yet). Thanks to @nanai97 for the screenshot that told the whole story. (#1067)
- **A 58-finding audit of every Settings panel, fixed end to end.** Highlights: the About page linked to the wrong project's GitHub; Arabic rendered left-to-right (RTL wiring was missing); a saved proxy could never be cleared after a reload; the HF-mirror and refinement panels vanished entirely when the backend was down; "Test now" on the HF token served five-minute-old cached results; factory reset only cleared part of what it promised; pronunciation previews ignored language-scoped entries; the hotkey recorder swallowed invalid presses in silence; Settings search could strand you with an empty sidebar — plus first component tests for previously untested panels, full i18n for five all-English panels, accessible names across inputs, confirmed destructive actions, deep links instead of dead-end advice, temp-file reclaim, and log-sharing workflows. (#1059, #1060, #1061, #1063, #1064)
## [0.3.16] — 2026-07-11
The quality release. Three long-standing frictions got structural fixes: **regenerating no longer destroys good takes** (a takes rail with starring and restore), **audiobooks stop redoing finished work** (per-sentence caching — edit one line, re-render one line; crashes resume where they stopped), and **dub translations stay consistent and fit their timeline** (auto-glossary + a naturalness pass, plus fit prediction before any GPU time is spent). Under the hood, every text path now speaks numbers, times, and abbreviations correctly, the VoxCPM2 engine gained upstream-alignment guards, and a Windows first-run breaker — model downloads completing but the cache ending up with broken file links — now self-heals automatically. Thank you @dmnobunaga for the razor-sharp diagnosis on that last one.
### Fixed
- **Windows: model downloads that finished but wouldn't load now repair themselves.** On machines without Developer Mode, the model cache could end up with all its multi-gigabyte files downloaded but the snapshot's file links broken — and the app reported a misleading "does not appear to have a file named model.safetensors". The app now detects the broken links on load failure, restores just the missing pieces (reusing everything already downloaded, and falling back to real file copies where links can't be trusted), and retries once; if repair is impossible, the error finally names the actual cache folder to delete. Root-caused in the wild by @dmnobunaga — thank you. (#1056)
- **VoxCPM2: cloning reference clips are now conditioned, and outputs lose their silent tails.** Reference audio used to reach the model completely raw; it now gets edge-silence trimming and a 30-second cap (fail-open — short clean clips pass through untouched), and generated audio gets a trailing-silence trim. The install hint also moved to `voxcpm>=2.0.3`, which carries an important Apple-Silicon audio-quality fix — older installs keep working and see an upgrade hint in the logs. (#1055)
- **Streaming TTS requests without an `emo_alpha` field no longer crash.** A minimal `/ws/tts` request hit a `KeyError` and returned an error frame instead of audio — found while giving that route its first tests. (#1054)
- **Long generations no longer risk a multi-gigabyte memory spike while being watermarked.** The invisible watermark (on by default) pushed the entire waveform through AudioSeal in a single call, and its memory use grows with audio length — a multi-minute generation demanded a single ~2 GB allocation, enough to fail outright on a 16 GB machine already holding a model ("DefaultCPUAllocator: not enough memory"). Watermark embedding — and the Verify-audio detector, which had the same flaw with uploaded files — now processes audio in ~30-second chunks, so peak memory stays flat no matter how long the audio is. Detection also got sharper for spliced files: it now reports the strongest chunk instead of a whole-file average. (#1045)
- **The ⊕ Insert token list no longer climbs out of the viewport.** In the voice-clone script panel, the insert popover (expression tags, CMU phoneme chips) always opened *upward* from the textarea — and since that input sits at the very top of the panel, the list disappeared past the top of the window with no way to see or scroll it. It now opens below the input, where there's always room. (owner-reported)
### Added
- **Edit one sentence, re-render one sentence.** Audiobook and Stories renders now cache every synthesized sentence individually (content-addressed, under the existing chapter cache): fixing a single line in a chapter reuses all the untouched audio, and an interrupted render — crash, quit, power loss — resumes from the sentences that already finished instead of redoing the whole chapter. One byte cap bounds both cache layers, and chapter caches from released versions keep working. (#1048)
- **Numbers, times, and abbreviations are spoken correctly in every engine.** A conservative normalization pass now runs before TTS everywhere (Studio, dubbing, audiobooks): "3:30" is read as a time, "2" as "two" (29 languages), "Dr." as "Doctor" — while stray control characters and markup remnants that trigger engine hallucinations are stripped. Deliberately cautious: when a rewrite could be wrong, the text is left alone, and your pronunciation-dictionary entries always have the final say. Toggleable (`text_normalization_enabled`, default on). The OpenAI-compatible API, streaming TTS, and the batch queue run the same pass, so every door into the engines speaks text identically. (#1049, #1054)
- **Dub translations stay consistent and sound natural (LLM engine).** Before translating, one pass over the whole transcript builds a terminology glossary (your manual glossary entries always win) that rides along on every segment, so names and terms stop drifting mid-video. After each segment's direct translation, an optional reflect pass critiques and rewrites stiff lines into natural spoken dialogue — any failure silently keeps the direct translation. Both toggleable in the Dub tab; the reflect toggle states its 3-calls-per-segment cost. (#1050)
- **The Dub tab now predicts which lines won't fit — before wasting GPU time on them.** After translation, each segment gets a duration estimate (self-calibrating to your engine and language from the segments already rendered) and a "Tight fit" or "Won't fit +Ns" badge when the dubbed audio can't match the timeline even with speed-up. An opt-in "Suggest shorter lines" option asks the LLM for a meaning-preserving shorter rewrite you can apply per segment — never applied automatically. (#1051)
- **Generation takes: star the good ones, restore any of them.** Regenerating no longer means losing the previous result — recent takes appear in the workspace history with replay, star/unstar, and one-click restore as the active output. History is now capped (Settings → Storage, default 200 takes): the oldest unstarred takes are pruned, starred ones are kept forever, and an audio file is only deleted when nothing else references it. (#1052)
- **A persistent mini-player for all the audio that used to play "invisibly".** Generated output, voice-profile and dub-segment previews, story lines, Gallery voices, and Projects renders all played through a bare audio pipe — no waveform, no seek, no time, and (until v0.3.15's stop pill) no way to stop them. A slim player bar now docks above the Logs footer whenever such audio plays, on every page: live waveform (decoded once from the audio already in memory — nothing is re-fetched), click/drag/keyboard seek, play/pause, elapsed/total time, what's-playing label, and a stop button. It replaces the stop-only pill, and because it's part of the app's layout rather than a floating overlay, the pill's "covers the Production Overrides row at 1440×900" overlap class can't come back. Stories line previews also route through it — which makes them stoppable *and* fixes them being silent on the macOS/Linux desktop builds (their old playback path used blob: URLs, which WebKit refuses to play). (no issue — owner request following #1032's stop-pill band-aid)
## [0.3.15] — 2026-07-10
The cold-start release. Three "why is this broken on my machine" mysteries got solved at their roots: **first generations stop dying at 300 seconds** (the timeout was counting the model download as generation time — @moduvoice measured it on a Tesla T4: 0% GPU for the full window), **updates stop deleting engines you installed yourself** (the updater's dependency sync removed anything not in the app's lockfile — including things our own UI told you to install), and **the "slower than v0.3.5" regression is found and fixed** (clone profiles without a transcript were silently re-running a full Whisper transcription on every single generate). Also: Clear History is back, auto-played audio is finally stoppable, @stronghamjji hardened the dub pipeline against wedged transcribes, and @shakib30's community Colab notebook is now the linked no-GPU path. Thank you all.
### Added
- **Agent Skills: `npx skills add debpalash/omnivoice-studio`.** Two installable [skills](https://skills.sh) now ship in the repo — `omnivoice` teaches any AI agent (Claude Code, Cursor, Codex, …) to speak and transcribe through your local install via the OpenAI-compatible API, including your cloned voices; `oss-maintainer` packages the maintainer methodology this project is run with.
### Fixed
- **A fresh install's first generation no longer dies at 300 seconds while the model is still downloading.** The generate timeout was one clock around everything — including the engine's lazy multi-GB weight download on a cold start — so first requests burned the whole budget on the download (0% GPU the entire time, as a contributor's Tesla T4 verification measured) and failed with a misleading "too heavy for the available compute" error. Model loading now runs first under its own, much larger budget; the generate clock starts only once the engine is warm. A genuinely stalled download gets a new error that says so and points at Settings → Models. (#1033, #1037, evidence from #1014)
- **The OpenAI-compatible speech endpoint stops silently discarding quality settings.** `POST /v1/audio/speech` accepted `num_step` and `guidance_scale` in the request body with a 200 OK — and dropped them without a word, so API callers couldn't reach the model's documented quality preset (`num_step: 32`). Both are now declared, validated, and passed through to the engine, matching the native `/generate` endpoint. Caught by a contributor's measured Tesla T4 verification pass. (#1014)
- **Updating no longer uninstalls engines you added yourself.** Optional engines installed with pip into the app's environment (VoxCPM2, KittenTTS — exactly what Settings → Engines' own install hints say to do) were silently deleted by every app update, because the update's dependency sync removed anything not in the app's lockfile. Routine updates now leave your additions alone; the repair path ("Clean & Retry") still restores the exact known-good state, since a broken environment is sometimes *caused* by an extra package. (#1029)
- **Voice cloning stops re-transcribing the same reference clip on every generate.** Since v0.3.6, a profile saved without a transcript (the default) triggered a full ASR model load *plus* a transcription of the reference on every single synthesis — the "TTS got much slower than v0.3.5, same settings" regression. The first auto-transcription is now saved onto the profile, and repeated ad-hoc uploads of the same clip reuse a content-keyed transcript cache — so the cost is paid once, not per request. A transcript you typed yourself is never overwritten. (#1032)
- **The Clear History button is back.** The workspace redesign moved generation history into the right-side panels but dropped the old sidebar's clear-all control, leaving one-by-one deletion as the only way to empty a long history. Both the Voice and Dub history panels now have a Clear History button (with a confirmation), scoped to that workspace's history. (#1032)
- **The audio that auto-plays after a render can finally be stopped anywhere.** The finished-render playback has no on-screen player, and the only stop control lived in the Voice workspace's action bar — audio started from the Dub workspace, a profile preview, or after navigating away simply played to the end. A stop button now appears above the status area whenever such playback is active, on every page. The existing Settings → Appearance "Auto-play preview" toggle now also governs this playback, as its description always promised. (#1032)
## [0.3.14] — 2026-07-09
+23 -1
View File
@@ -397,6 +397,20 @@ print(result.text)
Want the whole surface (100+ endpoints)? The full REST API reference is embedded in the app — **Settings → OpenAPI Reference** (Scalar-powered), or the `{}` button in the footer.
### 📓 Run on Google Colab (community)
No local GPU? A community member ([@shakib30](https://github.com/shakib30)) maintains a working Colab notebook: [shakib30/OmniVoice-Studio-google-colab](https://github.com/shakib30/OmniVoice-Studio-google-colab). Community-maintained — issues with the notebook go there; issues with OmniVoice itself come here.
### 🤝 Agent Skills
Teach your AI agent (Claude Code, Cursor, Codex, …) to use OmniVoice with one command:
```sh
npx skills add debpalash/omnivoice-studio
```
Ships two [skills](https://skills.sh): **`omnivoice`** — speak and transcribe through your local install (including your cloned voices) from any agent, free and offline; and **`oss-maintainer`** — the maintainer methodology this project is run with, for anyone running their own OSS project with an agent.
---
## 🗺️ Roadmap
@@ -525,7 +539,15 @@ Yes please — bug fixes, new TTS engine adapters, UI improvements, docs, transl
<details>
<summary><b>Is this really as good as ElevenLabs?</b></summary>
<br/>
For voice cloning and dubbing, yes — OmniVoice uses a state-of-the-art diffusion TTS model with 646 languages (ElevenLabs supports 32). Quality is comparable for most use cases. Where ElevenLabs wins is in their polished cloud API and pre-made voice library. OmniVoice wins on privacy, cost, language coverage, and customizability.
Honest answer: <b>it depends on what you're doing.</b>
<b>Where OmniVoice is genuinely competitive:</b> voice cloning from a clean reference clip (state-of-the-art open diffusion TTS), language coverage (646 languages vs. their 32), and everything structural — no per-character billing, no usage caps, no audio leaving your machine, full pipeline customizability (10 TTS engines, 10 ASR engines, your choice of translation).
<b>Where ElevenLabs still wins:</b> out-of-the-box consistency and polish, especially for English TTS. Their one model is heavily tuned; our quality depends on which engine you pick, your hardware, and — for cloning — the reference audio (a dry, close-mic clip clones dramatically better than a noisy or echoey one).
<b>For dubbing specifically:</b> a dub is a chain — transcription → translation → cloning → synthesis — and the output is only as good as its weakest link on <i>your</i> source material. Noisy or accented source audio degrades transcription, which degrades everything downstream; some language pairs translate better than others. If parts of a dub come out incoherent, check the segment table's <i>original</i> text first: if the transcription is already wrong there, switch the ASR engine (Settings → Engines) or use cleaner source audio — that's usually the fix, not the voice.
Try it on your real material — it's free and takes one download. Many users find it replaces ElevenLabs outright; some keep both for different jobs. Both outcomes are fine with us.
</details>
<details>
+5
View File
@@ -69,6 +69,11 @@ hiddenimports = [
# Pipeline
'yt_dlp', 'demucs', 'demucs.separate',
# Numbers→words for the pre-TTS text normalization pass
# (services/text_normalization.py). Imported inside a function (lazy),
# so pin it explicitly rather than trusting the tracer.
'num2words',
# OmniVoice's own package
'omnivoice', 'omnivoice.models', 'omnivoice.models.omnivoice',
]
+63 -24
View File
@@ -311,34 +311,60 @@ async def _prepare_synth(default_voice: str | None, language: str | None = None)
return info["synth"], info["sample_rate"], resolve, engine_id
def _render_chapter_cached(chapter, synth, sr, engine_id, resolve, cache_dir, lexicon=None):
def _render_chapter_cached(chapter, synth, sr, engine_id, resolve, cache_dir, lexicon=None,
language=None):
"""Render one chapter, content-addressed so a re-run reuses it (resume).
Returns ``(wav_path, duration_s, was_cached)``. The WAV lives at
``cache_dir/<key>.wav`` where ``key`` is :func:`chapter_cache_key` over the
chapter's spans + sample rate + engine + each voice's resolved signature
(+ the lexicon, so a lexicon edit re-renders), so an unchanged chapter is
never re-synthesized. Runs in the GPU-pool executor.
Returns ``(wav_path, duration_s, was_cached, seg_stats)``. Two cache
layers:
* Outer — the WAV at ``cache_dir/<key>.wav`` where ``key`` is
:func:`chapter_cache_key` over the chapter's spans + sample rate +
engine + each voice's resolved signature (+ the lexicon, so a lexicon
edit re-renders). A fully-unchanged chapter hits here and never touches
segment files; the key derivation is unchanged, so chapter caches
written by released versions keep hitting. ``seg_stats`` is ``None``.
* Inner — on a chapter miss, each spoken span goes through the
:class:`services.longform_render.SegmentCache` under
``cache_dir/segments``: cached segments load from disk, only the
edited/missing ones synthesize, and each fresh segment persists the
moment it renders (an interrupted chapter resumes from them).
``seg_stats`` is ``{"total": spoken_spans, "cached": reused}``.
Span text is normalized (``services.text_normalization``) up front — BEFORE
either cache key and BEFORE ``synthesize_chapter``'s lexicon pass, so the
per-project dictionary operates on normalized text and toggling / changing
normalization output naturally invalidates cached chapters and segments.
Runs in the GPU-pool executor.
"""
import json
import wave
from services.audio_io import atomic_save_wav
from services.longform_render import chapter_cache_key
from services.audiobook import Span
from services.longform_render import SegmentCache, chapter_cache_key
from services.pronunciation import normalize_lexicon
from services.text_normalization import normalize_for_tts
spans = [Span(voice_id=s.voice_id, text=normalize_for_tts(s.text, language),
pause_ms_after=s.pause_ms_after, speed=getattr(s, "speed", None))
for s in chapter.spans]
spans_tuples = [(s.voice_id, s.text, s.pause_ms_after, getattr(s, "speed", None))
for s in chapter.spans]
sig: dict = {}
for s in chapter.spans:
for s in spans]
voice_sigs: dict = {}
for s in spans:
k = s.voice_id or ""
if k not in sig:
if k not in voice_sigs:
v = resolve(s.voice_id)
sig[k] = f"{v.get('ref_audio')}|{v.get('ref_text')}|{v.get('instruct')}|{v.get('seed')}"
voice_sigs[k] = f"{v.get('ref_audio')}|{v.get('ref_text')}|{v.get('instruct')}|{v.get('seed')}"
sig: dict = dict(voice_sigs)
lex_sig = ""
if lexicon:
# Fold the lexicon into the cache key so editing pronunciations
# invalidates cached chapters (reserved key can't collide with a voice id).
sig["\x00lexicon"] = json.dumps(normalize_lexicon(lexicon), sort_keys=True)
lex_sig = json.dumps(normalize_lexicon(lexicon), sort_keys=True)
sig["\x00lexicon"] = lex_sig
key = chapter_cache_key(spans_tuples, sample_rate=sr, engine_id=engine_id, voice_sig=sig)
wav_path = os.path.join(cache_dir, f"{key}.wav")
@@ -346,13 +372,17 @@ def _render_chapter_cached(chapter, synth, sr, engine_id, resolve, cache_dir, le
try:
with wave.open(wav_path, "rb") as w:
dur = w.getnframes() / float(w.getframerate() or sr)
return wav_path, dur, True
return wav_path, dur, True, None
except Exception:
pass # corrupt cache entry — fall through and re-render
audio, dur = synthesize_chapter(chapter.spans, synth, sr, lexicon=lexicon)
seg_cache = SegmentCache(cache_dir, sample_rate=sr, engine_id=engine_id,
voice_sig=voice_sigs, extra_sig=lex_sig)
audio, dur = synthesize_chapter(spans, synth, sr, lexicon=lexicon,
segment_cache=seg_cache)
atomic_save_wav(wav_path, audio, sr)
return wav_path, dur, False
return wav_path, dur, False, {"total": seg_cache.hits + seg_cache.misses,
"cached": seg_cache.hits}
class AudiobookPreviewRequest(BaseModel):
@@ -383,14 +413,15 @@ async def audiobook_preview(req: AudiobookPreviewRequest) -> dict:
chapter = plan.chapters[req.chapter_index]
cache_dir = os.path.join(OUTPUTS_DIR, "longform_cache") # shared with _render_longform_sse
os.makedirs(cache_dir, exist_ok=True)
resolved_lang = _resolve_default_language(req.language, req.default_voice)
synth, sr, resolve, engine_id = await _prepare_synth(
req.default_voice,
language=_resolve_default_language(req.language, req.default_voice),
language=resolved_lang,
)
loop = asyncio.get_running_loop()
wav_path, dur, was_cached = await loop.run_in_executor(
wav_path, dur, was_cached, _seg_stats = await loop.run_in_executor(
_gpu_pool, _render_chapter_cached, chapter, synth, sr, engine_id, resolve, cache_dir,
req.lexicon,
req.lexicon, resolved_lang,
)
return {
"output": os.path.relpath(wav_path, OUTPUTS_DIR), # served via /audio
@@ -495,8 +526,9 @@ async def _render_longform_sse(
loop = asyncio.get_running_loop()
try:
resolved_lang = _resolve_default_language(language, default_voice)
synth, sr, resolve, engine_id = await _prepare_synth(
default_voice, language=_resolve_default_language(language, default_voice)
default_voice, language=resolved_lang
)
total = len(plan.chapters)
@@ -508,9 +540,10 @@ async def _render_longform_sse(
for i, chapter in enumerate(plan.chapters):
try:
wav_path, dur, was_cached = await loop.run_in_executor(
wav_path, dur, was_cached, seg_stats = await loop.run_in_executor(
_gpu_pool, _render_chapter_cached,
chapter, synth, sr, engine_id, resolve, cache_dir, lexicon,
resolved_lang,
)
except Exception: # isolate a bad chapter — keep going
logger.warning("[%s] chapter %d (%s) failed to render",
@@ -522,9 +555,15 @@ async def _render_longform_sse(
chapter_files.append(wav_path)
chapters_meta.append((chapter.title, int(round(dur * 1000))))
cached_n += 1 if was_cached else 0
yield _emit({"type": "chapter", "index": i, "total": total,
"title": chapter.title, "duration_s": round(dur, 2),
"cached": was_cached})
ev = {"type": "chapter", "index": i, "total": total,
"title": chapter.title, "duration_s": round(dur, 2),
"cached": was_cached}
if seg_stats is not None:
# Additive fields (old clients ignore them): segment-level
# reuse inside a re-rendered chapter.
ev["segments"] = seg_stats["total"]
ev["cached_segments"] = seg_stats["cached"]
yield _emit(ev)
if not chapter_files:
yield _emit({"type": "error", "error": "all chapters failed to render"})
+7
View File
@@ -287,6 +287,13 @@ async def _run_batch_pipeline(job_id: str, job: dict):
continue
def _gen(text=seg_text, lang=target_lang, dur=seg_duration):
# Normalize once at the segment's text→engine choke point —
# the same pre-pass as /generate and dub_generate's _gen.
# `lang` is the job's target language code. Pref-gated,
# idempotent, never raises.
from services.text_normalization import normalize_for_tts
text = normalize_for_tts(text, lang)
ref_audio = None
ref_text = None
+28 -6
View File
@@ -1008,9 +1008,22 @@ async def dub_transcribe_stream(
yield _sse_event("ping", {})
if clones:
from services.speaker_clone import refine_ref_texts
clones = await loop.run_in_executor(
_gpu_pool, lambda: refine_ref_texts(clones, _asr_backend),
)
# Bound the re-transcribe like every other ASR dispatch in
# this file (#730): a wedged transcribe would otherwise hold
# the GPU-pool worker forever and starve later work into a
# "can't reach backend". On timeout the guard resets the pool
# and raises — keep the original (unrefined) clones, matching
# refine_ref_text's own "failure is a strict no-op" fallback.
try:
clones = await run_transcribe_guarded(
_gpu_pool,
lambda: refine_ref_texts(clones, _asr_backend),
what="Dub clone ref-text refine",
)
except ASRTimeoutError as e:
logger.warning(
"clone ref-text refine timed out; keeping original ref_text: %s", e
)
# Wave 3.2: per-segment clone refs. Cut each long-enough segment's
# own reference from the vocals so the dub of each line matches the
# prosody of its source line. Short lines fall back to the
@@ -1031,9 +1044,18 @@ async def dub_transcribe_stream(
)
if seg_clones:
from services.speaker_clone import refine_ref_texts
seg_clones = await loop.run_in_executor(
_gpu_pool, lambda: refine_ref_texts(seg_clones, _asr_backend),
)
# Same guard as the per-speaker refine above (#730):
# keep the original seg_clones on a wedge/timeout.
try:
seg_clones = await run_transcribe_guarded(
_gpu_pool,
lambda: refine_ref_texts(seg_clones, _asr_backend),
what="Dub segment ref-text refine",
)
except ASRTimeoutError as e:
logger.warning(
"segment ref-text refine timed out; keeping original ref_text: %s", e
)
job["segment_clones"] = seg_clones
except Exception as e:
logger.warning("per-segment clone refs skipped: %s", e)
+33 -1
View File
@@ -338,6 +338,13 @@ async def dub_generate(job_id: str, req: DubRequest):
# retain every generated tensor in RAM until final assembly.
_pending_seg_writes: list[tuple] = []
# Calibration records for the pre-synthesis duration planner
# (services/duration_planner.py): text length + the NATURAL-rate TTS
# duration of every freshly synthesized segment. Only meaningful for
# the natural-rate strategies — strict_slot forces the audio to the
# slot length, which would poison the observed chars-per-second.
_natural_dur_records: dict[str, dict] = {}
# Phase 4.1 bench instrumentation: measure where incremental time goes.
# Only prints when regen_only is active (real-user incremental path).
_t_start = time.perf_counter()
@@ -429,6 +436,12 @@ async def dub_generate(job_id: str, req: DubRequest):
continue
def _gen(text, lang, instruct_str, dur_s, nstep, cfg, spd, profile_id, effect_preset):
# Normalize once at the segment's text→engine choke point
# (covers the OOM-retry generate below too, which reuses this
# closure's `text`). Pref-gated, idempotent, never raises.
from services.text_normalization import normalize_for_tts
text = normalize_for_tts(text, lang)
ref_audio = None
ref_text = None
used_seed = None
@@ -673,6 +686,15 @@ async def dub_generate(job_id: str, req: DubRequest):
sync_scores.append(sync_ratio)
# Duration-planner calibration sample: this text length spoke
# for this long at natural rate. Keyed by stable seg id and
# merged into the per-language job map after the loop.
if strategy != "strict_slot" and seg.text.strip() and generated_dur > 0:
_natural_dur_records[str(seg_id)] = {
"chars": len(seg.text.strip()),
"dur": round(generated_dur, 4),
}
# Build the fingerprint now (cheap) but defer the disk write
# and job flush to the batch-write phase after the GPU loop.
_seg_fp = None
@@ -773,6 +795,13 @@ async def dub_generate(job_id: str, req: DubRequest):
hashes[_sid] = _fp
quality_map[_sid] = _nstep
job["seg_hashes"] = dict(hashes)
# Duration-planner calibration: per-language (chars, natural dur)
# records. update() (not replace) so partial regens keep accumulating
# samples from earlier runs of this track.
if _natural_dur_records:
job.setdefault("seg_natural_durs_by_lang", {}).setdefault(
lang_code, {},
).update(_natural_dur_records)
# Single job flush instead of one per 8 segments.
_save_job(job_id, job)
_t_diskw = time.perf_counter() - _t_diskw_0
@@ -1246,8 +1275,11 @@ async def preview_segment(job_id: str, req: SegmentPreviewRequest):
instruct_str = row["instruct"]
lang = req.language if req.language != "Auto" else None
# Same normalization as the full dub render above, so a preview
# sounds exactly like the final segment. Pref-gated, never raises.
from services.text_normalization import normalize_for_tts
audio_out = backend.generate(
text=req.text,
text=normalize_for_tts(req.text, lang),
language=lang,
ref_audio=ref_audio,
ref_text=ref_text,
+235 -4
View File
@@ -9,7 +9,7 @@ from fastapi.responses import JSONResponse
from schemas.requests import TranslateRequest
from services.model_manager import _cpu_pool, _gpu_pool
from services.translator import cinematic_available, cinematic_refine_many, _cinematic_budget
from api.routers.dub_core import _get_job
from api.routers.dub_core import _get_job, _save_job
router = APIRouter()
logger = logging.getLogger("omnivoice.api")
@@ -202,6 +202,46 @@ def _resolve_source_lang(req: TranslateRequest) -> str:
return _guess_lang_from_text(getattr(req, "segments", None)) or "en"
def _resolve_translation_context(req, client, model_name: str, timeout: float,
src_lang: str) -> Optional[dict]:
"""Cached auto-glossary context (theme + terms) for this job/target.
Cache lives on the dub job dict (``job["translation_context"][target]``)
and persists through the existing ``job_data`` JSON blob via ``_save_job``
no schema change. A transcript fingerprint keys the cache so an edited
transcript re-extracts; an unchanged transcript costs zero LLM calls on
re-translate. Any failure returns None translation proceeds without
context, never fails because of it. Blocking; run in an executor.
"""
from services import translation_quality as tq
texts = [(s.text or "") for s in req.segments]
fp = tq.transcript_fingerprint(texts)
job = _get_job(req.job_id) if getattr(req, "job_id", None) else None
if job is not None:
cached = (job.get("translation_context") or {}).get(req.target_lang)
if isinstance(cached, dict) and cached.get("fingerprint") == fp:
return cached
ctx = tq.extract_context_sync(
client, model_name, timeout,
segment_texts=texts,
source_lang=src_lang,
target_lang=req.target_lang,
source_name=LANG_NAMES.get(src_lang, src_lang),
target_name=LANG_NAMES.get(req.target_lang, req.target_lang),
)
if ctx is None:
return None
ctx = {**ctx, "fingerprint": fp}
if job is not None:
try:
job.setdefault("translation_context", {})[req.target_lang] = ctx
_save_job(req.job_id, job)
except Exception: # noqa: BLE001 — persistence is best-effort
logger.debug("translation context persist skipped", exc_info=True)
return ctx
def _unload_nllb():
"""Release NLLB VRAM so TTS model can reload."""
global _nllb_model, _nllb_tokenizer
@@ -366,6 +406,27 @@ async def dub_translate(req: TranslateRequest):
)
return JSONResponse(status_code=400, content={"error": friendly})
from services import translation_quality as tq
# Two-stage quality toggles. None (old clients) = ON — an LLM
# translator is active on this branch by definition.
auto_glossary_on = req.auto_glossary if req.auto_glossary is not None else True
reflect_on = req.reflect if req.reflect is not None else True
# Stage 1 — auto-glossary: ONE pass over the full transcript for a
# theme summary + terminology map (cached per job/target/transcript),
# merged with the user's manual glossary (user entries win) and
# injected into every per-segment prompt below. With the toggle off
# the manual glossary still rides along — that costs no extra call.
auto_ctx = None
if auto_glossary_on:
auto_ctx = await loop.run_in_executor(
_cpu_pool, _resolve_translation_context,
req, client, model_name, llm_timeout, src_lang,
)
merged_terms = tq.merge_glossary(req.glossary, (auto_ctx or {}).get("terms"))
context_extra = tq.context_clause((auto_ctx or {}).get("theme", ""), merged_terms)
def _build_prompt(src_code: str, tgt_code: str) -> str:
"""Build a system prompt that resists hallucinations on small
local LLMs. Three things matter:
@@ -396,13 +457,19 @@ async def dub_translate(req: TranslateRequest):
dia_clause = ""
if req.dialect and str(req.dialect).lower().startswith(str(tgt_code).lower()[:2]):
dia_clause = dialect_clause(req.dialect)
return (
base = (
f"You are a professional dubbing translator. "
f"Translate the user's text from {src_name} into "
f"{tgt_name}.{script_clause}{dia_clause} "
f"Reply ONLY with the translated {tgt_name} text, do not "
f"add quotes, notes, headers, explanations, or commentary."
)
# Auto-glossary theme + merged terminology (user terms win) —
# every segment prompt carries the same brief, so recurring
# names/terms come out consistent across the whole dub.
if context_extra:
base = base + "\n\n" + context_extra
return base
def _translate_llm(seg):
if not seg.text or not seg.text.strip():
@@ -446,6 +513,33 @@ async def dub_translate(req: TranslateRequest):
seg.id, attempt + 1, last_err,
)
continue
# Stage 2 — reflect pass: critique→rewrite the direct
# translation into natural spoken dialogue. Returns None
# on ANY failure/timeout/divergence, in which case the
# direct translation stands — refinement can never fail
# a segment that already translated fine. The belt-and-
# braces except keeps that guarantee even if the helper
# itself ever raised: without it, the enclosing attempt
# handler would burn a retry on a segment that already
# translated successfully.
if reflect_on:
polished = None
try:
polished = tq.reflect_translation_sync(
client, model_name, llm_timeout,
source_text=seg.text,
direct_text=out_text,
source_lang=src_lang,
target_lang=tgt_code,
target_name=LANG_NAMES.get(tgt_code, tgt_code),
extra_clause=context_extra,
)
except Exception as e: # noqa: BLE001
logger.warning("reflect pass skipped for %s: %s",
seg.id, e)
if polished:
return {"id": seg.id, "text": polished,
"literal": out_text}
return {"id": seg.id, "text": out_text}
except Exception as e:
last_err = f"{type(e).__name__}: {e}"
@@ -624,6 +718,132 @@ async def dub_translate(req: TranslateRequest):
return JSONResponse(status_code=500, content={"error": str(e)})
def _stamp_duration_plan(rows, req) -> None:
"""Attach a pre-synthesis duration-plan verdict to every row (in place).
Pure planning (services/duration_planner.py): estimate the natural
speech duration of each row's FINAL text — self-calibrated from this
job's already-synthesized segments when possible — and classify it
against slot + borrowable gap using fit_planner's own caps. The verdict
rides on the row as ``plan`` so the segment table can badge tight/
impossible segments BEFORE any GPU time is spent. Informational only
generation is never blocked. Never raises.
"""
try:
from services.duration_planner import calibration_from_job, classify_segments
timed = [
s for s in req.segments
if getattr(s, "start", None) is not None and getattr(s, "end", None) is not None
]
if not timed:
return # old client — no timeline info, no plan
text_by_id = {str(r["id"]): (r.get("text") or "") for r in rows}
segs = sorted(
(
{
"id": str(s.id),
"start": float(s.start),
"end": float(s.end),
"text": text_by_id.get(str(s.id), ""),
}
for s in timed
),
key=lambda d: d["start"],
)
calib = None
total_dur = 0.0
if getattr(req, "job_id", None):
job = _get_job(req.job_id)
if job:
calib = calibration_from_job(job, req.target_lang)
total_dur = float(job.get("duration") or 0.0)
verdicts = {
v["id"]: v
for v in classify_segments(
segs, req.target_lang, calibration=calib, total_dur_s=total_dur,
)
}
for row in rows:
v = verdicts.get(str(row["id"]))
if v is None or row.get("error") or not (row.get("text") or "").strip():
continue
row["plan"] = {
"status": v["status"],
"est_dur_s": v["est_dur_s"],
"available_s": v["available_s"],
"est_overrun_s": v["est_overrun_s"],
"calibrated": v["calibrated"],
}
except Exception as e: # noqa: BLE001 — planning must never sink a translate
logger.debug("duration-plan stamping skipped: %s", e)
async def _apply_condense_pass(rows, req, loop) -> None:
"""Opt-in LLM condensation for ``impossible`` rows (in place).
Fans ``condense_for_slot`` out on the CPU pool under the same wall-clock
budget the cinematic phase uses, so a slow LLM can't hang the translate.
Suggestions land as ``plan.suggested_text`` the user applies them per
segment; the row's ``text`` is never touched here. Every failure mode
(no LLM, LLM error, divergent reply, budget) degrades to no suggestion.
"""
targets = [
row for row in rows
if (row.get("plan") or {}).get("status") == "impossible"
and (row.get("text") or "").strip() and not row.get("error")
]
if not targets:
return
try:
from services.duration_planner import calibration_from_job, condense_for_slot
calib = None
if getattr(req, "job_id", None):
job = _get_job(req.job_id)
if job:
calib = calibration_from_job(job, req.target_lang)
source_by_id = {str(s.id): s.text for s in req.segments}
sem = asyncio.Semaphore(int(os.environ.get("OMNIVOICE_LLM_CONCURRENCY", "6")))
async def _one(row):
async with sem:
res = await loop.run_in_executor(
_cpu_pool,
lambda: condense_for_slot(
row["text"],
available_s=float(row["plan"]["available_s"]),
target_lang=req.target_lang,
source_text=source_by_id.get(str(row["id"])),
calibration=calib,
),
)
if res.get("applied") and res.get("text"):
row["plan"]["suggested_text"] = res["text"]
row["plan"]["suggested_est_dur_s"] = res.get("est_dur_s")
tasks = [asyncio.ensure_future(_one(row)) for row in targets]
budget = _cinematic_budget()
done, pending = await asyncio.wait(
tasks, timeout=budget if budget and budget > 0 else None,
)
for task in pending:
task.cancel() # abandon the executor thread (#730 pattern)
for task in done:
exc = task.exception()
if exc is not None:
logger.warning("condense pass segment failed: %s", exc)
except Exception as e: # noqa: BLE001 — a suggestion pass must never sink a translate
logger.warning("condense pass skipped: %s", e)
async def _finalize_duration_plan(rows, req, loop) -> None:
"""Stamp plan verdicts on the FINAL row texts, then (opt-in) condense."""
_stamp_duration_plan(rows, req)
if getattr(req, "condense", False):
await _apply_condense_pass(rows, req, loop)
def _stamp_predicted_rate_ratio(translated, req) -> None:
"""Stamp a predicted ``rate_ratio`` on every row that has a known slot.
@@ -717,8 +937,10 @@ async def _maybe_cinematic(translated, req, src_lang, loop, *, already_llm=False
"quality_used": "fast",
**_dialect_flags(req, applied=(already_llm and bool(dialect_hint)))}
# Fast (and anything unrecognised) returns the plain translation unchanged.
# Fast (and anything unrecognised) returns the plain translation unchanged
# (plus the pre-synthesis duration-plan badges — no LLM needed for those).
if quality not in ("cinematic", "autofit"):
await _finalize_duration_plan(translated, req, loop)
return base
source_by_id: dict[str, str] = {str(s.id): s.text for s in req.segments}
@@ -738,15 +960,19 @@ async def _maybe_cinematic(translated, req, src_lang, loop, *, already_llm=False
if already_llm:
merged = []
for row in translated:
# A reflect-pass row already carries its pre-polish direct
# translation as `literal` — keep it instead of clobbering.
out = {"id": row["id"],
"text": row.get("text", "") or "",
"literal": row.get("text", "") or ""}
"literal": row.get("literal") or row.get("text", "") or ""}
if row.get("error"):
out["error"] = row["error"]
if "rate_ratio" in row:
out["rate_ratio"] = row["rate_ratio"]
merged.append(out)
await _apply_fit_pass(merged, req, slots_by_id, source_by_id, quality, loop, deadline)
# Plan AFTER the fit pass — verdicts must describe the final text.
await _finalize_duration_plan(merged, req, loop)
return {"translated": merged, "target_lang": req.target_lang,
"source_lang": src_lang, "quality_used": quality,
**_dialect_flags(req, applied=bool(dialect_hint))}
@@ -756,6 +982,7 @@ async def _maybe_cinematic(translated, req, src_lang, loop, *, already_llm=False
if not cinematic_available():
logger.warning("%s requested but no LLM configured — returning Fast result.", quality)
base["cinematic_skipped"] = "no-llm-configured"
await _finalize_duration_plan(translated, req, loop)
return base
directions: dict[str, str] = {
@@ -774,6 +1001,7 @@ async def _maybe_cinematic(translated, req, src_lang, loop, *, already_llm=False
pairs.append((seg_id, source_by_id.get(seg_id, ""), literal))
if not pairs:
await _finalize_duration_plan(translated, req, loop)
return base
refined = await cinematic_refine_many(
@@ -810,6 +1038,9 @@ async def _maybe_cinematic(translated, req, src_lang, loop, *, already_llm=False
# Phase 4.4 speech-rate fit pass — now concurrent + bounded (see helper).
await _apply_fit_pass(merged, req, slots_by_id, source_by_id, quality, loop, deadline)
# Plan AFTER the fit pass — verdicts must describe the final text.
await _finalize_duration_plan(merged, req, loop)
return {
"translated": merged,
"target_lang": req.target_lang,
+10 -2
View File
@@ -161,13 +161,18 @@ async def search_youtube(
list. Users are responsible for the licensing of whatever they import.
"""
try:
# yt-dlp is an importable module, never a PATH requirement — run it
# via the interpreter (honors the Settings → Audio tools overlay).
from services.media_tools import ytdlp_invocation
ytdlp_argv, ytdlp_env = ytdlp_invocation()
result = await spawn_subprocess(
"yt-dlp",
*ytdlp_argv,
"--dump-json",
"--remote-components", "ejs:github",
f"ytsearch{max_results}:{query}",
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
env=ytdlp_env,
)
stdout, stderr = await result.communicate()
@@ -218,8 +223,10 @@ async def download_youtube_clip(
temp_path = str(VOICE_GALLERY_DIR / f"{voice_id}.%(ext)s")
try:
from services.media_tools import ytdlp_invocation
ytdlp_argv, ytdlp_env = ytdlp_invocation()
cmd = [
"yt-dlp",
*ytdlp_argv,
"--remote-components", "ejs:github",
"-f",
"bestaudio",
@@ -239,6 +246,7 @@ async def download_youtube_clip(
*cmd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
env=ytdlp_env,
)
stdout, stderr = await result.communicate()
+221 -12
View File
@@ -12,6 +12,7 @@ import traceback
from typing import Optional
from fastapi import APIRouter, File, Form, UploadFile, HTTPException
from fastapi.responses import StreamingResponse
from pydantic import BaseModel
import sqlite3
from core.db import db_conn, ensure_schema
@@ -598,6 +599,32 @@ def _run_backend_inference(
_oom_friendly_reraise(e)
def _persist_profile_ref_text(profile_id: str, ref_text: str) -> None:
"""Cache an auto-transcribed reference transcript onto its profile row.
#1032 perf regression: profiles saved without a transcript re-ran a FULL
ASR model load + transcribe on every /generate (the #308 auto-transcribe
path). Persisting the first transcript makes subsequent generates read it
from the row like a user-entered one. The guarded UPDATE only ever fills
an empty column it can never overwrite a transcript the user typed or a
lock wrote and a failure is logged, never raised (best-effort, same
contract as the transcribe itself)."""
try:
with db_conn() as conn:
updated = conn.execute(
"UPDATE voice_profiles SET ref_text=? "
"WHERE id=? AND (ref_text IS NULL OR ref_text='')",
(ref_text, profile_id),
).rowcount
if updated:
event_bus.emit("profiles", {"action": "updated", "id": profile_id})
except Exception as e: # noqa: BLE001 — cache write must not break generate
logger.warning(
"could not persist auto-transcribed ref_text onto profile %s: %s",
profile_id, e,
)
@router.post("/generate")
async def generate_speech(
text: str = Form(...),
@@ -693,11 +720,51 @@ async def generate_speech(
raise HTTPException(status_code=400, detail=_routing["routing_reason"])
_routing_notice = routing_notice(_routing) # (status, reason) or None
# ── #1033/#1037: warm the engine under the LOAD budget, not the generate
# budget. A cold adapter lazily loads (and possibly downloads multi-GB
# weights) inside generate(), so a fresh install's first request burned
# its whole OMNIVOICE_GENERATE_TIMEOUT_S window on the download and died
# with a misleading "too heavy for the available compute" 503 (#1014
# measured it: 0% GPU util for the full 300s). Model loading gets its own,
# larger budget (OMNIVOICE_MODEL_LOAD_TIMEOUT, default 1200s) — the same
# split get_model() already has for the native engine. Once warm, this is
# a no-op per request.
if _backend is not None:
from services.model_manager import _model_load_timeout
try:
await run_on_gpu_pool_guarded(
_backend.ensure_ready,
what=f"TTS engine '{engine_id}' model load",
timeout=_model_load_timeout(),
)
# Builtin TimeoutError base, not GpuJobTimeoutError — reload-proof
# class identity (see the twin catch in openai_compat.py).
except TimeoutError as exc:
logger.warning("engine load exceeded the model-load budget: %s", exc)
raise HTTPException(
status_code=503,
detail=(
f"TTS engine '{engine_id}' did not finish loading within its "
f"model-load budget — on a first run this usually means the "
f"weight download is slow or stalled (check Settings → Models "
f"for progress), not that generation failed. Retry once the "
f"model shows as installed."
),
) from exc
ref_audio_path = None
cleanup_ref = False
used_seed = seed
resolved_profile_id = None
history_mode = None # profile.kind when a profile drives; else inferred at insert
# #1032: profile id to persist an auto-transcribed reference transcript to.
# Set only for a plain (unlocked) clone profile whose stored ref_text is
# empty — the case where every /generate re-ran a full ASR model load +
# transcribe of the same clip. Locked profiles are excluded (their ref
# audio is the locked take, and unlocking would leave a mismatched
# transcript paired with the original reference); design profiles are
# excluded (a re-render replaces the sample, stranding a stale transcript).
persist_ref_text_profile_id = None
if profile_id:
with db_conn() as conn:
@@ -743,6 +810,11 @@ async def generate_speech(
ref_audio_path = os.path.join(VOICES_DIR, row["ref_audio_path"]) if row["ref_audio_path"] else None
if not ref_text and row["ref_text"]:
ref_text = row["ref_text"]
elif ref_audio_path and not ref_text:
# Empty stored transcript → the auto-transcribe below will
# run; cache its result onto the profile so it runs ONCE,
# not on every generate (#1032 perf regression).
persist_ref_text_profile_id = profile_id
if not instruct and row["instruct"]:
instruct = row["instruct"]
if used_seed is None and row["seed"] is not None:
@@ -792,6 +864,11 @@ async def generate_speech(
except GpuJobTimeoutError as e:
logger.warning("reference transcribe hung (%s); using model ASR fallback", e)
ref_text = None
# #1032: cache the transcript onto its clone profile so the ASR model
# load + transcribe above happens once per profile, not per generate.
# Only fills an empty column — a user-entered transcript always wins.
if ref_text and persist_ref_text_profile_id:
_persist_profile_ref_text(persist_ref_text_profile_id, ref_text)
# #526: materialize a concrete seed when none was supplied (and no profile
# pinned one) so the take is reproducible and we can hand it back via the
@@ -801,6 +878,15 @@ async def generate_speech(
if used_seed is None:
used_seed = random.randint(0, 2**31 - 1)
# Engine-agnostic text normalization (junk strip, numbers→words,
# abbreviations) — AFTER `language` is fully resolved, and BEFORE the
# pronunciation dictionary so user dictionary entries operate on
# normalized text and respellings are never re-mangled (ordering rationale
# in services/text_normalization.py). Pref-gated (default ON), idempotent,
# never raises; applied exactly once per request, at this choke point.
from services.text_normalization import normalize_for_tts
text = normalize_for_tts(text, language)
# Expressive-TTS Spec 01: apply the user pronunciation dictionary + inline
# [[…]] one-off overrides to the text, here — AFTER `language` is fully
# resolved (a profile may fill it above) so per-language entries match the
@@ -905,6 +991,13 @@ async def generate_speech(
logger.warning("history write still failed after schema heal; returning audio anyway: %s", e2)
except Exception as e:
logger.warning("generation history write failed; returning audio anyway: %s", e)
# Retention cap: without it, takes (rows + WAVs in OUTPUTS_DIR) grow
# unbounded forever. Best-effort — a prune failure must never affect
# the generation that just succeeded.
try:
_prune_history_over_cap()
except Exception as e: # noqa: BLE001
logger.warning("history retention prune failed (non-fatal): %s", e)
event_bus.emit("generation_history", {"action": "created", "id": audio_id})
buffer = io.BytesIO()
@@ -977,17 +1070,101 @@ def _safe_output_path(name):
return candidate
def _remove_wav_if_unreferenced(conn, audio_path, exclude_ids=()):
"""Delete a history WAV from OUTPUTS_DIR — but only when no *other*
generation_history row still references the same file.
History WAVs are uniquely owned by their row (lock/save-as-profile COPY
into VOICES_DIR, exports copy to the user's destination), so this guard is
normally a no-op it exists so any future path that duplicates a row can
never make a delete/prune yank audio out from under a surviving take."""
if not audio_path:
return
p = _safe_output_path(audio_path)
if not p or not os.path.exists(p):
return
placeholders = ",".join("?" for _ in exclude_ids)
others = conn.execute(
"SELECT COUNT(*) FROM generation_history WHERE audio_path=?"
+ (f" AND id NOT IN ({placeholders})" if exclude_ids else ""),
(audio_path, *exclude_ids),
).fetchone()[0]
if others:
return
with contextlib.suppress(OSError):
os.remove(p)
# How many takes to keep before pruning the oldest UNstarred ones (rows + their
# WAVs). User-tunable via Settings → Storage; 0 = unlimited. The pref key is
# shared with api/routers/settings.py (the GET/PUT endpoint) — same pattern as
# perf.torch_compile_disabled, which settings.py and engine_env.py both name.
HISTORY_CAP_PREF_KEY = "generation_history_cap"
DEFAULT_HISTORY_CAP = 200
def _history_cap() -> int:
from core import prefs
try:
cap = int(prefs.get(HISTORY_CAP_PREF_KEY, DEFAULT_HISTORY_CAP))
except (TypeError, ValueError):
return DEFAULT_HISTORY_CAP
return max(0, cap)
def _prune_history_over_cap() -> int:
"""Retention: keep the newest ``_history_cap()`` takes; delete the oldest
UNstarred rows over the cap plus their WAVs (via the unreferenced guard).
Starred takes are never pruned even when they alone exceed the cap.
Returns the number of rows pruned."""
cap = _history_cap()
if cap <= 0:
return 0 # 0 = unlimited
with db_conn() as conn:
total = conn.execute("SELECT COUNT(*) FROM generation_history").fetchone()[0]
excess = total - cap
if excess <= 0:
return 0
victims = conn.execute(
"SELECT id, audio_path FROM generation_history "
"WHERE COALESCE(starred, 0)=0 ORDER BY created_at ASC LIMIT ?",
(excess,),
).fetchall()
if not victims:
return 0
victim_ids = [r["id"] for r in victims]
conn.executemany(
"DELETE FROM generation_history WHERE id=?", [(i,) for i in victim_ids]
)
for r in victims:
_remove_wav_if_unreferenced(conn, r["audio_path"], exclude_ids=victim_ids)
logger.info("history retention: pruned %d takes over the %d cap", len(victims), cap)
return len(victims)
@router.get("/history")
def list_history():
"""Newest 50 generations whose audio still exists on disk.
"""The newest 50 generations plus every starred take, newest first, kept to
rows whose audio still exists on disk.
Rows whose WAV was deleted out-of-band (cleared outputs dir, manual
cleanup) used to come back anyway and render dead players that 404 on
every fetch; prune them here so the UI never sees them again."""
Starred takes ride along past the 50-row window so a keeper can never age
off the rail. Rows whose WAV was deleted out-of-band (cleared outputs dir,
manual cleanup) used to come back anyway and render dead players that 404
on every fetch; prune them here so the UI never sees them again."""
query = (
"SELECT * FROM generation_history WHERE COALESCE(starred, 0)=1 "
"OR id IN (SELECT id FROM generation_history ORDER BY created_at DESC LIMIT 50) "
"ORDER BY created_at DESC"
)
with db_conn() as conn:
rows = conn.execute(
"SELECT * FROM generation_history ORDER BY created_at DESC LIMIT 50"
).fetchall()
try:
rows = conn.execute(query).fetchall()
except sqlite3.OperationalError:
# Same class as #710/#552: a DB that missed init or the additive
# `starred` column. Heal once and retry inside this connection.
ensure_schema()
rows = conn.execute(query).fetchall()
alive, stale_ids = [], []
for r in rows:
p = _safe_output_path(r["audio_path"]) if r["audio_path"] else None
@@ -1003,6 +1180,39 @@ def list_history():
logger.info("pruned %d stale history rows (audio file gone)", len(stale_ids))
return alive
class _StarBody(BaseModel):
starred: bool
@router.put("/history/{history_id}/starred")
def set_history_starred(history_id: str, body: _StarBody):
"""Star/unstar a take. Starred takes survive the retention cap and always
appear in GET /history regardless of the recency window."""
def _update():
with db_conn() as conn:
cur = conn.execute(
"UPDATE generation_history SET starred=? WHERE id=?",
(1 if body.starred else 0, history_id),
)
return cur.rowcount
try:
changed = _update()
except sqlite3.OperationalError as e:
# `no such column: starred` on a pre-migration DB (or the #710
# missing-table class) — heal the schema and retry once.
logger.warning("star update failed (%s); healing schema + retrying", e)
ensure_schema()
changed = _update()
if not changed:
raise HTTPException(
status_code=404,
detail="That take no longer exists — it may have been pruned or deleted.",
)
event_bus.emit("generation_history", {"action": "starred", "id": history_id})
return {"id": history_id, "starred": body.starred}
@router.delete("/history")
def clear_history():
with db_conn() as conn:
@@ -1020,11 +1230,10 @@ def clear_history():
def delete_single_history(history_id: str):
with db_conn() as conn:
row = conn.execute("SELECT audio_path FROM generation_history WHERE id=?", (history_id,)).fetchone()
if row and row["audio_path"]:
p = _safe_output_path(row["audio_path"])
if p and os.path.exists(p):
with contextlib.suppress(OSError):
os.remove(p)
conn.execute("DELETE FROM generation_history WHERE id=?", (history_id,))
if row:
# Row first, file second — the WAV goes only if no surviving take
# still references it (see _remove_wav_if_unreferenced).
_remove_wav_if_unreferenced(conn, row["audio_path"], exclude_ids=(history_id,))
event_bus.emit("generation_history", {"action": "deleted", "id": history_id})
return {"deleted": True}
+85
View File
@@ -0,0 +1,85 @@
"""Media-tools endpoints — the backend for Settings → Audio tools and the
wizard's invisible media-engine self-heal.
Every route is loopback-gated: ``custom-path`` / ``use-system`` point the app
at an arbitrary executable (an RCE primitive if remote-reachable), and the
rest mutate local state. Same contract as ``/system/set-env``.
"""
from __future__ import annotations
import logging
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from api.dependencies import require_loopback
logger = logging.getLogger("omnivoice.api")
router = APIRouter(dependencies=[Depends(require_loopback)])
class CustomPathRequest(BaseModel):
path: str
def _svc():
# Late import so a service-level failure surfaces as a 500 with detail,
# not an app-boot failure.
from services import media_tools
return media_tools
@router.get("/media-tools/status")
def media_tools_status():
"""Per-tool {ok, path, version, origin} + background-op states."""
return _svc().status()
@router.post("/media-tools/acquire")
def media_tools_acquire():
"""(Re-)fetch the pinned, checksummed static ffmpeg/ffprobe build in the
background. Idempotent; poll /media-tools/status for progress."""
return _svc().acquire_bundled()
# Literal ytdlp routes MUST register before the parametrized {tool} routes —
# FastAPI matches in declaration order, and `/media-tools/{tool}/restore`
# would otherwise swallow `/media-tools/ytdlp/restore` into a 400.
@router.post("/media-tools/ytdlp/update")
def media_tools_ytdlp_update():
"""Fetch the newest yt-dlp wheel (sha256-verified against PyPI metadata)
into the update-surviving overlay. Applies on next backend start."""
return _svc().update_ytdlp()
@router.post("/media-tools/ytdlp/restore")
def media_tools_ytdlp_restore():
"""Drop the overlay — the app-tested, locked yt-dlp takes over on next
start. Always safe (the locked install is never modified)."""
return _svc().restore_ytdlp()
@router.post("/media-tools/{tool}/custom-path")
def media_tools_custom_path(tool: str, body: CustomPathRequest):
try:
return _svc().set_custom_path(tool, body.path)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
@router.post("/media-tools/{tool}/use-system")
def media_tools_use_system(tool: str):
try:
return _svc().use_system(tool)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
except LookupError as e:
raise HTTPException(status_code=404, detail=str(e))
@router.post("/media-tools/{tool}/restore")
def media_tools_restore(tool: str):
try:
return _svc().restore_bundled(tool)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
+63 -1
View File
@@ -108,6 +108,23 @@ class SpeechRequest(BaseModel):
ge=0,
description="OmniVoice GGUF extension: long-form internal chunk threshold.",
)
# #1014: these two were silently DISCARDED before (pydantic ignores
# undeclared fields) — a 200 OK that quietly dropped the caller's quality
# knobs. Declared now and passed through, matching the native /generate
# form fields (defaults there: num_step=16, guidance_scale=2.0; the
# model's documented "quality" preset is num_step=32).
num_step: Optional[int] = Field(
default=None,
ge=1,
le=128,
description="OmniVoice extension: iterative unmasking steps (app default 16; 32 = the model's documented quality preset).",
)
guidance_scale: Optional[float] = Field(
default=None,
gt=0,
le=20,
description="OmniVoice extension: classifier-free guidance scale (app default 2.0).",
)
class TranscriptionResponse(BaseModel):
@@ -274,6 +291,10 @@ async def create_speech(req: SpeechRequest):
kw["chunk_duration"] = req.chunk_duration
if req.chunk_threshold is not None:
kw["chunk_threshold"] = req.chunk_threshold
if req.num_step is not None:
kw["num_step"] = req.num_step
if req.guidance_scale is not None:
kw["guidance_scale"] = req.guidance_scale
if req.language:
kw["language"] = req.language
if req.instruct:
@@ -311,11 +332,52 @@ async def create_speech(req: SpeechRequest):
# Not a profile ID — might be a KittenTTS preset or similar
kw["voice"] = voice
# Engine-agnostic text normalization (junk strip, numbers→words,
# abbreviations) at this route's text→engine choke point — the same
# pre-pass as /generate, applied exactly once per request. `req.language`
# is everything this route knows about the language (None → universal
# safety filters only). Pref-gated (default ON), idempotent, never raises.
from services.text_normalization import normalize_for_tts
text = normalize_for_tts(req.input, req.language)
# ── #1033/#1037/#1014: warm the engine under the LOAD budget before the
# generate clock starts. The T4 verification (#1014) measured a fresh
# install's first /v1/audio/speech burning its whole 300s generate budget
# on the multi-GB checkpoint download (0% GPU util throughout) and dying
# with a misleading "too heavy for the available compute" error. Model
# loading gets OMNIVOICE_MODEL_LOAD_TIMEOUT (default 1200s); once warm
# this is a per-request no-op.
from services.model_manager import _model_load_timeout
try:
await run_on_gpu_pool_guarded(
backend.ensure_ready,
what=f"TTS engine '{backend.id}' model load",
timeout=_model_load_timeout(),
)
# Catch the BUILTIN TimeoutError base, not GpuJobTimeoutError by name:
# several tests reload services.model_manager mid-suite, so a class
# imported at call time can differ in identity from the one the guard
# (bound at this module's import) actually raises — the except would
# silently miss. The builtin base has one identity forever. (Caught by
# this exact test failing CI-only, in full-suite order.)
except TimeoutError as e:
logger.warning("engine load exceeded the model-load budget: %s", e)
raise HTTPException(
status_code=503,
detail=(
f"TTS engine '{backend.id}' did not finish loading within its "
f"model-load budget — on a first run this usually means the weight "
f"download is slow or stalled (check Settings → Models for "
f"progress), not that generation failed. Retry once the model "
f"shows as installed."
),
) from e
try:
# Bounded + pool-reset on hang so a wedged TTS request can't starve the
# GPU pool and brick the backend (#730 class).
wav, sr = await run_on_gpu_pool_guarded(
lambda: _run_tts(backend, req.input, kw), what="OpenAI TTS generate")
lambda: _run_tts(backend, text, kw), what="OpenAI TTS generate")
except Exception as e:
logger.exception("OpenAI TTS failed: %s", e)
raise HTTPException(status_code=500, detail=str(e))
+71 -2
View File
@@ -77,8 +77,17 @@ def clear_hf_token(also_clear_hf_cli: bool = Query(False)):
@router.get("/hf-token/state")
def get_hf_token_state():
"""3-source HF token cascade state for the Settings UI."""
def get_hf_token_state(fresh: bool = Query(False)):
"""3-source HF token cascade state for the Settings UI.
``fresh=1`` drops the resolver's whoami validation cache first so the
response re-runs whoami for every source this is what the panel's
"Test now" button sends. Plain GETs (panel mounts) keep the 300s cache
so repeat Settings visits don't hammer the HF API.
"""
from services import token_resolver
if fresh:
token_resolver.invalidate_cache()
return _state_response()
@@ -124,6 +133,46 @@ def set_torch_compile_disabled(body: _TorchCompileBody):
return _torch_compile_state()
# ── Generation-history retention (Studio takes rail) ──────────────────────
class _HistoryRetentionBody(BaseModel):
cap: int = Field(
...,
ge=0,
le=100000,
description="Max takes kept before the oldest UNstarred ones (rows + WAVs) are pruned; 0 = unlimited",
)
def _history_retention_state() -> dict:
from api.routers.generation import DEFAULT_HISTORY_CAP, _history_cap
return {"cap": _history_cap(), "default": DEFAULT_HISTORY_CAP}
@router.get("/history-retention")
def get_history_retention():
"""Current generation-history retention cap (Settings → Storage)."""
return _history_retention_state()
@router.put("/history-retention")
def set_history_retention(body: _HistoryRetentionBody):
"""Persist the retention cap. Enforced after every generation: the oldest
unstarred takes over the cap are pruned (rows + their audio files);
starred takes are never pruned. 0 disables pruning entirely."""
from core import prefs
from api.routers.generation import HISTORY_CAP_PREF_KEY
try:
prefs.set_(HISTORY_CAP_PREF_KEY, int(body.cap))
except Exception:
logger.exception("set_history_retention failed")
raise HTTPException(status_code=500, detail="Failed to persist setting")
return _history_retention_state()
# ── Dictation refinement (parity program Wave 2.1 / Spec 3 phase 2) ───────
@@ -685,6 +734,26 @@ async def get_storage_report(refresh: bool = Query(False)):
raise HTTPException(status_code=500, detail="Failed to compute storage report")
@router.post("/storage/temp/clear")
async def clear_temp_files():
"""Delete OmniVoice-owned temp files (Settings → Storage → Temporary files).
Removes only the ``omnivoice*`` entries in the OS temp dir the exact
population the storage report's "temp" category counts — and invalidates
the cached report so the next scan reflects the reclaimed space. Partial
failures (files held open by a running job) are returned per entry.
"""
from services import storage_report
try:
result = await asyncio.to_thread(storage_report.clear_temp)
storage_report.clear_cache()
return result
except Exception:
logger.exception("clear temp files failed")
raise HTTPException(status_code=500, detail="Failed to clear temporary files")
# ── HF mirror endpoint (parity program Wave 4.3 / §R4 c) ──────────────────
# Restricted-network users (e.g. behind the Great Firewall) need to point
# huggingface_hub at a mirror. HF reads HF_ENDPOINT at import time, so a
+16 -54
View File
@@ -3,7 +3,9 @@
Extracted from the monolithic ``setup.py``.
- ``GET /setup/status`` missing-model gate for boot screen
- ``GET /setup/preflight`` system health check (OS, RAM, GPU, ffmpeg)
- ``GET /setup/preflight`` system health check (OS, RAM, disk, GPU, network
genuine user facts only; the media engine (ffmpeg/ffprobe/yt-dlp) is an
internal concern that self-heals via ``services.media_tools``)
- ``POST /setup/warmup`` background model pre-load
"""
from __future__ import annotations
@@ -12,7 +14,6 @@ import asyncio
import logging
import os
import platform as _platform
import shutil as _shutil
import sys
from fastapi import APIRouter
@@ -280,59 +281,19 @@ def preflight():
f"Fix write permissions on {cache} or point HF_HOME elsewhere.",
})
# ── FFmpeg
ffmpeg_path = None
# ── Media engine (ffmpeg/ffprobe/yt-dlp) — deliberately NOT a check row.
# These are internal dependencies the app provisions for itself, not user
# facts: when the resolution chain has no tier at all, preflight kicks the
# bundled acquisition in the background and the wizard shows a quiet
# progress line (a failure card only if that fails — with Retry / use a
# system copy). yt-dlp is an importable locked module and never appears.
# Power users manage all three in Settings → Audio tools.
media_tools = None
try:
from services.ffmpeg_utils import find_ffmpeg
ffmpeg_path = find_ffmpeg()
except Exception as e:
checks.append({
"id": "ffmpeg", "label": "FFmpeg", "status": "fail",
"detail": str(e)[:200],
"fix": "Install ffmpeg via your package manager "
"(brew install ffmpeg / apt install ffmpeg / choco install ffmpeg).",
})
else:
checks.append({
"id": "ffmpeg", "label": "FFmpeg", "status": "pass",
"detail": ffmpeg_path, "fix": None,
})
# ── FFprobe
ffprobe_path = None
try:
from services.ffmpeg_utils import find_ffprobe
ffprobe_path = find_ffprobe()
except Exception:
pass
if ffprobe_path:
checks.append({
"id": "ffprobe", "label": "FFprobe", "status": "pass",
"detail": ffprobe_path, "fix": None,
})
else:
checks.append({
"id": "ffprobe", "label": "FFprobe", "status": "warn",
"detail": "Not bundled alongside ffmpeg.",
"fix": "File-probe endpoint (/tools/probe) will 501. "
"Install system ffmpeg (includes ffprobe) to enable it.",
})
# ── yt-dlp
yt_dlp_path = _shutil.which("yt-dlp")
if yt_dlp_path:
rc_ytv, yt_ver = _run_cmd([yt_dlp_path, "--version"], timeout=3.0)
yt_version = yt_ver.strip() if rc_ytv == 0 else "unknown"
checks.append({
"id": "yt-dlp", "label": "yt-dlp", "status": "pass",
"detail": f"{yt_dlp_path} (v{yt_version})", "fix": None,
})
else:
checks.append({
"id": "yt-dlp", "label": "yt-dlp", "status": "warn",
"detail": "Not found in system PATH.",
"fix": "YouTube clip downloads in Voice Gallery will fail. Download the standalone binary from https://github.com/yt-dlp/yt-dlp/releases and place it in your PATH.",
})
from services.media_tools import summary as _media_summary
media_tools = _media_summary(auto_acquire=True)
except Exception as exc: # never break preflight on the media engine
logger.warning("preflight media_tools summary failed: %s", exc)
# ── GPU
gpu = _detect_gpu()
@@ -492,6 +453,7 @@ def preflight():
"disk_free_gb": round(free, 1),
},
"gpu_routing": gpu_routing,
"media_tools": media_tools,
}
+8 -6
View File
@@ -629,15 +629,17 @@ def system_notifications():
notes.append({
"id": "ffmpeg-missing",
"level": "error",
"title": "ffmpeg not found",
"title": "Media engine unavailable",
"message": (
"Video processing, audio conversion, and dubbing require ffmpeg. "
"Install it with: brew install ffmpeg (macOS) or apt install ffmpeg (Linux)."
"Video processing, audio conversion, and dubbing need the "
"media engine (ffmpeg), which the app normally provisions "
"itself. Open Settings > Audio tools and press Restore "
"bundled to re-download it, or point it at a system copy."
),
"action": {
"label": "Install guide",
"type": "link",
"target": "https://ffmpeg.org/download.html",
"label": "Open Audio tools",
"type": "settings-tab",
"target": "audio-tools",
},
})
+16 -1
View File
@@ -136,7 +136,10 @@ async def ws_tts(websocket: WebSocket):
kw["emo_text"] = data["emo_text"]
if data.get("emo_audio"):
kw["emo_audio"] = data["emo_audio"]
if data.get("emo_alpha") != 1.0:
# Default 1.0 when absent: a missing key must not trip the
# `!= 1.0` branch into a KeyError (any minimal request that
# omitted emo_alpha got an error frame instead of audio).
if data.get("emo_alpha", 1.0) != 1.0:
kw["emo_alpha"] = data["emo_alpha"]
# Resolve voice profile
@@ -168,6 +171,18 @@ async def ws_tts(websocket: WebSocket):
except Exception:
kw["voice"] = voice
# Engine-agnostic text normalization (junk strip,
# numbers→words, abbreviations) — the same pre-pass as
# /generate, applied exactly ONCE per request, on the whole
# text BEFORE the sentence chunker fans it out (so per-sentence
# generates never re-normalize, and expanded abbreviations
# can't confuse the sentence splitter). The request's
# `language` is all this route knows (None → universal safety
# filters only). Pref-gated (default ON), idempotent, never
# raises.
from services.text_normalization import normalize_for_tts
text = normalize_for_tts(text, data.get("language"))
# Wave 1.4: split the request into sentences so the first
# sentence's audio streams while later sentences are still
# synthesizing — this is the time-to-first-audio win. The
+5
View File
@@ -164,6 +164,11 @@ class PreflightResponse(BaseModel):
# Explicit field (PreflightResponse has no extra="allow") so the verdict
# survives serialization instead of being silently dropped.
gpu_routing: GpuRouting | None = None
# Media-engine verdict (ffmpeg/ffprobe) — NOT a check row: an internal
# dependency the app provisions for itself. Shape: {ready, acquire:
# {state, progress, error}}. The wizard renders a quiet progress line /
# failure card from it instead of "install ffmpeg" system requirements.
media_tools: dict | None = None
class InstallModelRequest(BaseModel):
+1
View File
@@ -70,6 +70,7 @@ _BASE_SCHEMA = """
duration_seconds REAL,
generation_time REAL,
seed INTEGER DEFAULT NULL,
starred INTEGER DEFAULT 0,
created_at REAL,
FOREIGN KEY (profile_id) REFERENCES voice_profiles(id)
);
+24 -8
View File
@@ -88,17 +88,33 @@ def _check_device() -> dict:
def _check_ffmpeg() -> dict:
"""Media engine (ffmpeg + ffprobe) — an internal dependency the app
bundles/acquires itself, so a failure here means the self-heal also has
nothing to work with (and the hint says where the controls live)."""
ffmpeg = ffprobe = None
try:
from services.ffmpeg_utils import find_ffmpeg
path = find_ffmpeg()
from services.ffmpeg_utils import find_ffmpeg, find_ffprobe
ffmpeg = find_ffmpeg()
ffprobe = find_ffprobe()
except Exception:
path = None
if path:
return _check("ffmpeg", "ffmpeg", OK, str(path))
pass
if ffmpeg and ffprobe:
return _check("ffmpeg", "Media engine (ffmpeg)", OK,
f"ffmpeg: {ffmpeg}; ffprobe: {ffprobe}")
if ffmpeg:
return _check(
"ffmpeg", "Media engine (ffmpeg)", WARN,
f"ffmpeg: {ffmpeg}; ffprobe missing",
"Media probing (Smart Fit, file inspection) is degraded. Open "
"Settings > Audio tools and press Restore bundled to fetch the "
"app's own ffprobe, or point it at a system copy there.",
)
return _check(
"ffmpeg", "ffmpeg", FAIL,
"not found on PATH or FFMPEG_PATH",
"Dubbing and audio conversion need ffmpeg: brew install ffmpeg (macOS), apt install ffmpeg (Linux), or set the path in Settings > General.",
"ffmpeg", "Media engine (ffmpeg)", FAIL,
"no runnable ffmpeg in any tier (sidecar, bundled, system, custom)",
"Dubbing and audio conversion are unavailable. The app normally "
"provisions ffmpeg itself — open Settings > Audio tools and press "
"Restore bundled (needs network once), or choose a system copy there.",
)
+12
View File
@@ -46,6 +46,7 @@ _HINTS: dict[str, str] = {
"UNSUPPORTED_VIDEO_URL": "This link isn't a directly downloadable video. Paste a direct video page (e.g. a youtube.com/watch?v=… or douyin.com/video/<id> link), not a share/profile/feed link — or download the file and drop it in directly.",
"VIDEO_DOWNLOAD_NETWORK": "The connection to the video server dropped mid-download (often a transient CDN/network blip or a regional rate-limit). Just retry — OmniVoice already cleaned up the partial download. If it keeps failing, check your network/VPN.",
"BROKEN_VENV": "The Python backend environment was moved or damaged. OmniVoice rebuilds it automatically on the next launch; if it keeps failing, use Clean & Retry on the setup screen.",
"MODEL_CACHE_CORRUPT": "The model cache had broken file links — snapshot entries that no longer point at their downloaded data (interrupted renames or antivirus interference can cause this). OmniVoice repairs this automatically and retries the load once. If the error persists, quit OmniVoice, delete the model's models--<org>--<name> folder inside the Hugging Face cache, and restart — the model re-downloads automatically.",
# HF_MIRROR_UNREACHABLE has a DYNAMIC hint (it names the configured mirror)
# — see hf_mirror_hint(); build_failure special-cases it.
}
@@ -231,6 +232,17 @@ def classify(reason: str) -> str:
# transformers + site-packages markers, which this signature lacks.
if "errno 22" in low:
return "OS_INVALID_ARGUMENT"
# An HF cache whose snapshot entries don't resolve (dangling symlinks /
# zero-byte stand-ins): transformers reports the weights missing ("does
# not appear to have a file named pytorch_model.bin or model.safetensors")
# even though the blobs are fully on disk. model_manager self-heals this
# (delete broken entries → snapshot_download → retry once); the class here
# covers both the raw transformers wording (any load surface can leak it)
# and OmniVoice's own repair messages, so the user-facing error and the
# auto bug report name the class and its automatic repair.
if ("does not appear to have a file named" in low
or "broken file link" in low):
return "MODEL_CACHE_CORRUPT"
if (
"could not import module" in low
or "autofeatureextractor" in low
+1 -1
View File
@@ -24,7 +24,7 @@ from pathlib import Path
# tests/test_app_version.py::test_all_version_files_in_lockstep and bumped by
# release.yml's version-bump job, so it stays equal to
# pyproject/tauri.conf/Cargo/package.json.
_FALLBACK_VERSION = "0.3.14"
_FALLBACK_VERSION = "0.3.17"
def _fallback_version() -> str:
+14
View File
@@ -195,6 +195,18 @@ try:
except Exception:
pass # prefs.json missing or broken — fine on first run
# ── Activate the yt-dlp user-update overlay (Settings → Audio tools) ──────
# Must run before anything imports yt_dlp so a user-updated version (stored
# under DATA_DIR, surviving app updates and uv drift syncs) wins over the
# locked wheel. Best-effort: a broken overlay must never block startup.
try:
from services.media_tools import activate_ytdlp_overlay
activate_ytdlp_overlay()
except Exception:
# Best-effort by design: a broken/corrupt overlay must never block
# startup — the locked wheel on sys.path is the fallback.
pass
warnings.filterwarnings("ignore", category=UserWarning)
torchaudio.set_audio_backend("soundfile")
@@ -375,6 +387,7 @@ from api.routers import (
longform_jobs,
pronunciation, # Expressive-TTS Spec 01: user pronunciation dictionary
settings as settings_router, # Phase 1 AUTH-03: HF token save/clear/state
media_tools as media_tools_router, # Audio tools: ffmpeg/ffprobe/yt-dlp management
)
from utils import hf_progress
@@ -1045,6 +1058,7 @@ app.include_router(audiobook.router)
app.include_router(longform_jobs.router)
app.include_router(pronunciation.router) # Expressive-TTS Spec 01: pronunciation dictionary
app.include_router(settings_router.router) # Phase 1 AUTH-03 endpoints
app.include_router(media_tools_router.router) # Settings → Audio tools + wizard media-engine self-heal
from api.routers import mcp_bindings as _mcp_bindings_router # noqa: E402
app.include_router(_mcp_bindings_router.router) # Wave 2.2 per-agent voice bindings
@@ -0,0 +1,65 @@
"""Generation takes: starred flag on generation_history
Revision ID: 0009_generation_history_starred
Revises: 0008_pronunciation_dictionary
Create Date: 2026-07-10 00:00:00.000000
Adds ``generation_history.starred INTEGER DEFAULT 0`` the "keep this
take" flag behind the Studio takes rail. Starred takes are exempt from the
retention cap that prunes old generations, and star/unstar round-trips through
``PUT /history/{id}/starred``.
Additive + idempotent (guarded by PRAGMA table_info, matching 0002/0003), so
re-running on a fresh-install DB where ``_BASE_SCHEMA`` already declares the
column is a no-op (Backward-compatible project data constraint). The same
column is mirrored into ``core/db.py::_BASE_SCHEMA`` so fresh installs and
migrated DBs converge on an identical end-state and DBs where alembic can't
run at all pick it up via ``_reconcile_additive_columns`` (the #552/#547
self-heal), the dual-path discipline.
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "0009_generation_history_starred"
down_revision: Union[str, None] = "0008_pronunciation_dictionary"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def _has_column(table: str, column: str) -> bool:
bind = op.get_bind()
rows = bind.execute(sa.text(f"PRAGMA table_info({table})")).fetchall()
return any(r[1] == column for r in rows)
def _has_table(name: str) -> bool:
bind = op.get_bind()
row = bind.execute(
sa.text("SELECT name FROM sqlite_master WHERE type='table' AND name=:n"),
{"n": name},
).fetchone()
return row is not None
def upgrade() -> None:
# A DB that somehow missed init has no generation_history at all — the
# startup self-heal (#710) creates it with the column already present, so
# ALTERing here would be both impossible and unnecessary.
if not _has_table("generation_history"):
return
if not _has_column("generation_history", "starred"):
# nullable + DEFAULT 0 to byte-match _BASE_SCHEMA's declaration
# (`starred INTEGER DEFAULT 0`) — the dual-path convergence test
# compares table shape between a migrated DB and a fresh install.
op.add_column(
"generation_history",
sa.Column("starred", sa.Integer(), nullable=True, server_default="0"),
)
def downgrade() -> None:
if _has_table("generation_history") and _has_column("generation_history", "starred"):
op.drop_column("generation_history", "starred")
+22
View File
@@ -122,6 +122,12 @@ class TranslateSegment(BaseModel):
# Available time slot (end - start, seconds) for rate-ratio prediction
# and the cinematic slot-fit pass. Same silent-drop fix as `direction`.
slot_seconds: Optional[float] = None
# Timeline position (seconds) — lets the duration planner borrow silence
# from the gap to the NEXT segment when classifying fits/tight/impossible
# (services/duration_planner.py). Optional: old clients that only send
# slot_seconds still get rate_ratio badges, just no plan verdicts.
start: Optional[float] = None
end: Optional[float] = None
class TranslateRequest(BaseModel):
segments: List[TranslateSegment]
@@ -137,6 +143,22 @@ class TranslateRequest(BaseModel):
# voseo: "vos sos" instead of "tú eres"). Non-LLM providers (Argos, NLLB,
# Google) can't honor it; the response then carries dialect_applied=false.
dialect: Optional[str] = None
# Two-stage LLM translation quality (provider="openai" only; MT engines
# ignore both). None = default ON for the LLM engine.
# auto_glossary — one up-front LLM pass over the full transcript extracts
# a theme summary + terminology map, merged with `glossary` (user
# entries win) and injected into every per-segment prompt.
# reflect — per-segment critique→rewrite polish after the direct
# translation (2 extra LLM calls per segment; failures silently keep
# the direct translation).
auto_glossary: Optional[bool] = None
reflect: Optional[bool] = None
# Opt-in LLM condensation (default OFF): for segments the duration
# planner classifies "impossible", ask the configured LLM for a shorter
# meaning-preserving rewrite and attach it as plan.suggested_text — a
# per-segment suggestion the user applies manually, never auto-applied.
# No LLM configured / LLM failure → silently no suggestion.
condense: Optional[bool] = False
class DubIngestUrlRequest(BaseModel):
url: str
+48
View File
@@ -29,6 +29,7 @@ import os
import re
import threading
from abc import ABC, abstractmethod
from collections import OrderedDict
from typing import Optional
logger = logging.getLogger("omnivoice.asr")
@@ -2026,6 +2027,38 @@ def get_active_asr_backend(*, asr_pipe=None) -> ASRBackend:
return cls()
# ── Reference-transcript cache (#1032) ──────────────────────────────────────
# `get_active_asr_backend()` returns a FRESH backend instance per call for the
# whisper family, so every `transcribe_reference` used to reload whisper
# weights from scratch — a multi-second (CPU: tens of seconds) hit on EVERY
# /generate whose reference clip has no stored transcript (#308 introduced the
# call; profiles saved without a transcript hit it per request). The reference
# audio is identical across those requests, so cache the *transcript* keyed by
# the file's content hash: no model or VRAM is held, repeated generates with
# the same clip skip ASR entirely. Bounded LRU; failures (None) are never
# cached so a transient ASR problem still retries next request.
_REF_TRANSCRIPT_CACHE_MAX = 64
_ref_transcript_cache: "OrderedDict[str, str]" = OrderedDict()
_ref_transcript_lock = threading.Lock()
def _ref_audio_fingerprint(audio_path: str) -> str | None:
"""sha256 of the clip's bytes, or None when unreadable (→ no caching).
Content-keyed (not path-keyed) because ad-hoc clone uploads land in a new
NamedTemporaryFile per request the path changes, the bytes don't.
Reference clips are seconds long, so hashing is negligible next to ASR."""
import hashlib
try:
h = hashlib.sha256()
with open(audio_path, "rb") as f:
for chunk in iter(lambda: f.read(1 << 20), b""):
h.update(chunk)
return h.hexdigest()
except OSError:
return None
def transcribe_reference(audio_path: str) -> str | None:
"""Transcribe a voice-clone reference clip with the active ASR backend.
@@ -2037,7 +2070,16 @@ def transcribe_reference(audio_path: str) -> str | None:
the model-attached pipeline is only reached when it is genuinely the last
resort. Returns ``None`` on any failure callers pass ``ref_text=None``
through and the model's built-in fallback still gets its chance.
Results are cached by audio content (#1032) — see the cache notes above.
"""
fingerprint = _ref_audio_fingerprint(audio_path)
if fingerprint is not None:
with _ref_transcript_lock:
cached = _ref_transcript_cache.get(fingerprint)
if cached is not None:
_ref_transcript_cache.move_to_end(fingerprint)
return cached
try:
backend = get_active_asr_backend()
except Exception as e: # noqa: BLE001 — never let ASR break generation
@@ -2061,6 +2103,12 @@ def transcribe_reference(audio_path: str) -> str | None:
(seg.get("text") or "").strip() for seg in result.get("segments", [])
)
text = (text or "").strip()
if text and fingerprint is not None:
with _ref_transcript_lock:
_ref_transcript_cache[fingerprint] = text
_ref_transcript_cache.move_to_end(fingerprint)
while len(_ref_transcript_cache) > _REF_TRANSCRIPT_CACHE_MAX:
_ref_transcript_cache.popitem(last=False)
return text or None
+39
View File
@@ -147,6 +147,45 @@ def normalize_audio(audio_tensor, target_dBFS=-2.0):
return audio_tensor
def trim_trailing_silence(
audio_tensor: torch.Tensor,
sample_rate: int,
keep_tail_s: float = 0.3,
) -> torch.Tensor:
"""Trim trailing near-silence from a generated clip, keeping a short
natural tail of ``keep_tail_s`` seconds after the last voiced sample.
Amplitude-based SILENCE trim only no content analysis of any kind.
Uses the same -50 dBFS silence floor as :func:`normalize_audio`: the last
sample above that floor marks the end of speech, and everything more than
``keep_tail_s`` past it is dropped.
Guaranteed no-op cases (input returned as-is, same object):
the trailing quiet span is already ``keep_tail_s`` (clean output);
the entire clip sits below the floor (dead render downstream
dead-render guards own that case, we must not shrink their evidence);
empty input.
Accepts ``(n,)`` or ``(channels, n)`` tensors; the returned tensor keeps
the input's shape convention.
"""
if audio_tensor.numel() == 0:
return audio_tensor
# -50 dBFS ≈ 0.00316 linear — matches normalize_audio's silence floor.
floor = 10 ** (-50.0 / 20.0)
envelope = torch.abs(audio_tensor)
if envelope.ndim > 1:
envelope = envelope.amax(dim=tuple(range(envelope.ndim - 1)))
voiced = torch.nonzero(envelope > floor)
if voiced.numel() == 0:
return audio_tensor
last_voiced = int(voiced[-1].item())
end = last_voiced + 1 + int(keep_tail_s * sample_rate)
if end >= audio_tensor.shape[-1]:
return audio_tensor
return audio_tensor[..., :end]
def apply_effects_chain(audio_tensor, sample_rate: int, chain: list[dict]) -> torch.Tensor:
"""Apply a chain of named effects to an audio tensor.
+20 -7
View File
@@ -101,6 +101,7 @@ def synthesize_chapter(
*,
crossfade_ms: int = 50,
lexicon: Optional[dict] = None,
segment_cache: Optional["object"] = None,
):
"""Render a chapter's spans to one waveform via an injected ``synth``.
@@ -111,6 +112,12 @@ def synthesize_chapter(
crossfaded; inter-span ``pause_ms_after`` becomes silence. ``lexicon`` (when
given) respells each span's text before chunking so the engine pronounces
tricky words correctly; a ``None``/empty lexicon is a no-op pass-through.
``segment_cache`` (when given a :class:`services.longform_render.
SegmentCache`) is consulted per spoken span: a cached segment WAV is reused
instead of synthesizing, and every freshly rendered span is stored the
moment it finishes so a one-sentence edit re-renders one segment and an
interrupted chapter resumes from its finished segments. Pauses are
synthesized silence and never touch the cache.
Returns ``(audio_tensor, duration_seconds)``. torch + chunked_tts are
imported lazily so this module stays import-light for the pure parser path.
@@ -122,13 +129,19 @@ def synthesize_chapter(
items: list = [] # ("a", tensor) for audio, ("s", n_samples) for silence
for span in spans:
if span.text:
chunks = split_text_into_chunks(apply_lexicon(span.text, lexicon))
rendered = [synth(c, span.voice_id, span.speed) for c in chunks]
rendered = [r for r in rendered if r is not None and getattr(r, "numel", lambda: 0)()]
if len(rendered) == 1:
items.append(("a", rendered[0]))
elif rendered:
items.append(("a", concatenate_audio_chunks(rendered, sample_rate, crossfade_ms=crossfade_ms)))
audio = segment_cache.load(span) if segment_cache is not None else None
if audio is None:
chunks = split_text_into_chunks(apply_lexicon(span.text, lexicon))
rendered = [synth(c, span.voice_id, span.speed) for c in chunks]
rendered = [r for r in rendered if r is not None and getattr(r, "numel", lambda: 0)()]
if len(rendered) == 1:
audio = rendered[0]
elif rendered:
audio = concatenate_audio_chunks(rendered, sample_rate, crossfade_ms=crossfade_ms)
if audio is not None and segment_cache is not None:
segment_cache.store(span, audio)
if audio is not None:
items.append(("a", audio))
if span.pause_ms_after > 0:
n = int(sample_rate * span.pause_ms_after / 1000.0)
if n > 0:
+329
View File
@@ -0,0 +1,329 @@
"""Pre-synthesis duration planning for dub segments.
The Smart Fit planner (services/fit_planner.py) reconciles dubbed audio
with the timeline AFTER synthesis by then a doomed segment has already
burned GPU time and can only be sped up or trimmed. This module predicts
BEFORE TTS whether a translated segment can possibly fit its slot, so the
UI can badge it (and optionally offer a shorter rewrite) while the text is
still cheap to change. It never blocks generation it informs.
Three pieces, all pure and unit-testable:
1. **Estimator** predict the natural speech duration of target-language
text. Self-calibrating: segments already synthesized in this job carry
``(chars, natural duration)`` records (written by dub_generate for every
natural-rate strategy), and the median chars-per-second of those is a
far better predictor for *this* voice/engine/language than any table.
With no (or too little) calibration data it falls back to the
conservative static per-language rate table in ``services.speech_rate``
(the same one the rate-ratio badge uses).
2. **Classifier** per segment, compare the estimate against the
*available* time: the slot plus silence borrowable from the gap to the
next segment (mirroring fit_planner's slack absorption, but with a
deliberate cap see ``GAP_BORROW_MAX_S``). The verdict thresholds are
derived from the SAME ``FitParams`` caps fit_planner enforces, so:
fits need max_audio_only_rate absorbed imperceptibly
tight need what the caps absorb audible speed-up and/or
video slow-down
impossible beyond the caps fit_planner will trim
3. **Condensation** (optional, caller-gated) for ``impossible`` segments,
ask the configured LLM for a meaning-preserving shorter rewrite
targeting the available duration. Strictly best-effort: no LLM, an LLM
error, or a divergent reply all degrade to a no-op.
No I/O, no torch; the only side-effectful function is ``condense_for_slot``
(network LLM call), which callers opt into explicitly.
"""
from __future__ import annotations
import logging
from dataclasses import dataclass
from typing import Iterable, Optional
from services.fit_planner import MAX_AUDIO_RATE_HARD, FitParams
from services.llm_backend import OffBackend, get_active_llm_backend
from services.speech_rate import expected_duration
# Shared LLM-output divergence guard (target-script + length window +
# critique-echo) — same seam speech_rate's Autofit pass uses.
from services.translator import refine_output_ok
logger = logging.getLogger("omnivoice.duration_planner")
# LLM Skills registry id — condensation is the same "make the line fit its
# slot" skill family as the Autofit pass, so it routes (and can be disabled)
# through the same Settings → LLM Skills entry.
_SKILL_ID = "slot_fitting"
# ── Calibration ─────────────────────────────────────────────────────────
# A calibration only counts once this many usable samples exist — below
# that, one odd segment (a sound effect, a mumbled clone ref) would swing
# the estimate more than the static table's error.
MIN_CALIBRATION_SAMPLES = 3
# Per-sample sanity floor: shorter/tinier segments carry more silence
# padding and TTS ramp-up than speech, so their chars/sec is noise.
MIN_SAMPLE_DUR_S = 0.4
MIN_SAMPLE_CHARS = 4
# How far a segment may borrow into the silent gap before the next segment
# (or the video tail). fit_planner itself absorbs the WHOLE gap, so this cap
# makes the pre-synthesis verdict deliberately conservative: a huge gap
# (scene change, music bed) is real slack at mix time, but planning speech
# to sprawl seconds past its slot is rarely what the user wants — and the
# estimate is fuzzy enough that promising it would over-sell.
GAP_BORROW_MAX_S = 3.0
@dataclass(frozen=True)
class Calibration:
"""Observed speech rate for one (job, language) pair."""
cps: float # chars per second at natural TTS rate
samples: int # how many segments backed it
def calibrate_cps(samples: Iterable[tuple[float, float]]) -> Optional[Calibration]:
"""Derive a chars-per-second calibration from ``(chars, natural_dur_s)``
pairs of already-synthesized segments.
Median of the per-segment rates robust against the occasional outlier
(a segment that's mostly a breath, an engine hiccup) that would drag a
mean. Returns None when fewer than ``MIN_CALIBRATION_SAMPLES`` usable
samples exist; callers then fall back to the static table.
"""
rates: list[float] = []
for chars, dur in samples:
try:
chars = float(chars)
dur = float(dur)
except (TypeError, ValueError):
continue
if dur >= MIN_SAMPLE_DUR_S and chars >= MIN_SAMPLE_CHARS:
rates.append(chars / dur)
if len(rates) < MIN_CALIBRATION_SAMPLES:
return None
rates.sort()
n = len(rates)
mid = n // 2
median = rates[mid] if n % 2 else (rates[mid - 1] + rates[mid]) / 2.0
if median <= 0:
return None
return Calibration(cps=median, samples=n)
def calibration_from_job(job: dict, lang: str) -> Optional[Calibration]:
"""Build a Calibration from the ``seg_natural_durs_by_lang`` records
dub_generate persists on the job. Tolerates any legacy/partial shape."""
try:
recs = (job.get("seg_natural_durs_by_lang") or {}).get(lang) or {}
return calibrate_cps(
(r.get("chars", 0), r.get("dur", 0))
for r in recs.values()
if isinstance(r, dict)
)
except Exception as e: # noqa: BLE001 — calibration is best-effort by design
logger.debug("calibration_from_job skipped: %s", e)
return None
# ── Estimator ───────────────────────────────────────────────────────────
def estimate_natural_duration(
text: str, lang: str, calibration: Optional[Calibration] = None,
) -> float:
"""Predicted natural-rate speech duration (seconds) of ``text``.
Calibrated rate when available, else the static per-language table
(``speech_rate.expected_duration``, 13 cps default for unknown codes).
"""
text = (text or "").strip()
if not text:
return 0.0
if calibration is not None and calibration.cps > 0:
return len(text) / calibration.cps
return expected_duration(text, lang)
# ── Classifier ──────────────────────────────────────────────────────────
def absorb_caps(params: FitParams) -> tuple[float, float]:
"""(fits_cap, absorb_cap) need-ratios aligned with fit_planner.
``fits_cap``: up to here the audio-only speed-up is imperceptible.
``absorb_cap``: up to here fit_planner's knobs absorb the overrun
(audio cap × video cap in hybrid mode; the legacy hard audio ceiling
when video retiming is off). Beyond it, fit_planner trims.
"""
if params.allow_video_retime:
return params.max_audio_only_rate, params.audio_rate_cap * params.video_slow_cap
return params.max_audio_only_rate, MAX_AUDIO_RATE_HARD
def classify_segments(
segments: list[dict],
target_lang: str,
*,
calibration: Optional[Calibration] = None,
fit_params: Optional[FitParams] = None,
total_dur_s: float = 0.0,
gap_borrow_max_s: float = GAP_BORROW_MAX_S,
) -> list[dict]:
"""Classify each segment's translated text against its timeline slot.
``segments``: chronological dicts with ``id``, ``start``, ``end``
(seconds) and ``text`` (the translated text about to be synthesized).
``total_dur_s``: original video duration (0/unknown the last segment
gets no tail borrow), mirroring ``fit_planner.plan_fit``.
Returns one dict per segment::
{id, status, est_dur_s, available_s, est_overrun_s, calibrated}
``status`` {"fits", "tight", "impossible"}; ``est_overrun_s`` is the
predicted seconds of speech past the available time (0 when it fits).
Pure function: no I/O, deterministic.
"""
params = fit_params or FitParams()
fits_cap, cap = absorb_caps(params)
n = len(segments)
out: list[dict] = []
for i, seg in enumerate(segments):
start = float(seg["start"])
end = float(seg["end"])
slot = max(0.0, end - start)
# Borrowable silence — fit_planner's slack absorption, capped.
if i + 1 < n:
gap = max(0.0, float(segments[i + 1]["start"]) - end)
borrow = min(max(0.0, gap - params.gap_guard_s), gap_borrow_max_s)
elif total_dur_s > 0:
borrow = min(max(0.0, float(total_dur_s) - end), gap_borrow_max_s)
else:
borrow = 0.0
available = slot + borrow
est = estimate_natural_duration(seg.get("text") or "", target_lang, calibration)
if est <= 0.0:
status = "fits"
overrun = 0.0
elif available <= 0.0:
status = "impossible"
overrun = est
else:
need = est / available
# Same boundary tolerance as fit_planner's _EPS: a need that
# lands exactly on a cap is absorbed, not escalated.
if need <= fits_cap + 1e-9:
status = "fits"
elif need <= cap + 1e-9:
status = "tight"
else:
status = "impossible"
overrun = max(0.0, est - available)
out.append({
"id": str(seg.get("id", f"seg_{i}")),
"status": status,
"est_dur_s": round(est, 3),
"available_s": round(available, 3),
"est_overrun_s": round(overrun, 3),
"calibrated": calibration is not None,
})
return out
# ── Optional LLM condensation ───────────────────────────────────────────
_CONDENSE_PROMPT = """\
You are a dubbing writer. The user will give you a translated line that is
TOO LONG for its time slot. Rewrite it shorter so it can be read aloud
within the target duration: cut filler words, tighten phrasing, and drop
the least essential clauses but preserve the meaning. Never change
character names, proper nouns, numbers, or technical terms. Stay in the
same language as the line.
Reply with ONLY the rewritten line. No quotes, no commentary."""
# Bound the LLM loop — condensation is a per-segment *suggestion*, not a
# fit guarantee, so two shots are plenty before degrading to a no-op.
_CONDENSE_ATTEMPTS = 2
def condense_for_slot(
text: str,
*,
available_s: float,
target_lang: str,
source_text: Optional[str] = None,
calibration: Optional[Calibration] = None,
) -> dict:
"""Meaning-preserving shorter rewrite of ``text`` targeting ``available_s``.
Returns ``{"text", "applied", "est_dur_s"}`` (+ ``"error"`` on the no-op
paths). ``applied=False`` keeps the input text untouched no LLM
configured, LLM failure, and divergent/too-aggressive replies all
degrade there. The best (shortest-estimate) candidate that passes the
divergence guard AND is actually shorter than the input wins; a reply
that fits ``available_s`` returns immediately.
"""
text = (text or "").strip()
base_est = estimate_natural_duration(text, target_lang, calibration)
if not text or available_s <= 0:
return {"text": text, "applied": False, "est_dur_s": round(base_est, 3),
"error": "nothing-to-condense"}
if base_est <= available_s:
return {"text": text, "applied": False, "est_dur_s": round(base_est, 3),
"error": "already-fits"}
from services import llm_skills
# `active=` forwards this module's (monkeypatch-able) name so the
# no-override path matches the plain get_active_llm_backend behavior.
llm = llm_skills.skill_backend(_SKILL_ID, active=lambda: get_active_llm_backend())
if isinstance(llm, OffBackend):
return {"text": text, "applied": False, "est_dur_s": round(base_est, 3),
"error": "no-llm"}
best: Optional[tuple[str, float]] = None # (candidate, est)
for attempt in range(1, _CONDENSE_ATTEMPTS + 1):
user_lines = [
f"Target language: {target_lang}",
f"Target duration: {available_s:.2f}s",
f"Current line: {text}",
f"Current reading duration: ~{base_est:.2f}s",
]
if source_text:
user_lines.append(f"Source line (for meaning): {source_text}")
if attempt > 1 and best is not None:
user_lines.append(
f"Your previous rewrite was still ~{best[1]:.2f}s. Cut further."
)
try:
reply = llm.chat(
system=_CONDENSE_PROMPT, user="\n".join(user_lines),
temperature=0.2, # pinned like Autofit — default 1.0 drifts/invents
)
except Exception as e: # noqa: BLE001 — LLM failure must no-op, never raise
logger.warning("condense attempt %d failed: %s", attempt, e)
break
candidate = (reply or "").strip()
if not candidate:
continue
ok, reason = refine_output_ok(text, candidate, target_lang)
if not ok:
logger.warning("condense attempt %d rejected (%s)", attempt, reason)
continue
est = estimate_natural_duration(candidate, target_lang, calibration)
if est >= base_est:
continue # not actually shorter — useless as a suggestion
if best is None or est < best[1]:
best = (candidate, est)
if est <= available_s:
break # fits — done
if best is None:
return {"text": text, "applied": False, "est_dur_s": round(base_est, 3),
"error": "condense-failed"}
return {"text": best[0], "applied": True, "est_dur_s": round(best[1], 3)}
+39 -5
View File
@@ -57,9 +57,13 @@ def find_ffmpeg():
"""Locate an ffmpeg binary.
Resolution order:
1. ``FFMPEG_PATH`` env var (set by Tauri when a sidecar is bundled).
1. ``FFMPEG_PATH`` env var (set by Tauri when a sidecar is bundled, or
by the user's Settings → Audio tools override via prefs).
2. ``imageio-ffmpeg`` pip package (ships a static binary per platform).
3. Common system paths / ``PATH``.
3. OmniVoice-acquired static bundle (``services.media_tools``) the
checksummed build the app downloads itself when nothing else
resolves; the only bundled tier that also ships ffprobe.
4. Common system paths / ``PATH``.
Returns the path string, or ``None`` if nothing found.
"""
@@ -78,7 +82,13 @@ def find_ffmpeg():
logger.debug("imageio_ffmpeg binary not usable at %s", candidate)
except Exception as e:
logger.debug("imageio_ffmpeg unavailable: %s", e)
# 3. Well-known system paths + PATH lookup
# 3. OmniVoice-acquired bundled static binary (never downloads here —
# acquisition is media_tools' background job; this only picks up an
# already-installed build).
candidate = _acquired_bundled("ffmpeg")
if candidate:
return candidate
# 4. Well-known system paths + PATH lookup
common = [
"/opt/homebrew/bin/ffmpeg",
"/usr/local/bin/ffmpeg",
@@ -95,6 +105,22 @@ def find_ffmpeg():
return None
def _acquired_bundled(tool: str) -> "str | None":
"""Already-acquired media_tools static binary, validated — or None.
Lazy import: media_tools imports from this module at its top, so this
module must only reach back at call time (no cycle).
"""
try:
from services.media_tools import bundled_tool_path
candidate = bundled_tool_path(tool)
if candidate and _binary_runs(candidate):
return candidate
except Exception as e:
logger.debug("media_tools bundled %s unavailable: %s", tool, e)
return None
def resolve_ffprobe() -> str | None:
"""Resolve an ffprobe binary path.
@@ -103,8 +129,12 @@ def resolve_ffprobe() -> str | None:
injected by Tauri pointing at the bundled sidecar (e.g.
``/usr/lib/omnivoice-studio/bin/ffprobe`` on .deb installs).
2. ``FFPROBE_PATH`` env var legacy alias kept for backward
compatibility with older Tauri shells / dev environments.
3. ``shutil.which("ffprobe")`` system ``PATH`` fallback.
compatibility with older Tauri shells / dev environments; also the
key Settings Audio tools persists a user override under.
3. OmniVoice-acquired static bundle (``services.media_tools``)
imageio-ffmpeg ships no ffprobe, so this is the bundled tier that
closes the source-install gap.
4. ``shutil.which("ffprobe")`` system ``PATH`` fallback.
Returns the resolved path string, or ``None`` if nothing found. Callers
that need a hard failure should use :func:`find_ffprobe` instead.
@@ -121,6 +151,10 @@ def resolve_ffprobe() -> str | None:
if resolved and _binary_runs(resolved):
return resolved
bundled = _acquired_bundled("ffprobe")
if bundled:
return bundled
system_probe = shutil.which("ffprobe")
if system_probe and _binary_runs(system_probe):
return system_probe
+292
View File
@@ -0,0 +1,292 @@
"""Self-heal for HF cache snapshots whose entries no longer resolve.
The Hugging Face hub cache stores each file's bytes once under
``models--<org>--<name>/blobs/<hash>`` and exposes every revision as
``snapshots/<rev>/<filename>`` entries that link into ``blobs/``. Several
real-world events leave a snapshot entry *broken* a dangling symlink (its
blob target doesn't exist) or a zero-byte stand-in file — while the actual
bytes are safely on disk under ``blobs/``: a blob-naming mismatch between
download modes, an interrupted rename mid-download, antivirus interference.
``os.path.isfile()`` on a dangling symlink is False, so transformers concludes
the weights are missing ("… does not appear to have a file named
pytorch_model.bin or model.safetensors") even though the multi-GB download
completed. A plain ``snapshot_download`` doesn't reliably fix this — depending
on hub version and platform symlink support, the existing-but-broken entry can
short-circuit the restore. Deleting exactly the broken entries first makes
``snapshot_download`` deterministically restore them (reusing completed blobs
where the naming matches, re-downloading only where it doesn't).
Conservative by design, repairing STATE rather than chasing one cause:
* never touches ``blobs/`` (the downloaded bytes),
* never touches snapshot entries that resolve,
* never force-redownloads healthy files,
* never raises any internal failure logs and returns a summary,
* a healthy cache is a cheap lstat/stat walk of ``snapshots/`` (no hashing,
no network) on every platform; the heal is generic, not Windows-gated.
"""
from __future__ import annotations
import logging
import os
logger = logging.getLogger("omnivoice.hf_cache_repair")
# Snapshot entries that are never legitimately zero bytes: weight formats and
# JSON/sentencepiece config-tokenizer files (an empty file is not valid JSON /
# not a valid serialized model). Zero-byte files with any OTHER suffix — an
# empty .txt, .md, .gitattributes, a marker file a repo genuinely ships empty —
# are left alone: when unsure, don't flag.
_NEVER_EMPTY_SUFFIXES = frozenset({
# weights / tensors
".safetensors", ".bin", ".pt", ".pth", ".ckpt", ".onnx", ".gguf",
".msgpack", ".h5", ".pb", ".tflite",
# config / tokenizer
".json", ".model", ".spm",
})
def _env_flag(name: str) -> bool:
return (os.environ.get(name) or "").strip().lower() in {"1", "true", "yes", "on"}
def hf_cache_home() -> str:
"""The hub cache root in effect. Mirrors huggingface_hub's resolution
(``HF_HUB_CACHE`` > ``HF_HOME``/hub > default) but reads the env at call
time hub's constants freeze at import, which is too early for tests and
for the Windows short-cache redirect in ``core.config``."""
env = (os.environ.get("HF_HUB_CACHE") or "").strip()
if env:
return env
hf_home = (os.environ.get("HF_HOME") or "").strip()
if hf_home:
return os.path.join(hf_home, "hub")
try:
from huggingface_hub.constants import HF_HUB_CACHE
return HF_HUB_CACHE
except Exception:
return os.path.join(os.path.expanduser("~"), ".cache", "huggingface", "hub")
def repo_cache_dir(repo_id: str, cache_dir: str | None = None) -> str:
"""The ``models--<org>--<name>`` folder for ``repo_id`` (repo_type=model)."""
return os.path.join(cache_dir or hf_cache_home(),
"models--" + repo_id.replace("/", "--"))
def _is_dangling_symlink(path: str) -> bool:
# islink() uses lstat (True even when the target is gone); exists()
# resolves the link — False for a dangling one. Never raises for a path
# that came out of os.walk.
return os.path.islink(path) and not os.path.exists(path)
def _is_suspicious_zero_byte(path: str) -> bool:
"""A zero-byte REGULAR file standing where model content must be.
Conservative: only weight/config-typed names are flagged (those are never
legitimately empty the bytes to restore them live in ``blobs/`` or on
the Hub); anything else is presumed intentional and left alone."""
if os.path.islink(path):
return False # resolving symlinks are handled by the dangling check
try:
if not os.path.isfile(path) or os.path.getsize(path) != 0:
return False
except OSError:
return False
return os.path.splitext(path)[1].lower() in _NEVER_EMPTY_SUFFIXES
def find_dangling_entries(repo_cache_dir: str) -> list[str]:
"""Broken entries under ``<repo_cache_dir>/snapshots/*/``: dangling
symlinks plus suspicious zero-byte regular files (see above).
Returns absolute paths. On a healthy cache this is a no-op scan a pure
lstat/stat walk of ``snapshots/`` (``blobs/`` is never visited), no
hashing, no network. Never raises."""
broken: list[str] = []
snapshots = os.path.join(repo_cache_dir, "snapshots")
if not os.path.isdir(snapshots):
return broken
try:
# followlinks=False: a dangling symlink is not a dir, so os.walk lists
# it among the files of its parent — exactly where we scan.
for root, _dirs, files in os.walk(snapshots, followlinks=False):
for name in files:
path = os.path.join(root, name)
if _is_dangling_symlink(path) or _is_suspicious_zero_byte(path):
broken.append(path)
except OSError as walk_err: # pragma: no cover - defensive
logger.warning("HF cache scan of %s aborted: %s", snapshots, walk_err)
return broken
def _force_copy_mode(cache_root: str) -> bool:
"""Best-effort: make huggingface_hub materialize snapshot entries as real
file COPIES instead of symlinks for the rest of this process.
Why: hub's ``are_symlinks_supported()`` probe can succeed in-process while
real snapshot symlink creation fails or produces broken links (Windows
without Developer Mode is the reported case) and the result is memoized
in the private ``file_download._are_symlinks_supported_in_dir`` dict, so a
plain ``snapshot_download`` retry would recreate the SAME dangling links.
Pre-seeding that memo with False flips hub into copy mode. It's private
API, so any failure (attribute/shape changed across hub versions) is
logged and reported as False the caller then skips the copy-mode pass
rather than crash. Deliberately NOT undone: on a host where links come
out broken, every later download should use copies too."""
try:
from pathlib import Path
import huggingface_hub.file_download as _fd
memo = getattr(_fd, "_are_symlinks_supported_in_dir", None)
if not isinstance(memo, dict):
raise TypeError(
f"_are_symlinks_supported_in_dir is {type(memo).__name__}, expected dict"
)
# Same key normalization hub's are_symlinks_supported() applies.
memo[str(Path(cache_root).expanduser().resolve())] = False
return True
except Exception as e:
logger.warning(
"Could not force copy-mode for the HF cache (%s) — "
"huggingface_hub's private memo may have changed; skipping the "
"copy-mode repair pass.", e,
)
return False
def repair_repo_cache(repo_id: str, cache_dir: str | None = None) -> dict:
"""Repair a repo's cache: delete broken snapshot entries (and ONLY those),
then ``snapshot_download`` to restore the missing files hub reuses
completed blobs where the naming matches and re-downloads otherwise.
Verified after the fact: if the restore recreated dangling links (a host
where hub's symlink probe passes but real links come out broken — Windows
without Developer Mode), force copy mode and repair once more so the
snapshot ends up with real files.
Returns a summary dict; never raises:
``found`` broken entries detected up front,
``removed`` entries actually deleted (both passes),
``restored`` True when a snapshot_download completed,
``outcome`` "healthy" | "healed_with_links" | "healed_with_copies"
| "repair_failed",
``ok`` True unless outcome == "repair_failed",
``error`` "" or why the repair failed.
"""
summary: dict = {
"repo_id": repo_id,
"repo_dir": "",
"found": 0,
"removed": 0,
"restored": False,
"outcome": "repair_failed",
"ok": False,
"error": "",
}
try:
cache_root = cache_dir or hf_cache_home()
repo_dir = repo_cache_dir(repo_id, cache_root)
summary["repo_dir"] = repo_dir
broken = find_dangling_entries(repo_dir)
summary["found"] = len(broken)
if not broken:
summary["ok"] = True # nothing broken → nothing to do
summary["outcome"] = "healthy"
return summary
if _env_flag("HF_HUB_OFFLINE") or _env_flag("TRANSFORMERS_OFFLINE"):
# Don't delete what we can't restore: offline mode means the
# follow-up snapshot_download is off the table.
summary["error"] = (
"Hugging Face offline mode is enabled "
"(HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE) — cannot restore files"
)
logger.warning(
"Model cache for %s has %d broken snapshot entr%s but HF "
"offline mode is set — skipping repair.",
repo_id, len(broken), "y" if len(broken) == 1 else "ies",
)
return summary
def _remove(paths: list[str]) -> int:
n = 0
for path in paths:
try:
os.remove(path) # removes the link/file itself, never a blob
n += 1
logger.info(
"HF cache self-heal: removed broken snapshot entry %s", path
)
except OSError as rm_err:
logger.warning(
"HF cache self-heal: could not remove broken entry %s: %s",
path, rm_err,
)
return n
summary["removed"] = _remove(broken)
if summary["removed"] == 0:
summary["error"] = "broken entries could not be removed"
return summary
from huggingface_hub import snapshot_download
dl_kwargs: dict = {"repo_id": repo_id}
if cache_dir:
dl_kwargs["cache_dir"] = cache_dir
endpoint = os.environ.get("HF_ENDPOINT")
if endpoint:
dl_kwargs["endpoint"] = endpoint
snapshot_download(**dl_kwargs)
summary["restored"] = True
# Verify-after-repair: hub's memoized symlink probe can claim support
# while the links it just recreated dangle again. If so, force copy
# mode and repair once more so real files land in the snapshot.
still_broken = find_dangling_entries(repo_dir)
if not still_broken:
summary["ok"] = True
summary["outcome"] = "healed_with_links"
logger.info(
"HF cache self-heal for %s: removed %d broken snapshot entr%s "
"and restored the snapshot from existing blobs / the Hub.",
repo_id, summary["removed"],
"y" if summary["removed"] == 1 else "ies",
)
return summary
logger.warning(
"HF cache self-heal for %s: the restore recreated %d broken "
"link(s) — forcing copy-mode and repairing once more.",
repo_id, len(still_broken),
)
if not _force_copy_mode(cache_root):
summary["error"] = (
"the snapshot restore recreated broken links and copy-mode "
"could not be forced"
)
return summary
summary["removed"] += _remove(still_broken)
snapshot_download(**dl_kwargs)
remaining = find_dangling_entries(repo_dir)
if remaining:
summary["error"] = (
f"{len(remaining)} snapshot entr"
f"{'y is' if len(remaining) == 1 else 'ies are'} still broken "
"after the copy-mode repair"
)
return summary
summary["ok"] = True
summary["outcome"] = "healed_with_copies"
logger.info(
"HF cache self-heal for %s: healed with real file copies "
"(symlinks on this host come out broken; hub stays in copy-mode "
"for the rest of this run).", repo_id,
)
return summary
except Exception as e: # never raise — repair is best-effort
summary["error"] = f"{type(e).__name__}: {e}"
logger.warning(
"HF cache self-heal for %s failed: %s", repo_id, summary["error"],
)
return summary
+146 -21
View File
@@ -18,10 +18,16 @@ reimplement it:
(+ optional cover art, loudness filter), output as ``m4b`` or ``mp3``.
* ``chapter_cache_key`` deterministic content hash so a re-run reuses
already-rendered chapters (resume) and re-renders only what changed.
* ``segment_cache_key`` / ``SegmentCache`` the inner cache layer: each
spoken span's WAV is content-addressed under ``<cache_dir>/segments`` so
editing one sentence re-renders one segment (not the chapter) and an
interrupted chapter render resumes from its finished segments.
Every function here is pure (string/argv in, string/argv out) so it's unit
tested without ffmpeg, torch, or a GPU. The impure ffmpeg run lives in the
caller (the audiobook router today; the stories job tomorrow).
The builders are pure (string/argv in, string/argv out) so they're unit tested
without ffmpeg, torch, or a GPU; the cache helpers (``prune_cache_dir``,
``SegmentCache``) touch only local files and import torch lazily. The impure
ffmpeg run lives in the caller (the audiobook router today; the stories job
tomorrow).
"""
from __future__ import annotations
@@ -65,30 +71,29 @@ def _escape_meta(value: str) -> str:
def prune_cache_dir(cache_dir: str, max_bytes: int = _CACHE_MAX_BYTES) -> tuple[int, int]:
"""Evict the oldest files in ``cache_dir`` until the total size is within
``max_bytes`` (LRU by mtime). The content-addressed chapter cache otherwise
``max_bytes`` (LRU by mtime). The content-addressed render cache otherwise
grows without bound uncompressed WAVs accumulate across every render.
Best-effort: returns ``(remaining_bytes, removed_count)`` and never raises
(a missing dir / unstattable file is just skipped). Call it *before* writing
a job's chapters so the fresh ones are never the eviction target.
Walks the whole tree, so chapter WAVs at the root and segment WAVs under
``segments/`` share ONE byte budget the cap holds no matter which layer
grew. Best-effort: returns ``(remaining_bytes, removed_count)`` and never
raises (a missing dir / unstattable file is just skipped). Call it *before*
writing a job's files so the fresh ones are never the eviction target.
"""
try:
names = os.listdir(cache_dir)
except OSError:
return (0, 0)
entries: list[tuple[float, int, str]] = []
total = 0
for name in names:
p = os.path.join(cache_dir, name)
try:
if not os.path.isfile(p):
for root, _dirs, names in os.walk(cache_dir):
for name in names:
p = os.path.join(root, name)
try:
if not os.path.isfile(p):
continue
size = os.path.getsize(p)
mtime = os.path.getmtime(p)
except OSError:
continue
size = os.path.getsize(p)
mtime = os.path.getmtime(p)
except OSError:
continue
entries.append((mtime, size, p))
total += size
entries.append((mtime, size, p))
total += size
if total <= max_bytes:
return (total, 0)
entries.sort() # oldest first
@@ -136,6 +141,126 @@ def chapter_cache_key(
return hashlib.sha1(raw.encode("utf-8"), usedforsecurity=False).hexdigest()[:20]
# ── Segment cache (sub-chapter granularity) ─────────────────────────────────
#: Segment WAVs live in a subdirectory of the chapter cache dir so both layers
#: share one root — and one byte cap (``prune_cache_dir`` walks the tree).
SEGMENT_SUBDIR = "segments"
def segment_cache_key(
text: str,
*,
sample_rate: int,
engine_id: str,
voice_id: Optional[str] = None,
voice_sig: str = "",
speed: Optional[float] = None,
extra_sig: str = "",
) -> str:
"""Deterministic content hash for ONE rendered segment (a single spoken
span). Same dimensions as :func:`chapter_cache_key` minus span order and
pauses (pauses are synthesized silence never cached): text, voice
identity (id + resolved signature), speed, sample rate, engine, plus
``extra_sig`` for anything else that changes the rendered audio (the
pronunciation lexicon today). Any change new key re-synthesize just
this segment.
"""
payload = {
"sr": int(sample_rate),
"engine": engine_id or "",
"voice": voice_id or "",
"text": text or "",
"speed": speed,
"voice_sig": voice_sig or "",
"extra": extra_sig or "",
}
raw = json.dumps(payload, sort_keys=True, ensure_ascii=False)
# Content-addressing only — not a security digest (see chapter_cache_key).
return hashlib.sha1(raw.encode("utf-8"), usedforsecurity=False).hexdigest()[:20]
class SegmentCache:
"""Content-addressed per-segment WAV store under ``cache_dir/segments``.
The chapter cache stays the fast outer layer a fully-unchanged chapter
hits at the chapter key and never touches segment files. This inner layer
makes a *changed* chapter cheap: only the edited/new segments synthesize
(the rest load from disk), and an interrupted chapter render resumes from
the segments that already finished, because each segment is persisted the
moment it renders.
``voice_sig`` maps ``voice_id or ""`` resolved-profile signature (same
strings the chapter key uses) so a profile edit invalidates segments too.
Load/store are best-effort: a missing/corrupt/foreign-rate file is a clean
cache miss (re-render), and a failed store never fails the render so
caches written by any app version degrade safely. torch/torchaudio import
lazily to keep this module import-light for the pure-builder callers.
"""
def __init__(
self,
cache_dir: str,
*,
sample_rate: int,
engine_id: str,
voice_sig: Optional[dict] = None,
extra_sig: str = "",
) -> None:
self.dir = os.path.join(cache_dir, SEGMENT_SUBDIR)
self.sample_rate = int(sample_rate)
self.engine_id = engine_id or ""
self.voice_sig = dict(voice_sig or {})
self.extra_sig = extra_sig or ""
self.hits = 0
self.misses = 0
def _path(self, span) -> str:
key = segment_cache_key(
span.text,
sample_rate=self.sample_rate,
engine_id=self.engine_id,
voice_id=span.voice_id,
voice_sig=self.voice_sig.get(span.voice_id or "", ""),
speed=getattr(span, "speed", None),
extra_sig=self.extra_sig,
)
return os.path.join(self.dir, f"{key}.wav")
def load(self, span):
"""Cached audio tensor for ``span``, or ``None`` (miss). A hit bumps
the file's mtime so LRU eviction sees the segment as recently used."""
path = self._path(span)
if not os.path.isfile(path):
self.misses += 1
return None
try:
import torchaudio
audio, sr = torchaudio.load(path)
except Exception:
self.misses += 1
return None # unreadable/corrupt entry — clean miss, re-render
if int(sr) != self.sample_rate or audio.numel() == 0:
self.misses += 1
return None # foreign-rate/empty entry — clean miss, re-render
try:
os.utime(path, None)
except OSError:
pass
self.hits += 1
return audio
def store(self, span, audio) -> None:
"""Persist a freshly rendered segment. Best-effort — a full disk or
unwritable cache dir must never fail the chapter render."""
try:
from services.audio_io import atomic_save_wav
os.makedirs(self.dir, exist_ok=True)
atomic_save_wav(self._path(span), audio, self.sample_rate)
except Exception:
pass
# ── Loudness normalization ──────────────────────────────────────────────────
@dataclass(frozen=True)
+637
View File
@@ -0,0 +1,637 @@
"""Media tools — ffmpeg / ffprobe / yt-dlp as an invisible internal concern.
Most users should never learn what ffmpeg is. This module makes the media
engine self-contained: it reports where each tool comes from, acquires a
bundled static build in the background when no tier of the resolution chain
(``services.ffmpeg_utils``) resolves, and gives power users explicit
control (custom path / system copy / restore bundled) through the
``/media-tools`` router persisted via the same ``env.FFMPEG_PATH`` /
``env.FFPROBE_PATH`` prefs convention the Settings env writer already uses,
so there is exactly one override mechanism.
Bundled-binary source (decision record)
---------------------------------------
The gap: ``imageio-ffmpeg`` (already a locked dep) ships a static *ffmpeg*
inside its platform wheels but **no ffprobe**, so source installs without a
system ffmpeg lose ``/tools/probe``, Smart-Fit duration checks, and VFR
detection. Two options were audited:
(a) the ``static-ffmpeg`` pip package ships BOTH binaries per platform via
lazy download. **Rejected**: it downloads from a *mutable* URL
(``.../ffmpeg_bins/raw/main/...`` the branch tip, not a pinned
release), performs **no checksum validation**, extracts into its own
``site-packages`` directory (read-only / non-existent in the frozen
PyInstaller backend), and drags in ``requests``/``filelock``/``progress``
plus a stdout spinner.
(b) fetch the same upstream static builds ourselves, pinned to an immutable
commit. **Chosen**: we download the platform zip from
``github.com/zackees/ffmpeg_bins`` at a pinned commit SHA (immutable
URL), verify size + SHA-256 against constants recorded from that
commit's git-LFS pointers, extract only ffmpeg/ffprobe into a
user-writable, update-surviving dir under ``DATA_DIR``, and trust a
binary only after the existing ``_binary_runs`` ``-version`` probe.
Stdlib-only (urllib honors HTTP(S)_PROXY), identical behavior on
macOS (arm64 + x86_64), Windows x64, and Linux (x64 + arm64), and zero
new Python dependencies.
yt-dlp updates (decision record)
--------------------------------
yt-dlp is an importable locked dep never a user-installed requirement.
But site support rots faster than app releases, so Settings offers a
user-triggered "Update". A plain in-venv upgrade was audited and rejected:
the app venv is uv-managed (no pip module), and the updater's drift sync
(#1029/#1030, ``uv sync --frozen --inexact``) preserves only packages *not*
in the lockfile yt-dlp IS locked, so an in-venv upgrade would be silently
reverted on the next app update, and the frozen build has no installer at
all. Instead we install the new wheel (pure-python, no required deps
matching the plain ``yt-dlp`` spec pinned in pyproject) into an **overlay
directory** under ``DATA_DIR`` SHA-256-verified against PyPI's own
metadata and prepend it to ``sys.path`` at startup. It survives app
updates and drift syncs, works identically in source and frozen builds, and
"Restore tested version" is simply deleting the overlay: the locked wheel
underneath was never touched.
"""
from __future__ import annotations
import hashlib
import logging
import os
import re
import shutil
import subprocess
import sys
import tempfile
import threading
import zipfile
from core.config import DATA_DIR
from core import prefs
from services.ffmpeg_utils import _binary_runs, _BINARY_OK
logger = logging.getLogger("omnivoice.media_tools")
# ── Pinned bundled build ────────────────────────────────────────────────────
# Immutable commit of github.com/zackees/ffmpeg_bins (the upstream the
# static-ffmpeg pip package also consumes, but pinned + checksummed here).
# SHA-256 values are the git-LFS oids of the v8.0 platform zips at this
# commit, independently verified by downloading and hashing.
_FFBIN_REPO = "zackees/ffmpeg_bins"
_FFBIN_COMMIT = "df95abcb0ce6efff710dda5ef28a2f6f1dc21493" # 2026-01-16
_FFBIN_TREE = "v8.0"
#: platform key → (sha256, size in bytes) of the zip at the pinned commit.
_FFBIN_SHA256 = {
"darwin": ("70fd5b21cb37b6ea97c8b584cf76b3cc6a90179831c9c269811b9716c28605fb", 53079896),
"darwin_arm64": ("b2da44a8169c4d09a97db996250690c3346f72e4795521d23d3dbb1e72421207", 41925556),
"linux": ("ca75b05e887c7a97676632f673031875847be83daa9794298fed9cef8cac14ad", 142008975),
"linux_arm64": ("e03efe471c03b999f10988d5db62ae3bd94837463291b3c7755528b100e97d6f", 131816005),
"win32": ("92662c2241e93fe71b3f3a01e94a0b0dc8cfad726019f96b83bc109ce44c5d0b", 72065209),
}
_PYPI_YTDLP_URL = "https://pypi.org/pypi/yt-dlp/json"
_DOWNLOAD_TIMEOUT_S = 30 # per-read socket timeout; downloads stream in chunks
_CHUNK = 256 * 1024
#: tool → env keys honored by the resolution chain, in precedence order.
_ENV_KEYS = {
"ffmpeg": ("FFMPEG_PATH",),
"ffprobe": ("OMNIVOICE_FFPROBE_PATH", "FFPROBE_PATH"),
}
#: tool → the env key the *user override* is persisted under (prefs `env.<KEY>`).
_PREF_ENV_KEY = {"ffmpeg": "FFMPEG_PATH", "ffprobe": "FFPROBE_PATH"}
TOOLS = ("ffmpeg", "ffprobe")
# ── Background-operation state (poll via status()) ─────────────────────────
_lock = threading.Lock()
_ops: dict[str, dict] = {
"acquire": {"state": "idle", "progress": 0.0, "error": None},
"ytdlp_update": {"state": "idle", "progress": 0.0, "error": None, "version": None},
}
_version_cache: dict[str, str] = {}
def _set_op(op: str, **fields) -> None:
with _lock:
_ops[op].update(fields)
def _op_snapshot() -> dict:
with _lock:
return {k: dict(v) for k, v in _ops.items()}
# ── Platform / paths ────────────────────────────────────────────────────────
def _platform_key() -> str:
import platform as _p
is_arm = _p.machine().lower() in ("arm64", "aarch64")
if sys.platform == "win32":
return "win32"
if sys.platform == "darwin":
return "darwin_arm64" if is_arm else "darwin"
if sys.platform.startswith("linux"):
return "linux_arm64" if is_arm else "linux"
return sys.platform
def media_tools_dir() -> str:
"""User-writable root for acquired binaries + the yt-dlp overlay.
Lives in DATA_DIR so it survives app updates (the app bundle / venv are
replaced wholesale on update; DATA_DIR is user state) and is writable in
frozen installs.
"""
return os.path.join(DATA_DIR, "media_tools")
def bundled_dir() -> str:
# Versioned by the pin so a future pin bump lands in a fresh dir and
# "Update" is a plain re-acquire — no in-place mutation of a live binary.
return os.path.join(media_tools_dir(), f"ffbin-{_FFBIN_COMMIT[:12]}", _platform_key())
def _exe(name: str) -> str:
return f"{name}.exe" if sys.platform == "win32" else name
def bundled_tool_path(tool: str) -> str | None:
"""Path of an already-acquired bundled binary, or None. Never downloads."""
p = os.path.join(bundled_dir(), _exe(tool))
return p if os.path.isfile(p) else None
def _bundle_url() -> str:
# github.com/<repo>/raw/<commit> redirects to the LFS media host and
# serves the real zip (raw.githubusercontent.com would return the
# 133-byte LFS pointer instead).
return f"https://github.com/{_FFBIN_REPO}/raw/{_FFBIN_COMMIT}/{_FFBIN_TREE}/{_platform_key()}.zip"
def _expected_bundle() -> tuple[str, str, int]:
"""(url, sha256, size) for this platform. Raises on unsupported platform."""
key = _platform_key()
if key not in _FFBIN_SHA256:
raise RuntimeError(f"no bundled media-engine build for platform '{key}'")
sha, size = _FFBIN_SHA256[key]
return _bundle_url(), sha, size
# ── Download helper ─────────────────────────────────────────────────────────
def _download(url: str, dest_path: str, expected_sha256: str,
expected_size: int | None, op: str) -> None:
"""Stream *url* to *dest_path*, hashing on the fly; raise on mismatch.
Progress is reported into ``_ops[op]["progress"]``. urllib honors the
HTTP(S)_PROXY env vars, so restricted-network users' proxy settings apply.
"""
import urllib.request
if not url.startswith("https://"):
raise ValueError("media-tools downloads must be https")
req = urllib.request.Request(url, headers={"User-Agent": "OmniVoice-Studio"})
hasher = hashlib.sha256()
done = 0
with urllib.request.urlopen(req, timeout=_DOWNLOAD_TIMEOUT_S) as resp:
total = expected_size or int(resp.headers.get("Content-Length") or 0)
with open(dest_path, "wb") as f:
while True:
chunk = resp.read(_CHUNK)
if not chunk:
break
f.write(chunk)
hasher.update(chunk)
done += len(chunk)
if total:
_set_op(op, progress=min(done / total, 1.0))
digest = hasher.hexdigest()
if expected_size is not None and done != expected_size:
raise RuntimeError(f"download size mismatch: got {done}, expected {expected_size}")
if digest != expected_sha256:
raise RuntimeError("download checksum mismatch — refusing to install")
# ── Bundled acquisition ─────────────────────────────────────────────────────
def acquire_bundled(wait: bool = False) -> dict:
"""Fetch + verify + install the pinned static ffmpeg/ffprobe build.
Idempotent: a no-op when the binaries are already present and runnable,
or when an acquisition is already running. Runs in a daemon thread so it
never blocks the caller (``wait=True`` is for tests/CLI use).
Returns the op-state snapshot.
"""
with _lock:
if _ops["acquire"]["state"] == "running":
return dict(_ops["acquire"])
_ops["acquire"].update(state="running", progress=0.0, error=None)
if all(bundled_tool_path(t) and _binary_runs(bundled_tool_path(t)) for t in TOOLS):
_set_op("acquire", state="done", progress=1.0)
return _op_snapshot()["acquire"]
def _worker():
try:
_do_acquire()
_set_op("acquire", state="done", progress=1.0, error=None)
logger.info("media-tools: bundled ffmpeg/ffprobe installed at %s", bundled_dir())
except Exception as e:
logger.warning("media-tools: bundled acquisition failed: %s", e)
_set_op("acquire", state="error", error=str(e)[:300])
if wait:
_worker()
else:
threading.Thread(target=_worker, name="media-tools-acquire", daemon=True).start()
return _op_snapshot()["acquire"]
def _do_acquire() -> None:
url, sha, size = _expected_bundle()
target = bundled_dir()
os.makedirs(os.path.dirname(target), exist_ok=True)
with tempfile.TemporaryDirectory(dir=os.path.dirname(target)) as tmp:
zip_path = os.path.join(tmp, "bundle.zip")
_download(url, zip_path, sha, size, op="acquire")
# Extract only the two binaries, flattened by basename — layout-agnostic
# and immune to zip-slip (we never honor archive paths).
wanted = {_exe(t): t for t in TOOLS}
staged = os.path.join(tmp, "staged")
os.makedirs(staged, exist_ok=True)
found: dict[str, str] = {}
with zipfile.ZipFile(zip_path) as zf:
for member in zf.infolist():
base = os.path.basename(member.filename)
if base in wanted and not member.is_dir():
out = os.path.join(staged, base)
with zf.open(member) as src, open(out, "wb") as dst:
shutil.copyfileobj(src, dst)
# Owner-only rwx — the backend process is the sole consumer
# of these binaries (least privilege; py/overly-permissive-file).
os.chmod(out, 0o700)
found[base] = out
missing = set(wanted) - set(found)
if missing:
raise RuntimeError(f"bundle is missing {sorted(missing)}")
# Probe BEFORE trusting — a corrupt / wrong-arch binary must never
# be installed (same contract as ffmpeg_utils._binary_runs at
# resolution time, applied at install time).
for base, path in found.items():
_BINARY_OK.pop(path, None)
if not _binary_runs(path):
raise RuntimeError(f"downloaded {base} failed its -version probe")
# Finalize: swap the staged dir into place.
if os.path.isdir(target):
shutil.rmtree(target, ignore_errors=True)
os.replace(staged, target)
# Resolution caches may hold negative verdicts for the old paths.
for t in TOOLS:
p = os.path.join(target, _exe(t))
_BINARY_OK.pop(p, None)
_version_cache.pop(p, None)
# ── Status / origin classification ─────────────────────────────────────────
def _tool_version(path: str) -> str | None:
cached = _version_cache.get(path)
if cached:
return cached
try:
out = subprocess.run(
[path, "-version"], capture_output=True, text=True, timeout=10, check=False,
).stdout
m = re.match(r"^(?:ffmpeg|ffprobe) version (\S+)", out or "")
if m:
_version_cache[path] = m.group(1)
return m.group(1)
except Exception as e:
logger.debug("version probe failed for %s: %s", os.path.basename(path), e)
return None
def _imageio_pkg_dir() -> str | None:
try:
import imageio_ffmpeg
return os.path.dirname(os.path.abspath(imageio_ffmpeg.__file__))
except Exception:
return None
def _classify_origin(tool: str, path: str) -> str:
"""sidecar | bundled | system | custom — where the resolved binary lives."""
rp = os.path.realpath(path)
for root in filter(None, (media_tools_dir(), _imageio_pkg_dir())):
if rp.startswith(os.path.realpath(root) + os.sep):
return "bundled"
for key in _ENV_KEYS[tool]:
v = os.environ.get(key)
if not v:
continue
if v == path or os.path.realpath(v) == rp or shutil.which(v) == path:
# The same env var serves two masters: the Tauri sidecar injects
# it at spawn; a user override persists it via prefs `env.<KEY>`.
return "custom" if prefs.get(f"env.{key}") else "sidecar"
return "system"
def _resolve(tool: str) -> str | None:
from services import ffmpeg_utils
if tool == "ffmpeg":
return ffmpeg_utils.find_ffmpeg()
return ffmpeg_utils.find_ffprobe()
def _ytdlp_status() -> dict:
"""yt-dlp is a python module, not a binary — status reads its version
without paying the full package import."""
info: dict = {"tool": "yt-dlp", "ok": False, "path": None, "version": None,
"origin": "bundled", "overlay_version": None,
"baseline_version": prefs.get("media_tools.ytdlp_baseline")}
try:
import importlib.util
spec = importlib.util.find_spec("yt_dlp")
origin = getattr(spec, "origin", None)
if origin:
pkg_dir = os.path.dirname(origin)
info["path"] = pkg_dir
info["ok"] = True
info["version"] = _read_ytdlp_version(pkg_dir)
if os.path.realpath(pkg_dir).startswith(
os.path.realpath(_ytdlp_overlay_dir()) + os.sep):
info["origin"] = "custom"
except Exception as e:
logger.debug("yt_dlp spec lookup failed: %s", e)
ov = _read_ytdlp_version(os.path.join(_ytdlp_overlay_dir(), "yt_dlp"))
info["overlay_version"] = ov
return info
def _read_ytdlp_version(pkg_dir: str) -> str | None:
try:
with open(os.path.join(pkg_dir, "version.py"), encoding="utf-8") as f:
m = re.search(r"__version__\s*=\s*['\"]([^'\"]+)['\"]", f.read())
return m.group(1) if m else None
except OSError:
return None
def status() -> dict:
"""Full media-tools report: per-tool {ok, path, version, origin} + op states."""
tools = {}
for tool in TOOLS:
path = _resolve(tool)
tools[tool] = {
"tool": tool,
"ok": bool(path),
"path": path,
"version": _tool_version(path) if path else None,
"origin": _classify_origin(tool, path) if path else None,
}
tools["ytdlp"] = _ytdlp_status()
ops = _op_snapshot()
return {
"ready": tools["ffmpeg"]["ok"] and tools["ffprobe"]["ok"],
"tools": tools,
"ops": ops,
"platform_key": _platform_key(),
}
def summary(auto_acquire: bool = False) -> dict:
"""Small preflight-embeddable verdict. With ``auto_acquire``, kicks off
the bundled download in the background when nothing resolves (first-run
self-heal) but never re-fires after a failed attempt (the wizard's
failure card owns the Retry)."""
st = status()
op = st["ops"]["acquire"]
if auto_acquire and not st["ready"] and op["state"] == "idle":
op = acquire_bundled()
return {
"ready": st["ready"],
"acquire": {"state": op["state"], "progress": op["progress"], "error": op["error"]},
}
# ── User overrides (persisted via the existing env-prefs convention) ───────
def _validate_binary_path(path: str) -> None:
# Same defense-in-depth as /system/set-env: no control chars, must be an
# existing file, and must actually run before we trust it.
if any(ord(c) < 0x20 or ord(c) == 0x7F for c in path):
raise ValueError("Invalid path: control characters are not allowed")
if not os.path.isfile(path):
raise ValueError(f"File not found: {path}")
_BINARY_OK.pop(path, None)
if not _binary_runs(path):
raise ValueError(
"That file exists but does not run as a media tool "
"(its `-version` probe failed) — wrong architecture or not executable."
)
def set_custom_path(tool: str, path: str) -> dict:
"""Pin *tool* to an explicit binary. Persists via prefs `env.<KEY>` —
the exact mechanism /system/set-env uses, so there is one override store."""
if tool not in TOOLS:
raise ValueError(f"unknown tool '{tool}'")
path = path.strip()
_validate_binary_path(path)
key = _PREF_ENV_KEY[tool]
os.environ[key] = path
prefs.set_(f"env.{key}", path)
_version_cache.pop(path, None)
logger.info("media-tools: %s pinned to user path (origin=%s)",
tool, _classify_origin(tool, path))
return status()["tools"][tool]
def use_system(tool: str) -> dict:
"""Auto-detect a system-installed copy and pin it."""
if tool not in TOOLS:
raise ValueError(f"unknown tool '{tool}'")
candidate = _detect_system(tool)
if not candidate:
raise LookupError(
f"No system {tool} found on PATH or in the usual install locations."
)
return set_custom_path(tool, candidate)
def _detect_system(tool: str) -> str | None:
roots = [r for r in (media_tools_dir(), _imageio_pkg_dir()) if r]
def _is_bundled(p: str) -> bool:
rp = os.path.realpath(p)
return any(rp.startswith(os.path.realpath(r) + os.sep) for r in roots)
candidates = [
f"/opt/homebrew/bin/{tool}",
f"/usr/local/bin/{tool}",
f"/usr/bin/{tool}",
f"C:\\ffmpeg\\bin\\{tool}.exe",
f"C:\\Program Files\\ffmpeg\\bin\\{tool}.exe",
tool,
]
for c in candidates:
resolved = shutil.which(c)
if resolved and not _is_bundled(resolved) and _binary_runs(resolved):
return resolved
return None
def restore_bundled(tool: str) -> dict:
"""Clear the user override so the chain resolves sidecar → bundled →
system again; kick acquisition if no bundled build is present. Always safe."""
if tool not in TOOLS:
raise ValueError(f"unknown tool '{tool}'")
for key in _ENV_KEYS[tool]:
if prefs.get(f"env.{key}"):
prefs.delete(f"env.{key}")
os.environ.pop(key, None)
_version_cache.clear()
if not (bundled_tool_path(tool) and _binary_runs(bundled_tool_path(tool))):
# No local bundled build to fall back to (imageio may still cover
# ffmpeg) — fetch ours in the background so the revert lands somewhere.
if not _resolve(tool):
acquire_bundled()
return status()["tools"][tool]
# ── yt-dlp overlay ──────────────────────────────────────────────────────────
def _ytdlp_overlay_dir() -> str:
return os.path.join(media_tools_dir(), "ytdlp_overlay")
def activate_ytdlp_overlay() -> bool:
"""Prepend the user-updated yt-dlp overlay to sys.path. Called once at
backend startup, before anything imports yt_dlp."""
overlay = _ytdlp_overlay_dir()
if os.path.isdir(os.path.join(overlay, "yt_dlp")) and overlay not in sys.path:
sys.path.insert(0, overlay)
logger.info("media-tools: yt-dlp overlay active (%s)",
_read_ytdlp_version(os.path.join(overlay, "yt_dlp")) or "?")
return True
return False
def _fetch_pypi_ytdlp() -> tuple[str, str, str]:
"""(version, wheel_url, sha256) of the latest yt-dlp wheel on PyPI."""
import json
import urllib.request
req = urllib.request.Request(_PYPI_YTDLP_URL, headers={"User-Agent": "OmniVoice-Studio"})
with urllib.request.urlopen(req, timeout=_DOWNLOAD_TIMEOUT_S) as resp:
meta = json.load(resp)
version = meta["info"]["version"]
for artifact in meta.get("urls", []):
if artifact.get("packagetype") == "bdist_wheel" and \
artifact["filename"].endswith("py3-none-any.whl"):
return version, artifact["url"], artifact["digests"]["sha256"]
raise RuntimeError(f"no universal wheel found for yt-dlp {version}")
def update_ytdlp(wait: bool = False) -> dict:
"""Install the newest yt-dlp into the overlay dir (background thread).
The wheel is verified against PyPI's own sha256 digest before a single
byte lands in the overlay; the swap is atomic (staged dir + os.replace).
Takes effect on the next backend start (the running process already
imported the old module) the UI shows the restart affordance.
"""
with _lock:
if _ops["ytdlp_update"]["state"] == "running":
return dict(_ops["ytdlp_update"])
_ops["ytdlp_update"].update(state="running", progress=0.0, error=None, version=None)
def _worker():
try:
version = _do_update_ytdlp()
_set_op("ytdlp_update", state="done", progress=1.0, version=version)
logger.info("media-tools: yt-dlp overlay updated to %s", version)
except Exception as e:
logger.warning("media-tools: yt-dlp update failed: %s", e)
_set_op("ytdlp_update", state="error", error=str(e)[:300])
if wait:
_worker()
else:
threading.Thread(target=_worker, name="media-tools-ytdlp", daemon=True).start()
return _op_snapshot()["ytdlp_update"]
def _do_update_ytdlp() -> str:
version, url, sha = _fetch_pypi_ytdlp()
# Record the locked ("tested") version once, before the first overlay
# ever activates — that's what "Restore tested version" reverts to.
if prefs.get("media_tools.ytdlp_baseline") is None:
current = _ytdlp_status()
if current["origin"] == "bundled" and current["version"]:
prefs.set_("media_tools.ytdlp_baseline", current["version"])
overlay = _ytdlp_overlay_dir()
os.makedirs(media_tools_dir(), exist_ok=True)
with tempfile.TemporaryDirectory(dir=media_tools_dir()) as tmp:
whl = os.path.join(tmp, "yt_dlp.whl")
_download(url, whl, sha, None, op="ytdlp_update")
staged = os.path.join(tmp, "staged")
with zipfile.ZipFile(whl) as zf:
for member in zf.infolist():
name = member.filename
# Only the package itself; wheels carry no absolute paths but
# guard against traversal anyway.
if not name.startswith("yt_dlp/") or ".." in name:
continue
zf.extract(member, staged)
got = _read_ytdlp_version(os.path.join(staged, "yt_dlp"))
if not got:
raise RuntimeError("downloaded wheel has no readable yt_dlp version")
if os.path.isdir(overlay):
shutil.rmtree(overlay, ignore_errors=True)
os.replace(staged, overlay)
return version
def ytdlp_invocation() -> "tuple[list[str], dict[str, str] | None]":
"""(argv prefix, env-or-None) for running the yt-dlp CLI.
Prefers ``[sys.executable, -m, yt_dlp]`` so the CLI always matches the
module the app ships (or the user's overlay — propagated via PYTHONPATH),
with no PATH requirement: yt-dlp is never something the user installs.
Frozen builds can't re-invoke an interpreter, so they keep the historical
PATH lookup as a last resort.
"""
if not getattr(sys, "frozen", False):
try:
import importlib.util
if importlib.util.find_spec("yt_dlp") is not None:
env = None
overlay = _ytdlp_overlay_dir()
if os.path.isdir(os.path.join(overlay, "yt_dlp")):
env = dict(os.environ)
env["PYTHONPATH"] = overlay + os.pathsep + env.get("PYTHONPATH", "")
return [sys.executable, "-m", "yt_dlp"], env
except Exception as e:
logger.debug("yt_dlp module CLI unavailable: %s", e)
exe = shutil.which("yt-dlp")
return ([exe] if exe else ["yt-dlp"]), None
def restore_ytdlp() -> dict:
"""Delete the overlay — the locked, tested yt-dlp underneath takes over on
next start. Always safe: the locked install was never modified."""
overlay = _ytdlp_overlay_dir()
if os.path.isdir(overlay):
shutil.rmtree(overlay, ignore_errors=True)
_set_op("ytdlp_update", state="idle", progress=0.0, error=None, version=None)
return _ytdlp_status()
+137 -39
View File
@@ -657,6 +657,78 @@ def _hf_offline() -> bool:
return _env_flag("HF_HUB_OFFLINE") or _env_flag("TRANSFORMERS_OFFLINE")
# ── Broken-snapshot-link self-heal ───────────────────────────────────
# A sibling of the incomplete-cache class above: the blobs are FULLY
# downloaded, but the snapshots/<rev>/ entries pointing at them are dangling
# symlinks (0 KB) or zero-byte stand-ins — blob-naming mismatches between
# download modes, interrupted renames, or antivirus interference all produce
# this state (reported on Windows, where the NTFS links show as 0 KB, but the
# heal is generic). os.path.isfile() on a dangling link is False, so
# transformers raises the same "does not appear to have a file named …"
# signature even though the bytes are on disk. The resume repair below can't
# fix it (snapshot_download may trust/short-circuit on the existing broken
# entry), so rung 0 of the recovery ladder deletes exactly the broken entries
# and restores them — see services.hf_cache_repair.
# Repos this process already attempted the link self-heal for — the retry
# after a repair may only happen ONCE per repo per process, so a cache that
# stays broken can't loop repair↔retry.
_LINK_REPAIR_ATTEMPTED: set[str] = set()
def _selfheal_broken_snapshot_links(checkpoint: str) -> bool:
"""Rung 0 of cache recovery: delete-and-restore broken snapshot entries.
Returns True only when broken entries were found, removed AND restored
i.e. retrying the load is worth it. At most one attempt per repo per
process. Never raises; when it returns False the legacy resume/force
ladder still runs."""
if checkpoint in _LINK_REPAIR_ATTEMPTED:
return False
_LINK_REPAIR_ATTEMPTED.add(checkpoint)
if os.path.isdir(checkpoint):
return False # a local-directory checkpoint doesn't use the hub cache
try:
from services.hf_cache_repair import repair_repo_cache
summary = repair_repo_cache(checkpoint)
except Exception as repair_err: # repair must never break the ladder
logger.warning("Snapshot-link self-heal for %s errored: %s",
checkpoint, repair_err)
return False
if summary.get("removed") and summary.get("ok"):
logger.warning(
"Model cache for %s had %d broken file link(s) — repaired "
"automatically (%s), retrying the load.",
checkpoint, summary["removed"],
summary.get("outcome") or "healed",
)
return True
if summary.get("found"):
logger.warning(
"Model cache for %s has %d broken file link(s) that could not be "
"auto-repaired (%s).",
checkpoint, summary["found"], summary.get("error") or "unknown",
)
return False
def _manual_cache_delete_hint(checkpoint: str) -> str:
"""Names the exact on-disk folder to delete when every auto-repair rung
failed "delete the model" is only actionable if the user can find it.
Empty for local-directory checkpoints (they don't live in the hub cache)."""
try:
if os.path.isdir(checkpoint):
return ""
from services.hf_cache_repair import repo_cache_dir
return (
f" If the problem persists, quit OmniVoice, delete "
f"{repo_cache_dir(checkpoint)} and restart — the model "
"re-downloads automatically."
)
except Exception:
return ""
# Why the LAST _repair_model_cache run failed ("" when it succeeded / hasn't
# run). #886: the "could not be auto-repaired" message used to drop the cause
# entirely, so a mirror outage, offline mode, or a full disk all read the same.
@@ -852,50 +924,76 @@ def _load_model_sync():
# cache never reaches this branch, so the fast path is untouched).
if not _is_incomplete_cache_error(e):
raise
_set_loading("loading_weights", "Repairing incomplete model cache…")
if not _repair_model_cache(checkpoint):
raise RuntimeError(
f"The TTS model cache for {checkpoint} is incomplete "
"(weights missing — usually an interrupted download)."
f"{_repair_failure_detail()} "
"Open Settings → Models, delete the OmniVoice TTS model, "
"and install it again."
) from e
_set_loading("loading_weights", f"Loading TTS weights on {device}")
try:
_model = _load()
except OSError as e2:
# Resume-repair ran but the cache is still unusable. The usual
# cause beyond "repo genuinely lacks weights" is a blob that's
# present with the right size but corrupt — snapshot_download's
# resume trusts it and never re-fetches it (#739). Force a full
# re-download (replaces corrupt blobs) and retry once more before
# falling back to the manual delete-and-reinstall message.
if _is_incomplete_cache_error(e2):
_set_loading("loading_weights", "Re-downloading model files…")
if _repair_model_cache(checkpoint, force=True):
try:
_model = _load()
except OSError as e3:
# Rung 0: broken snapshot links — the blobs are on disk but the
# snapshot entries don't resolve (dangling symlinks / zero-byte
# stand-ins). Delete exactly the broken entries, restore, and
# retry the load ONCE (guarded per repo per process). A cache
# without broken links falls straight through to the resume
# ladder below.
_model = None
if _selfheal_broken_snapshot_links(checkpoint):
_set_loading(
"loading_weights",
"Model cache had broken file links — repaired "
"automatically, retrying…",
)
try:
_model = _load()
except OSError as e_link:
if not _is_incomplete_cache_error(e_link):
raise
logger.warning(
"Load still failing after snapshot-link repair of %s"
"falling back to resume repair.", checkpoint,
)
e = e_link
_model = None
if _model is None:
_set_loading("loading_weights", "Repairing incomplete model cache…")
if not _repair_model_cache(checkpoint):
raise RuntimeError(
f"The TTS model cache for {checkpoint} is incomplete "
"(weights missing — usually an interrupted download)."
f"{_repair_failure_detail()} "
"Open Settings → Models, delete the OmniVoice TTS model, "
f"and install it again.{_manual_cache_delete_hint(checkpoint)}"
) from e
_set_loading("loading_weights", f"Loading TTS weights on {device}")
try:
_model = _load()
except OSError as e2:
# Resume-repair ran but the cache is still unusable. The usual
# cause beyond "repo genuinely lacks weights" is a blob that's
# present with the right size but corrupt — snapshot_download's
# resume trusts it and never re-fetches it (#739). Force a full
# re-download (replaces corrupt blobs) and retry once more before
# falling back to the manual delete-and-reinstall message.
if _is_incomplete_cache_error(e2):
_set_loading("loading_weights", "Re-downloading model files…")
if _repair_model_cache(checkpoint, force=True):
try:
_model = _load()
except OSError as e3:
raise RuntimeError(
f"The TTS model cache for {checkpoint} is incomplete "
"and could not be auto-repaired. Open Settings → "
"Models, delete the OmniVoice TTS model, and install "
f"it again.{_manual_cache_delete_hint(checkpoint)}"
) from e3
else:
raise RuntimeError(
f"The TTS model cache for {checkpoint} is incomplete "
"and could not be auto-repaired. Open Settings → "
"Models, delete the OmniVoice TTS model, and install "
"it again."
) from e3
f"The TTS model cache for {checkpoint} is incomplete and "
f"could not be auto-repaired.{_repair_failure_detail()} "
"Open Settings → Models, delete the OmniVoice TTS model, "
f"and install it again.{_manual_cache_delete_hint(checkpoint)}"
) from e2
else:
raise RuntimeError(
f"The TTS model cache for {checkpoint} is incomplete and "
f"could not be auto-repaired.{_repair_failure_detail()} "
"Open Settings → Models, delete the OmniVoice TTS model, "
"and install it again."
"could not be auto-repaired. Open Settings → Models, delete "
"the OmniVoice TTS model, and install it again."
f"{_manual_cache_delete_hint(checkpoint)}"
) from e2
else:
raise RuntimeError(
f"The TTS model cache for {checkpoint} is incomplete and "
"could not be auto-repaired. Open Settings → Models, delete "
"the OmniVoice TTS model, and install it again."
) from e2
try:
# plan-02 (#65): gate on Triton availability (+ user setting), not
+36
View File
@@ -469,3 +469,39 @@ def clear_cache() -> None:
"""Testing hook — drop the in-process cache."""
with _cache_lock:
_cache.update(key=None, ts=0.0, report=None)
def clear_temp(temp_root: str | None = None) -> dict:
"""Delete the app-owned ``omnivoice*`` entries in the OS temp dir.
Removes exactly the population ``build_report`` counts as the "temp"
category direct children of ``temp_root`` whose basename starts with
``omnivoice`` so nothing outside OmniVoice's own working files can ever
be swept up. Symlinked entries are unlinked, never followed, so a stray
``omnivoice*`` link cannot make this delete its target's contents.
Returns ``{"removed": [basenames], "freed_bytes": int, "errors":
[{"path", "error"}]}`` partial failures (e.g. a file held open by a
running job on Windows) are reported per entry instead of aborting.
"""
temp_root = temp_root if temp_root is not None else tempfile.gettempdir()
removed: list[str] = []
errors: list[dict] = []
freed = 0
deadline = time.monotonic() + CATEGORY_TIMEOUT_SECONDS
for p in sorted(glob.glob(os.path.join(glob.escape(temp_root), "omnivoice*"))):
try:
if os.path.islink(p):
size = 0
os.unlink(p)
elif os.path.isfile(p):
size = os.path.getsize(p)
os.unlink(p)
else:
size, _complete, _err = _dir_size(p, deadline)
shutil.rmtree(p)
removed.append(os.path.basename(p))
freed += size
except OSError as e:
errors.append({"path": p, "error": str(e)})
return {"removed": removed, "freed_bytes": freed, "errors": errors}
+492
View File
@@ -0,0 +1,492 @@
"""Engine-agnostic text normalization — a conservative pre-pass before TTS.
Raw user text trips TTS engines: digits, clock times, and title abbreviations
mispronounce; zero-width junk and pathological repeat runs cause hallucinations
and long dead air. This module cleans text *once*, at the point where each
pipeline hands text to an engine (single-shot /generate, dub segments,
longform chapters), so every engine benefits equally.
Design rules (load-bearing):
* **Conservative.** A false negative (digits left alone) is fine; a false
positive (mangled meaning) is not. Anything ambiguous thousands-grouped
numbers ("1,000"), ranges ("3-5"), version strings ("v2", "3.5.1"),
leading-zero codes ("007"), 7+-digit IDs is left unchanged. Roman
numerals are out of scope entirely ("I" is a pronoun).
* **Idempotent.** ``normalize_text(normalize_text(x)) == normalize_text(x)``:
number/abbreviation output contains no digits or matchable tokens and the
safety filters are fixed-point by construction, so an accidental second
pass through a pipeline is harmless.
* **Per-language.** Numbers go through ``num2words`` only for languages it
supports (``_NUM2WORDS_LANGS``; the request's ``language`` is a full
display name from frontend/src/languages.json or an ISO-ish code both
resolve via :func:`_num2words_lang`). Everything else keeps its digits.
Clock times / ordinals / currency are English-only (their spoken form is
language-specific); decimals only for locales whose num2words rendering
was vetted. CJK scripts pass through the safety filters untouched no
CJK punctuation is stripped and no words are injected into unsegmented
text.
* **Markup-safe.** The single-bracket grammar (``[voice:]``, ``[pause ]``,
SSML-lite) and inline ``[[]]`` pronunciation overrides are never touched:
the language passes skip every ``[]`` span (same shape as chunked_tts's
``_BRACKET_TAG_RE``), so ``[pause 300ms]`` / ``[rate 0.9]`` stay parseable.
Ordering vs. the pronunciation dictionary (audited 2026-07-10): normalization
runs **BEFORE** ``services.pronunciation.apply_pronunciation`` (and before the
audiobook ``apply_lexicon`` overlay). Rationale from the code:
1. Dictionary respellings are the user's explicit, final say. If
normalization ran second it would re-process them a respelling that
deliberately contains digits or an abbreviation must reach the engine
verbatim.
2. Users already write lexicon entries against display text (the lexicon
docstring's own example is ``{"Dr": "Doctor"}``); entries keyed on
normalized words keep firing, and the dictionary stays the override for
anything the normalizer produced.
3. Inline ``[[]]`` overrides resolve last inside ``apply_pronunciation``
(and their bracketed content is masked here), so the user retains a
per-occurrence override over any normalizer output.
Pinned by ``tests/test_text_normalization.py`` (dictionary-order test).
Gate: prefs key ``text_normalization_enabled`` (default ON) with env override
``OMNIVOICE_TEXT_NORMALIZATION`` the same env-wins contract as
``OMNIVOICE_PRONUNCIATION`` ("0"/"false"/"no"/"off" disable).
:func:`normalize_for_tts` is the gated entry point every pipeline calls; it
never raises normalization is never allowed to break synthesis.
"""
from __future__ import annotations
import logging
import os
import re
from typing import Callable, Optional
logger = logging.getLogger("omnivoice.text_normalization")
ENV_VAR = "OMNIVOICE_TEXT_NORMALIZATION"
PREF_KEY = "text_normalization_enabled"
# ── Language resolution ───────────────────────────────────────────────────────
#
# The `language` kwarg across the app is normally a full display name from
# frontend/src/languages.json ("English", "German", …) — see
# resolve_kokoro_lang_code in services/tts_backend.py — but ISO-ish codes
# ("en", "pt-BR") also flow through dub/API callers. Map both to a num2words
# locale; anything unmapped keeps its digits (false negatives are fine).
_FULL_NAME_TO_CODE = {
"english": "en",
"german": "de",
"spanish": "es",
"french": "fr",
"italian": "it",
"portuguese": "pt",
"dutch": "nl",
"russian": "ru",
"ukrainian": "uk",
"polish": "pl",
"turkish": "tr",
"czech": "cs",
"danish": "da",
"finnish": "fi",
"swedish": "sv",
"norwegian": "no",
"norwegian bokmål": "no",
"norwegian nynorsk": "no",
"romanian": "ro",
"hungarian": "hu",
"indonesian": "id",
"lithuanian": "lt",
"latvian": "lv",
"slovenian": "sl",
"serbian": "sr",
"hebrew": "he",
"persian": "fa",
"azerbaijani": "az",
"vietnamese": "vi",
"kazakh": "kz",
"standard arabic": "ar",
}
# ISO codes whose num2words locale name differs.
_ISO_ALIASES = {"kk": "kz"}
# Locales verified against the pinned num2words (cardinal + basic rendering).
# zh/ja/ko/th are deliberately absent: unsegmented scripts where injecting
# space-delimited words is wrong, and their engines read digits natively.
_NUM2WORDS_LANGS = frozenset({
"en", "de", "es", "fr", "it", "pt", "nl", "ru", "uk", "pl", "tr", "cs",
"da", "fi", "sv", "no", "ro", "hu", "id", "lt", "lv", "sl", "sr", "ar",
"he", "fa", "az", "vi", "kz",
})
# Locales whose num2words decimal rendering was vetted ("drei Komma fünf",
# "три целых пять десятых", …). tr/vi are excluded on purpose: their 0.5
# renders as "fifty" (wrong), so decimals keep their digits there.
_DECIMAL_LANGS = frozenset({
"en", "de", "es", "fr", "it", "pt", "nl", "ru", "uk", "pl", "cs", "da",
"no", "sv", "fi", "ro", "hu", "id",
})
# "50%" → "fifty <word>" only where the spoken percent word is unambiguous.
_PERCENT_WORD = {
"en": "percent",
"de": "Prozent",
"es": "por ciento",
"fr": "pour cent",
"it": "per cento",
"pt": "por cento",
"nl": "procent",
}
_ISO_CODE_RE = re.compile(r"^([a-z]{2,3})(?:[-_]|$)")
def _num2words_lang(language: Optional[str]) -> Optional[str]:
"""Resolve a request language (display name or ISO-ish code) to a
num2words locale, or ``None`` when digits should be left alone."""
if not language:
return None
s = str(language).strip().lower()
if not s or s == "auto":
return None
code = _FULL_NAME_TO_CODE.get(s)
if code:
return code
m = _ISO_CODE_RE.match(s)
if m:
c = _ISO_ALIASES.get(m.group(1), m.group(1))
if c in _NUM2WORDS_LANGS:
return c
return None
# ── Universal safety filters (all languages) ─────────────────────────────────
# Zero-width & bidi controls, C0/C1 controls (except \t \n \r), BOM, U+FFFD.
_ZW_CONTROL_RE = re.compile(
"[\u0000-\u0008\u000b\u000c\u000e-\u001f\u007f-\u009f"
"\u200b-\u200f\u202a-\u202e\u2060-\u2064\ufeff\ufffd]"
)
# A tiny, unambiguous HTML-entity leftover set. `&amp;` is decoded only when
# NOT followed by a letter/`#` — so double-encoded junk ("&amp;nbsp;") is left
# alone rather than decoded one layer per pass (idempotency).
_ENTITIES = {
"&nbsp;": " ",
"&quot;": '"',
"&#39;": "'",
"&apos;": "'",
"&hellip;": "",
"&mdash;": "",
"&ndash;": "",
}
_ENTITY_RE = re.compile(
"(?:" + "|".join(re.escape(k) for k in _ENTITIES) + "|&amp;(?![a-zA-Z#]))"
)
# Same ASCII punctuation char repeated more than 3 times → capped at 3
# ("!!!!!!!!" / "........." cause dead air and babble). CJK punctuation and
# letters are deliberately untouched ("Nooooo" is expressive).
_REPEAT_RE = re.compile(r"([!?.,;:~_*#=-])\1{3,}")
_HSPACE_RE = re.compile(r"[^\S\n]+") # horizontal whitespace runs → one space
_NEWLINE_RE = re.compile(r"\n{3,}") # blank-line floods → one blank line
def _safety_filters(text: str) -> str:
out = _ZW_CONTROL_RE.sub("", text)
out = _ENTITY_RE.sub(lambda m: _ENTITIES.get(m.group(0), "&"), out)
out = _REPEAT_RE.sub(lambda m: m.group(1) * 3, out)
out = _HSPACE_RE.sub(" ", out)
out = _NEWLINE_RE.sub("\n\n", out)
return out.strip()
# ── Bracket masking ──────────────────────────────────────────────────────────
#
# Language passes must never rewrite `[…]` spans: `[pause 300ms]` /
# `[rate 0.9]` / `[voice:NAME]` are grammar, and `[[term|replacement]]`
# belongs to the pronunciation layer. Bounded repetition keeps it linear.
_BRACKET_SPAN_RE = re.compile(r"\[[^\][\n]{0,128}\]")
def _outside_brackets(text: str, fn: Callable[[str], str]) -> str:
if "[" not in text:
return fn(text)
parts: list[str] = []
last = 0
for m in _BRACKET_SPAN_RE.finditer(text):
parts.append(fn(text[last:m.start()]))
parts.append(m.group(0))
last = m.end()
parts.append(fn(text[last:]))
return "".join(parts)
# ── Abbreviation expansion ────────────────────────────────────────────────────
#
# Per-language (key, expansion, guard) triples. Matching is case-sensitive
# (a lowercase "st." is NOT the title "St."); lowercase connective keys
# ("e.g.") get an auto-added sentence-initial variant. Guards:
# "cap" — only before a capitalized word (titles precede names; leaves
# street-suffix "Elm St." / "Elm Dr." untouched).
# "digit" — only before a number ("No. 5"; leaves the word "No." alone).
_ABBREVIATIONS: dict[str, list[tuple[str, str, Optional[str]]]] = {
"en": [
("Dr.", "Doctor", "cap"),
("Mr.", "Mister", "cap"),
("Mrs.", "Missus", "cap"),
("Prof.", "Professor", "cap"),
("St.", "Saint", "cap"),
("Mt.", "Mount", "cap"),
("Jr.", "Junior", None),
("Sr.", "Senior", None),
("vs.", "versus", None),
("etc.", "et cetera", None),
("e.g.", "for example", None),
("i.e.", "that is", None),
("approx.", "approximately", None),
("No.", "number", "digit"),
],
"de": [
("Dr.", "Doktor", "cap"),
("Prof.", "Professor", "cap"),
("Nr.", "Nummer", "digit"),
("z.B.", "zum Beispiel", None),
("z. B.", "zum Beispiel", None),
("d.h.", "das heißt", None),
("d. h.", "das heißt", None),
("usw.", "und so weiter", None),
("bzw.", "beziehungsweise", None),
("ca.", "circa", None),
],
"es": [
("Sr.", "Señor", "cap"),
("Sra.", "Señora", "cap"),
("Srta.", "Señorita", "cap"),
("Dr.", "Doctor", "cap"),
("Dra.", "Doctora", "cap"),
("Ud.", "usted", None),
("Uds.", "ustedes", None),
("etc.", "etcétera", None),
("núm.", "número", "digit"),
],
"fr": [
# "M." is deliberately absent: indistinguishable from a middle initial.
("Mme", "Madame", "cap"),
("Mmes", "Mesdames", "cap"),
("Mlle", "Mademoiselle", "cap"),
("Mlles", "Mesdemoiselles", "cap"),
("etc.", "et cetera", None),
("", "numéro", "digit"),
("", "Numéro", "digit"),
],
}
_GUARD_LOOKAHEAD = {
None: "",
"cap": r"(?=\s+[A-ZÀ-ÖØ-Þ])",
"digit": r"(?=\s*\d)",
}
def _compile_abbreviations() -> dict[str, tuple[re.Pattern, dict[str, str]]]:
compiled: dict[str, tuple[re.Pattern, dict[str, str]]] = {}
for lang, entries in _ABBREVIATIONS.items():
entries = list(entries)
# Sentence-initial variants for lowercase connectives ("E.g." → …).
for key, expansion, guard in list(entries):
if key[:1].islower():
cap_key = key[0].upper() + key[1:]
if not any(k == cap_key for k, _, _ in entries):
entries.append((cap_key, expansion[0].upper() + expansion[1:], guard))
entries.sort(key=lambda e: len(e[0]), reverse=True) # longest key wins
lookup = {key: expansion for key, expansion, _ in entries}
alts = []
for key, _, guard in entries:
suffix = r"(?!\w)" if key[-1:].isalnum() else ""
alts.append(f"{re.escape(key)}{suffix}{_GUARD_LOOKAHEAD[guard]}")
# Literal alternation with per-key guards; no nested quantifiers.
pattern = re.compile(r"(?<![\w.])(?:" + "|".join(alts) + ")")
compiled[lang] = (pattern, lookup)
return compiled
_ABBREV_COMPILED = _compile_abbreviations()
def _expand_abbreviations(text: str, lang: str) -> str:
entry = _ABBREV_COMPILED.get(lang)
if entry is None:
return text
pattern, lookup = entry
def _repl(m: re.Match) -> str:
return lookup.get(m.group(0), m.group(0))
return pattern.sub(_repl, text)
# ── Numbers → words ──────────────────────────────────────────────────────────
#
# Every pattern requires clean word boundaries: digits glued to letters
# ("MP3", "v2"), separators ("1,000", "3-5", "1/2", "12:34:56"), leading
# zeros ("007") or 7+ digits (IDs, phone numbers) are all left alone.
# EN-only clock time: H:MM, 0-23 hours. Rejects H:MM:SS (durations).
_TIME_RE = re.compile(r"(?<![\d:.,])([01]?\d|2[0-3]):([0-5]\d)(?![\d:])")
# EN-only ordinal, suffix verified in the callback ("2th" stays as-is).
_ORDINAL_RE = re.compile(r"(?<![\w.,])(\d{1,4})(st|nd|rd|th)\b")
# EN-only dollars: $N or $N.CC. "$1,000" is blocked by the lookahead.
_CURRENCY_RE = re.compile(r"(?<!\w)\$(\d{1,6})(?:\.(\d{2}))?(?![\d.,])")
_PERCENT_RE = re.compile(r"(?<![\w.,])(\d{1,6}(?:\.\d{1,4})?)\s?%")
_DECIMAL_RE = re.compile(
r"(?<![\w.,:/$%-])(\d{1,6})\.(\d{1,6})(?![\w:/%-])(?![.,]\d)"
)
_INTEGER_RE = re.compile(
r"(?<![\w.,:/$%-])(?!0\d)(\d{1,6})(?![\w:/%-])(?![.,]\d)"
)
_ORDINAL_SUFFIX = {1: "st", 2: "nd", 3: "rd"}
def _correct_ordinal_suffix(n: int) -> str:
if 10 <= n % 100 <= 13:
return "th"
return _ORDINAL_SUFFIX.get(n % 10, "th")
def _numbers_to_words(text: str, lang: str) -> str:
try:
from num2words import num2words
except ImportError: # pragma: no cover — direct dependency; belt & braces
return text
def _safe(m: re.Match, render: Callable[[re.Match], str]) -> str:
# Any num2words hiccup leaves this occurrence untouched.
try:
return render(m)
except Exception: # noqa: BLE001 — conservative: never mangle
return m.group(0)
if lang == "en":
def _time(m: re.Match) -> str:
h, mm = int(m.group(1)), int(m.group(2))
hw = num2words(h, lang="en")
if mm == 0:
return f"{hw} o'clock"
if mm < 10:
return f"{hw} oh {num2words(mm, lang='en')}"
return f"{hw} {num2words(mm, lang='en')}"
text = _TIME_RE.sub(lambda m: _safe(m, _time), text)
def _ordinal(m: re.Match) -> str:
n = int(m.group(1))
if m.group(2) != _correct_ordinal_suffix(n):
return m.group(0)
return num2words(n, lang="en", to="ordinal")
text = _ORDINAL_RE.sub(lambda m: _safe(m, _ordinal), text)
def _currency(m: re.Match) -> str:
dollars = int(m.group(1))
if m.group(2) is not None:
amount = float(f"{m.group(1)}.{m.group(2)}")
return num2words(amount, lang="en", to="currency", currency="USD")
unit = "dollar" if dollars == 1 else "dollars"
return f"{num2words(dollars, lang='en')} {unit}"
text = _CURRENCY_RE.sub(lambda m: _safe(m, _currency), text)
percent_word = _PERCENT_WORD.get(lang)
if percent_word:
def _percent(m: re.Match) -> str:
raw = m.group(1)
if "." in raw:
if lang not in _DECIMAL_LANGS:
return m.group(0)
value: object = float(raw)
else:
value = int(raw)
return f"{num2words(value, lang=lang)} {percent_word}"
text = _PERCENT_RE.sub(lambda m: _safe(m, _percent), text)
if lang in _DECIMAL_LANGS:
def _decimal(m: re.Match) -> str:
return num2words(float(f"{m.group(1)}.{m.group(2)}"), lang=lang)
text = _DECIMAL_RE.sub(lambda m: _safe(m, _decimal), text)
def _integer(m: re.Match) -> str:
raw = m.group(1)
n = int(raw)
if len(raw) == 4 and 1500 <= n <= 2099:
# Bare 4-digit numbers in this range read as years
# ("nineteen eighty-four"); fall back to cardinal where the
# locale has no year form (sv, vi).
try:
return num2words(n, lang=lang, to="year")
except Exception: # noqa: BLE001
pass
return num2words(n, lang=lang)
return _INTEGER_RE.sub(lambda m: _safe(m, _integer), text)
# ── Public API ───────────────────────────────────────────────────────────────
def normalize_text(text: str, language: Optional[str] = None) -> str:
"""Pure, idempotent normalization pass (no pref gate — see
:func:`normalize_for_tts` for the gated entry point pipelines call)."""
if not text:
return text or ""
out = _safety_filters(text)
lang = _num2words_lang(language)
if lang:
if lang in _ABBREV_COMPILED:
out = _outside_brackets(out, lambda t: _expand_abbreviations(t, lang))
out = _outside_brackets(out, lambda t: _numbers_to_words(t, lang))
return out
def normalization_enabled() -> bool:
"""Env wins (power-user override, mirrors OMNIVOICE_PRONUNCIATION);
otherwise the ``text_normalization_enabled`` pref, default ON."""
env = os.environ.get(ENV_VAR)
if env is not None:
return env.strip().lower() not in ("0", "false", "no", "off", "")
try:
from core import prefs
return bool(prefs.get(PREF_KEY, True))
except Exception: # noqa: BLE001 — prefs unreadable → default ON
return True
def normalize_for_tts(text: str, language: Optional[str] = None) -> str:
"""Gated + hardened entry point: pref/env toggle, never raises.
Every TTS pipeline calls this exactly once, at its textengine choke
point, BEFORE the pronunciation dictionary (see module docstring).
"""
if not text:
return text or ""
if not normalization_enabled():
return text
try:
return normalize_text(text, language)
except Exception: # noqa: BLE001 — normalization must never break synth
logger.warning("text normalization failed; using raw text", exc_info=True)
return text
+3 -2
View File
@@ -30,7 +30,7 @@ logger = logging.getLogger("omnivoice.token_resolver")
Source = Literal["app", "env", "hf-cli"]
_PRIORITY: tuple[Source, ...] = ("app", "env", "hf-cli")
_CACHE_TTL_SECONDS = 300.0 # See Open Question #4 — UI "Test now" calls invalidate.
_CACHE_TTL_SECONDS = 300.0 # UI "Test now" busts it via GET /hf-token/state?fresh=1.
@dataclass(frozen=True)
@@ -57,7 +57,8 @@ _CACHE_LOCK = threading.Lock()
def invalidate_cache() -> None:
"""Drop the whoami validation cache. Called by the Settings UI "Test now"
button (Plan 01-02) and by save/clear API endpoints (Task 3)."""
button (GET /api/settings/hf-token/state?fresh=1), by save/clear API
endpoints, and by on_401()."""
with _CACHE_LOCK:
_VALIDATION_CACHE.clear()
+282
View File
@@ -0,0 +1,282 @@
"""
Two-stage translation quality for the LLM dub engine (provider="openai").
Stage 1 auto-glossary. ONE up-front LLM pass over the full transcript
extracts a short theme summary plus a sourcetarget terminology map for the
target language. The caller merges it with the user's manual glossary
(user entries always win) and injects the result into every per-segment
translation prompt, so recurring names/terms are rendered the same way in
segment 3 and segment 300. The extraction result is cached on the dub job
dict (``job["translation_context"][target_lang]``) and rides the existing
``job_data`` JSON blob no schema change; a transcript fingerprint keys the
cache so edited segments re-extract.
Stage 2 reflect pass. After a segment's direct LLM translation, a
critique-then-rewrite step reviews the draft for wordiness / stiff or
unnatural register and produces the final natural line. It runs on the SAME
client/model the translation used (the dub_translation skill's provider).
Failure policy for BOTH stages: refinement must never fail a segment. Any
error, timeout, empty output, or divergent rewrite silently keeps the direct
translation callers get ``None`` back and move on.
MT engines (argos/nllb/google/deepl/) never reach this module: they have no
prompts to inject into and no LLM to critique with. The Cinematic/Autofit
refine for those engines lives in ``services/translator.py``.
"""
from __future__ import annotations
import hashlib
import logging
import os
from typing import Iterable, Optional
logger = logging.getLogger("omnivoice.translation_quality")
# ── Prompts ──────────────────────────────────────────────────────────────────
# The context pass runs ONCE per (job, target language, transcript); the
# reflect prompts run twice per segment — keep them short, verbosity = wall time.
_CONTEXT_PROMPT = """\
You are a dubbing terminology editor preparing a translation brief. The user
gives you the full source-language transcript of one video. Reply in this
exact plain-text format (no JSON, no code fences, no commentary):
THEME: one or two sentences what the video is about, its register
(casual / formal / technical) and audience.
TERM: SOURCE || TARGET
TERM: SOURCE || TARGET
TERM lines list proper nouns (people, places, brands, product names) and
recurring domain terms that must be translated identically every time, each
with your preferred {target_name} rendering. At most {max_terms} TERM lines;
fewer is better. Skip one-off words and anything trivially consistent."""
_REVIEW_PROMPT = """\
You are a dubbing script reviewer. The user gives you a source line and its
draft {target_name} translation. In 1-2 short sentences, point out where the
draft is wordy, stiff, or uses a register nobody would use in spoken
dialogue, and whether recurring terms follow the brief. If the draft already
sounds natural, say so. Reply ONLY with the critique no headers, no lists,
no code fences."""
_POLISH_PROMPT = """\
You are a dubbing script writer. Rewrite the draft translation using the
reviewer's notes so it reads like natural spoken {target_name}. Keep the
meaning faithful to the source line, keep required terminology, and never add
content that is not in the source. Prefer the same length or shorter than the
draft. The output MUST stay in the same language and script as the draft
never switch language or transliterate. Reply ONLY with the final translation
no quotes, no notes, no commentary."""
def _chat(client, model: str, timeout: float, *, system: str, user: str) -> str:
"""One-shot chat completion on the caller's client. Raises on failure."""
res = client.chat.completions.create(
model=model,
timeout=timeout,
temperature=0.2, # pinned like the direct-translate path — 1.0 drifts
messages=[
{"role": "system", "content": system},
{"role": "user", "content": user},
],
)
return (res.choices[0].message.content or "").strip()
# ── Stage 1: auto-glossary (theme + terminology) ────────────────────────────
def transcript_fingerprint(segment_texts: Iterable[str]) -> str:
"""Stable hash of the transcript, so the per-job context cache invalidates
when the user edits segments between translate runs."""
h = hashlib.sha256()
for t in segment_texts:
h.update((t or "").strip().encode("utf-8", errors="replace"))
h.update(b"\x00")
return h.hexdigest()[:16]
def extract_context_sync(
client,
model: str,
timeout: float,
*,
segment_texts: Iterable[str],
source_lang: str,
target_lang: str,
source_name: Optional[str] = None,
target_name: Optional[str] = None,
max_terms: int = 30,
) -> Optional[dict]:
"""One LLM pass over the whole transcript → ``{"theme", "terms"}``.
``terms`` is ``[{"source", "target"}]``. Returns None on ANY failure or
when the response yields neither a theme nor terms the caller proceeds
without context, never errors. Blocking; run in an executor.
"""
text = "\n".join(t.strip() for t in segment_texts if t and t.strip())
if not text:
return None
# Same cap as the explicit glossary auto-extract endpoint — one shared
# knob for "how much transcript may ride a single LLM context call".
try:
max_chars = int(os.environ.get("OMNIVOICE_GLOSSARY_MAX_CHARS", "12000"))
except ValueError:
max_chars = 12000
if len(text) > max_chars:
text = text[:max_chars] + "\n…[truncated]"
system = _CONTEXT_PROMPT.format(
target_name=target_name or target_lang, max_terms=max_terms,
)
user = (
f"Source language: {source_name or source_lang}\n"
f"Target language: {target_name or target_lang}\n"
f"Transcript:\n{text}"
)
try:
body = _chat(client, model, timeout, system=system, user=user)
except Exception as e: # noqa: BLE001 — context is an enhancement, never a gate
logger.warning("auto-glossary context pass failed: %s", e)
return None
theme = ""
terms: list[dict] = []
for line in body.splitlines():
line = line.strip()
if not line:
continue
upper = line.upper()
if upper.startswith("THEME:"):
theme = line[len("THEME:"):].strip()
continue
if upper.startswith("TERM:"):
line = line[len("TERM:"):].strip()
if "||" not in line:
continue
parts = [p.strip() for p in line.split("||")]
if len(parts) < 2 or not parts[0] or not parts[1]:
continue
terms.append({"source": parts[0], "target": parts[1]})
if len(terms) >= max_terms:
break
if not theme and not terms:
logger.warning("auto-glossary context pass returned nothing parseable")
return None
return {"theme": theme, "terms": terms}
def merge_glossary(
user_terms: Optional[Iterable[dict]],
auto_terms: Optional[Iterable[dict]],
) -> list[dict]:
"""Merge manual + auto glossaries. User entries ALWAYS win: an auto term
whose source matches a user source (case-insensitive) is dropped."""
merged: list[dict] = []
seen: set[str] = set()
for entry in user_terms or []:
src = (entry.get("source") or "").strip()
tgt = (entry.get("target") or "").strip()
if not src or not tgt:
continue
merged.append(entry)
seen.add(src.lower())
for entry in auto_terms or []:
src = (entry.get("source") or "").strip()
tgt = (entry.get("target") or "").strip()
if not src or not tgt or src.lower() in seen:
continue
merged.append({"source": src, "target": tgt})
seen.add(src.lower())
return merged
def context_clause(theme: str, terms: Optional[Iterable[dict]]) -> str:
"""Prompt fragment carrying the theme + merged glossary into every
per-segment translation prompt. Empty string when there's nothing."""
parts: list[str] = []
theme = (theme or "").strip()
if theme:
parts.append(f"Video context: {theme}")
lines = []
for entry in terms or []:
src = (entry.get("source") or "").strip()
tgt = (entry.get("target") or "").strip()
if not src or not tgt:
continue
note = (entry.get("note") or "").strip()
lines.append(f"- {src}{tgt}" + (f" (note: {note})" if note else ""))
if lines:
parts.append(
"Terminology — render every occurrence of a source term exactly "
"as its target:\n" + "\n".join(lines)
)
return "\n".join(parts)
# ── Stage 2: reflect pass (critique → rewrite) ──────────────────────────────
def reflect_translation_sync(
client,
model: str,
timeout: float,
*,
source_text: str,
direct_text: str,
source_lang: str,
target_lang: str,
target_name: Optional[str] = None,
extra_clause: str = "",
) -> Optional[str]:
"""Critique-then-rewrite the direct translation of one segment.
Returns the polished line, or None whenever the direct translation should
stand: any LLM failure/timeout, an empty rewrite, or a rewrite that
diverged from the draft (wrong script, runaway length, critique echoed
back the shared ``refine_output_ok`` guard). Never raises. Blocking;
run in an executor.
"""
if not direct_text or not direct_text.strip():
return None
tgt_name = target_name or target_lang
def _with_clause(base: str) -> str:
return base + "\n\n" + extra_clause if extra_clause.strip() else base
try:
review_user = (
f"Source ({source_lang}): {source_text}\n"
f"Draft translation ({target_lang}): {direct_text}"
)
critique = _chat(
client, model, timeout,
system=_with_clause(_REVIEW_PROMPT.format(target_name=tgt_name)),
user=review_user,
)
polish_user = review_user + f"\nReviewer's notes: {critique}"
polished = _chat(
client, model, timeout,
system=_with_clause(_POLISH_PROMPT.format(target_name=tgt_name)),
user=polish_user,
)
except Exception as e: # noqa: BLE001 — refinement must never fail a segment
logger.warning("reflect pass failed (%s) — keeping direct translation", e)
return None
polished = (polished or "").strip()
if not polished or polished == direct_text:
return None
# Same divergence guard the Cinematic ADAPT step uses: wrong script,
# runaway length, or the critique leaking through as the "translation".
from services.translator import refine_output_ok
ok, reason = refine_output_ok(direct_text, polished, target_lang, critique=critique)
if not ok:
logger.warning(
"reflect pass diverged for %s (%s) — keeping direct translation",
target_lang, reason,
)
return None
return polished
+239 -10
View File
@@ -6,7 +6,7 @@ A uniform protocol for every TTS engine. Today we ship:
OmniVoiceBackend wraps the current k2-fsa/OmniVoice model. Zero
behaviour change for existing callers.
VoxCPM2Backend thin stub that raises with a clear install hint
until `pip install voxcpm` is present and enabled.
until `pip install "voxcpm>=2.0.3"` is present and enabled.
Callers should use `get_active_tts_backend()` to pick the configured engine
instead of importing a specific class. The selection is controlled by the
@@ -55,6 +55,27 @@ def _mask_hf_tokens(value):
return _HF_TOKEN_MASK_RE.sub(_HF_TOKEN_MASK, value)
def _available_hint(msg) -> Optional[str]:
"""Advisory text carried by an *available* engine's ``is_available()``
message, or None when the message is a plain readiness echo.
Convention (established by VoxCPM2's version-floor hint): an engine
that is available but wants the user to know something returns
``(True, "ready — <advice>")``. This extracts ``<advice>`` so
:func:`list_backends` can surface it previously the whole message
was dropped for available rows (``reason`` is None when ok), so
upgrade hints never reached the UI. Plain "ready" / "ready (…)"
messages yield None. Output is token-masked like ``reason``.
"""
if not isinstance(msg, str):
return None
head, sep, advice = msg.partition("")
advice = advice.strip()
if not sep or not advice or not head.strip().lower().startswith("ready"):
return None
return _mask_hf_tokens(advice)
# ── HF Hub closed-client recovery (#880) ────────────────────────────────────
#
# huggingface_hub ≥1.x shares ONE global httpx client across every download.
@@ -142,6 +163,26 @@ class TTSBackend(ABC):
#: (e.g. "young female, warm tone, British accent") without reference audio.
supports_voice_design: bool = False
def ensure_ready(self) -> None:
"""Load model weights now (blocking), so callers can separate the
LOAD budget from the GENERATE budget (#1033/#1037 class).
Every adapter lazily loads inside ``generate()`` via a private
``_ensure_loaded()`` which meant a cold first call spent its whole
``OMNIVOICE_GENERATE_TIMEOUT_S`` window (default 300s) downloading /
loading weights and got killed with a misleading "too heavy for the
available compute" error (measured in the wild on a fresh install:
multi-GB checkpoint download, 0% GPU util, #1014). Routes call this
first under the model-load budget (``OMNIVOICE_MODEL_LOAD_TIMEOUT``,
default 1200s), then start the generate clock on an already-warm
engine. Default implementation dispatches to the adapter's own
``_ensure_loaded`` when present; engines without lazy state no-op.
Must be called on the GPU pool (it's blocking), same as generate.
"""
loader = getattr(self, "_ensure_loaded", None)
if callable(loader):
loader()
#: Whether this engine already emits mastered, studio-grade audio and should
#: therefore skip the shared apply_mastering() chain (highpass + Compressor,
#: tuned for OmniVoice's 24 kHz output). Studio engines like VoxCPM2 (native
@@ -387,9 +428,155 @@ class OmniVoiceBackend(TTSBackend):
# ── VoxCPM2 adapter (optional, scaffolded) ──────────────────────────────────
#: Minimum recommended `voxcpm` package version. 2.0.3 fixed an audio-quality
#: bug on Apple Silicon (low-precision dtypes on the MPS device produced
#: degraded output). A floor, NOT a pin: newer versions are fine, and an
#: already-installed older version keeps working — we only surface an upgrade
#: hint (is_available reason + load-time warning), never force a reinstall.
_VOXCPM_MIN_VERSION = "2.0.3"
#: Reference-clip cap for VoxCPM2 cloning (seconds). The `voxcpm` package no
#: longer trims reference audio internally, so an unbounded user clip would
#: condition the model on minutes of audio (slow, and past a point it stops
#: helping voice similarity). 30 s is a conservative upper bound.
_VOXCPM_REF_MAX_S = 30.0
#: Silence pad kept around the voiced region when trimming a reference clip —
#: a hard cut exactly at the first/last voiced sample clips consonant onsets.
_VOXCPM_REF_EDGE_PAD_S = 0.05
def _version_tuple(v: str) -> Optional[tuple[int, ...]]:
"""Parse the leading numeric components of a version string ("2.0.3"
(2, 0, 3), "2.1rc1" (2, 1)). Returns None when nothing numeric parses
callers treat that as 'unknown, assume fine' rather than failing."""
parts: list[int] = []
for piece in v.split("."):
digits = ""
for ch in piece:
if not ch.isdigit():
break
digits += ch
if not digits:
break
parts.append(int(digits))
return tuple(parts) if parts else None
def _voxcpm_installed_version() -> Optional[str]:
"""Installed `voxcpm` dist version, or None when undeterminable
(not installed, or importable without package metadata)."""
try:
from importlib.metadata import version
return version("voxcpm")
except Exception:
return None
def _voxcpm_upgrade_hint() -> Optional[str]:
"""Actionable upgrade hint when the installed `voxcpm` is older than
:data:`_VOXCPM_MIN_VERSION`, else None. Never raises; an unparseable or
unknown version yields None (don't nag users we can't be sure about)."""
installed = _voxcpm_installed_version()
if installed is None:
return None
have = _version_tuple(installed)
want = _version_tuple(_VOXCPM_MIN_VERSION)
if have is None or want is None or have >= want:
return None
return (
f"installed voxcpm {installed} is older than {_VOXCPM_MIN_VERSION}, "
"which fixed an audio-quality bug on Apple Silicon (low-precision "
"dtypes on MPS). The engine still works, but upgrading is "
'recommended: pip install --upgrade "voxcpm>=2.0.3"'
)
# Prepared-reference cache: (abspath, mtime_ns, size) → prepared path (which
# may be the original path itself when no trim/cap applied). Keeps repeat
# generations from re-reading + re-writing the same clip, and keeps the temp
# dir from filling with one copy per generate() call.
_VOXCPM_REF_PREP_CACHE: dict[tuple, str] = {}
def _prepare_voxcpm_ref(path: str) -> str:
"""Prepare a cloning reference clip for VoxCPM2.
The `voxcpm` package used to trim reference audio itself but no longer
does raw user clips reach the model unconditioned. This applies the
minimal, conservative preparation the model expects:
trim leading/trailing near-silence (amplitude threshold at the same
-50 dBFS floor `audio_dsp.normalize_audio` uses, with a small
:data:`_VOXCPM_REF_EDGE_PAD_S` pad kept on each side), and
cap the reference at :data:`_VOXCPM_REF_MAX_S` seconds from the
trimmed start.
Returns a path to the prepared WAV. Deliberately non-destructive and
fail-open: the ORIGINAL path is returned unchanged when the clip needs no
meaningful trim/cap (short clean clips pass through untouched), when the
whole clip sits below the silence floor (nothing to anchor a trim on), or
when anything at all goes wrong reference prep must never be the reason
a generation fails.
"""
try:
import numpy as np
import soundfile as sf
abspath = os.path.abspath(path)
st = os.stat(abspath)
cache_key = (abspath, st.st_mtime_ns, st.st_size)
cached = _VOXCPM_REF_PREP_CACHE.get(cache_key)
if cached is not None and (cached == abspath or os.path.exists(cached)):
return cached
audio, sr = sf.read(abspath, dtype="float32", always_2d=True) # (n, ch)
n = audio.shape[0]
if n == 0 or sr <= 0:
return path
# Silence floor: -50 dBFS, matching audio_dsp.normalize_audio. A clip
# that never rises above it is left alone (fail-open, see docstring).
floor = 10 ** (-50.0 / 20.0)
envelope = np.abs(audio).max(axis=1)
voiced = np.flatnonzero(envelope > floor)
if voiced.size == 0:
_VOXCPM_REF_PREP_CACHE[cache_key] = abspath
return path
pad = int(_VOXCPM_REF_EDGE_PAD_S * sr)
start = max(0, int(voiced[0]) - pad)
end = min(n, int(voiced[-1]) + 1 + pad)
cap = int(_VOXCPM_REF_MAX_S * sr)
end = min(end, start + cap)
# No-op path: nothing meaningful to cut (>0.1 s total) — hand the
# original file to the model byte-identical.
if (start + (n - end)) <= int(0.1 * sr):
_VOXCPM_REF_PREP_CACHE[cache_key] = abspath
return path
import tempfile
fd, prepared = tempfile.mkstemp(prefix="voxcpm_ref_", suffix=".wav")
os.close(fd)
sf.write(prepared, audio[start:end], sr)
_VOXCPM_REF_PREP_CACHE[cache_key] = prepared
logger.info(
"VoxCPM2: prepared reference clip %s%s (%.2fs → %.2fs; "
"silence trimmed, cap %.0fs)",
path, prepared, n / sr, (end - start) / sr, _VOXCPM_REF_MAX_S,
)
return prepared
except Exception as e: # noqa: BLE001 — prep is best-effort by contract
logger.warning(
"VoxCPM2: reference-clip preparation failed for %s — using the "
"raw clip: %s", path, e,
)
return path
class VoxCPM2Backend(TTSBackend):
"""OpenBMB VoxCPM2 wrapper — `pip install voxcpm` required.
"""OpenBMB VoxCPM2 wrapper — `pip install "voxcpm>=2.0.3"` required.
Ships as a scaffold: the class loads and reports unavailability cleanly
when the dep isn't installed, so Settings UI can gate the engine selector
@@ -417,10 +604,17 @@ class VoxCPM2Backend(TTSBackend):
import voxcpm # noqa: F401
except ImportError:
return False, (
"voxcpm package not installed. Install with `pip install voxcpm` "
"voxcpm package not installed. Install with "
'`pip install "voxcpm>=2.0.3"` '
"(requires Python ≥3.10, PyTorch ≥2.5). CUDA ≥12 recommended "
"for full speed; MPS (Apple Silicon) and CPU also supported."
)
# Version FLOOR, not pin: an older install still reports available
# (no forced reinstall), but the reason carries the upgrade hint and
# _ensure_loaded() logs it at load time.
hint = _voxcpm_upgrade_hint()
if hint:
return True, f"ready — {hint}"
return True, "ready"
@property
@@ -442,6 +636,9 @@ class VoxCPM2Backend(TTSBackend):
ok, msg = self.is_available()
if not ok:
raise RuntimeError(f"VoxCPM2 unavailable: {msg}")
hint = _voxcpm_upgrade_hint()
if hint:
logger.warning("VoxCPM2: %s", hint)
from voxcpm import VoxCPM # type: ignore[import-not-found]
checkpoint = os.environ.get("OMNIVOICE_VOXCPM_MODEL", "openbmb/VoxCPM2")
logger.info("Loading VoxCPM2 from %s", checkpoint)
@@ -471,14 +668,16 @@ class VoxCPM2Backend(TTSBackend):
cfg_value=kw.get("guidance_scale", 2.0),
inference_timesteps=kw.get("num_step", 10),
)
if isinstance(wav, np.ndarray):
wav = torch.from_numpy(wav).float()
if wav.ndim == 1:
wav = wav.unsqueeze(0)
return wav
return self._finalize(wav)
# ── Standard clone / instruct mode ──────────────────────────────
# Map our instruct prop onto VoxCPM2's inline "(instruct)prompt" prefix.
# The reference clip is prepared first (edge-silence trim + length
# cap) — the model no longer trims it internally, so a raw user clip
# would condition generation on dead air. Fail-open: on any prep
# problem the raw path is used, exactly as before.
if ref_audio:
ref_audio = _prepare_voxcpm_ref(ref_audio)
prompt = text
if instruct:
prompt = f"({instruct}){text}"
@@ -490,11 +689,26 @@ class VoxCPM2Backend(TTSBackend):
prompt_wav_path=ref_audio if ref_text else None,
prompt_text=ref_text,
)
return self._finalize(wav)
def _finalize(self, wav) -> torch.Tensor:
"""Normalize model output to a (1, n) float tensor and apply the
trailing-silence guard.
The guard is a SILENCE trim only: generations often end with a long
near-silent tail, which this cuts (keeping a short ~0.3 s natural
tail). It deliberately does NOT attempt to detect or judge trailing
*content* an output that ends in audible audio, wanted or not,
passes through unchanged, as does any output without a silent tail.
"""
import numpy as np
from services.audio_dsp import trim_trailing_silence
if isinstance(wav, np.ndarray):
wav = torch.from_numpy(wav).float()
if wav.ndim == 1:
wav = wav.unsqueeze(0)
return wav
return trim_trailing_silence(wav, self.sample_rate)
# ── MOSS-TTS-Nano adapter (tiny, CPU-friendly, 20 langs) ────────────────────
@@ -1439,7 +1653,7 @@ _INSTALL_HINTS: dict[str, str] = {
"cosyvoice": "git clone --recursive FunAudioLLM/CosyVoice + pip install -r requirements.txt + SoX",
"kittentts": "pip install kittentts (ONNX, CPU-only, ~80 MB)",
"mlx-audio": "pip install mlx-audio (Apple Silicon only)",
"voxcpm2": "pip install voxcpm (CPU/MPS supported; CUDA recommended for speed)",
"voxcpm2": 'pip install "voxcpm>=2.0.3" (floor: 2.0.3 fixed Apple-Silicon audio quality; CPU/MPS supported, CUDA recommended for speed)',
"moss-tts-nano": "git clone OpenMOSS/MOSS-TTS-Nano && pip install -e . (not on PyPI)",
"indextts2": "git clone index-tts/index-tts && uv pip install -e . (NOT uv sync --all-extras)",
"gpt-sovits": "External API server — start api_v2.py on port 9880",
@@ -1494,11 +1708,16 @@ def list_backends() -> list[dict]:
"display_name": str,
"available": bool,
"reason": Optional[str], # message when not available
"hint": Optional[str], # advice when available-but-has-advice
# (is_available "ready — <advice>" convention;
# e.g. VoxCPM2's >=2.0.3 upgrade hint)
"install_hint": Optional[str],
"setup_snippet": Optional[str], # exact `export VAR=...` for path-gated opt-in engines
"last_error": Optional[str], # cached most-recent failure
"isolation_mode": "in-process" | "subprocess",
"gpu_compat": list[str], # subset of {cuda, rocm, mps, xpu, cpu}
"supports_cloning": Optional[bool], # True/False from the class attr; None when
# model-dependent (property, e.g. mlx-audio)
"effective_device": str, # device this engine uses on THIS host
"routing_status": "accelerated" | "cpu_fallback" | "cpu_only" | "unavailable",
"routing_reason": Optional[str], # scrubbed; null when none
@@ -1553,11 +1772,21 @@ def list_backends() -> list[dict]:
else:
isolation = "in-process"
gpu_compat = getattr(cls, "gpu_compat", ("cpu",))
# Cloning capability: same descriptor guard as
# cloning_capable_engine_ids() — a class-level getattr on a *property*
# (mlx-audio: capability depends on the picked model) returns the
# descriptor, not a bool, so report None (= model-dependent) there
# instead of an always-truthy false positive.
_clone = getattr(cls, "supports_cloning", True)
out.append({
"id": bid,
"display_name": cls.display_name,
"available": ok,
"reason": None if ok else _mask_hf_tokens(msg),
# Available-but-has-advice (e.g. VoxCPM2's ">=2.0.3 recommended"
# upgrade hint). None unless ok and the message carries advice.
"hint": _available_hint(msg) if ok else None,
"supports_cloning": _clone if isinstance(_clone, bool) else None,
"install_hint": _INSTALL_HINTS.get(bid),
# Exact `export VAR=...` line for path-gated opt-in engines, or None.
"setup_snippet": _SETUP_SNIPPETS.get(bid),
+39 -6
View File
@@ -39,6 +39,27 @@ _audioseal_available: Optional[bool] = None
# This is our signature — every OmniVoice-generated audio carries it.
OMNI_MESSAGE = [0, 1, 0, 0, 1, 1, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1]
# Watermark ops run chunk-by-chunk: AudioSeal's activation memory grows
# linearly with input length — a single multi-minute waveform demands a
# multi-GB CPU buffer, which OOM'd a 16 GB machine mid-generate (#1045).
# 30 s bounds each call to tens of MB; the 16-bit message repeats throughout
# the audio, so per-chunk embedding/detection is equivalent.
_CHUNK_SECONDS = 30
def _iter_chunks(audio: torch.Tensor, sample_rate: int):
"""Yield ≤ ~_CHUNK_SECONDS slices of (batch, channels, samples) audio
along the time axis. A sub-second tail is folded into the previous chunk
(AudioSeal embeds poorly on very short segments)."""
total = audio.shape[-1]
step = _CHUNK_SECONDS * sample_rate
starts = list(range(0, total, step))
if len(starts) > 1 and total - starts[-1] < sample_rate:
starts.pop()
for i, start in enumerate(starts):
end = starts[i + 1] if i + 1 < len(starts) else total
yield audio[..., start:end]
def _check_available() -> bool:
"""Check if AudioSeal is installed and importable."""
@@ -135,7 +156,13 @@ def embed_watermark(
# AudioSeal operates at 16kHz internally; it handles resampling, but
# we need to inform it of the source rate for correct embedding.
watermarked = generator(audio, sample_rate=sample_rate, message=msg)
watermarked = torch.cat(
[
generator(seg, sample_rate=sample_rate, message=msg)
for seg in _iter_chunks(audio, sample_rate)
],
dim=-1,
)
# Restore original shape
if len(original_shape) == 2:
@@ -189,11 +216,17 @@ def detect_watermark(
else:
audio = waveform
result = detector.detect_watermark(audio, sample_rate=sample_rate, message_threshold=0.5)
# result is (detection_confidence, decoded_message)
confidence = float(result[0]) if isinstance(result, tuple) else 0.0
decoded_msg = result[1] if isinstance(result, tuple) and len(result) > 1 else None
# Detect per chunk and keep the best hit: bounds memory the same way
# embedding does, and a splice where only part of the file is
# OmniVoice audio still registers (a whole-file average would dilute it).
best_conf, decoded_msg = -1.0, None
for seg in _iter_chunks(audio, sample_rate):
result = detector.detect_watermark(seg, sample_rate=sample_rate, message_threshold=0.5)
seg_conf = float(result[0]) if isinstance(result, tuple) else 0.0
if seg_conf > best_conf:
best_conf = seg_conf
decoded_msg = result[1] if isinstance(result, tuple) and len(result) > 1 else None
confidence = max(best_conf, 0.0)
# Decode message bits
message_bits = ""
+10
View File
@@ -58,6 +58,10 @@ RUN uv pip install --system --no-cache .
# Copy application source
COPY backend/ ./backend/
COPY omnivoice/ ./omnivoice/
# Alembic config so schema migrations run natively on existing volumes
# (without it the backend fell back to the additive-column self-heal —
# functional, but the real migration chain is the first-class path).
COPY alembic.ini ./
# Copy the pre-built React frontend from the builder stage
COPY --from=frontend-builder /app/frontend/dist ./frontend/dist
@@ -65,6 +69,12 @@ COPY --from=frontend-builder /app/frontend/dist ./frontend/dist
# Expose the single unified API and UI port
EXPOSE 3900
# Image-level health probe (compose files define their own; this covers plain
# `docker run`). Generous start period: first boot creates the venv-less
# schema + may pull model metadata before /health answers.
HEALTHCHECK --interval=30s --timeout=5s --start-period=120s --retries=5 \
CMD curl -fsS http://127.0.0.1:3900/health || exit 1
# Mount points for persistent data (sqlite db, user voices, huggingface cache)
VOLUME ["/app/omnivoice_data"]
+40
View File
@@ -86,6 +86,46 @@ translation is produced:
Cinematic and Autofit **require an LLM** (below). If none is configured, they
fall back to Fast with a notice.
## Two-stage quality on the LLM engine (auto-glossary + reflect pass)
When the **LLM (OpenAI-compatible)** engine is the active translator, two extra
quality stages run by default. Both have checkboxes next to the Quality control
in the Dub tab's translation settings (they only appear for the LLM engine —
MT engines can't run either stage):
- **Auto glossary** — before the per-segment translation, ONE extra LLM pass
reads the whole transcript and extracts a short theme summary plus a
source → target terminology map. That brief rides every segment's translation
prompt, so character names, places, and recurring domain terms come out the
same in segment 3 and segment 300. It's merged with your manual glossary —
**your entries always win** on a clashing term. The result is cached with the
dub project per target language, so re-translating an unchanged transcript
costs zero extra calls; editing segments re-extracts.
- **Reflect pass** — after each segment's direct translation, the LLM critiques
the draft for wordiness and stiff/unnatural register, then rewrites it as
natural spoken dialogue. **This uses 3 LLM calls per segment instead of 1**
turn it off for long videos on slow or metered providers. If any refinement
step fails or times out, the direct translation is kept silently; refinement
can never fail a segment.
### Fit prediction (all quality levels)
Every translation additionally gets a **pre-synthesis fit check** — no LLM
needed. For each segment, OmniVoice predicts how long the translated line will
take to speak (self-calibrating to your voice/engine from segments already
generated in the job, with a per-language rate table as the cold-start
fallback) and compares it against the slot plus the silence it can borrow
before the next line. Segments the Smart Fit caps can only absorb with an
audible speed-up get a **Tight fit** badge; segments no fitting can save get a
**Won't fit +Ns** badge — so you can shorten the text *before* burning GPU
time on a line that would end up trimmed. Badges are informational only:
generation is never blocked.
**Suggest shorter lines** (checkbox under Quality, off by default) goes one
step further: for every "Won't fit" segment it asks the configured LLM for a
meaning-preserving shorter rewrite and offers it on the row as a one-click
**Use shorter rewrite** suggestion. It never rewrites anything automatically,
and with no LLM configured (or on any LLM error) it simply does nothing.
## LLM Providers (for Cinematic / Autofit)
**Settings → System → LLM Providers** is the one place to set up the LLM. Pick a
+60
View File
@@ -0,0 +1,60 @@
# Verified Tesla T4 (16GB) inference notes
Measured on a real NVIDIA Tesla T4 (16GB, Turing/sm_75), driver 550.163.01 (CUDA 12.8), torch
2.8.0+cu128, transformers 5.3.0, Python 3.11.15 (uv-managed). Engine under test: the default
`omnivoice` TTS backend (`OMNIVOICE_TTS_BACKEND=omnivoice`).
## Cold-cache first call can time out at 300s
The first `generate()` call lazily downloads the ~2.3GB `k2-fsa/OmniVoice` checkpoint, and that
download happens *inside* the `OMNIVOICE_GENERATE_TIMEOUT_S` budget (default 300s). On a fresh
install, the very first `POST /v1/audio/speech` can fail like this even though the GPU isn't
actually short on memory:
```
ERROR [omnivoice.openai_compat] OpenAI TTS failed: OpenAI TTS generate exceeded 300s and was
abandoned — the backend is running, but the job was too heavy for the available compute.
... most often the GPU is VRAM-starved ...
```
VRAM sampling during the failure showed a flat ~2GB with 0% GPU utilization for the whole 300s —
consistent with waiting on a download, not compute. Once the checkpoint is cached, the identical
request succeeds in ~1s (reproduced 5x: 1.574s / 1.034s / 1.065s / 0.995s / 0.911s).
**Workaround (no code change needed, both already exist):**
- For headless/API-only setups, pre-fetch the checkpoint before your first real TTS request:
```bash
curl -X POST http://localhost:3900/models/install \
-H "Content-Type: application/json" \
-d '{"repo_id": "k2-fsa/OmniVoice"}'
```
(`repo_id` is required — `InstallModelRequest` in `backend/api/schemas.py` rejects a bare/empty
body — and must match one of the entries in `KNOWN_MODELS`, e.g. the default engine's
`k2-fsa/OmniVoice`.) Progress streams over the existing `/setup/download-stream` SSE feed.
- Or raise `OMNIVOICE_GENERATE_TIMEOUT_S` for the first request.
## OpenAI-compatible endpoint doesn't expose `num_step` / `guidance_scale`
`POST /v1/audio/speech`'s request schema doesn't declare `num_step` or `guidance_scale` fields —
sending them in the JSON body returns `200 OK` but they're silently discarded (pydantic's default
`extra=ignore` behavior). The native multipart `POST /generate` endpoint *does* expose both as
explicit form fields, so use that endpoint if you need to control them.
Separately: the app's own default for `num_step` is 16 — half of the model's documented default of
32 (see `docs/generation-parameters.md`, "Use 16 for faster inference"). Not a bug, just not stated
that the app already runs the "fast" preset unless you override it via `/generate`.
## T4 acceleration checklist
| Option | Status |
|---|---|
| dtype | `torch.float16` hardcoded for the `omnivoice` engine (`model_manager.py`) — correct for Turing (no bf16 tensor cores this generation). No env var override for this engine specifically (ASR engines have `ASR_COMPUTE_TYPE`; `dots_tts`/`indextts` have their own precision vars; `omnivoice` doesn't). |
| Attention | `sdpa`, selected automatically since `flash_attn` isn't installed (`_supports_flash_attn_2=True` is declared but the package itself is absent) — safe on T4. |
| int8 | No int8 path for this engine (ASR's CTranslate2 `int8` and `sherpa-onnx`'s int8 ONNX models are separate/unrelated). |
| CUDA Graphs | No direct API usage in the app. Reachable indirectly via `torch.compile(mode="reduce-overhead")`, which the app attempts **by default** on this GPU (T4/sm_75 isn't in the framework's compile-exclusion list, unlike newer/Blackwell GPUs). The numbers above were measured with `TORCH_COMPILE_DISABLE=1` for a clean eager baseline. |
| torch.compile | Attempted by default on T4 (see above) — not evaluated further here. |
## VRAM
Peak measured: 2487 MiB (`nvidia-smi`) / 2.050 GB (`torch.cuda.max_memory_allocated()`) for the
default `omnivoice` engine — comfortably fits even the README's stated "minimum" (4GB) tier.
+1 -1
View File
@@ -13,7 +13,7 @@ and [`palashdeb/omnivoice-studio` on Docker Hub](https://hub.docker.com/r/palash
> |-----|--------------|
> | `:latest` | **Rolling preview** — latest commit on `main` (always one patch ahead of the last release). This is the preview channel; pin `:stable` for production. |
> | `:stable` | Most recent versioned release (updated on every `v*` git tag) |
> | `:0.3.6` | Exact release version |
> | `:0.3.17` | Exact release version |
> | `:0.3` | Latest patch within the 0.3 minor |
> | `:main` | Alias of the same rolling `main` build as `:latest` |
> | `:sha-xxxxxxx` | Specific commit (produced by manual workflow dispatch) |
+5 -8
View File
@@ -12,13 +12,11 @@ working OmniVoice Studio install on a Debian / Ubuntu / Fedora / Arch host.
- **~10 GB free disk** for the app, its Python environment, and model weights.
- Optional: an **NVIDIA driver** for CUDA GPU acceleration — the app runs
CPU-only without one. For AMD GPUs see [AMD GPU (ROCm)](#amd-gpu-rocm).
- Optional: **yt-dlp** for downloading YouTube/video clips directly in the
Voice Gallery and Dub tabs — `sudo apt install yt-dlp` (Debian/Ubuntu),
`sudo dnf install yt-dlp` (Fedora), or `sudo pacman -S yt-dlp` (Arch).
Without it those downloads fail; everything else works fine.
That's it — Python, FFmpeg, and the model weights are bundled or bootstrapped
by the app itself on first launch. No toolchain needed.
That's it — Python, FFmpeg/FFprobe, yt-dlp, and the model weights are bundled
or bootstrapped by the app itself on first launch. No toolchain needed. (If no
FFmpeg resolves anywhere, the app downloads its own checksummed static build
in the background during setup; **Settings → Audio tools** shows exactly which
binaries are in use and lets you override them or update yt-dlp.)
### Building from source
@@ -29,7 +27,6 @@ Everything above, plus the toolchain:
- **Python 3.11+** — typically `sudo apt install python3.11` on Debian/Ubuntu,
`sudo dnf install python3.11` on Fedora, or already installed on Arch.
- **Bun**`curl -fsSL https://bun.sh/install | bash`.
- **FFmpeg**`sudo apt install ffmpeg` (Debian/Ubuntu), `sudo dnf install ffmpeg-free` (Fedora), or `sudo pacman -S ffmpeg` (Arch).
- **Rust / Cargo**`curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh` or via your package manager (e.g., `sudo apt install rustc cargo`).
If you use rustup, reopen the shell or source `"$HOME/.cargo/env"` before running `bun run desktop-prod`.
- **GTK/WebKit deps** for the Tauri shell:
+6 -1
View File
@@ -35,10 +35,15 @@ Everything above, plus the toolchain:
and the C toolchain; `curl` ships with macOS).
- **Python 3.11+**`brew install python@3.11` (or use `pyenv` / the system Python if you already have ≥3.11).
- **Bun**`curl -fsSL https://bun.sh/install | bash`.
- **FFmpeg** (used by the dubbing + capture pipelines) — `brew install ffmpeg`.
- **Rust / Cargo**`curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh` or `brew install rust`.
If you use rustup, reopen the terminal or source `"$HOME/.cargo/env"` before running `bun run desktop-prod`.
FFmpeg/FFprobe and yt-dlp are **not** prerequisites on any install path: the
app resolves them itself (a static build ships with the Python environment;
if nothing resolves, the app downloads its own checksummed build on first
run). Power users can inspect or override the binaries in
**Settings → Audio tools** — including pointing at a Homebrew copy.
Optional but recommended:
- **A Hugging Face account** for diarization and the larger TTS models. See
+26
View File
@@ -167,6 +167,32 @@ that rely on `/usr/bin/ffprobe`.
**Fix:** see [linux.md#deb-ffprobe-conflict](linux.md#deb-ffprobe-conflict).
## 7b. "Media engine unavailable" / FFmpeg questions
FFmpeg, FFprobe, and yt-dlp are **not** things you install for OmniVoice.
The app resolves them itself, in order: a path provided by the desktop shell →
the static build shipped with the Python environment → the app's own
downloaded build → whatever is on your PATH. When nothing resolves at all
(some source installs on a fresh machine), the Setup Wizard downloads a
pinned, checksum-verified static build in the background — you'll see a
one-line "Preparing media engine…" progress and, only if that download fails,
a card with **Retry** and **Use a system copy**.
If a running install ever reports "Media engine unavailable":
1. Open **Settings → Audio tools**. Each row shows the binary actually in use
(version, path, and origin — Bundled / System / Custom).
2. Press **Restore bundled** to re-fetch the app's own build (needs network
once), or **Use system copy** / **Choose file…** to point at an FFmpeg you
already have. Installing via a package manager (`brew install ffmpeg`,
`sudo apt install ffmpeg`, `winget install ffmpeg`) also works — press
**Use system copy** afterwards.
The same panel updates **yt-dlp** (video imports): site support changes
faster than app releases, so when video-URL imports start failing, press
**Update** there — the new version survives app updates, and **Restore tested
version** reverts to the build the app shipped with.
## 8. Docker LAN access — media preview 404
**Symptom:** OmniVoice loads on `http://<lan-ip>:3900` but the audio preview
+4 -1
View File
@@ -76,7 +76,10 @@ Settings → Sharing → **Remote backend**:
- **Test connection** hits `{url}/health` and shows the remote's version and
device.
- **Save & reload** stores both in this browser/app and restarts the UI
against the remote.
against the remote. The URL must be a full `http://` or `https://` URL
(`gpu-box:3900` alone is rejected), and saving a URL that hasn't passed
**Test connection** asks for confirmation first — a wrong base would leave
the app unable to reach any backend until you change it back here.
Leave the URL empty to go back to the local backend.
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "omnivoice-studio",
"version": "0.3.14",
"version": "0.3.17",
"private": true,
"license": "AGPL-3.0-only",
"type": "module",
+1 -1
View File
@@ -2941,7 +2941,7 @@ dependencies = [
[[package]]
name = "omnivoice-studio"
version = "0.3.14"
version = "0.3.17"
dependencies = [
"arboard",
"dirs-next",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "omnivoice-studio"
version = "0.3.14"
version = "0.3.17"
description = "OmniVoice Studio AI voice cloning & dubbing desktop app"
authors = ["Debpalash"]
license = "AGPL-3.0-only"
+46 -5
View File
@@ -83,7 +83,41 @@ pub fn same_app_version(running: &str) -> bool {
!running.is_empty() && base(running) == base(env!("CARGO_PKG_VERSION"))
}
/// Deep health probe for the attach-to-a-running-backend shortcut.
///
/// `/health` and `/system/info` keep answering from a backend whose install
/// was deleted out from under it (files unlinked on disk, code already in
/// memory) — that zombie passes the version check and then 500s every real
/// route, so the UI looks alive but nothing works. Probe a DB-touching
/// endpoint and require an actual `200` status line before attaching;
/// anything else (500, timeout, refused) means the responder is not a
/// backend worth keeping.
pub fn backend_deep_healthy(port: u16) -> bool {
let url = format!("http://127.0.0.1:{}/profiles", port);
match raw_http_get(&url, Duration::from_millis(1500)) {
Ok(resp) => parse_http_status(&resp) == Some(200),
Err(_) => false,
}
}
/// Status code from a raw HTTP response ("HTTP/1.1 200 OK" → 200).
fn parse_http_status(response: &str) -> Option<u16> {
let line = response.lines().next()?;
line.split_whitespace().nth(1)?.parse().ok()
}
fn ureq_get_with_timeout(url: &str, timeout: Duration) -> Result<String, String> {
let buf = raw_http_get(url, timeout)?;
if let Some(idx) = buf.find("\r\n\r\n") {
Ok(buf[idx + 4..].to_string())
} else {
Err("no body".into())
}
}
/// One raw loopback HTTP GET, returning the FULL response (status line +
/// headers + body). Kept dependency-free on purpose — see module docs.
fn raw_http_get(url: &str, timeout: Duration) -> Result<String, String> {
let url = url.strip_prefix("http://").ok_or("only http:// supported")?;
let (host_port, path) = match url.find('/') {
Some(i) => (&url[..i], &url[i..]),
@@ -112,11 +146,7 @@ fn ureq_get_with_timeout(url: &str, timeout: Duration) -> Result<String, String>
stream.write_all(req.as_bytes()).map_err(|e| e.to_string())?;
let mut buf = String::new();
stream.read_to_string(&mut buf).map_err(|e| e.to_string())?;
if let Some(idx) = buf.find("\r\n\r\n") {
Ok(buf[idx + 4..].to_string())
} else {
Err("no body".into())
}
Ok(buf)
}
/// Kill whatever process owns the port.
@@ -428,6 +458,17 @@ mod tests {
assert_eq!(parse_app_version("<html>not json</html>"), None);
}
#[test]
fn parse_http_status_reads_the_status_line_only() {
assert_eq!(super::parse_http_status("HTTP/1.1 200 OK\r\nX: 500\r\n\r\nbody"), Some(200));
assert_eq!(
super::parse_http_status("HTTP/1.1 500 Internal Server Error\r\n\r\nInternal Server Error"),
Some(500)
);
assert_eq!(super::parse_http_status("garbage"), None);
assert_eq!(super::parse_http_status(""), None);
}
#[test]
fn same_app_version_matches_current_build_and_rejects_stale() {
let ours = env!("CARGO_PKG_VERSION");
+63 -7
View File
@@ -148,12 +148,24 @@ pub fn retry_bootstrap(app: tauri::AppHandle, state: tauri::State<'_, BootstrapS
}
match crate::backend::running_backend_version(backend_port()) {
Some(v) if crate::backend::same_app_version(&v) => {
log::info!(
"Port {} already serving OmniVoice backend v{} — attaching",
if crate::backend::backend_deep_healthy(backend_port()) {
log::info!(
"Port {} already serving OmniVoice backend v{} — attaching",
backend_port(), v
);
set_stage(&stage_handle, BootstrapStage::Ready);
return;
}
// Same version but a DB-touching probe fails: a backend whose
// install was wiped/corrupted while it kept running. Attaching
// would look alive and 500 on everything — replace it.
log::warn!(
"Port {} serves OmniVoice v{} but failed the deep health probe — replacing it",
backend_port(), v
);
set_stage(&stage_handle, BootstrapStage::Ready);
return;
set_backend_kill_intended(true); // deliberate kill, not a crash (#941)
crate::backend::kill_orphan_on_port(backend_port());
std::thread::sleep(Duration::from_millis(500));
}
Some(v) => {
// A healthy-but-stale backend from a previous version (the
@@ -713,6 +725,27 @@ fn sync_failure_is_torch_download(tail: &str) -> bool {
/// distro-matched ROCm builds torch's own index doesn't carry).
const ROCM_TORCH_INDEX: &str = "https://download.pytorch.org/whl/rocm6.4";
/// Args for the routine update-drift sync (#307 path) — the one that runs on
/// every app update when `uv.lock` changed. `--inexact` is the fix for #1029:
/// plain `uv sync` UNINSTALLS every package not in the lockfile, which
/// silently deleted user-pip-installed optional engines (voxcpm, kittentts —
/// packages the app's own Settings → Engines hints tell users to install
/// into this venv) on every single update. `--inexact` still installs/
/// upgrades everything the lockfile demands — locked deps stay exactly
/// correct — it just stops removing extras the user added on purpose.
///
/// Deliberately NOT applied to the repair sync (`repair_sync_args`): repair
/// runs when the venv is *broken*, and a user-installed extra is a plausible
/// cause — healing must restore the known-good locked state, extras
/// included-out. An engine lost to a repair is re-installable; a venv that
/// repair can't actually repair is a support thread.
const DRIFT_SYNC_ARGS: [&str; 5] = ["sync", "--frozen", "--inexact", "--no-dev", "--verbose"];
/// Exact-sync args for the venv-repair path — see `DRIFT_SYNC_ARGS` for why
/// repair stays exact while the update-drift sync preserves user extras.
const REPAIR_SYNC_ARGS_LOCKED: [&str; 4] = ["sync", "--frozen", "--no-dev", "--verbose"];
const REPAIR_SYNC_ARGS_UNLOCKED: [&str; 3] = ["sync", "--no-dev", "--verbose"];
/// `uv pip install` args that replace the default CUDA torch build with the AMD
/// ROCm wheel (#124). Opt-in (gated on OMNIVOICE_TORCH_VARIANT=rocm by the
/// caller); the detection side (`get_best_device`) already routes ROCm through
@@ -1254,7 +1287,7 @@ manually, then relaunch.",
drift_cmd.env("UV_INDEX_URL", "https://mirrors.aliyun.com/pypi/simple/");
}
drift_cmd
.args(["sync", "--frozen", "--no-dev", "--verbose"])
.args(DRIFT_SYNC_ARGS)
.current_dir(&project_dir);
match run_streaming(app, "installing_deps", &mut drift_cmd) {
Ok(ref s) if s.success() => {
@@ -1329,9 +1362,9 @@ the existing venv; newly added dependencies may be missing (#307)",
apply_uv_http_env(&mut repair_cmd);
let has_lockfile = project_dir.join("uv.lock").is_file();
if has_lockfile {
repair_cmd.args(["sync", "--frozen", "--no-dev", "--verbose"]);
repair_cmd.args(REPAIR_SYNC_ARGS_LOCKED);
} else {
repair_cmd.args(["sync", "--no-dev", "--verbose"]);
repair_cmd.args(REPAIR_SYNC_ARGS_UNLOCKED);
}
repair_cmd.current_dir(&project_dir);
let repair_status = run_streaming(app, "installing_deps", &mut repair_cmd);
@@ -1712,6 +1745,29 @@ mod tests {
use super::*;
use std::collections::HashMap;
#[test]
fn update_drift_sync_preserves_user_installed_engines() {
// #1029: the routine update sync must carry --inexact so a
// user-pip-installed optional engine (voxcpm, kittentts — packages
// the app's own Settings → Engines hints tell users to install into
// this venv) survives every update instead of being silently
// uninstalled. --frozen must stay (lockfile is the resolution truth).
assert!(DRIFT_SYNC_ARGS.contains(&"--inexact"),
"update-drift sync lost --inexact — user-installed engines get wiped on every update (#1029)");
assert!(DRIFT_SYNC_ARGS.contains(&"--frozen"));
}
#[test]
fn repair_sync_stays_exact() {
// Deliberate asymmetry with the drift sync: repair runs when the venv
// is BROKEN and a user-installed extra is a plausible cause — healing
// must restore the known-good locked state, extras included-out.
assert!(!REPAIR_SYNC_ARGS_LOCKED.contains(&"--inexact"),
"repair sync must stay exact — it's the recovery path when an extra broke the venv");
assert!(!REPAIR_SYNC_ARGS_UNLOCKED.contains(&"--inexact"));
assert!(REPAIR_SYNC_ARGS_LOCKED.contains(&"--frozen"));
}
#[test]
fn scrub_python_env_removes_bundled_runtime_vars() {
// #144: every uv/venv/pip subprocess must drop the AppImage's bundled
+15 -4
View File
@@ -807,12 +807,23 @@ pub fn run() {
}
match backend::running_backend_version(backend_port()) {
Some(v) if backend::same_app_version(&v) => {
log::info!(
"Port {} already serving OmniVoice backend v{} — attaching",
if backend::backend_deep_healthy(backend_port()) {
log::info!(
"Port {} already serving OmniVoice backend v{} — attaching",
backend_port(), v
);
set_stage(&stage_handle, BootstrapStage::Ready);
return;
}
// Same version but a DB-touching probe fails: a backend whose
// install was wiped/corrupted while it kept running. Attaching
// would look alive and 500 on everything — replace it.
log::warn!(
"Port {} serves OmniVoice v{} but failed the deep health probe — replacing it",
backend_port(), v
);
set_stage(&stage_handle, BootstrapStage::Ready);
return;
backend::kill_orphan_on_port(backend_port());
std::thread::sleep(Duration::from_millis(500));
}
Some(v) => {
// Healthy-but-stale backend from a previous version —
+66 -1
View File
@@ -42,6 +42,7 @@ import WorkspaceVoices from './components/WorkspaceVoices';
import WorkspaceProjects from './components/WorkspaceProjects';
import ErrorBoundary from './components/ErrorBoundary';
import FloatingPill from './components/FloatingPill';
import GlobalAudioPlayer from './components/GlobalAudioPlayer';
import BackendCrashNotice from './components/BackendCrashNotice';
// RemoteAuthGate is mounted at the true outermost provider in main-app.jsx so
// it covers all app states (setup check / wizard / bootstrap), not just the
@@ -83,6 +84,12 @@ import {
renameProject as apiRenameProject,
} from './api/projects';
import { exportAction, exportReveal, exportRecord } from './api/exports';
import {
clearHistory as apiClearHistory,
setHistoryStarred as apiSetHistoryStarred,
audioUrlWithCacheBust,
} from './api/generate';
import { clearDubHistory as apiClearDubHistory } from './api/dub';
import { isTauri, doubleClickMaximize, fileToMediaUrl, playBlobAudio } from './utils/media';
import { browserDownload } from './utils/download';
@@ -589,7 +596,7 @@ function App() {
fd.append('num_step', '16');
const res = await apiFetch(`${API}/generate`, { method: 'POST', body: fd });
const blob = await res.blob();
await playBlobAudio(blob);
await playBlobAudio(blob, { label: i18n.t('player.generated_audio') });
toast.success(i18n.t('firstrun.first_sound_done'), { duration: 7000 });
} catch {
/* silent — see above */
@@ -1135,6 +1142,33 @@ function App() {
toast.success(i18n.t('app.toast_restored_state'));
};
// Generation takes: star/unstar a take so it survives the retention cap and
// never ages off the rail. Optimistic errors only the WS
// generation_history event refreshes the list on success.
const toggleStarHistory = async (item) => {
try {
await apiSetHistoryStarred(item.id, !item.starred);
loadHistory();
} catch (err) {
toast.error(err.message);
}
};
// Load a past take back as the active output: fetch its WAV and hand it to
// the same global mini-player a fresh generation plays through.
const playTakeAsOutput = async (item) => {
try {
const res = await apiFetch(audioUrlWithCacheBust(item.audio_path));
if (!res.ok) throw new Error(`HTTP ${res.status}`);
const blob = await res.blob();
await playBlobAudio(blob, {
label: item.text || i18n.t('player.generated_audio'),
});
} catch (err) {
toast.error(i18n.t('history.load_take_failed', { message: err.message || '' }));
}
};
const deleteHistory = async (id, type) => {
if (!(await askConfirm('Delete this history item?'))) return;
try {
@@ -1151,6 +1185,27 @@ function App() {
}
};
// Clear-all for the workspace history panels (#1032). The control lived in
// the old left Sidebar; the workspace UX overhaul (#374) moved history into
// the right-side WorkspaceHistory panels and the button was dropped in the
// move restore it, scoped per workspace (voice = synth rows, dub = dubs).
const clearWorkspaceHistory = async (type) => {
const count = type === 'dub' ? dubHistory.length : history.length;
if (!(await askConfirm(i18n.t('sidebar.clear_confirm', { count })))) return;
try {
if (type === 'dub') {
await apiClearDubHistory();
loadDubHistory();
} else {
await apiClearHistory();
loadHistory();
}
toast.success(i18n.t('sidebar.history_cleared'));
} catch (err) {
toast.error(err.message);
}
};
// Install-plan screen outranks everything both on a true first run and
// when explicitly requested via `--setup`. Without this, a live backend
// answering /setup/status would route straight to the model wizard and the
@@ -1498,6 +1553,7 @@ function App() {
dubHistory={dubHistory}
restoreDubHistory={restoreDubHistory}
deleteHistory={deleteHistory}
clearHistory={() => clearWorkspaceHistory('dub')}
/>
</div>
)}
@@ -1594,6 +1650,9 @@ function App() {
handleNativeExport={handleNativeExport}
restoreHistory={restoreHistory}
deleteHistory={deleteHistory}
clearHistory={() => clearWorkspaceHistory('synth')}
toggleStarHistory={toggleStarHistory}
playTakeAsOutput={playTakeAsOutput}
/>
</div>
</div>
@@ -1700,6 +1759,12 @@ function App() {
</Suspense>
)}
{/* GLOBAL AUDIO MINI-PLAYER (grid row 3, above the footer)
Subsumes the #1032 PlaybackStopPill: waveform + seek + time + stop
for every playBlobAudio playback that has no on-screen player. As a
real grid row it can never overlap row-2 content or the footer. */}
<GlobalAudioPlayer />
{/* ═══ BOTTOM LOGS PANEL (VSCode-style) ═══ */}
<Suspense fallback={null}>
<LogsFooter />
+8
View File
@@ -16,6 +16,14 @@ export async function clearHistory(): Promise<Response> {
return apiFetch('/history', { method: 'DELETE' });
}
export async function setHistoryStarred(id: string, starred: boolean): Promise<unknown> {
return apiJson(`/history/${id}/starred`, {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ starred }),
});
}
export function audioUrl(filename: string): string {
return `${API}/audio/${filename}`;
}
+32
View File
@@ -88,6 +88,38 @@ export async function modelStatus(): Promise<ModelStatus> {
return apiJson<ModelStatus>('/model/status');
}
// ── Loaded-model residency (MM2-04 endpoints) ────────────────────────────
/** One entry from GET /model/loaded a model currently resident in memory.
* `engine_id`/`is_active_engine` attribute TTS-family entries to an engine
* (a model can stay resident after the user switches engines). */
export interface LoadedModel {
id: string; // 'tts' | 'asr' | 'diarization' | 'sidecar:<engine>'
name: string;
checkpoint: string;
device: string;
vram_mb: number;
unloadable: boolean;
note?: string;
engine_id?: string;
is_active_engine?: boolean | null;
}
export interface LoadedModelsResponse {
models: LoadedModel[];
count: number;
}
export async function listLoadedModels(): Promise<LoadedModelsResponse> {
return apiJson<LoadedModelsResponse>('/model/loaded');
}
/** Unload one resident model by its /model/loaded `id`. The model reloads
* lazily on next use unloading only frees memory, it never loses data. */
export async function unloadLoadedModel(modelId: string): Promise<unknown> {
return apiPost(`/model/unload/${encodeURIComponent(modelId)}`);
}
// ── Audio cleaning ───────────────────────────────────────────────────────
export async function cleanAudio(formData: FormData): Promise<Response> {
+19
View File
@@ -29,6 +29,14 @@ interface EngineBackend {
display_name: string;
available: boolean;
reason: string | null;
// Available-but-has-advice: the backend's `is_available()` returned ok with
// an advisory tail ("ready — <advice>", e.g. VoxCPM2's upgrade hint). Null
// for plain-ready and unavailable rows; absent on legacy payloads.
hint?: string | null;
// Cloning capability (TTS family): true/false from the backend class, null
// when model-dependent (mlx-audio's curated models differ). Only badge on
// an explicit true.
supports_cloning?: boolean | null;
install_hint?: string | null;
// Copy-paste-ready `export VAR=...` line for a path-gated opt-in engine
// (IndexTTS / MOSS-v1.5 / dots.tts / Confucius4), else null/absent.
@@ -244,6 +252,17 @@ export interface DubTranslateResponse {
text_original?: string;
rate_ratio?: number;
rate_error?: string;
/** Pre-synthesis duration plan (backend services/duration_planner.py). */
plan?: {
status: 'fits' | 'tight' | 'impossible';
est_dur_s: number;
available_s: number;
est_overrun_s: number;
calibrated: boolean;
/** Opt-in LLM condensation suggestion (request condense=true only). */
suggested_text?: string;
suggested_est_dur_s?: number;
};
}[];
}
+36
View File
@@ -275,6 +275,42 @@ function DubSegmentRow({
📖 {seg.rate_ratio.toFixed(2)}×
</span>
)}
{/* Pre-synthesis duration plan (backend duration_planner): warn about
tight/impossible segments BEFORE GPU time is spent. Informational
only generation is never blocked. */}
{seg.plan && (seg.plan.status === 'tight' || seg.plan.status === 'impossible') && (
<span
className="text-[0.48rem] mt-[1px] inline-flex items-center gap-[1px]"
style={{ color: seg.plan.status === 'impossible' ? '#fb4934' : '#fabd2f' }}
title={t(
seg.plan.status === 'impossible'
? 'segment.plan_impossible_title'
: 'segment.plan_tight_title',
{
est: (seg.plan.est_dur_s || 0).toFixed(1),
avail: (seg.plan.available_s || 0).toFixed(1),
seconds: (seg.plan.est_overrun_s || 0).toFixed(1),
},
)}
>
<AlertCircle size={8} />{' '}
{seg.plan.status === 'impossible'
? t('segment.plan_impossible', {
seconds: (seg.plan.est_overrun_s || 0).toFixed(1),
})
: t('segment.plan_tight')}
</span>
)}
{seg.plan && seg.plan.suggested_text && seg.plan.suggested_text !== seg.text && (
<button
onClick={() => onEditField(seg.id, 'text', seg.plan.suggested_text)}
disabled={disabled}
title={t('segment.plan_apply_title', { text: seg.plan.suggested_text })}
className="bg-transparent border-none text-[#83a598] cursor-pointer p-0 mt-[1px] text-[0.48rem] text-left"
>
{t('segment.plan_apply')}
</button>
)}
</span>
<input
File diff suppressed because it is too large Load Diff
+62
View File
@@ -0,0 +1,62 @@
import React from 'react';
import { cn } from '@/lib/utils';
/**
* EngineMark the per-engine identity mark for the Models & Engines
* Settings surfaces.
*
* A small monogram chip whose hue is derived deterministically from the
* engine id (the same trick as `models/format.js`'s `orgColor` for HF
* orgs), so the same engine is instantly recognizable everywhere it
* appears on these pages: the Engine Compatibility Matrix rows and the
* "in memory" residency chips. Purely decorative (`aria-hidden`) the
* engine's name and id are always rendered as text alongside it.
*
* Theme-safe by construction: the hue is fixed per engine, but the fill
* is a low-opacity `color-mix` over transparent and the glyph color is
* mixed toward `--chrome-fg`, so it stays legible on light and dark
* themes without per-theme overrides.
*/
/** Deterministic hue (0359) from an engine id. */
export function engineHue(id) {
const s = String(id || '');
let h = 0;
for (let i = 0; i < s.length; i++) h = (h * 31 + s.charCodeAt(i)) & 0xffff;
return h % 360;
}
/** Two-character monogram from an engine id ("mlx-audio" "MA",
* "voxcpm2" "VO"). Falls back to "?" for an empty id. */
export function engineMonogram(id) {
const parts = String(id || '')
.split(/[^a-z0-9]+/i)
.filter(Boolean);
if (parts.length === 0) return '?';
const mono = parts.length >= 2 ? parts[0][0] + parts[1][0] : parts[0].slice(0, 2);
return mono.toUpperCase();
}
export default function EngineMark({ id, size = 20, className = '' }) {
const accent = `hsl(${engineHue(id)} 62% 52%)`;
return (
<span
aria-hidden="true"
data-testid={`engine-mark-${id}`}
className={cn(
'inline-flex shrink-0 select-none items-center justify-center rounded-[5px] font-semibold tracking-[0.02em]',
className,
)}
style={{
width: size,
height: size,
fontSize: Math.max(8, Math.round(size * 0.42)),
background: `color-mix(in srgb, ${accent} 15%, transparent)`,
border: `1px solid color-mix(in srgb, ${accent} 40%, transparent)`,
color: `color-mix(in srgb, ${accent} 55%, var(--chrome-fg, currentColor))`,
}}
>
{engineMonogram(id)}
</span>
);
}
+1 -1
View File
@@ -286,7 +286,7 @@ export default function ExportModal({
return createPortal(
<div
className="pointer-events-none fixed inset-x-0 bottom-[var(--logs-footer-height,28px)] z-[90] flex justify-center"
className="pointer-events-none fixed inset-x-0 bottom-[calc(var(--logs-footer-height,28px)+var(--audio-dock-height,0px))] z-[90] flex justify-center"
role="dialog"
aria-modal="false"
aria-label={t('exportModal.export_options')}
@@ -0,0 +1,223 @@
/**
* GlobalAudioPlayer persistent bottom mini-player for "invisible" audio.
*
* `playBlobAudio` (playback source 'output') plays the generate auto-play,
* profile & dub-segment previews, story lines, gallery voices and Projects
* renders through a bare Audio()/AudioContext with no on-screen player. Its
* only global affordance used to be the stop-only PlaybackStopPill (#1032)
* this bar subsumes it: waveform (peaks decoded once from the blob already in
* hand), click/drag/keyboard seek, play/pause, elapsed/total time, a source
* label and a stop button, on every page (mounted once in App.jsx).
*
* Exclusion semantics are the pill's, unchanged: ONLY source 'output'
* renders here. Sources with their own visible player UI (WaveformPlayer
* instances, 'design-preview', 'demo-output') stay in-place.
*
* Layout: a real grid row of .app-container (row 3, directly above the
* LogsFooter see index.css). Content in row 2 physically ends at the bar's
* top edge, so the fixed-overlay overlap class the pill had at 1440×900
* (covering the studio's Production Overrides row) is impossible by
* construction. While visible it publishes --audio-dock-height so the fixed
* overlays that anchor above the footer (FloatingPill, VoicePreview,
* ExportModal, compare drawer) ride above the bar too.
*/
import React, { useEffect, useRef, useState } from 'react';
import { Pause, Play, Square } from 'lucide-react';
import { useTranslation } from 'react-i18next';
import {
pauseActivePlayback,
resumeActivePlayback,
seekActivePlayback,
stopActivePlayback,
usePlaybackTrack,
} from '../utils/playback';
const DOCK_H = 44; // collapsed-chrome scale: header/footer bars are 28px, player needs touch room
const fmt = (s) => {
if (!isFinite(s) || s < 0) s = 0;
const m = Math.floor(s / 60);
const sec = Math.floor(s % 60);
return `${m}:${String(sec).padStart(2, '0')}`;
};
// Same visual language as WaveformPlayer's wavesurfer config (bar width 2,
// gap 1, wave/progress colors), just hand-drawn on a canvas the peaks are
// precomputed in utils/media.js, so no wavesurfer instance (and no second
// decode/fetch) is needed here.
const WAVE_COLOR = 'rgba(168,153,132,0.45)';
const PROGRESS_COLOR = 'rgba(211,134,155,0.75)';
const CURSOR_COLOR = '#d3869b';
function WaveCanvas({ peaks, progress }) {
const wrapRef = useRef(null);
const canvasRef = useRef(null);
const [width, setWidth] = useState(0);
useEffect(() => {
const el = wrapRef.current;
if (!el || typeof ResizeObserver === 'undefined') return undefined;
const ro = new ResizeObserver(() => setWidth(el.clientWidth));
ro.observe(el);
setWidth(el.clientWidth);
return () => ro.disconnect();
}, []);
useEffect(() => {
const canvas = canvasRef.current;
const ctx = canvas?.getContext?.('2d');
if (!ctx) return; // jsdom / very old engines seek + time still work
const w = width || canvas.clientWidth;
const h = canvas.clientHeight || 28;
if (!w || !h) return;
const dpr = window.devicePixelRatio || 1;
canvas.width = w * dpr;
canvas.height = h * dpr;
ctx.scale(dpr, dpr);
ctx.clearRect(0, 0, w, h);
const playedX = Math.max(0, Math.min(1, progress)) * w;
if (peaks && peaks.length) {
const barW = 2;
const gap = 1;
const count = Math.max(1, Math.floor(w / (barW + gap)));
for (let i = 0; i < count; i++) {
const x = i * (barW + gap);
const peak = peaks[Math.floor((i / count) * peaks.length)] || 0;
const barH = Math.max(2, peak * (h - 2));
ctx.fillStyle = x + barW <= playedX ? PROGRESS_COLOR : WAVE_COLOR;
ctx.fillRect(x, (h - barH) / 2, barW, barH);
}
} else {
// No peaks (decode unavailable e.g. the Tauri streamed fallback):
// a plain progress track, same colors.
ctx.fillStyle = WAVE_COLOR;
ctx.fillRect(0, h / 2 - 1.5, w, 3);
ctx.fillStyle = PROGRESS_COLOR;
ctx.fillRect(0, h / 2 - 1.5, playedX, 3);
}
// Playhead cursor.
ctx.fillStyle = CURSOR_COLOR;
ctx.fillRect(Math.min(playedX, w - 1), 0, 1.5, h);
}, [peaks, progress, width]);
return (
<div ref={wrapRef} className="w-full h-full">
<canvas ref={canvasRef} className="block w-full h-full" aria-hidden="true" />
</div>
);
}
function PlayerBar({ track }) {
const { t } = useTranslation();
const { label, paused, currentTime, duration, peaks, canSeek, canPause } = track;
const scrubbingRef = useRef(false);
const seekable = canSeek && duration > 0;
const seekToClientX = (target, clientX) => {
const rect = target.getBoundingClientRect();
if (!rect.width) return;
const frac = Math.max(0, Math.min(1, (clientX - rect.left) / rect.width));
seekActivePlayback(frac * duration);
};
const onPointerDown = (e) => {
if (!seekable) return;
scrubbingRef.current = true;
e.currentTarget.setPointerCapture?.(e.pointerId);
seekToClientX(e.currentTarget, e.clientX);
};
const onPointerMove = (e) => {
if (!seekable || !scrubbingRef.current) return;
seekToClientX(e.currentTarget, e.clientX);
};
const endScrub = () => {
scrubbingRef.current = false;
};
const onKeyDown = (e) => {
if (!seekable) return;
if (e.key === 'ArrowRight') seekActivePlayback(Math.min(duration, currentTime + 5));
else if (e.key === 'ArrowLeft') seekActivePlayback(Math.max(0, currentTime - 5));
else if (e.key === 'Home') seekActivePlayback(0);
else if (e.key === 'End') seekActivePlayback(duration);
else return;
e.preventDefault();
};
return (
<div
className="global-audio-dock flex items-center gap-[10px] px-[10px] [background:var(--chrome-bg)] [border-top:1px_solid_var(--chrome-border)] [color:var(--chrome-fg)] select-none"
style={{ height: DOCK_H }}
role="region"
aria-label={t('player.now_playing')}
data-testid="global-audio-player"
>
{canPause && (
<button
type="button"
className="wf-player__btn shrink-0 inline-flex items-center justify-center w-[28px] h-[28px] border-none rounded-full cursor-pointer text-[color:var(--color-fg-inverse)] bg-[var(--color-brand)] [transition:background_0.15s_ease,transform_0.1s_ease] hover:bg-[var(--color-brand-hover)] active:scale-[0.94]"
onClick={paused ? resumeActivePlayback : pauseActivePlayback}
aria-label={paused ? t('player.play') : t('player.pause')}
>
{paused ? <Play size={14} /> : <Pause size={14} />}
</button>
)}
<span
className="shrink-0 max-w-[220px] truncate text-[11.5px] [color:var(--chrome-fg-muted)]"
title={label || t('player.untitled')}
>
{label || t('player.untitled')}
</span>
<div
className={`flex-1 min-w-0 h-[28px] ${seekable ? 'cursor-pointer' : 'cursor-default'}`}
role="slider"
tabIndex={seekable ? 0 : -1}
aria-label={t('player.seek')}
aria-valuemin={0}
aria-valuemax={Math.round(duration)}
aria-valuenow={Math.round(currentTime)}
aria-valuetext={`${fmt(currentTime)} / ${fmt(duration)}`}
aria-disabled={!seekable}
onPointerDown={onPointerDown}
onPointerMove={onPointerMove}
onPointerUp={endScrub}
onPointerCancel={endScrub}
onKeyDown={onKeyDown}
>
<WaveCanvas peaks={peaks} progress={duration > 0 ? currentTime / duration : 0} />
</div>
<span className="shrink-0 [font-variant-numeric:tabular-nums] text-[11px] [color:var(--chrome-fg-muted)] whitespace-nowrap">
{fmt(currentTime)} / {fmt(duration)}
</span>
<button
type="button"
className="shrink-0 flex items-center justify-center w-[var(--chrome-icon-btn)] h-[var(--chrome-icon-btn)] rounded-[3px] bg-transparent border-0 cursor-pointer [color:var(--chrome-fg-muted)] hover:[color:var(--chrome-fg)] hover:[background:var(--chrome-hover-bg)] focus-visible:[outline:2px_solid_var(--chrome-accent)] focus-visible:[outline-offset:1px]"
onClick={stopActivePlayback}
title={t('player.stop')}
aria-label={t('player.stop')}
>
<Square size={12} />
</button>
</div>
);
}
export default function GlobalAudioPlayer() {
const track = usePlaybackTrack();
// Exact PlaybackStopPill routing: only bare 'output' playback docks here.
const visible = track?.source === 'output';
// Publish the dock height so fixed overlays anchored above the LogsFooter
// (--logs-footer-height consumers) stack above the bar instead of over it.
useEffect(() => {
document.documentElement.style.setProperty(
'--audio-dock-height',
visible ? `${DOCK_H}px` : '0px',
);
return () => {
document.documentElement.style.setProperty('--audio-dock-height', '0px');
};
}, [visible]);
if (!visible) return null;
return <PlayerBar track={track} />;
}
+3
View File
@@ -798,6 +798,9 @@ export default function LogsFooter() {
if (notif.action.type === 'navigate') {
useAppStore.getState().setMode?.(notif.action.target);
setCollapsed(true);
} else if (notif.action.type === 'settings-tab') {
useAppStore.getState().openSettingsTab?.(notif.action.target);
setCollapsed(true);
} else if (notif.action.type === 'link') {
import('../api/external').then((m) => m.openExternal(notif.action.target));
}
+181
View File
@@ -0,0 +1,181 @@
/**
* Media engine invisible unless it needs help.
*
* The media engine (ffmpeg/ffprobe) is an internal dependency, not a system
* requirement: when the backend's resolution chain finds nothing, preflight
* already kicked a background download of the app's own pinned static build.
* This renders NOTHING when the engine is ready (the ideal outcome), a quiet
* one-line progress while acquiring, and an actionable card only on failure
* (Retry / use a copy already on the machine). yt-dlp never appears here
* it's an importable module, not a user task.
*/
import { useCallback, useEffect, useState } from 'react';
import { useTranslation } from 'react-i18next';
import { Loader } from 'lucide-react';
import { apiJson, apiFetch } from '../api/client';
import { Button } from '../ui';
export default function MediaEngineCard() {
const { t } = useTranslation();
const [status, setStatus] = useState(null);
const [detectError, setDetectError] = useState(null);
const [customPath, setCustomPath] = useState('');
const [showPathInput, setShowPathInput] = useState(false);
const [busy, setBusy] = useState(false);
const refresh = useCallback(async () => {
try {
const st = await apiJson('/media-tools/status');
setStatus(st);
return st;
} catch {
return null;
}
}, []);
useEffect(() => {
refresh();
}, [refresh]);
const acquiring = status?.ops?.acquire?.state === 'running';
useEffect(() => {
if (!acquiring) return undefined;
const iv = setInterval(refresh, 1500);
return () => clearInterval(iv);
}, [acquiring, refresh]);
const post = async (path, body) => {
setBusy(true);
setDetectError(null);
try {
const res = await apiFetch(path, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: body ? JSON.stringify(body) : undefined,
});
if (!res.ok) {
let detail = `HTTP ${res.status}`;
try {
detail = (await res.json())?.detail || detail;
} catch {
/* non-JSON body */
}
throw new Error(detail);
}
return true;
} catch (e) {
setDetectError(e?.message || String(e));
return false;
} finally {
setBusy(false);
refresh();
}
};
const useSystemCopy = async () => {
// ffprobe rides along: the resolver derives the sibling ffprobe from a
// resolved ffmpeg, so pinning ffmpeg is enough in the common case.
await post('/media-tools/ffmpeg/use-system');
};
const chooseFile = async () => {
try {
if ('__TAURI_INTERNALS__' in window) {
const { open } = await import('@tauri-apps/plugin-dialog');
const picked = await open({ multiple: false, directory: false, title: 'FFmpeg' });
if (typeof picked === 'string') {
await post('/media-tools/ffmpeg/custom-path', { path: picked });
return;
}
}
} catch {
/* picker unavailable — fall through to the inline input */
}
setShowPathInput(true);
};
if (!status || status.ready) return null; // the ideal outcome: nothing.
const op = status.ops?.acquire || {};
if (op.state === 'running' || op.state === 'idle') {
// idle-and-not-ready = preflight is about to kick the download (or a
// recheck is in flight) show the quiet line, never flash the card.
return (
<div
className="mt-3 flex items-center gap-2 text-xs text-fg-muted"
data-testid="media-engine-progress"
>
<Loader className="animate-spin" size={12} aria-hidden="true" />
{t('setup.media_engine_preparing', { defaultValue: 'Preparing media engine…' })}
{op.state === 'running' && ` ${Math.round((op.progress || 0) * 100)}%`}
</div>
);
}
return (
<div
className="mt-3 flex flex-col gap-1.5 rounded-md border border-border px-3 py-2.5"
data-testid="media-engine-card"
>
<span className="text-sm font-semibold">
{t('setup.media_engine_failed_title', { defaultValue: 'Media engine download failed' })}
</span>
<span className="text-xs leading-snug text-fg-muted">
{t('setup.media_engine_failed_desc', {
defaultValue:
"The app couldn't fetch its bundled audio/video engine (FFmpeg). Retry, or point it at a copy already on this computer.",
})}
</span>
{(op.error || detectError) && (
<span className="text-xs text-danger" role="alert" data-testid="media-engine-error">
{detectError || op.error}
</span>
)}
<div className="mt-1 flex flex-wrap items-center gap-2">
<Button
variant="subtle"
size="sm"
loading={busy}
disabled={busy}
onClick={() => post('/media-tools/acquire')}
data-testid="media-engine-retry"
>
{t('setup.media_engine_retry', { defaultValue: 'Retry' })}
</Button>
<Button
variant="ghost"
size="sm"
disabled={busy}
onClick={useSystemCopy}
data-testid="media-engine-use-system"
>
{t('setup.media_engine_use_system', { defaultValue: 'Use a system copy' })}
</Button>
<Button variant="ghost" size="sm" disabled={busy} onClick={chooseFile}>
{t('setup.media_engine_choose_file', { defaultValue: 'Choose file…' })}
</Button>
{showPathInput && (
<>
<input
type="text"
value={customPath}
onChange={(e) => setCustomPath(e.target.value)}
placeholder="/usr/bin/ffmpeg"
className="min-w-[220px] flex-1 rounded border border-border bg-transparent px-2 py-1 font-mono text-xs text-fg"
aria-label={t('settings.ffmpeg_input_aria', { defaultValue: 'FFmpeg path' })}
data-testid="media-engine-path"
/>
<Button
variant="subtle"
size="sm"
disabled={busy || !customPath.trim()}
onClick={() => post('/media-tools/ffmpeg/custom-path', { path: customPath.trim() })}
>
{t('credentials.save', { defaultValue: 'Save' })}
</Button>
</>
)}
</div>
</div>
);
}
@@ -0,0 +1,100 @@
import React from 'react';
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { fireEvent, render, screen, waitFor } from '@testing-library/react';
vi.mock('../api/client', () => ({
apiJson: vi.fn(),
apiFetch: vi.fn(),
}));
import { apiJson, apiFetch } from '../api/client';
import MediaEngineCard from './MediaEngineCard';
const statusWith = (ready, acquire) => ({
ready,
tools: {},
ops: { acquire: acquire || { state: 'idle', progress: 0, error: null } },
});
describe('MediaEngineCard — invisible-by-default media engine', () => {
beforeEach(() => {
vi.clearAllMocks();
apiFetch.mockResolvedValue({ ok: true, json: async () => ({}) });
});
it('renders NOTHING when the media engine is resolved (the ideal outcome)', async () => {
apiJson.mockResolvedValue(statusWith(true));
const { container } = render(<MediaEngineCard />);
await waitFor(() => expect(apiJson).toHaveBeenCalledWith('/media-tools/status'));
expect(container).toBeEmptyDOMElement();
expect(screen.queryByTestId('media-engine-card')).not.toBeInTheDocument();
});
it('shows only a quiet progress line while the bundled build downloads', async () => {
apiJson.mockResolvedValue(statusWith(false, { state: 'running', progress: 0.42, error: null }));
render(<MediaEngineCard />);
const line = await screen.findByTestId('media-engine-progress');
expect(line).toHaveTextContent('Preparing media engine…');
expect(line).toHaveTextContent('42%');
// No requirements-style card, no mention of package managers.
expect(screen.queryByTestId('media-engine-card')).not.toBeInTheDocument();
expect(document.body.textContent).not.toMatch(/brew|apt|choco/i);
});
it('shows the actionable failure card only when acquisition failed', async () => {
apiJson.mockResolvedValue(
statusWith(false, { state: 'error', progress: 0, error: 'download checksum mismatch' }),
);
render(<MediaEngineCard />);
const card = await screen.findByTestId('media-engine-card');
expect(card).toHaveTextContent('Media engine download failed');
expect(screen.getByTestId('media-engine-error')).toHaveTextContent(
'download checksum mismatch',
);
expect(screen.getByTestId('media-engine-retry')).toBeInTheDocument();
expect(screen.getByTestId('media-engine-use-system')).toBeInTheDocument();
});
it('Retry re-posts the acquisition endpoint', async () => {
apiJson.mockResolvedValue(statusWith(false, { state: 'error', error: 'boom' }));
render(<MediaEngineCard />);
fireEvent.click(await screen.findByTestId('media-engine-retry'));
await waitFor(() =>
expect(apiFetch).toHaveBeenCalledWith('/media-tools/acquire', expect.anything()),
);
});
it('Use a system copy posts use-system and surfaces a not-found detail', async () => {
apiJson.mockResolvedValue(statusWith(false, { state: 'error', error: 'boom' }));
apiFetch.mockResolvedValue({
ok: false,
status: 404,
json: async () => ({
detail: 'No system ffmpeg found on PATH or in the usual install locations.',
}),
});
render(<MediaEngineCard />);
fireEvent.click(await screen.findByTestId('media-engine-use-system'));
await waitFor(() =>
expect(apiFetch).toHaveBeenCalledWith('/media-tools/ffmpeg/use-system', expect.anything()),
);
expect(await screen.findByTestId('media-engine-error')).toHaveTextContent(
'No system ffmpeg found',
);
});
it('Choose file… falls back to an inline path input outside Tauri and saves it', async () => {
apiJson.mockResolvedValue(statusWith(false, { state: 'error', error: 'boom' }));
render(<MediaEngineCard />);
fireEvent.click(await screen.findByText('Choose file…'));
const input = await screen.findByTestId('media-engine-path');
fireEvent.change(input, { target: { value: '/usr/local/bin/ffmpeg' } });
fireEvent.click(screen.getByText('Save'));
await waitFor(() =>
expect(apiFetch).toHaveBeenCalledWith(
'/media-tools/ffmpeg/custom-path',
expect.objectContaining({ body: JSON.stringify({ path: '/usr/local/bin/ffmpeg' }) }),
),
);
});
});
+21 -31
View File
@@ -54,6 +54,7 @@ import {
import { parseScript } from '../utils/parseScript';
import { importToText } from '../utils/importStory';
import { generateSpeech, audioUrl } from '../api/generate';
import { playBlobAudio } from '../utils/media';
import { encodeAudio } from '../api/stories';
import { longformRender } from '../api/audiobook';
import { exportStems } from '../utils/storyExport';
@@ -423,14 +424,6 @@ export default function StoriesEditor({ profiles = [] }) {
return res.blob();
}, []);
const fetchChunkAudio = useCallback(
async (text, profileId, speed = 1.0) => {
const blob = await fetchChunkBlob(text, profileId, speed);
return URL.createObjectURL(blob);
},
[fetchChunkBlob],
);
const previewTrack = useCallback(
async (track) => {
const raw = (track.text || '').trim();
@@ -443,14 +436,18 @@ export default function StoriesEditor({ profiles = [] }) {
if (!hasStoryMarkers(raw)) {
try {
const url = await fetchChunkAudio(raw, pid, spd);
const blob = await fetchChunkBlob(raw, pid, spd);
const url = URL.createObjectURL(blob);
setTracks((prev) =>
prev.map((tk) =>
tk.id === track.id ? { ...tk, audioUrl: url, generating: false } : tk,
),
);
const audio = new Audio(url);
audio.play().catch(() => {});
// Shared playback path (labelled with the line text): registers with
// the single-playback manager + global mini-player, and unlike the
// old bare `new Audio(blobUrl)` actually plays under Tauri's
// WebKit, where blob: URLs are dead in media elements.
playBlobAudio(blob, { label: raw }).catch(() => {});
} catch (err) {
console.warn('Stories preview failed:', err);
setTracks((prev) =>
@@ -462,17 +459,15 @@ export default function StoriesEditor({ profiles = [] }) {
const parsed = parseStoryText(raw, pid);
try {
const audioUrls = await Promise.all(
const chunkBlobs = await Promise.all(
parsed.map((seg) =>
seg.type === 'chunk'
? fetchChunkAudio(seg.text, seg.profileId, spd)
? fetchChunkBlob(seg.text, seg.profileId, spd)
: Promise.resolve(null),
),
);
let cursor = 0;
const finish = () => {
for (let i = cursor; i < audioUrls.length; i++)
if (audioUrls[i]) URL.revokeObjectURL(audioUrls[i]);
setTracks((prev) =>
prev.map((tk) =>
tk.id === track.id ? { ...tk, generating: false, audioUrl: null } : tk,
@@ -482,26 +477,21 @@ export default function StoriesEditor({ profiles = [] }) {
const step = () => {
while (cursor < parsed.length) {
const seg = parsed[cursor];
const url = audioUrls[cursor];
const blob = chunkBlobs[cursor];
cursor++;
if (seg.type === 'pause') {
setTimeout(step, seg.seconds * 1000);
return;
}
if (seg.type === 'chunk' && url) {
const audio = new Audio(url);
audio.onended = () => {
URL.revokeObjectURL(url);
step();
};
audio.onerror = () => {
URL.revokeObjectURL(url);
step();
};
audio.play().catch(() => {
URL.revokeObjectURL(url);
step();
});
if (seg.type === 'chunk' && blob) {
// Chained through the shared playback path: each chunk claims
// the global manager (mini-player shows the line), a natural
// end (or a broken chunk) advances the chain, and stopping from
// the player/another claim cancels the rest of the chain.
playBlobAudio(blob, {
label: raw,
onDone: (reason) => (reason === 'stopped' ? finish() : step()),
}).catch(() => step());
return;
}
}
@@ -515,7 +505,7 @@ export default function StoriesEditor({ profiles = [] }) {
);
}
},
[fetchChunkAudio, cast, globalSpeed, setTracks],
[fetchChunkBlob, cast, globalSpeed, setTracks],
);
// Deliver a stitched WAV in the chosen format. MP3 routes through the backend
+1 -1
View File
@@ -97,7 +97,7 @@ export default function VoicePreview({
if (!open) return null;
return (
<div className="fixed bottom-[calc(var(--logs-footer-height,28px)+16px)] right-[16px] z-[900] w-[320px] bg-[var(--chrome-bg)] border border-solid border-transparent rounded-[12px] [box-shadow:0_8px_32px_rgba(0,0,0,0.4)] flex flex-col overflow-hidden animate-[voice-preview-in_0.2s_ease-out]">
<div className="fixed bottom-[calc(var(--logs-footer-height,28px)+var(--audio-dock-height,0px)+16px)] right-[16px] z-[900] w-[320px] bg-[var(--chrome-bg)] border border-solid border-transparent rounded-[12px] [box-shadow:0_8px_32px_rgba(0,0,0,0.4)] flex flex-col overflow-hidden animate-[voice-preview-in_0.2s_ease-out]">
<div className="flex items-center justify-between py-[10px] px-[14px] border-b border-solid border-b-transparent">
<span className="flex items-center gap-[6px] [font-family:var(--font-mono)] text-[0.72rem] font-semibold uppercase [letter-spacing:0.04em] text-[color:var(--chrome-fg)]">
<Volume2 size={13} /> {t('voicePreview.title')}
+69 -7
View File
@@ -22,6 +22,8 @@ import {
Lock,
Download as DownloadIcon,
FolderOpen,
Play,
Star,
Trash2,
} from 'lucide-react';
import WaveformPlayer from './WaveformPlayer';
@@ -31,6 +33,7 @@ const FILTERS = [
{ id: 'all', label: 'All' },
{ id: 'clone', label: 'Clone' },
{ id: 'design', label: 'Design' },
{ id: 'starred', label: 'Starred' },
];
/**
@@ -78,15 +81,35 @@ export default function WorkspaceHistory({
handleNativeExport,
restoreHistory,
deleteHistory,
clearHistory, // clear-all for this workspace's history (#1032)
toggleStarHistory, // generation takes: keep this take past the retention cap
playTakeAsOutput, // generation takes: replay a take as the active output
}) {
const { t } = useTranslation();
const [filter, setFilter] = useState('all');
const [expanded, setExpanded] = useState(null); // row id with un-clamped title
// Clear-all affordance (#1032): the old left Sidebar had one; the workspace
// UX overhaul (#374) moved history here and dropped it. Confirm + endpoint
// live in App.jsx (clearWorkspaceHistory) this only renders the button.
const clearAllButton = (count) =>
clearHistory && count > 0 ? (
<button
type="button"
className="history-action-btn danger flex-[0_0_auto]"
onClick={clearHistory}
title={t('sidebar.clear_history')}
>
<Trash2 size={10} /> {t('sidebar.clear_history')}
</button>
) : null;
// Voice workspace = clone + design generations (dub lives in its own workspace).
const items = useMemo(() => {
const synth = history.filter((h) => h.mode === 'clone' || h.mode === 'design');
return filter === 'all' ? synth : synth.filter((h) => h.mode === filter);
if (filter === 'all') return synth;
if (filter === 'starred') return synth.filter((h) => !!h.starred);
return synth.filter((h) => h.mode === filter);
}, [history, filter]);
// Dub variant: a flat list of dub jobs, no clone/design filter.
@@ -94,9 +117,12 @@ export default function WorkspaceHistory({
return (
<aside className="flex-[1_1_0] flex flex-col min-h-0 overflow-hidden">
<div className="flex-[0_0_auto] flex flex-col gap-[8px] py-[10px] px-[12px]">
<span className="inline-flex items-center gap-[6px] [font-family:var(--chrome-font-mono,var(--font-mono))] text-[0.72rem] font-semibold [letter-spacing:0.04em] uppercase text-[color:var(--chrome-fg-muted)]">
<History size={13} /> {t('history.dub_title', { defaultValue: 'Dub history' })}
</span>
<div className="flex items-center justify-between gap-[6px]">
<span className="inline-flex items-center gap-[6px] [font-family:var(--chrome-font-mono,var(--font-mono))] text-[0.72rem] font-semibold [letter-spacing:0.04em] uppercase text-[color:var(--chrome-fg-muted)]">
<History size={13} /> {t('history.dub_title', { defaultValue: 'Dub history' })}
</span>
{clearAllButton(dubHistory.length)}
</div>
</div>
<div className="flex-[1_1_auto] min-h-0 overflow-y-auto flex flex-col gap-[8px] p-[8px]">
{dubHistory.length === 0 ? (
@@ -156,9 +182,12 @@ export default function WorkspaceHistory({
return (
<aside className="flex-[1_1_0] flex flex-col min-h-0 overflow-hidden">
<div className="flex-[0_0_auto] flex flex-col gap-[8px] py-[10px] px-[12px]">
<span className="inline-flex items-center gap-[6px] [font-family:var(--chrome-font-mono,var(--font-mono))] text-[0.72rem] font-semibold [letter-spacing:0.04em] uppercase text-[color:var(--chrome-fg-muted)]">
<History size={13} /> {t('history.title', { defaultValue: 'History' })}
</span>
<div className="flex items-center justify-between gap-[6px]">
<span className="inline-flex items-center gap-[6px] [font-family:var(--chrome-font-mono,var(--font-mono))] text-[0.72rem] font-semibold [letter-spacing:0.04em] uppercase text-[color:var(--chrome-fg-muted)]">
<History size={13} /> {t('history.title', { defaultValue: 'History' })}
</span>
{clearAllButton(history.length)}
</div>
<div className="flex flex-wrap gap-[4px]">
{FILTERS.map((f) => (
<button
@@ -223,6 +252,39 @@ export default function WorkspaceHistory({
) : null}
{item.audio_path ? (
<div className="history-actions">
{toggleStarHistory ? (
<button
className={`history-action-btn history-action-icon ${item.starred ? 'accent' : ''}`}
onClick={(e) => {
e.stopPropagation();
toggleStarHistory(item);
}}
aria-pressed={!!item.starred}
data-testid={`take-star-${item.id}`}
title={
item.starred
? t('history.unstar_take', { defaultValue: 'Unstar — allow cleanup' })
: t('history.star_take', { defaultValue: 'Star — keep this take' })
}
>
<Star size={10} fill={item.starred ? 'currentColor' : 'none'} />
</button>
) : null}
{playTakeAsOutput ? (
<button
className="history-action-btn accent history-action-icon"
onClick={(e) => {
e.stopPropagation();
playTakeAsOutput(item);
}}
data-testid={`take-play-${item.id}`}
title={t('history.play_take', {
defaultValue: 'Load as active output',
})}
>
<Play size={10} />
</button>
) : null}
<button
className="history-action-btn accent"
onClick={(e) => {
@@ -0,0 +1,92 @@
// Generation takes: the Studio history rail's star / load-as-output actions.
import { describe, it, expect, vi, beforeAll } from 'vitest';
import { fireEvent, render, screen } from '@testing-library/react';
import React from 'react';
import WorkspaceHistory from './WorkspaceHistory';
beforeAll(() => {
// LazyWaveform defers the real <WaveformPlayer> behind an IntersectionObserver;
// a no-op stub keeps rows rendered without ever mounting the audio fetch.
global.IntersectionObserver = class {
observe() {}
disconnect() {}
unobserve() {}
};
});
const takes = [
{
id: 'aa1',
mode: 'clone',
text: 'first take',
audio_path: 'aa1.wav',
starred: 0,
created_at: 2,
},
{
id: 'bb2',
mode: 'design',
text: 'second take',
audio_path: 'bb2.wav',
starred: 1,
created_at: 1,
},
];
const noop = () => {};
function renderRail(overrides = {}) {
return render(
<WorkspaceHistory
history={takes}
handleSaveHistoryAsProfile={noop}
handleLockProfile={noop}
handleNativeExport={noop}
restoreHistory={noop}
deleteHistory={noop}
toggleStarHistory={noop}
playTakeAsOutput={noop}
{...overrides}
/>,
);
}
describe('WorkspaceHistory takes actions', () => {
it('star button reflects the starred state and calls the handler', () => {
const toggleStarHistory = vi.fn();
renderRail({ toggleStarHistory });
const unstarred = screen.getByTestId('take-star-aa1');
const starred = screen.getByTestId('take-star-bb2');
expect(unstarred).toHaveAttribute('aria-pressed', 'false');
expect(starred).toHaveAttribute('aria-pressed', 'true');
fireEvent.click(unstarred);
expect(toggleStarHistory).toHaveBeenCalledTimes(1);
expect(toggleStarHistory.mock.calls[0][0].id).toBe('aa1');
});
it('load-as-output button hands the take to the player handler', () => {
const playTakeAsOutput = vi.fn();
renderRail({ playTakeAsOutput });
fireEvent.click(screen.getByTestId('take-play-bb2'));
expect(playTakeAsOutput).toHaveBeenCalledTimes(1);
expect(playTakeAsOutput.mock.calls[0][0].id).toBe('bb2');
});
it('starred filter narrows the rail to starred takes only', () => {
renderRail();
fireEvent.click(screen.getByRole('button', { name: 'Starred' }));
expect(screen.queryByTestId('take-star-aa1')).toBeNull();
expect(screen.getByTestId('take-star-bb2')).toBeInTheDocument();
});
it('omits the takes actions when no handlers are passed (dub rail safety)', () => {
renderRail({ toggleStarHistory: undefined, playTakeAsOutput: undefined });
expect(screen.queryByTestId('take-star-aa1')).toBeNull();
expect(screen.queryByTestId('take-play-aa1')).toBeNull();
});
});
@@ -34,8 +34,13 @@ export default function ScriptPanel({
}) {
return (
<div className="flex flex-col gap-[6px] flex-none min-h-0 relative z-[2]">
{/* overflow-visible: the Insert popover opens above the textarea and
must escape the panel's box instead of being clipped (#481). */}
{/* overflow-visible: the Insert popover opens BELOW the textarea and
must escape the panel's box instead of being clipped (#481). It
used to open upward but the script input sits at the very top of
the clone modal, so the tag list (max-h 280px) climbed straight out
of the viewport with no way to see or scroll it (owner-reported,
screenshot showed the CMU chips clipped). Below always has room
here: the panel is the topmost element in every mount. */}
<div className={`${STUDIO_PANEL} relative z-[10] overflow-visible`}>
<div className="label-row">
<Command className="label-icon" size={14} />{' '}
@@ -100,7 +105,7 @@ export default function ScriptPanel({
)}
{insertOpen && (
<div
className="absolute right-[8px] bottom-[60px] z-20 flex flex-wrap gap-1 max-w-[min(360px,calc(100vw-16px))] max-h-[min(280px,calc(100vh-120px))] overflow-y-auto overscroll-contain p-2 bg-[var(--chrome-bg)] border border-transparent rounded-[10px] shadow-[0_8px_24px_rgba(0,0,0,0.45)]"
className="absolute right-[8px] top-[calc(100%+6px)] z-20 flex flex-wrap gap-1 max-w-[min(360px,calc(100vw-16px))] max-h-[min(280px,calc(100vh-120px))] overflow-y-auto overscroll-contain p-2 bg-[var(--chrome-bg)] border border-transparent rounded-[10px] shadow-[0_8px_24px_rgba(0,0,0,0.45)]"
role="menu"
>
{TAGS.map((tag) => (
@@ -109,6 +109,16 @@ export default function DubLeftColumn({
// configured, we route the user straight to the LLM Providers setup instead
// of dead-ending on a toast (#838).
const openSettingsTab = useAppStore((s) => s.openSettingsTab);
// Two-stage LLM translation quality only meaningful (and only rendered)
// when the LLM engine is the active translator. Persisted prefs.
const autoGlossary = useAppStore((s) => s.autoGlossary);
const setAutoGlossary = useAppStore((s) => s.setAutoGlossary);
const reflectPass = useAppStore((s) => s.reflectPass);
const setReflectPass = useAppStore((s) => s.setReflectPass);
// Opt-in LLM condensation suggestions for segments the duration planner
// classifies as impossible to fit (default OFF needs an LLM).
const condenseSuggest = useAppStore((s) => s.condenseSuggest);
const setCondenseSuggest = useAppStore((s) => s.setCondenseSuggest);
// Frozen-build (packaged/signed, read-only site-packages) escape-hatch
// popover: pip install is impossible, so we surface the copyable command +
// a one-click switch to the always-bundled Argos engine + a docs deeplink.
@@ -646,7 +656,54 @@ export default function DubLeftColumn({
{ value: 'cinematic', label: t('dub.cinematic_quality') },
]}
/>
{/* Opt-in (default OFF): when the duration planner marks a
translated line "impossible" for its slot, ask the LLM for a
shorter rewrite the user can apply per segment. */}
<label
className="flex items-center gap-[4px] mt-[3px] text-[0.55rem] text-fg-muted cursor-pointer select-none"
title={t('dub.condense_title')}
>
<input
type="checkbox"
checked={condenseSuggest}
onChange={(e) => setCondenseSuggest(e.target.checked)}
className="cursor-pointer"
/>
{t('dub.condense_label')}
</label>
</div>
{/* LLM engine only: auto-glossary + reflect pass. Both default ON;
the reflect tooltip is explicit that it multiplies LLM calls. */}
{translateProvider === 'openai' && (
<div
className={`${FIELD} flex-[0_0_auto] ${FIELD_RESP} justify-end gap-[2px] pb-[2px]`}
>
<label
className="flex items-center gap-[4px] text-[0.6rem] text-[var(--chrome-fg-muted)] cursor-pointer whitespace-nowrap"
title={t('dub.auto_glossary_title')}
>
<input
type="checkbox"
className="accent-[var(--color-brand)] cursor-pointer"
checked={autoGlossary}
onChange={(e) => setAutoGlossary(e.target.checked)}
/>
<span>{t('dub.auto_glossary_label')}</span>
</label>
<label
className="flex items-center gap-[4px] text-[0.6rem] text-[var(--chrome-fg-muted)] cursor-pointer whitespace-nowrap"
title={t('dub.reflect_title')}
>
<input
type="checkbox"
className="accent-[var(--color-brand)] cursor-pointer"
checked={reflectPass}
onChange={(e) => setReflectPass(e.target.checked)}
/>
<span>{t('dub.reflect_label')}</span>
</label>
</div>
)}
<div className={`${FIELD} flex-[1_1_90px] min-w-[64px] ${FIELD_RESP}`}>
<div className={FIELD_LABEL}>
<UserSquare2 className="label-icon" size={9} /> {t('dub.style')}{' '}
@@ -75,7 +75,7 @@ export default function CommunityZone({
onToggleFavorite={toggleFavorite}
onPreview={(item) =>
item.audio?.url
? onPlayAudio(item.audio.url, item.id)
? onPlayAudio(item.audio.url, item.id, item.name)
: flash(
t('gallery.no_preview', {
defaultValue: 'No preview — add it with "Use voice" to hear it.',
+55 -10
View File
@@ -12,12 +12,50 @@ import {
import { useTranslation } from 'react-i18next';
import { openExternal } from '../../api/external';
import { resolveAboutVersion } from '../../utils/appVersion';
import { REPO_URL } from '../../utils/bugReport';
import { Button, Badge } from '../../ui';
import { SettingsSection } from './primitives';
import { CATEGORY_BY_ID } from './settingsCategories';
import { useAppStore } from '../../store';
import { isTauri } from './native';
import Row from './Row';
/**
* Where a failing self-check can be fixed inside the app diagnose check id
* (backend/core/diagnose.py) Settings category id. Checks without an in-app
* fix (python, backend, ) render their hint as plain text only.
*/
const CHECK_FIX_CATEGORY = {
ffmpeg: 'network',
hf_token: 'credentials',
disk: 'storage',
data_dir: 'storage',
engines: 'engines',
gpu_routing: 'engines',
device: 'performance',
ram: 'performance',
deep_synth: 'logs',
};
/** Small "Open <category>" deep-link into the Settings hub. */
function OpenCategoryButton({ categoryId }) {
const { t } = useTranslation();
const cat = CATEGORY_BY_ID[categoryId];
if (!cat) return null;
return (
<Button
size="sm"
variant="subtle"
onClick={() => useAppStore.getState().openSettingsTab(categoryId)}
>
{t('about.open_fix_category', {
defaultValue: 'Open {{category}}',
category: t(cat.labelKey, { defaultValue: cat.defaultLabel }),
})}
</Button>
);
}
/**
* Settings About.
*
@@ -52,7 +90,16 @@ export default function AboutTab({
/>
<Row
label={t('about.hf_token')}
value={info?.has_hf_token ? t('about.yes') : t('about.no')}
value={
info?.has_hf_token ? (
t('about.yes')
) : (
<span className="inline-flex flex-wrap items-center gap-[var(--space-3)]">
{t('about.no')}
<OpenCategoryButton categoryId="credentials" />
</span>
)
}
/>
<div className="settings-link-row mt-[var(--space-5)] flex flex-wrap gap-[var(--space-4)]">
@@ -92,18 +139,10 @@ export default function AboutTab({
variant="subtle"
size="md"
leading={<ExternalLink size={12} />}
onClick={() => openExternal('https://github.com/k2-fsa/OmniVoice')}
onClick={() => openExternal(REPO_URL)}
>
{t('about.github')}
</Button>
<Button
variant="subtle"
size="md"
leading={<ExternalLink size={12} />}
onClick={() => openExternal('https://huggingface.co/k2-fsa/OmniVoice')}
>
{t('about.model_card')}
</Button>
<Button
variant="subtle"
size="md"
@@ -136,6 +175,12 @@ export default function AboutTab({
{c.hint}
</span>
)}
{c.status !== 'ok' && CHECK_FIX_CATEGORY[c.id] && (
<>
{' '}
<OpenCategoryButton categoryId={CHECK_FIX_CATEGORY[c.id]} />
</>
)}
</span>
}
/>
@@ -0,0 +1,96 @@
import { describe, it, expect, beforeEach, vi } from 'vitest';
import { fireEvent, render, screen } from '@testing-library/react';
import React from 'react';
import AboutTab from './AboutTab';
import { REPO_URL } from '../../utils/bugReport';
import { openExternal } from '../../api/external';
import { useAppStore } from '../../store';
vi.mock('../../api/external', () => ({ openExternal: vi.fn() }));
const noop = () => {};
const baseProps = {
appVersion: '0.0.0-test',
tauriVersion: null,
info: { has_hf_token: true },
checkForUpdates: noop,
updateState: 'idle',
selfCheck: null,
selfCheckRunning: false,
runSelfCheck: noop,
bundleBuilding: false,
saveDiagnosticBundle: noop,
copyDiagnostics: noop,
};
describe('AboutTab — external links', () => {
beforeEach(() => {
vi.clearAllMocks();
});
it('the GitHub button opens the canonical repo (derived from the shared REPO_URL constant)', () => {
render(<AboutTab {...baseProps} />);
fireEvent.click(screen.getByRole('button', { name: 'OmniVoice on GitHub' }));
expect(openExternal).toHaveBeenCalledWith(REPO_URL);
// Belt-and-braces: the constant itself must point at this project, not a
// lookalike (the original bug linked github.com/k2-fsa/OmniVoice).
expect(REPO_URL).toBe('https://github.com/debpalash/OmniVoice-Studio');
});
it('has no "Model card" link — the app is multi-engine with no single model card', () => {
render(<AboutTab {...baseProps} />);
expect(screen.queryByRole('button', { name: /model card/i })).toBeNull();
});
});
describe('AboutTab — fixable problems deep-link into Settings', () => {
beforeEach(() => {
vi.clearAllMocks();
useAppStore.getState().setMode('launchpad');
useAppStore.getState().setPendingSettingsTab(null);
});
it('HF token "no" offers an Open Credentials action instead of dead-ending', () => {
render(<AboutTab {...baseProps} info={{ has_hf_token: false }} />);
fireEvent.click(screen.getByRole('button', { name: 'Open Credentials' }));
expect(useAppStore.getState().mode).toBe('settings');
expect(useAppStore.getState().pendingSettingsTab).toBe('credentials');
});
it('HF token "yes" renders no Credentials action', () => {
render(<AboutTab {...baseProps} info={{ has_hf_token: true }} />);
expect(screen.queryByRole('button', { name: 'Open Credentials' })).toBeNull();
});
it('a failing self-check renders an "Open <category>" button for its fix destination', () => {
const selfCheck = {
checks: [
{
id: 'ffmpeg',
label: 'ffmpeg',
status: 'fail',
detail: 'not found on PATH or FFMPEG_PATH',
hint: 'Dubbing and audio conversion need ffmpeg.',
},
{ id: 'python', label: 'Python runtime', status: 'ok', detail: '3.12', hint: null },
],
summary: { ok: false, failures: 1 },
};
render(<AboutTab {...baseProps} selfCheck={selfCheck} />);
fireEvent.click(screen.getByRole('button', { name: 'Open Network' }));
expect(useAppStore.getState().mode).toBe('settings');
expect(useAppStore.getState().pendingSettingsTab).toBe('network');
});
it('passing checks render no deep-link button', () => {
const selfCheck = {
checks: [
{ id: 'ffmpeg', label: 'ffmpeg', status: 'ok', detail: '/usr/bin/ffmpeg', hint: null },
],
summary: { ok: true, failures: 0 },
};
render(<AboutTab {...baseProps} selfCheck={selfCheck} />);
expect(screen.queryByRole('button', { name: /^Open / })).toBeNull();
});
});
+9 -14
View File
@@ -7,39 +7,34 @@
* while OmniVoice plays audio doesn't transcribe the playback. Off by default
* dictation uses the standard MediaRecorder path and behaves identically on
* every platform. The pref is the zustand `aecEnabled` flag (persisted); no
* backend round-trip needed.
* backend round-trip needed. All strings go through i18n (`dictation.aec_*`).
*/
import React from 'react';
import { Volume2 } from 'lucide-react';
import { useTranslation } from 'react-i18next';
import { useAppStore } from '../../store';
import { SettingsSection, SettingRow, SettingsToggle } from './primitives';
export default function AecPanel() {
const { t } = useTranslation();
const aecEnabled = useAppStore((s) => s.aecEnabled);
const setAecEnabled = useAppStore((s) => s.setAecEnabled);
return (
<SettingsSection
icon={Volume2}
title="Dictate while audio plays"
description="Cancel OmniVoice's own playback out of the microphone."
title={t('dictation.aec_title')}
description={t('dictation.aec_description')}
>
<SettingRow
title="Enable echo cancellation for dictation"
subtitle="experimental"
hint={
<>
Cancels OmniVoice's own playback out of the microphone so you can dictate while a
preview, dub, or video is playing without the transcript picking up what the app is
saying. Adds a small amount of audio processing; leave it off if you never dictate over
playback.
</>
}
title={t('dictation.aec_row_title')}
subtitle={t('dictation.aec_experimental')}
hint={t('dictation.aec_hint')}
control={
<SettingsToggle
checked={aecEnabled}
onChange={setAecEnabled}
aria-label="Enable echo cancellation for dictation"
aria-label={t('dictation.aec_row_title')}
/>
}
/>
+140 -122
View File
@@ -25,15 +25,6 @@ import { CheckCircle2, KeyRound, RefreshCw, Save, Trash2, XCircle } from 'lucide
import { apiJson, apiPost, apiFetch, API } from '../../api/client';
import { SettingsSection, InfoHint } from './primitives';
const EMPTY_STATE = {
sources: [
{ source: 'app', set: false, masked: null, whoami_user: null, whoami_ok: false },
{ source: 'env', set: false, masked: null, whoami_user: null, whoami_ok: false },
{ source: 'hf-cli', set: false, masked: null, whoami_user: null, whoami_ok: false },
],
active: null,
};
export default function ApiKeysPanel() {
const { t } = useTranslation();
const SOURCE_LABELS = {
@@ -52,7 +43,9 @@ export default function ApiKeysPanel() {
defaultValue: 'Written by `huggingface-cli login`. Read-only from the UI.',
}),
};
const [state, setState] = useState(EMPTY_STATE);
// null until the first GET lands the panel renders a "checking" placeholder
// instead of flashing a false amber "not set" verdict for every source.
const [state, setState] = useState(null);
const [loading, setLoading] = useState(false);
const [tokenInput, setTokenInput] = useState('');
const [saving, setSaving] = useState(false);
@@ -60,27 +53,36 @@ export default function ApiKeysPanel() {
const [alsoClearCli, setAlsoClearCli] = useState(false);
const [error, setError] = useState(null);
const refresh = useCallback(async () => {
setLoading(true);
setError(null);
try {
const data = await apiJson('/api/settings/hf-token/state');
setState(data);
} catch (e) {
setError(
e?.message ||
t('settings.hf_token_load_error', { defaultValue: 'Failed to load token state' }),
);
} finally {
setLoading(false);
}
}, [t]);
// `fresh` busts the backend's 300s whoami cache used by "Test now" so it
// really re-runs whoami instead of echoing a cached (possibly stale) verdict.
// Plain mounts/refreshes keep the cache so Settings visits stay cheap.
const refresh = useCallback(
async ({ fresh = false } = {}) => {
setLoading(true);
setError(null);
try {
const data = await apiJson(`/api/settings/hf-token/state${fresh ? '?fresh=1' : ''}`);
setState(data);
} catch (e) {
setError(
e?.message ||
t('settings.hf_token_load_error', { defaultValue: 'Failed to load token state' }),
);
} finally {
setLoading(false);
}
},
[t],
);
useEffect(() => {
refresh();
}, [refresh]);
const onSave = async () => {
// `saving` mirrors the button's disabled state for the input's Enter path,
// closing the double-submit hole (Enter fired POSTs while one was in flight).
if (saving) return;
const token = tokenInput.trim();
if (!token) return;
setSaving(true);
@@ -131,7 +133,7 @@ export default function ApiKeysPanel() {
<button
type="button"
className="inline-flex cursor-pointer items-center gap-[5px] rounded-[var(--chrome-radius-pill)] [border:1px_solid_var(--chrome-border)] bg-transparent px-[var(--space-4)] py-[var(--space-2)] text-[length:var(--text-sm)] font-medium text-[var(--chrome-fg)] hover:enabled:bg-[var(--chrome-hover-bg)] disabled:cursor-not-allowed disabled:opacity-50"
onClick={refresh}
onClick={() => refresh({ fresh: true })}
disabled={loading}
aria-label={testNowLabel}
title={t('settings.hf_token_test_now_title', {
@@ -151,106 +153,122 @@ export default function ApiKeysPanel() {
</div>
)}
<div
className="flex flex-col gap-[var(--space-3)]"
role="table"
aria-label={t('settings.hf_token_sources', { defaultValue: 'HF token sources' })}
>
{state.sources.map((row) => {
const isActive = state.active === row.source;
return (
<div
key={row.source}
className={`apikeys-row ${isActive ? 'apikeys-row--active' : ''}`}
role="row"
data-source={row.source}
>
<div className="flex items-center justify-between gap-[var(--space-3)]">
<span className="inline-flex items-center gap-[var(--space-2)] text-[length:var(--text-md)] font-medium text-[var(--chrome-fg)]">
{SOURCE_LABELS[row.source]}
<InfoHint>{SOURCE_HELP[row.source]}</InfoHint>
</span>
{isActive && (
<span className="apikeys-badge apikeys-badge--active">
{t('settings.hf_token_active', { defaultValue: 'Active' })}
{!state ? (
// First load still in flight (or failed the banner above explains and
// "Test now" doubles as retry). Never show a wrong "not set" verdict.
<div
className="py-[var(--space-4)] text-[length:var(--text-sm)] text-[var(--chrome-fg-muted)]"
role="status"
data-testid="hf-token-loading"
>
{loading && t('settings.hf_token_checking', { defaultValue: 'Checking token sources…' })}
</div>
) : (
/* Visually a stack of cards, not a data grid list semantics are the
valid ARIA fit (the old role="table" had rows with no cells). */
<div
className="flex flex-col gap-[var(--space-3)]"
role="list"
aria-label={t('settings.hf_token_sources', { defaultValue: 'HF token sources' })}
>
{state.sources.map((row) => {
const isActive = state.active === row.source;
return (
<div
key={row.source}
className={`apikeys-row ${isActive ? 'apikeys-row--active' : ''}`}
role="listitem"
data-source={row.source}
>
<div className="flex items-center justify-between gap-[var(--space-3)]">
<span className="inline-flex items-center gap-[var(--space-2)] text-[length:var(--text-md)] font-medium text-[var(--chrome-fg)]">
{SOURCE_LABELS[row.source]}
<InfoHint>{SOURCE_HELP[row.source]}</InfoHint>
</span>
)}
</div>
<div className="flex flex-wrap items-center gap-[var(--space-3)] text-[length:var(--text-sm)] text-[var(--chrome-fg-muted)]">
{row.set ? (
<>
<span
className="inline-flex items-center gap-[4px] text-[var(--chrome-severity-ok)]"
aria-label={t('settings.hf_token_set', { defaultValue: 'set' })}
>
<CheckCircle2 size={12} />{' '}
{t('settings.hf_token_set', { defaultValue: 'set' })}
{isActive && (
<span className="apikeys-badge apikeys-badge--active">
{t('settings.hf_token_active', { defaultValue: 'Active' })}
</span>
{row.masked && (
<code className="rounded-[4px] bg-[var(--chrome-hover-bg)] px-[6px] py-[1px] font-mono text-[length:var(--text-xs)]">
{row.masked}
</code>
)}
{row.whoami_ok ? (
<span className="inline-flex items-center gap-[4px] text-[var(--chrome-severity-ok)]">
<CheckCircle2 size={12} />{' '}
{row.whoami_user ||
t('settings.hf_token_verified', { defaultValue: 'verified' })}
</span>
) : (
<span className="inline-flex items-center gap-[4px] text-[var(--chrome-severity-err)]">
<XCircle size={12} />{' '}
{t('settings.hf_token_whoami_failed', { defaultValue: 'whoami failed' })}
</span>
)}
</>
) : (
<span className="inline-flex items-center gap-[4px] text-[var(--chrome-severity-warn)]">
<XCircle size={12} />{' '}
{t('settings.hf_token_not_set', { defaultValue: 'not set' })}
</span>
)}
</div>
{row.source === 'app' && (
<div className="mt-[var(--space-2)] flex flex-wrap items-center gap-[var(--space-3)]">
<input
type="password"
className="box-border min-w-0 max-w-full flex-[1_1_220px] rounded-[var(--chrome-radius-pill)] [border:1px_solid_var(--chrome-border)] bg-[var(--chrome-hover-bg)] px-[var(--space-3)] py-[var(--space-2)] font-mono text-[length:var(--text-sm)] text-[var(--chrome-fg)] focus:border-[var(--chrome-accent)] focus:outline-none"
placeholder="hf_…"
aria-label={t('settings.hf_token_input', { defaultValue: 'HuggingFace token' })}
value={tokenInput}
onChange={(e) => setTokenInput(e.target.value)}
onKeyDown={(e) => {
if (e.key === 'Enter') onSave();
}}
autoComplete="off"
spellCheck={false}
/>
<button
type="button"
className="inline-flex cursor-pointer items-center gap-[5px] rounded-[var(--chrome-radius-pill)] [border:1px_solid_var(--chrome-accent)] bg-[color-mix(in_srgb,var(--chrome-accent)_25%,var(--chrome-bg))] px-[var(--space-4)] py-[var(--space-2)] text-[length:var(--text-sm)] font-medium text-[var(--chrome-fg)] hover:enabled:bg-[var(--chrome-hover-bg)] disabled:cursor-not-allowed disabled:opacity-50"
onClick={onSave}
disabled={!tokenInput.trim() || saving}
>
<Save size={12} /> {t('common.save')}
</button>
{row.set && (
<button
type="button"
className="inline-flex cursor-pointer items-center gap-[5px] rounded-[var(--chrome-radius-pill)] [border:1px_solid_color-mix(in_srgb,var(--chrome-severity-err)_35%,var(--chrome-border))] bg-[var(--chrome-bg)] px-[var(--space-4)] py-[var(--space-2)] text-[length:var(--text-sm)] font-medium text-[var(--chrome-severity-err)] hover:enabled:bg-[var(--chrome-hover-bg)] disabled:cursor-not-allowed disabled:opacity-50"
onClick={() => setClearOpen(true)}
disabled={saving}
>
<Trash2 size={12} />{' '}
{t('settings.hf_token_clear_short', { defaultValue: 'Clear' })}
</button>
)}
</div>
)}
</div>
);
})}
</div>
<div className="flex flex-wrap items-center gap-[var(--space-3)] text-[length:var(--text-sm)] text-[var(--chrome-fg-muted)]">
{row.set ? (
<>
<span
className="inline-flex items-center gap-[4px] text-[var(--chrome-severity-ok)]"
aria-label={t('settings.hf_token_set', { defaultValue: 'set' })}
>
<CheckCircle2 size={12} />{' '}
{t('settings.hf_token_set', { defaultValue: 'set' })}
</span>
{row.masked && (
<code className="rounded-[4px] bg-[var(--chrome-hover-bg)] px-[6px] py-[1px] font-mono text-[length:var(--text-xs)]">
{row.masked}
</code>
)}
{row.whoami_ok ? (
<span className="inline-flex items-center gap-[4px] text-[var(--chrome-severity-ok)]">
<CheckCircle2 size={12} />{' '}
{row.whoami_user ||
t('settings.hf_token_verified', { defaultValue: 'verified' })}
</span>
) : (
<span className="inline-flex items-center gap-[4px] text-[var(--chrome-severity-err)]">
<XCircle size={12} />{' '}
{t('settings.hf_token_whoami_failed', { defaultValue: 'whoami failed' })}
</span>
)}
</>
) : (
<span className="inline-flex items-center gap-[4px] text-[var(--chrome-severity-warn)]">
<XCircle size={12} />{' '}
{t('settings.hf_token_not_set', { defaultValue: 'not set' })}
</span>
)}
</div>
{row.source === 'app' && (
<div className="mt-[var(--space-2)] flex flex-wrap items-center gap-[var(--space-3)]">
<input
type="password"
className="box-border min-w-0 max-w-full flex-[1_1_220px] rounded-[var(--chrome-radius-pill)] [border:1px_solid_var(--chrome-border)] bg-[var(--chrome-hover-bg)] px-[var(--space-3)] py-[var(--space-2)] font-mono text-[length:var(--text-sm)] text-[var(--chrome-fg)] focus:border-[var(--chrome-accent)] focus:outline-none"
placeholder="hf_…"
aria-label={t('settings.hf_token_input', {
defaultValue: 'HuggingFace token',
})}
value={tokenInput}
onChange={(e) => setTokenInput(e.target.value)}
onKeyDown={(e) => {
if (e.key === 'Enter') onSave();
}}
autoComplete="off"
spellCheck={false}
/>
<button
type="button"
className="inline-flex cursor-pointer items-center gap-[5px] rounded-[var(--chrome-radius-pill)] [border:1px_solid_var(--chrome-accent)] bg-[color-mix(in_srgb,var(--chrome-accent)_25%,var(--chrome-bg))] px-[var(--space-4)] py-[var(--space-2)] text-[length:var(--text-sm)] font-medium text-[var(--chrome-fg)] hover:enabled:bg-[var(--chrome-hover-bg)] disabled:cursor-not-allowed disabled:opacity-50"
onClick={onSave}
disabled={!tokenInput.trim() || saving}
>
<Save size={12} /> {t('common.save')}
</button>
{row.set && (
<button
type="button"
className="inline-flex cursor-pointer items-center gap-[5px] rounded-[var(--chrome-radius-pill)] [border:1px_solid_color-mix(in_srgb,var(--chrome-severity-err)_35%,var(--chrome-border))] bg-[var(--chrome-bg)] px-[var(--space-4)] py-[var(--space-2)] text-[length:var(--text-sm)] font-medium text-[var(--chrome-severity-err)] hover:enabled:bg-[var(--chrome-hover-bg)] disabled:cursor-not-allowed disabled:opacity-50"
onClick={() => setClearOpen(true)}
disabled={saving}
>
<Trash2 size={12} />{' '}
{t('settings.hf_token_clear_short', { defaultValue: 'Clear' })}
</button>
)}
</div>
)}
</div>
);
})}
</div>
)}
{clearOpen && (
<div
@@ -150,7 +150,7 @@ describe('ApiKeysPanel', () => {
});
});
it('"Test now" button refetches state', async () => {
it('"Test now" busts the whoami cache (?fresh=1); plain mounts stay cached', async () => {
const fetchMock = mockFetchSequence(
{ status: 200, body: STATE_THREE_UNSET },
{ status: 200, body: STATE_THREE_UNSET },
@@ -159,11 +159,88 @@ describe('ApiKeysPanel', () => {
render(<ApiKeysPanel />);
await waitFor(() => screen.getByPlaceholderText(/hf_/));
// Mount GET keeps the backend cache no fresh param.
expect(fetchMock.mock.calls[0][0]).not.toMatch(/fresh=1/);
const testBtn = screen.getByRole('button', { name: /test now/i });
fireEvent.click(testBtn);
// The button claims to re-run whoami, so it must actually bypass the
// backend's 300s validation cache.
await waitFor(() => {
expect(fetchMock.mock.calls.length).toBeGreaterThanOrEqual(2);
expect(fetchMock.mock.calls[1][0]).toMatch(/\/api\/settings\/hf-token\/state\?fresh=1$/);
});
});
it('initial load shows a checking placeholder, never a false "not set" verdict', async () => {
let resolveFetch;
global.fetch = vi.fn(
() =>
new Promise((resolve) => {
resolveFetch = resolve;
}),
);
const { container } = render(<ApiKeysPanel />);
// While the GET is in flight: placeholder, no source rows, no verdicts.
expect(screen.getByTestId('hf-token-loading')).toBeInTheDocument();
expect(screen.queryByText(/not set/i)).toBeNull();
expect(container.querySelectorAll('.apikeys-row').length).toBe(0);
resolveFetch({
ok: true,
status: 200,
json: async () => STATE_APP_ACTIVE,
text: async () => JSON.stringify(STATE_APP_ACTIVE),
});
await waitFor(() => {
expect(container.querySelectorAll('.apikeys-row').length).toBe(3);
expect(screen.queryByTestId('hf-token-loading')).toBeNull();
});
});
it('renders the sources as a valid ARIA list (no cell-less table)', async () => {
global.fetch = mockFetchOnce(STATE_THREE_UNSET);
render(<ApiKeysPanel />);
const list = await screen.findByRole('list', { name: /HF token sources/i });
expect(list.querySelectorAll('[role="listitem"]').length).toBe(3);
});
it('Enter while a save is in flight does not fire a duplicate POST', async () => {
let resolvePost;
const fetchMock = vi
.fn()
.mockResolvedValueOnce({
ok: true,
status: 200,
json: async () => STATE_THREE_UNSET,
text: async () => JSON.stringify(STATE_THREE_UNSET),
})
.mockImplementation(
() =>
new Promise((resolve) => {
resolvePost = resolve;
}),
);
global.fetch = fetchMock;
render(<ApiKeysPanel />);
const input = await screen.findByPlaceholderText(/hf_/);
fireEvent.change(input, { target: { value: 'hf_newtoken123' } });
fireEvent.keyDown(input, { key: 'Enter' });
fireEvent.keyDown(input, { key: 'Enter' }); // Save button is disabled; Enter must be too.
fireEvent.keyDown(input, { key: 'Enter' });
await waitFor(() => {
const posts = fetchMock.mock.calls.filter(([, opts]) => opts?.method === 'POST');
expect(posts.length).toBe(1);
});
resolvePost({
ok: true,
status: 200,
json: async () => STATE_APP_ACTIVE,
text: async () => JSON.stringify(STATE_APP_ACTIVE),
});
});
});
@@ -21,6 +21,39 @@ const THEMES = [
{ id: 'catppuccin', label: 'Catppuccin', dot: '#cba6f7' },
];
/**
* WAI-ARIA radio-group keyboard support for the theme-dot / font-tile pickers:
* arrow keys move selection (wrapping), Home/End jump to the ends, and focus
* follows selection. Pair with `radioTabIndex` for the roving tabindex so the
* group occupies a single tab stop, as the announced role promises.
*/
function radioGroupKeyDown(e, values, current, select) {
const STEP = { ArrowRight: 1, ArrowDown: 1, ArrowLeft: -1, ArrowUp: -1 };
let next;
if (e.key in STEP) {
const idx = Math.max(0, values.indexOf(current));
next = values[(idx + STEP[e.key] + values.length) % values.length];
} else if (e.key === 'Home') {
next = values[0];
} else if (e.key === 'End') {
next = values[values.length - 1];
}
if (!next) return;
e.preventDefault();
select(next);
const el = e.currentTarget
.closest('[role="radiogroup"]')
?.querySelector(`[data-radio-value="${next}"]`);
el?.focus();
}
/** Roving tabindex: only the checked radio (or the first, if none is checked
* e.g. a stale persisted value) is tabbable. */
function radioTabIndex(values, current, value) {
const focusable = values.includes(current) ? current : values[0];
return value === focusable ? 0 : -1;
}
export default function AppearancePanel() {
const { t } = useTranslation();
const uiScale = useAppStore((s) => s.uiScale);
@@ -37,6 +70,8 @@ export default function AppearancePanel() {
const scaleLabel = t('settings.ui_scale', { defaultValue: 'UI scale' });
const themeLabel = t('settings.color_theme', { defaultValue: 'Color theme' });
const fontLabel = t('settings.font', { defaultValue: 'Font' });
const themeIds = THEMES.map((th) => th.id);
const fontIds = FONT_OPTIONS.map((f) => f.id);
return (
<SettingsSection
@@ -89,10 +124,13 @@ export default function AppearancePanel() {
className={`appearance-panel__theme-dot ${theme === th.id ? 'is-active' : ''}`}
style={{ '--dot-color': th.dot }}
onClick={() => setTheme(th.id)}
onKeyDown={(e) => radioGroupKeyDown(e, themeIds, theme, setTheme)}
title={th.label}
aria-label={th.label}
aria-checked={theme === th.id}
role="radio"
tabIndex={radioTabIndex(themeIds, theme, th.id)}
data-radio-value={th.id}
/>
))}
</div>
@@ -117,10 +155,13 @@ export default function AppearancePanel() {
role="radio"
aria-checked={font === f.id}
aria-label={f.label}
tabIndex={radioTabIndex(fontIds, font, f.id)}
data-radio-value={f.id}
data-testid={`appearance-font-${f.id}`}
className={`appearance-panel__font-tile ${font === f.id ? 'is-active' : ''}`}
style={{ fontFamily: FONT_STACKS[f.id] || 'var(--font-sans)' }}
onClick={() => setFont(f.id)}
onKeyDown={(e) => radioGroupKeyDown(e, fontIds, font, setFont)}
>
<span className="appearance-panel__font-sample">Ag</span>
<span className="appearance-panel__font-name">{f.label}</span>
@@ -50,6 +50,56 @@ describe('AppearancePanel — global font selection', () => {
});
});
describe('AppearancePanel — WAI-ARIA radio-group keyboard pattern', () => {
const fontIds = FONT_OPTIONS.map((f) => f.id);
beforeEach(() => {
useAppStore.getState().setFont(fontIds[0]);
useAppStore.getState().setTheme('gruvbox');
document.documentElement.style.removeProperty('--font-sans');
});
it('roving tabindex: only the checked font tile is tabbable', () => {
render(<AppearancePanel />);
expect(screen.getByTestId(`appearance-font-${fontIds[0]}`)).toHaveAttribute('tabindex', '0');
for (const id of fontIds.slice(1)) {
expect(screen.getByTestId(`appearance-font-${id}`)).toHaveAttribute('tabindex', '-1');
}
});
it('ArrowRight moves font selection and focus to the next tile', () => {
render(<AppearancePanel />);
const first = screen.getByTestId(`appearance-font-${fontIds[0]}`);
first.focus();
fireEvent.keyDown(first, { key: 'ArrowRight' });
expect(useAppStore.getState().font).toBe(fontIds[1]);
const second = screen.getByTestId(`appearance-font-${fontIds[1]}`);
expect(second).toHaveFocus();
expect(second).toHaveAttribute('aria-checked', 'true');
// Roving tabindex followed the selection.
expect(second).toHaveAttribute('tabindex', '0');
expect(first).toHaveAttribute('tabindex', '-1');
});
it('ArrowLeft wraps from the first font to the last', () => {
render(<AppearancePanel />);
const first = screen.getByTestId(`appearance-font-${fontIds[0]}`);
first.focus();
fireEvent.keyDown(first, { key: 'ArrowLeft' });
expect(useAppStore.getState().font).toBe(fontIds[fontIds.length - 1]);
});
it('arrow keys move the theme-dot selection too', () => {
render(<AppearancePanel />);
const gruvbox = screen.getByRole('radio', { name: 'Gruvbox' });
gruvbox.focus();
fireEvent.keyDown(gruvbox, { key: 'ArrowDown' });
expect(useAppStore.getState().theme).toBe('midnight');
expect(screen.getByRole('radio', { name: 'Midnight' })).toHaveFocus();
});
});
describe('AppearancePanel — auto-play preview toggle (#666)', () => {
it('defaults to ON (preserves existing auto-play behavior)', () => {
expect(useAppStore.getState().autoPlayPreview).toBe(true);
@@ -27,7 +27,12 @@ export default function AsrOpenAICompatPanel() {
const [apiKey, setApiKey] = useState('');
const [hasKey, setHasKey] = useState(false);
const [saving, setSaving] = useState(false);
const [saved, setSaved] = useState(false);
const [error, setError] = useState(null);
// Last server-acknowledged values: the one Save button persists all three
// fields, so it stays disabled until something actually differs (dirty) and
// a successful save shows an explicit "Saved" confirmation.
const [server, setServer] = useState({ base_url: '', model: '' });
const refresh = useCallback(async () => {
setError(null);
@@ -37,6 +42,7 @@ export default function AsrOpenAICompatPanel() {
setModel(d?.model || '');
setHasKey(Boolean(d?.has_key));
setApiKey(''); // the key is never returned the field always starts blank
setServer({ base_url: d?.base_url || '', model: d?.model || '' });
} catch (e) {
setError(e?.message || t('models.asrOpenAICompatLoadError'));
}
@@ -68,6 +74,8 @@ export default function AsrOpenAICompatPanel() {
setModel(d.model || '');
setHasKey(Boolean(d.has_key));
setApiKey('');
setServer({ base_url: d.base_url || '', model: d.model || '' });
setSaved(true);
} catch (e) {
setError(e?.message || t('models.asrOpenAICompatSaveError'));
} finally {
@@ -75,6 +83,8 @@ export default function AsrOpenAICompatPanel() {
}
};
const dirty = baseUrl !== server.base_url || model !== server.model || apiKey !== '';
return (
<SettingsSection
icon={Mic}
@@ -139,11 +149,20 @@ export default function AsrOpenAICompatPanel() {
size="sm"
onClick={save}
loading={saving}
disabled={saving}
disabled={saving || !dirty}
data-testid="asr-openai-compat-save"
>
{t('common.save')}
</Button>
{saved && !dirty && !saving && (
<span
className="text-[length:var(--text-xs)] text-[color:var(--chrome-fg-dim)]"
role="status"
data-testid="asr-openai-compat-saved"
>
{t('models.asrOpenAICompatSaved')}
</span>
)}
</>
}
/>
@@ -0,0 +1,397 @@
/**
* Settings Audio tools the power-user surface for the media tools most
* users never see (the wizard + backend provision them invisibly).
*
* One row per tool:
* FFmpeg / FFprobe version + origin badge (Bundled / System / Custom /
* App package) + path; actions: Use system copy (auto-detect),
* Choose file (picker in Tauri, inline path input everywhere),
* Restore bundled (always-safe revert). The section header carries
* "Update bundled build" (one download covers both binaries).
* yt-dlp module version + Update (fetches the newest wheel into an
* update-surviving overlay; applies on restart) + Restore tested version.
*
* Absorbs the FFmpeg-path override that used to live in Settings Network
* same backend store (prefs `env.FFMPEG_PATH`), one control surface.
*/
import React, { useCallback, useEffect, useRef, useState } from 'react';
import { toast } from 'react-hot-toast';
import { useTranslation } from 'react-i18next';
import { AudioLines, Film, ScanSearch, DownloadCloud } from 'lucide-react';
import { Button, Badge } from '../../ui';
import { SettingsSection, SettingRow, SettingsInput } from './primitives';
import RestartBadge from './RestartBadge';
import { isTauri } from './native';
const ORIGIN_TONE = {
bundled: 'success',
sidecar: 'success',
system: 'info',
custom: 'warn',
};
function OriginBadge({ origin }) {
const { t } = useTranslation();
if (!origin) return null;
const labels = {
bundled: t('settings.audio_tools_origin_bundled', { defaultValue: 'Bundled' }),
system: t('settings.audio_tools_origin_system', { defaultValue: 'System' }),
custom: t('settings.audio_tools_origin_custom', { defaultValue: 'Custom' }),
sidecar: t('settings.audio_tools_origin_sidecar', { defaultValue: 'App package' }),
};
return (
<Badge tone={ORIGIN_TONE[origin] || 'neutral'} size="xs" data-testid={`origin-${origin}`}>
{labels[origin] || origin}
</Badge>
);
}
/** Open the OS file picker in Tauri; return the chosen path or null. */
async function pickBinary(title) {
if (!isTauri()) return null;
try {
const { open } = await import('@tauri-apps/plugin-dialog');
const picked = await open({ multiple: false, directory: false, title });
return typeof picked === 'string' ? picked : null;
} catch {
return null;
}
}
function BinaryRow({ tool, info, onAction, busy }) {
const { t } = useTranslation();
const [path, setPath] = useState('');
const [showInput, setShowInput] = useState(false);
const label = tool === 'ffmpeg' ? 'FFmpeg' : 'FFprobe';
const chooseFile = async () => {
const picked = await pickBinary(label);
if (picked) {
onAction(`/media-tools/${tool}/custom-path`, { path: picked });
} else {
// Web preview / picker unavailable fall back to the inline input.
setShowInput(true);
}
};
return (
<SettingRow
align="start"
stack
icon={tool === 'ffmpeg' ? Film : ScanSearch}
title={
<>
{label}
<OriginBadge origin={info?.origin} />
{!info?.ok && (
<Badge tone="warn" size="xs">
{t('settings.audio_tools_not_found', { defaultValue: 'Not available' })}
</Badge>
)}
</>
}
note={
info?.ok ? (
<>
{info.version ||
t('settings.audio_tools_version_unknown', { defaultValue: 'version unknown' })}
{' — '}
<code className="font-mono">{info.path}</code>
</>
) : (
t(`settings.audio_tools_${tool}_desc`)
)
}
control={
<>
<Button
size="sm"
variant="ghost"
disabled={busy}
onClick={() => onAction(`/media-tools/${tool}/use-system`)}
aria-label={`${label}: ${t('settings.audio_tools_use_system')}`}
>
{t('settings.audio_tools_use_system', { defaultValue: 'Use system copy' })}
</Button>
<Button
size="sm"
variant="ghost"
disabled={busy}
onClick={chooseFile}
aria-label={`${label}: ${t('settings.audio_tools_choose_file')}`}
>
{t('settings.audio_tools_choose_file', { defaultValue: 'Choose file…' })}
</Button>
<Button
size="sm"
variant="ghost"
disabled={busy}
onClick={() => onAction(`/media-tools/${tool}/restore`)}
aria-label={`${label}: ${t('settings.audio_tools_restore')}`}
>
{t('settings.audio_tools_restore', { defaultValue: 'Restore bundled' })}
</Button>
{showInput && (
<>
<SettingsInput
placeholder={tool === 'ffmpeg' ? '/usr/bin/ffmpeg' : '/usr/bin/ffprobe'}
value={path}
onChange={(e) => setPath(e.target.value)}
onKeyDown={(e) =>
e.key === 'Enter' &&
path.trim() &&
onAction(`/media-tools/${tool}/custom-path`, { path: path.trim() })
}
aria-label={t('settings.audio_tools_path_input_aria', {
tool: label,
defaultValue: '{{tool}} binary path',
})}
/>
<Button
size="sm"
variant="subtle"
disabled={busy || !path.trim()}
onClick={() => onAction(`/media-tools/${tool}/custom-path`, { path: path.trim() })}
>
{t('credentials.save', { defaultValue: 'Save' })}
</Button>
</>
)}
</>
}
/>
);
}
export default function AudioToolsPanel() {
const { t } = useTranslation();
const [status, setStatus] = useState(null);
const [busy, setBusy] = useState(false);
const acquireWasRunning = useRef(false);
const ytdlpWasRunning = useRef(false);
const load = useCallback(async () => {
try {
const { apiJson } = await import('../../api/client');
const st = await apiJson('/media-tools/status');
setStatus(st);
return st;
} catch {
return null;
}
}, []);
useEffect(() => {
load();
}, [load]);
// Poll while a background op runs; toast exactly once on the edge.
const acquire = status?.ops?.acquire;
const ytdlpOp = status?.ops?.ytdlp_update;
useEffect(() => {
if (acquire?.state === 'running') acquireWasRunning.current = true;
else if (acquireWasRunning.current) {
acquireWasRunning.current = false;
if (acquire?.state === 'done') {
toast.success(
t('settings.audio_tools_bundle_done', { defaultValue: 'Bundled media engine ready.' }),
);
} else if (acquire?.state === 'error') {
toast.error(
t('settings.audio_tools_bundle_failed', {
message: acquire.error,
defaultValue: 'Bundled download failed: {{message}}',
}),
);
}
}
if (ytdlpOp?.state === 'running') ytdlpWasRunning.current = true;
else if (ytdlpWasRunning.current) {
ytdlpWasRunning.current = false;
if (ytdlpOp?.state === 'done') {
toast.success(
t('settings.audio_tools_ytdlp_updated', {
version: ytdlpOp.version,
defaultValue: 'yt-dlp {{version}} installed — restart the backend to apply.',
}),
);
} else if (ytdlpOp?.state === 'error') {
toast.error(
t('settings.audio_tools_ytdlp_update_failed', {
message: ytdlpOp.error,
defaultValue: 'yt-dlp update failed: {{message}}',
}),
);
}
}
if (acquire?.state !== 'running' && ytdlpOp?.state !== 'running') return undefined;
const iv = setInterval(load, 1500);
return () => clearInterval(iv);
}, [acquire?.state, ytdlpOp?.state, load, t, acquire?.error, ytdlpOp?.error, ytdlpOp?.version]);
const post = useCallback(
async (path, body) => {
setBusy(true);
try {
const { apiFetch } = await import('../../api/client');
const res = await apiFetch(path, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: body ? JSON.stringify(body) : undefined,
});
if (!res.ok) {
let detail = `HTTP ${res.status}`;
try {
detail = (await res.json())?.detail || detail;
} catch {
/* non-JSON error body */
}
throw new Error(detail);
}
return true;
} catch (e) {
toast.error(
t('settings.audio_tools_path_failed', {
message: e.message,
defaultValue: "Couldn't set path: {{message}}",
}),
);
return false;
} finally {
setBusy(false);
load();
}
},
[load, t],
);
const onToolAction = useCallback(
async (path, body) => {
const ok = await post(path, body);
if (ok && (path.endsWith('/custom-path') || path.endsWith('/use-system'))) {
toast.success(
t('settings.audio_tools_path_set', {
tool: path.includes('ffprobe') ? 'FFprobe' : 'FFmpeg',
path: body?.path || t('settings.audio_tools_origin_system', { defaultValue: 'System' }),
defaultValue: '{{tool}} now uses {{path}}',
}),
);
} else if (ok && path.endsWith('/restore')) {
toast.success(
t('settings.audio_tools_restored', {
tool: path.includes('ffprobe') ? 'FFprobe' : 'FFmpeg',
defaultValue: '{{tool}} restored to the app-managed build.',
}),
);
}
},
[post, t],
);
const ytdlp = status?.tools?.ytdlp;
const ytdlpNeedsRestart =
ytdlpOp?.state === 'done' ||
(ytdlp?.overlay_version && ytdlp.overlay_version !== ytdlp.version);
return (
<SettingsSection
icon={AudioLines}
title={t('settings.audio_tools', { defaultValue: 'Audio tools' })}
description={t('settings.audio_tools_desc', {
defaultValue:
'The media engine (FFmpeg, FFprobe) and video downloader (yt-dlp) the app manages for you.',
})}
actions={
<Button
size="sm"
variant="ghost"
leading={<DownloadCloud size={12} />}
loading={acquire?.state === 'running'}
disabled={busy || acquire?.state === 'running'}
onClick={() => post('/media-tools/acquire')}
aria-label={t('settings.audio_tools_update_bundle', {
defaultValue: 'Update bundled build',
})}
>
{acquire?.state === 'running'
? t('settings.audio_tools_bundle_updating', {
percent: Math.round((acquire.progress || 0) * 100),
defaultValue: 'Downloading bundled build… {{percent}}%',
})
: t('settings.audio_tools_update_bundle', { defaultValue: 'Update bundled build' })}
</Button>
}
>
<BinaryRow tool="ffmpeg" info={status?.tools?.ffmpeg} onAction={onToolAction} busy={busy} />
<BinaryRow tool="ffprobe" info={status?.tools?.ffprobe} onAction={onToolAction} busy={busy} />
<SettingRow
align="start"
stack
icon={DownloadCloud}
title={
<>
{t('settings.audio_tools_ytdlp', { defaultValue: 'yt-dlp (video downloader)' })}
{ytdlp?.origin && (
<OriginBadge origin={ytdlp.origin === 'custom' ? 'custom' : 'bundled'} />
)}
{ytdlpNeedsRestart && <RestartBadge />}
</>
}
note={
<>
{ytdlp?.version ||
t('settings.audio_tools_version_unknown', { defaultValue: 'version unknown' })}
{' — '}
{t('settings.audio_tools_ytdlp_desc', {
defaultValue:
'Powers video/clip imports. Site support changes faster than app releases — update it here when imports start failing.',
})}
</>
}
hint={t('settings.audio_tools_manual_hint', {
defaultValue:
'Prefer your package manager? Install FFmpeg yourself (macOS: brew install ffmpeg · Debian/Ubuntu: sudo apt install ffmpeg · Windows: winget install ffmpeg) and press Use system copy. Nothing is ever installed system-wide by the app.',
})}
control={
<>
<Button
size="sm"
variant="subtle"
loading={ytdlpOp?.state === 'running'}
disabled={busy || ytdlpOp?.state === 'running'}
onClick={() => post('/media-tools/ytdlp/update')}
aria-label={`yt-dlp: ${t('settings.audio_tools_ytdlp_update', { defaultValue: 'Update' })}`}
>
{t('settings.audio_tools_ytdlp_update', { defaultValue: 'Update' })}
</Button>
{(ytdlp?.origin === 'custom' || ytdlp?.overlay_version) && (
<Button
size="sm"
variant="ghost"
disabled={busy || ytdlpOp?.state === 'running'}
onClick={async () => {
const ok = await post('/media-tools/ytdlp/restore');
if (ok) {
toast.success(
t('settings.audio_tools_ytdlp_restored', {
defaultValue: 'Tested yt-dlp restored — restart the backend to apply.',
}),
);
}
}}
aria-label={`yt-dlp: ${t('settings.audio_tools_ytdlp_restore', { defaultValue: 'Restore tested version' })}`}
data-testid="ytdlp-restore"
>
{t('settings.audio_tools_ytdlp_restore', {
defaultValue: 'Restore tested version',
})}
{ytdlp?.baseline_version ? ` (${ytdlp.baseline_version})` : ''}
</Button>
)}
</>
}
/>
</SettingsSection>
);
}
@@ -0,0 +1,155 @@
import React from 'react';
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { fireEvent, render, screen, waitFor } from '@testing-library/react';
vi.mock('react-hot-toast', () => ({
default: { error: vi.fn(), success: vi.fn() },
toast: { error: vi.fn(), success: vi.fn() },
}));
vi.mock('../../api/client', () => ({
apiJson: vi.fn(),
apiFetch: vi.fn(),
}));
import { toast } from 'react-hot-toast';
import { apiJson, apiFetch } from '../../api/client';
import AudioToolsPanel from './AudioToolsPanel';
const STATUS = {
ready: true,
platform_key: 'darwin_arm64',
tools: {
ffmpeg: {
tool: 'ffmpeg',
ok: true,
path: '/data/media_tools/ffbin-abc/darwin_arm64/ffmpeg',
version: '7.0',
origin: 'bundled',
},
ffprobe: {
tool: 'ffprobe',
ok: true,
path: '/opt/homebrew/bin/ffprobe',
version: '8.1.1',
origin: 'system',
},
ytdlp: {
tool: 'yt-dlp',
ok: true,
path: '/venv/site-packages/yt_dlp',
version: '2026.06.09',
origin: 'bundled',
overlay_version: null,
baseline_version: null,
},
},
ops: {
acquire: { state: 'idle', progress: 0, error: null },
ytdlp_update: { state: 'idle', progress: 0, error: null, version: null },
},
};
const okResponse = { ok: true, json: async () => ({}) };
describe('AudioToolsPanel — power-user surface for the media tools', () => {
beforeEach(() => {
vi.clearAllMocks();
apiJson.mockResolvedValue(JSON.parse(JSON.stringify(STATUS)));
apiFetch.mockResolvedValue(okResponse);
});
it('renders one row per tool with version, path, and origin badge', async () => {
render(<AudioToolsPanel />);
await waitFor(() => expect(apiJson).toHaveBeenCalledWith('/media-tools/status'));
expect(await screen.findByText('FFmpeg')).toBeInTheDocument();
expect(screen.getByText('FFprobe')).toBeInTheDocument();
expect(screen.getByText('yt-dlp (video downloader)')).toBeInTheDocument();
// ffmpeg + yt-dlp are both app-managed here; ffprobe is a system copy.
const bundled = screen.getAllByTestId('origin-bundled');
expect(bundled).toHaveLength(2);
expect(bundled[0]).toHaveTextContent('Bundled');
expect(screen.getByTestId('origin-system')).toHaveTextContent('System');
expect(screen.getByText('/opt/homebrew/bin/ffprobe')).toBeInTheDocument();
expect(screen.getByText(/2026\.06\.09/)).toBeInTheDocument();
});
it('Use system copy posts the endpoint and toasts success', async () => {
render(<AudioToolsPanel />);
fireEvent.click(await screen.findByLabelText('FFmpeg: Use system copy'));
await waitFor(() =>
expect(apiFetch).toHaveBeenCalledWith('/media-tools/ffmpeg/use-system', expect.anything()),
);
await waitFor(() => expect(toast.success).toHaveBeenCalled());
});
it('Restore bundled is per-tool and always available (safe revert)', async () => {
render(<AudioToolsPanel />);
fireEvent.click(await screen.findByLabelText('FFprobe: Restore bundled'));
await waitFor(() =>
expect(apiFetch).toHaveBeenCalledWith('/media-tools/ffprobe/restore', expect.anything()),
);
await waitFor(() => expect(toast.success).toHaveBeenCalled());
});
it('surfaces the backend error detail on a failed action', async () => {
apiFetch.mockResolvedValue({
ok: false,
status: 400,
json: async () => ({ detail: 'That file exists but does not run as a media tool' }),
});
render(<AudioToolsPanel />);
fireEvent.click(await screen.findByLabelText('FFmpeg: Use system copy'));
await waitFor(() => expect(toast.error).toHaveBeenCalled());
expect(String(toast.error.mock.calls[0][0])).toContain('does not run as a media tool');
});
it('yt-dlp row: Update posts the update endpoint', async () => {
render(<AudioToolsPanel />);
fireEvent.click(await screen.findByLabelText('yt-dlp: Update'));
await waitFor(() =>
expect(apiFetch).toHaveBeenCalledWith('/media-tools/ytdlp/update', expect.anything()),
);
});
it('yt-dlp row: Restore tested version appears only when an overlay is active', async () => {
const { unmount } = render(<AudioToolsPanel />);
await screen.findByText('yt-dlp (video downloader)');
expect(screen.queryByTestId('ytdlp-restore')).not.toBeInTheDocument();
unmount();
const overlaid = JSON.parse(JSON.stringify(STATUS));
overlaid.tools.ytdlp.origin = 'custom';
overlaid.tools.ytdlp.overlay_version = '2026.07.01';
overlaid.tools.ytdlp.version = '2026.07.01';
overlaid.tools.ytdlp.baseline_version = '2026.06.09';
apiJson.mockResolvedValue(overlaid);
render(<AudioToolsPanel />);
const restore = await screen.findByTestId('ytdlp-restore');
expect(restore).toHaveTextContent('Restore tested version (2026.06.09)');
fireEvent.click(restore);
await waitFor(() =>
expect(apiFetch).toHaveBeenCalledWith('/media-tools/ytdlp/restore', expect.anything()),
);
});
it('section header offers Update bundled build (one download covers both binaries)', async () => {
render(<AudioToolsPanel />);
fireEvent.click(await screen.findByLabelText('Update bundled build'));
await waitFor(() =>
expect(apiFetch).toHaveBeenCalledWith('/media-tools/acquire', expect.anything()),
);
});
it('package-manager commands are copy-only prose, never buttons', async () => {
render(<AudioToolsPanel />);
await screen.findByText('FFmpeg');
// The InfoHint copy mentions brew/apt as a secondary affordance, but no
// button/control runs a package manager.
const buttons = screen.getAllByRole('button').map((b) => b.textContent || '');
expect(buttons.join(' ')).not.toMatch(/brew|apt|winget|choco/i);
});
});
+19 -35
View File
@@ -1,19 +1,26 @@
import React, { useCallback, useRef } from 'react';
import React, { useCallback } from 'react';
import { toast } from 'react-hot-toast';
import { useTranslation } from 'react-i18next';
import { addBreadcrumb } from '../../utils/breadcrumbs';
import { listEngines, selectEngine } from '../../api/engines';
import { selectEngine } from '../../api/engines';
import { notifyEngineSelected } from '../../utils/engineSelectToast';
import EngineCompatibilityMatrix from '../EngineCompatibilityMatrix';
import { SETTINGS_SECTION_SURFACE } from './primitives';
/** One pinned matrix per family, stacked in this order. ASR used to be
* reachable only through the matrix's family tabs, which read as a
* TTS-only table README even promised a Settings ASR picker that
* didn't exist (UX gap found during #877). Every family now gets a
* visible picker; `OMNIVOICE_*_BACKEND` env vars still win over any pick. */
const FAMILIES = ['tts', 'asr', 'llm'];
/** Settings → Engines: ONE section, one matrix, a TTS / ASR / LLM tab strip.
*
* The page used to stack three pinned per-family matrices; with every row
* free to grow (wrapping names, stacked badges, inline failure prose) a
* single engine could fill a viewport and the ASR/LLM pickers lived below
* the fold. The matrix's family tab strip (Radix Segmented roving
* tabindex + arrow keys, active engine named in each tab caption) now
* presents one family at a time instead, over compact fixed-height rows.
*
* Data contract is unchanged: the single mounted matrix issues exactly one
* GET /engines + one GET /model/loaded per Settings open (switching tabs
* re-slices the same payload no refetch), `openSettingsTab('engines')`
* still lands here, and `OMNIVOICE_*_BACKEND` env vars still win over any
* pick made in the UI. */
export default function EnginesTab() {
const { t } = useTranslation();
@@ -40,32 +47,9 @@ export default function EnginesTab() {
[t],
);
// The stacked matrices all consume the same GET /engines payload share
// one in-flight request so opening the tab probes every engine once, not
// once per family. A per-matrix Refresh after the shared promise settles
// still triggers a fresh fetch.
const inflightList = useRef(null);
const listEnginesShared = useCallback(() => {
if (!inflightList.current) {
inflightList.current = listEngines().finally(() => {
inflightList.current = null;
});
}
return inflightList.current;
}, []);
return (
<>
{FAMILIES.map((family) => (
<section key={family} className={SETTINGS_SECTION_SURFACE} data-slot="settings-section">
<EngineCompatibilityMatrix
family={family}
showFamilyTabs={false}
onSelect={onSelect}
apiListEngines={listEnginesShared}
/>
</section>
))}
</>
<section className={SETTINGS_SECTION_SURFACE} data-slot="settings-section">
<EngineCompatibilityMatrix family="tts" onSelect={onSelect} />
</section>
);
}
@@ -15,7 +15,15 @@ vi.mock('../../api/engines', () => ({
selfTestEngine: vi.fn(),
}));
// Residency layer (/model/loaded) mocked so the matrix never hits the
// network in tests; the single-probe behavior is asserted below.
vi.mock('../../api/system', () => ({
listLoadedModels: vi.fn(),
unloadLoadedModel: vi.fn(),
}));
import { listEngines, selectEngine } from '../../api/engines';
import { listLoadedModels } from '../../api/system';
import EnginesTab from './EnginesTab';
function entry(id, name) {
@@ -43,31 +51,60 @@ const ENGINES = {
llm: { active: 'off', backends: [entry('off', 'Off (test)')] },
};
/** Click the family tab whose label text is `label` (TTS / ASR / LLM). */
function clickFamilyTab(label) {
const tab = Array.from(document.querySelectorAll('.engine-matrix__tab-family')).find(
(el) => el.textContent === label,
);
expect(tab).toBeTruthy();
fireEvent.click(tab.closest('button'));
}
describe('EnginesTab', () => {
beforeEach(() => {
vi.clearAllMocks();
listEngines.mockResolvedValue(ENGINES);
listLoadedModels.mockResolvedValue({ models: [], count: 0 });
});
it('renders a pinned picker per family — TTS, ASR and LLM all visible at once', async () => {
it('renders ONE tabbed section — TTS/ASR/LLM tab strip, one family at a time', async () => {
render(<EnginesTab />);
await waitFor(() => screen.getByText('WhisperX (test)'));
await waitFor(() => screen.getByText('OmniVoice (test)'));
// One named section per family (the ASR picker used to be tucked behind
// a family tab inside a single TTS-titled matrix no picker to find).
expect(screen.getByText('TTS Engines')).toBeInTheDocument();
expect(screen.getByText('ASR Engines')).toBeInTheDocument();
expect(screen.getByText('LLM Engines')).toBeInTheDocument();
// Pinned matrices render no family switcher.
expect(document.querySelector('.engine-matrix__tab-family')).toBeNull();
// One settings card, not three stacked per-family matrices.
expect(document.querySelectorAll('[data-slot="settings-section"]').length).toBe(1);
// The tab strip offers all three families (with the active engine caption).
expect(document.querySelectorAll('.engine-matrix__tab-family').length).toBe(3);
// Only the selected family's engines are on screen.
expect(screen.queryByText('WhisperX (test)')).not.toBeInTheDocument();
expect(screen.queryByText('Off (test)')).not.toBeInTheDocument();
});
it('the stacked matrices share one GET /engines on mount', async () => {
it('switching to the ASR tab shows ASR engines without refetching /engines', async () => {
render(<EnginesTab />);
await waitFor(() => screen.getByText('OmniVoice (test)'));
clickFamilyTab('ASR');
await waitFor(() => screen.getByText('WhisperX (test)'));
expect(screen.getByText('OpenAI-compatible ASR (test)')).toBeInTheDocument();
expect(screen.queryByText('OmniVoice (test)')).not.toBeInTheDocument();
// Tab switches re-slice the already-fetched payload no second request.
expect(listEngines).toHaveBeenCalledTimes(1);
});
it('fetches GET /engines exactly once on mount', async () => {
render(<EnginesTab />);
await waitFor(() => screen.getByText('OmniVoice (test)'));
expect(listEngines).toHaveBeenCalledTimes(1);
});
it('probes GET /model/loaded exactly once on mount', async () => {
render(<EnginesTab />);
await waitFor(() => screen.getByText('OmniVoice (test)'));
await waitFor(() => expect(listLoadedModels).toHaveBeenCalled());
expect(listLoadedModels).toHaveBeenCalledTimes(1);
});
it('clicking Use on an ASR engine selects it with family="asr"', async () => {
selectEngine.mockResolvedValue({
family: 'asr',
@@ -78,6 +115,9 @@ describe('EnginesTab', () => {
routing_reason: null,
});
render(<EnginesTab />);
await waitFor(() => screen.getByText('OmniVoice (test)'));
clickFamilyTab('ASR');
await waitFor(() => screen.getByText('OpenAI-compatible ASR (test)'));
fireEvent.click(screen.getByRole('button', { name: /use openai-compatible asr \(test\)/i }));
@@ -56,8 +56,14 @@ export default function GeneralTab() {
value={reviewMode}
onChange={setReviewMode}
items={[
{ value: 'on', label: t('engines.review_on') },
{ value: 'off', label: t('engines.review_off') },
{
value: 'on',
label: t('settings.review_mode_on', { defaultValue: 'Pause for review' }),
},
{
value: 'off',
label: t('settings.review_mode_off', { defaultValue: 'Run straight through' }),
},
]}
/>
}
@@ -10,13 +10,17 @@
* PUT /api/settings/hf-mirror body {url} (empty url clears official)
*/
import React, { useCallback, useEffect, useState } from 'react';
import { Globe } from 'lucide-react';
import { Globe, RefreshCw } from 'lucide-react';
import toast from 'react-hot-toast';
import { useTranslation } from 'react-i18next';
import { apiJson, apiFetch } from '../../api/client';
import { SettingsSection, SettingRow, SettingsInput } from './primitives';
import { Button } from '../../ui';
import RestartBadge from './RestartBadge';
/** Normalize a mirror URL for equality checks (trailing slashes, whitespace). */
const normalizeMirror = (u) => (u || '').trim().replace(/\/+$/, '');
export default function HFMirrorPanel() {
const { t } = useTranslation();
const [state, setState] = useState(null);
@@ -52,6 +56,7 @@ export default function HFMirrorPanel() {
const d = await res.json();
setUrl(d.configured || '');
setRestart(Boolean(d.restart_required));
toast.success(t('models.mirror_saved', { defaultValue: 'Mirror setting saved' }));
refresh();
} catch (e) {
setError(e?.message || t('models.mirror_save_error'));
@@ -60,8 +65,10 @@ export default function HFMirrorPanel() {
}
};
if (!state) return null;
const configured = normalizeMirror(state?.configured);
// Always render the section shell: a restricted-network user whose backend
// GET failed is exactly the user who needs this panel never let it vanish.
return (
<SettingsSection
icon={Globe}
@@ -75,54 +82,84 @@ export default function HFMirrorPanel() {
</div>
)}
<SettingRow
stack
title={t('models.mirror_preset_title')}
hint={t('models.mirror_preset_hint')}
control={
<div className="flex flex-wrap items-center gap-[6px] min-w-0 max-w-full">
{state.presets.map((p) => (
<Button
variant="preset"
key={p.label}
onClick={() => save(p.url)}
disabled={saving}
data-testid={`hf-preset-${p.url || 'official'}`}
>
{p.label}
</Button>
))}
</div>
}
/>
{!state && !error && (
<div
data-testid="hf-mirror-loading"
className="py-[var(--space-4)] text-[color:var(--chrome-fg-muted)] text-[length:var(--text-sm)]"
>
{t('common.loading')}
</div>
)}
<SettingRow
stack
title="HF_ENDPOINT"
subtitle={restart ? t('models.mirror_restart_note') : undefined}
control={
<>
<SettingsInput
mono
type="text"
value={url}
onChange={(e) => setUrl(e.target.value)}
placeholder="https://hf-mirror.com"
data-testid="hf-mirror-url"
/>
<Button
variant="subtle"
size="sm"
onClick={() => save(url)}
loading={saving}
disabled={saving}
data-testid="hf-mirror-save"
>
{t('common.save')}
</Button>
</>
}
/>
{!state && error && (
<Button
variant="subtle"
size="sm"
leading={<RefreshCw size={13} aria-hidden="true" />}
onClick={refresh}
data-testid="hf-mirror-retry"
>
{t('models.mirror_retry', { defaultValue: 'Retry' })}
</Button>
)}
{state && (
<>
<SettingRow
stack
title={t('models.mirror_preset_title')}
hint={t('models.mirror_preset_hint')}
control={
<div className="flex flex-wrap items-center gap-[6px] min-w-0 max-w-full">
{state.presets.map((p) => (
<Button
variant="preset"
key={p.label}
active={normalizeMirror(p.url) === configured}
onClick={() => save(p.url)}
disabled={saving}
data-testid={`hf-preset-${p.url || 'official'}`}
>
{p.label}
</Button>
))}
</div>
}
/>
<SettingRow
stack
title={t('models.mirror_custom_url', { defaultValue: 'Custom mirror URL' })}
note={t('models.mirror_custom_url_note', {
defaultValue: 'Sets the HF_ENDPOINT environment variable for Hugging Face downloads.',
})}
subtitle={restart ? t('models.mirror_restart_note') : undefined}
control={
<>
<SettingsInput
mono
type="text"
value={url}
onChange={(e) => setUrl(e.target.value)}
placeholder="https://hf-mirror.com"
aria-label={t('models.mirror_custom_url', { defaultValue: 'Custom mirror URL' })}
data-testid="hf-mirror-url"
/>
<Button
variant="subtle"
size="sm"
onClick={() => save(url)}
loading={saving}
disabled={saving}
data-testid="hf-mirror-save"
>
{t('common.save')}
</Button>
</>
}
/>
</>
)}
</SettingsSection>
);
}
@@ -0,0 +1,89 @@
import React from 'react';
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { fireEvent, render, screen, waitFor } from '@testing-library/react';
vi.mock('react-hot-toast', () => ({
default: { error: vi.fn(), success: vi.fn() },
}));
vi.mock('../../api/client', () => ({
apiJson: vi.fn(),
apiFetch: vi.fn(),
}));
import toast from 'react-hot-toast';
import { apiJson, apiFetch } from '../../api/client';
import HFMirrorPanel from './HFMirrorPanel';
const STATE = {
configured: 'https://hf-mirror.com',
effective: 'https://hf-mirror.com',
presets: [
{ label: 'Official (huggingface.co)', url: '' },
{ label: 'hf-mirror.com (community, China)', url: 'https://hf-mirror.com' },
],
};
describe('HFMirrorPanel', () => {
beforeEach(() => {
vi.clearAllMocks();
});
it('keeps the panel visible with an error and a Retry when the initial GET fails', async () => {
// The restricted-network user whose backend GET 500s is exactly the user
// who needs this panel it must never silently vanish.
apiJson.mockRejectedValueOnce(new Error('HTTP 500'));
render(<HFMirrorPanel />);
expect(await screen.findByRole('alert')).toHaveTextContent('HTTP 500');
expect(screen.getByText('Hugging Face mirror')).toBeInTheDocument();
// Retry re-fetches and renders the rows.
apiJson.mockResolvedValueOnce(STATE);
fireEvent.click(screen.getByTestId('hf-mirror-retry'));
expect(await screen.findByTestId('hf-mirror-url')).toBeInTheDocument();
expect(screen.queryByRole('alert')).not.toBeInTheDocument();
});
it('shows a loading state while the GET is in flight (never an empty gap)', () => {
apiJson.mockReturnValue(new Promise(() => {}));
render(<HFMirrorPanel />);
expect(screen.getByText('Hugging Face mirror')).toBeInTheDocument();
expect(screen.getByTestId('hf-mirror-loading')).toBeInTheDocument();
});
it('marks the configured preset as active', async () => {
apiJson.mockResolvedValue(STATE);
render(<HFMirrorPanel />);
const mirror = await screen.findByTestId('hf-preset-https://hf-mirror.com');
const official = screen.getByTestId('hf-preset-official');
expect(mirror).toHaveAttribute('aria-pressed', 'true');
expect(official).toHaveAttribute('aria-pressed', 'false');
});
it('labels the custom-URL row in plain language and toasts on save', async () => {
apiJson.mockResolvedValue(STATE);
apiFetch.mockResolvedValue({
json: async () => ({ configured: 'https://mirror.example', restart_required: true }),
});
render(<HFMirrorPanel />);
// Plain translated label (HF_ENDPOINT is a subtitle detail, not the title),
// and the input carries an accessible name.
const input = await screen.findByLabelText('Custom mirror URL');
expect(screen.getByText('Custom mirror URL')).toBeInTheDocument();
fireEvent.change(input, { target: { value: 'https://mirror.example' } });
fireEvent.click(screen.getByTestId('hf-mirror-save'));
await waitFor(() => expect(toast.success).toHaveBeenCalledWith('Mirror setting saved'));
expect(apiFetch).toHaveBeenCalledWith(
'/api/settings/hf-mirror',
expect.objectContaining({ method: 'PUT' }),
);
});
});
@@ -0,0 +1,154 @@
/**
* Settings Storage Generation history retention (generation takes).
*
* Every synthesis is kept as a "take" (row + WAV in outputs/). Without a cap
* they grow unbounded, so the backend prunes the oldest UNstarred takes over
* this limit after each generation. Starred takes are never pruned. 0 = keep
* everything.
*
* Endpoints:
* GET /api/settings/history-retention {cap, default}
* PUT /api/settings/history-retention body {cap}
*/
import React, { useCallback, useEffect, useState } from 'react';
import { History } from 'lucide-react';
import toast from 'react-hot-toast';
import { useTranslation } from 'react-i18next';
import { apiJson, apiFetch } from '../../api/client';
import { Button } from '../../ui';
import { SettingsSection, SettingRow, InfoHint } from './primitives';
export default function HistoryRetentionPanel() {
const { t } = useTranslation();
const [cap, setCap] = useState('');
const [def, setDef] = useState(200);
const [loading, setLoading] = useState(true);
const [loaded, setLoaded] = useState(false);
const [saving, setSaving] = useState(false);
const [error, setError] = useState(null);
const refresh = useCallback(async () => {
setLoading(true);
setError(null);
try {
const d = await apiJson('/api/settings/history-retention');
setCap(String(d?.cap ?? ''));
if (Number.isInteger(d?.default)) setDef(d.default);
setLoaded(true);
} catch (e) {
if (e?.status === 404) {
// Backend older than this panel leave the default hint in place.
setLoaded(true);
} else {
// Transport failure / 500: the shown default may not be the real cap,
// so say so and hold Save until a load succeeds.
setError(
e?.message ||
t('settings.history_retention_load_failed', {
defaultValue: 'Could not load the current retention limit',
}),
);
}
} finally {
setLoading(false);
}
}, [t]);
useEffect(() => {
refresh();
}, [refresh]);
const save = async () => {
const n = Number.parseInt(cap, 10);
if (!Number.isInteger(n) || n < 0) {
toast.error(t('settings.history_retention_invalid', { defaultValue: 'Enter 0 or more' }));
return;
}
setSaving(true);
try {
// apiFetch throws ApiError on any non-OK response.
const res = await apiFetch('/api/settings/history-retention', {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ cap: n }),
});
const b = await res.json();
setCap(String(b?.cap ?? n));
toast.success(
t('settings.history_retention_saved', { defaultValue: 'Retention limit saved' }),
);
} catch (e) {
toast.error(
e?.message ||
t('settings.history_retention_save_failed', { defaultValue: 'Could not save' }),
);
} finally {
setSaving(false);
}
};
return (
<SettingsSection
icon={History}
title={t('settings.history_retention', { defaultValue: 'Generation history' })}
description={t('settings.history_retention_desc', {
defaultValue: 'How many takes to keep before the oldest are cleaned up.',
})}
actions={
<InfoHint label={t('settings.history_retention', { defaultValue: 'Generation history' })}>
{t('settings.history_retention_help', {
defaultValue:
'After each generation, the oldest unstarred takes over this limit are removed along with their audio files. Starred takes are always kept. Set 0 to keep everything.',
})}
</InfoHint>
}
>
{error && (
<div className="perfpanel__error" role="alert">
{error}
</div>
)}
<SettingRow
title={t('settings.history_retention_cap', { defaultValue: 'Takes to keep' })}
subtitle={t('settings.history_retention_cap_hint', {
defaultValue: 'Starred takes never count against cleanup · 0 = unlimited',
count: def,
})}
control={
<div className="flex items-center gap-[var(--space-3)]">
<input
className="box-border w-[110px] rounded-[var(--chrome-radius-pill)] [border:1px_solid_var(--chrome-border)] bg-[var(--chrome-input-bg)] px-[var(--space-3)] py-[var(--space-2)] font-[family-name:var(--chrome-font-mono)] text-[length:var(--text-base)] text-[var(--chrome-fg)] focus-visible:border-[var(--chrome-accent)] focus-visible:shadow-[var(--focus-ring)] focus-visible:outline-none"
type="number"
min="0"
step="1"
value={cap}
placeholder={String(def)}
onChange={(e) => setCap(e.target.value)}
onKeyDown={(e) => {
if (e.key === 'Enter' && !saving && !loading && loaded) {
e.preventDefault();
save();
}
}}
disabled={saving || loading}
aria-label={t('settings.history_retention_cap', { defaultValue: 'Takes to keep' })}
data-testid="history-retention-input"
/>
<Button
variant="primary"
size="sm"
onClick={save}
loading={saving}
disabled={loading || !loaded}
data-testid="history-retention-save"
>
{saving
? t('common.saving', { defaultValue: 'Saving…' })
: t('common.save', { defaultValue: 'Save' })}
</Button>
</div>
}
/>
</SettingsSection>
);
}
@@ -0,0 +1,103 @@
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { fireEvent, render, screen, waitFor } from '@testing-library/react';
import React from 'react';
function mockFetchSequence(...responses) {
const fn = vi.fn();
for (const r of responses) {
fn.mockResolvedValueOnce({
ok: r.status >= 200 && r.status < 300,
status: r.status,
json: async () => r.body,
text: async () => JSON.stringify(r.body),
});
}
return fn;
}
import HistoryRetentionPanel from './HistoryRetentionPanel';
describe('HistoryRetentionPanel', () => {
beforeEach(() => {
vi.restoreAllMocks();
});
it('renders the current cap from GET state', async () => {
global.fetch = mockFetchSequence({ status: 200, body: { cap: 200, default: 200 } });
render(<HistoryRetentionPanel />);
await waitFor(() => {
expect(screen.getByTestId('history-retention-input')).toHaveValue(200);
});
});
it('save PUTs the edited cap', async () => {
const fetchMock = mockFetchSequence(
{ status: 200, body: { cap: 200, default: 200 } }, // initial GET
{ status: 200, body: { cap: 50, default: 200 } }, // PUT
);
global.fetch = fetchMock;
render(<HistoryRetentionPanel />);
await waitFor(() => screen.getByTestId('history-retention-input'));
fireEvent.change(screen.getByTestId('history-retention-input'), { target: { value: '50' } });
fireEvent.click(screen.getByTestId('history-retention-save'));
await waitFor(() => {
const put = fetchMock.mock.calls.find(([_u, opts]) => opts && opts.method === 'PUT');
expect(put).toBeTruthy();
expect(put[0]).toMatch(/\/api\/settings\/history-retention$/);
expect(JSON.parse(put[1].body)).toEqual({ cap: 50 });
});
});
it('saves on Enter in the input', async () => {
const fetchMock = mockFetchSequence(
{ status: 200, body: { cap: 200, default: 200 } }, // initial GET
{ status: 200, body: { cap: 75, default: 200 } }, // PUT
);
global.fetch = fetchMock;
render(<HistoryRetentionPanel />);
await waitFor(() => screen.getByTestId('history-retention-input'));
const input = screen.getByTestId('history-retention-input');
fireEvent.change(input, { target: { value: '75' } });
fireEvent.keyDown(input, { key: 'Enter' });
await waitFor(() => {
const put = fetchMock.mock.calls.find(([_u, opts]) => opts && opts.method === 'PUT');
expect(put).toBeTruthy();
expect(JSON.parse(put[1].body)).toEqual({ cap: 75 });
});
});
it('surfaces a load failure (500) and holds Save until a load succeeds', async () => {
global.fetch = mockFetchSequence({ status: 500, body: { detail: 'db locked' } });
render(<HistoryRetentionPanel />);
await waitFor(() => expect(screen.getByRole('alert')).toBeInTheDocument());
expect(screen.getByRole('alert')).toHaveTextContent(/db locked/);
expect(screen.getByTestId('history-retention-save')).toBeDisabled();
});
it('stays silent and usable on a 404 (backend older than the panel)', async () => {
global.fetch = mockFetchSequence({ status: 404, body: { detail: 'Not Found' } });
render(<HistoryRetentionPanel />);
await waitFor(() => expect(screen.getByTestId('history-retention-save')).not.toBeDisabled());
expect(screen.queryByRole('alert')).not.toBeInTheDocument();
// The hardcoded default hint stays in place.
expect(screen.getByTestId('history-retention-input')).toHaveAttribute('placeholder', '200');
});
it('rejects a negative cap client-side without a PUT', async () => {
const fetchMock = mockFetchSequence({ status: 200, body: { cap: 200, default: 200 } });
global.fetch = fetchMock;
render(<HistoryRetentionPanel />);
await waitFor(() => screen.getByTestId('history-retention-input'));
fireEvent.change(screen.getByTestId('history-retention-input'), { target: { value: '-5' } });
fireEvent.click(screen.getByTestId('history-retention-save'));
await waitFor(() => {
expect(fetchMock.mock.calls.filter(([_u, o]) => o && o.method === 'PUT')).toHaveLength(0);
});
});
});
+36 -5
View File
@@ -32,11 +32,21 @@ function keyEventToAccelerator(e) {
return [...mods, key].join('+');
}
// A pure modifier press means the user is still building the chord stay
// quiet. Anything else that fails to produce an accelerator (a bare letter,
// F5, Space) is a real rejection and deserves visible feedback.
function isPureModifierEvent(e) {
return /^(Meta|Control|Alt|Shift|OS)/.test(e.key || '');
}
export default function HotkeyTab() {
const { t } = useTranslation();
const [current, setCurrent] = useState('');
const [recording, setRecording] = useState(false);
const [pending, setPending] = useState('');
// True after a modifier-less press while recording drives the inline
// "add a modifier" feedback instead of listening forever in silence.
const [rejected, setRejected] = useState(false);
const [saving, setSaving] = useState(false);
const tauri = isTauri();
@@ -55,7 +65,9 @@ export default function HotkeyTab() {
}, [tauri]);
// While recording, swallow keystrokes globally and convert the next real
// press into an accelerator string. Escape cancels.
// press into an accelerator string. Escape cancels; losing window focus
// cancels too so a stray click outside doesn't leave a global
// key-swallowing listener armed forever.
useEffect(() => {
if (!recording) return;
const onKeyDown = (e) => {
@@ -64,16 +76,28 @@ export default function HotkeyTab() {
if (e.key === 'Escape') {
setRecording(false);
setPending('');
setRejected(false);
return;
}
const accel = keyEventToAccelerator(e);
if (accel) {
setPending(accel);
setRecording(false);
setRejected(false);
return;
}
if (!isPureModifierEvent(e)) setRejected(true);
};
const onBlur = () => {
setRecording(false);
setRejected(false);
};
window.addEventListener('keydown', onKeyDown, true);
return () => window.removeEventListener('keydown', onKeyDown, true);
window.addEventListener('blur', onBlur);
return () => {
window.removeEventListener('keydown', onKeyDown, true);
window.removeEventListener('blur', onBlur);
};
}, [recording]);
const save = async () => {
@@ -123,7 +147,11 @@ export default function HotkeyTab() {
<SettingRow
title={recording ? t('capture.press_key') : t('capture.new_shortcut')}
hint={<Trans i18nKey="capture.desc_detail" components={{ 1: <code />, 2: <code /> }} />}
control={recording ? t('capture.listening') : pending || '—'}
control={
recording
? (rejected && t('capture.needs_modifier')) || t('capture.listening')
: pending || '—'
}
mono
/>
@@ -132,13 +160,16 @@ export default function HotkeyTab() {
size="sm"
variant="subtle"
onClick={() => {
// Toggle: while recording, the same button cancels (Esc still
// works too) re-clicking must not silently re-arm the recorder.
setPending('');
setRecording(true);
setRejected(false);
setRecording(!recording);
}}
disabled={!tauri || saving}
leading={<Keyboard size={12} />}
>
{recording ? t('capture.recording') : t('capture.record_shortcut')}
{recording ? t('common.cancel') : t('capture.record_shortcut')}
</Button>
<Button
size="sm"
@@ -0,0 +1,70 @@
import { describe, it, expect, beforeEach, vi } from 'vitest';
import { fireEvent, render, screen } from '@testing-library/react';
import React from 'react';
import HotkeyTab from './HotkeyTab';
// Recording is only armed in the desktop shell; pretend we are in it and
// stub the two shortcut IPC commands.
vi.mock('./native', () => ({ isTauri: () => true }));
vi.mock('@tauri-apps/api/core', () => ({
invoke: vi.fn(async (cmd) => (cmd === 'get_dictation_shortcut' ? 'CmdOrCtrl+Shift+Space' : '')),
}));
async function startRecording() {
render(<HotkeyTab />);
// Wait for the mount-time shortcut load so state updates stay inside act().
await screen.findByText('CmdOrCtrl+Shift+Space');
fireEvent.click(screen.getByRole('button', { name: 'Record shortcut' }));
expect(screen.getByText(/listening/)).toBeInTheDocument();
}
describe('HotkeyTab — recording feedback and cancel affordances', () => {
beforeEach(() => {
vi.clearAllMocks();
});
it('a modifier-less key press shows "add a modifier" feedback instead of silence', async () => {
await startRecording();
fireEvent.keyDown(window, { key: 'a', code: 'KeyA' });
expect(screen.getByText(/Add a modifier/)).toBeInTheDocument();
// Still recording the button stays in its cancel state.
expect(screen.getByRole('button', { name: 'Cancel' })).toBeInTheDocument();
});
it('a pure modifier press (chord in progress) does NOT trigger the rejection message', async () => {
await startRecording();
fireEvent.keyDown(window, { key: 'Control', code: 'ControlLeft', ctrlKey: true });
expect(screen.queryByText(/Add a modifier/)).toBeNull();
expect(screen.getByText(/listening/)).toBeInTheDocument();
});
it('a modifier+key press captures the accelerator and clears the rejection state', async () => {
await startRecording();
fireEvent.keyDown(window, { key: 'a', code: 'KeyA' }); // rejected first
fireEvent.keyDown(window, { key: 'a', code: 'KeyA', ctrlKey: true });
expect(screen.getByText('Ctrl+A')).toBeInTheDocument();
expect(screen.queryByText(/Add a modifier/)).toBeNull();
expect(screen.getByRole('button', { name: 'Record shortcut' })).toBeInTheDocument();
});
it('clicking the record button while recording cancels instead of re-arming', async () => {
await startRecording();
fireEvent.click(screen.getByRole('button', { name: 'Cancel' }));
expect(screen.queryByText(/listening/)).toBeNull();
expect(screen.getByRole('button', { name: 'Record shortcut' })).toBeInTheDocument();
});
it('losing window focus cancels recording (no global key-swallower left armed)', async () => {
await startRecording();
fireEvent(window, new Event('blur'));
expect(screen.queryByText(/listening/)).toBeNull();
expect(screen.getByRole('button', { name: 'Record shortcut' })).toBeInTheDocument();
});
it('Escape cancels recording', async () => {
await startRecording();
fireEvent.keyDown(window, { key: 'Escape', code: 'Escape' });
expect(screen.queryByText(/listening/)).toBeNull();
});
});
@@ -115,8 +115,11 @@ export default function LLMProvidersPanel() {
populate(providers, id);
};
// Returns true when the PUT (and refresh) succeeded Test / Fetch models
// gate on it so they never probe the previously-stored config after a
// failed save (which could show a green "Test ok" beside a save error).
const save = async (makeActive) => {
if (!current) return;
if (!current) return false;
setSaving(true);
setError(null);
try {
@@ -138,8 +141,10 @@ export default function LLMProvidersPanel() {
// pins the choice the env banner already explains and the suggested
// button is disabled.
setSavedInactive(Boolean(data) && data.active !== current.id && !current.active_from_env);
return true;
} catch (e) {
setError(e?.message || t('settings.llmp_save_failed'));
return false;
} finally {
setSaving(false);
}
@@ -151,8 +156,10 @@ export default function LLMProvidersPanel() {
setTest(null);
setError(null);
try {
// Save first so the probe sees the just-typed key/URL.
await save(false);
// Save first so the probe sees the just-typed key/URL. If the save
// failed, stop: probing the stale stored config would contradict the
// save error with a misleading green badge.
if (!(await save(false))) return;
const res = await apiPost(`/api/settings/llm-providers/${current.id}/test`);
setTest(res);
} catch (e) {
@@ -167,8 +174,9 @@ export default function LLMProvidersPanel() {
setLoadingModels(true);
setError(null);
try {
// Save non-key fields first so the probe uses the just-typed base URL.
await save(false);
// Save non-key fields first so the probe uses the just-typed base URL;
// abort on a failed save (same stale-config trap as runTest).
if (!(await save(false))) return;
const res = await apiJson(`/api/settings/llm-providers/${current.id}/models`);
if (res.ok) {
setModels(res.models || []);
@@ -187,6 +195,8 @@ export default function LLMProvidersPanel() {
};
if (!providers.length) {
// A failed initial GET used to dead-end here (nothing re-runs refresh
// without a remount) the Retry button is the way back in.
return (
<SettingsSection
icon={Brain}
@@ -195,7 +205,15 @@ export default function LLMProvidersPanel() {
>
{error && (
<div className="perfpanel__error" role="alert">
{error}
<span className="mr-[8px]">{error}</span>
<Button
variant="subtle"
size="sm"
onClick={() => refresh()}
data-testid="llm-provider-retry"
>
{t('settings.retry', { defaultValue: 'Retry' })}
</Button>
</div>
)}
</SettingsSection>
@@ -200,6 +200,53 @@ describe('LLMProvidersPanel', () => {
expect(screen.getByText(/not yet used for translation/)).toBeInTheDocument();
});
it('a failed implicit save aborts Test — no probe against the stale stored config', async () => {
const fetchMock = mockFetchSequence(
{ body: PROVIDERS }, // mount GET
{ status: 500, body: { detail: 'disk full' } }, // save PUT fails
// nothing else queued: the /test POST must never fire
);
global.fetch = fetchMock;
render(<LLMProvidersPanel />);
fireEvent.click(await screen.findByTestId('llm-provider-test'));
// The save error is the single message
await waitFor(() => expect(screen.getByRole('alert')).toHaveTextContent(/disk full/));
// with no contradictory green "Test ok" badge and no /test round-trip.
expect(screen.queryByText(/ok —/)).toBeNull();
expect(fetchMock.mock.calls.some(([url]) => String(url).includes('/test'))).toBe(false);
});
it('a failed implicit save aborts Fetch models', async () => {
const fetchMock = mockFetchSequence(
{ body: PROVIDERS }, // mount GET
{ status: 500, body: { detail: 'disk full' } }, // save PUT fails
);
global.fetch = fetchMock;
render(<LLMProvidersPanel />);
fireEvent.click(await screen.findByTestId('llm-provider-models'));
await waitFor(() => expect(screen.getByRole('alert')).toHaveTextContent(/disk full/));
expect(fetchMock.mock.calls.some(([url]) => String(url).includes('/models'))).toBe(false);
});
it('initial-load failure offers a Retry that refetches (no remount needed)', async () => {
const fetchMock = mockFetchSequence(
{ status: 500, body: { detail: 'backend hiccup' } }, // mount GET fails
{ body: PROVIDERS }, // retry GET succeeds
);
global.fetch = fetchMock;
render(<LLMProvidersPanel />);
const retry = await screen.findByTestId('llm-provider-retry');
expect(screen.getByRole('alert')).toHaveTextContent(/backend hiccup/);
fireEvent.click(retry);
const select = await screen.findByTestId('llm-provider-select');
await waitFor(() => expect(select.value).toBe('groq'));
expect(screen.queryByTestId('llm-provider-retry')).toBeNull();
});
it('no notice when the saved provider IS the active one', async () => {
global.fetch = mockFetchSequence(
{ body: PROVIDERS }, // mount GET (active: groq)
@@ -136,6 +136,10 @@ export default function LLMSkillsPanel() {
value={skill.provider_override || ''}
onChange={(e) => update(skill.id, { provider_override: e.target.value })}
disabled={!skill.enabled || busy === skill.id}
aria-label={t('settings.llmskills_route_for', {
defaultValue: 'Provider for {{skill}}',
skill: t(skill.name_key),
})}
data-testid={`llm-skill-provider-${skill.id}`}
>
<option value="">{t('settings.llmskills_use_active')}</option>
@@ -121,6 +121,14 @@ describe('LLMSkillsPanel', () => {
expect(put.mock.calls[0][1]).toEqual({ provider_override: 'ollama' });
});
it('the per-skill routing Select carries an accessible name', async () => {
global.fetch = mockFetch(routes);
render(<LLMSkillsPanel />);
const select = await screen.findByTestId('llm-skill-provider-cinematic_translation');
// Announced as "Provider for <skill>" not an unlabeled combobox.
expect(select).toHaveAccessibleName('Provider for Cinematic & Autofit translation');
});
it('shows the needs-setup badge + LLM Providers link when no provider resolves', async () => {
const unready = {
skills: SKILLS.skills.map((s) => ({
+67 -3
View File
@@ -1,6 +1,9 @@
import React from 'react';
import { FileText, RefreshCw, Trash2, AlertCircle } from 'lucide-react';
import React, { useEffect, useRef } from 'react';
import { Copy, FileText, FolderOpen, RefreshCw, Trash2, AlertCircle } from 'lucide-react';
import toast from 'react-hot-toast';
import { useTranslation } from 'react-i18next';
import { exportReveal } from '../../api/exports';
import { copyText } from '../../utils/copyText';
import { Segmented, Button, Badge } from '../../ui';
import { SettingsSection } from './primitives';
import ReportBugButton from '../ReportBugButton';
@@ -21,6 +24,37 @@ export default function LogsTab({
onClearLogs,
}) {
const { t } = useTranslation();
const scrollRef = useRef(null);
// Fresh log loads land scrolled to the newest entries the tail is the
// whole point of checking logs; without this the viewer opens at the oldest
// line of the tailed window on every refresh.
useEffect(() => {
const el = scrollRef.current;
if (el) el.scrollTop = el.scrollHeight;
}, [logs]);
// The frontend "log" is an in-memory buffer there is no file to reveal.
const hasLogFile = logSource !== 'frontend' && !!logMeta.exists && !!logMeta.path;
const openLogFolder = async () => {
try {
await exportReveal({ path: logMeta.path });
} catch (e) {
toast.error(
e?.message || t('settings.open_folder_failed', { defaultValue: 'Could not open folder' }),
);
}
};
const copyLogs = async () => {
const ok = await copyText(logs.join(''));
if (ok) {
toast.success(t('logs.log_copied', { source: t(`common.${logSource}`) }));
} else {
toast.error(t('logs.copy_failed_short', { defaultValue: 'Could not copy the log' }));
}
};
return (
<SettingsSection
@@ -29,6 +63,16 @@ export default function LogsTab({
actions={
<>
<ReportBugButton />
<Button
variant="subtle"
size="sm"
onClick={copyLogs}
disabled={logs.length === 0}
leading={<Copy size={11} />}
data-testid="logs-copy"
>
{t('logs.copy_visible', { defaultValue: 'Copy visible log' })}
</Button>
<Button
variant="subtle"
size="sm"
@@ -48,17 +92,37 @@ export default function LogsTab({
items={LOG_SOURCE_DEFS.map((d) => ({ ...d, label: t(`common.${d.key}`) }))}
value={logSource}
onChange={setLogSource}
aria-label={t('logs.source', { defaultValue: 'Log source' })}
/>
<div className="settings-log-meta flex items-center gap-[var(--space-4)] my-[var(--space-4)] font-mono text-[var(--text-base)] text-[var(--chrome-fg-dim)]">
<span>{logMeta.path || '—'}</span>
{hasLogFile && (
<Button
variant="ghost"
size="sm"
onClick={openLogFolder}
leading={<FolderOpen size={11} />}
title={logMeta.path}
data-testid="logs-open-folder"
>
{t('settings.storage_open_folder', { defaultValue: 'Open folder' })}
</Button>
)}
{logSource === 'tauri' && !logMeta.exists && (
<Badge tone="warn">
<AlertCircle size={11} /> {t('logs.no_tauri_log')}
</Badge>
)}
</div>
<div className="bg-[var(--chrome-bg)] [border:1px_solid_var(--chrome-border)] rounded-[var(--chrome-radius-pill)] px-[12px] py-[10px] max-h-[280px] overflow-auto font-mono text-[0.72rem] text-[var(--chrome-fg-muted)] whitespace-pre-wrap break-words">
<div
ref={scrollRef}
tabIndex={0}
role="log"
aria-label={t('settings.logs')}
data-testid="logs-scroll"
className="bg-[var(--chrome-bg)] [border:1px_solid_var(--chrome-border)] rounded-[var(--chrome-radius-pill)] px-[12px] py-[10px] max-h-[280px] overflow-auto font-mono text-[0.72rem] text-[var(--chrome-fg-muted)] whitespace-pre-wrap break-words focus-visible:outline-none focus-visible:border-[var(--chrome-accent)] focus-visible:shadow-[var(--focus-ring)]"
>
{logs.length === 0 ? (
<span className="settings-log__empty font-sans text-[var(--chrome-fg-dim)]">
{logSource === 'frontend'
@@ -0,0 +1,85 @@
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { fireEvent, render, screen, waitFor } from '@testing-library/react';
import React from 'react';
import LogsTab from './LogsTab';
const LINES = ['[10:00:00] boot\n', '[10:00:01] ready\n'];
function renderTab(overrides = {}) {
const props = {
logSource: 'backend',
setLogSource: vi.fn(),
logs: LINES,
logMeta: { path: '/home/u/.omnivoice/omnivoice.log', exists: true },
loadingLogs: false,
refreshLogs: vi.fn(),
onClearLogs: vi.fn(),
...overrides,
};
return { ...render(<LogsTab {...props} />), props };
}
describe('LogsTab', () => {
beforeEach(() => {
vi.restoreAllMocks();
});
it('offers Open folder for on-disk logs and reveals via /export/reveal', async () => {
const fetchMock = vi.fn().mockResolvedValue({
ok: true,
status: 200,
json: async () => ({ success: true }),
text: async () => '{"success":true}',
});
global.fetch = fetchMock;
renderTab();
fireEvent.click(screen.getByTestId('logs-open-folder'));
await waitFor(() => {
const call = fetchMock.mock.calls.find(([u]) => u.endsWith('/export/reveal'));
expect(call).toBeTruthy();
expect(JSON.parse(call[1].body)).toEqual({ path: '/home/u/.omnivoice/omnivoice.log' });
});
});
it('hides Open folder for the in-memory frontend buffer and missing files', () => {
renderTab({ logSource: 'frontend', logMeta: { path: 'in-memory (last 500)', exists: true } });
expect(screen.queryByTestId('logs-open-folder')).not.toBeInTheDocument();
renderTab({ logSource: 'tauri', logMeta: { path: '—', exists: false } });
expect(screen.queryByTestId('logs-open-folder')).not.toBeInTheDocument();
});
it('copies the visible tail to the clipboard', async () => {
const writeText = vi.fn().mockResolvedValue(undefined);
Object.defineProperty(navigator, 'clipboard', { value: { writeText }, configurable: true });
renderTab();
fireEvent.click(screen.getByTestId('logs-copy'));
await waitFor(() => expect(writeText).toHaveBeenCalledWith(LINES.join('')));
Object.defineProperty(navigator, 'clipboard', { value: undefined, configurable: true });
});
it('disables Copy when there is nothing to copy', () => {
renderTab({ logs: [] });
expect(screen.getByTestId('logs-copy')).toBeDisabled();
});
it('log viewport is keyboard-reachable and labelled', () => {
renderTab();
const box = screen.getByTestId('logs-scroll');
expect(box).toHaveAttribute('tabindex', '0');
expect(box).toHaveAttribute('role', 'log');
expect(box).toHaveAccessibleName('Logs');
});
it('scrolls to the newest entries when logs load', () => {
const { rerender, props } = renderTab({ logs: [] });
const box = screen.getByTestId('logs-scroll');
Object.defineProperty(box, 'scrollHeight', { value: 640, configurable: true });
rerender(<LogsTab {...props} logs={LINES} />);
expect(box.scrollTop).toBe(640);
});
});
@@ -9,19 +9,31 @@
* GET /api/mcp/bindings
* PUT /api/mcp/bindings {client_id, label?, profile_id?, default_engine?}
* DELETE /api/mcp/bindings/{client_id}
*
* The API's `default_engine` field is intentionally NOT editable here it is
* an MCP-side capability (agents can request an engine per docs/mcp.md); the
* panel only manages the voice routing a user actually reasons about.
*/
import React, { useCallback, useEffect, useState } from 'react';
import { useTranslation } from 'react-i18next';
import { Bot, Trash2 } from 'lucide-react';
import { apiJson, apiFetch } from '../../api/client';
import { listProfiles } from '../../api/profiles';
import { SettingsSection, SettingRow, SettingsInput } from './primitives';
import { askConfirm } from './native';
import { SettingsSection, SettingRow, SettingsInput, InfoHint } from './primitives';
import { Button, Badge, Select } from '../../ui';
const MCP_DOCS_URL = 'https://github.com/debpalash/OmniVoice-Studio/blob/main/docs/mcp.md';
export default function MCPBindingsPanel() {
const { t } = useTranslation();
const [bindings, setBindings] = useState([]);
const [profiles, setProfiles] = useState([]);
const [clientId, setClientId] = useState('');
const [label, setLabel] = useState('');
const [profileId, setProfileId] = useState('');
const [adding, setAdding] = useState(false);
const [deletingId, setDeletingId] = useState(null);
const [error, setError] = useState(null);
const refresh = useCallback(async () => {
@@ -31,47 +43,92 @@ export default function MCPBindingsPanel() {
setBindings(b);
setProfiles(p);
} catch (e) {
setError(e?.message || 'Failed to load MCP bindings');
setError(
e?.message ||
t('settings.mcp_load_failed', { defaultValue: 'Failed to load MCP bindings' }),
);
}
}, []);
}, [t]);
useEffect(() => {
refresh();
}, [refresh]);
const profileName = (id) => profiles.find((p) => p.id === id)?.name || id || '—';
const profileName = (id) =>
profiles.find((p) => p.id === id)?.name ||
id ||
t('settings.mcp_default_voice', { defaultValue: 'Default voice' });
const onAdd = async () => {
if (!clientId.trim()) return;
if (!clientId.trim() || adding) return;
setAdding(true);
setError(null);
try {
await apiFetch('/api/mcp/bindings', {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ client_id: clientId.trim(), profile_id: profileId || null }),
body: JSON.stringify({
client_id: clientId.trim(),
label: label.trim() || null,
profile_id: profileId || null,
}),
});
setClientId('');
setLabel('');
setProfileId('');
refresh();
await refresh();
} catch (e) {
setError(e?.message || 'Failed to save binding');
setError(
e?.message || t('settings.mcp_save_failed', { defaultValue: 'Failed to save binding' }),
);
} finally {
setAdding(false);
}
};
const onDelete = async (cid) => {
if (deletingId) return;
const confirmed = await askConfirm(
t('settings.mcp_delete_confirm', {
defaultValue: 'Remove the voice binding for “{{clientId}}”?',
clientId: cid,
}),
t('settings.mcp_delete_confirm_title', { defaultValue: 'Remove binding' }),
);
if (!confirmed) return;
setDeletingId(cid);
setError(null);
let failure = null;
try {
await apiFetch(`/api/mcp/bindings/${encodeURIComponent(cid)}`, { method: 'DELETE' });
refresh();
} catch (e) {
setError(e?.message || 'Failed to delete binding');
failure =
e?.message || t('settings.mcp_delete_failed', { defaultValue: 'Failed to delete binding' });
}
// Re-sync even on failure: a 404 means the row was already gone the list
// must not keep showing it. refresh() clears error state, so re-apply the
// delete failure afterwards.
await refresh();
if (failure) setError(failure);
setDeletingId(null);
};
return (
<SettingsSection
icon={Bot}
title="MCP voice bindings"
description="Bind an agent's client id to a voice profile."
title={t('settings.mcp_title', { defaultValue: 'MCP voice bindings' })}
description={t('settings.mcp_desc', {
defaultValue:
'Give each MCP agent its own voice — bind the client id an agent sends to a voice profile.',
})}
actions={
<InfoHint learnMoreHref={MCP_DOCS_URL}>
{t('settings.mcp_hint', {
defaultValue:
'Agents reach OmniVoice at /mcp and identify themselves with a client id (e.g. claude-code). Bind that id to a voice so the agent always speaks in that profile.',
})}
</InfoHint>
}
>
{error && (
<div className="perfpanel__error" role="alert">
@@ -79,16 +136,22 @@ export default function MCPBindingsPanel() {
</div>
)}
{bindings.length === 0 && !error && (
<p
className="m-0 py-[var(--space-3)] text-[length:var(--text-xs)] text-[color:var(--chrome-fg-dim)] leading-[1.5]"
data-testid="mcp-empty"
>
{t('settings.mcp_empty', {
defaultValue: "No bindings yet — add an agent's client id below.",
})}
</p>
)}
{bindings.map((b) => (
<SettingRow
key={b.client_id}
title={b.label || b.client_id}
hint={
<>
Agents reach OmniVoice at <code>/mcp</code>. Bind an agent's client id to a voice so
it speaks in that profile. See <code>docs/mcp.md</code>.
</>
}
subtitle={b.label ? b.client_id : undefined}
control={
<>
<Badge tone="neutral">{profileName(b.profile_id)}</Badge>
@@ -96,7 +159,11 @@ export default function MCPBindingsPanel() {
variant="danger"
size="sm"
onClick={() => onDelete(b.client_id)}
aria-label={`Remove ${b.client_id}`}
disabled={deletingId === b.client_id}
aria-label={t('settings.mcp_remove', {
defaultValue: 'Remove {{clientId}}',
clientId: b.client_id,
})}
data-testid={`mcp-del-${b.client_id}`}
>
<Trash2 size={12} />
@@ -107,31 +174,54 @@ export default function MCPBindingsPanel() {
))}
<SettingRow
title="Add binding"
title={t('settings.mcp_add_title', { defaultValue: 'Add binding' })}
stack
control={
<>
<SettingsInput
type="text"
value={clientId}
onChange={(e) => setClientId(e.target.value)}
placeholder="client id (e.g. claude-code)"
placeholder={t('settings.mcp_client_id_placeholder', {
defaultValue: 'Client ID (e.g. claude-code)',
})}
aria-label={t('settings.mcp_client_id', { defaultValue: 'Client ID' })}
data-testid="mcp-client-id"
/>
<SettingsInput
type="text"
value={label}
onChange={(e) => setLabel(e.target.value)}
placeholder={t('settings.mcp_label_placeholder', {
defaultValue: 'Label (optional)',
})}
aria-label={t('settings.mcp_label', { defaultValue: 'Label' })}
data-testid="mcp-label"
/>
<Select
size="sm"
value={profileId}
onChange={(e) => setProfileId(e.target.value)}
aria-label={t('settings.mcp_voice_profile', { defaultValue: 'Voice profile' })}
data-testid="mcp-profile"
>
<option value="">default voice</option>
<option value="">
{t('settings.mcp_default_voice', { defaultValue: 'Default voice' })}
</option>
{profiles.map((p) => (
<option key={p.id} value={p.id}>
{p.name}
</option>
))}
</Select>
<Button variant="subtle" size="sm" onClick={onAdd} data-testid="mcp-add">
Bind
<Button
variant="subtle"
size="sm"
onClick={onAdd}
disabled={!clientId.trim() || adding}
data-testid="mcp-add"
>
{t('settings.mcp_add', { defaultValue: 'Add binding' })}
</Button>
</>
}
@@ -0,0 +1,164 @@
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { fireEvent, render, screen, waitFor } from '@testing-library/react';
import React from 'react';
// Deterministic confirm: tests flip `confirmAnswer` per case (the real
// askConfirm routes through the Tauri dialog plugin / window.confirm).
let confirmAnswer = true;
const askConfirmMock = vi.fn(async () => confirmAnswer);
vi.mock('./native', () => ({
isTauri: () => false,
askConfirm: (...args) => askConfirmMock(...args),
}));
const PROFILES = [
{ id: 'morgan', name: 'Morgan' },
{ id: 'scarlett', name: 'Scarlett' },
];
vi.mock('../../api/profiles', () => ({
listProfiles: vi.fn(async () => PROFILES),
}));
import MCPBindingsPanel from './MCPBindingsPanel';
const BINDINGS = [
{ client_id: 'claude-code', label: 'Claude Code', profile_id: 'morgan' },
{ client_id: 'cursor', label: null, profile_id: null },
];
function mockFetchSequence(...responses) {
const fn = vi.fn();
for (const r of responses) {
fn.mockResolvedValueOnce({
ok: (r.status ?? 200) >= 200 && (r.status ?? 200) < 300,
status: r.status ?? 200,
json: async () => r.body,
text: async () => JSON.stringify(r.body),
});
}
return fn;
}
describe('MCPBindingsPanel', () => {
beforeEach(() => {
vi.clearAllMocks();
confirmAnswer = true;
});
it('renders bindings with label (falling back to client id) and profile badge', async () => {
global.fetch = mockFetchSequence({ body: BINDINGS });
render(<MCPBindingsPanel />);
expect(await screen.findByText('Claude Code')).toBeInTheDocument();
// Unlabelled binding falls back to its client id as the row title.
expect(screen.getByText('cursor')).toBeInTheDocument();
// Profile badge on the bound row ("Morgan" also exists as a select option).
expect(screen.getAllByText('Morgan').some((el) => el.tagName !== 'OPTION')).toBe(true);
});
it('empty list shows the first-run guidance instead of a bare add row', async () => {
global.fetch = mockFetchSequence({ body: [] });
render(<MCPBindingsPanel />);
expect(await screen.findByTestId('mcp-empty')).toHaveTextContent(/No bindings yet/);
});
it('load failure surfaces the error (and no stale empty-state hint)', async () => {
// HTTP 500 (not a transport error) apiFetch never retries HTTP errors,
// so the test stays fast and deterministic.
global.fetch = mockFetchSequence({ status: 500, body: { detail: 'boom' } });
render(<MCPBindingsPanel />);
await waitFor(() => expect(screen.getByRole('alert')).toBeInTheDocument());
expect(screen.queryByTestId('mcp-empty')).toBeNull();
});
it('Add binding PUTs client id + optional label + profile, then refreshes', async () => {
const fetchMock = mockFetchSequence(
{ body: [] }, // mount GET
{ body: { client_id: 'cline', label: 'Cline', profile_id: 'scarlett' } }, // PUT
{ body: [{ client_id: 'cline', label: 'Cline', profile_id: 'scarlett' }] }, // refresh GET
);
global.fetch = fetchMock;
render(<MCPBindingsPanel />);
await screen.findByTestId('mcp-empty');
fireEvent.change(screen.getByTestId('mcp-client-id'), { target: { value: ' cline ' } });
fireEvent.change(screen.getByTestId('mcp-label'), { target: { value: 'Cline' } });
fireEvent.change(screen.getByTestId('mcp-profile'), { target: { value: 'scarlett' } });
fireEvent.click(screen.getByTestId('mcp-add'));
await waitFor(() => {
const put = fetchMock.mock.calls.find(([, opts]) => opts?.method === 'PUT');
expect(put).toBeTruthy();
expect(put[0]).toMatch(/\/api\/mcp\/bindings$/);
expect(JSON.parse(put[1].body)).toEqual({
client_id: 'cline',
label: 'Cline',
profile_id: 'scarlett',
});
});
// Inputs reset after a successful add; the new row renders.
await screen.findByText('Cline');
expect(screen.getByTestId('mcp-client-id').value).toBe('');
});
it('Add button is disabled with an empty client id', async () => {
global.fetch = mockFetchSequence({ body: [] });
render(<MCPBindingsPanel />);
await screen.findByTestId('mcp-empty');
expect(screen.getByTestId('mcp-add')).toBeDisabled();
});
it('delete asks for confirmation and DELETEs on confirm', async () => {
const fetchMock = mockFetchSequence(
{ body: BINDINGS }, // mount GET
{ body: { deleted: 'cursor' } }, // DELETE
{ body: [BINDINGS[0]] }, // refresh GET
);
global.fetch = fetchMock;
render(<MCPBindingsPanel />);
fireEvent.click(await screen.findByTestId('mcp-del-cursor'));
await waitFor(() => {
expect(askConfirmMock).toHaveBeenCalledWith(
expect.stringContaining('cursor'),
expect.any(String),
);
const del = fetchMock.mock.calls.find(([, opts]) => opts?.method === 'DELETE');
expect(del).toBeTruthy();
expect(del[0]).toMatch(/\/api\/mcp\/bindings\/cursor$/);
});
await waitFor(() => expect(screen.queryByTestId('mcp-del-cursor')).toBeNull());
});
it('declining the confirmation sends no DELETE', async () => {
confirmAnswer = false;
const fetchMock = mockFetchSequence({ body: BINDINGS });
global.fetch = fetchMock;
render(<MCPBindingsPanel />);
fireEvent.click(await screen.findByTestId('mcp-del-cursor'));
await waitFor(() => expect(askConfirmMock).toHaveBeenCalled());
expect(fetchMock.mock.calls.find(([, opts]) => opts?.method === 'DELETE')).toBeUndefined();
});
it('a failed delete (already gone: 404) still re-syncs the list', async () => {
const fetchMock = mockFetchSequence(
{ body: BINDINGS }, // mount GET
{ status: 404, body: { detail: 'No binding for that client id' } }, // DELETE fails
{ body: [BINDINGS[0]] }, // refresh GET row is gone server-side
);
global.fetch = fetchMock;
render(<MCPBindingsPanel />);
fireEvent.click(await screen.findByTestId('mcp-del-cursor'));
// The stale row disappears even though the DELETE errored.
await waitFor(() => expect(screen.queryByTestId('mcp-del-cursor')).toBeNull());
});
it('controls carry accessible names', async () => {
global.fetch = mockFetchSequence({ body: BINDINGS });
render(<MCPBindingsPanel />);
await screen.findByText('Claude Code');
expect(screen.getByLabelText('Client ID')).toBeInTheDocument();
expect(screen.getByLabelText('Label')).toBeInTheDocument();
expect(screen.getByLabelText('Voice profile')).toBeInTheDocument();
expect(screen.getByRole('button', { name: 'Remove cursor' })).toBeInTheDocument();
});
});

Some files were not shown because too many files have changed in this diff Show More