Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2a22952f9c | ||
|
|
ff56865cf7 | ||
|
|
9c81e3389d | ||
|
|
3f62471bad | ||
|
|
254f071b45 | ||
|
|
d1522cbba0 | ||
|
|
d3e88c0f3e | ||
|
|
fb0508f4f6 | ||
|
|
855a72038b | ||
|
|
e0a7b2202d | ||
|
|
f7be62207e | ||
|
|
7762bc48bd | ||
|
|
5562aa16a7 | ||
|
|
34c8a33628 | ||
|
|
17e2bb5ed5 | ||
|
|
fbac0de817 | ||
|
|
2e71cc3744 | ||
|
|
e816a2c24e | ||
|
|
fd083749c6 | ||
|
|
b603b9f78d | ||
|
|
2f3e40db24 | ||
|
|
6978f90f16 | ||
|
|
17ae952810 | ||
|
|
55c40be307 | ||
|
|
7f59d5f8fe | ||
|
|
d3ec4ed371 | ||
|
|
a99f1fdff9 | ||
|
|
236c727cd4 | ||
|
|
efc99be337 | ||
|
|
5ce9d0e51d | ||
|
|
7dbb95fa15 | ||
|
|
60b29b4006 | ||
|
|
04d6d0cb7a | ||
|
|
2695ef97ae | ||
|
|
800207ddb5 | ||
|
|
2d073cfaec | ||
|
|
b1e83658f0 | ||
|
|
29b6f30f2b | ||
|
|
72e979f1d9 | ||
|
|
6c6207a132 | ||
|
|
0effe485f4 | ||
|
|
32407bc781 | ||
|
|
df870e9ed3 | ||
|
|
323892b36c | ||
|
|
36ee06c7cc | ||
|
|
9bfd8f9a13 | ||
|
|
cb6c72b409 | ||
|
|
a721c1fdcf | ||
|
|
80f10289fe | ||
|
|
a0ad314736 | ||
|
|
1808a373a1 | ||
|
|
67789fb31c | ||
|
|
da4bef8e42 | ||
|
|
0076d0067e | ||
|
|
faf34348c8 | ||
|
|
69ce697ee5 | ||
|
|
93a3cb260a | ||
|
|
33379890ad | ||
|
|
7f8a42ce51 | ||
|
|
4de3c824d0 | ||
|
|
11fbbd3f6d | ||
|
|
f546f04c8e | ||
|
|
270b3b1c4c | ||
|
|
5a7d9cc05c |
@@ -74,9 +74,10 @@ jobs:
|
||||
- name: Validate install docs against desktop-prod.sh
|
||||
run: python scripts/validate-install-docs.py
|
||||
|
||||
# `backend/tests/` stubs core.config in sys.modules to avoid the heavy
|
||||
# main app import chain — that pollutes import state for other modules,
|
||||
# so it runs in its own pytest session to stay isolated from tests/.
|
||||
# `backend/tests/` mounts routers on bare FastAPI apps (no heavy main
|
||||
# import chain) with a hermetic data dir from its conftest.py. It no
|
||||
# longer stubs sys.modules, so mixed sessions with tests/ are safe;
|
||||
# the separate session is kept for cheaper, clearer CI output.
|
||||
- name: Run pytest (backend/tests, isolated)
|
||||
run: uv run pytest backend/tests/ -q --tb=short
|
||||
|
||||
@@ -149,7 +150,7 @@ jobs:
|
||||
- os: windows-2022
|
||||
label: Windows
|
||||
rust_target: x86_64-pc-windows-msvc
|
||||
- os: ubuntu-22.04
|
||||
- os: ubuntu-24.04
|
||||
label: Linux
|
||||
rust_target: x86_64-unknown-linux-gnu
|
||||
runs-on: ${{ matrix.os }}
|
||||
|
||||
@@ -213,13 +213,24 @@ jobs:
|
||||
bundles: "msi,updater"
|
||||
|
||||
# Linux: ship .AppImage only. AppImage is universal (no distro
|
||||
# package-manager dep), runs on any glibc-2.31+ host, and is the
|
||||
# package-manager dep), runs on any glibc-2.39+ host, and is the
|
||||
# Linux auto-update target. The .deb target was dropped: tauri-bundler
|
||||
# fails it with "Failed to create control scripts: No such file or
|
||||
# directory" (no custom deb config of ours is at fault) — revisit on a
|
||||
# tauri-cli bump. FUSE unavailability on GH runners is handled via
|
||||
# APPIMAGE_EXTRACT_AND_RUN=1.
|
||||
- os: ubuntu-22.04
|
||||
#
|
||||
# Bumped from ubuntu-22.04 → ubuntu-24.04 (#961): the AppImage
|
||||
# bundles whatever `libwebkit2gtk-4.1-dev` the build runner's apt
|
||||
# repos resolve (see the "Linux system deps" step below) — 22.04's
|
||||
# was meaningfully stale relative to what current Ubuntu/Fedora
|
||||
# ship, and AppRun's LD_LIBRARY_PATH makes that bundled, stale copy
|
||||
# take priority over a healthy system WebKitGTK at runtime. Raises
|
||||
# the AppImage's glibc floor from 2.35 to 2.39 — pre-2022 distros
|
||||
# (Ubuntu <22.04, Debian <12) lose support; no report of anyone on
|
||||
# something that old has come in, and the project's own install
|
||||
# docs already assume Debian 12 / Ubuntu 22.04+.
|
||||
- os: ubuntu-24.04
|
||||
arch: x86_64-unknown-linux-gnu
|
||||
label: "Linux x64"
|
||||
rust_target: x86_64-unknown-linux-gnu
|
||||
|
||||
@@ -0,0 +1,205 @@
|
||||
# Adjacent open-source projects — research notes (2026-07-10)
|
||||
|
||||
Owner-requested research on five neighboring projects, read against OmniVoice
|
||||
Studio's current feature-maturity map. Each section ends with what we should
|
||||
take from it. Priorities are consolidated at the bottom.
|
||||
|
||||
| Project | Stars | License | Status | Why it matters to us |
|
||||
|---|---|---|---|---|
|
||||
| [Real-Time-Voice-Cloning](https://github.com/CorentinJ/Real-Time-Voice-Cloning) | ~60k | MIT | Retired (models frozen 2019, maintainer quit 2020) | Positioning/SEO opportunity, cautionary tales |
|
||||
| [VoxCPM](https://github.com/OpenBMB/VoxCPM) | ~33k | Apache-2.0 | Very active (VoxCPM2, Apr 2026) | **Upstream of our `voxcpm2` engine** — sync items below |
|
||||
| [ebook2audiobook](https://github.com/DrewThomasson/ebook2audiobook) | ~19.5k | Apache-2.0 (default XTTS weights are CPML non-commercial) | Very active, weekly releases | The playbook for our weakest shipped surface (audiobook) |
|
||||
| [VideoLingo](https://github.com/Huanshere/VideoLingo) | ~17.7k | Apache-2.0 | Active, bursty | Dub-pipeline techniques (translation loop, timeline fit) |
|
||||
| [voicebox](https://github.com/jamiepine/voicebox) | ~40.2k | MIT | Very active, post-viral triage debt | **Direct competitor** — same stack, same pitch, 10x the audience |
|
||||
|
||||
## 1. Real-Time-Voice-Cloning — the retired ancestor
|
||||
|
||||
The 2019 SV2TTS implementation ("clone a voice in 5 seconds") that created the
|
||||
DIY voice-cloning category. Explicitly retired: the maintainer said in 2020 he
|
||||
won't develop it again; the README now calls itself old and redirects users to
|
||||
Chatterbox. Models are frozen 2019 checkpoints — 16 kHz, English-only, weak
|
||||
similarity, Tacotron+WaveRNN. Community PRs keep the install alive (uv
|
||||
one-command install landed Sept 2025), but ~163 open issues are mostly "how do
|
||||
I make it sound good" — the answer is: you can't.
|
||||
|
||||
**Integrating it as an engine: no.** Strictly worse than everything we ship,
|
||||
plus PyQt/legacy baggage.
|
||||
|
||||
**Take:**
|
||||
- 60k stars of traffic reads a README that says "go elsewhere," and the
|
||||
redirect target is a model repo, not a product. An honest
|
||||
"Real-Time-Voice-Cloning alternative" comparison page is cheap, truthful,
|
||||
and lands exactly our pitch (local, free, modern quality, 646 languages,
|
||||
actual installer).
|
||||
- Its headline copy discipline ("Clone a voice in 5 seconds, generate
|
||||
arbitrary speech in real-time") is better than ours; our 3-second-reference
|
||||
claim deserves the same outcome-first, time-boxed phrasing.
|
||||
- Its failure modes validate our Core Value: out-of-band model links rotted
|
||||
for years; a research toolbox without packaging drowned in install issues.
|
||||
|
||||
## 2. VoxCPM — upstream of our `voxcpm2` engine
|
||||
|
||||
Tokenizer-free TTS on a MiniCPM-4 backbone. Current model is **VoxCPM2**
|
||||
(Apr 2026): 2B params, 30 languages + 9 Chinese dialects, 48 kHz, ~8 GB VRAM,
|
||||
RTF ~0.30 (0.13 with Nano-vLLM). Latest tag v2.0.3 (May 2026); main has
|
||||
unreleased seed support and timestamp alignment. Apache-2.0, healthy cadence,
|
||||
~868k monthly HF downloads.
|
||||
|
||||
**Sync items for our integration** (we install `voxcpm` unpinned):
|
||||
|
||||
1. **Floor the install at `voxcpm>=2.0.3`** — it carries the MPS
|
||||
audio-quality fix (low-precision dtypes promoted to float32 on Apple
|
||||
Silicon). Directly relevant to our default-platform-parity rule.
|
||||
2. **v2.0.1 removed reference-audio auto-trim** — if we hand raw user clips
|
||||
to cloning, we now own trim/normalize. Verify our clone path; cloning
|
||||
quality may have silently regressed when upstream released 2.0.1.
|
||||
3. **Trailing-audio guard**: end-of-audio gibberish/hallucination is a known
|
||||
open upstream bug (#352). A trailing-silence/garbage trim on our side is
|
||||
cheap insurance.
|
||||
4. **Later, when tagged**: seed support (reproducible generation — currently
|
||||
buggy upstream, #351) and timestamp alignment (useful for dub sync);
|
||||
`generate_streaming()` is a candidate for `tts_stream.py`.
|
||||
5. **Risk**: unpinned dependency + active upstream = next release lands
|
||||
silently in fresh installs. Consider pinning a tested range.
|
||||
|
||||
## 3. ebook2audiobook — the audiobook playbook
|
||||
|
||||
Any-format ebook (epub/pdf/docx/even scanned images via OCR) → Calibre
|
||||
normalize to EPUB → TOC/spine chapters ("blocks") → per-language sentence
|
||||
split → per-sentence TTS → chapterized m4b with metadata/cover. Gradio UI +
|
||||
headless CLI + Docker for every accelerator. Engine roster is 2023-era Coqui
|
||||
(XTTSv2 default, Bark, Piper, MMS…), with voice-conversion post-processing to
|
||||
fake cloning on non-cloning engines. 19.5k stars, near-weekly releases, only
|
||||
4 open issues.
|
||||
|
||||
This is the mature version of exactly the surface where we're weakest: our
|
||||
audiobook/stories feature is a thin UI over per-chapter render caching, with
|
||||
no server-side ebook parsing and no per-segment regeneration.
|
||||
|
||||
**Take (prioritized):**
|
||||
1. **Per-sentence render cache + content-hashed blocks + missing-file
|
||||
resume.** Every sentence is its own file; restart re-renders only what's
|
||||
missing; editing a block invalidates only that block. This closes our
|
||||
biggest audiobook gap (per-chapter cache, no crash resume) and is the same
|
||||
span-level model spec 03 already calls for — dub's `incremental.py`
|
||||
pattern, extended to longform.
|
||||
2. **Normalize-to-EPUB ingestion** (Calibre `ebook-convert`) instead of
|
||||
building N format parsers; blocks carry keep/drop flags for front matter.
|
||||
3. **Engine-agnostic text-normalization pre-pass**: per-language abbreviation
|
||||
maps, num2words, roman numerals, and a non-text character filter that
|
||||
kills TTS hallucination triggers. Benefits every engine we ship, not just
|
||||
audiobooks.
|
||||
4. **Chapterized m4b output** (ffmpeg FFMETADATA chapters, cover art, VTT
|
||||
sidecar) — small work, high perceived value.
|
||||
5. **Inline voice/pause tags** for multi-voice narration — our cloning
|
||||
quality makes this worth more to us than it is to them.
|
||||
|
||||
**Where we already win:** native desktop UX, modern engine quality
|
||||
(CosyVoice3/IndexTTS2/VoxCPM2 vs 2023 Coqui), real zero-shot cloning without
|
||||
VC hacks, no Calibre-wall install, and a commercially-clean default engine
|
||||
(their default XTTS weights are CPML non-commercial).
|
||||
|
||||
## 4. VideoLingo — dub-pipeline techniques
|
||||
|
||||
"Netflix-quality subtitles + dubbing" as a 14-stage Streamlit pipeline:
|
||||
yt-dlp → WhisperX word-level ASR → spaCy + LLM two-candidate semantic split →
|
||||
summarize-first terminology glossary → 3-step Translate–Reflect–Adapt →
|
||||
length-constrained subtitles → duration-aware dub-chunk planning →
|
||||
per-chunk reference audio → TTS → merge. Its recommended path is
|
||||
cloud-heavy (API LLM/TTS, optionally API ASR); fully-local is possible but
|
||||
fragile. Single-speaker only — it explicitly gave up on diarized multi-voice
|
||||
dubbing. Apache-2.0, ~17.7k stars, bursty maintenance, install pain on
|
||||
Windows/CUDA.
|
||||
|
||||
**Take (prioritized):**
|
||||
1. **Translate–Reflect–Adapt** — add a reflection/critique pass to our
|
||||
per-segment translation prompt. Prompt-level change, meaningful quality
|
||||
win on idiomatic output.
|
||||
2. **Summarize-first glossary** — extract theme + terminology once per video,
|
||||
inject into every segment's translation. Fixes term drift on long videos.
|
||||
3. **Duration-aware chunk planning** — estimate TTS duration *before*
|
||||
generating; classify each line ok / needs-speedup / impossible; borrow
|
||||
inter-subtitle gap time and merge adjacent segments before resorting to
|
||||
atempo; for impossible lines, LLM-trim filler from the dub text instead of
|
||||
chipmunking. Our smart-fit handles the tail of this; their pre-planning
|
||||
avoids generating doomed audio at all.
|
||||
4. **Two-candidate split prompt** — generate two `[br]` segmentations, have
|
||||
the LLM pick, instead of accepting the first.
|
||||
|
||||
**Where we already win:** fully local by design, per-segment regeneration +
|
||||
directorial AI (they have coarse folder-state resume, no per-segment redo),
|
||||
cross-platform installers, cloning stable across languages. Their
|
||||
single-speaker ceiling is our opening if diarized multi-voice dubbing ever
|
||||
ships.
|
||||
|
||||
## 5. voicebox — the direct competitor
|
||||
|
||||
Jamie Pine's (Spacedrive founder) "open-source AI voice studio. Clone,
|
||||
dictate, create." — architecturally a near-twin: **Tauri + React/TS +
|
||||
FastAPI/Python + SQLite**, MIT, local-first, explicitly pitched as
|
||||
ElevenLabs-out + WisprFlow-in replacement. Launched Jan 29, 2026; the launch
|
||||
post did ~17M views on X, and it sits at **~40.2k stars** with ~10 community
|
||||
contributors and heavy AI co-authorship. Latest tagged release v0.5.0
|
||||
(Apr 2026); main is active but untagged for ~10 weeks, with **434 open
|
||||
issues / 105 open PRs** — a polished happy path with thin edges.
|
||||
|
||||
Engines: Qwen3-TTS 0.6B/1.7B (flagship cloner), Qwen CustomVoice, LuxTTS,
|
||||
Chatterbox Multilingual (23 langs) + Turbo, HumeAI TADA, Kokoro. Features
|
||||
where they lead: global-hotkey dictation overlay with LLM transcript cleanup
|
||||
(macOS-verified), Pedalboard post-FX chain, generation versioning/starring,
|
||||
multi-track Stories editor, **MCP per-client voice bindings** ("Claude Code
|
||||
speaks in your cloned voice") used as a viral wedge, DirectML/Intel-Arc
|
||||
coverage, and an agent-facing CONTRIBUTING pattern that farms drive-by
|
||||
contributions.
|
||||
|
||||
Two strategic facts:
|
||||
|
||||
- **They are adding accounts.** "Log in with browser" auth for a
|
||||
`voicebox.sh` cloud tier merged July 5 (their PR #812). Open-core with a
|
||||
paid cloud is visibly forming — which cuts against the pitch that won them
|
||||
their audience.
|
||||
- **Press already flagged their missing consent/misuse policy** — we ship
|
||||
watermarking by default and consent attestation in `.ovsvoice`.
|
||||
|
||||
**Where we're ahead:** 646 languages vs 23, video dubbing (they have none),
|
||||
voice design from text descriptions (roadmap item for them, shipped for us),
|
||||
engine breadth (CosyVoice3/VoxCPM2/IndexTTS2/GPT-SoVITS/sherpa-onnx), and
|
||||
backward-compat/release discipline.
|
||||
|
||||
**Take:**
|
||||
1. **Positioning: own "no accounts, ever."** Their cloud login is our
|
||||
opening — state the local-first guarantee in the README as a permanent
|
||||
commitment, next to the 646-language and dubbing advantages they can't
|
||||
match today.
|
||||
2. **Tell the MCP agent-voice story loudly.** We already ship an MCP server
|
||||
and Agent Skills; per-client voice bindings + a speak-in-your-voice demo
|
||||
was their single best growth hook and costs us mostly marketing effort.
|
||||
3. **Generation versioning/starring and post-FX presets** — cheap,
|
||||
high-perceived-value Studio features worth absorbing.
|
||||
4. **Watch their triage debt** (434 open issues): our absorb-or-decline
|
||||
queue discipline is a real contributor-trust differentiator — keep it.
|
||||
|
||||
## Consolidated priorities
|
||||
|
||||
Ordered by (user impact on already-shipped surfaces) × (effort):
|
||||
|
||||
1. **voxcpm2 upstream sync** (§2 items 1–3): version floor, ref-clip trim
|
||||
audit, trailing-audio guard. Small, protects an engine users already run.
|
||||
2. **Dub translation quality loop** (§4 items 1–2): reflect pass + glossary.
|
||||
Prompt-level, no new deps, lifts the flagship dubbing feature.
|
||||
3. **Audiobook maturity via per-sentence cache + resume** (§3 item 1): the
|
||||
established pattern for the feature the maturity survey ranked weakest —
|
||||
and it's the same architecture spec 03 already prescribes.
|
||||
4. **Text-normalization pre-pass** (§3 item 3): engine-agnostic hallucination
|
||||
reduction; pairs with the pronunciation dictionary we already shipped.
|
||||
5. **Duration-aware dub planning** (§4 item 3) and **chapterized m4b export**
|
||||
(§3 item 4): next tier, both self-contained.
|
||||
6. **Competitive positioning vs voicebox** (§5 items 1–2): own "no accounts,
|
||||
ever" while they onboard a cloud tier, and tell the MCP agent-voice story
|
||||
we already technically ship.
|
||||
7. **RTVC comparison/migration page** (§1): marketing, not engineering;
|
||||
cheap and honest.
|
||||
|
||||
*Method note: compiled from five parallel research passes over the repos'
|
||||
READMEs, releases, issues, and (for ebook2audiobook) source; figures as of
|
||||
2026-07-10.*
|
||||
+131
-1
@@ -6,7 +6,137 @@ The format is loosely based on [Keep a Changelog](https://keepachangelog.com/).
|
||||
Versions track the desktop app (`tauri.conf.json` + `frontend/src-tauri/Cargo.toml`).
|
||||
The bundled TTS model package (`pyproject.toml`) is versioned independently.
|
||||
|
||||
## [Unreleased]
|
||||
## [0.3.18] — 2026-07-12
|
||||
|
||||
The self-sufficiency release. Two long-standing "works on my network / works after four terminal commands" walls came down: model downloads now find a reachable Hugging Face endpoint on their own (no more restricted-network first-run dead-ends), and IndexTTS-2 — previously the only engine that demanded a manual clone-venv-install ritual — installs itself with one click. Under the hood, a test-debt sweep hardened the suite that guards all of it.
|
||||
|
||||
### Added
|
||||
|
||||
- **IndexTTS-2 installs itself now — one click in Settings → Engines.** The emotion-controlled cloning engine used to demand four terminal steps (clone the repo, create a venv, `uv pip install`, set an environment variable); the row now has an Install button that does all of it — source fetch (git, with a no-git tarball fallback), an isolated venv that keeps its `transformers<5` away from the app, the ~6 GB model weights (via your configured/auto-selected Hugging Face endpoint), and configuration — with step-by-step progress, a disk-space check before anything is written, and resumable repair if anything is interrupted. The engine is usable the moment the job finishes, no restart; existing manual installs are detected and left untouched, and the manual steps remain as a collapsible fallback. The provisioner is parametrized so future sidecar engines (MOSS-v1.5, dots.tts, Confucius4) can reuse it. (#1083)
|
||||
|
||||
- **Model downloads now find a reachable Hugging Face endpoint on their own.** On networks where huggingface.co is blocked or slow (the class of first-run dead-ends behind #984), the app quietly probes the official endpoint and the hf-mirror.com community mirror, picks whichever actually works, remembers the choice, and re-checks only when a download fails or the pick goes stale — so a restricted-network first run reaches a working voice instead of a wall of connection errors. Anyone who already set a mirror (env var, pref, or Settings) stays exactly where they pointed: explicit choices are never auto-switched, and Settings → Models → Hugging Face mirror now shows the automatic pick with its measured latency plus a "Test again" button. Probes only touch the two download hosts — no geo-IP, no telemetry — and every download stays checksum-verified by `huggingface_hub` regardless of endpoint. (#1082)
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Concurrent settings writes can no longer drop each other.** Two parts of the app saving preferences at the same moment (say, an engine install finishing while you change a setting) could silently lose whichever save landed first; preference writes are now serialized, with a regression test. Found and fixed as part of the IndexTTS-2 installer work. (#1083)
|
||||
|
||||
### CI
|
||||
|
||||
- **Test-suite debt sweep.** Exports coverage (26 new tests, which caught two real router bugs), fp16 default-dtype leak instrumentation, and a suite-order pollution class root-caused at its source — the checks that guard every release got stricter. (#1081)
|
||||
|
||||
## [0.3.17] — 2026-07-11
|
||||
|
||||
The polish release. The dubbing workspace can no longer trap you — an interrupted dub session used to relaunch into an eternal spinner that even reinstalling couldn't clear (thank you @nanai97 for the screenshot that cracked it). A 58-finding audit of every Settings panel got fixed end to end, **FFmpeg and yt-dlp stopped being your problem** (the app provisions its own, with a new Audio tools panel when you want control), the Engines and Models pages went compact and tabbed, the launcher stopped trusting half-dead backends, and the app finally opens at 100% scale.
|
||||
|
||||
### Added
|
||||
|
||||
- **FFmpeg, FFprobe, and yt-dlp stopped being your problem.** The setup wizard no longer lists them as system requirements with "brew install" homework — the app provisions them itself: shipped installs already bundle them, and when nothing is found the backend downloads its own checksum-pinned static build in the background, showing a single actionable card only if that fails. A new **Settings → Audio tools** panel gives back the control: per-tool version and origin (App package / Bundled / System / Custom), update / use-system / choose-file / restore-bundled — and **one-click yt-dlp updates** that survive app upgrades, because video-site support changes faster than releases. Install docs updated to match. (#1071)
|
||||
|
||||
- **The Engines and Models pages got compact and tabbed.** Engines is now one section with TTS / ASR / LLM tabs; every engine is a strict two-line, fixed-height row with truncated text and aligned status / GPU / isolation / action columns, so the whole engine list fits one screen — details like "Why unavailable?" expand below the row instead of stretching it. Models rows tightened the same way. (#1072)
|
||||
|
||||
- **The Engines and Models pages got a full readability-and-features pass.** Every engine row now carries a small identity mark and honest capability badges (voice cloning, device routing with the reason on hover, sidecar isolation), and engines that are ready-but-have-advice finally say so — upgrade hints used to be dropped before reaching the UI. The model store gains a filter, disk-space context next to downloads, "in memory — safe to unload" indicators, copyable setup snippets for opt-in engines, and empty states that tell you what to do next. (#1058)
|
||||
|
||||
### Fixed
|
||||
|
||||
- **The app no longer attaches to a "zombie" backend that looks alive but fails everything.** If a backend process survived while its install was replaced or deleted underneath it, it kept answering health checks from memory — so the next launch attached to it and every real request failed with a confusing access-control error. The launcher now runs a deeper probe (an endpoint that actually touches the database) before attaching, and replaces any backend that fails it. The local dev/test scripts also now terminate running instances before wiping data, which is how this state was produced. (#1077)
|
||||
|
||||
- **The app opens at 100% scale by default.** New installs rendered everything at 130% zoom, which read as oversized on typical displays. Fresh sessions now start at native size; if you already picked a scale in Settings → Appearance, your choice is kept. (#1074)
|
||||
|
||||
- **The app no longer relaunches into a dead "generating" dub session — the blank-pane-and-spinner trap.** The saved dub session was restoring its in-flight state verbatim: quit (or crash) while a dub was generating and every subsequent launch waited forever for work that died with the process — and reinstalling couldn't clear it. Interrupted sessions now reopen on the segment editor with all your work intact (or the upload screen if nothing was transcribed yet). Thanks to @nanai97 for the screenshot that told the whole story. (#1067)
|
||||
|
||||
- **A 58-finding audit of every Settings panel, fixed end to end.** Highlights: the About page linked to the wrong project's GitHub; Arabic rendered left-to-right (RTL wiring was missing); a saved proxy could never be cleared after a reload; the HF-mirror and refinement panels vanished entirely when the backend was down; "Test now" on the HF token served five-minute-old cached results; factory reset only cleared part of what it promised; pronunciation previews ignored language-scoped entries; the hotkey recorder swallowed invalid presses in silence; Settings search could strand you with an empty sidebar — plus first component tests for previously untested panels, full i18n for five all-English panels, accessible names across inputs, confirmed destructive actions, deep links instead of dead-end advice, temp-file reclaim, and log-sharing workflows. (#1059, #1060, #1061, #1063, #1064)
|
||||
|
||||
## [0.3.16] — 2026-07-11
|
||||
|
||||
The quality release. Three long-standing frictions got structural fixes: **regenerating no longer destroys good takes** (a takes rail with starring and restore), **audiobooks stop redoing finished work** (per-sentence caching — edit one line, re-render one line; crashes resume where they stopped), and **dub translations stay consistent and fit their timeline** (auto-glossary + a naturalness pass, plus fit prediction before any GPU time is spent). Under the hood, every text path now speaks numbers, times, and abbreviations correctly, the VoxCPM2 engine gained upstream-alignment guards, and a Windows first-run breaker — model downloads completing but the cache ending up with broken file links — now self-heals automatically. Thank you @dmnobunaga for the razor-sharp diagnosis on that last one.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Windows: model downloads that finished but wouldn't load now repair themselves.** On machines without Developer Mode, the model cache could end up with all its multi-gigabyte files downloaded but the snapshot's file links broken — and the app reported a misleading "does not appear to have a file named model.safetensors". The app now detects the broken links on load failure, restores just the missing pieces (reusing everything already downloaded, and falling back to real file copies where links can't be trusted), and retries once; if repair is impossible, the error finally names the actual cache folder to delete. Root-caused in the wild by @dmnobunaga — thank you. (#1056)
|
||||
|
||||
- **VoxCPM2: cloning reference clips are now conditioned, and outputs lose their silent tails.** Reference audio used to reach the model completely raw; it now gets edge-silence trimming and a 30-second cap (fail-open — short clean clips pass through untouched), and generated audio gets a trailing-silence trim. The install hint also moved to `voxcpm>=2.0.3`, which carries an important Apple-Silicon audio-quality fix — older installs keep working and see an upgrade hint in the logs. (#1055)
|
||||
|
||||
- **Streaming TTS requests without an `emo_alpha` field no longer crash.** A minimal `/ws/tts` request hit a `KeyError` and returned an error frame instead of audio — found while giving that route its first tests. (#1054)
|
||||
|
||||
- **Long generations no longer risk a multi-gigabyte memory spike while being watermarked.** The invisible watermark (on by default) pushed the entire waveform through AudioSeal in a single call, and its memory use grows with audio length — a multi-minute generation demanded a single ~2 GB allocation, enough to fail outright on a 16 GB machine already holding a model ("DefaultCPUAllocator: not enough memory"). Watermark embedding — and the Verify-audio detector, which had the same flaw with uploaded files — now processes audio in ~30-second chunks, so peak memory stays flat no matter how long the audio is. Detection also got sharper for spliced files: it now reports the strongest chunk instead of a whole-file average. (#1045)
|
||||
|
||||
- **The ⊕ Insert token list no longer climbs out of the viewport.** In the voice-clone script panel, the insert popover (expression tags, CMU phoneme chips) always opened *upward* from the textarea — and since that input sits at the very top of the panel, the list disappeared past the top of the window with no way to see or scroll it. It now opens below the input, where there's always room. (owner-reported)
|
||||
|
||||
### Added
|
||||
|
||||
- **Edit one sentence, re-render one sentence.** Audiobook and Stories renders now cache every synthesized sentence individually (content-addressed, under the existing chapter cache): fixing a single line in a chapter reuses all the untouched audio, and an interrupted render — crash, quit, power loss — resumes from the sentences that already finished instead of redoing the whole chapter. One byte cap bounds both cache layers, and chapter caches from released versions keep working. (#1048)
|
||||
|
||||
- **Numbers, times, and abbreviations are spoken correctly in every engine.** A conservative normalization pass now runs before TTS everywhere (Studio, dubbing, audiobooks): "3:30" is read as a time, "2" as "two" (29 languages), "Dr." as "Doctor" — while stray control characters and markup remnants that trigger engine hallucinations are stripped. Deliberately cautious: when a rewrite could be wrong, the text is left alone, and your pronunciation-dictionary entries always have the final say. Toggleable (`text_normalization_enabled`, default on). The OpenAI-compatible API, streaming TTS, and the batch queue run the same pass, so every door into the engines speaks text identically. (#1049, #1054)
|
||||
|
||||
- **Dub translations stay consistent and sound natural (LLM engine).** Before translating, one pass over the whole transcript builds a terminology glossary (your manual glossary entries always win) that rides along on every segment, so names and terms stop drifting mid-video. After each segment's direct translation, an optional reflect pass critiques and rewrites stiff lines into natural spoken dialogue — any failure silently keeps the direct translation. Both toggleable in the Dub tab; the reflect toggle states its 3-calls-per-segment cost. (#1050)
|
||||
|
||||
- **The Dub tab now predicts which lines won't fit — before wasting GPU time on them.** After translation, each segment gets a duration estimate (self-calibrating to your engine and language from the segments already rendered) and a "Tight fit" or "Won't fit +Ns" badge when the dubbed audio can't match the timeline even with speed-up. An opt-in "Suggest shorter lines" option asks the LLM for a meaning-preserving shorter rewrite you can apply per segment — never applied automatically. (#1051)
|
||||
|
||||
- **Generation takes: star the good ones, restore any of them.** Regenerating no longer means losing the previous result — recent takes appear in the workspace history with replay, star/unstar, and one-click restore as the active output. History is now capped (Settings → Storage, default 200 takes): the oldest unstarred takes are pruned, starred ones are kept forever, and an audio file is only deleted when nothing else references it. (#1052)
|
||||
|
||||
- **A persistent mini-player for all the audio that used to play "invisibly".** Generated output, voice-profile and dub-segment previews, story lines, Gallery voices, and Projects renders all played through a bare audio pipe — no waveform, no seek, no time, and (until v0.3.15's stop pill) no way to stop them. A slim player bar now docks above the Logs footer whenever such audio plays, on every page: live waveform (decoded once from the audio already in memory — nothing is re-fetched), click/drag/keyboard seek, play/pause, elapsed/total time, what's-playing label, and a stop button. It replaces the stop-only pill, and because it's part of the app's layout rather than a floating overlay, the pill's "covers the Production Overrides row at 1440×900" overlap class can't come back. Stories line previews also route through it — which makes them stoppable *and* fixes them being silent on the macOS/Linux desktop builds (their old playback path used blob: URLs, which WebKit refuses to play). (no issue — owner request following #1032's stop-pill band-aid)
|
||||
|
||||
## [0.3.15] — 2026-07-10
|
||||
|
||||
The cold-start release. Three "why is this broken on my machine" mysteries got solved at their roots: **first generations stop dying at 300 seconds** (the timeout was counting the model download as generation time — @moduvoice measured it on a Tesla T4: 0% GPU for the full window), **updates stop deleting engines you installed yourself** (the updater's dependency sync removed anything not in the app's lockfile — including things our own UI told you to install), and **the "slower than v0.3.5" regression is found and fixed** (clone profiles without a transcript were silently re-running a full Whisper transcription on every single generate). Also: Clear History is back, auto-played audio is finally stoppable, @stronghamjji hardened the dub pipeline against wedged transcribes, and @shakib30's community Colab notebook is now the linked no-GPU path. Thank you all.
|
||||
|
||||
|
||||
### Added
|
||||
|
||||
- **Agent Skills: `npx skills add debpalash/omnivoice-studio`.** Two installable [skills](https://skills.sh) now ship in the repo — `omnivoice` teaches any AI agent (Claude Code, Cursor, Codex, …) to speak and transcribe through your local install via the OpenAI-compatible API, including your cloned voices; `oss-maintainer` packages the maintainer methodology this project is run with.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **A fresh install's first generation no longer dies at 300 seconds while the model is still downloading.** The generate timeout was one clock around everything — including the engine's lazy multi-GB weight download on a cold start — so first requests burned the whole budget on the download (0% GPU the entire time, as a contributor's Tesla T4 verification measured) and failed with a misleading "too heavy for the available compute" error. Model loading now runs first under its own, much larger budget; the generate clock starts only once the engine is warm. A genuinely stalled download gets a new error that says so and points at Settings → Models. (#1033, #1037, evidence from #1014)
|
||||
- **The OpenAI-compatible speech endpoint stops silently discarding quality settings.** `POST /v1/audio/speech` accepted `num_step` and `guidance_scale` in the request body with a 200 OK — and dropped them without a word, so API callers couldn't reach the model's documented quality preset (`num_step: 32`). Both are now declared, validated, and passed through to the engine, matching the native `/generate` endpoint. Caught by a contributor's measured Tesla T4 verification pass. (#1014)
|
||||
- **Updating no longer uninstalls engines you added yourself.** Optional engines installed with pip into the app's environment (VoxCPM2, KittenTTS — exactly what Settings → Engines' own install hints say to do) were silently deleted by every app update, because the update's dependency sync removed anything not in the app's lockfile. Routine updates now leave your additions alone; the repair path ("Clean & Retry") still restores the exact known-good state, since a broken environment is sometimes *caused* by an extra package. (#1029)
|
||||
- **Voice cloning stops re-transcribing the same reference clip on every generate.** Since v0.3.6, a profile saved without a transcript (the default) triggered a full ASR model load *plus* a transcription of the reference on every single synthesis — the "TTS got much slower than v0.3.5, same settings" regression. The first auto-transcription is now saved onto the profile, and repeated ad-hoc uploads of the same clip reuse a content-keyed transcript cache — so the cost is paid once, not per request. A transcript you typed yourself is never overwritten. (#1032)
|
||||
- **The Clear History button is back.** The workspace redesign moved generation history into the right-side panels but dropped the old sidebar's clear-all control, leaving one-by-one deletion as the only way to empty a long history. Both the Voice and Dub history panels now have a Clear History button (with a confirmation), scoped to that workspace's history. (#1032)
|
||||
- **The audio that auto-plays after a render can finally be stopped anywhere.** The finished-render playback has no on-screen player, and the only stop control lived in the Voice workspace's action bar — audio started from the Dub workspace, a profile preview, or after navigating away simply played to the end. A stop button now appears above the status area whenever such playback is active, on every page. The existing Settings → Appearance "Auto-play preview" toggle now also governs this playback, as its description always promised. (#1032)
|
||||
|
||||
## [0.3.14] — 2026-07-09
|
||||
|
||||
A fast follow to v0.3.13: **every engine family now has a visible picker.** Settings → Engines showed only a TTS table, with the ASR and LLM pickers hidden behind a low-discoverability tab — so the 10 transcription engines (including the new OpenAI-compatible backend) looked unswitchable without env vars. Now all three families get their own table. Also in: the Linux AppImage's white-screen auto-workaround now checks the WebKitGTK it actually ships (not whatever your system reports), and installing to a different drive on Windows is properly documented.
|
||||
|
||||
### Added
|
||||
|
||||
- **ASR engines get the same Settings picker TTS has.** Settings → Engines now shows a visible picker table per family — TTS, ASR, and LLM — instead of a single TTS-titled table with the other families tucked behind a tab (README even promised a Settings ASR picker that didn't exist). The OpenAI-compatible backend and the 9 local ASR engines become selectable with one click, no env vars needed; an explicit `OMNIVOICE_ASR_BACKEND` still wins over the Settings pick, so pinned setups behave exactly as before. (no issue — UX gap found during #877)
|
||||
|
||||
### Fixed
|
||||
|
||||
- **The Linux AppImage's white-screen auto-workaround now checks the right WebKitGTK.** The launcher decided whether to apply the compositing workaround by asking the *system's* `pkg-config` — but the version that actually runs is the *bundled* one, which the AppImage prioritizes. On any machine where the two diverge (e.g. building from source with newer dev packages installed), the detection read the wrong number and could skip a workaround the running library needed. The build now stamps the bundled version into the AppImage at package time, and the launcher reads that stamp — correct by construction. The launcher's shell tests also now run in CI, which they previously never did. (#961 follow-up)
|
||||
|
||||
### Docs
|
||||
|
||||
- **Windows: installing to a different drive is documented** — the wizard's directory picker works for any local drive; mapped network drives are a Windows Installer limitation (not installable-to by design); and the big data (models/voices) moves independently via Settings → Storage or Portable mode. (#938)
|
||||
|
||||
## [0.3.13] — 2026-07-09
|
||||
|
||||
The community-fixes release. Two contributors didn't just report bugs — they diagnosed them to the exact line and submitted the fixes that shipped: **voice cloning on mlx-audio's CSM model works for the first time**, and **macOS live recording finally gets its microphone permission prompt** (both @MahdiHedhli). A third reporter's A/B analysis fixed **cross-language dubs speaking the wrong language**. On top of that: a backend shutdown race that produced confusing crash-on-quit reports is fixed, the Linux AppImage stops shipping a stale WebKitGTK that white-screened current distros, and a new OpenAI-compatible transcription backend opens a path to Qwen3-ASR today. Thank you to everyone who filed, diagnosed, and contributed — this release is mostly yours.
|
||||
|
||||
### Added
|
||||
|
||||
- **A path to Qwen3-ASR today: generic OpenAI-compatible transcription.** The direct integration is still blocked on `transformers>=5.13` stabilizing upstream, but a community member proposed splitting the work — add a backend that talks to any OpenAI-compatible transcription server right now. Point OmniVoice at a self-hosted Qwen3-ASR/FunASR/SenseVoice server, or OpenAI's own API, configured in Settings → Models. No install; audio does leave your machine to whichever server you configure, unlike every other ASR engine. (#877)
|
||||
|
||||
### Fixed
|
||||
|
||||
- **The Linux AppImage no longer white-screens on current distros with a healthy system WebKitGTK.** The release build ran on an older CI base image, and the resulting AppImage bundles whatever `libwebkit2gtk` that image's apt repos resolve — which the AppImage's own `LD_LIBRARY_PATH` then prioritizes over your system's newer, healthy copy at runtime. A from-source build (which links straight against your system library) worked fine on the exact same machine where the shipped AppImage didn't — that split was the tell. Bumped the release build to a current Ubuntu LTS. Raises the AppImage's minimum host to glibc 2.39 (Ubuntu 24.04+); no reports from anyone on an older distro. (#961)
|
||||
- **Backend shutdown no longer races a still-loading model, surfacing a confusing crash on restart.** Quitting the app while a model was still loading in the background let shutdown report itself "done" while a background thread was still mid-import; tearing the process down under that thread produced a misleading error (a generic transformers import-failure message, unrelated to the real cause) that looked like a real crash rather than a timing issue. All background tasks are now properly cancelled and awaited before shutdown proceeds. (#1000, likely the same class behind #941 and #979)
|
||||
- **Cross-language dub no longer speaks the source-language reference line verbatim.** Auto-generated speaker clones pair an audio slice with the ASR segment's own text field, assuming the two agree — but ASR segment text and its timestamps routinely drift (a trailing word audible in the clip but missing from the text, or vice versa). A mismatched (reference audio, reference text) pair breaks zero-shot TTS prompt priming badly enough that the clone can emit the reference text itself instead of the target-language line it was asked to speak. Each reference clip is now re-transcribed after it's written, so the pair matches by construction — reported with an exceptionally clear root-cause diagnosis and a working A/B repro. (#1004)
|
||||
- **Voice Gallery errors now say what actually went wrong.** "Use voice", "Preview", search, upload, save, delete, and trim in the Gallery all showed the same hardcoded guess ("the engine may be loading") on ANY failure — a 500, a validation error, a genuinely unrelated bug — discarding the real, already-clean backend error message in the process. Every one of those now shows the actual error.
|
||||
- **Voice cloning on mlx-audio's CSM model no longer crashes with an opaque "list index out of range".** `MLXAudioBackend.generate()` read `voice`/`ref_audio`/`language`/`speed` from its kwargs but silently dropped `ref_text` — CSM only builds its cloning context when both `ref_audio` and `ref_text` are present, so cloning on this engine could never have worked as shipped. Reported with the exact root cause and a working fix. (#1012, #1013)
|
||||
- **A dub segment's free-text style tags no longer 400 the segment preview.** A validator-safe instruct builder already keeps Studio and Clone generation from round-tripping a 400 on unsupported free-text (a preset's raw attrs, an old profile's stray descriptive phrase) — but the Dub tab's segment preview, and saving a profile from a clone or from history, built their instruct strings directly and skipped it. Same guard now applies everywhere an instruct string is sent. (#1010)
|
||||
- **The dub editor's play button no longer sticks permanently disabled after an audio-decode hiccup.** When the initial WaveSurfer decode fails, the timeline falls back to loading pre-computed peaks — the waveform draws fine, but the button's enabled state only relied on the `ready` event firing again for that recovery load, which it didn't reliably do. Each fallback path now confirms readiness explicitly once it settles.
|
||||
- **macOS: live recording finally works — the microphone permission prompt now actually appears.** The app never showed up in System Settings → Privacy & Security → Microphone because macOS never saw a legitimate request: Tauri enables Hardened Runtime by default, which blocks microphone hardware access unless the matching entitlement is in the signed bundle — and it wasn't. Diagnosed to the exact mechanism and fixed by a community contributor (@MahdiHedhli), who also corrected our initial mis-read of this as an upstream WebKit limitation. (#1013, #1016)
|
||||
- **Quitting during a slow model load waits longer before giving up.** A post-merge code review of the shutdown-race fix flagged that its 3-second wait could still be outrun by a cold model import on a slow disk, reproducing the original confusing-crash-on-quit in rare cases. The wait is now 20 seconds — imperceptible on a normal quit (tasks finish or cancel in milliseconds), only felt in the exact case it protects. (#1020)
|
||||
|
||||
### Changed
|
||||
|
||||
- **Removed the donate heart from the nav rail.** Support OmniVoice is still one click away from Settings and the Contact page.
|
||||
|
||||
### CI
|
||||
|
||||
- **The "flaky trio" is root-caused and neutralized.** Three tests failed intermittently on CI — never locally — across unrelated PRs, costing a re-run each time. Cause: a leaked half-precision torch default from some earlier test in CI's ordering (the giveaway: a failing assertion's observed value was exactly float16(0.1)). An autouse test-suite guard now resets the leak between tests and names the offending test in CI output when it fires. (#1021)
|
||||
|
||||
## [0.3.12] — 2026-07-08
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
|
||||
**OmniVoice Studio**
|
||||
|
||||
OmniVoice Studio is an open-source, fully-local ElevenLabs alternative — a desktop app for voice cloning, voice design, video dubbing, and real-time dictation across 646 languages. It runs entirely on the user's machine (CUDA/MPS/ROCm/CPU auto-detect), with no API keys, no accounts, and no cloud dependencies. It's an active beta with a growing user base who hit it with real workloads (50-video batches, multi-engine setups, edge-OS platforms) and report friction in GitHub Issues and Discord. The latest stable release is **v0.3.5**; `main` rolls ahead at **v0.3.6** (latest release + 1 patch — see the Versioning rule below).
|
||||
OmniVoice Studio is an open-source, fully-local ElevenLabs alternative — a desktop app for voice cloning, voice design, video dubbing, and real-time dictation across 646 languages. It runs entirely on the user's machine (CUDA/MPS/ROCm/CPU auto-detect), with no API keys, no accounts, and no cloud dependencies. It's an active beta with a growing user base who hit it with real workloads (50-video batches, multi-engine setups, edge-OS platforms) and report friction in GitHub Issues and Discord. The current version lives in `frontend/package.json` (the single source of truth — see Versioning); the latest stable tag is on the [Releases page](https://github.com/debpalash/OmniVoice-Studio/releases/latest). With `AUTO_VERSION_BUMP` off (the current owner setting), `main` holds at the released version between releases.
|
||||
|
||||
**Core Value:** **A first-run that actually works.** A user who downloads the installer (or clones the repo) should reach a working voice-cloning or dubbing output without hitting a wall — and when something does go wrong, the error or docs should tell them exactly what to do.
|
||||
|
||||
@@ -16,175 +16,21 @@ Everything else (new engines, fancy features) is downstream of "the thing instal
|
||||
- **Default features must work on every platform (strict rule, 2026-05-20):** A feature that ships in default mode — out-of-the-box, no user customization, no opt-in toggle — must behave identically on macOS, Windows, and Linux. Platform-specific *implementation code* is allowed for OS APIs / shells / packaging, but the user-visible *default behavior* cannot diverge. Platform-only features (e.g., a macOS-only global shortcut, a Windows-only path picker) must go behind explicit user opt-in: Settings toggle, env var, or CLI flag. When a default doesn't work on a platform, that's a P0 bug — either fix it on the missing platform or move it behind opt-in. No third option.
|
||||
- **Backward-compatible project data**: Existing `omnivoice_data/` (user voices, projects, settings) must keep working without manual migration. Any DB schema change goes through alembic with a tested upgrade path.
|
||||
- **Local-first guarantee preserved**: Auto bug reporting (new addition) must be **opt-in**, must submit only to GitHub Issues (no third-party telemetry endpoint), and the app must remain fully functional with reporting disabled. No required cloud calls, accounts, or API keys.
|
||||
- **Beta release cadence (no RC, no ceremony — strict rule, 2026-05-20):** the v0.3.x line has **no release candidates, no 48h soak, no formal release ceremony**. Every fix goes continuous-to-main; the owner tags a patch (`v0.3.Z`) from main whenever the current state is worth cutting. No `-rc` tags. No phased release. No `v0.4` deferrals while the v0.3.x line is open — every open issue and every open community PR gets absorbed into the v0.3.x line or explicitly declined. Users follow `main` for previews; users wanting stable stay on the latest tagged release (currently **v0.3.5**). ROADMAP.md's Phase 6 "Release/Verify/Retro" entries are obsolete unless the user revives them.
|
||||
- **Beta release cadence (no RC, no ceremony — strict rule, 2026-05-20):** the v0.3.x line has **no release candidates, no 48h soak, no formal release ceremony**. Every fix goes continuous-to-main; the owner tags a patch (`v0.3.Z`) from main whenever the current state is worth cutting. No `-rc` tags. No phased release. No `v0.4` deferrals while the v0.3.x line is open — every open issue and every open community PR gets absorbed into the v0.3.x line or explicitly declined. Users follow `main` for previews; users wanting stable stay on the latest tagged release. ROADMAP.md's Phase 6 "Release/Verify/Retro" entries are obsolete unless the user revives them.
|
||||
<!-- GSD:project-end -->
|
||||
|
||||
<!-- GSD:stack-start source:research/STACK.md -->
|
||||
## Technology Stack
|
||||
|
||||
## Recommended Stack — Per Capability
|
||||
### Capability 1 — HuggingFace Token Persistence (issue #35)
|
||||
| Technology | Version | Purpose | Why Recommended |
|
||||
|------------|---------|---------|-----------------|
|
||||
| `huggingface_hub` (already pinned transitively by `transformers>=5.3.0`) | `≥1.12.x` (latest 2026) | Auth + cache + token storage | Canonical, used by every HF library already in the stack. `HfFolder` is **superseded** in v1.x by the higher-level `login()` / `auth_list()` / `auth_switch()` API. |
|
||||
| `keyring` (Python) | `≥25.x` | Optional OS-keychain backing | Only adopt if a future hardening pass wants Keychain/Credential-Manager/SecretService. **Not recommended for this milestone** — adds a native dep (`dbus`, `pywin32`) per platform with no real security win over `0600` file storage in `HF_HOME`. |
|
||||
| Shell | One-liner to persist `HF_TOKEN` |
|
||||
|-------|---------------------------------|
|
||||
| macOS zsh (default since 10.15) | `echo 'export HF_TOKEN=hf_xxx' >> ~/.zshrc && source ~/.zshrc` |
|
||||
| Linux bash | `echo 'export HF_TOKEN=hf_xxx' >> ~/.bashrc && source ~/.bashrc` |
|
||||
| Windows PowerShell (user scope) | `[Environment]::SetEnvironmentVariable("HF_TOKEN","hf_xxx","User")` (new shells only) |
|
||||
| Windows cmd | `setx HF_TOKEN "hf_xxx"` (user scope, new shells only) |
|
||||
- [HF environment variables docs](https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables) — HIGH confidence (official, current)
|
||||
- [HF authentication API docs](https://huggingface.co/docs/huggingface_hub/en/package_reference/authentication) — HIGH confidence
|
||||
- [Microsoft `setx` docs](https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/setx) — HIGH confidence
|
||||
### Capability 2 — In-App Structured Bug Reporting (opt-in, GitHub Issues)
|
||||
| Technology | Version | Purpose | Why Recommended |
|
||||
|------------|---------|---------|-----------------|
|
||||
| GitHub REST API `POST /repos/{owner}/{repo}/issues` | `2026-03-10` API version | Server-side issue creation | Official, stable. Requires auth. |
|
||||
| **Prefilled-URL pattern** (`github.com/{owner}/{repo}/issues/new?title=…&body=…&labels=…`) | n/a | Zero-auth fallback | **This is the recommended primary path for v0.3.x.** No token needed, no GitHub App registration needed, user's browser opens with a prefilled form, they review and click Submit. They own the issue, the OSS project gets the report, and OmniVoice never holds a credential. |
|
||||
| `gh-app-jwt` + GitHub App (Rust crate `octocrab` or Python `pygithub`) | only if we later want fully-automated submission | Programmatic posting under an app identity | **Defer to a later milestone.** Requires registering a public GitHub App, hosting a token-exchange endpoint, and managing rate-limit quotas — disproportionate for stabilization scope. |
|
||||
| `platform`, `psutil`, `torch.cuda` (already in deps) | already pinned | Capture OS, CPU/GPU/VRAM info | No new deps. |
|
||||
| `httpx` (already in `dev-dependencies`, promote to runtime if needed) | `≥0.28.1` | HTTP for the API call path (if/when we add auth) | Modern async-first, already used in test suite. |
|
||||
- ✓ No token storage in OmniVoice → no security surface
|
||||
- ✓ Opt-in by definition (user has to click Submit on github.com)
|
||||
- ✓ User owns the issue → can be replied to, edited, closed by them
|
||||
- ✓ Zero infra cost — no proxy, no app, no rate-limit management
|
||||
- ✓ Works identically on macOS / Windows / Linux via Tauri's `shell.open`
|
||||
- ✓ Survives our project being forked (just change the URL)
|
||||
- OS name + version (`platform.platform()`)
|
||||
- Python version (`sys.version`)
|
||||
- OmniVoice version (`pyproject.toml`)
|
||||
- Backend git SHA (if installed from source) or installer build ID
|
||||
- CPU model, RAM (`psutil.cpu_count()`, `psutil.virtual_memory()`)
|
||||
- GPU vendor/model/VRAM (`torch.cuda.get_device_name()`, `torch.cuda.mem_get_info()`, MPS detect)
|
||||
- Active TTS engine + list of installed engines
|
||||
- Frontend: bun version, OS shell
|
||||
- Last error message + stack trace if launched from an error toast
|
||||
- Audio file contents (privacy — reference samples may contain user's voice)
|
||||
- File paths containing `/Users/<name>/` (strip home dir → `~/`)
|
||||
- HF token, OpenAI keys, any env var matching `*TOKEN*|*KEY*|*SECRET*`
|
||||
- [GitHub URL query parameters for issues](https://docs.github.com/en/issues/tracking-your-work-with-issues/using-issues/creating-an-issue#creating-an-issue-from-a-url-query) — HIGH confidence
|
||||
- [sindresorhus/new-github-issue-url](https://github.com/sindresorhus/new-github-issue-url) — HIGH (widely used reference impl)
|
||||
- [GitHub REST API: Create an issue](https://docs.github.com/en/rest/issues/issues#create-an-issue) — HIGH confidence (for the future auto-submit path)
|
||||
- [sentry-tauri](https://github.com/timfish/sentry-tauri) — reviewed, **rejected for milestone** due to local-first constraint
|
||||
### Capability 3 — `uv venv` Mirror Fallback for Restricted Networks (issues #57, #60)
|
||||
| Technology | Version | Purpose | Why Recommended |
|
||||
|------------|---------|---------|-----------------|
|
||||
| `uv` (already used) | `≥0.5.x` | Python+venv bootstrap | Existing dep. |
|
||||
| `UV_PYTHON_INSTALL_MIRROR` env var | uv `0.4.x`+ | Override python-build-standalone download URL | **Official, current.** Replaces `https://github.com/astral-sh/python-build-standalone/releases/download/...` in download URL construction. No built-in fallback if mirror fails. |
|
||||
| `UV_PYTHON_PREFERENCE=only-system` (or CLI flag `--python-preference only-system`) | uv `0.4.x`+ | Skip the python-build-standalone download entirely; use the user's system Python | **The reliable escape hatch** when no mirror works. Requires a compatible Python `>=3.11` to already be on PATH. |
|
||||
| `UV_HTTP_TIMEOUT`, `UV_HTTP_CONNECT_TIMEOUT`, `UV_HTTP_RETRIES` | uv `0.4.x`+ | Tune retry behavior for flaky links | Defaults are 30s / 10s / 3 — bump to 120s / 30s / 5 for restricted networks. |
|
||||
# Pseudocode for the bootstrap
|
||||
# Final fallback: don't download Python at all
|
||||
- `UV_DEFAULT_INDEX=https://pypi.tuna.tsinghua.edu.cn/simple` (Tsinghua — fastest in China)
|
||||
- `UV_DEFAULT_INDEX=https://mirrors.aliyun.com/pypi/simple` (Aliyun fallback)
|
||||
- Russia: no major government-blessed PyPI mirror; users typically tunnel via VPN. Document this honestly rather than ship a broken default.
|
||||
- [uv environment variables reference](https://docs.astral.sh/uv/reference/environment/) — HIGH (official)
|
||||
- [uv issue #5224 — python-build-standalone mirror support](https://github.com/astral-sh/uv/issues/5224) — HIGH (the feature was added)
|
||||
- [uv issue #14187 — venv on Chinese network](https://github.com/astral-sh/uv/issues/14187) — HIGH (confirms real user pain, no built-in fallback)
|
||||
- [uv python-versions concepts](https://github.com/astral-sh/uv/blob/main/docs/concepts/python-versions.md) — HIGH (documents `python-preference` semantics)
|
||||
- [dautovri/mirrors-china](https://github.com/dautovri/mirrors-china) — MEDIUM (community-maintained mirror list; verify each URL still works before shipping)
|
||||
### Capability 4 — Supertonic-3 TTS Engine
|
||||
| Technology | Version | Purpose | Why Recommended |
|
||||
|------------|---------|---------|-----------------|
|
||||
| `supertonic` (PyPI) | `1.3.1` (latest, May 18 2026 — Phase 3 Wave 1 to verify constructor signature before bump) | Official Supertonic-3 inference SDK | Authoritative wrapper from Supertone Inc. Wraps the ONNX session orchestration so we don't have to. |
|
||||
| `onnxruntime` | `≥1.17.x` (any recent) | ONNX inference runtime | Already a transitive dep of WhisperX (via CTranslate2 path is separate, but `onnxruntime` itself ships for kittentts and audioseal). Verify with `uv tree` after adding — should resolve cleanly. |
|
||||
| `huggingface_hub` (already pinned) | `≥1.12.x` | Model weight download (~400 MB on first use) | Reuses existing HF token + cache infrastructure. The user's existing `HF_TOKEN` (Capability 1) works for the Supertonic model download too. |
|
||||
| `numpy`, `soundfile` (already pinned) | already pinned | Audio I/O + array math | No new deps. |
|
||||
- `text_encoder.onnx`
|
||||
- `latent_denoiser.onnx`
|
||||
- `voice_decoder.onnx`
|
||||
- 44.1 kHz sample rate, 24-dim latent, 128-dim style
|
||||
- ~99M parameters total
|
||||
- Tokenizer: `AutoTokenizer.from_pretrained(model_path)` — loads from `tokenizer.json` shipped with model
|
||||
- [Supertone/supertonic-3 model card](https://huggingface.co/Supertone/supertonic-3) — HIGH (official)
|
||||
- [supertone-inc/supertonic GitHub](https://github.com/supertone-inc/supertonic) — HIGH (official)
|
||||
- [supertonic PyPI page](https://pypi.org/project/supertonic/) — HIGH (`1.3.1` confirmed 2026-05-18; same publisher, MIT, same 4 deps)
|
||||
- [onnx-community/Supertonic-TTS-ONNX](https://huggingface.co/onnx-community/Supertonic-TTS-ONNX) — HIGH (ONNX file structure details)
|
||||
### Capability 5 — Cross-Platform Documentation Tooling
|
||||
| Technology | Version | Purpose | Why Recommended |
|
||||
|------------|---------|---------|-----------------|
|
||||
| Plain Markdown in `docs/` + GitHub-rendered (current state) | n/a | Install tutorial, troubleshooting | Zero new infra. Renders inline on GitHub for issue-replies. No build step to break. |
|
||||
| Existing `scripts/smoke-test.sh` + Playwright `tests/` (already in `package.json`) | already pinned | Verify install paths actually work | **This is the real solution to "docs drift."** If smoke-test exercises the install path described in docs, docs that drift will break CI. |
|
||||
| **Future** (defer): Astro Starlight | `≥0.30` | Standalone docs site at `docs.omnivoice.studio` | Adopt only when docs exceed ~20 markdown files and need search/versioning. Tauri, the framework OmniVoice already depends on, uses Starlight — well-traveled choice. Material for MkDocs entered maintenance mode in November 2025 per Docsio's 2026 review — **avoid** for new docs. |
|
||||
| Project | What they do |
|
||||
|---------|--------------|
|
||||
| **OBS Studio** | Docs at `obsproject.com/docs` (Sphinx, separate repo). Install paths in README, wiki for community-contributed. CI doesn't gate on docs drift. |
|
||||
| **Audacity** | Manual at `manual.audacityteam.org` (MediaWiki). README is minimal. Install path = "use the installer." No automated sync. |
|
||||
| **Tauri** | Docs at `v2.tauri.app` (Astro Starlight, separate repo `tauri-apps/tauri-docs`). README is minimal. Heavy reliance on community contributions and PR review. |
|
||||
| **VS Code** | Docs at `code.visualstudio.com/docs` (separate repo, Markdown). README is minimal. Manual sync; docs team is staffed. |
|
||||
- [Tauri docs (Astro Starlight)](https://github.com/tauri-apps/tauri-docs) — HIGH (reference for "if we ever move off README")
|
||||
- [OBS Studio docs](https://docs.obsproject.com/) — HIGH (Sphinx, separate site reference)
|
||||
- [Audacity Manual](https://manual.audacityteam.org/) — HIGH (MediaWiki reference)
|
||||
- [Docsio: Material for MkDocs 2026 review (maintenance mode)](https://docsio.co/blog/mkdocs-material) — MEDIUM (third-party review, but signal aligns with project's own GitHub activity)
|
||||
- [Docsio: Starlight 2026 review](https://docsio.co/blog/starlight-docs) — MEDIUM
|
||||
## Installation
|
||||
# No new Python dependencies needed for Capabilities 1, 2, 3, 5.
|
||||
# Only Capability 4 adds a runtime dep:
|
||||
# Verify no regressions:
|
||||
# Should show single versions of each; no duplicates.
|
||||
## Alternatives Considered
|
||||
| Recommended | Alternative | When to Use Alternative |
|
||||
|-------------|-------------|-------------------------|
|
||||
| HF token via in-app Settings → `huggingface_hub.login()` | OS keyring via `keyring` package | Only if a security hardening milestone later demands OS-native credential storage. Not worth the cross-platform native-dep cost for v0.3.x. |
|
||||
| Prefilled-URL GitHub Issues | GitHub App + device flow + authenticated POST | When milestone budget can afford registering a public GitHub App and hosting a token-exchange function. Defer. |
|
||||
| Prefilled-URL GitHub Issues | Sentry / `sentry-tauri` | Never — violates the "no third-party telemetry endpoint" constraint in PROJECT.md. |
|
||||
| `UV_PYTHON_INSTALL_MIRROR` chain + `only-system` fallback | Bundle Python in the Tauri installer | Adds ~30 MB to every installer for ~5% of users. Revisit if the bootstrap is still a top complaint in v0.4. |
|
||||
| In-repo Markdown docs | Astro Starlight standalone site | When docs grow past ~20 pages and need full-text search. Tauri provides a precedent if/when we get there. |
|
||||
| In-repo Markdown docs | MkDocs / Material for MkDocs | **Avoid** for new sites — Material for MkDocs is in maintenance mode as of Nov 2025. |
|
||||
## What NOT to Use
|
||||
| Avoid | Why | Use Instead |
|
||||
|-------|-----|-------------|
|
||||
| `HfFolder.save_token()` directly | Older API; v1.x `login()` does the same plus git-credential integration and is the documented path | `huggingface_hub.login(token=val, add_to_git_credential=False)` |
|
||||
| Setting `HF_TOKEN` via shell rc files as the *only* persistence mechanism | Different per OS, fragile, opaque to the user, breaks in installer-launched processes that don't source shell rc | Write to `$HF_HOME/token` via `login()`. Document env var as override only. |
|
||||
| `setx` for HF token persistence | Doesn't propagate to current shell; common source of "I set it but it's empty" bug reports | `[Environment]::SetEnvironmentVariable(...,"User")` in PowerShell, or the in-app Settings field |
|
||||
| PAT-based GitHub Issues posting from OmniVoice | Would require shipping or asking for a token; breaks local-first promise | Prefilled-URL pattern (user submits from their browser) |
|
||||
| `sentry-tauri` for OmniVoice | Third-party telemetry endpoint — violates PROJECT.md constraint | Local-only `backend.log` rotation + opt-in prefilled-URL reporter |
|
||||
| `hf_transfer` for downloads | Deprecated in favor of `hf-xet` per HF docs | Default `huggingface_hub` (uses `hf-xet` automatically when available) |
|
||||
| `--python-preference managed` (default) without mirror config in restricted-network installers | Hits GitHub CDN, times out, user sees raw `uv` error | Configure `UV_PYTHON_INSTALL_MIRROR` + retry chain + `only-system` final fallback |
|
||||
| Material for MkDocs as a *new* docs choice | Entered maintenance mode November 2025 | If docs site is eventually needed, use Astro Starlight (Tauri precedent) |
|
||||
## Stack Patterns by Variant
|
||||
- Set `UV_PYTHON_INSTALL_MIRROR` to one of the gh-proxy URLs at install time
|
||||
- Set `UV_DEFAULT_INDEX=https://pypi.tuna.tsinghua.edu.cn/simple` (China) or document VPN requirement (Russia)
|
||||
- Fall back to `UV_PYTHON_PREFERENCE=only-system` if all mirrors fail
|
||||
- Increase `UV_HTTP_TIMEOUT=120`, `UV_HTTP_RETRIES=5`
|
||||
- Default path: in-app Settings field → `login()` → file at `$HF_HOME/token`
|
||||
- Power-user path: `export HF_TOKEN=...` in shell rc (documented but not promoted)
|
||||
- Both paths are read at HF library import time; env var wins on conflict
|
||||
- Default path: in-app "Report a bug" → prefilled GitHub Issues URL → user reviews + submits in browser
|
||||
- All optional capture toggles default ON except "include reproduction file" (privacy)
|
||||
- No path posts to any URL except `github.com/{owner}/{repo}/issues/new` (rendered locally as a URL, opened via `shell.open`)
|
||||
- `uv add supertonic` → new TTSBackend subclass in `backend/services/tts_backend.py`
|
||||
- Auto-detected and added to the engine picker in Settings
|
||||
- ~400 MB model download on first synthesize call, cached in `$HF_HUB_CACHE`
|
||||
- Existing IndexTTS/CosyVoice/etc. installs are untouched (no shared model weights)
|
||||
## Version Compatibility
|
||||
| Package A | Compatible With | Notes |
|
||||
|-----------|-----------------|-------|
|
||||
| `supertonic@1.3.1` | `onnxruntime>=1.17`, `numpy>=1.24`, `huggingface_hub>=0.20` | All deps already satisfied transitively by current `pyproject.toml`. |
|
||||
| `huggingface_hub>=1.12` | `transformers>=5.3.0` (current pin) | `HfFolder` retained as deprecated alias; `login()`/`get_token()` are the canonical APIs. |
|
||||
| `uv>=0.5` | `UV_PYTHON_INSTALL_MIRROR`, `UV_PYTHON_PREFERENCE` | Both env vars stable since uv 0.4.x. |
|
||||
| Tauri v2 + `@tauri-apps/api/shell` | `shell.open()` for the prefilled-URL pattern | Already in the desktop app; no new permission needed beyond what the existing "open external link" plugin grants. |
|
||||
## Sources
|
||||
- [Hugging Face Hub environment variables](https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables) — HIGH (verified against v1.12.1 docs, current 2026)
|
||||
- [Hugging Face Hub authentication API](https://huggingface.co/docs/huggingface_hub/en/package_reference/authentication) — HIGH (verified `login()` is the canonical 1.x API)
|
||||
- [Microsoft `setx` reference](https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/setx) — HIGH (confirms "current shell" gotcha)
|
||||
- [PowerShell `about_Environment_Variables`](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_environment_variables) — HIGH
|
||||
- [uv environment variables reference](https://docs.astral.sh/uv/reference/environment/) — HIGH (verified all mirror + retry env vars)
|
||||
- [uv issue #5224 — python-build-standalone mirror](https://github.com/astral-sh/uv/issues/5224) — HIGH (feature shipped)
|
||||
- [uv issue #14187 — venv on Chinese network](https://github.com/astral-sh/uv/issues/14187) — HIGH (confirms user pain, justifies fallback chain)
|
||||
- [uv `python-preference` semantics](https://github.com/astral-sh/uv/blob/main/docs/concepts/python-versions.md) — HIGH
|
||||
- [Supertone/supertonic-3 model card](https://huggingface.co/Supertone/supertonic-3) — HIGH (official, 99M params, 31 languages, OpenRAIL-M)
|
||||
- [supertone-inc/supertonic GitHub](https://github.com/supertone-inc/supertonic) — HIGH (official inference API)
|
||||
- [supertonic 1.3.1 on PyPI](https://pypi.org/project/supertonic/) — HIGH (released 2026-05-18, MIT code license; bumped from 1.2.3 after Phase 3 research)
|
||||
- [onnx-community/Supertonic-TTS-ONNX](https://huggingface.co/onnx-community/Supertonic-TTS-ONNX) — HIGH (ONNX file structure)
|
||||
- [GitHub Docs: Authenticating to the REST API](https://docs.github.com/en/rest/authentication/authenticating-to-the-rest-api) — HIGH
|
||||
- [GitHub Docs: Generating a user access token for a GitHub App](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-a-user-access-token-for-a-github-app) — HIGH (device flow reference)
|
||||
- [sindresorhus/new-github-issue-url](https://github.com/sindresorhus/new-github-issue-url) — HIGH (canonical prefilled-URL reference impl)
|
||||
- [sentry-tauri](https://github.com/timfish/sentry-tauri) — MEDIUM (reviewed, rejected on PROJECT.md constraint, not on quality)
|
||||
- [dautovri/mirrors-china](https://github.com/dautovri/mirrors-china) — MEDIUM (community-maintained, verify URLs are still live before pinning in production)
|
||||
- [Tauri 2 docs (Astro Starlight reference)](https://v2.tauri.app/) — HIGH (precedent for docs framework if we ever migrate)
|
||||
- [Docsio: Material for MkDocs entered maintenance mode Nov 2025](https://docsio.co/blog/mkdocs-material) — MEDIUM (third-party review, but signal aligns with the project's own GitHub commit activity)
|
||||
The May-2026 stack research that used to live here served five capabilities that have all since shipped (HF-token Settings panel, prefilled-URL bug reporting, uv mirror fallback for restricted networks, the Supertonic-3 engine, in-repo Markdown docs). Follow the patterns in the code itself; the durable *don'ts* that research established:
|
||||
|
||||
- **No third-party telemetry endpoints, ever** (`sentry-tauri` was evaluated and rejected) — bug reporting stays opt-in via prefilled GitHub-issue URLs.
|
||||
- **No PAT/token-based GitHub posting from the app** — the user submits from their own browser.
|
||||
- **Don't recommend `setx` for env vars on Windows** (silent truncation, no current-shell propagation) — use the in-app Settings panel or PowerShell `[Environment]::SetEnvironmentVariable`.
|
||||
- **Don't adopt Material for MkDocs** for any future docs site (maintenance mode since Nov 2025) — Astro Starlight is the precedent if docs ever outgrow the repo.
|
||||
- **`hf_transfer` is deprecated** — default `huggingface_hub` (hf-xet) handles downloads.
|
||||
|
||||
For anything new: prefer what's already pinned in `pyproject.toml` / `frontend/package.json`, and check `uv tree` for conflicts before adding a dependency.
|
||||
<!-- GSD:stack-end -->
|
||||
|
||||
<!-- GSD:conventions-start source:CONVENTIONS.md -->
|
||||
@@ -222,16 +68,9 @@ No project skills found. Add skills to any of: `.claude/skills/`, `.agents/skill
|
||||
<!-- GSD:skills-end -->
|
||||
|
||||
<!-- GSD:workflow-start source:GSD defaults -->
|
||||
## GSD Workflow Enforcement
|
||||
## Workflow
|
||||
|
||||
Before using Edit, Write, or other file-changing tools, start work through a GSD command so planning artifacts and execution context stay in sync.
|
||||
|
||||
Use these entry points:
|
||||
- `/gsd-quick` for small fixes, doc updates, and ad-hoc tasks
|
||||
- `/gsd-debug` for investigation and bug fixing
|
||||
- `/gsd-execute-phase` for planned phase work
|
||||
|
||||
Do not make direct repo edits outside a GSD workflow unless the user explicitly asks to bypass it.
|
||||
Direct repo edits are authorized (owner decision, 2026-07-08). The GSD command gate that used to live here referenced `/gsd-quick` / `/gsd-debug` / `/gsd-execute-phase` skills that are not installed in this environment; the owner chose to keep working directly rather than restore them. The working conventions that matter are in **Conventions** above — versioning, docs-sync, changelog, localization, fix quality, keep-main-green — plus: gate every merge on the "Tests (backend + frontend)" check passing and the PR being MERGEABLE, and check the open-PR queue before implementing any community-reported fix (contributors may have already submitted one).
|
||||
<!-- GSD:workflow-end -->
|
||||
|
||||
|
||||
|
||||
@@ -266,7 +266,7 @@ Professional-grade voice AI, minus the subscription and the cloud.
|
||||
|
||||
| | **Minimum** | **Recommended** |
|
||||
|---|---|---|
|
||||
| **OS** | Windows 10, macOS 12+ (Apple Silicon), Ubuntu 20.04+ | Any modern 64-bit OS |
|
||||
| **OS** | Windows 10, macOS 12+ (Apple Silicon), Ubuntu 24.04+ (glibc 2.39+) | Any modern 64-bit OS |
|
||||
| **RAM** | 8 GB | 16 GB+ |
|
||||
| **VRAM (GPU)** | 4 GB (auto-offloads TTS to CPU) | 8 GB+ (NVIDIA RTX 3060+) |
|
||||
| **Disk** | 10 GB free (models + cache) | 20 GB+ SSD |
|
||||
@@ -322,10 +322,10 @@ Professional-grade voice AI, minus the subscription and the cloud.
|
||||
|
||||
### 🎧 ASR Engines
|
||||
|
||||
**9 engines, all fully local** — they power dictation, video dubbing, and subtitles. **WhisperX** is the cross-platform default (~100 languages, word-level timing); the rest are opt-in and auto-detected. Switch in **Settings → ASR Engine** or via the `OMNIVOICE_ASR_BACKEND` env var.
|
||||
**10 engines** — they power dictation, video dubbing, and subtitles. **WhisperX** is the cross-platform default (~100 languages, word-level timing); the rest are opt-in and auto-detected. Switch in **Settings → Engines** (the ASR Engines table — same picker TTS has), or pin one with the `OMNIVOICE_ASR_BACKEND` env var (the env var wins over the Settings pick). Nine run fully on-device; one (OpenAI-compatible) is an optional remote client for pointing at Qwen3-ASR or another compatible server — see below.
|
||||
|
||||
<details>
|
||||
<summary><b>📊 The full lineup</b> — 9 engines, what each is best at, and compute-type notes</summary>
|
||||
<summary><b>📊 The full lineup</b> — 10 engines, what each is best at, and compute-type notes</summary>
|
||||
|
||||
<br/>
|
||||
|
||||
@@ -340,6 +340,7 @@ Professional-grade voice AI, minus the subscription and the cloud.
|
||||
| **Moonshine** | `moonshine` | English | Edge / low-latency, ONNX |
|
||||
| **FunASR** | `funasr` | 50+ | All-in-one multilingual — built-in VAD + inline speaker diarization (SenseVoice) |
|
||||
| **sherpa-onnx** (live dictation) | `sherpa-onnx-asr` | 25 EU + 90+ | Live, faster-than-real-time dictation — small streaming/offline ONNX models (Parakeet TDT v3/v2, streaming Zipformer & Paraformer, Whisper Tiny), CPU, identical on macOS / Windows / Linux. Picked per-model in **Settings → Voice**. |
|
||||
| **OpenAI-compatible** ⚠️ remote | `openai-compat-asr` | Server-dependent | A path to **Qwen3-ASR** today (self-hosted server, no transformers wait), any OpenAI-compatible transcription endpoint, or OpenAI's own API — no install, configure in **Settings → Models**. Audio leaves your machine to whatever server you point it at; see [docs/engines/openai-compatible-asr.md](docs/engines/openai-compatible-asr.md). |
|
||||
|
||||
> Whisper-family engines cover ~100 languages; **FunASR / SenseVoice** adds an all-in-one multilingual path with built-in voice-activity detection and inline speaker diarization. **sherpa-onnx** powers the live dictation model picker — you talk and text appears as you speak. Every engine runs on-device — no API keys, no cloud.
|
||||
|
||||
@@ -396,6 +397,20 @@ print(result.text)
|
||||
|
||||
Want the whole surface (100+ endpoints)? The full REST API reference is embedded in the app — **Settings → OpenAPI Reference** (Scalar-powered), or the `{}` button in the footer.
|
||||
|
||||
### 📓 Run on Google Colab (community)
|
||||
|
||||
No local GPU? A community member ([@shakib30](https://github.com/shakib30)) maintains a working Colab notebook: [shakib30/OmniVoice-Studio-google-colab](https://github.com/shakib30/OmniVoice-Studio-google-colab). Community-maintained — issues with the notebook go there; issues with OmniVoice itself come here.
|
||||
|
||||
### 🤝 Agent Skills
|
||||
|
||||
Teach your AI agent (Claude Code, Cursor, Codex, …) to use OmniVoice with one command:
|
||||
|
||||
```sh
|
||||
npx skills add debpalash/omnivoice-studio
|
||||
```
|
||||
|
||||
Ships two [skills](https://skills.sh): **`omnivoice`** — speak and transcribe through your local install (including your cloned voices) from any agent, free and offline; and **`oss-maintainer`** — the maintainer methodology this project is run with, for anyone running their own OSS project with an agent.
|
||||
|
||||
---
|
||||
|
||||
## 🗺️ Roadmap
|
||||
@@ -524,7 +539,15 @@ Yes please — bug fixes, new TTS engine adapters, UI improvements, docs, transl
|
||||
<details>
|
||||
<summary><b>Is this really as good as ElevenLabs?</b></summary>
|
||||
<br/>
|
||||
For voice cloning and dubbing, yes — OmniVoice uses a state-of-the-art diffusion TTS model with 646 languages (ElevenLabs supports 32). Quality is comparable for most use cases. Where ElevenLabs wins is in their polished cloud API and pre-made voice library. OmniVoice wins on privacy, cost, language coverage, and customizability.
|
||||
Honest answer: <b>it depends on what you're doing.</b>
|
||||
|
||||
<b>Where OmniVoice is genuinely competitive:</b> voice cloning from a clean reference clip (state-of-the-art open diffusion TTS), language coverage (646 languages vs. their 32), and everything structural — no per-character billing, no usage caps, no audio leaving your machine, full pipeline customizability (10 TTS engines, 10 ASR engines, your choice of translation).
|
||||
|
||||
<b>Where ElevenLabs still wins:</b> out-of-the-box consistency and polish, especially for English TTS. Their one model is heavily tuned; our quality depends on which engine you pick, your hardware, and — for cloning — the reference audio (a dry, close-mic clip clones dramatically better than a noisy or echoey one).
|
||||
|
||||
<b>For dubbing specifically:</b> a dub is a chain — transcription → translation → cloning → synthesis — and the output is only as good as its weakest link on <i>your</i> source material. Noisy or accented source audio degrades transcription, which degrades everything downstream; some language pairs translate better than others. If parts of a dub come out incoherent, check the segment table's <i>original</i> text first: if the transcription is already wrong there, switch the ASR engine (Settings → Engines) or use cleaner source audio — that's usually the fix, not the voice.
|
||||
|
||||
Try it on your real material — it's free and takes one download. Many users find it replaces ElevenLabs outright; some keep both for different jobs. Both outcomes are fine with us.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
|
||||
@@ -69,6 +69,11 @@ hiddenimports = [
|
||||
# Pipeline
|
||||
'yt_dlp', 'demucs', 'demucs.separate',
|
||||
|
||||
# Numbers→words for the pre-TTS text normalization pass
|
||||
# (services/text_normalization.py). Imported inside a function (lazy),
|
||||
# so pin it explicitly rather than trusting the tracer.
|
||||
'num2words',
|
||||
|
||||
# OmniVoice's own package
|
||||
'omnivoice', 'omnivoice.models', 'omnivoice.models.omnivoice',
|
||||
]
|
||||
|
||||
@@ -311,34 +311,60 @@ async def _prepare_synth(default_voice: str | None, language: str | None = None)
|
||||
return info["synth"], info["sample_rate"], resolve, engine_id
|
||||
|
||||
|
||||
def _render_chapter_cached(chapter, synth, sr, engine_id, resolve, cache_dir, lexicon=None):
|
||||
def _render_chapter_cached(chapter, synth, sr, engine_id, resolve, cache_dir, lexicon=None,
|
||||
language=None):
|
||||
"""Render one chapter, content-addressed so a re-run reuses it (resume).
|
||||
|
||||
Returns ``(wav_path, duration_s, was_cached)``. The WAV lives at
|
||||
``cache_dir/<key>.wav`` where ``key`` is :func:`chapter_cache_key` over the
|
||||
chapter's spans + sample rate + engine + each voice's resolved signature
|
||||
(+ the lexicon, so a lexicon edit re-renders), so an unchanged chapter is
|
||||
never re-synthesized. Runs in the GPU-pool executor.
|
||||
Returns ``(wav_path, duration_s, was_cached, seg_stats)``. Two cache
|
||||
layers:
|
||||
|
||||
* Outer — the WAV at ``cache_dir/<key>.wav`` where ``key`` is
|
||||
:func:`chapter_cache_key` over the chapter's spans + sample rate +
|
||||
engine + each voice's resolved signature (+ the lexicon, so a lexicon
|
||||
edit re-renders). A fully-unchanged chapter hits here and never touches
|
||||
segment files; the key derivation is unchanged, so chapter caches
|
||||
written by released versions keep hitting. ``seg_stats`` is ``None``.
|
||||
* Inner — on a chapter miss, each spoken span goes through the
|
||||
:class:`services.longform_render.SegmentCache` under
|
||||
``cache_dir/segments``: cached segments load from disk, only the
|
||||
edited/missing ones synthesize, and each fresh segment persists the
|
||||
moment it renders (an interrupted chapter resumes from them).
|
||||
``seg_stats`` is ``{"total": spoken_spans, "cached": reused}``.
|
||||
|
||||
Span text is normalized (``services.text_normalization``) up front — BEFORE
|
||||
either cache key and BEFORE ``synthesize_chapter``'s lexicon pass, so the
|
||||
per-project dictionary operates on normalized text and toggling / changing
|
||||
normalization output naturally invalidates cached chapters and segments.
|
||||
|
||||
Runs in the GPU-pool executor.
|
||||
"""
|
||||
import json
|
||||
import wave
|
||||
|
||||
from services.audio_io import atomic_save_wav
|
||||
from services.longform_render import chapter_cache_key
|
||||
from services.audiobook import Span
|
||||
from services.longform_render import SegmentCache, chapter_cache_key
|
||||
from services.pronunciation import normalize_lexicon
|
||||
from services.text_normalization import normalize_for_tts
|
||||
|
||||
spans = [Span(voice_id=s.voice_id, text=normalize_for_tts(s.text, language),
|
||||
pause_ms_after=s.pause_ms_after, speed=getattr(s, "speed", None))
|
||||
for s in chapter.spans]
|
||||
spans_tuples = [(s.voice_id, s.text, s.pause_ms_after, getattr(s, "speed", None))
|
||||
for s in chapter.spans]
|
||||
sig: dict = {}
|
||||
for s in chapter.spans:
|
||||
for s in spans]
|
||||
voice_sigs: dict = {}
|
||||
for s in spans:
|
||||
k = s.voice_id or ""
|
||||
if k not in sig:
|
||||
if k not in voice_sigs:
|
||||
v = resolve(s.voice_id)
|
||||
sig[k] = f"{v.get('ref_audio')}|{v.get('ref_text')}|{v.get('instruct')}|{v.get('seed')}"
|
||||
voice_sigs[k] = f"{v.get('ref_audio')}|{v.get('ref_text')}|{v.get('instruct')}|{v.get('seed')}"
|
||||
sig: dict = dict(voice_sigs)
|
||||
lex_sig = ""
|
||||
if lexicon:
|
||||
# Fold the lexicon into the cache key so editing pronunciations
|
||||
# invalidates cached chapters (reserved key can't collide with a voice id).
|
||||
sig["\x00lexicon"] = json.dumps(normalize_lexicon(lexicon), sort_keys=True)
|
||||
lex_sig = json.dumps(normalize_lexicon(lexicon), sort_keys=True)
|
||||
sig["\x00lexicon"] = lex_sig
|
||||
key = chapter_cache_key(spans_tuples, sample_rate=sr, engine_id=engine_id, voice_sig=sig)
|
||||
wav_path = os.path.join(cache_dir, f"{key}.wav")
|
||||
|
||||
@@ -346,13 +372,17 @@ def _render_chapter_cached(chapter, synth, sr, engine_id, resolve, cache_dir, le
|
||||
try:
|
||||
with wave.open(wav_path, "rb") as w:
|
||||
dur = w.getnframes() / float(w.getframerate() or sr)
|
||||
return wav_path, dur, True
|
||||
return wav_path, dur, True, None
|
||||
except Exception:
|
||||
pass # corrupt cache entry — fall through and re-render
|
||||
|
||||
audio, dur = synthesize_chapter(chapter.spans, synth, sr, lexicon=lexicon)
|
||||
seg_cache = SegmentCache(cache_dir, sample_rate=sr, engine_id=engine_id,
|
||||
voice_sig=voice_sigs, extra_sig=lex_sig)
|
||||
audio, dur = synthesize_chapter(spans, synth, sr, lexicon=lexicon,
|
||||
segment_cache=seg_cache)
|
||||
atomic_save_wav(wav_path, audio, sr)
|
||||
return wav_path, dur, False
|
||||
return wav_path, dur, False, {"total": seg_cache.hits + seg_cache.misses,
|
||||
"cached": seg_cache.hits}
|
||||
|
||||
|
||||
class AudiobookPreviewRequest(BaseModel):
|
||||
@@ -383,14 +413,15 @@ async def audiobook_preview(req: AudiobookPreviewRequest) -> dict:
|
||||
chapter = plan.chapters[req.chapter_index]
|
||||
cache_dir = os.path.join(OUTPUTS_DIR, "longform_cache") # shared with _render_longform_sse
|
||||
os.makedirs(cache_dir, exist_ok=True)
|
||||
resolved_lang = _resolve_default_language(req.language, req.default_voice)
|
||||
synth, sr, resolve, engine_id = await _prepare_synth(
|
||||
req.default_voice,
|
||||
language=_resolve_default_language(req.language, req.default_voice),
|
||||
language=resolved_lang,
|
||||
)
|
||||
loop = asyncio.get_running_loop()
|
||||
wav_path, dur, was_cached = await loop.run_in_executor(
|
||||
wav_path, dur, was_cached, _seg_stats = await loop.run_in_executor(
|
||||
_gpu_pool, _render_chapter_cached, chapter, synth, sr, engine_id, resolve, cache_dir,
|
||||
req.lexicon,
|
||||
req.lexicon, resolved_lang,
|
||||
)
|
||||
return {
|
||||
"output": os.path.relpath(wav_path, OUTPUTS_DIR), # served via /audio
|
||||
@@ -495,8 +526,9 @@ async def _render_longform_sse(
|
||||
loop = asyncio.get_running_loop()
|
||||
|
||||
try:
|
||||
resolved_lang = _resolve_default_language(language, default_voice)
|
||||
synth, sr, resolve, engine_id = await _prepare_synth(
|
||||
default_voice, language=_resolve_default_language(language, default_voice)
|
||||
default_voice, language=resolved_lang
|
||||
)
|
||||
|
||||
total = len(plan.chapters)
|
||||
@@ -508,9 +540,10 @@ async def _render_longform_sse(
|
||||
|
||||
for i, chapter in enumerate(plan.chapters):
|
||||
try:
|
||||
wav_path, dur, was_cached = await loop.run_in_executor(
|
||||
wav_path, dur, was_cached, seg_stats = await loop.run_in_executor(
|
||||
_gpu_pool, _render_chapter_cached,
|
||||
chapter, synth, sr, engine_id, resolve, cache_dir, lexicon,
|
||||
resolved_lang,
|
||||
)
|
||||
except Exception: # isolate a bad chapter — keep going
|
||||
logger.warning("[%s] chapter %d (%s) failed to render",
|
||||
@@ -522,9 +555,15 @@ async def _render_longform_sse(
|
||||
chapter_files.append(wav_path)
|
||||
chapters_meta.append((chapter.title, int(round(dur * 1000))))
|
||||
cached_n += 1 if was_cached else 0
|
||||
yield _emit({"type": "chapter", "index": i, "total": total,
|
||||
"title": chapter.title, "duration_s": round(dur, 2),
|
||||
"cached": was_cached})
|
||||
ev = {"type": "chapter", "index": i, "total": total,
|
||||
"title": chapter.title, "duration_s": round(dur, 2),
|
||||
"cached": was_cached}
|
||||
if seg_stats is not None:
|
||||
# Additive fields (old clients ignore them): segment-level
|
||||
# reuse inside a re-rendered chapter.
|
||||
ev["segments"] = seg_stats["total"]
|
||||
ev["cached_segments"] = seg_stats["cached"]
|
||||
yield _emit(ev)
|
||||
|
||||
if not chapter_files:
|
||||
yield _emit({"type": "error", "error": "all chapters failed to render"})
|
||||
|
||||
@@ -287,6 +287,13 @@ async def _run_batch_pipeline(job_id: str, job: dict):
|
||||
continue
|
||||
|
||||
def _gen(text=seg_text, lang=target_lang, dur=seg_duration):
|
||||
# Normalize once at the segment's text→engine choke point —
|
||||
# the same pre-pass as /generate and dub_generate's _gen.
|
||||
# `lang` is the job's target language code. Pref-gated,
|
||||
# idempotent, never raises.
|
||||
from services.text_normalization import normalize_for_tts
|
||||
text = normalize_for_tts(text, lang)
|
||||
|
||||
ref_audio = None
|
||||
ref_text = None
|
||||
|
||||
|
||||
@@ -1006,6 +1006,24 @@ async def dub_transcribe_stream(
|
||||
clones = done.pop().result()
|
||||
break
|
||||
yield _sse_event("ping", {})
|
||||
if clones:
|
||||
from services.speaker_clone import refine_ref_texts
|
||||
# Bound the re-transcribe like every other ASR dispatch in
|
||||
# this file (#730): a wedged transcribe would otherwise hold
|
||||
# the GPU-pool worker forever and starve later work into a
|
||||
# "can't reach backend". On timeout the guard resets the pool
|
||||
# and raises — keep the original (unrefined) clones, matching
|
||||
# refine_ref_text's own "failure is a strict no-op" fallback.
|
||||
try:
|
||||
clones = await run_transcribe_guarded(
|
||||
_gpu_pool,
|
||||
lambda: refine_ref_texts(clones, _asr_backend),
|
||||
what="Dub clone ref-text refine",
|
||||
)
|
||||
except ASRTimeoutError as e:
|
||||
logger.warning(
|
||||
"clone ref-text refine timed out; keeping original ref_text: %s", e
|
||||
)
|
||||
# Wave 3.2: per-segment clone refs. Cut each long-enough segment's
|
||||
# own reference from the vocals so the dub of each line matches the
|
||||
# prosody of its source line. Short lines fall back to the
|
||||
@@ -1025,6 +1043,19 @@ async def dub_transcribe_stream(
|
||||
),
|
||||
)
|
||||
if seg_clones:
|
||||
from services.speaker_clone import refine_ref_texts
|
||||
# Same guard as the per-speaker refine above (#730):
|
||||
# keep the original seg_clones on a wedge/timeout.
|
||||
try:
|
||||
seg_clones = await run_transcribe_guarded(
|
||||
_gpu_pool,
|
||||
lambda: refine_ref_texts(seg_clones, _asr_backend),
|
||||
what="Dub segment ref-text refine",
|
||||
)
|
||||
except ASRTimeoutError as e:
|
||||
logger.warning(
|
||||
"segment ref-text refine timed out; keeping original ref_text: %s", e
|
||||
)
|
||||
job["segment_clones"] = seg_clones
|
||||
except Exception as e:
|
||||
logger.warning("per-segment clone refs skipped: %s", e)
|
||||
|
||||
@@ -338,6 +338,13 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
# retain every generated tensor in RAM until final assembly.
|
||||
_pending_seg_writes: list[tuple] = []
|
||||
|
||||
# Calibration records for the pre-synthesis duration planner
|
||||
# (services/duration_planner.py): text length + the NATURAL-rate TTS
|
||||
# duration of every freshly synthesized segment. Only meaningful for
|
||||
# the natural-rate strategies — strict_slot forces the audio to the
|
||||
# slot length, which would poison the observed chars-per-second.
|
||||
_natural_dur_records: dict[str, dict] = {}
|
||||
|
||||
# Phase 4.1 bench instrumentation: measure where incremental time goes.
|
||||
# Only prints when regen_only is active (real-user incremental path).
|
||||
_t_start = time.perf_counter()
|
||||
@@ -429,6 +436,12 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
continue
|
||||
|
||||
def _gen(text, lang, instruct_str, dur_s, nstep, cfg, spd, profile_id, effect_preset):
|
||||
# Normalize once at the segment's text→engine choke point
|
||||
# (covers the OOM-retry generate below too, which reuses this
|
||||
# closure's `text`). Pref-gated, idempotent, never raises.
|
||||
from services.text_normalization import normalize_for_tts
|
||||
text = normalize_for_tts(text, lang)
|
||||
|
||||
ref_audio = None
|
||||
ref_text = None
|
||||
used_seed = None
|
||||
@@ -673,6 +686,15 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
|
||||
sync_scores.append(sync_ratio)
|
||||
|
||||
# Duration-planner calibration sample: this text length spoke
|
||||
# for this long at natural rate. Keyed by stable seg id and
|
||||
# merged into the per-language job map after the loop.
|
||||
if strategy != "strict_slot" and seg.text.strip() and generated_dur > 0:
|
||||
_natural_dur_records[str(seg_id)] = {
|
||||
"chars": len(seg.text.strip()),
|
||||
"dur": round(generated_dur, 4),
|
||||
}
|
||||
|
||||
# Build the fingerprint now (cheap) but defer the disk write
|
||||
# and job flush to the batch-write phase after the GPU loop.
|
||||
_seg_fp = None
|
||||
@@ -773,6 +795,13 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
hashes[_sid] = _fp
|
||||
quality_map[_sid] = _nstep
|
||||
job["seg_hashes"] = dict(hashes)
|
||||
# Duration-planner calibration: per-language (chars, natural dur)
|
||||
# records. update() (not replace) so partial regens keep accumulating
|
||||
# samples from earlier runs of this track.
|
||||
if _natural_dur_records:
|
||||
job.setdefault("seg_natural_durs_by_lang", {}).setdefault(
|
||||
lang_code, {},
|
||||
).update(_natural_dur_records)
|
||||
# Single job flush instead of one per 8 segments.
|
||||
_save_job(job_id, job)
|
||||
_t_diskw = time.perf_counter() - _t_diskw_0
|
||||
@@ -1246,8 +1275,11 @@ async def preview_segment(job_id: str, req: SegmentPreviewRequest):
|
||||
instruct_str = row["instruct"]
|
||||
|
||||
lang = req.language if req.language != "Auto" else None
|
||||
# Same normalization as the full dub render above, so a preview
|
||||
# sounds exactly like the final segment. Pref-gated, never raises.
|
||||
from services.text_normalization import normalize_for_tts
|
||||
audio_out = backend.generate(
|
||||
text=req.text,
|
||||
text=normalize_for_tts(req.text, lang),
|
||||
language=lang,
|
||||
ref_audio=ref_audio,
|
||||
ref_text=ref_text,
|
||||
|
||||
@@ -9,7 +9,7 @@ from fastapi.responses import JSONResponse
|
||||
from schemas.requests import TranslateRequest
|
||||
from services.model_manager import _cpu_pool, _gpu_pool
|
||||
from services.translator import cinematic_available, cinematic_refine_many, _cinematic_budget
|
||||
from api.routers.dub_core import _get_job
|
||||
from api.routers.dub_core import _get_job, _save_job
|
||||
|
||||
router = APIRouter()
|
||||
logger = logging.getLogger("omnivoice.api")
|
||||
@@ -202,6 +202,46 @@ def _resolve_source_lang(req: TranslateRequest) -> str:
|
||||
return _guess_lang_from_text(getattr(req, "segments", None)) or "en"
|
||||
|
||||
|
||||
def _resolve_translation_context(req, client, model_name: str, timeout: float,
|
||||
src_lang: str) -> Optional[dict]:
|
||||
"""Cached auto-glossary context (theme + terms) for this job/target.
|
||||
|
||||
Cache lives on the dub job dict (``job["translation_context"][target]``)
|
||||
and persists through the existing ``job_data`` JSON blob via ``_save_job``
|
||||
— no schema change. A transcript fingerprint keys the cache so an edited
|
||||
transcript re-extracts; an unchanged transcript costs zero LLM calls on
|
||||
re-translate. Any failure returns None — translation proceeds without
|
||||
context, never fails because of it. Blocking; run in an executor.
|
||||
"""
|
||||
from services import translation_quality as tq
|
||||
|
||||
texts = [(s.text or "") for s in req.segments]
|
||||
fp = tq.transcript_fingerprint(texts)
|
||||
job = _get_job(req.job_id) if getattr(req, "job_id", None) else None
|
||||
if job is not None:
|
||||
cached = (job.get("translation_context") or {}).get(req.target_lang)
|
||||
if isinstance(cached, dict) and cached.get("fingerprint") == fp:
|
||||
return cached
|
||||
ctx = tq.extract_context_sync(
|
||||
client, model_name, timeout,
|
||||
segment_texts=texts,
|
||||
source_lang=src_lang,
|
||||
target_lang=req.target_lang,
|
||||
source_name=LANG_NAMES.get(src_lang, src_lang),
|
||||
target_name=LANG_NAMES.get(req.target_lang, req.target_lang),
|
||||
)
|
||||
if ctx is None:
|
||||
return None
|
||||
ctx = {**ctx, "fingerprint": fp}
|
||||
if job is not None:
|
||||
try:
|
||||
job.setdefault("translation_context", {})[req.target_lang] = ctx
|
||||
_save_job(req.job_id, job)
|
||||
except Exception: # noqa: BLE001 — persistence is best-effort
|
||||
logger.debug("translation context persist skipped", exc_info=True)
|
||||
return ctx
|
||||
|
||||
|
||||
def _unload_nllb():
|
||||
"""Release NLLB VRAM so TTS model can reload."""
|
||||
global _nllb_model, _nllb_tokenizer
|
||||
@@ -366,6 +406,27 @@ async def dub_translate(req: TranslateRequest):
|
||||
)
|
||||
return JSONResponse(status_code=400, content={"error": friendly})
|
||||
|
||||
from services import translation_quality as tq
|
||||
|
||||
# Two-stage quality toggles. None (old clients) = ON — an LLM
|
||||
# translator is active on this branch by definition.
|
||||
auto_glossary_on = req.auto_glossary if req.auto_glossary is not None else True
|
||||
reflect_on = req.reflect if req.reflect is not None else True
|
||||
|
||||
# Stage 1 — auto-glossary: ONE pass over the full transcript for a
|
||||
# theme summary + terminology map (cached per job/target/transcript),
|
||||
# merged with the user's manual glossary (user entries win) and
|
||||
# injected into every per-segment prompt below. With the toggle off
|
||||
# the manual glossary still rides along — that costs no extra call.
|
||||
auto_ctx = None
|
||||
if auto_glossary_on:
|
||||
auto_ctx = await loop.run_in_executor(
|
||||
_cpu_pool, _resolve_translation_context,
|
||||
req, client, model_name, llm_timeout, src_lang,
|
||||
)
|
||||
merged_terms = tq.merge_glossary(req.glossary, (auto_ctx or {}).get("terms"))
|
||||
context_extra = tq.context_clause((auto_ctx or {}).get("theme", ""), merged_terms)
|
||||
|
||||
def _build_prompt(src_code: str, tgt_code: str) -> str:
|
||||
"""Build a system prompt that resists hallucinations on small
|
||||
local LLMs. Three things matter:
|
||||
@@ -396,13 +457,19 @@ async def dub_translate(req: TranslateRequest):
|
||||
dia_clause = ""
|
||||
if req.dialect and str(req.dialect).lower().startswith(str(tgt_code).lower()[:2]):
|
||||
dia_clause = dialect_clause(req.dialect)
|
||||
return (
|
||||
base = (
|
||||
f"You are a professional dubbing translator. "
|
||||
f"Translate the user's text from {src_name} into "
|
||||
f"{tgt_name}.{script_clause}{dia_clause} "
|
||||
f"Reply ONLY with the translated {tgt_name} text, do not "
|
||||
f"add quotes, notes, headers, explanations, or commentary."
|
||||
)
|
||||
# Auto-glossary theme + merged terminology (user terms win) —
|
||||
# every segment prompt carries the same brief, so recurring
|
||||
# names/terms come out consistent across the whole dub.
|
||||
if context_extra:
|
||||
base = base + "\n\n" + context_extra
|
||||
return base
|
||||
|
||||
def _translate_llm(seg):
|
||||
if not seg.text or not seg.text.strip():
|
||||
@@ -446,6 +513,33 @@ async def dub_translate(req: TranslateRequest):
|
||||
seg.id, attempt + 1, last_err,
|
||||
)
|
||||
continue
|
||||
# Stage 2 — reflect pass: critique→rewrite the direct
|
||||
# translation into natural spoken dialogue. Returns None
|
||||
# on ANY failure/timeout/divergence, in which case the
|
||||
# direct translation stands — refinement can never fail
|
||||
# a segment that already translated fine. The belt-and-
|
||||
# braces except keeps that guarantee even if the helper
|
||||
# itself ever raised: without it, the enclosing attempt
|
||||
# handler would burn a retry on a segment that already
|
||||
# translated successfully.
|
||||
if reflect_on:
|
||||
polished = None
|
||||
try:
|
||||
polished = tq.reflect_translation_sync(
|
||||
client, model_name, llm_timeout,
|
||||
source_text=seg.text,
|
||||
direct_text=out_text,
|
||||
source_lang=src_lang,
|
||||
target_lang=tgt_code,
|
||||
target_name=LANG_NAMES.get(tgt_code, tgt_code),
|
||||
extra_clause=context_extra,
|
||||
)
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.warning("reflect pass skipped for %s: %s",
|
||||
seg.id, e)
|
||||
if polished:
|
||||
return {"id": seg.id, "text": polished,
|
||||
"literal": out_text}
|
||||
return {"id": seg.id, "text": out_text}
|
||||
except Exception as e:
|
||||
last_err = f"{type(e).__name__}: {e}"
|
||||
@@ -624,6 +718,132 @@ async def dub_translate(req: TranslateRequest):
|
||||
return JSONResponse(status_code=500, content={"error": str(e)})
|
||||
|
||||
|
||||
def _stamp_duration_plan(rows, req) -> None:
|
||||
"""Attach a pre-synthesis duration-plan verdict to every row (in place).
|
||||
|
||||
Pure planning (services/duration_planner.py): estimate the natural
|
||||
speech duration of each row's FINAL text — self-calibrated from this
|
||||
job's already-synthesized segments when possible — and classify it
|
||||
against slot + borrowable gap using fit_planner's own caps. The verdict
|
||||
rides on the row as ``plan`` so the segment table can badge tight/
|
||||
impossible segments BEFORE any GPU time is spent. Informational only —
|
||||
generation is never blocked. Never raises.
|
||||
"""
|
||||
try:
|
||||
from services.duration_planner import calibration_from_job, classify_segments
|
||||
|
||||
timed = [
|
||||
s for s in req.segments
|
||||
if getattr(s, "start", None) is not None and getattr(s, "end", None) is not None
|
||||
]
|
||||
if not timed:
|
||||
return # old client — no timeline info, no plan
|
||||
text_by_id = {str(r["id"]): (r.get("text") or "") for r in rows}
|
||||
segs = sorted(
|
||||
(
|
||||
{
|
||||
"id": str(s.id),
|
||||
"start": float(s.start),
|
||||
"end": float(s.end),
|
||||
"text": text_by_id.get(str(s.id), ""),
|
||||
}
|
||||
for s in timed
|
||||
),
|
||||
key=lambda d: d["start"],
|
||||
)
|
||||
calib = None
|
||||
total_dur = 0.0
|
||||
if getattr(req, "job_id", None):
|
||||
job = _get_job(req.job_id)
|
||||
if job:
|
||||
calib = calibration_from_job(job, req.target_lang)
|
||||
total_dur = float(job.get("duration") or 0.0)
|
||||
verdicts = {
|
||||
v["id"]: v
|
||||
for v in classify_segments(
|
||||
segs, req.target_lang, calibration=calib, total_dur_s=total_dur,
|
||||
)
|
||||
}
|
||||
for row in rows:
|
||||
v = verdicts.get(str(row["id"]))
|
||||
if v is None or row.get("error") or not (row.get("text") or "").strip():
|
||||
continue
|
||||
row["plan"] = {
|
||||
"status": v["status"],
|
||||
"est_dur_s": v["est_dur_s"],
|
||||
"available_s": v["available_s"],
|
||||
"est_overrun_s": v["est_overrun_s"],
|
||||
"calibrated": v["calibrated"],
|
||||
}
|
||||
except Exception as e: # noqa: BLE001 — planning must never sink a translate
|
||||
logger.debug("duration-plan stamping skipped: %s", e)
|
||||
|
||||
|
||||
async def _apply_condense_pass(rows, req, loop) -> None:
|
||||
"""Opt-in LLM condensation for ``impossible`` rows (in place).
|
||||
|
||||
Fans ``condense_for_slot`` out on the CPU pool under the same wall-clock
|
||||
budget the cinematic phase uses, so a slow LLM can't hang the translate.
|
||||
Suggestions land as ``plan.suggested_text`` — the user applies them per
|
||||
segment; the row's ``text`` is never touched here. Every failure mode
|
||||
(no LLM, LLM error, divergent reply, budget) degrades to no suggestion.
|
||||
"""
|
||||
targets = [
|
||||
row for row in rows
|
||||
if (row.get("plan") or {}).get("status") == "impossible"
|
||||
and (row.get("text") or "").strip() and not row.get("error")
|
||||
]
|
||||
if not targets:
|
||||
return
|
||||
try:
|
||||
from services.duration_planner import calibration_from_job, condense_for_slot
|
||||
|
||||
calib = None
|
||||
if getattr(req, "job_id", None):
|
||||
job = _get_job(req.job_id)
|
||||
if job:
|
||||
calib = calibration_from_job(job, req.target_lang)
|
||||
source_by_id = {str(s.id): s.text for s in req.segments}
|
||||
sem = asyncio.Semaphore(int(os.environ.get("OMNIVOICE_LLM_CONCURRENCY", "6")))
|
||||
|
||||
async def _one(row):
|
||||
async with sem:
|
||||
res = await loop.run_in_executor(
|
||||
_cpu_pool,
|
||||
lambda: condense_for_slot(
|
||||
row["text"],
|
||||
available_s=float(row["plan"]["available_s"]),
|
||||
target_lang=req.target_lang,
|
||||
source_text=source_by_id.get(str(row["id"])),
|
||||
calibration=calib,
|
||||
),
|
||||
)
|
||||
if res.get("applied") and res.get("text"):
|
||||
row["plan"]["suggested_text"] = res["text"]
|
||||
row["plan"]["suggested_est_dur_s"] = res.get("est_dur_s")
|
||||
|
||||
tasks = [asyncio.ensure_future(_one(row)) for row in targets]
|
||||
budget = _cinematic_budget()
|
||||
done, pending = await asyncio.wait(
|
||||
tasks, timeout=budget if budget and budget > 0 else None,
|
||||
)
|
||||
for task in pending:
|
||||
task.cancel() # abandon the executor thread (#730 pattern)
|
||||
for task in done:
|
||||
exc = task.exception()
|
||||
if exc is not None:
|
||||
logger.warning("condense pass segment failed: %s", exc)
|
||||
except Exception as e: # noqa: BLE001 — a suggestion pass must never sink a translate
|
||||
logger.warning("condense pass skipped: %s", e)
|
||||
|
||||
|
||||
async def _finalize_duration_plan(rows, req, loop) -> None:
|
||||
"""Stamp plan verdicts on the FINAL row texts, then (opt-in) condense."""
|
||||
_stamp_duration_plan(rows, req)
|
||||
if getattr(req, "condense", False):
|
||||
await _apply_condense_pass(rows, req, loop)
|
||||
|
||||
|
||||
def _stamp_predicted_rate_ratio(translated, req) -> None:
|
||||
"""Stamp a predicted ``rate_ratio`` on every row that has a known slot.
|
||||
|
||||
@@ -717,8 +937,10 @@ async def _maybe_cinematic(translated, req, src_lang, loop, *, already_llm=False
|
||||
"quality_used": "fast",
|
||||
**_dialect_flags(req, applied=(already_llm and bool(dialect_hint)))}
|
||||
|
||||
# Fast (and anything unrecognised) returns the plain translation unchanged.
|
||||
# Fast (and anything unrecognised) returns the plain translation unchanged
|
||||
# (plus the pre-synthesis duration-plan badges — no LLM needed for those).
|
||||
if quality not in ("cinematic", "autofit"):
|
||||
await _finalize_duration_plan(translated, req, loop)
|
||||
return base
|
||||
|
||||
source_by_id: dict[str, str] = {str(s.id): s.text for s in req.segments}
|
||||
@@ -738,15 +960,19 @@ async def _maybe_cinematic(translated, req, src_lang, loop, *, already_llm=False
|
||||
if already_llm:
|
||||
merged = []
|
||||
for row in translated:
|
||||
# A reflect-pass row already carries its pre-polish direct
|
||||
# translation as `literal` — keep it instead of clobbering.
|
||||
out = {"id": row["id"],
|
||||
"text": row.get("text", "") or "",
|
||||
"literal": row.get("text", "") or ""}
|
||||
"literal": row.get("literal") or row.get("text", "") or ""}
|
||||
if row.get("error"):
|
||||
out["error"] = row["error"]
|
||||
if "rate_ratio" in row:
|
||||
out["rate_ratio"] = row["rate_ratio"]
|
||||
merged.append(out)
|
||||
await _apply_fit_pass(merged, req, slots_by_id, source_by_id, quality, loop, deadline)
|
||||
# Plan AFTER the fit pass — verdicts must describe the final text.
|
||||
await _finalize_duration_plan(merged, req, loop)
|
||||
return {"translated": merged, "target_lang": req.target_lang,
|
||||
"source_lang": src_lang, "quality_used": quality,
|
||||
**_dialect_flags(req, applied=bool(dialect_hint))}
|
||||
@@ -756,6 +982,7 @@ async def _maybe_cinematic(translated, req, src_lang, loop, *, already_llm=False
|
||||
if not cinematic_available():
|
||||
logger.warning("%s requested but no LLM configured — returning Fast result.", quality)
|
||||
base["cinematic_skipped"] = "no-llm-configured"
|
||||
await _finalize_duration_plan(translated, req, loop)
|
||||
return base
|
||||
|
||||
directions: dict[str, str] = {
|
||||
@@ -774,6 +1001,7 @@ async def _maybe_cinematic(translated, req, src_lang, loop, *, already_llm=False
|
||||
pairs.append((seg_id, source_by_id.get(seg_id, ""), literal))
|
||||
|
||||
if not pairs:
|
||||
await _finalize_duration_plan(translated, req, loop)
|
||||
return base
|
||||
|
||||
refined = await cinematic_refine_many(
|
||||
@@ -810,6 +1038,9 @@ async def _maybe_cinematic(translated, req, src_lang, loop, *, already_llm=False
|
||||
# Phase 4.4 speech-rate fit pass — now concurrent + bounded (see helper).
|
||||
await _apply_fit_pass(merged, req, slots_by_id, source_by_id, quality, loop, deadline)
|
||||
|
||||
# Plan AFTER the fit pass — verdicts must describe the final text.
|
||||
await _finalize_duration_plan(merged, req, loop)
|
||||
|
||||
return {
|
||||
"translated": merged,
|
||||
"target_lang": req.target_lang,
|
||||
|
||||
@@ -156,6 +156,97 @@ async def uninstall_translation_engine(engine_id: str):
|
||||
return {"status": "uninstalled", "engine": engine_id, "package": pkg, "log_tail": out[-800:]}
|
||||
|
||||
|
||||
# ── One-click sidecar-engine install (IndexTTS-2 & friends) ────────────────
|
||||
#
|
||||
# Sidecar engines (dedicated venv + source checkout + weights, isolated from
|
||||
# the parent's transformers>=5.3) used to require four manual terminal steps.
|
||||
# These routes drive services.sidecar_install: POST starts a resumable
|
||||
# background job, GET polls its step-by-step status (the Settings → Engines
|
||||
# Install button polls this), DELETE removes an app-managed install.
|
||||
#
|
||||
# Path namespace: /engines/sidecar/{engine_id}/… — NOT /engines/{engine_id}/…
|
||||
# — because a dynamic segment there would shadow pre-existing literal routes
|
||||
# (this router registers before sonitranslate's, so a dynamic
|
||||
# POST /engines/{engine_id}/install would swallow
|
||||
# POST /engines/sonitranslate/install). Mirrors the
|
||||
# /engines/translation/{engine_id}/install namespace pattern.
|
||||
#
|
||||
# Loopback-gated: installing spawns subprocesses (git/uv) and writes to the
|
||||
# data directory — only the local desktop frontend may trigger it. The job
|
||||
# runs fine in packaged builds: the venv lives under the user data dir, not
|
||||
# inside the signed app bundle, and uv resolves via OMNIVOICE_BUNDLED_UV/PATH.
|
||||
|
||||
|
||||
@router.post(
|
||||
"/engines/sidecar/{engine_id}/install",
|
||||
dependencies=[Depends(require_loopback)],
|
||||
)
|
||||
def install_sidecar_engine(engine_id: str):
|
||||
"""Start (or report) the one-click install for a sidecar engine.
|
||||
|
||||
Returns ``{status: "started"|"already_running"|"already_installed"}``.
|
||||
404 for engines that have no sidecar installer — the response names the
|
||||
translation-engine route so a mis-aimed client can self-correct.
|
||||
"""
|
||||
from services import sidecar_install
|
||||
try:
|
||||
return sidecar_install.start_install(engine_id)
|
||||
except KeyError:
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=(
|
||||
f"No one-click installer for engine {engine_id!r}. Sidecar "
|
||||
f"installers exist for: {sorted(sidecar_install.SPECS)}. "
|
||||
"(Translation engines install via POST "
|
||||
"/engines/translation/{id}/install.)"
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@router.get(
|
||||
"/engines/sidecar/{engine_id}/install/status",
|
||||
dependencies=[Depends(require_loopback)],
|
||||
)
|
||||
def sidecar_install_status(engine_id: str):
|
||||
"""Step-by-step status of the sidecar install job (poll while running).
|
||||
|
||||
Shape: ``{engine_id, installed, managed, install_dir, job}`` where job is
|
||||
null before the first run, else ``{state, steps[], log[], error,
|
||||
remediation, weights_progress, started_at, finished_at}``.
|
||||
"""
|
||||
from services import sidecar_install
|
||||
try:
|
||||
return sidecar_install.get_status(engine_id)
|
||||
except KeyError:
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=f"No one-click installer for engine {engine_id!r}.",
|
||||
)
|
||||
|
||||
|
||||
@router.delete(
|
||||
"/engines/sidecar/{engine_id}/install",
|
||||
dependencies=[Depends(require_loopback)],
|
||||
)
|
||||
def uninstall_sidecar_engine(engine_id: str):
|
||||
"""Remove an app-managed sidecar install (checkout + venv + weights) and
|
||||
clear the persisted path. Refuses user-managed installs (a clone the user
|
||||
made themselves) and installs with a job still running."""
|
||||
from services import sidecar_install
|
||||
try:
|
||||
res = sidecar_install.uninstall(engine_id)
|
||||
except KeyError:
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=f"No one-click installer for engine {engine_id!r}.",
|
||||
)
|
||||
if res["status"] == "install_in_progress":
|
||||
raise HTTPException(status_code=409, detail="Install is still running — wait for it to finish.")
|
||||
if res["status"] == "not_managed":
|
||||
raise HTTPException(status_code=400, detail=res["detail"])
|
||||
return res
|
||||
|
||||
|
||||
# ── Engine health-check (Plan 02-04 / ENGINE-06) ───────────────────────────
|
||||
#
|
||||
# The Compat Matrix UI's "Test engine" button calls into this endpoint so
|
||||
|
||||
@@ -21,12 +21,17 @@ def _safe_destination(raw: str) -> str:
|
||||
status_code=400,
|
||||
detail="Export needs a destination folder. Pick where the file should go and try again.",
|
||||
)
|
||||
dest = os.path.realpath(os.path.expanduser(raw))
|
||||
if not os.path.isabs(dest):
|
||||
expanded = os.path.expanduser(raw)
|
||||
# Check BEFORE realpath(): realpath absolutizes a relative path against
|
||||
# the server's cwd, which made this check dead code — a relative
|
||||
# destination silently exported to a cwd-dependent location instead of
|
||||
# the documented 400 (regression-tested in tests/test_exports_api.py).
|
||||
if not os.path.isabs(expanded):
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail="The destination needs to be a full path (e.g. /Users/you/Movies/OmniVoice) — not relative.",
|
||||
)
|
||||
dest = os.path.realpath(expanded)
|
||||
parent = os.path.dirname(dest)
|
||||
if not parent or not os.path.isdir(parent):
|
||||
raise HTTPException(
|
||||
@@ -39,7 +44,10 @@ def _safe_destination(raw: str) -> str:
|
||||
def _safe_source(filename: str) -> str:
|
||||
"""Resolve a source filename against OUTPUTS_DIR / dub outputs, blocking traversal."""
|
||||
base = os.path.basename(filename or "")
|
||||
if not base or base != filename:
|
||||
# "." and ".." are their own basename, so they'd slip past the
|
||||
# base != filename check and only die later on realpath containment —
|
||||
# reject them up front with the same 400 as any other malformed name.
|
||||
if not base or base != filename or base in (".", ".."):
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail="The file to export has an unexpected name. Try re-generating the audio and exporting again.",
|
||||
|
||||
@@ -161,13 +161,18 @@ async def search_youtube(
|
||||
list. Users are responsible for the licensing of whatever they import.
|
||||
"""
|
||||
try:
|
||||
# yt-dlp is an importable module, never a PATH requirement — run it
|
||||
# via the interpreter (honors the Settings → Audio tools overlay).
|
||||
from services.media_tools import ytdlp_invocation
|
||||
ytdlp_argv, ytdlp_env = ytdlp_invocation()
|
||||
result = await spawn_subprocess(
|
||||
"yt-dlp",
|
||||
*ytdlp_argv,
|
||||
"--dump-json",
|
||||
"--remote-components", "ejs:github",
|
||||
f"ytsearch{max_results}:{query}",
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
env=ytdlp_env,
|
||||
)
|
||||
stdout, stderr = await result.communicate()
|
||||
|
||||
@@ -218,8 +223,10 @@ async def download_youtube_clip(
|
||||
temp_path = str(VOICE_GALLERY_DIR / f"{voice_id}.%(ext)s")
|
||||
|
||||
try:
|
||||
from services.media_tools import ytdlp_invocation
|
||||
ytdlp_argv, ytdlp_env = ytdlp_invocation()
|
||||
cmd = [
|
||||
"yt-dlp",
|
||||
*ytdlp_argv,
|
||||
"--remote-components", "ejs:github",
|
||||
"-f",
|
||||
"bestaudio",
|
||||
@@ -239,6 +246,7 @@ async def download_youtube_clip(
|
||||
*cmd,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
env=ytdlp_env,
|
||||
)
|
||||
stdout, stderr = await result.communicate()
|
||||
|
||||
|
||||
@@ -12,6 +12,7 @@ import traceback
|
||||
from typing import Optional
|
||||
from fastapi import APIRouter, File, Form, UploadFile, HTTPException
|
||||
from fastapi.responses import StreamingResponse
|
||||
from pydantic import BaseModel
|
||||
|
||||
import sqlite3
|
||||
from core.db import db_conn, ensure_schema
|
||||
@@ -598,6 +599,32 @@ def _run_backend_inference(
|
||||
_oom_friendly_reraise(e)
|
||||
|
||||
|
||||
def _persist_profile_ref_text(profile_id: str, ref_text: str) -> None:
|
||||
"""Cache an auto-transcribed reference transcript onto its profile row.
|
||||
|
||||
#1032 perf regression: profiles saved without a transcript re-ran a FULL
|
||||
ASR model load + transcribe on every /generate (the #308 auto-transcribe
|
||||
path). Persisting the first transcript makes subsequent generates read it
|
||||
from the row like a user-entered one. The guarded UPDATE only ever fills
|
||||
an empty column — it can never overwrite a transcript the user typed or a
|
||||
lock wrote — and a failure is logged, never raised (best-effort, same
|
||||
contract as the transcribe itself)."""
|
||||
try:
|
||||
with db_conn() as conn:
|
||||
updated = conn.execute(
|
||||
"UPDATE voice_profiles SET ref_text=? "
|
||||
"WHERE id=? AND (ref_text IS NULL OR ref_text='')",
|
||||
(ref_text, profile_id),
|
||||
).rowcount
|
||||
if updated:
|
||||
event_bus.emit("profiles", {"action": "updated", "id": profile_id})
|
||||
except Exception as e: # noqa: BLE001 — cache write must not break generate
|
||||
logger.warning(
|
||||
"could not persist auto-transcribed ref_text onto profile %s: %s",
|
||||
profile_id, e,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/generate")
|
||||
async def generate_speech(
|
||||
text: str = Form(...),
|
||||
@@ -693,11 +720,51 @@ async def generate_speech(
|
||||
raise HTTPException(status_code=400, detail=_routing["routing_reason"])
|
||||
_routing_notice = routing_notice(_routing) # (status, reason) or None
|
||||
|
||||
# ── #1033/#1037: warm the engine under the LOAD budget, not the generate
|
||||
# budget. A cold adapter lazily loads (and possibly downloads multi-GB
|
||||
# weights) inside generate(), so a fresh install's first request burned
|
||||
# its whole OMNIVOICE_GENERATE_TIMEOUT_S window on the download and died
|
||||
# with a misleading "too heavy for the available compute" 503 (#1014
|
||||
# measured it: 0% GPU util for the full 300s). Model loading gets its own,
|
||||
# larger budget (OMNIVOICE_MODEL_LOAD_TIMEOUT, default 1200s) — the same
|
||||
# split get_model() already has for the native engine. Once warm, this is
|
||||
# a no-op per request.
|
||||
if _backend is not None:
|
||||
from services.model_manager import _model_load_timeout
|
||||
try:
|
||||
await run_on_gpu_pool_guarded(
|
||||
_backend.ensure_ready,
|
||||
what=f"TTS engine '{engine_id}' model load",
|
||||
timeout=_model_load_timeout(),
|
||||
)
|
||||
# Builtin TimeoutError base, not GpuJobTimeoutError — reload-proof
|
||||
# class identity (see the twin catch in openai_compat.py).
|
||||
except TimeoutError as exc:
|
||||
logger.warning("engine load exceeded the model-load budget: %s", exc)
|
||||
raise HTTPException(
|
||||
status_code=503,
|
||||
detail=(
|
||||
f"TTS engine '{engine_id}' did not finish loading within its "
|
||||
f"model-load budget — on a first run this usually means the "
|
||||
f"weight download is slow or stalled (check Settings → Models "
|
||||
f"for progress), not that generation failed. Retry once the "
|
||||
f"model shows as installed."
|
||||
),
|
||||
) from exc
|
||||
|
||||
ref_audio_path = None
|
||||
cleanup_ref = False
|
||||
used_seed = seed
|
||||
resolved_profile_id = None
|
||||
history_mode = None # profile.kind when a profile drives; else inferred at insert
|
||||
# #1032: profile id to persist an auto-transcribed reference transcript to.
|
||||
# Set only for a plain (unlocked) clone profile whose stored ref_text is
|
||||
# empty — the case where every /generate re-ran a full ASR model load +
|
||||
# transcribe of the same clip. Locked profiles are excluded (their ref
|
||||
# audio is the locked take, and unlocking would leave a mismatched
|
||||
# transcript paired with the original reference); design profiles are
|
||||
# excluded (a re-render replaces the sample, stranding a stale transcript).
|
||||
persist_ref_text_profile_id = None
|
||||
|
||||
if profile_id:
|
||||
with db_conn() as conn:
|
||||
@@ -743,6 +810,11 @@ async def generate_speech(
|
||||
ref_audio_path = os.path.join(VOICES_DIR, row["ref_audio_path"]) if row["ref_audio_path"] else None
|
||||
if not ref_text and row["ref_text"]:
|
||||
ref_text = row["ref_text"]
|
||||
elif ref_audio_path and not ref_text:
|
||||
# Empty stored transcript → the auto-transcribe below will
|
||||
# run; cache its result onto the profile so it runs ONCE,
|
||||
# not on every generate (#1032 perf regression).
|
||||
persist_ref_text_profile_id = profile_id
|
||||
if not instruct and row["instruct"]:
|
||||
instruct = row["instruct"]
|
||||
if used_seed is None and row["seed"] is not None:
|
||||
@@ -792,6 +864,11 @@ async def generate_speech(
|
||||
except GpuJobTimeoutError as e:
|
||||
logger.warning("reference transcribe hung (%s); using model ASR fallback", e)
|
||||
ref_text = None
|
||||
# #1032: cache the transcript onto its clone profile so the ASR model
|
||||
# load + transcribe above happens once per profile, not per generate.
|
||||
# Only fills an empty column — a user-entered transcript always wins.
|
||||
if ref_text and persist_ref_text_profile_id:
|
||||
_persist_profile_ref_text(persist_ref_text_profile_id, ref_text)
|
||||
|
||||
# #526: materialize a concrete seed when none was supplied (and no profile
|
||||
# pinned one) so the take is reproducible and we can hand it back via the
|
||||
@@ -801,6 +878,15 @@ async def generate_speech(
|
||||
if used_seed is None:
|
||||
used_seed = random.randint(0, 2**31 - 1)
|
||||
|
||||
# Engine-agnostic text normalization (junk strip, numbers→words,
|
||||
# abbreviations) — AFTER `language` is fully resolved, and BEFORE the
|
||||
# pronunciation dictionary so user dictionary entries operate on
|
||||
# normalized text and respellings are never re-mangled (ordering rationale
|
||||
# in services/text_normalization.py). Pref-gated (default ON), idempotent,
|
||||
# never raises; applied exactly once per request, at this choke point.
|
||||
from services.text_normalization import normalize_for_tts
|
||||
text = normalize_for_tts(text, language)
|
||||
|
||||
# Expressive-TTS Spec 01: apply the user pronunciation dictionary + inline
|
||||
# [[…]] one-off overrides to the text, here — AFTER `language` is fully
|
||||
# resolved (a profile may fill it above) so per-language entries match the
|
||||
@@ -905,6 +991,13 @@ async def generate_speech(
|
||||
logger.warning("history write still failed after schema heal; returning audio anyway: %s", e2)
|
||||
except Exception as e:
|
||||
logger.warning("generation history write failed; returning audio anyway: %s", e)
|
||||
# Retention cap: without it, takes (rows + WAVs in OUTPUTS_DIR) grow
|
||||
# unbounded forever. Best-effort — a prune failure must never affect
|
||||
# the generation that just succeeded.
|
||||
try:
|
||||
_prune_history_over_cap()
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.warning("history retention prune failed (non-fatal): %s", e)
|
||||
event_bus.emit("generation_history", {"action": "created", "id": audio_id})
|
||||
|
||||
buffer = io.BytesIO()
|
||||
@@ -977,17 +1070,101 @@ def _safe_output_path(name):
|
||||
return candidate
|
||||
|
||||
|
||||
def _remove_wav_if_unreferenced(conn, audio_path, exclude_ids=()):
|
||||
"""Delete a history WAV from OUTPUTS_DIR — but only when no *other*
|
||||
generation_history row still references the same file.
|
||||
|
||||
History WAVs are uniquely owned by their row (lock/save-as-profile COPY
|
||||
into VOICES_DIR, exports copy to the user's destination), so this guard is
|
||||
normally a no-op — it exists so any future path that duplicates a row can
|
||||
never make a delete/prune yank audio out from under a surviving take."""
|
||||
if not audio_path:
|
||||
return
|
||||
p = _safe_output_path(audio_path)
|
||||
if not p or not os.path.exists(p):
|
||||
return
|
||||
placeholders = ",".join("?" for _ in exclude_ids)
|
||||
others = conn.execute(
|
||||
"SELECT COUNT(*) FROM generation_history WHERE audio_path=?"
|
||||
+ (f" AND id NOT IN ({placeholders})" if exclude_ids else ""),
|
||||
(audio_path, *exclude_ids),
|
||||
).fetchone()[0]
|
||||
if others:
|
||||
return
|
||||
with contextlib.suppress(OSError):
|
||||
os.remove(p)
|
||||
|
||||
|
||||
# How many takes to keep before pruning the oldest UNstarred ones (rows + their
|
||||
# WAVs). User-tunable via Settings → Storage; 0 = unlimited. The pref key is
|
||||
# shared with api/routers/settings.py (the GET/PUT endpoint) — same pattern as
|
||||
# perf.torch_compile_disabled, which settings.py and engine_env.py both name.
|
||||
HISTORY_CAP_PREF_KEY = "generation_history_cap"
|
||||
DEFAULT_HISTORY_CAP = 200
|
||||
|
||||
|
||||
def _history_cap() -> int:
|
||||
from core import prefs
|
||||
|
||||
try:
|
||||
cap = int(prefs.get(HISTORY_CAP_PREF_KEY, DEFAULT_HISTORY_CAP))
|
||||
except (TypeError, ValueError):
|
||||
return DEFAULT_HISTORY_CAP
|
||||
return max(0, cap)
|
||||
|
||||
|
||||
def _prune_history_over_cap() -> int:
|
||||
"""Retention: keep the newest ``_history_cap()`` takes; delete the oldest
|
||||
UNstarred rows over the cap plus their WAVs (via the unreferenced guard).
|
||||
Starred takes are never pruned — even when they alone exceed the cap.
|
||||
Returns the number of rows pruned."""
|
||||
cap = _history_cap()
|
||||
if cap <= 0:
|
||||
return 0 # 0 = unlimited
|
||||
with db_conn() as conn:
|
||||
total = conn.execute("SELECT COUNT(*) FROM generation_history").fetchone()[0]
|
||||
excess = total - cap
|
||||
if excess <= 0:
|
||||
return 0
|
||||
victims = conn.execute(
|
||||
"SELECT id, audio_path FROM generation_history "
|
||||
"WHERE COALESCE(starred, 0)=0 ORDER BY created_at ASC LIMIT ?",
|
||||
(excess,),
|
||||
).fetchall()
|
||||
if not victims:
|
||||
return 0
|
||||
victim_ids = [r["id"] for r in victims]
|
||||
conn.executemany(
|
||||
"DELETE FROM generation_history WHERE id=?", [(i,) for i in victim_ids]
|
||||
)
|
||||
for r in victims:
|
||||
_remove_wav_if_unreferenced(conn, r["audio_path"], exclude_ids=victim_ids)
|
||||
logger.info("history retention: pruned %d takes over the %d cap", len(victims), cap)
|
||||
return len(victims)
|
||||
|
||||
|
||||
@router.get("/history")
|
||||
def list_history():
|
||||
"""Newest 50 generations whose audio still exists on disk.
|
||||
"""The newest 50 generations plus every starred take, newest first, kept to
|
||||
rows whose audio still exists on disk.
|
||||
|
||||
Rows whose WAV was deleted out-of-band (cleared outputs dir, manual
|
||||
cleanup) used to come back anyway and render dead players that 404 on
|
||||
every fetch; prune them here so the UI never sees them again."""
|
||||
Starred takes ride along past the 50-row window so a keeper can never age
|
||||
off the rail. Rows whose WAV was deleted out-of-band (cleared outputs dir,
|
||||
manual cleanup) used to come back anyway and render dead players that 404
|
||||
on every fetch; prune them here so the UI never sees them again."""
|
||||
query = (
|
||||
"SELECT * FROM generation_history WHERE COALESCE(starred, 0)=1 "
|
||||
"OR id IN (SELECT id FROM generation_history ORDER BY created_at DESC LIMIT 50) "
|
||||
"ORDER BY created_at DESC"
|
||||
)
|
||||
with db_conn() as conn:
|
||||
rows = conn.execute(
|
||||
"SELECT * FROM generation_history ORDER BY created_at DESC LIMIT 50"
|
||||
).fetchall()
|
||||
try:
|
||||
rows = conn.execute(query).fetchall()
|
||||
except sqlite3.OperationalError:
|
||||
# Same class as #710/#552: a DB that missed init or the additive
|
||||
# `starred` column. Heal once and retry inside this connection.
|
||||
ensure_schema()
|
||||
rows = conn.execute(query).fetchall()
|
||||
alive, stale_ids = [], []
|
||||
for r in rows:
|
||||
p = _safe_output_path(r["audio_path"]) if r["audio_path"] else None
|
||||
@@ -1003,6 +1180,39 @@ def list_history():
|
||||
logger.info("pruned %d stale history rows (audio file gone)", len(stale_ids))
|
||||
return alive
|
||||
|
||||
|
||||
class _StarBody(BaseModel):
|
||||
starred: bool
|
||||
|
||||
|
||||
@router.put("/history/{history_id}/starred")
|
||||
def set_history_starred(history_id: str, body: _StarBody):
|
||||
"""Star/unstar a take. Starred takes survive the retention cap and always
|
||||
appear in GET /history regardless of the recency window."""
|
||||
def _update():
|
||||
with db_conn() as conn:
|
||||
cur = conn.execute(
|
||||
"UPDATE generation_history SET starred=? WHERE id=?",
|
||||
(1 if body.starred else 0, history_id),
|
||||
)
|
||||
return cur.rowcount
|
||||
|
||||
try:
|
||||
changed = _update()
|
||||
except sqlite3.OperationalError as e:
|
||||
# `no such column: starred` on a pre-migration DB (or the #710
|
||||
# missing-table class) — heal the schema and retry once.
|
||||
logger.warning("star update failed (%s); healing schema + retrying", e)
|
||||
ensure_schema()
|
||||
changed = _update()
|
||||
if not changed:
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail="That take no longer exists — it may have been pruned or deleted.",
|
||||
)
|
||||
event_bus.emit("generation_history", {"action": "starred", "id": history_id})
|
||||
return {"id": history_id, "starred": body.starred}
|
||||
|
||||
@router.delete("/history")
|
||||
def clear_history():
|
||||
with db_conn() as conn:
|
||||
@@ -1020,11 +1230,10 @@ def clear_history():
|
||||
def delete_single_history(history_id: str):
|
||||
with db_conn() as conn:
|
||||
row = conn.execute("SELECT audio_path FROM generation_history WHERE id=?", (history_id,)).fetchone()
|
||||
if row and row["audio_path"]:
|
||||
p = _safe_output_path(row["audio_path"])
|
||||
if p and os.path.exists(p):
|
||||
with contextlib.suppress(OSError):
|
||||
os.remove(p)
|
||||
conn.execute("DELETE FROM generation_history WHERE id=?", (history_id,))
|
||||
if row:
|
||||
# Row first, file second — the WAV goes only if no surviving take
|
||||
# still references it (see _remove_wav_if_unreferenced).
|
||||
_remove_wav_if_unreferenced(conn, row["audio_path"], exclude_ids=(history_id,))
|
||||
event_bus.emit("generation_history", {"action": "deleted", "id": history_id})
|
||||
return {"deleted": True}
|
||||
|
||||
@@ -0,0 +1,85 @@
|
||||
"""Media-tools endpoints — the backend for Settings → Audio tools and the
|
||||
wizard's invisible media-engine self-heal.
|
||||
|
||||
Every route is loopback-gated: ``custom-path`` / ``use-system`` point the app
|
||||
at an arbitrary executable (an RCE primitive if remote-reachable), and the
|
||||
rest mutate local state. Same contract as ``/system/set-env``.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel
|
||||
|
||||
from api.dependencies import require_loopback
|
||||
|
||||
logger = logging.getLogger("omnivoice.api")
|
||||
router = APIRouter(dependencies=[Depends(require_loopback)])
|
||||
|
||||
|
||||
class CustomPathRequest(BaseModel):
|
||||
path: str
|
||||
|
||||
|
||||
def _svc():
|
||||
# Late import so a service-level failure surfaces as a 500 with detail,
|
||||
# not an app-boot failure.
|
||||
from services import media_tools
|
||||
return media_tools
|
||||
|
||||
|
||||
@router.get("/media-tools/status")
|
||||
def media_tools_status():
|
||||
"""Per-tool {ok, path, version, origin} + background-op states."""
|
||||
return _svc().status()
|
||||
|
||||
|
||||
@router.post("/media-tools/acquire")
|
||||
def media_tools_acquire():
|
||||
"""(Re-)fetch the pinned, checksummed static ffmpeg/ffprobe build in the
|
||||
background. Idempotent; poll /media-tools/status for progress."""
|
||||
return _svc().acquire_bundled()
|
||||
|
||||
|
||||
# Literal ytdlp routes MUST register before the parametrized {tool} routes —
|
||||
# FastAPI matches in declaration order, and `/media-tools/{tool}/restore`
|
||||
# would otherwise swallow `/media-tools/ytdlp/restore` into a 400.
|
||||
@router.post("/media-tools/ytdlp/update")
|
||||
def media_tools_ytdlp_update():
|
||||
"""Fetch the newest yt-dlp wheel (sha256-verified against PyPI metadata)
|
||||
into the update-surviving overlay. Applies on next backend start."""
|
||||
return _svc().update_ytdlp()
|
||||
|
||||
|
||||
@router.post("/media-tools/ytdlp/restore")
|
||||
def media_tools_ytdlp_restore():
|
||||
"""Drop the overlay — the app-tested, locked yt-dlp takes over on next
|
||||
start. Always safe (the locked install is never modified)."""
|
||||
return _svc().restore_ytdlp()
|
||||
|
||||
|
||||
@router.post("/media-tools/{tool}/custom-path")
|
||||
def media_tools_custom_path(tool: str, body: CustomPathRequest):
|
||||
try:
|
||||
return _svc().set_custom_path(tool, body.path)
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
|
||||
@router.post("/media-tools/{tool}/use-system")
|
||||
def media_tools_use_system(tool: str):
|
||||
try:
|
||||
return _svc().use_system(tool)
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
except LookupError as e:
|
||||
raise HTTPException(status_code=404, detail=str(e))
|
||||
|
||||
|
||||
@router.post("/media-tools/{tool}/restore")
|
||||
def media_tools_restore(tool: str):
|
||||
try:
|
||||
return _svc().restore_bundled(tool)
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
@@ -108,6 +108,23 @@ class SpeechRequest(BaseModel):
|
||||
ge=0,
|
||||
description="OmniVoice GGUF extension: long-form internal chunk threshold.",
|
||||
)
|
||||
# #1014: these two were silently DISCARDED before (pydantic ignores
|
||||
# undeclared fields) — a 200 OK that quietly dropped the caller's quality
|
||||
# knobs. Declared now and passed through, matching the native /generate
|
||||
# form fields (defaults there: num_step=16, guidance_scale=2.0; the
|
||||
# model's documented "quality" preset is num_step=32).
|
||||
num_step: Optional[int] = Field(
|
||||
default=None,
|
||||
ge=1,
|
||||
le=128,
|
||||
description="OmniVoice extension: iterative unmasking steps (app default 16; 32 = the model's documented quality preset).",
|
||||
)
|
||||
guidance_scale: Optional[float] = Field(
|
||||
default=None,
|
||||
gt=0,
|
||||
le=20,
|
||||
description="OmniVoice extension: classifier-free guidance scale (app default 2.0).",
|
||||
)
|
||||
|
||||
|
||||
class TranscriptionResponse(BaseModel):
|
||||
@@ -274,6 +291,10 @@ async def create_speech(req: SpeechRequest):
|
||||
kw["chunk_duration"] = req.chunk_duration
|
||||
if req.chunk_threshold is not None:
|
||||
kw["chunk_threshold"] = req.chunk_threshold
|
||||
if req.num_step is not None:
|
||||
kw["num_step"] = req.num_step
|
||||
if req.guidance_scale is not None:
|
||||
kw["guidance_scale"] = req.guidance_scale
|
||||
if req.language:
|
||||
kw["language"] = req.language
|
||||
if req.instruct:
|
||||
@@ -311,11 +332,52 @@ async def create_speech(req: SpeechRequest):
|
||||
# Not a profile ID — might be a KittenTTS preset or similar
|
||||
kw["voice"] = voice
|
||||
|
||||
# Engine-agnostic text normalization (junk strip, numbers→words,
|
||||
# abbreviations) at this route's text→engine choke point — the same
|
||||
# pre-pass as /generate, applied exactly once per request. `req.language`
|
||||
# is everything this route knows about the language (None → universal
|
||||
# safety filters only). Pref-gated (default ON), idempotent, never raises.
|
||||
from services.text_normalization import normalize_for_tts
|
||||
text = normalize_for_tts(req.input, req.language)
|
||||
|
||||
# ── #1033/#1037/#1014: warm the engine under the LOAD budget before the
|
||||
# generate clock starts. The T4 verification (#1014) measured a fresh
|
||||
# install's first /v1/audio/speech burning its whole 300s generate budget
|
||||
# on the multi-GB checkpoint download (0% GPU util throughout) and dying
|
||||
# with a misleading "too heavy for the available compute" error. Model
|
||||
# loading gets OMNIVOICE_MODEL_LOAD_TIMEOUT (default 1200s); once warm
|
||||
# this is a per-request no-op.
|
||||
from services.model_manager import _model_load_timeout
|
||||
try:
|
||||
await run_on_gpu_pool_guarded(
|
||||
backend.ensure_ready,
|
||||
what=f"TTS engine '{backend.id}' model load",
|
||||
timeout=_model_load_timeout(),
|
||||
)
|
||||
# Catch the BUILTIN TimeoutError base, not GpuJobTimeoutError by name:
|
||||
# several tests reload services.model_manager mid-suite, so a class
|
||||
# imported at call time can differ in identity from the one the guard
|
||||
# (bound at this module's import) actually raises — the except would
|
||||
# silently miss. The builtin base has one identity forever. (Caught by
|
||||
# this exact test failing CI-only, in full-suite order.)
|
||||
except TimeoutError as e:
|
||||
logger.warning("engine load exceeded the model-load budget: %s", e)
|
||||
raise HTTPException(
|
||||
status_code=503,
|
||||
detail=(
|
||||
f"TTS engine '{backend.id}' did not finish loading within its "
|
||||
f"model-load budget — on a first run this usually means the weight "
|
||||
f"download is slow or stalled (check Settings → Models for "
|
||||
f"progress), not that generation failed. Retry once the model "
|
||||
f"shows as installed."
|
||||
),
|
||||
) from e
|
||||
|
||||
try:
|
||||
# Bounded + pool-reset on hang so a wedged TTS request can't starve the
|
||||
# GPU pool and brick the backend (#730 class).
|
||||
wav, sr = await run_on_gpu_pool_guarded(
|
||||
lambda: _run_tts(backend, req.input, kw), what="OpenAI TTS generate")
|
||||
lambda: _run_tts(backend, text, kw), what="OpenAI TTS generate")
|
||||
except Exception as e:
|
||||
logger.exception("OpenAI TTS failed: %s", e)
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
|
||||
@@ -77,8 +77,17 @@ def clear_hf_token(also_clear_hf_cli: bool = Query(False)):
|
||||
|
||||
|
||||
@router.get("/hf-token/state")
|
||||
def get_hf_token_state():
|
||||
"""3-source HF token cascade state for the Settings UI."""
|
||||
def get_hf_token_state(fresh: bool = Query(False)):
|
||||
"""3-source HF token cascade state for the Settings UI.
|
||||
|
||||
``fresh=1`` drops the resolver's whoami validation cache first so the
|
||||
response re-runs whoami for every source — this is what the panel's
|
||||
"Test now" button sends. Plain GETs (panel mounts) keep the 300s cache
|
||||
so repeat Settings visits don't hammer the HF API.
|
||||
"""
|
||||
from services import token_resolver
|
||||
if fresh:
|
||||
token_resolver.invalidate_cache()
|
||||
return _state_response()
|
||||
|
||||
|
||||
@@ -124,6 +133,46 @@ def set_torch_compile_disabled(body: _TorchCompileBody):
|
||||
return _torch_compile_state()
|
||||
|
||||
|
||||
# ── Generation-history retention (Studio takes rail) ──────────────────────
|
||||
|
||||
|
||||
class _HistoryRetentionBody(BaseModel):
|
||||
cap: int = Field(
|
||||
...,
|
||||
ge=0,
|
||||
le=100000,
|
||||
description="Max takes kept before the oldest UNstarred ones (rows + WAVs) are pruned; 0 = unlimited",
|
||||
)
|
||||
|
||||
|
||||
def _history_retention_state() -> dict:
|
||||
from api.routers.generation import DEFAULT_HISTORY_CAP, _history_cap
|
||||
|
||||
return {"cap": _history_cap(), "default": DEFAULT_HISTORY_CAP}
|
||||
|
||||
|
||||
@router.get("/history-retention")
|
||||
def get_history_retention():
|
||||
"""Current generation-history retention cap (Settings → Storage)."""
|
||||
return _history_retention_state()
|
||||
|
||||
|
||||
@router.put("/history-retention")
|
||||
def set_history_retention(body: _HistoryRetentionBody):
|
||||
"""Persist the retention cap. Enforced after every generation: the oldest
|
||||
unstarred takes over the cap are pruned (rows + their audio files);
|
||||
starred takes are never pruned. 0 disables pruning entirely."""
|
||||
from core import prefs
|
||||
from api.routers.generation import HISTORY_CAP_PREF_KEY
|
||||
|
||||
try:
|
||||
prefs.set_(HISTORY_CAP_PREF_KEY, int(body.cap))
|
||||
except Exception:
|
||||
logger.exception("set_history_retention failed")
|
||||
raise HTTPException(status_code=500, detail="Failed to persist setting")
|
||||
return _history_retention_state()
|
||||
|
||||
|
||||
# ── Dictation refinement (parity program Wave 2.1 / Spec 3 phase 2) ───────
|
||||
|
||||
|
||||
@@ -685,6 +734,26 @@ async def get_storage_report(refresh: bool = Query(False)):
|
||||
raise HTTPException(status_code=500, detail="Failed to compute storage report")
|
||||
|
||||
|
||||
@router.post("/storage/temp/clear")
|
||||
async def clear_temp_files():
|
||||
"""Delete OmniVoice-owned temp files (Settings → Storage → Temporary files).
|
||||
|
||||
Removes only the ``omnivoice*`` entries in the OS temp dir — the exact
|
||||
population the storage report's "temp" category counts — and invalidates
|
||||
the cached report so the next scan reflects the reclaimed space. Partial
|
||||
failures (files held open by a running job) are returned per entry.
|
||||
"""
|
||||
from services import storage_report
|
||||
|
||||
try:
|
||||
result = await asyncio.to_thread(storage_report.clear_temp)
|
||||
storage_report.clear_cache()
|
||||
return result
|
||||
except Exception:
|
||||
logger.exception("clear temp files failed")
|
||||
raise HTTPException(status_code=500, detail="Failed to clear temporary files")
|
||||
|
||||
|
||||
# ── HF mirror endpoint (parity program Wave 4.3 / §R4 c) ──────────────────
|
||||
# Restricted-network users (e.g. behind the Great Firewall) need to point
|
||||
# huggingface_hub at a mirror. HF reads HF_ENDPOINT at import time, so a
|
||||
@@ -704,27 +773,63 @@ _HF_MIRROR_PRESETS = [
|
||||
|
||||
class _HFMirrorBody(BaseModel):
|
||||
url: str = Field("", description="HF_ENDPOINT URL; empty string clears it (official endpoint)")
|
||||
mode: str | None = Field(
|
||||
None,
|
||||
description=(
|
||||
"'auto' switches to automatic endpoint selection (clears any "
|
||||
"explicit endpoint); 'manual' (or omitted — back-compat with older "
|
||||
"clients) pins the given url as an explicit choice."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@router.get("/hf-mirror")
|
||||
def get_hf_mirror():
|
||||
def _hf_mirror_state() -> dict:
|
||||
"""The full GET /hf-mirror payload. Auto info comes from the CACHED race
|
||||
decision only — reading settings never probes the network."""
|
||||
from core import user_env
|
||||
from services import endpoint_race
|
||||
|
||||
configured = user_env.get_user_env(_HF_ENDPOINT_ENV) or ""
|
||||
try:
|
||||
mode = "manual" if configured else endpoint_race.mode()
|
||||
auto = endpoint_race.cached_decision() if mode == "auto" else None
|
||||
opt_out = endpoint_race.env_opt_out()
|
||||
except Exception: # a broken prefs file must never 500 the settings page
|
||||
logger.exception("hf-mirror auto state unavailable")
|
||||
mode, auto, opt_out = "manual", None, False
|
||||
return {
|
||||
# The value that will apply after restart (persisted), and what's
|
||||
# live in this process (env may differ until then).
|
||||
"configured": configured,
|
||||
"effective": os.environ.get(_HF_ENDPOINT_ENV, ""),
|
||||
"presets": _HF_MIRROR_PRESETS,
|
||||
# Automatic endpoint selection (services.endpoint_race): "auto" only
|
||||
# when nothing explicit is configured anywhere. `auto` is the cached
|
||||
# race decision ({endpoint, reachable, latency_ms, checked_at,
|
||||
# results}) or null when never raced / in manual mode.
|
||||
"mode": mode,
|
||||
"auto": auto,
|
||||
"auto_opt_out": opt_out,
|
||||
}
|
||||
|
||||
|
||||
@router.get("/hf-mirror")
|
||||
def get_hf_mirror():
|
||||
return _hf_mirror_state()
|
||||
|
||||
|
||||
@router.put("/hf-mirror")
|
||||
def set_hf_mirror(body: _HFMirrorBody):
|
||||
from core import user_env
|
||||
from services import endpoint_race
|
||||
|
||||
url = (body.url or "").strip().rstrip("/")
|
||||
mode = (body.mode or "manual").strip().lower()
|
||||
if mode not in {"auto", "manual"}:
|
||||
raise HTTPException(status_code=400, detail="mode must be 'auto' or 'manual'")
|
||||
# Auto mode = no explicit endpoint anywhere; a persisted endpoint would
|
||||
# read as an explicit choice, so switching to Auto clears it (plus the
|
||||
# `hf_endpoint` pref fallback the download paths resolve).
|
||||
url = "" if mode == "auto" else (body.url or "").strip().rstrip("/")
|
||||
if url and not url.startswith(("http://", "https://")):
|
||||
raise HTTPException(status_code=400, detail="Mirror URL must start with http(s)://")
|
||||
# Compare against the currently-persisted value (normalised the same way) so
|
||||
@@ -739,15 +844,89 @@ def set_hf_mirror(body: _HFMirrorBody):
|
||||
else:
|
||||
user_env.unset_user_env(_HF_ENDPOINT_ENV)
|
||||
os.environ.pop(_HF_ENDPOINT_ENV, None)
|
||||
from core import prefs
|
||||
if mode == "auto":
|
||||
prefs.delete("hf_endpoint") # the pref fallback is explicit config too
|
||||
endpoint_race.set_mode_pref(mode)
|
||||
except Exception:
|
||||
logger.exception("set_hf_mirror failed")
|
||||
raise HTTPException(status_code=500, detail="Failed to persist mirror setting")
|
||||
if mode == "auto":
|
||||
# Freshly chosen Auto should show a real pick immediately — race now
|
||||
# unless a fresh cached decision already exists (probes are ≤3 s and
|
||||
# this is an explicit user action, not a hot path).
|
||||
try:
|
||||
endpoint_race.ensure_decision()
|
||||
except Exception:
|
||||
logger.exception("endpoint race after switching to auto failed")
|
||||
# Model Store downloads pick up the new mirror immediately — the download
|
||||
# path resolves the endpoint per-call and we updated os.environ above. Only
|
||||
# transformers-side model *loads* (which read HF_ENDPOINT at import time)
|
||||
# need a restart, so restart_required is True ONLY when the value actually
|
||||
# changed — a no-op re-save never asks for a restart.
|
||||
return {"configured": url, "restart_required": changed, "presets": _HF_MIRROR_PRESETS}
|
||||
return {**_hf_mirror_state(), "restart_required": changed}
|
||||
|
||||
|
||||
@router.post("/hf-mirror/test")
|
||||
def test_hf_mirror():
|
||||
"""Re-run the endpoint race now (the Auto panel's "Test again").
|
||||
|
||||
Forces fresh probes and re-caches the decision. In manual mode this is a
|
||||
no-op (an explicit endpoint is never auto-switched) — the response simply
|
||||
reflects the current state."""
|
||||
from services import endpoint_race
|
||||
|
||||
try:
|
||||
endpoint_race.ensure_decision(force=True)
|
||||
except Exception:
|
||||
logger.exception("hf-mirror endpoint test failed")
|
||||
raise HTTPException(status_code=500, detail="Endpoint test failed")
|
||||
return _hf_mirror_state()
|
||||
|
||||
|
||||
# ── OpenAI-compatible remote ASR (#877) ─────────────────────────────────────
|
||||
# A path to Qwen3-ASR/FunASR/SenseVoice — or OpenAI's own Whisper API — today,
|
||||
# without waiting on transformers to ship a direct Qwen3-ASR integration.
|
||||
# base_url/model are plain settings_store text rows; the key is encrypted via
|
||||
# settings_store.set_secret — same convention as /llm-providers, never
|
||||
# returned to the client, '' clears it, omitted/None leaves it unchanged.
|
||||
|
||||
|
||||
class _ASROpenAICompatBody(BaseModel):
|
||||
base_url: str | None = None
|
||||
model: str | None = None
|
||||
api_key: str | None = Field(None, description="'' clears it, None leaves unchanged")
|
||||
|
||||
|
||||
@router.get("/asr-openai-compat")
|
||||
def get_asr_openai_compat():
|
||||
from services import asr_backend
|
||||
|
||||
return {
|
||||
"base_url": asr_backend.resolve_openai_compat_asr_base_url(),
|
||||
"model": asr_backend.resolve_openai_compat_asr_model(),
|
||||
"has_key": asr_backend.openai_compat_asr_has_key(),
|
||||
}
|
||||
|
||||
|
||||
@router.put("/asr-openai-compat")
|
||||
def set_asr_openai_compat(body: _ASROpenAICompatBody):
|
||||
from services import asr_backend, settings_store
|
||||
|
||||
if body.base_url is not None:
|
||||
url = body.base_url.strip().rstrip("/")
|
||||
if url and not url.startswith(("http://", "https://")):
|
||||
raise HTTPException(status_code=400, detail="Base URL must start with http(s)://")
|
||||
settings_store.set_text(asr_backend._ASR_OPENAI_COMPAT_BASE_URL_KEY, url)
|
||||
if body.model is not None:
|
||||
settings_store.set_text(
|
||||
asr_backend._ASR_OPENAI_COMPAT_MODEL_KEY, body.model.strip() or "whisper-1"
|
||||
)
|
||||
if body.api_key is not None:
|
||||
settings_store.set_secret(
|
||||
asr_backend._ASR_OPENAI_COMPAT_SECRET_NAME, body.api_key.strip()
|
||||
)
|
||||
return get_asr_openai_compat()
|
||||
|
||||
|
||||
# ── Updates panel: shipped changelog + pre-migration DB backup state ────────
|
||||
|
||||
@@ -72,12 +72,15 @@ def _download_max_workers() -> int:
|
||||
|
||||
|
||||
def _download_endpoint() -> "str | None":
|
||||
"""Optional HF endpoint override (FDL-10 mirror path, opt-in). Returned as a
|
||||
per-call ``endpoint=`` rather than a process-wide HF_ENDPOINT mutation. A
|
||||
"""Optional HF endpoint override, per-call ``endpoint=`` rather than a
|
||||
process-wide HF_ENDPOINT mutation. Explicit configuration (FDL-10 mirror
|
||||
path: HF_ENDPOINT env / ``hf_endpoint`` pref / Settings) always wins; when
|
||||
nothing was chosen, the automatic endpoint selection's cached pick applies
|
||||
(services.endpoint_race — probe-based, cached, never probes here). A
|
||||
mirror routes through the classic LFS path (no Xet) — documented in
|
||||
docs/downloading-models.md."""
|
||||
ep = prefs.resolve("hf_endpoint", env="HF_ENDPOINT", default=None)
|
||||
return ep or None
|
||||
from services import endpoint_race
|
||||
return endpoint_race.effective_endpoint()
|
||||
|
||||
|
||||
def apply_xet_env() -> None:
|
||||
@@ -488,6 +491,23 @@ async def install_model(req: InstallModelRequest):
|
||||
"model install %s: attempt %d/%d failed (%s); retry in %ds",
|
||||
req.repo_id, _attempt, _max_attempts, net_err, _backoff,
|
||||
)
|
||||
# Endpoint failover (auto mode only, once per repo per
|
||||
# process): a network-classified failure re-races the
|
||||
# endpoints and, when the winner changed, the next attempt
|
||||
# retries on it — so a mid-download endpoint outage heals
|
||||
# instead of burning every retry on a dead host. Explicit
|
||||
# user endpoints are never switched.
|
||||
from services import endpoint_race
|
||||
if endpoint_race.reselect_after_failure(req.repo_id, str(net_err)):
|
||||
_endpoint = _download_endpoint()
|
||||
if _endpoint:
|
||||
dl_kwargs["endpoint"] = _endpoint
|
||||
else:
|
||||
dl_kwargs.pop("endpoint", None)
|
||||
logger.info(
|
||||
"model install %s: endpoint failover — retrying on %s",
|
||||
req.repo_id, _endpoint or "https://huggingface.co",
|
||||
)
|
||||
hf_progress.emit({
|
||||
"repo_id": req.repo_id,
|
||||
"filename": req.repo_id,
|
||||
|
||||
@@ -3,7 +3,9 @@
|
||||
Extracted from the monolithic ``setup.py``.
|
||||
|
||||
- ``GET /setup/status`` — missing-model gate for boot screen
|
||||
- ``GET /setup/preflight`` — system health check (OS, RAM, GPU, ffmpeg…)
|
||||
- ``GET /setup/preflight`` — system health check (OS, RAM, disk, GPU, network —
|
||||
genuine user facts only; the media engine (ffmpeg/ffprobe/yt-dlp) is an
|
||||
internal concern that self-heals via ``services.media_tools``)
|
||||
- ``POST /setup/warmup`` — background model pre-load
|
||||
"""
|
||||
from __future__ import annotations
|
||||
@@ -12,7 +14,6 @@ import asyncio
|
||||
import logging
|
||||
import os
|
||||
import platform as _platform
|
||||
import shutil as _shutil
|
||||
import sys
|
||||
|
||||
from fastapi import APIRouter
|
||||
@@ -202,6 +203,114 @@ def _hf_endpoint_host() -> tuple[str, int]:
|
||||
return "huggingface.co", 443
|
||||
|
||||
|
||||
def _network_check() -> dict:
|
||||
"""The preflight "network" check row — auto-race or explicit-endpoint probe.
|
||||
|
||||
Auto mode (nothing explicitly configured): force a fresh endpoint race —
|
||||
preflight IS the connectivity health check, and the cached winner is what
|
||||
model downloads will use. Manual mode: probe exactly the configured
|
||||
endpoint (never auto-switch an explicit choice), keeping the mirror
|
||||
quick-pick affordance when the official endpoint is blocked.
|
||||
"""
|
||||
auto_decision = None
|
||||
try:
|
||||
from services import endpoint_race
|
||||
if endpoint_race.mode() == "auto":
|
||||
auto_decision = endpoint_race.ensure_decision(force=True)
|
||||
except Exception as exc: # the race must never break preflight
|
||||
logger.warning("preflight endpoint race failed: %s", exc)
|
||||
|
||||
if auto_decision is not None:
|
||||
from urllib.parse import urlsplit
|
||||
from services.endpoint_race import CANONICAL_ENDPOINT
|
||||
|
||||
picked = auto_decision["endpoint"]
|
||||
picked_host = urlsplit(picked).hostname or picked
|
||||
latency = auto_decision.get("latency_ms")
|
||||
latency_s = f" ({latency:.0f} ms)" if isinstance(latency, (int, float)) else ""
|
||||
results = {r["endpoint"]: r for r in auto_decision.get("results", [])}
|
||||
canonical_ok = bool(results.get(CANONICAL_ENDPOINT, {}).get("reachable"))
|
||||
mirror_reachable = any(
|
||||
r.get("reachable") for ep, r in results.items() if ep != CANONICAL_ENDPOINT
|
||||
)
|
||||
if auto_decision.get("reachable"):
|
||||
if picked == CANONICAL_ENDPOINT:
|
||||
detail = f"Reachable{latency_s}"
|
||||
elif not canonical_ok:
|
||||
detail = (
|
||||
f"huggingface.co is unreachable on this network — using the "
|
||||
f"community mirror {picked_host}{latency_s} for model "
|
||||
"downloads. Downloads are checksum-verified by Hugging Face "
|
||||
"regardless of endpoint; change anytime in Settings → "
|
||||
"Models → Hugging Face mirror."
|
||||
)
|
||||
else:
|
||||
detail = (
|
||||
f"Both endpoints reachable — {picked_host}{latency_s} "
|
||||
"selected (decisively faster here). Change anytime in "
|
||||
"Settings → Models → Hugging Face mirror."
|
||||
)
|
||||
status, fix = "pass", None
|
||||
else:
|
||||
status = "warn"
|
||||
detail = "No Hugging Face endpoint reachable"
|
||||
fix = (
|
||||
"Neither huggingface.co nor the hf-mirror.com community mirror "
|
||||
"responded — check internet connection, VPN, or firewall. You "
|
||||
"can continue — models already downloaded keep working "
|
||||
"offline; a custom mirror can be configured below."
|
||||
)
|
||||
return {
|
||||
"id": "network", "label": f"Network ({picked_host})",
|
||||
"status": status, "detail": detail, "fix": fix,
|
||||
# Frontend affordance hint: the wizard offers the mirror
|
||||
# quick-pick when the check didn't pass (PreflightCheck allows
|
||||
# extras). `endpoint` documents the auto pick for the UI.
|
||||
"mirror_reachable": mirror_reachable,
|
||||
"endpoint": picked,
|
||||
}
|
||||
|
||||
# Manual mode (explicit endpoint) — probe exactly what the user chose.
|
||||
net_host, net_port = _hf_endpoint_host()
|
||||
net_ok = _probe_network(net_host, net_port)
|
||||
mirror_reachable = False
|
||||
if not net_ok and net_host == "huggingface.co":
|
||||
# Official endpoint blocked — if the community mirror is reachable,
|
||||
# tell the user exactly which switch unblocks them.
|
||||
mirror_reachable = _probe_network("hf-mirror.com")
|
||||
if net_ok:
|
||||
net_fix = None
|
||||
elif mirror_reachable:
|
||||
net_fix = (
|
||||
"huggingface.co is blocked on this network, but the hf-mirror.com "
|
||||
"community mirror is reachable — apply it below and re-check. "
|
||||
"Model downloads will use the mirror immediately."
|
||||
)
|
||||
elif net_host != "huggingface.co":
|
||||
net_fix = (
|
||||
f"Your configured Hugging Face mirror ({net_host}) is unreachable "
|
||||
"— it may be down or blocked. Pick another mirror or the official "
|
||||
"endpoint below, or continue offline: models already downloaded "
|
||||
"keep working."
|
||||
)
|
||||
else:
|
||||
net_fix = (
|
||||
"Check internet connection, VPN, or corporate firewall whitelist "
|
||||
"for huggingface.co. You can continue — models already downloaded "
|
||||
"keep working offline; new downloads need a connection or a "
|
||||
"mirror (configurable below)."
|
||||
)
|
||||
return {
|
||||
"id": "network", "label": f"Network ({net_host})",
|
||||
"status": "pass" if net_ok else "warn",
|
||||
"detail": "Reachable" if net_ok else f"Unreachable on port {net_port}",
|
||||
"fix": net_fix,
|
||||
# Frontend affordance hint: the wizard offers the mirror quick-pick
|
||||
# when the endpoint is unreachable (PreflightCheck allows extras).
|
||||
"mirror_reachable": mirror_reachable,
|
||||
}
|
||||
|
||||
|
||||
def _ram_gb() -> float:
|
||||
try:
|
||||
import psutil
|
||||
@@ -280,59 +389,19 @@ def preflight():
|
||||
f"Fix write permissions on {cache} or point HF_HOME elsewhere.",
|
||||
})
|
||||
|
||||
# ── FFmpeg
|
||||
ffmpeg_path = None
|
||||
# ── Media engine (ffmpeg/ffprobe/yt-dlp) — deliberately NOT a check row.
|
||||
# These are internal dependencies the app provisions for itself, not user
|
||||
# facts: when the resolution chain has no tier at all, preflight kicks the
|
||||
# bundled acquisition in the background and the wizard shows a quiet
|
||||
# progress line (a failure card only if that fails — with Retry / use a
|
||||
# system copy). yt-dlp is an importable locked module and never appears.
|
||||
# Power users manage all three in Settings → Audio tools.
|
||||
media_tools = None
|
||||
try:
|
||||
from services.ffmpeg_utils import find_ffmpeg
|
||||
ffmpeg_path = find_ffmpeg()
|
||||
except Exception as e:
|
||||
checks.append({
|
||||
"id": "ffmpeg", "label": "FFmpeg", "status": "fail",
|
||||
"detail": str(e)[:200],
|
||||
"fix": "Install ffmpeg via your package manager "
|
||||
"(brew install ffmpeg / apt install ffmpeg / choco install ffmpeg).",
|
||||
})
|
||||
else:
|
||||
checks.append({
|
||||
"id": "ffmpeg", "label": "FFmpeg", "status": "pass",
|
||||
"detail": ffmpeg_path, "fix": None,
|
||||
})
|
||||
|
||||
# ── FFprobe
|
||||
ffprobe_path = None
|
||||
try:
|
||||
from services.ffmpeg_utils import find_ffprobe
|
||||
ffprobe_path = find_ffprobe()
|
||||
except Exception:
|
||||
pass
|
||||
if ffprobe_path:
|
||||
checks.append({
|
||||
"id": "ffprobe", "label": "FFprobe", "status": "pass",
|
||||
"detail": ffprobe_path, "fix": None,
|
||||
})
|
||||
else:
|
||||
checks.append({
|
||||
"id": "ffprobe", "label": "FFprobe", "status": "warn",
|
||||
"detail": "Not bundled alongside ffmpeg.",
|
||||
"fix": "File-probe endpoint (/tools/probe) will 501. "
|
||||
"Install system ffmpeg (includes ffprobe) to enable it.",
|
||||
})
|
||||
|
||||
# ── yt-dlp
|
||||
yt_dlp_path = _shutil.which("yt-dlp")
|
||||
if yt_dlp_path:
|
||||
rc_ytv, yt_ver = _run_cmd([yt_dlp_path, "--version"], timeout=3.0)
|
||||
yt_version = yt_ver.strip() if rc_ytv == 0 else "unknown"
|
||||
checks.append({
|
||||
"id": "yt-dlp", "label": "yt-dlp", "status": "pass",
|
||||
"detail": f"{yt_dlp_path} (v{yt_version})", "fix": None,
|
||||
})
|
||||
else:
|
||||
checks.append({
|
||||
"id": "yt-dlp", "label": "yt-dlp", "status": "warn",
|
||||
"detail": "Not found in system PATH.",
|
||||
"fix": "YouTube clip downloads in Voice Gallery will fail. Download the standalone binary from https://github.com/yt-dlp/yt-dlp/releases and place it in your PATH.",
|
||||
})
|
||||
from services.media_tools import summary as _media_summary
|
||||
media_tools = _media_summary(auto_acquire=True)
|
||||
except Exception as exc: # never break preflight on the media engine
|
||||
logger.warning("preflight media_tools summary failed: %s", exc)
|
||||
|
||||
# ── GPU
|
||||
gpu = _detect_gpu()
|
||||
@@ -424,50 +493,21 @@ def preflight():
|
||||
"status": r_status, "detail": r_detail, "fix": r_fix,
|
||||
})
|
||||
|
||||
# ── Network — probes the HF endpoint actually in effect (mirror-aware),
|
||||
# and a dead network is a WARNING, not a blocker. The app is local-first:
|
||||
# already-downloaded models work offline, and a hard fail here dead-ends
|
||||
# restricted-network users (e.g. China, where huggingface.co is blocked)
|
||||
# on the very first screen — before they can reach the mirror setting
|
||||
# that fixes it. Model downloads surface their own actionable errors.
|
||||
net_host, net_port = _hf_endpoint_host()
|
||||
net_ok = _probe_network(net_host, net_port)
|
||||
mirror_reachable = False
|
||||
if not net_ok and net_host == "huggingface.co":
|
||||
# Official endpoint blocked — if the community mirror is reachable,
|
||||
# tell the user exactly which switch unblocks them.
|
||||
mirror_reachable = _probe_network("hf-mirror.com")
|
||||
if net_ok:
|
||||
net_fix = None
|
||||
elif mirror_reachable:
|
||||
net_fix = (
|
||||
"huggingface.co is blocked on this network, but the hf-mirror.com "
|
||||
"community mirror is reachable — apply it below and re-check. "
|
||||
"Model downloads will use the mirror immediately."
|
||||
)
|
||||
elif net_host != "huggingface.co":
|
||||
net_fix = (
|
||||
f"Your configured Hugging Face mirror ({net_host}) is unreachable "
|
||||
"— it may be down or blocked. Pick another mirror or the official "
|
||||
"endpoint below, or continue offline: models already downloaded "
|
||||
"keep working."
|
||||
)
|
||||
else:
|
||||
net_fix = (
|
||||
"Check internet connection, VPN, or corporate firewall whitelist "
|
||||
"for huggingface.co. You can continue — models already downloaded "
|
||||
"keep working offline; new downloads need a connection or a "
|
||||
"mirror (configurable below)."
|
||||
)
|
||||
checks.append({
|
||||
"id": "network", "label": f"Network ({net_host})",
|
||||
"status": "pass" if net_ok else "warn",
|
||||
"detail": "Reachable" if net_ok else f"Unreachable on port {net_port}",
|
||||
"fix": net_fix,
|
||||
# Frontend affordance hint: the wizard offers the mirror quick-pick
|
||||
# when the endpoint is unreachable (PreflightCheck allows extras).
|
||||
"mirror_reachable": mirror_reachable,
|
||||
})
|
||||
# ── Network — a dead network is a WARNING, not a blocker. The app is
|
||||
# local-first: already-downloaded models work offline, and a hard fail
|
||||
# here dead-ends restricted-network users (e.g. China, where
|
||||
# huggingface.co is blocked) on the very first screen — before they can
|
||||
# reach the mirror setting that fixes it. Model downloads surface their
|
||||
# own actionable errors.
|
||||
#
|
||||
# With NO explicit endpoint configured, preflight runs the automatic
|
||||
# endpoint race (services.endpoint_race): both the official endpoint and
|
||||
# the community mirror are probed, the winner is cached for downloads,
|
||||
# and the copy states the outcome honestly — so a blocked huggingface.co
|
||||
# no longer needs the user to find the mirror setting at all. An explicit
|
||||
# endpoint (Settings / HF_ENDPOINT / pref) keeps the single-endpoint
|
||||
# probe: the user's choice is never auto-switched.
|
||||
checks.append(_network_check())
|
||||
|
||||
# Aggregate
|
||||
any_fail = any(c["status"] == "fail" for c in checks)
|
||||
@@ -492,6 +532,7 @@ def preflight():
|
||||
"disk_free_gb": round(free, 1),
|
||||
},
|
||||
"gpu_routing": gpu_routing,
|
||||
"media_tools": media_tools,
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -629,15 +629,17 @@ def system_notifications():
|
||||
notes.append({
|
||||
"id": "ffmpeg-missing",
|
||||
"level": "error",
|
||||
"title": "ffmpeg not found",
|
||||
"title": "Media engine unavailable",
|
||||
"message": (
|
||||
"Video processing, audio conversion, and dubbing require ffmpeg. "
|
||||
"Install it with: brew install ffmpeg (macOS) or apt install ffmpeg (Linux)."
|
||||
"Video processing, audio conversion, and dubbing need the "
|
||||
"media engine (ffmpeg), which the app normally provisions "
|
||||
"itself. Open Settings > Audio tools and press Restore "
|
||||
"bundled to re-download it, or point it at a system copy."
|
||||
),
|
||||
"action": {
|
||||
"label": "Install guide",
|
||||
"type": "link",
|
||||
"target": "https://ffmpeg.org/download.html",
|
||||
"label": "Open Audio tools",
|
||||
"type": "settings-tab",
|
||||
"target": "audio-tools",
|
||||
},
|
||||
})
|
||||
|
||||
@@ -752,6 +754,17 @@ PERSISTENT_KEYS = {
|
||||
"OMNIVOICE_PORT", "OMNIVOICE_SHARE_PORT", "OMNIVOICE_UI_PORT",
|
||||
}
|
||||
|
||||
# Sidecar-engine install dirs (OMNIVOICE_INDEXTTS_DIR, …). The one-click
|
||||
# installer persists these via prefs.json `env.*` (restored at startup in
|
||||
# main.py); merging them here lets users inspect/clear them from the same
|
||||
# Settings env panel as every other persisted var. Single-sourced from the
|
||||
# installer's SPECS so a future sidecar engine can't forget to register.
|
||||
try:
|
||||
from services.sidecar_install import persistent_env_vars as _sidecar_env_vars
|
||||
PERSISTENT_KEYS |= _sidecar_env_vars()
|
||||
except Exception: # pragma: no cover — defensive: env panel > installer wiring
|
||||
pass
|
||||
|
||||
# Keys whose value must be a valid TCP port (1024–65535). Validated before
|
||||
# being set so a bad value never reaches uvicorn / the share listener.
|
||||
_PORT_KEYS = {"OMNIVOICE_PORT", "OMNIVOICE_SHARE_PORT", "OMNIVOICE_UI_PORT"}
|
||||
|
||||
@@ -136,7 +136,10 @@ async def ws_tts(websocket: WebSocket):
|
||||
kw["emo_text"] = data["emo_text"]
|
||||
if data.get("emo_audio"):
|
||||
kw["emo_audio"] = data["emo_audio"]
|
||||
if data.get("emo_alpha") != 1.0:
|
||||
# Default 1.0 when absent: a missing key must not trip the
|
||||
# `!= 1.0` branch into a KeyError (any minimal request that
|
||||
# omitted emo_alpha got an error frame instead of audio).
|
||||
if data.get("emo_alpha", 1.0) != 1.0:
|
||||
kw["emo_alpha"] = data["emo_alpha"]
|
||||
|
||||
# Resolve voice profile
|
||||
@@ -168,6 +171,18 @@ async def ws_tts(websocket: WebSocket):
|
||||
except Exception:
|
||||
kw["voice"] = voice
|
||||
|
||||
# Engine-agnostic text normalization (junk strip,
|
||||
# numbers→words, abbreviations) — the same pre-pass as
|
||||
# /generate, applied exactly ONCE per request, on the whole
|
||||
# text BEFORE the sentence chunker fans it out (so per-sentence
|
||||
# generates never re-normalize, and expanded abbreviations
|
||||
# can't confuse the sentence splitter). The request's
|
||||
# `language` is all this route knows (None → universal safety
|
||||
# filters only). Pref-gated (default ON), idempotent, never
|
||||
# raises.
|
||||
from services.text_normalization import normalize_for_tts
|
||||
text = normalize_for_tts(text, data.get("language"))
|
||||
|
||||
# Wave 1.4: split the request into sentences so the first
|
||||
# sentence's audio streams while later sentences are still
|
||||
# synthesizing — this is the time-to-first-audio win. The
|
||||
|
||||
@@ -164,6 +164,11 @@ class PreflightResponse(BaseModel):
|
||||
# Explicit field (PreflightResponse has no extra="allow") so the verdict
|
||||
# survives serialization instead of being silently dropped.
|
||||
gpu_routing: GpuRouting | None = None
|
||||
# Media-engine verdict (ffmpeg/ffprobe) — NOT a check row: an internal
|
||||
# dependency the app provisions for itself. Shape: {ready, acquire:
|
||||
# {state, progress, error}}. The wizard renders a quiet progress line /
|
||||
# failure card from it instead of "install ffmpeg" system requirements.
|
||||
media_tools: dict | None = None
|
||||
|
||||
|
||||
class InstallModelRequest(BaseModel):
|
||||
|
||||
@@ -70,6 +70,7 @@ _BASE_SCHEMA = """
|
||||
duration_seconds REAL,
|
||||
generation_time REAL,
|
||||
seed INTEGER DEFAULT NULL,
|
||||
starred INTEGER DEFAULT 0,
|
||||
created_at REAL,
|
||||
FOREIGN KEY (profile_id) REFERENCES voice_profiles(id)
|
||||
);
|
||||
|
||||
@@ -88,17 +88,33 @@ def _check_device() -> dict:
|
||||
|
||||
|
||||
def _check_ffmpeg() -> dict:
|
||||
"""Media engine (ffmpeg + ffprobe) — an internal dependency the app
|
||||
bundles/acquires itself, so a failure here means the self-heal also has
|
||||
nothing to work with (and the hint says where the controls live)."""
|
||||
ffmpeg = ffprobe = None
|
||||
try:
|
||||
from services.ffmpeg_utils import find_ffmpeg
|
||||
path = find_ffmpeg()
|
||||
from services.ffmpeg_utils import find_ffmpeg, find_ffprobe
|
||||
ffmpeg = find_ffmpeg()
|
||||
ffprobe = find_ffprobe()
|
||||
except Exception:
|
||||
path = None
|
||||
if path:
|
||||
return _check("ffmpeg", "ffmpeg", OK, str(path))
|
||||
pass
|
||||
if ffmpeg and ffprobe:
|
||||
return _check("ffmpeg", "Media engine (ffmpeg)", OK,
|
||||
f"ffmpeg: {ffmpeg}; ffprobe: {ffprobe}")
|
||||
if ffmpeg:
|
||||
return _check(
|
||||
"ffmpeg", "Media engine (ffmpeg)", WARN,
|
||||
f"ffmpeg: {ffmpeg}; ffprobe missing",
|
||||
"Media probing (Smart Fit, file inspection) is degraded. Open "
|
||||
"Settings > Audio tools and press Restore bundled to fetch the "
|
||||
"app's own ffprobe, or point it at a system copy there.",
|
||||
)
|
||||
return _check(
|
||||
"ffmpeg", "ffmpeg", FAIL,
|
||||
"not found on PATH or FFMPEG_PATH",
|
||||
"Dubbing and audio conversion need ffmpeg: brew install ffmpeg (macOS), apt install ffmpeg (Linux), or set the path in Settings > General.",
|
||||
"ffmpeg", "Media engine (ffmpeg)", FAIL,
|
||||
"no runnable ffmpeg in any tier (sidecar, bundled, system, custom)",
|
||||
"Dubbing and audio conversion are unavailable. The app normally "
|
||||
"provisions ffmpeg itself — open Settings > Audio tools and press "
|
||||
"Restore bundled (needs network once), or choose a system copy there.",
|
||||
)
|
||||
|
||||
|
||||
|
||||
+25
-1
@@ -46,6 +46,7 @@ _HINTS: dict[str, str] = {
|
||||
"UNSUPPORTED_VIDEO_URL": "This link isn't a directly downloadable video. Paste a direct video page (e.g. a youtube.com/watch?v=… or douyin.com/video/<id> link), not a share/profile/feed link — or download the file and drop it in directly.",
|
||||
"VIDEO_DOWNLOAD_NETWORK": "The connection to the video server dropped mid-download (often a transient CDN/network blip or a regional rate-limit). Just retry — OmniVoice already cleaned up the partial download. If it keeps failing, check your network/VPN.",
|
||||
"BROKEN_VENV": "The Python backend environment was moved or damaged. OmniVoice rebuilds it automatically on the next launch; if it keeps failing, use Clean & Retry on the setup screen.",
|
||||
"MODEL_CACHE_CORRUPT": "The model cache had broken file links — snapshot entries that no longer point at their downloaded data (interrupted renames or antivirus interference can cause this). OmniVoice repairs this automatically and retries the load once. If the error persists, quit OmniVoice, delete the model's models--<org>--<name> folder inside the Hugging Face cache, and restart — the model re-downloads automatically.",
|
||||
# HF_MIRROR_UNREACHABLE has a DYNAMIC hint (it names the configured mirror)
|
||||
# — see hf_mirror_hint(); build_failure special-cases it.
|
||||
}
|
||||
@@ -101,6 +102,18 @@ _HF_CONTEXT_MARKERS = (
|
||||
)
|
||||
|
||||
|
||||
def is_hf_connectivity_error(reason: Optional[str]) -> bool:
|
||||
"""True when *reason* looks like a network/connectivity failure of an HF
|
||||
download (DNS, refused/reset connections, timeouts, hub locate errors).
|
||||
|
||||
Signature match only — callers already in a download context (the model
|
||||
install worker, the cache auto-repair, the endpoint failover in
|
||||
services.endpoint_race) don't need the HF-context markers that
|
||||
``hf_mirror_hint`` requires. Never raises."""
|
||||
low = (reason or "").lower()
|
||||
return any(sig in low for sig in _HF_CONNECTIVITY_SIGNATURES)
|
||||
|
||||
|
||||
def configured_hf_mirror() -> str:
|
||||
"""The non-default Hugging Face endpoint (mirror) in effect, or "".
|
||||
|
||||
@@ -137,7 +150,7 @@ def hf_mirror_hint(reason: Optional[str]) -> str:
|
||||
if not mirror:
|
||||
return ""
|
||||
low = (reason or "").lower()
|
||||
if not any(sig in low for sig in _HF_CONNECTIVITY_SIGNATURES):
|
||||
if not is_hf_connectivity_error(low):
|
||||
return ""
|
||||
try:
|
||||
host = (urlsplit(mirror).netloc or "").lower()
|
||||
@@ -231,6 +244,17 @@ def classify(reason: str) -> str:
|
||||
# transformers + site-packages markers, which this signature lacks.
|
||||
if "errno 22" in low:
|
||||
return "OS_INVALID_ARGUMENT"
|
||||
# An HF cache whose snapshot entries don't resolve (dangling symlinks /
|
||||
# zero-byte stand-ins): transformers reports the weights missing ("does
|
||||
# not appear to have a file named pytorch_model.bin or model.safetensors")
|
||||
# even though the blobs are fully on disk. model_manager self-heals this
|
||||
# (delete broken entries → snapshot_download → retry once); the class here
|
||||
# covers both the raw transformers wording (any load surface can leak it)
|
||||
# and OmniVoice's own repair messages, so the user-facing error and the
|
||||
# auto bug report name the class and its automatic repair.
|
||||
if ("does not appear to have a file named" in low
|
||||
or "broken file link" in low):
|
||||
return "MODEL_CACHE_CORRUPT"
|
||||
if (
|
||||
"could not import module" in low
|
||||
or "autofeatureextractor" in low
|
||||
|
||||
+17
-6
@@ -15,6 +15,7 @@ import json
|
||||
import logging
|
||||
import os
|
||||
import tempfile
|
||||
import threading
|
||||
from typing import Any, Optional
|
||||
|
||||
from core.config import DATA_DIR
|
||||
@@ -23,6 +24,14 @@ logger = logging.getLogger("omnivoice.prefs")
|
||||
|
||||
_PREFS_PATH = os.path.join(DATA_DIR, "prefs.json")
|
||||
|
||||
# Serializes the load-modify-save cycle of every mutation. Writers run on
|
||||
# many threads (FastAPI's request threadpool, background workers like the
|
||||
# sidecar-engine installer); without the lock two concurrent set_/delete
|
||||
# calls interleave their read-modify-write and the later save silently
|
||||
# drops the other's key. Reads stay lock-free — the atomic os.replace in
|
||||
# _save guarantees they never see a torn file.
|
||||
_MUTATE_LOCK = threading.RLock()
|
||||
|
||||
|
||||
def _load() -> dict:
|
||||
try:
|
||||
@@ -60,16 +69,18 @@ def get(key: str, default: Any = None) -> Any:
|
||||
|
||||
|
||||
def set_(key: str, value: Any) -> None:
|
||||
data = _load()
|
||||
data[key] = value
|
||||
_save(data)
|
||||
with _MUTATE_LOCK:
|
||||
data = _load()
|
||||
data[key] = value
|
||||
_save(data)
|
||||
|
||||
|
||||
def delete(key: str) -> None:
|
||||
"""Remove *key* from prefs.json if present."""
|
||||
data = _load()
|
||||
data.pop(key, None)
|
||||
_save(data)
|
||||
with _MUTATE_LOCK:
|
||||
data = _load()
|
||||
data.pop(key, None)
|
||||
_save(data)
|
||||
|
||||
|
||||
def resolve(key: str, *, env: Optional[str] = None, default: Any = None) -> Any:
|
||||
|
||||
@@ -24,7 +24,7 @@ from pathlib import Path
|
||||
# tests/test_app_version.py::test_all_version_files_in_lockstep and bumped by
|
||||
# release.yml's version-bump job, so it stays equal to
|
||||
# pyproject/tauri.conf/Cargo/package.json.
|
||||
_FALLBACK_VERSION = "0.3.12"
|
||||
_FALLBACK_VERSION = "0.3.18"
|
||||
|
||||
|
||||
def _fallback_version() -> str:
|
||||
|
||||
@@ -41,9 +41,7 @@ from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
@@ -145,11 +143,12 @@ def _venv_python_path(venv_dir: Path) -> Path:
|
||||
"""Return the python executable path inside a venv directory.
|
||||
|
||||
Handles the Unix (``bin/python``) vs Windows (``Scripts/python.exe``)
|
||||
layout. No filesystem access — caller checks .is_file().
|
||||
layout. No filesystem access — caller checks .is_file(). Delegates to
|
||||
the canonical implementation in :mod:`services.sidecar_install` so the
|
||||
cross-platform venv-layout rule lives in exactly one place.
|
||||
"""
|
||||
if sys.platform == "win32":
|
||||
return venv_dir / "Scripts" / "python.exe"
|
||||
return venv_dir / "bin" / "python"
|
||||
from services.sidecar_install import _venv_python
|
||||
return _venv_python(venv_dir)
|
||||
|
||||
|
||||
def _probe_paths() -> list[Path]:
|
||||
@@ -189,14 +188,14 @@ def _venv_can_import_indextts(python_path: Path) -> bool:
|
||||
|
||||
|
||||
def _locate_uv() -> Optional[str]:
|
||||
"""Find the uv binary — bundled first (Tauri-set env var), else PATH."""
|
||||
bundled = os.environ.get("OMNIVOICE_BUNDLED_UV")
|
||||
if bundled and Path(bundled).is_file():
|
||||
return bundled
|
||||
sys_uv = shutil.which("uv")
|
||||
if sys_uv:
|
||||
return sys_uv
|
||||
return None
|
||||
"""Find the uv binary — bundled first (Tauri-set env var), else PATH.
|
||||
|
||||
Delegates to :mod:`services.sidecar_install`'s canonical resolver so the
|
||||
bundled-uv contract (env var name, precedence) can't drift between this
|
||||
lazy bootstrap and the one-click installer.
|
||||
"""
|
||||
from services.sidecar_install import _locate_uv as _canonical_locate_uv
|
||||
return _canonical_locate_uv()
|
||||
|
||||
|
||||
def _bootstrap_engines_venv(indextts_clone: Path) -> Path:
|
||||
|
||||
+75
-8
@@ -195,6 +195,18 @@ try:
|
||||
except Exception:
|
||||
pass # prefs.json missing or broken — fine on first run
|
||||
|
||||
# ── Activate the yt-dlp user-update overlay (Settings → Audio tools) ──────
|
||||
# Must run before anything imports yt_dlp so a user-updated version (stored
|
||||
# under DATA_DIR, surviving app updates and uv drift syncs) wins over the
|
||||
# locked wheel. Best-effort: a broken overlay must never block startup.
|
||||
try:
|
||||
from services.media_tools import activate_ytdlp_overlay
|
||||
activate_ytdlp_overlay()
|
||||
except Exception:
|
||||
# Best-effort by design: a broken/corrupt overlay must never block
|
||||
# startup — the locked wheel on sys.path is the fallback.
|
||||
pass
|
||||
|
||||
warnings.filterwarnings("ignore", category=UserWarning)
|
||||
torchaudio.set_audio_backend("soundfile")
|
||||
|
||||
@@ -375,6 +387,7 @@ from api.routers import (
|
||||
longform_jobs,
|
||||
pronunciation, # Expressive-TTS Spec 01: user pronunciation dictionary
|
||||
settings as settings_router, # Phase 1 AUTH-03: HF token save/clear/state
|
||||
media_tools as media_tools_router, # Audio tools: ffmpeg/ffprobe/yt-dlp management
|
||||
)
|
||||
from utils import hf_progress
|
||||
|
||||
@@ -504,6 +517,35 @@ async def _start_mcp_session_manager(session_manager, *, timeout: float):
|
||||
return task, stop, mounted
|
||||
|
||||
|
||||
async def _cancel_and_await_tasks(*tasks, timeout: float = 3.0) -> None:
|
||||
"""Cancel each background task and give it a bounded chance to actually
|
||||
finish before shutdown proceeds — ``None`` entries are skipped (a task
|
||||
that's conditionally created, e.g. ``capture_preload_task``, may not
|
||||
exist).
|
||||
|
||||
``task.cancel()`` alone is not enough for a task awaiting
|
||||
``run_in_executor()``: once the underlying OS thread is inside blocking
|
||||
native/import work, cancellation can't stop it, so cancel-and-move-on lets
|
||||
shutdown finish while that thread is still running — invisible to
|
||||
asyncio, but very much alive when the interpreter starts tearing down
|
||||
module state under it (#1000 class). Awaiting with a bound (instead of
|
||||
just cancelling) gives an early-stage task a real chance to exit cleanly
|
||||
first; a task that's genuinely still deep in blocking work times out here
|
||||
same as before, and the caller's own GPU-pool reset handles that case.
|
||||
"""
|
||||
for t in tasks:
|
||||
if t is None:
|
||||
continue
|
||||
t.cancel()
|
||||
for t in tasks:
|
||||
if t is None:
|
||||
continue
|
||||
try:
|
||||
await asyncio.wait_for(t, timeout=timeout)
|
||||
except (asyncio.CancelledError, asyncio.TimeoutError):
|
||||
pass
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
# Startup watchdog (#632): a silent hang during startup (e.g. a model-load /
|
||||
@@ -578,6 +620,7 @@ async def lifespan(app: FastAPI):
|
||||
# lean and the first dictation is instant instead of a cold model load.
|
||||
# OMNIVOICE_PRELOAD_CAPTURE_ASR=0 opts out; the warm-up is also skipped
|
||||
# under 4 GB free RAM (checked at warm time, not boot time).
|
||||
capture_preload_task = None # only assigned when the preload actually runs (#1000 class)
|
||||
if _env_flag("OMNIVOICE_PRELOAD_CAPTURE_ASR", default=True):
|
||||
async def _preload_capture_asr():
|
||||
await asyncio.sleep(_capture_preload_delay_s())
|
||||
@@ -646,14 +689,33 @@ async def lifespan(app: FastAPI):
|
||||
pass
|
||||
except Exception:
|
||||
pass
|
||||
idle_task.cancel()
|
||||
worker_task.cancel()
|
||||
# Wait for tasks to finish their current iteration
|
||||
for t in (idle_task, worker_task):
|
||||
try:
|
||||
await asyncio.wait_for(t, timeout=3.0)
|
||||
except (asyncio.CancelledError, asyncio.TimeoutError):
|
||||
pass
|
||||
# preload_task/capture_preload_task matter most here (#1000 class): a quit
|
||||
# mid-preload used to fall straight through to "Shutdown: done." while the
|
||||
# model load was still running on a GPU-pool thread — cancel() can't stop
|
||||
# a thread already inside blocking import/load work, so the process
|
||||
# reported a clean exit while that background thread was still mid-
|
||||
# `import transformers`, and got torn down by interpreter finalization
|
||||
# instead. That surfaced as a misleading "Could not import module
|
||||
# 'AutoFeatureExtractor'" — transformers' own generic lazy-import wrapper,
|
||||
# not a real dependency problem. Awaiting here lets an early-stage load
|
||||
# (still importing, not yet mid weight-download) finish cleanly before we
|
||||
# report done; a load that's genuinely deep into a multi-GB download still
|
||||
# times out — _reset_gpu_pool() below abandons it either way.
|
||||
#
|
||||
# 20s, not the original 3s (code-review finding post-merge): a cold
|
||||
# transformers import alone can take longer than 3s on a slow disk or a
|
||||
# first-ever launch, so the original bound left a real residual window —
|
||||
# cancellation detaches the asyncio task, but the underlying OS thread
|
||||
# keeps running past it, and shutdown could still report "done" while
|
||||
# that thread was alive. Python cannot forcibly kill a running thread, so
|
||||
# no finite bound eliminates this outright — 20s just shrinks the window
|
||||
# from "any preload" to "an unusually slow cold-import," which is the
|
||||
# practical ceiling before a longer shutdown itself becomes the
|
||||
# complaint. A thread that's still running past 20s was never going to
|
||||
# finish in a shutdown-appropriate timeframe regardless.
|
||||
await _cancel_and_await_tasks(
|
||||
idle_task, worker_task, preload_task, capture_preload_task, timeout=20.0,
|
||||
)
|
||||
# Unload the model and free GPU memory
|
||||
try:
|
||||
import services.model_manager as mm
|
||||
@@ -661,6 +723,10 @@ async def lifespan(app: FastAPI):
|
||||
mm.model = None
|
||||
logger.info("Shutdown: model unloaded.")
|
||||
mm.free_vram()
|
||||
# Abandon a still-running preload's GPU-pool thread (Python can't kill
|
||||
# a thread mid blocking call) so it can't outlive this shutdown block
|
||||
# holding a reference into module state that's about to be torn down.
|
||||
mm._reset_gpu_pool()
|
||||
except Exception:
|
||||
pass
|
||||
# Run GC to release any remaining references
|
||||
@@ -992,6 +1058,7 @@ app.include_router(audiobook.router)
|
||||
app.include_router(longform_jobs.router)
|
||||
app.include_router(pronunciation.router) # Expressive-TTS Spec 01: pronunciation dictionary
|
||||
app.include_router(settings_router.router) # Phase 1 AUTH-03 endpoints
|
||||
app.include_router(media_tools_router.router) # Settings → Audio tools + wizard media-engine self-heal
|
||||
from api.routers import mcp_bindings as _mcp_bindings_router # noqa: E402
|
||||
app.include_router(_mcp_bindings_router.router) # Wave 2.2 per-agent voice bindings
|
||||
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
"""Generation takes: starred flag on generation_history
|
||||
|
||||
Revision ID: 0009_generation_history_starred
|
||||
Revises: 0008_pronunciation_dictionary
|
||||
Create Date: 2026-07-10 00:00:00.000000
|
||||
|
||||
Adds ``generation_history.starred INTEGER DEFAULT 0`` — the "keep this
|
||||
take" flag behind the Studio takes rail. Starred takes are exempt from the
|
||||
retention cap that prunes old generations, and star/unstar round-trips through
|
||||
``PUT /history/{id}/starred``.
|
||||
|
||||
Additive + idempotent (guarded by PRAGMA table_info, matching 0002/0003), so
|
||||
re-running on a fresh-install DB where ``_BASE_SCHEMA`` already declares the
|
||||
column is a no-op (Backward-compatible project data constraint). The same
|
||||
column is mirrored into ``core/db.py::_BASE_SCHEMA`` so fresh installs and
|
||||
migrated DBs converge on an identical end-state — and DBs where alembic can't
|
||||
run at all pick it up via ``_reconcile_additive_columns`` (the #552/#547
|
||||
self-heal), the dual-path discipline.
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision: str = "0009_generation_history_starred"
|
||||
down_revision: Union[str, None] = "0008_pronunciation_dictionary"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def _has_column(table: str, column: str) -> bool:
|
||||
bind = op.get_bind()
|
||||
rows = bind.execute(sa.text(f"PRAGMA table_info({table})")).fetchall()
|
||||
return any(r[1] == column for r in rows)
|
||||
|
||||
|
||||
def _has_table(name: str) -> bool:
|
||||
bind = op.get_bind()
|
||||
row = bind.execute(
|
||||
sa.text("SELECT name FROM sqlite_master WHERE type='table' AND name=:n"),
|
||||
{"n": name},
|
||||
).fetchone()
|
||||
return row is not None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# A DB that somehow missed init has no generation_history at all — the
|
||||
# startup self-heal (#710) creates it with the column already present, so
|
||||
# ALTERing here would be both impossible and unnecessary.
|
||||
if not _has_table("generation_history"):
|
||||
return
|
||||
if not _has_column("generation_history", "starred"):
|
||||
# nullable + DEFAULT 0 to byte-match _BASE_SCHEMA's declaration
|
||||
# (`starred INTEGER DEFAULT 0`) — the dual-path convergence test
|
||||
# compares table shape between a migrated DB and a fresh install.
|
||||
op.add_column(
|
||||
"generation_history",
|
||||
sa.Column("starred", sa.Integer(), nullable=True, server_default="0"),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
if _has_table("generation_history") and _has_column("generation_history", "starred"):
|
||||
op.drop_column("generation_history", "starred")
|
||||
@@ -122,6 +122,12 @@ class TranslateSegment(BaseModel):
|
||||
# Available time slot (end - start, seconds) for rate-ratio prediction
|
||||
# and the cinematic slot-fit pass. Same silent-drop fix as `direction`.
|
||||
slot_seconds: Optional[float] = None
|
||||
# Timeline position (seconds) — lets the duration planner borrow silence
|
||||
# from the gap to the NEXT segment when classifying fits/tight/impossible
|
||||
# (services/duration_planner.py). Optional: old clients that only send
|
||||
# slot_seconds still get rate_ratio badges, just no plan verdicts.
|
||||
start: Optional[float] = None
|
||||
end: Optional[float] = None
|
||||
|
||||
class TranslateRequest(BaseModel):
|
||||
segments: List[TranslateSegment]
|
||||
@@ -137,6 +143,22 @@ class TranslateRequest(BaseModel):
|
||||
# voseo: "vos sos" instead of "tú eres"). Non-LLM providers (Argos, NLLB,
|
||||
# Google) can't honor it; the response then carries dialect_applied=false.
|
||||
dialect: Optional[str] = None
|
||||
# Two-stage LLM translation quality (provider="openai" only; MT engines
|
||||
# ignore both). None = default ON for the LLM engine.
|
||||
# auto_glossary — one up-front LLM pass over the full transcript extracts
|
||||
# a theme summary + terminology map, merged with `glossary` (user
|
||||
# entries win) and injected into every per-segment prompt.
|
||||
# reflect — per-segment critique→rewrite polish after the direct
|
||||
# translation (2 extra LLM calls per segment; failures silently keep
|
||||
# the direct translation).
|
||||
auto_glossary: Optional[bool] = None
|
||||
reflect: Optional[bool] = None
|
||||
# Opt-in LLM condensation (default OFF): for segments the duration
|
||||
# planner classifies "impossible", ask the configured LLM for a shorter
|
||||
# meaning-preserving rewrite and attach it as plan.suggested_text — a
|
||||
# per-segment suggestion the user applies manually, never auto-applied.
|
||||
# No LLM configured / LLM failure → silently no suggestion.
|
||||
condense: Optional[bool] = False
|
||||
|
||||
class DubIngestUrlRequest(BaseModel):
|
||||
url: str
|
||||
|
||||
@@ -29,6 +29,8 @@ import os
|
||||
import re
|
||||
import threading
|
||||
from abc import ABC, abstractmethod
|
||||
from collections import OrderedDict
|
||||
from typing import Optional
|
||||
|
||||
logger = logging.getLogger("omnivoice.asr")
|
||||
|
||||
@@ -1634,6 +1636,161 @@ class FunASRBackend(ASRBackend):
|
||||
pass
|
||||
|
||||
|
||||
# ── OpenAI-compatible remote transcription (#877 — Qwen3-ASR / FunASR / any
|
||||
# compatible server, today, without waiting on transformers to catch up) ──
|
||||
#
|
||||
# transformers doesn't yet ship a stable Qwen3-ASR integration (issue #877),
|
||||
# but a self-hosted Qwen3-ASR/FunASR/SenseVoice server exposing an
|
||||
# OpenAI-compatible `POST /v1/audio/transcriptions` endpoint — or OpenAI's own
|
||||
# Whisper API — is usable right now. This backend is a pure network client:
|
||||
# no model runs locally, so it needs no install and claims no GPU.
|
||||
#
|
||||
# Settings mirror the LLM-providers convention exactly (services/
|
||||
# llm_providers.py): base_url/model are plain settings_store text rows; the
|
||||
# API key is Fernet-encrypted via settings_store.set_secret/get_secret — never
|
||||
# a .env row, never echoed back to the client. Optional: some self-hosted
|
||||
# servers (vLLM, LM Studio-style) don't check the key at all.
|
||||
|
||||
_ASR_OPENAI_COMPAT_BASE_URL_KEY = "asr.openai_compat.base_url"
|
||||
_ASR_OPENAI_COMPAT_MODEL_KEY = "asr.openai_compat.model"
|
||||
_ASR_OPENAI_COMPAT_SECRET_NAME = "asr_openai_compat_key"
|
||||
|
||||
|
||||
def resolve_openai_compat_asr_base_url() -> str:
|
||||
from services import settings_store
|
||||
return (
|
||||
os.environ.get("ASR_OPENAI_COMPAT_BASE_URL")
|
||||
or settings_store.get_text(_ASR_OPENAI_COMPAT_BASE_URL_KEY)
|
||||
or ""
|
||||
)
|
||||
|
||||
|
||||
def resolve_openai_compat_asr_model() -> str:
|
||||
from services import settings_store
|
||||
return (
|
||||
os.environ.get("ASR_OPENAI_COMPAT_MODEL")
|
||||
or settings_store.get_text(_ASR_OPENAI_COMPAT_MODEL_KEY)
|
||||
or "whisper-1"
|
||||
)
|
||||
|
||||
|
||||
def resolve_openai_compat_asr_api_key() -> Optional[str]:
|
||||
"""Env → encrypted stored key → None. Unlike LLM providers, no 'local'
|
||||
sentinel: many self-hosted transcription servers accept an empty/omitted
|
||||
Authorization header outright, so the OpenAI SDK is constructed with
|
||||
``api_key="not-needed"`` (a non-empty placeholder the SDK requires) when
|
||||
this returns None, rather than treating a keyless server as unconfigured.
|
||||
"""
|
||||
from services import settings_store
|
||||
return os.environ.get("ASR_OPENAI_COMPAT_API_KEY") or settings_store.get_secret(
|
||||
_ASR_OPENAI_COMPAT_SECRET_NAME
|
||||
)
|
||||
|
||||
|
||||
def openai_compat_asr_has_key() -> bool:
|
||||
"""Whether a key is configured, without ever decrypting it — mirrors
|
||||
llm_providers.has_key()'s no-plaintext-round-trip contract."""
|
||||
from services import settings_store
|
||||
if os.environ.get("ASR_OPENAI_COMPAT_API_KEY"):
|
||||
return True
|
||||
return _ASR_OPENAI_COMPAT_SECRET_NAME in settings_store.list_secret_names()
|
||||
|
||||
|
||||
class OpenAICompatASRBackend(ASRBackend):
|
||||
"""Remote transcription via any OpenAI-compatible server.
|
||||
|
||||
Adapts whatever the server returns into this module's expected shape.
|
||||
Prefers `response_format="verbose_json"` for real per-segment timestamps
|
||||
(OpenAI's own API and most compatible servers support it); falls back to
|
||||
plain text with rough single-segment bounds — mirroring
|
||||
MoonshineASRBackend's degraded shape — for minimal servers that reject it.
|
||||
"""
|
||||
id = "openai-compat-asr"
|
||||
display_name = "OpenAI-compatible (remote server)"
|
||||
gpu_compat = ("cpu",) # network client only — no local compute
|
||||
|
||||
def __init__(self):
|
||||
self._base_url = resolve_openai_compat_asr_base_url()
|
||||
self._model = resolve_openai_compat_asr_model()
|
||||
|
||||
@classmethod
|
||||
def is_available(cls) -> tuple[bool, str]:
|
||||
if not resolve_openai_compat_asr_base_url():
|
||||
return False, "Configure a server endpoint in Settings → Engines"
|
||||
try:
|
||||
import openai # noqa: F401
|
||||
except ImportError:
|
||||
return False, "openai package not installed. Install with: uv pip install openai"
|
||||
return True, "ready"
|
||||
|
||||
def _client(self):
|
||||
from openai import OpenAI
|
||||
api_key = resolve_openai_compat_asr_api_key() or "not-needed"
|
||||
# max_retries=0: mirrors llm_skills.resolve_skill_client — a
|
||||
# rate-limited/slow server retrying inside the SDK would blow past
|
||||
# whatever bounded timeout the caller (dub transcribe, dictation)
|
||||
# expects from a single call.
|
||||
return OpenAI(base_url=self._base_url, api_key=api_key, max_retries=0)
|
||||
|
||||
def transcribe(self, audio_path: str, *, word_timestamps: bool = True) -> dict:
|
||||
logger.info(
|
||||
"OpenAI-compat ASR transcribing %s (base_url=%s, model=%s)",
|
||||
audio_path, self._base_url, self._model,
|
||||
)
|
||||
client = self._client()
|
||||
try:
|
||||
with open(audio_path, "rb") as f:
|
||||
try:
|
||||
resp = client.audio.transcriptions.create(
|
||||
file=f, model=self._model, response_format="verbose_json",
|
||||
)
|
||||
except Exception:
|
||||
# Minimal/older compatible servers reject verbose_json
|
||||
# outright — retry plain before treating it as a real
|
||||
# failure. Re-open: the SDK may have partially consumed
|
||||
# the file handle on the first attempt.
|
||||
f.seek(0)
|
||||
resp = client.audio.transcriptions.create(
|
||||
file=f, model=self._model, response_format="json",
|
||||
)
|
||||
except Exception as exc:
|
||||
# Never leak a raw SDK/httpx exception object (auth headers,
|
||||
# connection internals) straight into a user-facing message —
|
||||
# same convention as generation.py's _safe_exc_text (#977 class).
|
||||
raise RuntimeError(
|
||||
f"OpenAI-compatible ASR server at {self._base_url!r} failed: "
|
||||
f"{type(exc).__name__}: {exc}"
|
||||
) from exc
|
||||
return self._adapt_response(resp)
|
||||
|
||||
@staticmethod
|
||||
def _adapt_response(resp) -> dict:
|
||||
segments_out = []
|
||||
# verbose_json: resp.segments is a list of objects with start/end/text.
|
||||
raw_segments = getattr(resp, "segments", None)
|
||||
if raw_segments:
|
||||
for seg in raw_segments:
|
||||
seg_dict = seg if isinstance(seg, dict) else seg.model_dump()
|
||||
segments_out.append({
|
||||
"text": (seg_dict.get("text") or "").strip(),
|
||||
"start": seg_dict.get("start", 0.0),
|
||||
"end": seg_dict.get("end", 0.0),
|
||||
"words": [], # word-level timing isn't part of this API
|
||||
})
|
||||
else:
|
||||
# Plain text response (json/text format) — single-segment shape,
|
||||
# matching MoonshineASRBackend's degraded fallback exactly.
|
||||
text = (getattr(resp, "text", None) or "").strip()
|
||||
if text:
|
||||
segments_out.append({"text": text, "start": 0.0, "end": None, "words": []})
|
||||
chunks = [
|
||||
{"text": seg["text"], "timestamp": (seg["start"], seg["end"])}
|
||||
for seg in segments_out
|
||||
]
|
||||
language = getattr(resp, "language", None) or "en"
|
||||
return {"chunks": chunks, "segments": segments_out, "language": language}
|
||||
|
||||
|
||||
def _isolated_faster_whisper():
|
||||
"""Lazy import so the subprocess_asr → subprocess_backend chain isn't
|
||||
pulled in at registry definition time."""
|
||||
@@ -1689,6 +1846,7 @@ _REGISTRY: dict[str, type[ASRBackend]] = _LazyASRRegistry({
|
||||
"moonshine": MoonshineASRBackend,
|
||||
"funasr": FunASRBackend,
|
||||
"sherpa-onnx-asr": SherpaDictationBackend,
|
||||
"openai-compat-asr": OpenAICompatASRBackend,
|
||||
# "faster-whisper-isolated": resolved lazily (crash-isolated subprocess).
|
||||
})
|
||||
|
||||
@@ -1713,6 +1871,13 @@ _INSTALL_HINTS: dict[str, str] = {
|
||||
"moonshine": "pip install useful-moonshine (edge/CPU-optimized ASR)",
|
||||
"funasr": "pip install funasr (SenseVoiceSmall + FSMN-VAD; CUDA or CPU)",
|
||||
"sherpa-onnx-asr": "uv add sherpa-onnx (ONNX live dictation; CPU, cross-platform)",
|
||||
"openai-compat-asr": (
|
||||
"No install needed — configure a server endpoint in Settings → "
|
||||
"Engines. Points OmniVoice at any OpenAI-compatible transcription "
|
||||
"server (a self-hosted Qwen3-ASR/FunASR/SenseVoice server, OpenAI's "
|
||||
"own Whisper API, or similar) — a path to Qwen3-ASR today, without "
|
||||
"waiting on a direct transformers integration."
|
||||
),
|
||||
"faster-whisper-isolated": (
|
||||
"No extra install (reuses faster-whisper). Escape hatch for hanging "
|
||||
"transcribes: runs ASR in a separate process that can be force-killed "
|
||||
@@ -1862,6 +2027,38 @@ def get_active_asr_backend(*, asr_pipe=None) -> ASRBackend:
|
||||
return cls()
|
||||
|
||||
|
||||
# ── Reference-transcript cache (#1032) ──────────────────────────────────────
|
||||
# `get_active_asr_backend()` returns a FRESH backend instance per call for the
|
||||
# whisper family, so every `transcribe_reference` used to reload whisper
|
||||
# weights from scratch — a multi-second (CPU: tens of seconds) hit on EVERY
|
||||
# /generate whose reference clip has no stored transcript (#308 introduced the
|
||||
# call; profiles saved without a transcript hit it per request). The reference
|
||||
# audio is identical across those requests, so cache the *transcript* keyed by
|
||||
# the file's content hash: no model or VRAM is held, repeated generates with
|
||||
# the same clip skip ASR entirely. Bounded LRU; failures (None) are never
|
||||
# cached so a transient ASR problem still retries next request.
|
||||
_REF_TRANSCRIPT_CACHE_MAX = 64
|
||||
_ref_transcript_cache: "OrderedDict[str, str]" = OrderedDict()
|
||||
_ref_transcript_lock = threading.Lock()
|
||||
|
||||
|
||||
def _ref_audio_fingerprint(audio_path: str) -> str | None:
|
||||
"""sha256 of the clip's bytes, or None when unreadable (→ no caching).
|
||||
|
||||
Content-keyed (not path-keyed) because ad-hoc clone uploads land in a new
|
||||
NamedTemporaryFile per request — the path changes, the bytes don't.
|
||||
Reference clips are seconds long, so hashing is negligible next to ASR."""
|
||||
import hashlib
|
||||
try:
|
||||
h = hashlib.sha256()
|
||||
with open(audio_path, "rb") as f:
|
||||
for chunk in iter(lambda: f.read(1 << 20), b""):
|
||||
h.update(chunk)
|
||||
return h.hexdigest()
|
||||
except OSError:
|
||||
return None
|
||||
|
||||
|
||||
def transcribe_reference(audio_path: str) -> str | None:
|
||||
"""Transcribe a voice-clone reference clip with the active ASR backend.
|
||||
|
||||
@@ -1873,7 +2070,16 @@ def transcribe_reference(audio_path: str) -> str | None:
|
||||
the model-attached pipeline is only reached when it is genuinely the last
|
||||
resort. Returns ``None`` on any failure — callers pass ``ref_text=None``
|
||||
through and the model's built-in fallback still gets its chance.
|
||||
|
||||
Results are cached by audio content (#1032) — see the cache notes above.
|
||||
"""
|
||||
fingerprint = _ref_audio_fingerprint(audio_path)
|
||||
if fingerprint is not None:
|
||||
with _ref_transcript_lock:
|
||||
cached = _ref_transcript_cache.get(fingerprint)
|
||||
if cached is not None:
|
||||
_ref_transcript_cache.move_to_end(fingerprint)
|
||||
return cached
|
||||
try:
|
||||
backend = get_active_asr_backend()
|
||||
except Exception as e: # noqa: BLE001 — never let ASR break generation
|
||||
@@ -1897,6 +2103,12 @@ def transcribe_reference(audio_path: str) -> str | None:
|
||||
(seg.get("text") or "").strip() for seg in result.get("segments", [])
|
||||
)
|
||||
text = (text or "").strip()
|
||||
if text and fingerprint is not None:
|
||||
with _ref_transcript_lock:
|
||||
_ref_transcript_cache[fingerprint] = text
|
||||
_ref_transcript_cache.move_to_end(fingerprint)
|
||||
while len(_ref_transcript_cache) > _REF_TRANSCRIPT_CACHE_MAX:
|
||||
_ref_transcript_cache.popitem(last=False)
|
||||
return text or None
|
||||
|
||||
|
||||
|
||||
@@ -147,6 +147,45 @@ def normalize_audio(audio_tensor, target_dBFS=-2.0):
|
||||
return audio_tensor
|
||||
|
||||
|
||||
def trim_trailing_silence(
|
||||
audio_tensor: torch.Tensor,
|
||||
sample_rate: int,
|
||||
keep_tail_s: float = 0.3,
|
||||
) -> torch.Tensor:
|
||||
"""Trim trailing near-silence from a generated clip, keeping a short
|
||||
natural tail of ``keep_tail_s`` seconds after the last voiced sample.
|
||||
|
||||
Amplitude-based SILENCE trim only — no content analysis of any kind.
|
||||
Uses the same -50 dBFS silence floor as :func:`normalize_audio`: the last
|
||||
sample above that floor marks the end of speech, and everything more than
|
||||
``keep_tail_s`` past it is dropped.
|
||||
|
||||
Guaranteed no-op cases (input returned as-is, same object):
|
||||
• the trailing quiet span is already ≤ ``keep_tail_s`` (clean output);
|
||||
• the entire clip sits below the floor (dead render — downstream
|
||||
dead-render guards own that case, we must not shrink their evidence);
|
||||
• empty input.
|
||||
|
||||
Accepts ``(n,)`` or ``(channels, n)`` tensors; the returned tensor keeps
|
||||
the input's shape convention.
|
||||
"""
|
||||
if audio_tensor.numel() == 0:
|
||||
return audio_tensor
|
||||
# -50 dBFS ≈ 0.00316 linear — matches normalize_audio's silence floor.
|
||||
floor = 10 ** (-50.0 / 20.0)
|
||||
envelope = torch.abs(audio_tensor)
|
||||
if envelope.ndim > 1:
|
||||
envelope = envelope.amax(dim=tuple(range(envelope.ndim - 1)))
|
||||
voiced = torch.nonzero(envelope > floor)
|
||||
if voiced.numel() == 0:
|
||||
return audio_tensor
|
||||
last_voiced = int(voiced[-1].item())
|
||||
end = last_voiced + 1 + int(keep_tail_s * sample_rate)
|
||||
if end >= audio_tensor.shape[-1]:
|
||||
return audio_tensor
|
||||
return audio_tensor[..., :end]
|
||||
|
||||
|
||||
def apply_effects_chain(audio_tensor, sample_rate: int, chain: list[dict]) -> torch.Tensor:
|
||||
"""Apply a chain of named effects to an audio tensor.
|
||||
|
||||
|
||||
@@ -101,6 +101,7 @@ def synthesize_chapter(
|
||||
*,
|
||||
crossfade_ms: int = 50,
|
||||
lexicon: Optional[dict] = None,
|
||||
segment_cache: Optional["object"] = None,
|
||||
):
|
||||
"""Render a chapter's spans to one waveform via an injected ``synth``.
|
||||
|
||||
@@ -111,6 +112,12 @@ def synthesize_chapter(
|
||||
crossfaded; inter-span ``pause_ms_after`` becomes silence. ``lexicon`` (when
|
||||
given) respells each span's text before chunking so the engine pronounces
|
||||
tricky words correctly; a ``None``/empty lexicon is a no-op pass-through.
|
||||
``segment_cache`` (when given — a :class:`services.longform_render.
|
||||
SegmentCache`) is consulted per spoken span: a cached segment WAV is reused
|
||||
instead of synthesizing, and every freshly rendered span is stored the
|
||||
moment it finishes — so a one-sentence edit re-renders one segment and an
|
||||
interrupted chapter resumes from its finished segments. Pauses are
|
||||
synthesized silence and never touch the cache.
|
||||
|
||||
Returns ``(audio_tensor, duration_seconds)``. torch + chunked_tts are
|
||||
imported lazily so this module stays import-light for the pure parser path.
|
||||
@@ -122,13 +129,19 @@ def synthesize_chapter(
|
||||
items: list = [] # ("a", tensor) for audio, ("s", n_samples) for silence
|
||||
for span in spans:
|
||||
if span.text:
|
||||
chunks = split_text_into_chunks(apply_lexicon(span.text, lexicon))
|
||||
rendered = [synth(c, span.voice_id, span.speed) for c in chunks]
|
||||
rendered = [r for r in rendered if r is not None and getattr(r, "numel", lambda: 0)()]
|
||||
if len(rendered) == 1:
|
||||
items.append(("a", rendered[0]))
|
||||
elif rendered:
|
||||
items.append(("a", concatenate_audio_chunks(rendered, sample_rate, crossfade_ms=crossfade_ms)))
|
||||
audio = segment_cache.load(span) if segment_cache is not None else None
|
||||
if audio is None:
|
||||
chunks = split_text_into_chunks(apply_lexicon(span.text, lexicon))
|
||||
rendered = [synth(c, span.voice_id, span.speed) for c in chunks]
|
||||
rendered = [r for r in rendered if r is not None and getattr(r, "numel", lambda: 0)()]
|
||||
if len(rendered) == 1:
|
||||
audio = rendered[0]
|
||||
elif rendered:
|
||||
audio = concatenate_audio_chunks(rendered, sample_rate, crossfade_ms=crossfade_ms)
|
||||
if audio is not None and segment_cache is not None:
|
||||
segment_cache.store(span, audio)
|
||||
if audio is not None:
|
||||
items.append(("a", audio))
|
||||
if span.pause_ms_after > 0:
|
||||
n = int(sample_rate * span.pause_ms_after / 1000.0)
|
||||
if n > 0:
|
||||
|
||||
@@ -151,9 +151,14 @@ async def generate_segments_batched(
|
||||
# Raw: skip all DSP — return raw model output
|
||||
return audio_out
|
||||
|
||||
# TODO(#312): this route runs the OmniVoice model directly (not the active
|
||||
# backend), so VoxCPM2 never reaches it. When these routes become
|
||||
# engine-aware, guard with `if not getattr(backend, "applies_own_mastering", False)`.
|
||||
# NOTE(#312, closed): the live routes (generation.py,
|
||||
# dub_generate.py, batch.py, tts_stream.py) are engine-aware
|
||||
# and honor `applies_own_mastering` themselves. This module
|
||||
# still drives the OmniVoice model directly and currently has
|
||||
# NO call sites — it's an unintegrated throughput experiment.
|
||||
# If it is ever wired into a route, thread the active backend
|
||||
# through instead of `model` and guard the mastering below
|
||||
# with `if not getattr(backend, "applies_own_mastering", False)`.
|
||||
mastered = apply_mastering(audio_out, sample_rate=sr)
|
||||
effect_chain = get_effect_chain(seg_effect_preset)
|
||||
if effect_chain:
|
||||
|
||||
@@ -0,0 +1,329 @@
|
||||
"""Pre-synthesis duration planning for dub segments.
|
||||
|
||||
The Smart Fit planner (services/fit_planner.py) reconciles dubbed audio
|
||||
with the timeline AFTER synthesis — by then a doomed segment has already
|
||||
burned GPU time and can only be sped up or trimmed. This module predicts
|
||||
BEFORE TTS whether a translated segment can possibly fit its slot, so the
|
||||
UI can badge it (and optionally offer a shorter rewrite) while the text is
|
||||
still cheap to change. It never blocks generation — it informs.
|
||||
|
||||
Three pieces, all pure and unit-testable:
|
||||
|
||||
1. **Estimator** — predict the natural speech duration of target-language
|
||||
text. Self-calibrating: segments already synthesized in this job carry
|
||||
``(chars, natural duration)`` records (written by dub_generate for every
|
||||
natural-rate strategy), and the median chars-per-second of those is a
|
||||
far better predictor for *this* voice/engine/language than any table.
|
||||
With no (or too little) calibration data it falls back to the
|
||||
conservative static per-language rate table in ``services.speech_rate``
|
||||
(the same one the rate-ratio badge uses).
|
||||
|
||||
2. **Classifier** — per segment, compare the estimate against the
|
||||
*available* time: the slot plus silence borrowable from the gap to the
|
||||
next segment (mirroring fit_planner's slack absorption, but with a
|
||||
deliberate cap — see ``GAP_BORROW_MAX_S``). The verdict thresholds are
|
||||
derived from the SAME ``FitParams`` caps fit_planner enforces, so:
|
||||
|
||||
fits need ≤ max_audio_only_rate — absorbed imperceptibly
|
||||
tight need ≤ what the caps absorb — audible speed-up and/or
|
||||
video slow-down
|
||||
impossible beyond the caps — fit_planner will trim
|
||||
|
||||
3. **Condensation** (optional, caller-gated) — for ``impossible`` segments,
|
||||
ask the configured LLM for a meaning-preserving shorter rewrite
|
||||
targeting the available duration. Strictly best-effort: no LLM, an LLM
|
||||
error, or a divergent reply all degrade to a no-op.
|
||||
|
||||
No I/O, no torch; the only side-effectful function is ``condense_for_slot``
|
||||
(network LLM call), which callers opt into explicitly.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
from typing import Iterable, Optional
|
||||
|
||||
from services.fit_planner import MAX_AUDIO_RATE_HARD, FitParams
|
||||
from services.llm_backend import OffBackend, get_active_llm_backend
|
||||
from services.speech_rate import expected_duration
|
||||
# Shared LLM-output divergence guard (target-script + length window +
|
||||
# critique-echo) — same seam speech_rate's Autofit pass uses.
|
||||
from services.translator import refine_output_ok
|
||||
|
||||
logger = logging.getLogger("omnivoice.duration_planner")
|
||||
|
||||
# LLM Skills registry id — condensation is the same "make the line fit its
|
||||
# slot" skill family as the Autofit pass, so it routes (and can be disabled)
|
||||
# through the same Settings → LLM Skills entry.
|
||||
_SKILL_ID = "slot_fitting"
|
||||
|
||||
# ── Calibration ─────────────────────────────────────────────────────────
|
||||
|
||||
# A calibration only counts once this many usable samples exist — below
|
||||
# that, one odd segment (a sound effect, a mumbled clone ref) would swing
|
||||
# the estimate more than the static table's error.
|
||||
MIN_CALIBRATION_SAMPLES = 3
|
||||
# Per-sample sanity floor: shorter/tinier segments carry more silence
|
||||
# padding and TTS ramp-up than speech, so their chars/sec is noise.
|
||||
MIN_SAMPLE_DUR_S = 0.4
|
||||
MIN_SAMPLE_CHARS = 4
|
||||
|
||||
# How far a segment may borrow into the silent gap before the next segment
|
||||
# (or the video tail). fit_planner itself absorbs the WHOLE gap, so this cap
|
||||
# makes the pre-synthesis verdict deliberately conservative: a huge gap
|
||||
# (scene change, music bed) is real slack at mix time, but planning speech
|
||||
# to sprawl seconds past its slot is rarely what the user wants — and the
|
||||
# estimate is fuzzy enough that promising it would over-sell.
|
||||
GAP_BORROW_MAX_S = 3.0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Calibration:
|
||||
"""Observed speech rate for one (job, language) pair."""
|
||||
cps: float # chars per second at natural TTS rate
|
||||
samples: int # how many segments backed it
|
||||
|
||||
|
||||
def calibrate_cps(samples: Iterable[tuple[float, float]]) -> Optional[Calibration]:
|
||||
"""Derive a chars-per-second calibration from ``(chars, natural_dur_s)``
|
||||
pairs of already-synthesized segments.
|
||||
|
||||
Median of the per-segment rates — robust against the occasional outlier
|
||||
(a segment that's mostly a breath, an engine hiccup) that would drag a
|
||||
mean. Returns None when fewer than ``MIN_CALIBRATION_SAMPLES`` usable
|
||||
samples exist; callers then fall back to the static table.
|
||||
"""
|
||||
rates: list[float] = []
|
||||
for chars, dur in samples:
|
||||
try:
|
||||
chars = float(chars)
|
||||
dur = float(dur)
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
if dur >= MIN_SAMPLE_DUR_S and chars >= MIN_SAMPLE_CHARS:
|
||||
rates.append(chars / dur)
|
||||
if len(rates) < MIN_CALIBRATION_SAMPLES:
|
||||
return None
|
||||
rates.sort()
|
||||
n = len(rates)
|
||||
mid = n // 2
|
||||
median = rates[mid] if n % 2 else (rates[mid - 1] + rates[mid]) / 2.0
|
||||
if median <= 0:
|
||||
return None
|
||||
return Calibration(cps=median, samples=n)
|
||||
|
||||
|
||||
def calibration_from_job(job: dict, lang: str) -> Optional[Calibration]:
|
||||
"""Build a Calibration from the ``seg_natural_durs_by_lang`` records
|
||||
dub_generate persists on the job. Tolerates any legacy/partial shape."""
|
||||
try:
|
||||
recs = (job.get("seg_natural_durs_by_lang") or {}).get(lang) or {}
|
||||
return calibrate_cps(
|
||||
(r.get("chars", 0), r.get("dur", 0))
|
||||
for r in recs.values()
|
||||
if isinstance(r, dict)
|
||||
)
|
||||
except Exception as e: # noqa: BLE001 — calibration is best-effort by design
|
||||
logger.debug("calibration_from_job skipped: %s", e)
|
||||
return None
|
||||
|
||||
|
||||
# ── Estimator ───────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def estimate_natural_duration(
|
||||
text: str, lang: str, calibration: Optional[Calibration] = None,
|
||||
) -> float:
|
||||
"""Predicted natural-rate speech duration (seconds) of ``text``.
|
||||
|
||||
Calibrated rate when available, else the static per-language table
|
||||
(``speech_rate.expected_duration``, 13 cps default for unknown codes).
|
||||
"""
|
||||
text = (text or "").strip()
|
||||
if not text:
|
||||
return 0.0
|
||||
if calibration is not None and calibration.cps > 0:
|
||||
return len(text) / calibration.cps
|
||||
return expected_duration(text, lang)
|
||||
|
||||
|
||||
# ── Classifier ──────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def absorb_caps(params: FitParams) -> tuple[float, float]:
|
||||
"""(fits_cap, absorb_cap) need-ratios aligned with fit_planner.
|
||||
|
||||
``fits_cap``: up to here the audio-only speed-up is imperceptible.
|
||||
``absorb_cap``: up to here fit_planner's knobs absorb the overrun
|
||||
(audio cap × video cap in hybrid mode; the legacy hard audio ceiling
|
||||
when video retiming is off). Beyond it, fit_planner trims.
|
||||
"""
|
||||
if params.allow_video_retime:
|
||||
return params.max_audio_only_rate, params.audio_rate_cap * params.video_slow_cap
|
||||
return params.max_audio_only_rate, MAX_AUDIO_RATE_HARD
|
||||
|
||||
|
||||
def classify_segments(
|
||||
segments: list[dict],
|
||||
target_lang: str,
|
||||
*,
|
||||
calibration: Optional[Calibration] = None,
|
||||
fit_params: Optional[FitParams] = None,
|
||||
total_dur_s: float = 0.0,
|
||||
gap_borrow_max_s: float = GAP_BORROW_MAX_S,
|
||||
) -> list[dict]:
|
||||
"""Classify each segment's translated text against its timeline slot.
|
||||
|
||||
``segments``: chronological dicts with ``id``, ``start``, ``end``
|
||||
(seconds) and ``text`` (the translated text about to be synthesized).
|
||||
``total_dur_s``: original video duration (0/unknown → the last segment
|
||||
gets no tail borrow), mirroring ``fit_planner.plan_fit``.
|
||||
|
||||
Returns one dict per segment::
|
||||
|
||||
{id, status, est_dur_s, available_s, est_overrun_s, calibrated}
|
||||
|
||||
``status`` ∈ {"fits", "tight", "impossible"}; ``est_overrun_s`` is the
|
||||
predicted seconds of speech past the available time (0 when it fits).
|
||||
Pure function: no I/O, deterministic.
|
||||
"""
|
||||
params = fit_params or FitParams()
|
||||
fits_cap, cap = absorb_caps(params)
|
||||
n = len(segments)
|
||||
out: list[dict] = []
|
||||
for i, seg in enumerate(segments):
|
||||
start = float(seg["start"])
|
||||
end = float(seg["end"])
|
||||
slot = max(0.0, end - start)
|
||||
|
||||
# Borrowable silence — fit_planner's slack absorption, capped.
|
||||
if i + 1 < n:
|
||||
gap = max(0.0, float(segments[i + 1]["start"]) - end)
|
||||
borrow = min(max(0.0, gap - params.gap_guard_s), gap_borrow_max_s)
|
||||
elif total_dur_s > 0:
|
||||
borrow = min(max(0.0, float(total_dur_s) - end), gap_borrow_max_s)
|
||||
else:
|
||||
borrow = 0.0
|
||||
available = slot + borrow
|
||||
|
||||
est = estimate_natural_duration(seg.get("text") or "", target_lang, calibration)
|
||||
if est <= 0.0:
|
||||
status = "fits"
|
||||
overrun = 0.0
|
||||
elif available <= 0.0:
|
||||
status = "impossible"
|
||||
overrun = est
|
||||
else:
|
||||
need = est / available
|
||||
# Same boundary tolerance as fit_planner's _EPS: a need that
|
||||
# lands exactly on a cap is absorbed, not escalated.
|
||||
if need <= fits_cap + 1e-9:
|
||||
status = "fits"
|
||||
elif need <= cap + 1e-9:
|
||||
status = "tight"
|
||||
else:
|
||||
status = "impossible"
|
||||
overrun = max(0.0, est - available)
|
||||
|
||||
out.append({
|
||||
"id": str(seg.get("id", f"seg_{i}")),
|
||||
"status": status,
|
||||
"est_dur_s": round(est, 3),
|
||||
"available_s": round(available, 3),
|
||||
"est_overrun_s": round(overrun, 3),
|
||||
"calibrated": calibration is not None,
|
||||
})
|
||||
return out
|
||||
|
||||
|
||||
# ── Optional LLM condensation ───────────────────────────────────────────
|
||||
|
||||
_CONDENSE_PROMPT = """\
|
||||
You are a dubbing writer. The user will give you a translated line that is
|
||||
TOO LONG for its time slot. Rewrite it shorter so it can be read aloud
|
||||
within the target duration: cut filler words, tighten phrasing, and drop
|
||||
the least essential clauses — but preserve the meaning. Never change
|
||||
character names, proper nouns, numbers, or technical terms. Stay in the
|
||||
same language as the line.
|
||||
Reply with ONLY the rewritten line. No quotes, no commentary."""
|
||||
|
||||
# Bound the LLM loop — condensation is a per-segment *suggestion*, not a
|
||||
# fit guarantee, so two shots are plenty before degrading to a no-op.
|
||||
_CONDENSE_ATTEMPTS = 2
|
||||
|
||||
|
||||
def condense_for_slot(
|
||||
text: str,
|
||||
*,
|
||||
available_s: float,
|
||||
target_lang: str,
|
||||
source_text: Optional[str] = None,
|
||||
calibration: Optional[Calibration] = None,
|
||||
) -> dict:
|
||||
"""Meaning-preserving shorter rewrite of ``text`` targeting ``available_s``.
|
||||
|
||||
Returns ``{"text", "applied", "est_dur_s"}`` (+ ``"error"`` on the no-op
|
||||
paths). ``applied=False`` keeps the input text untouched — no LLM
|
||||
configured, LLM failure, and divergent/too-aggressive replies all
|
||||
degrade there. The best (shortest-estimate) candidate that passes the
|
||||
divergence guard AND is actually shorter than the input wins; a reply
|
||||
that fits ``available_s`` returns immediately.
|
||||
"""
|
||||
text = (text or "").strip()
|
||||
base_est = estimate_natural_duration(text, target_lang, calibration)
|
||||
if not text or available_s <= 0:
|
||||
return {"text": text, "applied": False, "est_dur_s": round(base_est, 3),
|
||||
"error": "nothing-to-condense"}
|
||||
if base_est <= available_s:
|
||||
return {"text": text, "applied": False, "est_dur_s": round(base_est, 3),
|
||||
"error": "already-fits"}
|
||||
|
||||
from services import llm_skills
|
||||
# `active=` forwards this module's (monkeypatch-able) name so the
|
||||
# no-override path matches the plain get_active_llm_backend behavior.
|
||||
llm = llm_skills.skill_backend(_SKILL_ID, active=lambda: get_active_llm_backend())
|
||||
if isinstance(llm, OffBackend):
|
||||
return {"text": text, "applied": False, "est_dur_s": round(base_est, 3),
|
||||
"error": "no-llm"}
|
||||
|
||||
best: Optional[tuple[str, float]] = None # (candidate, est)
|
||||
for attempt in range(1, _CONDENSE_ATTEMPTS + 1):
|
||||
user_lines = [
|
||||
f"Target language: {target_lang}",
|
||||
f"Target duration: {available_s:.2f}s",
|
||||
f"Current line: {text}",
|
||||
f"Current reading duration: ~{base_est:.2f}s",
|
||||
]
|
||||
if source_text:
|
||||
user_lines.append(f"Source line (for meaning): {source_text}")
|
||||
if attempt > 1 and best is not None:
|
||||
user_lines.append(
|
||||
f"Your previous rewrite was still ~{best[1]:.2f}s. Cut further."
|
||||
)
|
||||
try:
|
||||
reply = llm.chat(
|
||||
system=_CONDENSE_PROMPT, user="\n".join(user_lines),
|
||||
temperature=0.2, # pinned like Autofit — default 1.0 drifts/invents
|
||||
)
|
||||
except Exception as e: # noqa: BLE001 — LLM failure must no-op, never raise
|
||||
logger.warning("condense attempt %d failed: %s", attempt, e)
|
||||
break
|
||||
candidate = (reply or "").strip()
|
||||
if not candidate:
|
||||
continue
|
||||
ok, reason = refine_output_ok(text, candidate, target_lang)
|
||||
if not ok:
|
||||
logger.warning("condense attempt %d rejected (%s)", attempt, reason)
|
||||
continue
|
||||
est = estimate_natural_duration(candidate, target_lang, calibration)
|
||||
if est >= base_est:
|
||||
continue # not actually shorter — useless as a suggestion
|
||||
if best is None or est < best[1]:
|
||||
best = (candidate, est)
|
||||
if est <= available_s:
|
||||
break # fits — done
|
||||
|
||||
if best is None:
|
||||
return {"text": text, "applied": False, "est_dur_s": round(base_est, 3),
|
||||
"error": "condense-failed"}
|
||||
return {"text": best[0], "applied": True, "est_dur_s": round(best[1], 3)}
|
||||
@@ -0,0 +1,470 @@
|
||||
"""Automatic Hugging Face endpoint selection — probe, pick, remember.
|
||||
|
||||
Restricted-network first-runs (e.g. China, where huggingface.co is blocked)
|
||||
used to dead-end until the user found the mirror setting. This service makes
|
||||
that class of failure self-healing: it *races* the official endpoint against
|
||||
the community mirror with real connectivity probes and remembers the winner,
|
||||
so model downloads work out of the box wherever at least one endpoint is
|
||||
reachable.
|
||||
|
||||
Principles (owner-set):
|
||||
|
||||
- **Probes are the truth.** The decision comes only from actual reachability
|
||||
and latency measurements against endpoints the app would legitimately
|
||||
download from. Device locale/timezone is used *only* to order which
|
||||
endpoint gets probed first — never to decide. No geo-IP lookups, no
|
||||
third-party calls, no telemetry.
|
||||
- **Explicit choices are never auto-switched.** A user with an endpoint
|
||||
configured anywhere (Settings → Models, ``HF_ENDPOINT`` env, the
|
||||
``hf_endpoint`` pref) is in manual mode; auto applies only where nothing
|
||||
was chosen. ``OMNIVOICE_HF_ENDPOINT_MODE=manual`` is a hard env opt-out.
|
||||
- **Sticky, canonical-first decisions.** With both endpoints reachable the
|
||||
official endpoint wins unless the mirror is *decisively* faster
|
||||
(``MIRROR_SPEEDUP_FACTOR``× on latency, confirmed by an optional small
|
||||
ranged-GET throughput sample) — so noise can't flap users onto a mirror.
|
||||
The decision is cached in prefs and re-raced only on: no cached decision
|
||||
(first run), a network-classified download failure, an explicit
|
||||
"Test again", or a decision older than ``DECISION_MAX_AGE_S``.
|
||||
- **Integrity is a non-issue.** huggingface_hub verifies every download by
|
||||
etag/sha regardless of endpoint, so a mirror cannot silently corrupt
|
||||
models.
|
||||
|
||||
Application is **per-call**: download paths (Model Store installs, the model
|
||||
cache auto-repair) pass the effective endpoint as an ``endpoint=`` kwarg. The
|
||||
auto decision is never written to ``HF_ENDPOINT``/user_env — doing so would
|
||||
make it indistinguishable from an explicit user choice.
|
||||
|
||||
Pure and mocked-transport-testable: ``race()`` takes injectable probers, and
|
||||
tests patch the module-level ``probe_endpoint`` / ``throughput_probe``
|
||||
(resolved at call time). Stdlib only.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import threading
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from dataclasses import asdict, dataclass
|
||||
from typing import Callable, Optional
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
logger = logging.getLogger("omnivoice.endpoint_race")
|
||||
|
||||
CANONICAL_ENDPOINT = "https://huggingface.co"
|
||||
COMMUNITY_MIRROR = "https://hf-mirror.com"
|
||||
|
||||
# Hard env opt-out: any of these values disables auto selection entirely.
|
||||
MODE_ENV = "OMNIVOICE_HF_ENDPOINT_MODE"
|
||||
_OPT_OUT_VALUES = {"manual", "off", "0", "false", "no"}
|
||||
|
||||
# prefs keys (core.prefs conventions: env > prefs.json > default).
|
||||
_MODE_PREF = "hf_endpoint_mode" # "auto" | "manual"; absent → default
|
||||
_DECISION_PREF = "hf_endpoint_auto" # cached decision dict (see race())
|
||||
|
||||
DECISION_MAX_AGE_S = 7 * 24 * 3600.0 # re-race a decision older than 7 days
|
||||
PROBE_TIMEOUT_S = 3.0 # short: a probe is not a download
|
||||
MIRROR_SPEEDUP_FACTOR = 3.0 # mirror must be ≥3× faster to win
|
||||
|
||||
# Small, stable, long-lived public file for the optional ranged-GET
|
||||
# throughput tiebreak (mirrors proxy the same /resolve/ paths).
|
||||
_THROUGHPUT_SAMPLE_PATH = "/openai-community/gpt2/resolve/main/model.safetensors"
|
||||
_THROUGHPUT_SAMPLE_BYTES = 256 * 1024
|
||||
|
||||
# Serialises race+persist so concurrent callers can't double-race.
|
||||
_race_lock = threading.Lock()
|
||||
|
||||
# Repos this process already re-raced for after a download failure — the
|
||||
# failover may only happen ONCE per repo per process (same guard pattern as
|
||||
# model_manager._LINK_REPAIR_ATTEMPTED) so a network that stays broken can't
|
||||
# loop probe↔retry.
|
||||
_FAILOVER_ATTEMPTED: set[str] = set()
|
||||
|
||||
|
||||
@dataclass
|
||||
class ProbeResult:
|
||||
endpoint: str
|
||||
reachable: bool
|
||||
latency_ms: Optional[float] = None
|
||||
error: str = "" # "", "timeout", "dns", "tls", "refused", "unreachable"
|
||||
|
||||
|
||||
# ── Probes (the only network code in this module) ───────────────────────────
|
||||
|
||||
def _classify_probe_error(exc: Exception) -> str:
|
||||
"""Coarse failure class for a probe, for logs/UI — never raises."""
|
||||
import socket
|
||||
import ssl
|
||||
|
||||
if isinstance(exc, (socket.timeout, TimeoutError)):
|
||||
return "timeout"
|
||||
if isinstance(exc, ssl.SSLError):
|
||||
return "tls"
|
||||
reason = getattr(exc, "reason", None)
|
||||
if isinstance(reason, socket.gaierror):
|
||||
return "dns"
|
||||
if isinstance(reason, (socket.timeout, TimeoutError)):
|
||||
return "timeout"
|
||||
if isinstance(reason, ssl.SSLError):
|
||||
return "tls"
|
||||
if isinstance(exc, ConnectionRefusedError) or isinstance(reason, ConnectionRefusedError):
|
||||
return "refused"
|
||||
return "unreachable"
|
||||
|
||||
|
||||
def probe_endpoint(endpoint: str, timeout: float = PROBE_TIMEOUT_S) -> ProbeResult:
|
||||
"""HTTPS reachability + latency: one HEAD to the endpoint root.
|
||||
|
||||
Any HTTP response (even an error status) counts as reachable — the probe
|
||||
measures whether the network path works, not whether a specific resource
|
||||
exists. Never raises."""
|
||||
url = endpoint.rstrip("/") + "/"
|
||||
req = urllib.request.Request(url, method="HEAD", headers={"User-Agent": "OmniVoice-endpoint-probe"})
|
||||
start = time.monotonic()
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=timeout):
|
||||
pass
|
||||
except urllib.error.HTTPError:
|
||||
pass # the server answered → reachable
|
||||
except Exception as exc:
|
||||
return ProbeResult(endpoint=endpoint, reachable=False, error=_classify_probe_error(exc))
|
||||
return ProbeResult(
|
||||
endpoint=endpoint,
|
||||
reachable=True,
|
||||
latency_ms=round((time.monotonic() - start) * 1000.0, 1),
|
||||
)
|
||||
|
||||
|
||||
def throughput_probe(endpoint: str, timeout: float = PROBE_TIMEOUT_S) -> Optional[float]:
|
||||
"""Bytes/second over a small ranged GET of a stable public file, or None.
|
||||
|
||||
Used only as a tiebreak confirmation when latency says the mirror is
|
||||
decisively faster — throughput is what a multi-GB download actually
|
||||
feels. Best-effort; any failure returns None (tiebreak skipped)."""
|
||||
url = endpoint.rstrip("/") + _THROUGHPUT_SAMPLE_PATH
|
||||
req = urllib.request.Request(
|
||||
url,
|
||||
headers={
|
||||
"Range": f"bytes=0-{_THROUGHPUT_SAMPLE_BYTES - 1}",
|
||||
"User-Agent": "OmniVoice-endpoint-probe",
|
||||
},
|
||||
)
|
||||
deadline = time.monotonic() + timeout
|
||||
total = 0
|
||||
start = time.monotonic()
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
while total < _THROUGHPUT_SAMPLE_BYTES and time.monotonic() < deadline:
|
||||
chunk = resp.read(min(65536, _THROUGHPUT_SAMPLE_BYTES - total))
|
||||
if not chunk:
|
||||
break
|
||||
total += len(chunk)
|
||||
except Exception:
|
||||
return None
|
||||
elapsed = max(time.monotonic() - start, 1e-6)
|
||||
if total <= 0:
|
||||
return None
|
||||
return total / elapsed
|
||||
|
||||
|
||||
# ── Locale/timezone probe-ORDER hint (stdlib only, never a decision) ────────
|
||||
|
||||
def _hint_sources() -> tuple[list[str], list[str]]:
|
||||
"""(locale strings, timezone strings) from the environment — best-effort."""
|
||||
locs: list[str] = []
|
||||
for key in ("LC_ALL", "LC_MESSAGES", "LANG"):
|
||||
v = os.environ.get(key)
|
||||
if v:
|
||||
locs.append(v)
|
||||
try:
|
||||
import locale as _locale
|
||||
|
||||
locs.extend(x for x in _locale.getlocale() if x)
|
||||
except Exception:
|
||||
pass
|
||||
tzs: list[str] = []
|
||||
tz_env = os.environ.get("TZ")
|
||||
if tz_env:
|
||||
tzs.append(tz_env)
|
||||
try:
|
||||
tzs.extend(x for x in time.tzname if x)
|
||||
except Exception:
|
||||
pass
|
||||
return locs, tzs
|
||||
|
||||
|
||||
_CN_TZ_NAMES = {"asia/shanghai", "asia/chongqing", "asia/urumqi", "asia/harbin"}
|
||||
|
||||
|
||||
def cn_probe_hint(
|
||||
locale_strings: Optional[list[str]] = None,
|
||||
tz_strings: Optional[list[str]] = None,
|
||||
) -> bool:
|
||||
"""True when device language/region/timezone *suggests* mainland China.
|
||||
|
||||
Purely cosmetic: it reorders which endpoint gets probed first (so the
|
||||
likely winner's result lands soonest); every candidate is always probed
|
||||
and the decision comes from the probes alone (VPNs, expats, and corporate
|
||||
networks make region a lie). Never raises."""
|
||||
try:
|
||||
if locale_strings is None or tz_strings is None:
|
||||
env_locs, env_tzs = _hint_sources()
|
||||
locale_strings = env_locs if locale_strings is None else locale_strings
|
||||
tz_strings = env_tzs if tz_strings is None else tz_strings
|
||||
for raw in locale_strings:
|
||||
norm = raw.strip().lower().replace("-", "_")
|
||||
if "zh_cn" in norm or "zh_hans" in norm or "china" in norm:
|
||||
return True
|
||||
for raw in tz_strings:
|
||||
norm = raw.strip().lower()
|
||||
if norm in _CN_TZ_NAMES or "china standard time" in norm:
|
||||
return True
|
||||
except Exception:
|
||||
pass
|
||||
return False
|
||||
|
||||
|
||||
def candidates(cn_hint: Optional[bool] = None) -> list[str]:
|
||||
"""The endpoint registry, probe-ordered by the locale/timezone hint."""
|
||||
if cn_hint is None:
|
||||
cn_hint = cn_probe_hint()
|
||||
if cn_hint:
|
||||
return [COMMUNITY_MIRROR, CANONICAL_ENDPOINT]
|
||||
return [CANONICAL_ENDPOINT, COMMUNITY_MIRROR]
|
||||
|
||||
|
||||
# ── Mode / explicit-setting resolution ──────────────────────────────────────
|
||||
|
||||
def env_opt_out() -> bool:
|
||||
return (os.environ.get(MODE_ENV) or "").strip().lower() in _OPT_OUT_VALUES
|
||||
|
||||
|
||||
def explicit_endpoint() -> str:
|
||||
"""The endpoint the user explicitly configured, or "".
|
||||
|
||||
Same resolution the download paths use: ``HF_ENDPOINT`` env (what
|
||||
Settings → Models persists via user_env and what main.py loads at boot)
|
||||
with the ``hf_endpoint`` pref as fallback. Unlike
|
||||
``core.failure.configured_hf_mirror`` this does NOT filter the official
|
||||
endpoint — explicitly choosing huggingface.co is still an explicit
|
||||
choice. Never raises."""
|
||||
ep = (os.environ.get("HF_ENDPOINT") or "").strip().rstrip("/")
|
||||
if ep:
|
||||
return ep
|
||||
try:
|
||||
from core import prefs
|
||||
|
||||
return str(prefs.get("hf_endpoint", "") or "").strip().rstrip("/")
|
||||
except Exception:
|
||||
return ""
|
||||
|
||||
|
||||
def mode() -> str:
|
||||
"""``"auto"`` or ``"manual"``. Manual whenever the user opted out via
|
||||
env, has an explicit endpoint anywhere, or picked a manual mode in
|
||||
Settings (including explicitly choosing the official endpoint)."""
|
||||
if env_opt_out():
|
||||
return "manual"
|
||||
if explicit_endpoint():
|
||||
return "manual"
|
||||
try:
|
||||
from core import prefs
|
||||
|
||||
if str(prefs.get(_MODE_PREF, "") or "").strip().lower() == "manual":
|
||||
return "manual"
|
||||
except Exception:
|
||||
pass
|
||||
return "auto"
|
||||
|
||||
|
||||
def set_mode_pref(value: str) -> None:
|
||||
"""Persist the Settings-panel mode choice ("auto" | "manual")."""
|
||||
from core import prefs
|
||||
|
||||
prefs.set_(_MODE_PREF, value)
|
||||
|
||||
|
||||
# ── Decision cache (prefs conventions) ──────────────────────────────────────
|
||||
|
||||
def cached_decision() -> Optional[dict]:
|
||||
"""The persisted race decision, or None. Shape-validated; never raises."""
|
||||
try:
|
||||
from core import prefs
|
||||
|
||||
d = prefs.get(_DECISION_PREF)
|
||||
except Exception:
|
||||
return None
|
||||
if (
|
||||
isinstance(d, dict)
|
||||
and isinstance(d.get("endpoint"), str)
|
||||
and d.get("endpoint")
|
||||
and isinstance(d.get("checked_at"), (int, float))
|
||||
):
|
||||
return d
|
||||
return None
|
||||
|
||||
|
||||
def _store_decision(decision: dict) -> None:
|
||||
try:
|
||||
from core import prefs
|
||||
|
||||
prefs.set_(_DECISION_PREF, decision)
|
||||
except Exception: # a broken prefs file must never break downloads
|
||||
logger.warning("could not persist endpoint decision", exc_info=True)
|
||||
|
||||
|
||||
def decision_is_fresh(decision: Optional[dict], now: Optional[float] = None) -> bool:
|
||||
if not decision:
|
||||
return False
|
||||
now = time.time() if now is None else now
|
||||
age = now - float(decision.get("checked_at") or 0)
|
||||
return 0 <= age <= DECISION_MAX_AGE_S
|
||||
|
||||
|
||||
# ── The race ────────────────────────────────────────────────────────────────
|
||||
|
||||
def race(
|
||||
endpoints: Optional[list[str]] = None,
|
||||
prober: Optional[Callable[[str], ProbeResult]] = None,
|
||||
throughput_prober: Optional[Callable[[str], Optional[float]]] = None,
|
||||
now: Optional[float] = None,
|
||||
) -> dict:
|
||||
"""Probe all candidates in parallel and decide. Pure given the probers.
|
||||
|
||||
Policy: reachable beats unreachable; with both reachable the canonical
|
||||
endpoint wins unless the mirror is ≥``MIRROR_SPEEDUP_FACTOR``× faster on
|
||||
latency AND the ranged-GET throughput sample doesn't contradict it (a
|
||||
failed/unavailable throughput probe leaves the latency verdict standing).
|
||||
Neither reachable → canonical, ``reachable=False`` (nothing works anyway;
|
||||
the offline copy owns messaging).
|
||||
|
||||
Returns ``{"endpoint", "reachable", "latency_ms", "checked_at",
|
||||
"results": [...]}``.
|
||||
"""
|
||||
cands = endpoints if endpoints is not None else candidates()
|
||||
# Resolve module attrs at call time so tests can patch probe_endpoint /
|
||||
# throughput_probe and every caller (preflight, Settings) picks it up.
|
||||
do_probe = prober if prober is not None else probe_endpoint
|
||||
do_throughput = throughput_prober if throughput_prober is not None else throughput_probe
|
||||
|
||||
with ThreadPoolExecutor(max_workers=max(1, len(cands))) as pool:
|
||||
results = list(pool.map(do_probe, cands))
|
||||
|
||||
by_endpoint = {r.endpoint: r for r in results}
|
||||
canonical = by_endpoint.get(CANONICAL_ENDPOINT)
|
||||
reachable = [r for r in results if r.reachable and r.latency_ms is not None]
|
||||
reachable.sort(key=lambda r: r.latency_ms)
|
||||
|
||||
if not reachable:
|
||||
winner = ProbeResult(endpoint=CANONICAL_ENDPOINT, reachable=False,
|
||||
error=(canonical.error if canonical else "unreachable"))
|
||||
elif canonical is None or not canonical.reachable:
|
||||
winner = reachable[0]
|
||||
else:
|
||||
winner = canonical
|
||||
fastest_mirror = next((r for r in reachable if r.endpoint != CANONICAL_ENDPOINT), None)
|
||||
if (
|
||||
fastest_mirror is not None
|
||||
and canonical.latency_ms is not None
|
||||
and fastest_mirror.latency_ms * MIRROR_SPEEDUP_FACTOR <= canonical.latency_ms
|
||||
):
|
||||
# Decisive latency win — confirm with throughput (what a real
|
||||
# multi-GB download feels) before moving the user off canonical.
|
||||
tp_mirror = do_throughput(fastest_mirror.endpoint)
|
||||
tp_canonical = do_throughput(CANONICAL_ENDPOINT)
|
||||
if tp_mirror is not None and tp_canonical is not None and tp_mirror < tp_canonical:
|
||||
winner = canonical # latency was noise; canonical still wins
|
||||
else:
|
||||
winner = fastest_mirror
|
||||
|
||||
decision = {
|
||||
"endpoint": winner.endpoint,
|
||||
"reachable": winner.reachable,
|
||||
"latency_ms": winner.latency_ms,
|
||||
"checked_at": time.time() if now is None else now,
|
||||
"results": [asdict(r) for r in results],
|
||||
}
|
||||
logger.info(
|
||||
"HF endpoint race: picked %s (reachable=%s, latency=%sms) from %s",
|
||||
winner.endpoint, winner.reachable, winner.latency_ms,
|
||||
[(r.endpoint, r.reachable, r.latency_ms) for r in results],
|
||||
)
|
||||
return decision
|
||||
|
||||
|
||||
def ensure_decision(
|
||||
force: bool = False,
|
||||
prober: Optional[Callable[[str], ProbeResult]] = None,
|
||||
throughput_prober: Optional[Callable[[str], Optional[float]]] = None,
|
||||
) -> Optional[dict]:
|
||||
"""The current auto decision, racing only when needed. None in manual mode.
|
||||
|
||||
Races when: no cached decision (first run), the cache is stale
|
||||
(>``DECISION_MAX_AGE_S``), or ``force=True`` (preflight, "Test again",
|
||||
download-failure failover). Otherwise the cached decision is returned
|
||||
untouched — launches stay probe-free."""
|
||||
if mode() != "auto":
|
||||
return None
|
||||
with _race_lock:
|
||||
d = cached_decision()
|
||||
if not force and decision_is_fresh(d):
|
||||
return d
|
||||
d = race(prober=prober, throughput_prober=throughput_prober)
|
||||
_store_decision(d)
|
||||
return d
|
||||
|
||||
|
||||
def effective_endpoint() -> Optional[str]:
|
||||
"""The endpoint downloads should pass as ``endpoint=``, or None (canonical).
|
||||
|
||||
Explicit user configuration always wins; in auto mode this returns the
|
||||
cached decision's mirror when one was picked. NEVER probes — safe on the
|
||||
per-download hot path. Never raises."""
|
||||
try:
|
||||
ep = explicit_endpoint()
|
||||
if ep:
|
||||
return ep
|
||||
if mode() != "auto":
|
||||
return None
|
||||
d = cached_decision()
|
||||
if d and d.get("reachable") and d["endpoint"] != CANONICAL_ENDPOINT:
|
||||
return d["endpoint"]
|
||||
except Exception:
|
||||
logger.warning("effective_endpoint failed; using canonical", exc_info=True)
|
||||
return None
|
||||
|
||||
|
||||
def reselect_after_failure(repo_id: str, reason: Optional[str] = None) -> bool:
|
||||
"""After a network-classified download failure: re-race once and report
|
||||
whether the effective endpoint changed (the caller then retries on it).
|
||||
|
||||
Guarded once per repo per process (mirrors the cache-recovery ladder's
|
||||
retry-once guard) so a network that stays broken can't loop probe↔retry.
|
||||
No-op in manual mode and for non-network failures. Never raises."""
|
||||
try:
|
||||
if mode() != "auto":
|
||||
return False
|
||||
if reason is not None:
|
||||
from core.failure import is_hf_connectivity_error
|
||||
|
||||
if not is_hf_connectivity_error(reason):
|
||||
return False
|
||||
if repo_id in _FAILOVER_ATTEMPTED:
|
||||
return False
|
||||
_FAILOVER_ATTEMPTED.add(repo_id)
|
||||
before = effective_endpoint()
|
||||
ensure_decision(force=True)
|
||||
after = effective_endpoint()
|
||||
if after != before:
|
||||
logger.warning(
|
||||
"HF endpoint failover for %s: %s → %s (download failed with a "
|
||||
"network error; retrying on the new endpoint)",
|
||||
repo_id, before or CANONICAL_ENDPOINT, after or CANONICAL_ENDPOINT,
|
||||
)
|
||||
return True
|
||||
return False
|
||||
except Exception:
|
||||
logger.warning("endpoint failover for %s errored", repo_id, exc_info=True)
|
||||
return False
|
||||
@@ -57,9 +57,13 @@ def find_ffmpeg():
|
||||
"""Locate an ffmpeg binary.
|
||||
|
||||
Resolution order:
|
||||
1. ``FFMPEG_PATH`` env var (set by Tauri when a sidecar is bundled).
|
||||
1. ``FFMPEG_PATH`` env var (set by Tauri when a sidecar is bundled, or
|
||||
by the user's Settings → Audio tools override via prefs).
|
||||
2. ``imageio-ffmpeg`` pip package (ships a static binary per platform).
|
||||
3. Common system paths / ``PATH``.
|
||||
3. OmniVoice-acquired static bundle (``services.media_tools``) — the
|
||||
checksummed build the app downloads itself when nothing else
|
||||
resolves; the only bundled tier that also ships ffprobe.
|
||||
4. Common system paths / ``PATH``.
|
||||
|
||||
Returns the path string, or ``None`` if nothing found.
|
||||
"""
|
||||
@@ -78,7 +82,13 @@ def find_ffmpeg():
|
||||
logger.debug("imageio_ffmpeg binary not usable at %s", candidate)
|
||||
except Exception as e:
|
||||
logger.debug("imageio_ffmpeg unavailable: %s", e)
|
||||
# 3. Well-known system paths + PATH lookup
|
||||
# 3. OmniVoice-acquired bundled static binary (never downloads here —
|
||||
# acquisition is media_tools' background job; this only picks up an
|
||||
# already-installed build).
|
||||
candidate = _acquired_bundled("ffmpeg")
|
||||
if candidate:
|
||||
return candidate
|
||||
# 4. Well-known system paths + PATH lookup
|
||||
common = [
|
||||
"/opt/homebrew/bin/ffmpeg",
|
||||
"/usr/local/bin/ffmpeg",
|
||||
@@ -95,6 +105,22 @@ def find_ffmpeg():
|
||||
return None
|
||||
|
||||
|
||||
def _acquired_bundled(tool: str) -> "str | None":
|
||||
"""Already-acquired media_tools static binary, validated — or None.
|
||||
|
||||
Lazy import: media_tools imports from this module at its top, so this
|
||||
module must only reach back at call time (no cycle).
|
||||
"""
|
||||
try:
|
||||
from services.media_tools import bundled_tool_path
|
||||
candidate = bundled_tool_path(tool)
|
||||
if candidate and _binary_runs(candidate):
|
||||
return candidate
|
||||
except Exception as e:
|
||||
logger.debug("media_tools bundled %s unavailable: %s", tool, e)
|
||||
return None
|
||||
|
||||
|
||||
def resolve_ffprobe() -> str | None:
|
||||
"""Resolve an ffprobe binary path.
|
||||
|
||||
@@ -103,8 +129,12 @@ def resolve_ffprobe() -> str | None:
|
||||
injected by Tauri pointing at the bundled sidecar (e.g.
|
||||
``/usr/lib/omnivoice-studio/bin/ffprobe`` on .deb installs).
|
||||
2. ``FFPROBE_PATH`` env var — legacy alias kept for backward
|
||||
compatibility with older Tauri shells / dev environments.
|
||||
3. ``shutil.which("ffprobe")`` — system ``PATH`` fallback.
|
||||
compatibility with older Tauri shells / dev environments; also the
|
||||
key Settings → Audio tools persists a user override under.
|
||||
3. OmniVoice-acquired static bundle (``services.media_tools``) —
|
||||
imageio-ffmpeg ships no ffprobe, so this is the bundled tier that
|
||||
closes the source-install gap.
|
||||
4. ``shutil.which("ffprobe")`` — system ``PATH`` fallback.
|
||||
|
||||
Returns the resolved path string, or ``None`` if nothing found. Callers
|
||||
that need a hard failure should use :func:`find_ffprobe` instead.
|
||||
@@ -121,6 +151,10 @@ def resolve_ffprobe() -> str | None:
|
||||
if resolved and _binary_runs(resolved):
|
||||
return resolved
|
||||
|
||||
bundled = _acquired_bundled("ffprobe")
|
||||
if bundled:
|
||||
return bundled
|
||||
|
||||
system_probe = shutil.which("ffprobe")
|
||||
if system_probe and _binary_runs(system_probe):
|
||||
return system_probe
|
||||
|
||||
@@ -0,0 +1,292 @@
|
||||
"""Self-heal for HF cache snapshots whose entries no longer resolve.
|
||||
|
||||
The Hugging Face hub cache stores each file's bytes once under
|
||||
``models--<org>--<name>/blobs/<hash>`` and exposes every revision as
|
||||
``snapshots/<rev>/<filename>`` entries that link into ``blobs/``. Several
|
||||
real-world events leave a snapshot entry *broken* — a dangling symlink (its
|
||||
blob target doesn't exist) or a zero-byte stand-in file — while the actual
|
||||
bytes are safely on disk under ``blobs/``: a blob-naming mismatch between
|
||||
download modes, an interrupted rename mid-download, antivirus interference.
|
||||
|
||||
``os.path.isfile()`` on a dangling symlink is False, so transformers concludes
|
||||
the weights are missing ("… does not appear to have a file named
|
||||
pytorch_model.bin or model.safetensors") even though the multi-GB download
|
||||
completed. A plain ``snapshot_download`` doesn't reliably fix this — depending
|
||||
on hub version and platform symlink support, the existing-but-broken entry can
|
||||
short-circuit the restore. Deleting exactly the broken entries first makes
|
||||
``snapshot_download`` deterministically restore them (reusing completed blobs
|
||||
where the naming matches, re-downloading only where it doesn't).
|
||||
|
||||
Conservative by design, repairing STATE rather than chasing one cause:
|
||||
* never touches ``blobs/`` (the downloaded bytes),
|
||||
* never touches snapshot entries that resolve,
|
||||
* never force-redownloads healthy files,
|
||||
* never raises — any internal failure logs and returns a summary,
|
||||
* a healthy cache is a cheap lstat/stat walk of ``snapshots/`` (no hashing,
|
||||
no network) on every platform; the heal is generic, not Windows-gated.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
|
||||
logger = logging.getLogger("omnivoice.hf_cache_repair")
|
||||
|
||||
# Snapshot entries that are never legitimately zero bytes: weight formats and
|
||||
# JSON/sentencepiece config-tokenizer files (an empty file is not valid JSON /
|
||||
# not a valid serialized model). Zero-byte files with any OTHER suffix — an
|
||||
# empty .txt, .md, .gitattributes, a marker file a repo genuinely ships empty —
|
||||
# are left alone: when unsure, don't flag.
|
||||
_NEVER_EMPTY_SUFFIXES = frozenset({
|
||||
# weights / tensors
|
||||
".safetensors", ".bin", ".pt", ".pth", ".ckpt", ".onnx", ".gguf",
|
||||
".msgpack", ".h5", ".pb", ".tflite",
|
||||
# config / tokenizer
|
||||
".json", ".model", ".spm",
|
||||
})
|
||||
|
||||
|
||||
def _env_flag(name: str) -> bool:
|
||||
return (os.environ.get(name) or "").strip().lower() in {"1", "true", "yes", "on"}
|
||||
|
||||
|
||||
def hf_cache_home() -> str:
|
||||
"""The hub cache root in effect. Mirrors huggingface_hub's resolution
|
||||
(``HF_HUB_CACHE`` > ``HF_HOME``/hub > default) but reads the env at call
|
||||
time — hub's constants freeze at import, which is too early for tests and
|
||||
for the Windows short-cache redirect in ``core.config``."""
|
||||
env = (os.environ.get("HF_HUB_CACHE") or "").strip()
|
||||
if env:
|
||||
return env
|
||||
hf_home = (os.environ.get("HF_HOME") or "").strip()
|
||||
if hf_home:
|
||||
return os.path.join(hf_home, "hub")
|
||||
try:
|
||||
from huggingface_hub.constants import HF_HUB_CACHE
|
||||
return HF_HUB_CACHE
|
||||
except Exception:
|
||||
return os.path.join(os.path.expanduser("~"), ".cache", "huggingface", "hub")
|
||||
|
||||
|
||||
def repo_cache_dir(repo_id: str, cache_dir: str | None = None) -> str:
|
||||
"""The ``models--<org>--<name>`` folder for ``repo_id`` (repo_type=model)."""
|
||||
return os.path.join(cache_dir or hf_cache_home(),
|
||||
"models--" + repo_id.replace("/", "--"))
|
||||
|
||||
|
||||
def _is_dangling_symlink(path: str) -> bool:
|
||||
# islink() uses lstat (True even when the target is gone); exists()
|
||||
# resolves the link — False for a dangling one. Never raises for a path
|
||||
# that came out of os.walk.
|
||||
return os.path.islink(path) and not os.path.exists(path)
|
||||
|
||||
|
||||
def _is_suspicious_zero_byte(path: str) -> bool:
|
||||
"""A zero-byte REGULAR file standing where model content must be.
|
||||
|
||||
Conservative: only weight/config-typed names are flagged (those are never
|
||||
legitimately empty — the bytes to restore them live in ``blobs/`` or on
|
||||
the Hub); anything else is presumed intentional and left alone."""
|
||||
if os.path.islink(path):
|
||||
return False # resolving symlinks are handled by the dangling check
|
||||
try:
|
||||
if not os.path.isfile(path) or os.path.getsize(path) != 0:
|
||||
return False
|
||||
except OSError:
|
||||
return False
|
||||
return os.path.splitext(path)[1].lower() in _NEVER_EMPTY_SUFFIXES
|
||||
|
||||
|
||||
def find_dangling_entries(repo_cache_dir: str) -> list[str]:
|
||||
"""Broken entries under ``<repo_cache_dir>/snapshots/*/``: dangling
|
||||
symlinks plus suspicious zero-byte regular files (see above).
|
||||
|
||||
Returns absolute paths. On a healthy cache this is a no-op scan — a pure
|
||||
lstat/stat walk of ``snapshots/`` (``blobs/`` is never visited), no
|
||||
hashing, no network. Never raises."""
|
||||
broken: list[str] = []
|
||||
snapshots = os.path.join(repo_cache_dir, "snapshots")
|
||||
if not os.path.isdir(snapshots):
|
||||
return broken
|
||||
try:
|
||||
# followlinks=False: a dangling symlink is not a dir, so os.walk lists
|
||||
# it among the files of its parent — exactly where we scan.
|
||||
for root, _dirs, files in os.walk(snapshots, followlinks=False):
|
||||
for name in files:
|
||||
path = os.path.join(root, name)
|
||||
if _is_dangling_symlink(path) or _is_suspicious_zero_byte(path):
|
||||
broken.append(path)
|
||||
except OSError as walk_err: # pragma: no cover - defensive
|
||||
logger.warning("HF cache scan of %s aborted: %s", snapshots, walk_err)
|
||||
return broken
|
||||
|
||||
|
||||
def _force_copy_mode(cache_root: str) -> bool:
|
||||
"""Best-effort: make huggingface_hub materialize snapshot entries as real
|
||||
file COPIES instead of symlinks for the rest of this process.
|
||||
|
||||
Why: hub's ``are_symlinks_supported()`` probe can succeed in-process while
|
||||
real snapshot symlink creation fails or produces broken links (Windows
|
||||
without Developer Mode is the reported case) — and the result is memoized
|
||||
in the private ``file_download._are_symlinks_supported_in_dir`` dict, so a
|
||||
plain ``snapshot_download`` retry would recreate the SAME dangling links.
|
||||
Pre-seeding that memo with False flips hub into copy mode. It's private
|
||||
API, so any failure (attribute/shape changed across hub versions) is
|
||||
logged and reported as False — the caller then skips the copy-mode pass
|
||||
rather than crash. Deliberately NOT undone: on a host where links come
|
||||
out broken, every later download should use copies too."""
|
||||
try:
|
||||
from pathlib import Path
|
||||
import huggingface_hub.file_download as _fd
|
||||
|
||||
memo = getattr(_fd, "_are_symlinks_supported_in_dir", None)
|
||||
if not isinstance(memo, dict):
|
||||
raise TypeError(
|
||||
f"_are_symlinks_supported_in_dir is {type(memo).__name__}, expected dict"
|
||||
)
|
||||
# Same key normalization hub's are_symlinks_supported() applies.
|
||||
memo[str(Path(cache_root).expanduser().resolve())] = False
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
"Could not force copy-mode for the HF cache (%s) — "
|
||||
"huggingface_hub's private memo may have changed; skipping the "
|
||||
"copy-mode repair pass.", e,
|
||||
)
|
||||
return False
|
||||
|
||||
|
||||
def repair_repo_cache(repo_id: str, cache_dir: str | None = None) -> dict:
|
||||
"""Repair a repo's cache: delete broken snapshot entries (and ONLY those),
|
||||
then ``snapshot_download`` to restore the missing files — hub reuses
|
||||
completed blobs where the naming matches and re-downloads otherwise.
|
||||
|
||||
Verified after the fact: if the restore recreated dangling links (a host
|
||||
where hub's symlink probe passes but real links come out broken — Windows
|
||||
without Developer Mode), force copy mode and repair once more so the
|
||||
snapshot ends up with real files.
|
||||
|
||||
Returns a summary dict; never raises:
|
||||
``found`` broken entries detected up front,
|
||||
``removed`` entries actually deleted (both passes),
|
||||
``restored`` True when a snapshot_download completed,
|
||||
``outcome`` "healthy" | "healed_with_links" | "healed_with_copies"
|
||||
| "repair_failed",
|
||||
``ok`` True unless outcome == "repair_failed",
|
||||
``error`` "" or why the repair failed.
|
||||
"""
|
||||
summary: dict = {
|
||||
"repo_id": repo_id,
|
||||
"repo_dir": "",
|
||||
"found": 0,
|
||||
"removed": 0,
|
||||
"restored": False,
|
||||
"outcome": "repair_failed",
|
||||
"ok": False,
|
||||
"error": "",
|
||||
}
|
||||
try:
|
||||
cache_root = cache_dir or hf_cache_home()
|
||||
repo_dir = repo_cache_dir(repo_id, cache_root)
|
||||
summary["repo_dir"] = repo_dir
|
||||
broken = find_dangling_entries(repo_dir)
|
||||
summary["found"] = len(broken)
|
||||
if not broken:
|
||||
summary["ok"] = True # nothing broken → nothing to do
|
||||
summary["outcome"] = "healthy"
|
||||
return summary
|
||||
if _env_flag("HF_HUB_OFFLINE") or _env_flag("TRANSFORMERS_OFFLINE"):
|
||||
# Don't delete what we can't restore: offline mode means the
|
||||
# follow-up snapshot_download is off the table.
|
||||
summary["error"] = (
|
||||
"Hugging Face offline mode is enabled "
|
||||
"(HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE) — cannot restore files"
|
||||
)
|
||||
logger.warning(
|
||||
"Model cache for %s has %d broken snapshot entr%s but HF "
|
||||
"offline mode is set — skipping repair.",
|
||||
repo_id, len(broken), "y" if len(broken) == 1 else "ies",
|
||||
)
|
||||
return summary
|
||||
|
||||
def _remove(paths: list[str]) -> int:
|
||||
n = 0
|
||||
for path in paths:
|
||||
try:
|
||||
os.remove(path) # removes the link/file itself, never a blob
|
||||
n += 1
|
||||
logger.info(
|
||||
"HF cache self-heal: removed broken snapshot entry %s", path
|
||||
)
|
||||
except OSError as rm_err:
|
||||
logger.warning(
|
||||
"HF cache self-heal: could not remove broken entry %s: %s",
|
||||
path, rm_err,
|
||||
)
|
||||
return n
|
||||
|
||||
summary["removed"] = _remove(broken)
|
||||
if summary["removed"] == 0:
|
||||
summary["error"] = "broken entries could not be removed"
|
||||
return summary
|
||||
|
||||
from huggingface_hub import snapshot_download
|
||||
|
||||
dl_kwargs: dict = {"repo_id": repo_id}
|
||||
if cache_dir:
|
||||
dl_kwargs["cache_dir"] = cache_dir
|
||||
endpoint = os.environ.get("HF_ENDPOINT")
|
||||
if endpoint:
|
||||
dl_kwargs["endpoint"] = endpoint
|
||||
snapshot_download(**dl_kwargs)
|
||||
summary["restored"] = True
|
||||
|
||||
# Verify-after-repair: hub's memoized symlink probe can claim support
|
||||
# while the links it just recreated dangle again. If so, force copy
|
||||
# mode and repair once more so real files land in the snapshot.
|
||||
still_broken = find_dangling_entries(repo_dir)
|
||||
if not still_broken:
|
||||
summary["ok"] = True
|
||||
summary["outcome"] = "healed_with_links"
|
||||
logger.info(
|
||||
"HF cache self-heal for %s: removed %d broken snapshot entr%s "
|
||||
"and restored the snapshot from existing blobs / the Hub.",
|
||||
repo_id, summary["removed"],
|
||||
"y" if summary["removed"] == 1 else "ies",
|
||||
)
|
||||
return summary
|
||||
logger.warning(
|
||||
"HF cache self-heal for %s: the restore recreated %d broken "
|
||||
"link(s) — forcing copy-mode and repairing once more.",
|
||||
repo_id, len(still_broken),
|
||||
)
|
||||
if not _force_copy_mode(cache_root):
|
||||
summary["error"] = (
|
||||
"the snapshot restore recreated broken links and copy-mode "
|
||||
"could not be forced"
|
||||
)
|
||||
return summary
|
||||
summary["removed"] += _remove(still_broken)
|
||||
snapshot_download(**dl_kwargs)
|
||||
remaining = find_dangling_entries(repo_dir)
|
||||
if remaining:
|
||||
summary["error"] = (
|
||||
f"{len(remaining)} snapshot entr"
|
||||
f"{'y is' if len(remaining) == 1 else 'ies are'} still broken "
|
||||
"after the copy-mode repair"
|
||||
)
|
||||
return summary
|
||||
summary["ok"] = True
|
||||
summary["outcome"] = "healed_with_copies"
|
||||
logger.info(
|
||||
"HF cache self-heal for %s: healed with real file copies "
|
||||
"(symlinks on this host come out broken; hub stays in copy-mode "
|
||||
"for the rest of this run).", repo_id,
|
||||
)
|
||||
return summary
|
||||
except Exception as e: # never raise — repair is best-effort
|
||||
summary["error"] = f"{type(e).__name__}: {e}"
|
||||
logger.warning(
|
||||
"HF cache self-heal for %s failed: %s", repo_id, summary["error"],
|
||||
)
|
||||
return summary
|
||||
@@ -18,10 +18,16 @@ reimplement it:
|
||||
(+ optional cover art, loudness filter), output as ``m4b`` or ``mp3``.
|
||||
* ``chapter_cache_key`` — deterministic content hash so a re-run reuses
|
||||
already-rendered chapters (resume) and re-renders only what changed.
|
||||
* ``segment_cache_key`` / ``SegmentCache`` — the inner cache layer: each
|
||||
spoken span's WAV is content-addressed under ``<cache_dir>/segments`` so
|
||||
editing one sentence re-renders one segment (not the chapter) and an
|
||||
interrupted chapter render resumes from its finished segments.
|
||||
|
||||
Every function here is pure (string/argv in, string/argv out) so it's unit
|
||||
tested without ffmpeg, torch, or a GPU. The impure ffmpeg run lives in the
|
||||
caller (the audiobook router today; the stories job tomorrow).
|
||||
The builders are pure (string/argv in, string/argv out) so they're unit tested
|
||||
without ffmpeg, torch, or a GPU; the cache helpers (``prune_cache_dir``,
|
||||
``SegmentCache``) touch only local files and import torch lazily. The impure
|
||||
ffmpeg run lives in the caller (the audiobook router today; the stories job
|
||||
tomorrow).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -65,30 +71,29 @@ def _escape_meta(value: str) -> str:
|
||||
|
||||
def prune_cache_dir(cache_dir: str, max_bytes: int = _CACHE_MAX_BYTES) -> tuple[int, int]:
|
||||
"""Evict the oldest files in ``cache_dir`` until the total size is within
|
||||
``max_bytes`` (LRU by mtime). The content-addressed chapter cache otherwise
|
||||
``max_bytes`` (LRU by mtime). The content-addressed render cache otherwise
|
||||
grows without bound — uncompressed WAVs accumulate across every render.
|
||||
|
||||
Best-effort: returns ``(remaining_bytes, removed_count)`` and never raises
|
||||
(a missing dir / unstattable file is just skipped). Call it *before* writing
|
||||
a job's chapters so the fresh ones are never the eviction target.
|
||||
Walks the whole tree, so chapter WAVs at the root and segment WAVs under
|
||||
``segments/`` share ONE byte budget — the cap holds no matter which layer
|
||||
grew. Best-effort: returns ``(remaining_bytes, removed_count)`` and never
|
||||
raises (a missing dir / unstattable file is just skipped). Call it *before*
|
||||
writing a job's files so the fresh ones are never the eviction target.
|
||||
"""
|
||||
try:
|
||||
names = os.listdir(cache_dir)
|
||||
except OSError:
|
||||
return (0, 0)
|
||||
entries: list[tuple[float, int, str]] = []
|
||||
total = 0
|
||||
for name in names:
|
||||
p = os.path.join(cache_dir, name)
|
||||
try:
|
||||
if not os.path.isfile(p):
|
||||
for root, _dirs, names in os.walk(cache_dir):
|
||||
for name in names:
|
||||
p = os.path.join(root, name)
|
||||
try:
|
||||
if not os.path.isfile(p):
|
||||
continue
|
||||
size = os.path.getsize(p)
|
||||
mtime = os.path.getmtime(p)
|
||||
except OSError:
|
||||
continue
|
||||
size = os.path.getsize(p)
|
||||
mtime = os.path.getmtime(p)
|
||||
except OSError:
|
||||
continue
|
||||
entries.append((mtime, size, p))
|
||||
total += size
|
||||
entries.append((mtime, size, p))
|
||||
total += size
|
||||
if total <= max_bytes:
|
||||
return (total, 0)
|
||||
entries.sort() # oldest first
|
||||
@@ -136,6 +141,126 @@ def chapter_cache_key(
|
||||
return hashlib.sha1(raw.encode("utf-8"), usedforsecurity=False).hexdigest()[:20]
|
||||
|
||||
|
||||
# ── Segment cache (sub-chapter granularity) ─────────────────────────────────
|
||||
|
||||
#: Segment WAVs live in a subdirectory of the chapter cache dir so both layers
|
||||
#: share one root — and one byte cap (``prune_cache_dir`` walks the tree).
|
||||
SEGMENT_SUBDIR = "segments"
|
||||
|
||||
|
||||
def segment_cache_key(
|
||||
text: str,
|
||||
*,
|
||||
sample_rate: int,
|
||||
engine_id: str,
|
||||
voice_id: Optional[str] = None,
|
||||
voice_sig: str = "",
|
||||
speed: Optional[float] = None,
|
||||
extra_sig: str = "",
|
||||
) -> str:
|
||||
"""Deterministic content hash for ONE rendered segment (a single spoken
|
||||
span). Same dimensions as :func:`chapter_cache_key` minus span order and
|
||||
pauses (pauses are synthesized silence — never cached): text, voice
|
||||
identity (id + resolved signature), speed, sample rate, engine, plus
|
||||
``extra_sig`` for anything else that changes the rendered audio (the
|
||||
pronunciation lexicon today). Any change → new key → re-synthesize just
|
||||
this segment.
|
||||
"""
|
||||
payload = {
|
||||
"sr": int(sample_rate),
|
||||
"engine": engine_id or "",
|
||||
"voice": voice_id or "",
|
||||
"text": text or "",
|
||||
"speed": speed,
|
||||
"voice_sig": voice_sig or "",
|
||||
"extra": extra_sig or "",
|
||||
}
|
||||
raw = json.dumps(payload, sort_keys=True, ensure_ascii=False)
|
||||
# Content-addressing only — not a security digest (see chapter_cache_key).
|
||||
return hashlib.sha1(raw.encode("utf-8"), usedforsecurity=False).hexdigest()[:20]
|
||||
|
||||
|
||||
class SegmentCache:
|
||||
"""Content-addressed per-segment WAV store under ``cache_dir/segments``.
|
||||
|
||||
The chapter cache stays the fast outer layer — a fully-unchanged chapter
|
||||
hits at the chapter key and never touches segment files. This inner layer
|
||||
makes a *changed* chapter cheap: only the edited/new segments synthesize
|
||||
(the rest load from disk), and an interrupted chapter render resumes from
|
||||
the segments that already finished, because each segment is persisted the
|
||||
moment it renders.
|
||||
|
||||
``voice_sig`` maps ``voice_id or ""`` → resolved-profile signature (same
|
||||
strings the chapter key uses) so a profile edit invalidates segments too.
|
||||
Load/store are best-effort: a missing/corrupt/foreign-rate file is a clean
|
||||
cache miss (re-render), and a failed store never fails the render — so
|
||||
caches written by any app version degrade safely. torch/torchaudio import
|
||||
lazily to keep this module import-light for the pure-builder callers.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
cache_dir: str,
|
||||
*,
|
||||
sample_rate: int,
|
||||
engine_id: str,
|
||||
voice_sig: Optional[dict] = None,
|
||||
extra_sig: str = "",
|
||||
) -> None:
|
||||
self.dir = os.path.join(cache_dir, SEGMENT_SUBDIR)
|
||||
self.sample_rate = int(sample_rate)
|
||||
self.engine_id = engine_id or ""
|
||||
self.voice_sig = dict(voice_sig or {})
|
||||
self.extra_sig = extra_sig or ""
|
||||
self.hits = 0
|
||||
self.misses = 0
|
||||
|
||||
def _path(self, span) -> str:
|
||||
key = segment_cache_key(
|
||||
span.text,
|
||||
sample_rate=self.sample_rate,
|
||||
engine_id=self.engine_id,
|
||||
voice_id=span.voice_id,
|
||||
voice_sig=self.voice_sig.get(span.voice_id or "", ""),
|
||||
speed=getattr(span, "speed", None),
|
||||
extra_sig=self.extra_sig,
|
||||
)
|
||||
return os.path.join(self.dir, f"{key}.wav")
|
||||
|
||||
def load(self, span):
|
||||
"""Cached audio tensor for ``span``, or ``None`` (miss). A hit bumps
|
||||
the file's mtime so LRU eviction sees the segment as recently used."""
|
||||
path = self._path(span)
|
||||
if not os.path.isfile(path):
|
||||
self.misses += 1
|
||||
return None
|
||||
try:
|
||||
import torchaudio
|
||||
audio, sr = torchaudio.load(path)
|
||||
except Exception:
|
||||
self.misses += 1
|
||||
return None # unreadable/corrupt entry — clean miss, re-render
|
||||
if int(sr) != self.sample_rate or audio.numel() == 0:
|
||||
self.misses += 1
|
||||
return None # foreign-rate/empty entry — clean miss, re-render
|
||||
try:
|
||||
os.utime(path, None)
|
||||
except OSError:
|
||||
pass
|
||||
self.hits += 1
|
||||
return audio
|
||||
|
||||
def store(self, span, audio) -> None:
|
||||
"""Persist a freshly rendered segment. Best-effort — a full disk or
|
||||
unwritable cache dir must never fail the chapter render."""
|
||||
try:
|
||||
from services.audio_io import atomic_save_wav
|
||||
os.makedirs(self.dir, exist_ok=True)
|
||||
atomic_save_wav(self._path(span), audio, self.sample_rate)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
# ── Loudness normalization ──────────────────────────────────────────────────
|
||||
|
||||
@dataclass(frozen=True)
|
||||
|
||||
@@ -0,0 +1,637 @@
|
||||
"""Media tools — ffmpeg / ffprobe / yt-dlp as an invisible internal concern.
|
||||
|
||||
Most users should never learn what ffmpeg is. This module makes the media
|
||||
engine self-contained: it reports where each tool comes from, acquires a
|
||||
bundled static build in the background when no tier of the resolution chain
|
||||
(``services.ffmpeg_utils``) resolves, and gives power users explicit
|
||||
control (custom path / system copy / restore bundled) through the
|
||||
``/media-tools`` router — persisted via the same ``env.FFMPEG_PATH`` /
|
||||
``env.FFPROBE_PATH`` prefs convention the Settings env writer already uses,
|
||||
so there is exactly one override mechanism.
|
||||
|
||||
Bundled-binary source (decision record)
|
||||
---------------------------------------
|
||||
The gap: ``imageio-ffmpeg`` (already a locked dep) ships a static *ffmpeg*
|
||||
inside its platform wheels but **no ffprobe**, so source installs without a
|
||||
system ffmpeg lose ``/tools/probe``, Smart-Fit duration checks, and VFR
|
||||
detection. Two options were audited:
|
||||
|
||||
(a) the ``static-ffmpeg`` pip package — ships BOTH binaries per platform via
|
||||
lazy download. **Rejected**: it downloads from a *mutable* URL
|
||||
(``.../ffmpeg_bins/raw/main/...`` — the branch tip, not a pinned
|
||||
release), performs **no checksum validation**, extracts into its own
|
||||
``site-packages`` directory (read-only / non-existent in the frozen
|
||||
PyInstaller backend), and drags in ``requests``/``filelock``/``progress``
|
||||
plus a stdout spinner.
|
||||
|
||||
(b) fetch the same upstream static builds ourselves, pinned to an immutable
|
||||
commit. **Chosen**: we download the platform zip from
|
||||
``github.com/zackees/ffmpeg_bins`` at a pinned commit SHA (immutable
|
||||
URL), verify size + SHA-256 against constants recorded from that
|
||||
commit's git-LFS pointers, extract only ffmpeg/ffprobe into a
|
||||
user-writable, update-surviving dir under ``DATA_DIR``, and trust a
|
||||
binary only after the existing ``_binary_runs`` ``-version`` probe.
|
||||
Stdlib-only (urllib honors HTTP(S)_PROXY), identical behavior on
|
||||
macOS (arm64 + x86_64), Windows x64, and Linux (x64 + arm64), and zero
|
||||
new Python dependencies.
|
||||
|
||||
yt-dlp updates (decision record)
|
||||
--------------------------------
|
||||
yt-dlp is an importable locked dep — never a user-installed requirement.
|
||||
But site support rots faster than app releases, so Settings offers a
|
||||
user-triggered "Update". A plain in-venv upgrade was audited and rejected:
|
||||
the app venv is uv-managed (no pip module), and the updater's drift sync
|
||||
(#1029/#1030, ``uv sync --frozen --inexact``) preserves only packages *not*
|
||||
in the lockfile — yt-dlp IS locked, so an in-venv upgrade would be silently
|
||||
reverted on the next app update, and the frozen build has no installer at
|
||||
all. Instead we install the new wheel (pure-python, no required deps —
|
||||
matching the plain ``yt-dlp`` spec pinned in pyproject) into an **overlay
|
||||
directory** under ``DATA_DIR`` — SHA-256-verified against PyPI's own
|
||||
metadata — and prepend it to ``sys.path`` at startup. It survives app
|
||||
updates and drift syncs, works identically in source and frozen builds, and
|
||||
"Restore tested version" is simply deleting the overlay: the locked wheel
|
||||
underneath was never touched.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import zipfile
|
||||
|
||||
from core.config import DATA_DIR
|
||||
from core import prefs
|
||||
from services.ffmpeg_utils import _binary_runs, _BINARY_OK
|
||||
|
||||
logger = logging.getLogger("omnivoice.media_tools")
|
||||
|
||||
# ── Pinned bundled build ────────────────────────────────────────────────────
|
||||
# Immutable commit of github.com/zackees/ffmpeg_bins (the upstream the
|
||||
# static-ffmpeg pip package also consumes, but pinned + checksummed here).
|
||||
# SHA-256 values are the git-LFS oids of the v8.0 platform zips at this
|
||||
# commit, independently verified by downloading and hashing.
|
||||
_FFBIN_REPO = "zackees/ffmpeg_bins"
|
||||
_FFBIN_COMMIT = "df95abcb0ce6efff710dda5ef28a2f6f1dc21493" # 2026-01-16
|
||||
_FFBIN_TREE = "v8.0"
|
||||
|
||||
#: platform key → (sha256, size in bytes) of the zip at the pinned commit.
|
||||
_FFBIN_SHA256 = {
|
||||
"darwin": ("70fd5b21cb37b6ea97c8b584cf76b3cc6a90179831c9c269811b9716c28605fb", 53079896),
|
||||
"darwin_arm64": ("b2da44a8169c4d09a97db996250690c3346f72e4795521d23d3dbb1e72421207", 41925556),
|
||||
"linux": ("ca75b05e887c7a97676632f673031875847be83daa9794298fed9cef8cac14ad", 142008975),
|
||||
"linux_arm64": ("e03efe471c03b999f10988d5db62ae3bd94837463291b3c7755528b100e97d6f", 131816005),
|
||||
"win32": ("92662c2241e93fe71b3f3a01e94a0b0dc8cfad726019f96b83bc109ce44c5d0b", 72065209),
|
||||
}
|
||||
|
||||
_PYPI_YTDLP_URL = "https://pypi.org/pypi/yt-dlp/json"
|
||||
|
||||
_DOWNLOAD_TIMEOUT_S = 30 # per-read socket timeout; downloads stream in chunks
|
||||
_CHUNK = 256 * 1024
|
||||
|
||||
#: tool → env keys honored by the resolution chain, in precedence order.
|
||||
_ENV_KEYS = {
|
||||
"ffmpeg": ("FFMPEG_PATH",),
|
||||
"ffprobe": ("OMNIVOICE_FFPROBE_PATH", "FFPROBE_PATH"),
|
||||
}
|
||||
#: tool → the env key the *user override* is persisted under (prefs `env.<KEY>`).
|
||||
_PREF_ENV_KEY = {"ffmpeg": "FFMPEG_PATH", "ffprobe": "FFPROBE_PATH"}
|
||||
|
||||
TOOLS = ("ffmpeg", "ffprobe")
|
||||
|
||||
# ── Background-operation state (poll via status()) ─────────────────────────
|
||||
|
||||
_lock = threading.Lock()
|
||||
_ops: dict[str, dict] = {
|
||||
"acquire": {"state": "idle", "progress": 0.0, "error": None},
|
||||
"ytdlp_update": {"state": "idle", "progress": 0.0, "error": None, "version": None},
|
||||
}
|
||||
_version_cache: dict[str, str] = {}
|
||||
|
||||
|
||||
def _set_op(op: str, **fields) -> None:
|
||||
with _lock:
|
||||
_ops[op].update(fields)
|
||||
|
||||
|
||||
def _op_snapshot() -> dict:
|
||||
with _lock:
|
||||
return {k: dict(v) for k, v in _ops.items()}
|
||||
|
||||
|
||||
# ── Platform / paths ────────────────────────────────────────────────────────
|
||||
|
||||
def _platform_key() -> str:
|
||||
import platform as _p
|
||||
is_arm = _p.machine().lower() in ("arm64", "aarch64")
|
||||
if sys.platform == "win32":
|
||||
return "win32"
|
||||
if sys.platform == "darwin":
|
||||
return "darwin_arm64" if is_arm else "darwin"
|
||||
if sys.platform.startswith("linux"):
|
||||
return "linux_arm64" if is_arm else "linux"
|
||||
return sys.platform
|
||||
|
||||
|
||||
def media_tools_dir() -> str:
|
||||
"""User-writable root for acquired binaries + the yt-dlp overlay.
|
||||
|
||||
Lives in DATA_DIR so it survives app updates (the app bundle / venv are
|
||||
replaced wholesale on update; DATA_DIR is user state) and is writable in
|
||||
frozen installs.
|
||||
"""
|
||||
return os.path.join(DATA_DIR, "media_tools")
|
||||
|
||||
|
||||
def bundled_dir() -> str:
|
||||
# Versioned by the pin so a future pin bump lands in a fresh dir and
|
||||
# "Update" is a plain re-acquire — no in-place mutation of a live binary.
|
||||
return os.path.join(media_tools_dir(), f"ffbin-{_FFBIN_COMMIT[:12]}", _platform_key())
|
||||
|
||||
|
||||
def _exe(name: str) -> str:
|
||||
return f"{name}.exe" if sys.platform == "win32" else name
|
||||
|
||||
|
||||
def bundled_tool_path(tool: str) -> str | None:
|
||||
"""Path of an already-acquired bundled binary, or None. Never downloads."""
|
||||
p = os.path.join(bundled_dir(), _exe(tool))
|
||||
return p if os.path.isfile(p) else None
|
||||
|
||||
|
||||
def _bundle_url() -> str:
|
||||
# github.com/<repo>/raw/<commit> redirects to the LFS media host and
|
||||
# serves the real zip (raw.githubusercontent.com would return the
|
||||
# 133-byte LFS pointer instead).
|
||||
return f"https://github.com/{_FFBIN_REPO}/raw/{_FFBIN_COMMIT}/{_FFBIN_TREE}/{_platform_key()}.zip"
|
||||
|
||||
|
||||
def _expected_bundle() -> tuple[str, str, int]:
|
||||
"""(url, sha256, size) for this platform. Raises on unsupported platform."""
|
||||
key = _platform_key()
|
||||
if key not in _FFBIN_SHA256:
|
||||
raise RuntimeError(f"no bundled media-engine build for platform '{key}'")
|
||||
sha, size = _FFBIN_SHA256[key]
|
||||
return _bundle_url(), sha, size
|
||||
|
||||
|
||||
# ── Download helper ─────────────────────────────────────────────────────────
|
||||
|
||||
def _download(url: str, dest_path: str, expected_sha256: str,
|
||||
expected_size: int | None, op: str) -> None:
|
||||
"""Stream *url* to *dest_path*, hashing on the fly; raise on mismatch.
|
||||
|
||||
Progress is reported into ``_ops[op]["progress"]``. urllib honors the
|
||||
HTTP(S)_PROXY env vars, so restricted-network users' proxy settings apply.
|
||||
"""
|
||||
import urllib.request
|
||||
|
||||
if not url.startswith("https://"):
|
||||
raise ValueError("media-tools downloads must be https")
|
||||
req = urllib.request.Request(url, headers={"User-Agent": "OmniVoice-Studio"})
|
||||
hasher = hashlib.sha256()
|
||||
done = 0
|
||||
with urllib.request.urlopen(req, timeout=_DOWNLOAD_TIMEOUT_S) as resp:
|
||||
total = expected_size or int(resp.headers.get("Content-Length") or 0)
|
||||
with open(dest_path, "wb") as f:
|
||||
while True:
|
||||
chunk = resp.read(_CHUNK)
|
||||
if not chunk:
|
||||
break
|
||||
f.write(chunk)
|
||||
hasher.update(chunk)
|
||||
done += len(chunk)
|
||||
if total:
|
||||
_set_op(op, progress=min(done / total, 1.0))
|
||||
digest = hasher.hexdigest()
|
||||
if expected_size is not None and done != expected_size:
|
||||
raise RuntimeError(f"download size mismatch: got {done}, expected {expected_size}")
|
||||
if digest != expected_sha256:
|
||||
raise RuntimeError("download checksum mismatch — refusing to install")
|
||||
|
||||
|
||||
# ── Bundled acquisition ─────────────────────────────────────────────────────
|
||||
|
||||
def acquire_bundled(wait: bool = False) -> dict:
|
||||
"""Fetch + verify + install the pinned static ffmpeg/ffprobe build.
|
||||
|
||||
Idempotent: a no-op when the binaries are already present and runnable,
|
||||
or when an acquisition is already running. Runs in a daemon thread so it
|
||||
never blocks the caller (``wait=True`` is for tests/CLI use).
|
||||
Returns the op-state snapshot.
|
||||
"""
|
||||
with _lock:
|
||||
if _ops["acquire"]["state"] == "running":
|
||||
return dict(_ops["acquire"])
|
||||
_ops["acquire"].update(state="running", progress=0.0, error=None)
|
||||
|
||||
if all(bundled_tool_path(t) and _binary_runs(bundled_tool_path(t)) for t in TOOLS):
|
||||
_set_op("acquire", state="done", progress=1.0)
|
||||
return _op_snapshot()["acquire"]
|
||||
|
||||
def _worker():
|
||||
try:
|
||||
_do_acquire()
|
||||
_set_op("acquire", state="done", progress=1.0, error=None)
|
||||
logger.info("media-tools: bundled ffmpeg/ffprobe installed at %s", bundled_dir())
|
||||
except Exception as e:
|
||||
logger.warning("media-tools: bundled acquisition failed: %s", e)
|
||||
_set_op("acquire", state="error", error=str(e)[:300])
|
||||
|
||||
if wait:
|
||||
_worker()
|
||||
else:
|
||||
threading.Thread(target=_worker, name="media-tools-acquire", daemon=True).start()
|
||||
return _op_snapshot()["acquire"]
|
||||
|
||||
|
||||
def _do_acquire() -> None:
|
||||
url, sha, size = _expected_bundle()
|
||||
target = bundled_dir()
|
||||
os.makedirs(os.path.dirname(target), exist_ok=True)
|
||||
|
||||
with tempfile.TemporaryDirectory(dir=os.path.dirname(target)) as tmp:
|
||||
zip_path = os.path.join(tmp, "bundle.zip")
|
||||
_download(url, zip_path, sha, size, op="acquire")
|
||||
|
||||
# Extract only the two binaries, flattened by basename — layout-agnostic
|
||||
# and immune to zip-slip (we never honor archive paths).
|
||||
wanted = {_exe(t): t for t in TOOLS}
|
||||
staged = os.path.join(tmp, "staged")
|
||||
os.makedirs(staged, exist_ok=True)
|
||||
found: dict[str, str] = {}
|
||||
with zipfile.ZipFile(zip_path) as zf:
|
||||
for member in zf.infolist():
|
||||
base = os.path.basename(member.filename)
|
||||
if base in wanted and not member.is_dir():
|
||||
out = os.path.join(staged, base)
|
||||
with zf.open(member) as src, open(out, "wb") as dst:
|
||||
shutil.copyfileobj(src, dst)
|
||||
# Owner-only rwx — the backend process is the sole consumer
|
||||
# of these binaries (least privilege; py/overly-permissive-file).
|
||||
os.chmod(out, 0o700)
|
||||
found[base] = out
|
||||
missing = set(wanted) - set(found)
|
||||
if missing:
|
||||
raise RuntimeError(f"bundle is missing {sorted(missing)}")
|
||||
|
||||
# Probe BEFORE trusting — a corrupt / wrong-arch binary must never
|
||||
# be installed (same contract as ffmpeg_utils._binary_runs at
|
||||
# resolution time, applied at install time).
|
||||
for base, path in found.items():
|
||||
_BINARY_OK.pop(path, None)
|
||||
if not _binary_runs(path):
|
||||
raise RuntimeError(f"downloaded {base} failed its -version probe")
|
||||
|
||||
# Finalize: swap the staged dir into place.
|
||||
if os.path.isdir(target):
|
||||
shutil.rmtree(target, ignore_errors=True)
|
||||
os.replace(staged, target)
|
||||
|
||||
# Resolution caches may hold negative verdicts for the old paths.
|
||||
for t in TOOLS:
|
||||
p = os.path.join(target, _exe(t))
|
||||
_BINARY_OK.pop(p, None)
|
||||
_version_cache.pop(p, None)
|
||||
|
||||
|
||||
# ── Status / origin classification ─────────────────────────────────────────
|
||||
|
||||
def _tool_version(path: str) -> str | None:
|
||||
cached = _version_cache.get(path)
|
||||
if cached:
|
||||
return cached
|
||||
try:
|
||||
out = subprocess.run(
|
||||
[path, "-version"], capture_output=True, text=True, timeout=10, check=False,
|
||||
).stdout
|
||||
m = re.match(r"^(?:ffmpeg|ffprobe) version (\S+)", out or "")
|
||||
if m:
|
||||
_version_cache[path] = m.group(1)
|
||||
return m.group(1)
|
||||
except Exception as e:
|
||||
logger.debug("version probe failed for %s: %s", os.path.basename(path), e)
|
||||
return None
|
||||
|
||||
|
||||
def _imageio_pkg_dir() -> str | None:
|
||||
try:
|
||||
import imageio_ffmpeg
|
||||
return os.path.dirname(os.path.abspath(imageio_ffmpeg.__file__))
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _classify_origin(tool: str, path: str) -> str:
|
||||
"""sidecar | bundled | system | custom — where the resolved binary lives."""
|
||||
rp = os.path.realpath(path)
|
||||
for root in filter(None, (media_tools_dir(), _imageio_pkg_dir())):
|
||||
if rp.startswith(os.path.realpath(root) + os.sep):
|
||||
return "bundled"
|
||||
for key in _ENV_KEYS[tool]:
|
||||
v = os.environ.get(key)
|
||||
if not v:
|
||||
continue
|
||||
if v == path or os.path.realpath(v) == rp or shutil.which(v) == path:
|
||||
# The same env var serves two masters: the Tauri sidecar injects
|
||||
# it at spawn; a user override persists it via prefs `env.<KEY>`.
|
||||
return "custom" if prefs.get(f"env.{key}") else "sidecar"
|
||||
return "system"
|
||||
|
||||
|
||||
def _resolve(tool: str) -> str | None:
|
||||
from services import ffmpeg_utils
|
||||
if tool == "ffmpeg":
|
||||
return ffmpeg_utils.find_ffmpeg()
|
||||
return ffmpeg_utils.find_ffprobe()
|
||||
|
||||
|
||||
def _ytdlp_status() -> dict:
|
||||
"""yt-dlp is a python module, not a binary — status reads its version
|
||||
without paying the full package import."""
|
||||
info: dict = {"tool": "yt-dlp", "ok": False, "path": None, "version": None,
|
||||
"origin": "bundled", "overlay_version": None,
|
||||
"baseline_version": prefs.get("media_tools.ytdlp_baseline")}
|
||||
try:
|
||||
import importlib.util
|
||||
spec = importlib.util.find_spec("yt_dlp")
|
||||
origin = getattr(spec, "origin", None)
|
||||
if origin:
|
||||
pkg_dir = os.path.dirname(origin)
|
||||
info["path"] = pkg_dir
|
||||
info["ok"] = True
|
||||
info["version"] = _read_ytdlp_version(pkg_dir)
|
||||
if os.path.realpath(pkg_dir).startswith(
|
||||
os.path.realpath(_ytdlp_overlay_dir()) + os.sep):
|
||||
info["origin"] = "custom"
|
||||
except Exception as e:
|
||||
logger.debug("yt_dlp spec lookup failed: %s", e)
|
||||
ov = _read_ytdlp_version(os.path.join(_ytdlp_overlay_dir(), "yt_dlp"))
|
||||
info["overlay_version"] = ov
|
||||
return info
|
||||
|
||||
|
||||
def _read_ytdlp_version(pkg_dir: str) -> str | None:
|
||||
try:
|
||||
with open(os.path.join(pkg_dir, "version.py"), encoding="utf-8") as f:
|
||||
m = re.search(r"__version__\s*=\s*['\"]([^'\"]+)['\"]", f.read())
|
||||
return m.group(1) if m else None
|
||||
except OSError:
|
||||
return None
|
||||
|
||||
|
||||
def status() -> dict:
|
||||
"""Full media-tools report: per-tool {ok, path, version, origin} + op states."""
|
||||
tools = {}
|
||||
for tool in TOOLS:
|
||||
path = _resolve(tool)
|
||||
tools[tool] = {
|
||||
"tool": tool,
|
||||
"ok": bool(path),
|
||||
"path": path,
|
||||
"version": _tool_version(path) if path else None,
|
||||
"origin": _classify_origin(tool, path) if path else None,
|
||||
}
|
||||
tools["ytdlp"] = _ytdlp_status()
|
||||
ops = _op_snapshot()
|
||||
return {
|
||||
"ready": tools["ffmpeg"]["ok"] and tools["ffprobe"]["ok"],
|
||||
"tools": tools,
|
||||
"ops": ops,
|
||||
"platform_key": _platform_key(),
|
||||
}
|
||||
|
||||
|
||||
def summary(auto_acquire: bool = False) -> dict:
|
||||
"""Small preflight-embeddable verdict. With ``auto_acquire``, kicks off
|
||||
the bundled download in the background when nothing resolves (first-run
|
||||
self-heal) — but never re-fires after a failed attempt (the wizard's
|
||||
failure card owns the Retry)."""
|
||||
st = status()
|
||||
op = st["ops"]["acquire"]
|
||||
if auto_acquire and not st["ready"] and op["state"] == "idle":
|
||||
op = acquire_bundled()
|
||||
return {
|
||||
"ready": st["ready"],
|
||||
"acquire": {"state": op["state"], "progress": op["progress"], "error": op["error"]},
|
||||
}
|
||||
|
||||
|
||||
# ── User overrides (persisted via the existing env-prefs convention) ───────
|
||||
|
||||
def _validate_binary_path(path: str) -> None:
|
||||
# Same defense-in-depth as /system/set-env: no control chars, must be an
|
||||
# existing file, and must actually run before we trust it.
|
||||
if any(ord(c) < 0x20 or ord(c) == 0x7F for c in path):
|
||||
raise ValueError("Invalid path: control characters are not allowed")
|
||||
if not os.path.isfile(path):
|
||||
raise ValueError(f"File not found: {path}")
|
||||
_BINARY_OK.pop(path, None)
|
||||
if not _binary_runs(path):
|
||||
raise ValueError(
|
||||
"That file exists but does not run as a media tool "
|
||||
"(its `-version` probe failed) — wrong architecture or not executable."
|
||||
)
|
||||
|
||||
|
||||
def set_custom_path(tool: str, path: str) -> dict:
|
||||
"""Pin *tool* to an explicit binary. Persists via prefs `env.<KEY>` —
|
||||
the exact mechanism /system/set-env uses, so there is one override store."""
|
||||
if tool not in TOOLS:
|
||||
raise ValueError(f"unknown tool '{tool}'")
|
||||
path = path.strip()
|
||||
_validate_binary_path(path)
|
||||
key = _PREF_ENV_KEY[tool]
|
||||
os.environ[key] = path
|
||||
prefs.set_(f"env.{key}", path)
|
||||
_version_cache.pop(path, None)
|
||||
logger.info("media-tools: %s pinned to user path (origin=%s)",
|
||||
tool, _classify_origin(tool, path))
|
||||
return status()["tools"][tool]
|
||||
|
||||
|
||||
def use_system(tool: str) -> dict:
|
||||
"""Auto-detect a system-installed copy and pin it."""
|
||||
if tool not in TOOLS:
|
||||
raise ValueError(f"unknown tool '{tool}'")
|
||||
candidate = _detect_system(tool)
|
||||
if not candidate:
|
||||
raise LookupError(
|
||||
f"No system {tool} found on PATH or in the usual install locations."
|
||||
)
|
||||
return set_custom_path(tool, candidate)
|
||||
|
||||
|
||||
def _detect_system(tool: str) -> str | None:
|
||||
roots = [r for r in (media_tools_dir(), _imageio_pkg_dir()) if r]
|
||||
|
||||
def _is_bundled(p: str) -> bool:
|
||||
rp = os.path.realpath(p)
|
||||
return any(rp.startswith(os.path.realpath(r) + os.sep) for r in roots)
|
||||
|
||||
candidates = [
|
||||
f"/opt/homebrew/bin/{tool}",
|
||||
f"/usr/local/bin/{tool}",
|
||||
f"/usr/bin/{tool}",
|
||||
f"C:\\ffmpeg\\bin\\{tool}.exe",
|
||||
f"C:\\Program Files\\ffmpeg\\bin\\{tool}.exe",
|
||||
tool,
|
||||
]
|
||||
for c in candidates:
|
||||
resolved = shutil.which(c)
|
||||
if resolved and not _is_bundled(resolved) and _binary_runs(resolved):
|
||||
return resolved
|
||||
return None
|
||||
|
||||
|
||||
def restore_bundled(tool: str) -> dict:
|
||||
"""Clear the user override so the chain resolves sidecar → bundled →
|
||||
system again; kick acquisition if no bundled build is present. Always safe."""
|
||||
if tool not in TOOLS:
|
||||
raise ValueError(f"unknown tool '{tool}'")
|
||||
for key in _ENV_KEYS[tool]:
|
||||
if prefs.get(f"env.{key}"):
|
||||
prefs.delete(f"env.{key}")
|
||||
os.environ.pop(key, None)
|
||||
_version_cache.clear()
|
||||
if not (bundled_tool_path(tool) and _binary_runs(bundled_tool_path(tool))):
|
||||
# No local bundled build to fall back to (imageio may still cover
|
||||
# ffmpeg) — fetch ours in the background so the revert lands somewhere.
|
||||
if not _resolve(tool):
|
||||
acquire_bundled()
|
||||
return status()["tools"][tool]
|
||||
|
||||
|
||||
# ── yt-dlp overlay ──────────────────────────────────────────────────────────
|
||||
|
||||
def _ytdlp_overlay_dir() -> str:
|
||||
return os.path.join(media_tools_dir(), "ytdlp_overlay")
|
||||
|
||||
|
||||
def activate_ytdlp_overlay() -> bool:
|
||||
"""Prepend the user-updated yt-dlp overlay to sys.path. Called once at
|
||||
backend startup, before anything imports yt_dlp."""
|
||||
overlay = _ytdlp_overlay_dir()
|
||||
if os.path.isdir(os.path.join(overlay, "yt_dlp")) and overlay not in sys.path:
|
||||
sys.path.insert(0, overlay)
|
||||
logger.info("media-tools: yt-dlp overlay active (%s)",
|
||||
_read_ytdlp_version(os.path.join(overlay, "yt_dlp")) or "?")
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _fetch_pypi_ytdlp() -> tuple[str, str, str]:
|
||||
"""(version, wheel_url, sha256) of the latest yt-dlp wheel on PyPI."""
|
||||
import json
|
||||
import urllib.request
|
||||
req = urllib.request.Request(_PYPI_YTDLP_URL, headers={"User-Agent": "OmniVoice-Studio"})
|
||||
with urllib.request.urlopen(req, timeout=_DOWNLOAD_TIMEOUT_S) as resp:
|
||||
meta = json.load(resp)
|
||||
version = meta["info"]["version"]
|
||||
for artifact in meta.get("urls", []):
|
||||
if artifact.get("packagetype") == "bdist_wheel" and \
|
||||
artifact["filename"].endswith("py3-none-any.whl"):
|
||||
return version, artifact["url"], artifact["digests"]["sha256"]
|
||||
raise RuntimeError(f"no universal wheel found for yt-dlp {version}")
|
||||
|
||||
|
||||
def update_ytdlp(wait: bool = False) -> dict:
|
||||
"""Install the newest yt-dlp into the overlay dir (background thread).
|
||||
|
||||
The wheel is verified against PyPI's own sha256 digest before a single
|
||||
byte lands in the overlay; the swap is atomic (staged dir + os.replace).
|
||||
Takes effect on the next backend start (the running process already
|
||||
imported the old module) — the UI shows the restart affordance.
|
||||
"""
|
||||
with _lock:
|
||||
if _ops["ytdlp_update"]["state"] == "running":
|
||||
return dict(_ops["ytdlp_update"])
|
||||
_ops["ytdlp_update"].update(state="running", progress=0.0, error=None, version=None)
|
||||
|
||||
def _worker():
|
||||
try:
|
||||
version = _do_update_ytdlp()
|
||||
_set_op("ytdlp_update", state="done", progress=1.0, version=version)
|
||||
logger.info("media-tools: yt-dlp overlay updated to %s", version)
|
||||
except Exception as e:
|
||||
logger.warning("media-tools: yt-dlp update failed: %s", e)
|
||||
_set_op("ytdlp_update", state="error", error=str(e)[:300])
|
||||
|
||||
if wait:
|
||||
_worker()
|
||||
else:
|
||||
threading.Thread(target=_worker, name="media-tools-ytdlp", daemon=True).start()
|
||||
return _op_snapshot()["ytdlp_update"]
|
||||
|
||||
|
||||
def _do_update_ytdlp() -> str:
|
||||
version, url, sha = _fetch_pypi_ytdlp()
|
||||
|
||||
# Record the locked ("tested") version once, before the first overlay
|
||||
# ever activates — that's what "Restore tested version" reverts to.
|
||||
if prefs.get("media_tools.ytdlp_baseline") is None:
|
||||
current = _ytdlp_status()
|
||||
if current["origin"] == "bundled" and current["version"]:
|
||||
prefs.set_("media_tools.ytdlp_baseline", current["version"])
|
||||
|
||||
overlay = _ytdlp_overlay_dir()
|
||||
os.makedirs(media_tools_dir(), exist_ok=True)
|
||||
with tempfile.TemporaryDirectory(dir=media_tools_dir()) as tmp:
|
||||
whl = os.path.join(tmp, "yt_dlp.whl")
|
||||
_download(url, whl, sha, None, op="ytdlp_update")
|
||||
staged = os.path.join(tmp, "staged")
|
||||
with zipfile.ZipFile(whl) as zf:
|
||||
for member in zf.infolist():
|
||||
name = member.filename
|
||||
# Only the package itself; wheels carry no absolute paths but
|
||||
# guard against traversal anyway.
|
||||
if not name.startswith("yt_dlp/") or ".." in name:
|
||||
continue
|
||||
zf.extract(member, staged)
|
||||
got = _read_ytdlp_version(os.path.join(staged, "yt_dlp"))
|
||||
if not got:
|
||||
raise RuntimeError("downloaded wheel has no readable yt_dlp version")
|
||||
if os.path.isdir(overlay):
|
||||
shutil.rmtree(overlay, ignore_errors=True)
|
||||
os.replace(staged, overlay)
|
||||
return version
|
||||
|
||||
|
||||
def ytdlp_invocation() -> "tuple[list[str], dict[str, str] | None]":
|
||||
"""(argv prefix, env-or-None) for running the yt-dlp CLI.
|
||||
|
||||
Prefers ``[sys.executable, -m, yt_dlp]`` so the CLI always matches the
|
||||
module the app ships (or the user's overlay — propagated via PYTHONPATH),
|
||||
with no PATH requirement: yt-dlp is never something the user installs.
|
||||
Frozen builds can't re-invoke an interpreter, so they keep the historical
|
||||
PATH lookup as a last resort.
|
||||
"""
|
||||
if not getattr(sys, "frozen", False):
|
||||
try:
|
||||
import importlib.util
|
||||
if importlib.util.find_spec("yt_dlp") is not None:
|
||||
env = None
|
||||
overlay = _ytdlp_overlay_dir()
|
||||
if os.path.isdir(os.path.join(overlay, "yt_dlp")):
|
||||
env = dict(os.environ)
|
||||
env["PYTHONPATH"] = overlay + os.pathsep + env.get("PYTHONPATH", "")
|
||||
return [sys.executable, "-m", "yt_dlp"], env
|
||||
except Exception as e:
|
||||
logger.debug("yt_dlp module CLI unavailable: %s", e)
|
||||
exe = shutil.which("yt-dlp")
|
||||
return ([exe] if exe else ["yt-dlp"]), None
|
||||
|
||||
|
||||
def restore_ytdlp() -> dict:
|
||||
"""Delete the overlay — the locked, tested yt-dlp underneath takes over on
|
||||
next start. Always safe: the locked install was never modified."""
|
||||
overlay = _ytdlp_overlay_dir()
|
||||
if os.path.isdir(overlay):
|
||||
shutil.rmtree(overlay, ignore_errors=True)
|
||||
_set_op("ytdlp_update", state="idle", progress=0.0, error=None, version=None)
|
||||
return _ytdlp_status()
|
||||
@@ -657,6 +657,78 @@ def _hf_offline() -> bool:
|
||||
return _env_flag("HF_HUB_OFFLINE") or _env_flag("TRANSFORMERS_OFFLINE")
|
||||
|
||||
|
||||
# ── Broken-snapshot-link self-heal ───────────────────────────────────
|
||||
# A sibling of the incomplete-cache class above: the blobs are FULLY
|
||||
# downloaded, but the snapshots/<rev>/ entries pointing at them are dangling
|
||||
# symlinks (0 KB) or zero-byte stand-ins — blob-naming mismatches between
|
||||
# download modes, interrupted renames, or antivirus interference all produce
|
||||
# this state (reported on Windows, where the NTFS links show as 0 KB, but the
|
||||
# heal is generic). os.path.isfile() on a dangling link is False, so
|
||||
# transformers raises the same "does not appear to have a file named …"
|
||||
# signature even though the bytes are on disk. The resume repair below can't
|
||||
# fix it (snapshot_download may trust/short-circuit on the existing broken
|
||||
# entry), so rung 0 of the recovery ladder deletes exactly the broken entries
|
||||
# and restores them — see services.hf_cache_repair.
|
||||
|
||||
# Repos this process already attempted the link self-heal for — the retry
|
||||
# after a repair may only happen ONCE per repo per process, so a cache that
|
||||
# stays broken can't loop repair↔retry.
|
||||
_LINK_REPAIR_ATTEMPTED: set[str] = set()
|
||||
|
||||
|
||||
def _selfheal_broken_snapshot_links(checkpoint: str) -> bool:
|
||||
"""Rung 0 of cache recovery: delete-and-restore broken snapshot entries.
|
||||
|
||||
Returns True only when broken entries were found, removed AND restored —
|
||||
i.e. retrying the load is worth it. At most one attempt per repo per
|
||||
process. Never raises; when it returns False the legacy resume/force
|
||||
ladder still runs."""
|
||||
if checkpoint in _LINK_REPAIR_ATTEMPTED:
|
||||
return False
|
||||
_LINK_REPAIR_ATTEMPTED.add(checkpoint)
|
||||
if os.path.isdir(checkpoint):
|
||||
return False # a local-directory checkpoint doesn't use the hub cache
|
||||
try:
|
||||
from services.hf_cache_repair import repair_repo_cache
|
||||
summary = repair_repo_cache(checkpoint)
|
||||
except Exception as repair_err: # repair must never break the ladder
|
||||
logger.warning("Snapshot-link self-heal for %s errored: %s",
|
||||
checkpoint, repair_err)
|
||||
return False
|
||||
if summary.get("removed") and summary.get("ok"):
|
||||
logger.warning(
|
||||
"Model cache for %s had %d broken file link(s) — repaired "
|
||||
"automatically (%s), retrying the load.",
|
||||
checkpoint, summary["removed"],
|
||||
summary.get("outcome") or "healed",
|
||||
)
|
||||
return True
|
||||
if summary.get("found"):
|
||||
logger.warning(
|
||||
"Model cache for %s has %d broken file link(s) that could not be "
|
||||
"auto-repaired (%s).",
|
||||
checkpoint, summary["found"], summary.get("error") or "unknown",
|
||||
)
|
||||
return False
|
||||
|
||||
|
||||
def _manual_cache_delete_hint(checkpoint: str) -> str:
|
||||
"""Names the exact on-disk folder to delete when every auto-repair rung
|
||||
failed — "delete the model" is only actionable if the user can find it.
|
||||
Empty for local-directory checkpoints (they don't live in the hub cache)."""
|
||||
try:
|
||||
if os.path.isdir(checkpoint):
|
||||
return ""
|
||||
from services.hf_cache_repair import repo_cache_dir
|
||||
return (
|
||||
f" If the problem persists, quit OmniVoice, delete "
|
||||
f"{repo_cache_dir(checkpoint)} and restart — the model "
|
||||
"re-downloads automatically."
|
||||
)
|
||||
except Exception:
|
||||
return ""
|
||||
|
||||
|
||||
# Why the LAST _repair_model_cache run failed ("" when it succeeded / hasn't
|
||||
# run). #886: the "could not be auto-repaired" message used to drop the cause
|
||||
# entirely, so a mirror outage, offline mode, or a full disk all read the same.
|
||||
@@ -714,7 +786,13 @@ def _repair_model_cache(checkpoint: str, *, force: bool = False) -> bool:
|
||||
_last_repair_error = f"{type(imp_err).__name__}: {imp_err}"
|
||||
return False
|
||||
dl_kwargs: dict = {"repo_id": checkpoint}
|
||||
endpoint = os.environ.get("HF_ENDPOINT")
|
||||
# Explicit endpoint (HF_ENDPOINT / pref) wins; otherwise the automatic
|
||||
# endpoint selection's cached pick applies (services.endpoint_race).
|
||||
try:
|
||||
from services import endpoint_race
|
||||
endpoint = endpoint_race.effective_endpoint()
|
||||
except Exception: # endpoint resolution must never break the repair
|
||||
endpoint = os.environ.get("HF_ENDPOINT")
|
||||
if endpoint:
|
||||
dl_kwargs["endpoint"] = endpoint
|
||||
if force:
|
||||
@@ -765,8 +843,29 @@ def _repair_model_cache(checkpoint: str, *, force: bool = False) -> bool:
|
||||
checkpoint, attempt, retries, e,
|
||||
)
|
||||
_last_repair_error = f"{type(e).__name__}: {e}"
|
||||
if attempt < retries and backoff:
|
||||
time.sleep(backoff * attempt)
|
||||
if attempt < retries:
|
||||
# Endpoint failover (auto mode only, once per repo per
|
||||
# process — same guard pattern as the snapshot-link rung): a
|
||||
# network-classified repair failure re-races the endpoints so
|
||||
# the next attempt retries on the winner instead of burning
|
||||
# every retry on a dead host. Explicit user endpoints are
|
||||
# never switched.
|
||||
try:
|
||||
from services import endpoint_race
|
||||
if endpoint_race.reselect_after_failure(checkpoint, str(e)):
|
||||
new_ep = endpoint_race.effective_endpoint()
|
||||
if new_ep:
|
||||
dl_kwargs["endpoint"] = new_ep
|
||||
else:
|
||||
dl_kwargs.pop("endpoint", None)
|
||||
logger.info(
|
||||
"Auto-repair of %s: endpoint failover — retrying on %s",
|
||||
checkpoint, new_ep or "https://huggingface.co",
|
||||
)
|
||||
except Exception: # failover must never break the ladder
|
||||
pass
|
||||
if backoff:
|
||||
time.sleep(backoff * attempt)
|
||||
return False
|
||||
|
||||
|
||||
@@ -852,50 +951,76 @@ def _load_model_sync():
|
||||
# cache never reaches this branch, so the fast path is untouched).
|
||||
if not _is_incomplete_cache_error(e):
|
||||
raise
|
||||
_set_loading("loading_weights", "Repairing incomplete model cache…")
|
||||
if not _repair_model_cache(checkpoint):
|
||||
raise RuntimeError(
|
||||
f"The TTS model cache for {checkpoint} is incomplete "
|
||||
"(weights missing — usually an interrupted download)."
|
||||
f"{_repair_failure_detail()} "
|
||||
"Open Settings → Models, delete the OmniVoice TTS model, "
|
||||
"and install it again."
|
||||
) from e
|
||||
_set_loading("loading_weights", f"Loading TTS weights on {device}…")
|
||||
try:
|
||||
_model = _load()
|
||||
except OSError as e2:
|
||||
# Resume-repair ran but the cache is still unusable. The usual
|
||||
# cause beyond "repo genuinely lacks weights" is a blob that's
|
||||
# present with the right size but corrupt — snapshot_download's
|
||||
# resume trusts it and never re-fetches it (#739). Force a full
|
||||
# re-download (replaces corrupt blobs) and retry once more before
|
||||
# falling back to the manual delete-and-reinstall message.
|
||||
if _is_incomplete_cache_error(e2):
|
||||
_set_loading("loading_weights", "Re-downloading model files…")
|
||||
if _repair_model_cache(checkpoint, force=True):
|
||||
try:
|
||||
_model = _load()
|
||||
except OSError as e3:
|
||||
# Rung 0: broken snapshot links — the blobs are on disk but the
|
||||
# snapshot entries don't resolve (dangling symlinks / zero-byte
|
||||
# stand-ins). Delete exactly the broken entries, restore, and
|
||||
# retry the load ONCE (guarded per repo per process). A cache
|
||||
# without broken links falls straight through to the resume
|
||||
# ladder below.
|
||||
_model = None
|
||||
if _selfheal_broken_snapshot_links(checkpoint):
|
||||
_set_loading(
|
||||
"loading_weights",
|
||||
"Model cache had broken file links — repaired "
|
||||
"automatically, retrying…",
|
||||
)
|
||||
try:
|
||||
_model = _load()
|
||||
except OSError as e_link:
|
||||
if not _is_incomplete_cache_error(e_link):
|
||||
raise
|
||||
logger.warning(
|
||||
"Load still failing after snapshot-link repair of %s — "
|
||||
"falling back to resume repair.", checkpoint,
|
||||
)
|
||||
e = e_link
|
||||
_model = None
|
||||
if _model is None:
|
||||
_set_loading("loading_weights", "Repairing incomplete model cache…")
|
||||
if not _repair_model_cache(checkpoint):
|
||||
raise RuntimeError(
|
||||
f"The TTS model cache for {checkpoint} is incomplete "
|
||||
"(weights missing — usually an interrupted download)."
|
||||
f"{_repair_failure_detail()} "
|
||||
"Open Settings → Models, delete the OmniVoice TTS model, "
|
||||
f"and install it again.{_manual_cache_delete_hint(checkpoint)}"
|
||||
) from e
|
||||
_set_loading("loading_weights", f"Loading TTS weights on {device}…")
|
||||
try:
|
||||
_model = _load()
|
||||
except OSError as e2:
|
||||
# Resume-repair ran but the cache is still unusable. The usual
|
||||
# cause beyond "repo genuinely lacks weights" is a blob that's
|
||||
# present with the right size but corrupt — snapshot_download's
|
||||
# resume trusts it and never re-fetches it (#739). Force a full
|
||||
# re-download (replaces corrupt blobs) and retry once more before
|
||||
# falling back to the manual delete-and-reinstall message.
|
||||
if _is_incomplete_cache_error(e2):
|
||||
_set_loading("loading_weights", "Re-downloading model files…")
|
||||
if _repair_model_cache(checkpoint, force=True):
|
||||
try:
|
||||
_model = _load()
|
||||
except OSError as e3:
|
||||
raise RuntimeError(
|
||||
f"The TTS model cache for {checkpoint} is incomplete "
|
||||
"and could not be auto-repaired. Open Settings → "
|
||||
"Models, delete the OmniVoice TTS model, and install "
|
||||
f"it again.{_manual_cache_delete_hint(checkpoint)}"
|
||||
) from e3
|
||||
else:
|
||||
raise RuntimeError(
|
||||
f"The TTS model cache for {checkpoint} is incomplete "
|
||||
"and could not be auto-repaired. Open Settings → "
|
||||
"Models, delete the OmniVoice TTS model, and install "
|
||||
"it again."
|
||||
) from e3
|
||||
f"The TTS model cache for {checkpoint} is incomplete and "
|
||||
f"could not be auto-repaired.{_repair_failure_detail()} "
|
||||
"Open Settings → Models, delete the OmniVoice TTS model, "
|
||||
f"and install it again.{_manual_cache_delete_hint(checkpoint)}"
|
||||
) from e2
|
||||
else:
|
||||
raise RuntimeError(
|
||||
f"The TTS model cache for {checkpoint} is incomplete and "
|
||||
f"could not be auto-repaired.{_repair_failure_detail()} "
|
||||
"Open Settings → Models, delete the OmniVoice TTS model, "
|
||||
"and install it again."
|
||||
"could not be auto-repaired. Open Settings → Models, delete "
|
||||
"the OmniVoice TTS model, and install it again."
|
||||
f"{_manual_cache_delete_hint(checkpoint)}"
|
||||
) from e2
|
||||
else:
|
||||
raise RuntimeError(
|
||||
f"The TTS model cache for {checkpoint} is incomplete and "
|
||||
"could not be auto-repaired. Open Settings → Models, delete "
|
||||
"the OmniVoice TTS model, and install it again."
|
||||
) from e2
|
||||
|
||||
try:
|
||||
# plan-02 (#65): gate on Triton availability (+ user setting), not
|
||||
@@ -957,7 +1082,13 @@ def _load_model_sync():
|
||||
except Exception: # never let failure-formatting mask the real error
|
||||
err_msg = str(exc)
|
||||
_set_loading("error", "Model loading failed", error=err_msg)
|
||||
logger.error("Model loading failed: %s", str(exc))
|
||||
# #1000 class: transformers' lazy-import machinery wraps ANY disruption
|
||||
# to an inner import (including one interrupted by process teardown)
|
||||
# in a generic "Could not import module X. Are this object's
|
||||
# requirements defined correctly?" — logging only str(exc) discarded
|
||||
# the real cause in __cause__/__context__ and made a shutdown race
|
||||
# look like a broken install. exc_info surfaces the full chain.
|
||||
logger.error("Model loading failed: %s", str(exc), exc_info=exc)
|
||||
raise
|
||||
finally:
|
||||
unregister_listener(lid)
|
||||
@@ -1089,7 +1220,11 @@ async def preload_model():
|
||||
model = await _load_model_with_timeout()
|
||||
logger.info("Preload complete — model ready.")
|
||||
except Exception as e:
|
||||
logger.warning("Model preload failed (non-fatal): %s", e)
|
||||
# See the matching exc_info note on the _load_model_sync handler above
|
||||
# (#1000 class) — the full chain, not just str(e), is what actually
|
||||
# distinguishes a real dependency problem from a shutdown-interrupted
|
||||
# import.
|
||||
logger.warning("Model preload failed (non-fatal): %s", e, exc_info=e)
|
||||
|
||||
def get_model_status():
|
||||
is_loaded = model is not None
|
||||
|
||||
@@ -0,0 +1,921 @@
|
||||
"""One-click sidecar-engine provisioner (issue: IndexTTS-2 in-app install).
|
||||
|
||||
Some engines (IndexTTS-2 today; MOSS-v1.5 / dots.tts / Confucius4 are the
|
||||
same shape) can't live in the app venv because they pin a ``transformers``
|
||||
version that conflicts with the parent's ``>=5.3``. They run as sidecars:
|
||||
a source checkout + a dedicated venv + (for IndexTTS-2) model weights in
|
||||
``<checkout>/checkpoints/``. Until now provisioning that trio was four
|
||||
manual terminal steps; this module turns it into a resumable background
|
||||
job the Settings → Engines UI can start and poll.
|
||||
|
||||
Design notes (single source of truth for the choices):
|
||||
|
||||
* **Fetch: git primary, tarball fallback.** ``git clone --depth 1`` is the
|
||||
primary path (fast, matches the documented manual flow, and leaves a
|
||||
repo the user can update). When git is absent — common on Windows — we
|
||||
fall back to downloading the GitHub source tarball over HTTPS (httpx,
|
||||
honours proxy env vars) and extracting it with :mod:`tarfile`. A
|
||||
``pip install git+https://…`` path was rejected because the engine
|
||||
*directory* must exist on disk anyway: the sidecar resolves its venv
|
||||
and model weights relative to it.
|
||||
* **Managed install root:** ``DATA_DIR/engines/<engine_id>/`` — always
|
||||
user-writable (works in frozen/packaged builds where ``backend/`` is
|
||||
read-only), survives app updates, and never collides with a user's own
|
||||
clone. A user-managed install (env var already pointing at their clone)
|
||||
is left completely alone.
|
||||
* **Weights ARE part of the install** for engines whose sidecar loads
|
||||
from ``<checkout>/<weights_subdir>/`` (IndexTTS-2's ``main.py`` reads
|
||||
``$OMNIVOICE_INDEXTTS_DIR/checkpoints/config.yaml`` — verified). The
|
||||
download goes through ``huggingface_hub.snapshot_download`` with the
|
||||
endpoint from :mod:`services.endpoint_race` (HF endpoint auto-select;
|
||||
**no hardcoded huggingface.co**) and the token from
|
||||
:mod:`services.token_resolver`.
|
||||
* **Idempotent + resumable:** every step no-ops when its output is
|
||||
already healthy and repairs it when it is half-there (a checkout
|
||||
without ``pyproject.toml`` is re-fetched; a venv that can't import the
|
||||
probe module is re-installed; ``snapshot_download`` resumes weights).
|
||||
* **Persistence:** on success the checkout path is written to
|
||||
``os.environ[<env_var>]`` (the engine's bootstrap reads the env var, so
|
||||
it works immediately — no restart) and to ``prefs.json`` under
|
||||
``env.<env_var>`` (restored into the environment at startup by
|
||||
``main.py``), the same mechanism Settings' env panel uses.
|
||||
|
||||
Cross-platform: no symlinks, no shell strings (argv lists only), venv
|
||||
layout resolved per-OS (``Scripts/python.exe`` vs ``bin/python``).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tarfile
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
from collections import deque
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Callable, Optional
|
||||
|
||||
from core.config import DATA_DIR
|
||||
|
||||
logger = logging.getLogger("omnivoice.sidecar_install")
|
||||
|
||||
_GIB = 1024 ** 3
|
||||
|
||||
# Headroom kept free on the target volume on top of the estimated install
|
||||
# size. Same needs-X/have-Y message shape as the model-install guard
|
||||
# (api.routers.setup.models.disk_space_error), but a deliberately SMALLER
|
||||
# floor than its MIN_FREE_GB=10: required_bytes here is already a
|
||||
# conservative over-estimate, so stacking the full model-cache headroom on
|
||||
# top would block legitimate installs on ~15 GB-free machines.
|
||||
MIN_FREE_GB = 5
|
||||
|
||||
# Bounded in-memory log per job (last N lines survive; enough for the UI's
|
||||
# log tail and for the failure remediation to quote real output).
|
||||
_LOG_MAX_LINES = 200
|
||||
|
||||
_GIT_CLONE_TIMEOUT_S = 600
|
||||
_TARBALL_TIMEOUT_S = 600
|
||||
_UV_VENV_TIMEOUT_S = 300
|
||||
_UV_PIP_INSTALL_TIMEOUT_S = 3600
|
||||
_IMPORT_PROBE_TIMEOUT_S = 120
|
||||
|
||||
|
||||
# ── Spec ───────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SidecarSpec:
|
||||
"""Everything the provisioner needs to install one sidecar engine.
|
||||
|
||||
Parametrized so future sidecar engines (MOSS-v1.5, dots.tts,
|
||||
Confucius4) become one SPECS entry, not another installer.
|
||||
"""
|
||||
|
||||
engine_id: str
|
||||
display_name: str
|
||||
repo_url: str # git clone URL (primary fetch path)
|
||||
tarball_url: str # source tarball (fallback when git is absent)
|
||||
checkout_dirname: str # directory name of the checkout under the managed root
|
||||
env_var: str # env var the engine's bootstrap reads (install dir)
|
||||
probe_module: str # python -c "import <probe_module>" proves the venv works
|
||||
weights_repo_id: Optional[str] = None # HF repo downloaded into <checkout>/<weights_subdir>
|
||||
weights_subdir: str = "checkpoints"
|
||||
docs_path: str = "docs/engines" # where the manual-install fallback lives
|
||||
required_bytes: int = 12 * _GIB # conservative source+venv+weights estimate for preflight
|
||||
# Called after a successful install/uninstall so the engine's memoised
|
||||
# venv resolution re-probes (import inside the lambda — never at module load).
|
||||
invalidate: Callable[[], None] = field(default=lambda: None)
|
||||
# Cheap "is a healthy install already present?" probe (file existence only).
|
||||
installed_probe: Callable[[], bool] = field(default=lambda: False)
|
||||
|
||||
|
||||
def _indextts_invalidate() -> None:
|
||||
from engines.indextts import bootstrap
|
||||
bootstrap.invalidate()
|
||||
|
||||
|
||||
def _indextts_installed() -> bool:
|
||||
from engines.indextts.bootstrap import is_indextts_installed
|
||||
return is_indextts_installed()
|
||||
|
||||
|
||||
SPECS: dict[str, SidecarSpec] = {
|
||||
"indextts2": SidecarSpec(
|
||||
engine_id="indextts2",
|
||||
display_name="IndexTTS-2",
|
||||
repo_url="https://github.com/index-tts/index-tts.git",
|
||||
tarball_url="https://github.com/index-tts/index-tts/archive/refs/heads/main.tar.gz",
|
||||
checkout_dirname="index-tts",
|
||||
env_var="OMNIVOICE_INDEXTTS_DIR",
|
||||
probe_module="indextts.infer_v2",
|
||||
weights_repo_id="IndexTeam/IndexTTS-2",
|
||||
weights_subdir="checkpoints",
|
||||
docs_path="docs/engines/indextts.md",
|
||||
# ~0.1 GB source + up to ~6 GB venv (torch + transformers<5) +
|
||||
# ~6 GB weights. Deliberately conservative; the preflight subtracts
|
||||
# whatever a partial install already put on disk.
|
||||
required_bytes=12 * _GIB,
|
||||
invalidate=_indextts_invalidate,
|
||||
installed_probe=_indextts_installed,
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def get_spec(engine_id: str) -> Optional[SidecarSpec]:
|
||||
return SPECS.get(engine_id)
|
||||
|
||||
|
||||
def persistent_env_vars() -> set[str]:
|
||||
"""Env vars the provisioner persists — merged into the Settings env-var
|
||||
allowlist (api.routers.system.PERSISTENT_KEYS) so users can inspect or
|
||||
clear them from the same panel as every other persisted var."""
|
||||
return {s.env_var for s in SPECS.values()}
|
||||
|
||||
|
||||
# ── Paths ──────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def managed_root(spec: SidecarSpec) -> Path:
|
||||
"""Per-engine managed install root (checkout lives inside it)."""
|
||||
return Path(DATA_DIR) / "engines" / spec.engine_id
|
||||
|
||||
|
||||
def managed_checkout(spec: SidecarSpec) -> Path:
|
||||
return managed_root(spec) / spec.checkout_dirname
|
||||
|
||||
|
||||
def _venv_python(venv_dir: Path) -> Path:
|
||||
"""Venv python path, per-OS layout (Windows: Scripts/, POSIX: bin/)."""
|
||||
if sys.platform == "win32":
|
||||
return venv_dir / "Scripts" / "python.exe"
|
||||
return venv_dir / "bin" / "python"
|
||||
|
||||
|
||||
def _locate_uv() -> Optional[str]:
|
||||
"""Find uv: bundled (Tauri-set OMNIVOICE_BUNDLED_UV) first, then PATH.
|
||||
|
||||
Same resolution order as engines.indextts.bootstrap._locate_uv — the
|
||||
canonical uv-resolution pattern for sidecar venvs.
|
||||
"""
|
||||
bundled = os.environ.get("OMNIVOICE_BUNDLED_UV")
|
||||
if bundled and Path(bundled).is_file():
|
||||
return bundled
|
||||
return shutil.which("uv")
|
||||
|
||||
|
||||
# ── Disk preflight ─────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _dir_size_bytes(path: Path) -> int:
|
||||
"""Best-effort recursive size (bytes already spent by a partial install)."""
|
||||
total = 0
|
||||
try:
|
||||
for root, _dirs, files in os.walk(path):
|
||||
for f in files:
|
||||
try:
|
||||
total += os.path.getsize(os.path.join(root, f))
|
||||
except OSError:
|
||||
continue
|
||||
except OSError:
|
||||
pass # unreadable dir — treat as zero bytes spent
|
||||
return total
|
||||
|
||||
|
||||
def disk_free_bytes(path: Path) -> int:
|
||||
"""Free bytes on the volume backing *path* (nearest existing ancestor).
|
||||
Never raises; 0 when the volume can't be probed."""
|
||||
try:
|
||||
p = path.resolve()
|
||||
while not p.exists():
|
||||
parent = p.parent
|
||||
if parent == p:
|
||||
break
|
||||
p = parent
|
||||
return int(shutil.disk_usage(str(p)).free)
|
||||
except Exception:
|
||||
return 0
|
||||
|
||||
|
||||
def disk_space_error(spec: SidecarSpec) -> Optional[str]:
|
||||
"""Actionable message when the estimated remaining install won't fit
|
||||
(needs X + headroom Y, have Z — same shape as the model-install guard);
|
||||
``None`` when it fits or the volume can't be probed."""
|
||||
root = managed_root(spec)
|
||||
already = _dir_size_bytes(root)
|
||||
remaining = max(0, spec.required_bytes - already)
|
||||
free = disk_free_bytes(root)
|
||||
if free <= 0:
|
||||
return None # can't probe → never block on missing information
|
||||
required = remaining + MIN_FREE_GB * _GIB
|
||||
if free >= required:
|
||||
return None
|
||||
|
||||
def _gb(n: int) -> str:
|
||||
return f"{n / _GIB:.1f} GB"
|
||||
|
||||
return (
|
||||
f"Not enough disk space to install {spec.display_name}: it needs about "
|
||||
f"{_gb(remaining)} plus {MIN_FREE_GB} GB free headroom ({_gb(required)} total), "
|
||||
f"but only {_gb(free)} is free at {root}. Free up space and retry."
|
||||
)
|
||||
|
||||
|
||||
# ── Job state ──────────────────────────────────────────────────────────────
|
||||
|
||||
STEP_IDS = (
|
||||
"preflight",
|
||||
"fetch_source",
|
||||
"create_venv",
|
||||
"install_deps",
|
||||
"verify",
|
||||
"fetch_weights",
|
||||
"persist",
|
||||
)
|
||||
|
||||
_jobs: dict[str, dict] = {}
|
||||
_jobs_lock = threading.Lock()
|
||||
|
||||
# Guards each job's log deque: the worker thread appends while the status
|
||||
# poll copies it, and list() over a deque raises RuntimeError if it mutates
|
||||
# mid-iteration. One module-level lock is plenty — appends are tiny and at
|
||||
# most one job runs per engine.
|
||||
_log_lock = threading.Lock()
|
||||
|
||||
|
||||
class _StepError(Exception):
|
||||
"""Install-step failure carrying user-facing remediation text."""
|
||||
|
||||
def __init__(self, message: str, remediation: str):
|
||||
super().__init__(message)
|
||||
self.remediation = remediation
|
||||
|
||||
|
||||
def _new_job(engine_id: str) -> dict:
|
||||
return {
|
||||
"engine_id": engine_id,
|
||||
"state": "running",
|
||||
"steps": [{"id": s, "state": "pending", "detail": None} for s in STEP_IDS],
|
||||
"log": deque(maxlen=_LOG_MAX_LINES),
|
||||
"error": None,
|
||||
"remediation": None,
|
||||
"weights_progress": None,
|
||||
"started_at": time.time(),
|
||||
"finished_at": None,
|
||||
}
|
||||
|
||||
|
||||
def _job_step(job: dict, step_id: str) -> dict:
|
||||
return next(s for s in job["steps"] if s["id"] == step_id)
|
||||
|
||||
|
||||
def _log(job: dict, line: str) -> None:
|
||||
line = line.rstrip()
|
||||
if line:
|
||||
with _log_lock:
|
||||
job["log"].append(line)
|
||||
logger.info("[%s install] %s", job["engine_id"], line)
|
||||
|
||||
|
||||
def _serialize_job(job: Optional[dict]) -> Optional[dict]:
|
||||
if job is None:
|
||||
return None
|
||||
out = dict(job)
|
||||
with _log_lock:
|
||||
out["log"] = list(job["log"])
|
||||
out["steps"] = [dict(s) for s in job["steps"]]
|
||||
return out
|
||||
|
||||
|
||||
def get_status(engine_id: str) -> dict:
|
||||
"""Install state + last/current job for one engine. Cheap (file probes)."""
|
||||
spec = get_spec(engine_id)
|
||||
if spec is None:
|
||||
raise KeyError(engine_id)
|
||||
with _jobs_lock:
|
||||
job = _serialize_job(_jobs.get(engine_id))
|
||||
installed = _healthy(spec)
|
||||
checkout = managed_checkout(spec)
|
||||
env_dir = os.environ.get(spec.env_var)
|
||||
return {
|
||||
"engine_id": engine_id,
|
||||
"installed": installed,
|
||||
# True when the on-disk install is the app-managed one (uninstallable
|
||||
# from the app). A user's own clone is never "managed".
|
||||
"managed": bool(
|
||||
checkout.is_dir()
|
||||
and (not env_dir or Path(env_dir) == checkout)
|
||||
),
|
||||
"install_dir": env_dir or (str(checkout) if checkout.is_dir() else None),
|
||||
"job": job,
|
||||
}
|
||||
|
||||
|
||||
def _safe_installed(spec: SidecarSpec) -> bool:
|
||||
try:
|
||||
return bool(spec.installed_probe())
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def _user_managed_dir(spec: SidecarSpec) -> Optional[Path]:
|
||||
"""The user's own install dir when the env var points anywhere but the
|
||||
app-managed checkout; None for managed/unset (ours to provision)."""
|
||||
env_dir = os.environ.get(spec.env_var)
|
||||
if env_dir and Path(env_dir) != managed_checkout(spec):
|
||||
return Path(env_dir)
|
||||
return None
|
||||
|
||||
|
||||
def _healthy(spec: SidecarSpec) -> bool:
|
||||
"""A COMPLETE install: for a user-managed dir, trust the engine's own
|
||||
probe (their clone, their layout); for the app-managed install require
|
||||
the venv AND the fully-downloaded weights, so a partial install repairs
|
||||
instead of reporting already_installed."""
|
||||
if _user_managed_dir(spec) is not None:
|
||||
return _safe_installed(spec)
|
||||
checkout = managed_checkout(spec)
|
||||
if not checkout.is_dir():
|
||||
# No managed install at all. A legacy install may still exist (e.g.
|
||||
# IndexTTS's old lazy-bootstrap venv under backend/engines/) — trust
|
||||
# the engine's own probe so we never re-provision over a working one.
|
||||
return _safe_installed(spec)
|
||||
if not _venv_python(checkout / ".venv").is_file():
|
||||
return False
|
||||
if spec.weights_repo_id and not _weights_present(spec):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _persist(spec: SidecarSpec) -> None:
|
||||
"""Point the engine at the managed checkout: process env for immediate
|
||||
use, prefs.json ``env.*`` for the next launch, and invalidate the
|
||||
engine's memoised venv resolution so it re-probes without a restart."""
|
||||
checkout = managed_checkout(spec)
|
||||
os.environ[spec.env_var] = str(checkout)
|
||||
from core import prefs
|
||||
prefs.set_(f"env.{spec.env_var}", str(checkout))
|
||||
try:
|
||||
spec.invalidate()
|
||||
except Exception:
|
||||
pass # best-effort cache invalidation — the env var is already set
|
||||
|
||||
|
||||
def start_install(engine_id: str) -> dict:
|
||||
"""Start (or report) the install job for *engine_id*.
|
||||
|
||||
Returns ``{"status": "started"|"already_running"|"already_installed", ...}``.
|
||||
Raises KeyError for an engine with no sidecar spec.
|
||||
"""
|
||||
spec = get_spec(engine_id)
|
||||
if spec is None:
|
||||
raise KeyError(engine_id)
|
||||
with _jobs_lock:
|
||||
existing = _jobs.get(engine_id)
|
||||
if existing and existing["state"] == "running":
|
||||
return {"status": "already_running", "engine": engine_id}
|
||||
# A healthy install (user-managed or app-managed) never reinstalls;
|
||||
# a PARTIAL managed install falls through so the job repairs it.
|
||||
if _healthy(spec):
|
||||
# Self-heal: a healthy MANAGED install whose env var was lost
|
||||
# (e.g. prefs.json wiped) just needs re-pointing, not a reinstall.
|
||||
if (
|
||||
_user_managed_dir(spec) is None
|
||||
and _venv_python(managed_checkout(spec) / ".venv").is_file()
|
||||
and not _safe_installed(spec)
|
||||
):
|
||||
_persist(spec)
|
||||
return {"status": "already_installed", "engine": engine_id}
|
||||
job = _new_job(engine_id)
|
||||
_jobs[engine_id] = job
|
||||
th = threading.Thread(
|
||||
target=_run_install, args=(spec, job),
|
||||
name=f"sidecar-install-{engine_id}", daemon=True,
|
||||
)
|
||||
th.start()
|
||||
return {"status": "started", "engine": engine_id}
|
||||
|
||||
|
||||
def uninstall(engine_id: str) -> dict:
|
||||
"""Remove the app-managed install and clear the persisted path.
|
||||
|
||||
Refuses to touch a user-managed install (env var pointing anywhere but
|
||||
the managed checkout) — those were never ours to delete.
|
||||
"""
|
||||
spec = get_spec(engine_id)
|
||||
if spec is None:
|
||||
raise KeyError(engine_id)
|
||||
with _jobs_lock:
|
||||
job = _jobs.get(engine_id)
|
||||
if job and job["state"] == "running":
|
||||
return {"status": "install_in_progress", "engine": engine_id}
|
||||
env_dir = os.environ.get(spec.env_var)
|
||||
checkout = managed_checkout(spec)
|
||||
if env_dir and Path(env_dir) != checkout:
|
||||
return {
|
||||
"status": "not_managed",
|
||||
"engine": engine_id,
|
||||
"detail": (
|
||||
f"{spec.display_name} points at {env_dir}, which OmniVoice did not "
|
||||
f"install. Remove that directory yourself if you want it gone, or "
|
||||
f"clear {spec.env_var} in Settings."
|
||||
),
|
||||
}
|
||||
root = managed_root(spec)
|
||||
removed = root.is_dir()
|
||||
shutil.rmtree(root, ignore_errors=True)
|
||||
if env_dir: # only ever the managed checkout at this point
|
||||
os.environ.pop(spec.env_var, None)
|
||||
from core import prefs
|
||||
if prefs.get(f"env.{spec.env_var}") == str(checkout):
|
||||
prefs.delete(f"env.{spec.env_var}")
|
||||
try:
|
||||
spec.invalidate()
|
||||
except Exception:
|
||||
pass # best-effort cache invalidation — uninstall already succeeded
|
||||
with _jobs_lock:
|
||||
_jobs.pop(engine_id, None)
|
||||
return {"status": "uninstalled" if removed else "not_installed", "engine": engine_id}
|
||||
|
||||
|
||||
# ── Worker ─────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _run_install(spec: SidecarSpec, job: dict) -> None:
|
||||
step_fns: list[tuple[str, Callable[[SidecarSpec, dict], None]]] = [
|
||||
("preflight", _step_preflight),
|
||||
("fetch_source", _step_fetch_source),
|
||||
("create_venv", _step_create_venv),
|
||||
("install_deps", _step_install_deps),
|
||||
("verify", _step_verify),
|
||||
("fetch_weights", _step_fetch_weights),
|
||||
("persist", _step_persist),
|
||||
]
|
||||
try:
|
||||
for step_id, fn in step_fns:
|
||||
step = _job_step(job, step_id)
|
||||
step["state"] = "running"
|
||||
try:
|
||||
fn(spec, job)
|
||||
except _StepError:
|
||||
step["state"] = "error"
|
||||
raise
|
||||
except Exception as exc: # noqa: BLE001 — surfaced into the job
|
||||
step["state"] = "error"
|
||||
raise _StepError(
|
||||
f"{type(exc).__name__}: {exc}",
|
||||
"Re-run the install — it resumes from where it stopped. If it "
|
||||
f"keeps failing, see {spec.docs_path} for the manual steps.",
|
||||
) from exc
|
||||
if step["state"] == "running":
|
||||
step["state"] = "done"
|
||||
job["state"] = "succeeded"
|
||||
_log(job, f"{spec.display_name} installed successfully.")
|
||||
except _StepError as exc:
|
||||
job["state"] = "failed"
|
||||
job["error"] = str(exc)
|
||||
job["remediation"] = exc.remediation
|
||||
_log(job, f"FAILED: {exc}")
|
||||
finally:
|
||||
job["finished_at"] = time.time()
|
||||
|
||||
|
||||
def _step_preflight(spec: SidecarSpec, job: dict) -> None:
|
||||
if _locate_uv() is None:
|
||||
raise _StepError(
|
||||
"uv was not found (checked the bundled path via OMNIVOICE_BUNDLED_UV, "
|
||||
"then PATH).",
|
||||
"Install uv from https://docs.astral.sh/uv/ and relaunch OmniVoice, or "
|
||||
"set OMNIVOICE_BUNDLED_UV to the absolute path of a uv binary.",
|
||||
)
|
||||
err = disk_space_error(spec)
|
||||
if err:
|
||||
raise _StepError(err, "Free up disk space (or move OmniVoice's data directory "
|
||||
"to a larger volume) and retry.")
|
||||
managed_root(spec).mkdir(parents=True, exist_ok=True)
|
||||
_job_step(job, "preflight")["detail"] = "uv found, disk space OK"
|
||||
_log(job, "Preflight OK — uv resolved and enough free disk space.")
|
||||
|
||||
|
||||
def _step_fetch_source(spec: SidecarSpec, job: dict) -> None:
|
||||
step = _job_step(job, "fetch_source")
|
||||
checkout = managed_checkout(spec)
|
||||
if (checkout / "pyproject.toml").is_file():
|
||||
step["state"] = "done"
|
||||
step["detail"] = "source already present"
|
||||
_log(job, f"Source already present at {checkout} — skipping fetch.")
|
||||
return
|
||||
if checkout.exists():
|
||||
# Half-fetched checkout (no pyproject.toml) — repair by refetching.
|
||||
_log(job, f"Removing incomplete checkout at {checkout} …")
|
||||
shutil.rmtree(checkout, ignore_errors=True)
|
||||
|
||||
git = shutil.which("git")
|
||||
if git:
|
||||
_log(job, f"Cloning {spec.repo_url} (git, depth 1) …")
|
||||
rc = _run_logged(job, [git, "clone", "--depth", "1", spec.repo_url, str(checkout)],
|
||||
timeout=_GIT_CLONE_TIMEOUT_S)
|
||||
if rc == 0 and (checkout / "pyproject.toml").is_file():
|
||||
step["detail"] = "git clone"
|
||||
return
|
||||
_log(job, f"git clone failed (exit {rc}) — falling back to source tarball.")
|
||||
shutil.rmtree(checkout, ignore_errors=True)
|
||||
else:
|
||||
_log(job, "git not found — using the source-tarball fallback.")
|
||||
|
||||
_fetch_tarball(spec, job, checkout)
|
||||
if not (checkout / "pyproject.toml").is_file():
|
||||
raise _StepError(
|
||||
f"Fetched source at {checkout} has no pyproject.toml — the download "
|
||||
"appears incomplete or the upstream layout changed.",
|
||||
"Re-run the install; if it keeps failing, clone the repository "
|
||||
f"manually and set {spec.env_var} to the clone (see the engine docs).",
|
||||
)
|
||||
step["detail"] = "source tarball"
|
||||
|
||||
|
||||
def _fetch_tarball(spec: SidecarSpec, job: dict, checkout: Path) -> None:
|
||||
"""Download + extract the GitHub source tarball (no git required).
|
||||
|
||||
Extraction is member-validated (no absolute paths / parent escapes) and
|
||||
never uses symlinks, so it behaves identically on Windows.
|
||||
"""
|
||||
import httpx
|
||||
|
||||
root = managed_root(spec)
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
_log(job, f"Downloading {spec.tarball_url} …")
|
||||
fd, tmp_tar = tempfile.mkstemp(suffix=".tar.gz", dir=str(root))
|
||||
try:
|
||||
with os.fdopen(fd, "wb") as out:
|
||||
with httpx.stream(
|
||||
"GET", spec.tarball_url, follow_redirects=True,
|
||||
timeout=_TARBALL_TIMEOUT_S,
|
||||
) as resp:
|
||||
resp.raise_for_status()
|
||||
for chunk in resp.iter_bytes():
|
||||
out.write(chunk)
|
||||
_log(job, "Extracting source tarball …")
|
||||
with tempfile.TemporaryDirectory(dir=str(root)) as tmp_dir:
|
||||
with tarfile.open(tmp_tar, "r:gz") as tf:
|
||||
try:
|
||||
tf.extractall(tmp_dir, filter="data") # stdlib safe-extract (3.11.4+)
|
||||
except TypeError: # pragma: no cover — pre-filter= interpreters
|
||||
_safe_extract_members(tf, tmp_dir)
|
||||
entries = [p for p in Path(tmp_dir).iterdir() if p.is_dir()]
|
||||
if len(entries) != 1:
|
||||
raise _StepError(
|
||||
f"Unexpected tarball layout ({len(entries)} top-level dirs).",
|
||||
"Re-run the install; if it keeps failing, clone the repository "
|
||||
f"manually and set {spec.env_var} (see the engine docs).",
|
||||
)
|
||||
# os.replace-style move keeps this atomic-ish on the same volume.
|
||||
shutil.move(str(entries[0]), str(checkout))
|
||||
finally:
|
||||
try:
|
||||
os.unlink(tmp_tar)
|
||||
except OSError:
|
||||
pass # temp tarball already gone / locked — harmless leftover
|
||||
|
||||
|
||||
def _safe_extract_members(tf: "tarfile.TarFile", dest: str) -> None:
|
||||
"""Tar-slip-guarded extraction for interpreters without
|
||||
``extractall(filter="data")`` (Python < 3.11.4).
|
||||
|
||||
Mirrors what the "data" filter enforces: only regular files and
|
||||
directories (no symlinks/hardlinks/devices — also keeps Windows
|
||||
behaviour identical), no absolute paths, and every resolved target must
|
||||
stay inside *dest*.
|
||||
"""
|
||||
dest_abs = os.path.abspath(dest)
|
||||
for member in tf.getmembers():
|
||||
if not (member.isreg() or member.isdir()):
|
||||
continue # drop symlinks/hardlinks/devices/fifos
|
||||
name = member.name
|
||||
if name.startswith(("/", "\\")) or ".." in name.replace("\\", "/").split("/"):
|
||||
continue # absolute path or parent-dir escape
|
||||
target = os.path.abspath(os.path.join(dest, name))
|
||||
if os.path.commonpath([dest_abs, target]) != dest_abs:
|
||||
continue # resolved outside the extraction dir
|
||||
tf.extract(member, dest)
|
||||
|
||||
|
||||
def _step_create_venv(spec: SidecarSpec, job: dict) -> None:
|
||||
step = _job_step(job, "create_venv")
|
||||
checkout = managed_checkout(spec)
|
||||
venv_dir = checkout / ".venv"
|
||||
py = _venv_python(venv_dir)
|
||||
if py.is_file():
|
||||
step["state"] = "done"
|
||||
step["detail"] = "venv already present"
|
||||
_log(job, f"Venv already present at {venv_dir} — skipping.")
|
||||
return
|
||||
uv = _locate_uv()
|
||||
_log(job, f"Creating venv at {venv_dir} …")
|
||||
rc = _run_logged(job, [uv, "venv", str(venv_dir)], timeout=_UV_VENV_TIMEOUT_S)
|
||||
if rc != 0 or not py.is_file():
|
||||
raise _StepError(
|
||||
f"uv venv failed (exit {rc}) at {venv_dir}.",
|
||||
"Check the log above for the uv error; free disk space or fix "
|
||||
"permissions on the data directory, then re-run the install.",
|
||||
)
|
||||
step["detail"] = "venv created"
|
||||
|
||||
|
||||
def _step_install_deps(spec: SidecarSpec, job: dict) -> None:
|
||||
"""`uv pip install -e <checkout>` into the dedicated venv.
|
||||
|
||||
Deliberately NOT `uv sync` — sync would apply the sidecar's lockfile
|
||||
semantics; `uv pip install -e` resolves the sidecar's own pins
|
||||
(e.g. transformers<5) inside ITS venv, never touching the parent app.
|
||||
Idempotent: re-running repairs a partial dependency set.
|
||||
"""
|
||||
checkout = managed_checkout(spec)
|
||||
py = _venv_python(checkout / ".venv")
|
||||
uv = _locate_uv()
|
||||
_log(job, f"Installing {spec.display_name} into its venv (this can take several minutes) …")
|
||||
rc = _run_logged(
|
||||
job,
|
||||
[uv, "pip", "install", "--python", str(py), "-e", str(checkout)],
|
||||
timeout=_UV_PIP_INSTALL_TIMEOUT_S,
|
||||
)
|
||||
if rc != 0:
|
||||
raise _StepError(
|
||||
f"uv pip install -e failed (exit {rc}).",
|
||||
"Usually a network hiccup — re-run the install to resume. Behind a "
|
||||
"proxy, set HTTPS_PROXY in Settings → Environment first.",
|
||||
)
|
||||
_job_step(job, "install_deps")["detail"] = "dependencies installed"
|
||||
|
||||
|
||||
def _step_verify(spec: SidecarSpec, job: dict) -> None:
|
||||
checkout = managed_checkout(spec)
|
||||
py = _venv_python(checkout / ".venv")
|
||||
_log(job, f"Verifying `import {spec.probe_module}` inside the venv …")
|
||||
try:
|
||||
proc = subprocess.run(
|
||||
[str(py), "-c", f"import {spec.probe_module}"],
|
||||
capture_output=True, timeout=_IMPORT_PROBE_TIMEOUT_S,
|
||||
)
|
||||
except (subprocess.TimeoutExpired, OSError) as exc:
|
||||
raise _StepError(
|
||||
f"Import probe failed to run: {exc}",
|
||||
"Re-run the install; if it keeps failing, delete the engine in "
|
||||
"Settings → Engines and install again.",
|
||||
) from exc
|
||||
if proc.returncode != 0:
|
||||
tail = proc.stderr.decode("utf-8", errors="replace")[-500:]
|
||||
raise _StepError(
|
||||
f"`import {spec.probe_module}` failed in the new venv: {tail}",
|
||||
"Re-run the install — dependency resolution resumes and repairs "
|
||||
"partial installs. If it keeps failing, use the manual install in "
|
||||
"the engine docs.",
|
||||
)
|
||||
_job_step(job, "verify")["detail"] = f"import {spec.probe_module} OK"
|
||||
_log(job, "Venv verified.")
|
||||
|
||||
|
||||
# Written into the weights dir after snapshot_download COMPLETES. A partial
|
||||
# multi-shard download can leave config.yaml + several plausible shards on
|
||||
# disk, so file heuristics alone would declare a killed-mid-download install
|
||||
# healthy and never resume it (the sidecar edition of #352). Only this
|
||||
# installer writes the marker; user-managed clones never hit this path.
|
||||
_WEIGHTS_COMPLETE_MARKER = ".omnivoice_weights_complete"
|
||||
|
||||
|
||||
def _weights_present(spec: SidecarSpec) -> bool:
|
||||
"""True only for a COMPLETED weights download: the completion marker
|
||||
plus a sanity floor (config.yaml + one ≥5 MB weight file — the same
|
||||
truncated-download floor the model store uses)."""
|
||||
wdir = managed_checkout(spec) / spec.weights_subdir
|
||||
if not (wdir / _WEIGHTS_COMPLETE_MARKER).is_file():
|
||||
return False
|
||||
return _weights_floor_ok(wdir)
|
||||
|
||||
|
||||
def _weights_floor_ok(wdir: Path) -> bool:
|
||||
if not (wdir / "config.yaml").is_file():
|
||||
return False
|
||||
floor = 5 * 1024 * 1024
|
||||
try:
|
||||
for root, _dirs, files in os.walk(wdir):
|
||||
for f in files:
|
||||
try:
|
||||
if os.path.getsize(os.path.join(root, f)) >= floor:
|
||||
return True
|
||||
except OSError:
|
||||
continue
|
||||
except OSError:
|
||||
pass # unreadable weights dir — treat as not present
|
||||
return False
|
||||
|
||||
|
||||
def _step_fetch_weights(spec: SidecarSpec, job: dict) -> None:
|
||||
step = _job_step(job, "fetch_weights")
|
||||
if not spec.weights_repo_id:
|
||||
step["state"] = "skipped"
|
||||
step["detail"] = "engine has no bundled-weights requirement"
|
||||
return
|
||||
if _weights_present(spec):
|
||||
step["state"] = "done"
|
||||
step["detail"] = "weights already present"
|
||||
_log(job, "Model weights already present — skipping download.")
|
||||
return
|
||||
|
||||
wdir = managed_checkout(spec) / spec.weights_subdir
|
||||
wdir.mkdir(parents=True, exist_ok=True)
|
||||
_log(job, f"Downloading {spec.weights_repo_id} → {wdir} (several GB — resumable) …")
|
||||
|
||||
from huggingface_hub import snapshot_download
|
||||
from services import endpoint_race
|
||||
from services.token_resolver import resolve as resolve_token
|
||||
from utils import hf_progress
|
||||
|
||||
# Mirror per-file byte progress into the job so the polling UI can show
|
||||
# it — same tqdm hook the model store's SSE feed uses.
|
||||
def _listener(ev: dict) -> None:
|
||||
try:
|
||||
# Only mirror events for OUR repo — a concurrent model-store
|
||||
# download must not scribble its progress into this job.
|
||||
if ev.get("repo_id") not in (None, spec.weights_repo_id):
|
||||
return
|
||||
job["weights_progress"] = {
|
||||
"filename": ev.get("filename"),
|
||||
"downloaded": ev.get("downloaded"),
|
||||
"total": ev.get("total"),
|
||||
"pct": ev.get("pct"),
|
||||
}
|
||||
except Exception:
|
||||
pass # progress mirroring is advisory — never break the download
|
||||
|
||||
listener_id = hf_progress.register_listener(_listener)
|
||||
repo_token = hf_progress.current_repo_id.set(spec.weights_repo_id)
|
||||
try:
|
||||
# Tracks the repo's default branch on purpose (same policy as every
|
||||
# other model download in the app — see setup/download.py): the
|
||||
# source checkout is unpinned upstream `main` anyway, and hf_hub
|
||||
# checksum-verifies each artifact. Hence the B615 waiver below.
|
||||
kwargs: dict = {
|
||||
"repo_id": spec.weights_repo_id,
|
||||
"local_dir": str(wdir),
|
||||
"token": resolve_token(),
|
||||
}
|
||||
endpoint = endpoint_race.effective_endpoint()
|
||||
if endpoint:
|
||||
kwargs["endpoint"] = endpoint
|
||||
tqdm_cls = hf_progress.tracked_tqdm_class()
|
||||
if tqdm_cls is not None:
|
||||
kwargs["tqdm_class"] = tqdm_cls
|
||||
try:
|
||||
snapshot_download(**kwargs) # nosec B615 — deliberate default-branch policy, see above
|
||||
except Exception as exc:
|
||||
raise _StepError(
|
||||
f"Model weight download failed: {exc}",
|
||||
"Re-run the install — the download resumes where it stopped. "
|
||||
"Check Settings → Network (HF endpoint / proxy) if it keeps failing.",
|
||||
) from exc
|
||||
finally:
|
||||
hf_progress.unregister_listener(listener_id)
|
||||
hf_progress.current_repo_id.reset(repo_token)
|
||||
|
||||
if not _weights_floor_ok(wdir):
|
||||
raise _StepError(
|
||||
"Weight download finished but no plausible weight files were found — "
|
||||
"the download was likely interrupted.",
|
||||
"Re-run the install to resume the download.",
|
||||
)
|
||||
# snapshot_download returned AND the sanity floor holds → mark complete,
|
||||
# so _weights_present/_healthy stop treating this dir as a partial.
|
||||
(wdir / _WEIGHTS_COMPLETE_MARKER).write_text(
|
||||
f"{spec.weights_repo_id}\n{time.time():.0f}\n", encoding="utf-8",
|
||||
)
|
||||
step["detail"] = "weights downloaded"
|
||||
_log(job, "Model weights downloaded.")
|
||||
|
||||
|
||||
def _step_persist(spec: SidecarSpec, job: dict) -> None:
|
||||
_persist(spec)
|
||||
_job_step(job, "persist")["detail"] = f"{spec.env_var}={managed_checkout(spec)}"
|
||||
_log(job, f"Saved {spec.env_var} — the engine is ready to use, no restart needed.")
|
||||
|
||||
|
||||
# ── Subprocess runner with live log capture ────────────────────────────────
|
||||
|
||||
|
||||
def _run_logged(job: dict, argv: list[str], *, timeout: float) -> int:
|
||||
"""Run *argv*, streaming combined stdout+stderr lines into the job log.
|
||||
|
||||
Returns the exit code; -1 on timeout (process tree killed) or spawn
|
||||
failure. argv-list only — never a shell string — so paths with spaces
|
||||
are safe on every platform.
|
||||
|
||||
The stdout drain runs on its own daemon thread and the main flow blocks
|
||||
on ``proc.wait(timeout=…)``. That bounds the step even when a grandchild
|
||||
(uv resolver worker, git helper) inherits the pipe and outlives the
|
||||
killed child — a blocking ``for line in proc.stdout`` on this thread
|
||||
would hang past the timeout waiting for pipe EOF.
|
||||
"""
|
||||
popen_kwargs: dict = {}
|
||||
if os.name == "posix":
|
||||
# New session → we can kill the whole process group on timeout
|
||||
# instead of only the direct child.
|
||||
popen_kwargs["start_new_session"] = True
|
||||
try:
|
||||
proc = subprocess.Popen(
|
||||
argv,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
text=True,
|
||||
encoding="utf-8",
|
||||
errors="replace",
|
||||
**popen_kwargs,
|
||||
)
|
||||
except OSError as exc:
|
||||
_log(job, f"failed to spawn {argv[0]}: {exc}")
|
||||
return -1
|
||||
|
||||
def _drain() -> None:
|
||||
try:
|
||||
assert proc.stdout is not None
|
||||
for line in proc.stdout:
|
||||
_log(job, line)
|
||||
except (OSError, ValueError):
|
||||
pass # pipe closed by the timeout kill — nothing left to read
|
||||
|
||||
drain = threading.Thread(target=_drain, daemon=True)
|
||||
drain.start()
|
||||
try:
|
||||
rc = proc.wait(timeout=timeout)
|
||||
except subprocess.TimeoutExpired:
|
||||
_kill_tree(proc)
|
||||
_log(job, f"process timed out after {timeout:.0f}s — killed")
|
||||
return -1
|
||||
drain.join(5.0) # give the drain a moment to flush the tail
|
||||
return rc if rc is not None else -1
|
||||
|
||||
|
||||
def _kill_tree(proc: "subprocess.Popen") -> None:
|
||||
"""Kill the child and its whole process tree, on every platform.
|
||||
|
||||
POSIX: the child was started in its own session, so SIGKILL the group.
|
||||
Windows: ``proc.kill()`` only terminates the direct child — a git/uv
|
||||
helper it spawned would keep running (and writing into the checkout)
|
||||
past our timeout — so use ``taskkill /T`` to fell the tree.
|
||||
"""
|
||||
if os.name == "posix":
|
||||
import signal
|
||||
try:
|
||||
os.killpg(proc.pid, signal.SIGKILL)
|
||||
return
|
||||
except (ProcessLookupError, PermissionError, OSError):
|
||||
pass # group already gone / not ours — fall through to plain kill
|
||||
else: # Windows
|
||||
try:
|
||||
subprocess.run(
|
||||
["taskkill", "/F", "/T", "/PID", str(proc.pid)],
|
||||
capture_output=True, timeout=15,
|
||||
)
|
||||
return
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
pass # taskkill unavailable/failed — fall through to plain kill
|
||||
try:
|
||||
proc.kill()
|
||||
except OSError:
|
||||
pass # process already exited
|
||||
|
||||
|
||||
__all__ = [
|
||||
"SPECS",
|
||||
"SidecarSpec",
|
||||
"disk_space_error",
|
||||
"get_spec",
|
||||
"get_status",
|
||||
"managed_checkout",
|
||||
"managed_root",
|
||||
"persistent_env_vars",
|
||||
"start_install",
|
||||
"uninstall",
|
||||
]
|
||||
@@ -223,6 +223,60 @@ def extract_segment_refs(
|
||||
return out
|
||||
|
||||
|
||||
def refine_ref_text(ref_audio_path: str, asr_backend, fallback_text: str) -> str:
|
||||
"""Re-transcribe a written reference clip and return that transcript.
|
||||
|
||||
`extract_speaker_clones`/`extract_segment_refs` pair each audio slice with
|
||||
the ASR segment's OWN text field, on the assumption that the segment's
|
||||
timestamps and its transcribed text agree. They routinely don't — Whisper
|
||||
(and friends) frequently drift on segment boundaries: a trailing word
|
||||
audible in `[start, end]` but missing from `text`, or vice versa. When the
|
||||
(ref_audio, ref_text) pair disagrees, zero-shot TTS prompt-priming breaks
|
||||
down and the clone can speak the mismatched reference text itself instead
|
||||
of the target-language text it was given to synthesize (issue #1004).
|
||||
|
||||
Re-transcribing the *actual written clip* guarantees the pair matches by
|
||||
construction — the model doesn't care whether the original ASR text was
|
||||
right, only that ref_text is what's really in ref_audio. `asr_backend` is
|
||||
the caller's already-loaded active backend (duck-typed:
|
||||
`.transcribe(path, word_timestamps=...) -> dict` with a `chunks` list of
|
||||
`{"text": ...}`); the model is already warm, so this costs one more short
|
||||
transcribe call, not a fresh load. Falls back to `fallback_text` — never
|
||||
raises — so a re-transcribe failure is a strict no-op, never a regression
|
||||
from the original (matching) behavior.
|
||||
"""
|
||||
if asr_backend is None:
|
||||
return fallback_text
|
||||
try:
|
||||
result = asr_backend.transcribe(ref_audio_path, word_timestamps=False)
|
||||
text = " ".join(
|
||||
(c.get("text") or "").strip() for c in (result.get("chunks") or [])
|
||||
).strip()
|
||||
return text or fallback_text
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
"speaker_clone: re-transcribe of %s failed, keeping original ref_text: %s",
|
||||
ref_audio_path, e,
|
||||
)
|
||||
return fallback_text
|
||||
|
||||
|
||||
def refine_ref_texts(clones: dict[str, dict], asr_backend) -> dict[str, dict]:
|
||||
"""Apply `refine_ref_text` to every entry's `ref_text` in place.
|
||||
|
||||
Batches the whole dict (per-speaker `clones` from `extract_speaker_clones`
|
||||
or per-segment `seg_clones` from `extract_segment_refs`) into the single
|
||||
executor round-trip the caller submits to the GPU pool, rather than one
|
||||
dispatch per reference. Mutates and returns `clones` for a convenient
|
||||
call-and-reassign at the call site.
|
||||
"""
|
||||
for entry in clones.values():
|
||||
entry["ref_text"] = refine_ref_text(
|
||||
entry["ref_audio"], asr_backend, entry.get("ref_text", "")
|
||||
)
|
||||
return clones
|
||||
|
||||
|
||||
# ── Internals ───────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
|
||||
@@ -469,3 +469,39 @@ def clear_cache() -> None:
|
||||
"""Testing hook — drop the in-process cache."""
|
||||
with _cache_lock:
|
||||
_cache.update(key=None, ts=0.0, report=None)
|
||||
|
||||
|
||||
def clear_temp(temp_root: str | None = None) -> dict:
|
||||
"""Delete the app-owned ``omnivoice*`` entries in the OS temp dir.
|
||||
|
||||
Removes exactly the population ``build_report`` counts as the "temp"
|
||||
category — direct children of ``temp_root`` whose basename starts with
|
||||
``omnivoice`` — so nothing outside OmniVoice's own working files can ever
|
||||
be swept up. Symlinked entries are unlinked, never followed, so a stray
|
||||
``omnivoice*`` link cannot make this delete its target's contents.
|
||||
|
||||
Returns ``{"removed": [basenames], "freed_bytes": int, "errors":
|
||||
[{"path", "error"}]}`` — partial failures (e.g. a file held open by a
|
||||
running job on Windows) are reported per entry instead of aborting.
|
||||
"""
|
||||
temp_root = temp_root if temp_root is not None else tempfile.gettempdir()
|
||||
removed: list[str] = []
|
||||
errors: list[dict] = []
|
||||
freed = 0
|
||||
deadline = time.monotonic() + CATEGORY_TIMEOUT_SECONDS
|
||||
for p in sorted(glob.glob(os.path.join(glob.escape(temp_root), "omnivoice*"))):
|
||||
try:
|
||||
if os.path.islink(p):
|
||||
size = 0
|
||||
os.unlink(p)
|
||||
elif os.path.isfile(p):
|
||||
size = os.path.getsize(p)
|
||||
os.unlink(p)
|
||||
else:
|
||||
size, _complete, _err = _dir_size(p, deadline)
|
||||
shutil.rmtree(p)
|
||||
removed.append(os.path.basename(p))
|
||||
freed += size
|
||||
except OSError as e:
|
||||
errors.append({"path": p, "error": str(e)})
|
||||
return {"removed": removed, "freed_bytes": freed, "errors": errors}
|
||||
|
||||
@@ -0,0 +1,492 @@
|
||||
"""Engine-agnostic text normalization — a conservative pre-pass before TTS.
|
||||
|
||||
Raw user text trips TTS engines: digits, clock times, and title abbreviations
|
||||
mispronounce; zero-width junk and pathological repeat runs cause hallucinations
|
||||
and long dead air. This module cleans text *once*, at the point where each
|
||||
pipeline hands text to an engine (single-shot /generate, dub segments,
|
||||
longform chapters), so every engine benefits equally.
|
||||
|
||||
Design rules (load-bearing):
|
||||
|
||||
* **Conservative.** A false negative (digits left alone) is fine; a false
|
||||
positive (mangled meaning) is not. Anything ambiguous — thousands-grouped
|
||||
numbers ("1,000"), ranges ("3-5"), version strings ("v2", "3.5.1"),
|
||||
leading-zero codes ("007"), 7+-digit IDs — is left unchanged. Roman
|
||||
numerals are out of scope entirely ("I" is a pronoun).
|
||||
* **Idempotent.** ``normalize_text(normalize_text(x)) == normalize_text(x)``:
|
||||
number/abbreviation output contains no digits or matchable tokens and the
|
||||
safety filters are fixed-point by construction, so an accidental second
|
||||
pass through a pipeline is harmless.
|
||||
* **Per-language.** Numbers go through ``num2words`` only for languages it
|
||||
supports (``_NUM2WORDS_LANGS``; the request's ``language`` is a full
|
||||
display name from frontend/src/languages.json or an ISO-ish code — both
|
||||
resolve via :func:`_num2words_lang`). Everything else keeps its digits.
|
||||
Clock times / ordinals / currency are English-only (their spoken form is
|
||||
language-specific); decimals only for locales whose num2words rendering
|
||||
was vetted. CJK scripts pass through the safety filters untouched — no
|
||||
CJK punctuation is stripped and no words are injected into unsegmented
|
||||
text.
|
||||
* **Markup-safe.** The single-bracket grammar (``[voice:…]``, ``[pause …]``,
|
||||
SSML-lite) and inline ``[[…]]`` pronunciation overrides are never touched:
|
||||
the language passes skip every ``[…]`` span (same shape as chunked_tts's
|
||||
``_BRACKET_TAG_RE``), so ``[pause 300ms]`` / ``[rate 0.9]`` stay parseable.
|
||||
|
||||
Ordering vs. the pronunciation dictionary (audited 2026-07-10): normalization
|
||||
runs **BEFORE** ``services.pronunciation.apply_pronunciation`` (and before the
|
||||
audiobook ``apply_lexicon`` overlay). Rationale from the code:
|
||||
|
||||
1. Dictionary respellings are the user's explicit, final say. If
|
||||
normalization ran second it would re-process them — a respelling that
|
||||
deliberately contains digits or an abbreviation must reach the engine
|
||||
verbatim.
|
||||
2. Users already write lexicon entries against display text (the lexicon
|
||||
docstring's own example is ``{"Dr": "Doctor"}``); entries keyed on
|
||||
normalized words keep firing, and the dictionary stays the override for
|
||||
anything the normalizer produced.
|
||||
3. Inline ``[[…]]`` overrides resolve last inside ``apply_pronunciation``
|
||||
(and their bracketed content is masked here), so the user retains a
|
||||
per-occurrence override over any normalizer output.
|
||||
|
||||
Pinned by ``tests/test_text_normalization.py`` (dictionary-order test).
|
||||
|
||||
Gate: prefs key ``text_normalization_enabled`` (default ON) with env override
|
||||
``OMNIVOICE_TEXT_NORMALIZATION`` — the same env-wins contract as
|
||||
``OMNIVOICE_PRONUNCIATION`` ("0"/"false"/"no"/"off" disable).
|
||||
:func:`normalize_for_tts` is the gated entry point every pipeline calls; it
|
||||
never raises — normalization is never allowed to break synthesis.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
from typing import Callable, Optional
|
||||
|
||||
logger = logging.getLogger("omnivoice.text_normalization")
|
||||
|
||||
ENV_VAR = "OMNIVOICE_TEXT_NORMALIZATION"
|
||||
PREF_KEY = "text_normalization_enabled"
|
||||
|
||||
|
||||
# ── Language resolution ───────────────────────────────────────────────────────
|
||||
#
|
||||
# The `language` kwarg across the app is normally a full display name from
|
||||
# frontend/src/languages.json ("English", "German", …) — see
|
||||
# resolve_kokoro_lang_code in services/tts_backend.py — but ISO-ish codes
|
||||
# ("en", "pt-BR") also flow through dub/API callers. Map both to a num2words
|
||||
# locale; anything unmapped keeps its digits (false negatives are fine).
|
||||
|
||||
_FULL_NAME_TO_CODE = {
|
||||
"english": "en",
|
||||
"german": "de",
|
||||
"spanish": "es",
|
||||
"french": "fr",
|
||||
"italian": "it",
|
||||
"portuguese": "pt",
|
||||
"dutch": "nl",
|
||||
"russian": "ru",
|
||||
"ukrainian": "uk",
|
||||
"polish": "pl",
|
||||
"turkish": "tr",
|
||||
"czech": "cs",
|
||||
"danish": "da",
|
||||
"finnish": "fi",
|
||||
"swedish": "sv",
|
||||
"norwegian": "no",
|
||||
"norwegian bokmål": "no",
|
||||
"norwegian nynorsk": "no",
|
||||
"romanian": "ro",
|
||||
"hungarian": "hu",
|
||||
"indonesian": "id",
|
||||
"lithuanian": "lt",
|
||||
"latvian": "lv",
|
||||
"slovenian": "sl",
|
||||
"serbian": "sr",
|
||||
"hebrew": "he",
|
||||
"persian": "fa",
|
||||
"azerbaijani": "az",
|
||||
"vietnamese": "vi",
|
||||
"kazakh": "kz",
|
||||
"standard arabic": "ar",
|
||||
}
|
||||
|
||||
# ISO codes whose num2words locale name differs.
|
||||
_ISO_ALIASES = {"kk": "kz"}
|
||||
|
||||
# Locales verified against the pinned num2words (cardinal + basic rendering).
|
||||
# zh/ja/ko/th are deliberately absent: unsegmented scripts where injecting
|
||||
# space-delimited words is wrong, and their engines read digits natively.
|
||||
_NUM2WORDS_LANGS = frozenset({
|
||||
"en", "de", "es", "fr", "it", "pt", "nl", "ru", "uk", "pl", "tr", "cs",
|
||||
"da", "fi", "sv", "no", "ro", "hu", "id", "lt", "lv", "sl", "sr", "ar",
|
||||
"he", "fa", "az", "vi", "kz",
|
||||
})
|
||||
|
||||
# Locales whose num2words decimal rendering was vetted ("drei Komma fünf",
|
||||
# "три целых пять десятых", …). tr/vi are excluded on purpose: their 0.5
|
||||
# renders as "fifty" (wrong), so decimals keep their digits there.
|
||||
_DECIMAL_LANGS = frozenset({
|
||||
"en", "de", "es", "fr", "it", "pt", "nl", "ru", "uk", "pl", "cs", "da",
|
||||
"no", "sv", "fi", "ro", "hu", "id",
|
||||
})
|
||||
|
||||
# "50%" → "fifty <word>" only where the spoken percent word is unambiguous.
|
||||
_PERCENT_WORD = {
|
||||
"en": "percent",
|
||||
"de": "Prozent",
|
||||
"es": "por ciento",
|
||||
"fr": "pour cent",
|
||||
"it": "per cento",
|
||||
"pt": "por cento",
|
||||
"nl": "procent",
|
||||
}
|
||||
|
||||
_ISO_CODE_RE = re.compile(r"^([a-z]{2,3})(?:[-_]|$)")
|
||||
|
||||
|
||||
def _num2words_lang(language: Optional[str]) -> Optional[str]:
|
||||
"""Resolve a request language (display name or ISO-ish code) to a
|
||||
num2words locale, or ``None`` when digits should be left alone."""
|
||||
if not language:
|
||||
return None
|
||||
s = str(language).strip().lower()
|
||||
if not s or s == "auto":
|
||||
return None
|
||||
code = _FULL_NAME_TO_CODE.get(s)
|
||||
if code:
|
||||
return code
|
||||
m = _ISO_CODE_RE.match(s)
|
||||
if m:
|
||||
c = _ISO_ALIASES.get(m.group(1), m.group(1))
|
||||
if c in _NUM2WORDS_LANGS:
|
||||
return c
|
||||
return None
|
||||
|
||||
|
||||
# ── Universal safety filters (all languages) ─────────────────────────────────
|
||||
|
||||
# Zero-width & bidi controls, C0/C1 controls (except \t \n \r), BOM, U+FFFD.
|
||||
_ZW_CONTROL_RE = re.compile(
|
||||
"[\u0000-\u0008\u000b\u000c\u000e-\u001f\u007f-\u009f"
|
||||
"\u200b-\u200f\u202a-\u202e\u2060-\u2064\ufeff\ufffd]"
|
||||
)
|
||||
|
||||
# A tiny, unambiguous HTML-entity leftover set. `&` is decoded only when
|
||||
# NOT followed by a letter/`#` — so double-encoded junk ("&nbsp;") is left
|
||||
# alone rather than decoded one layer per pass (idempotency).
|
||||
_ENTITIES = {
|
||||
" ": " ",
|
||||
""": '"',
|
||||
"'": "'",
|
||||
"'": "'",
|
||||
"…": "…",
|
||||
"—": "—",
|
||||
"–": "–",
|
||||
}
|
||||
_ENTITY_RE = re.compile(
|
||||
"(?:" + "|".join(re.escape(k) for k in _ENTITIES) + "|&(?![a-zA-Z#]))"
|
||||
)
|
||||
|
||||
# Same ASCII punctuation char repeated more than 3 times → capped at 3
|
||||
# ("!!!!!!!!" / "........." cause dead air and babble). CJK punctuation and
|
||||
# letters are deliberately untouched ("Nooooo" is expressive).
|
||||
_REPEAT_RE = re.compile(r"([!?.,;:~_*#=-])\1{3,}")
|
||||
|
||||
_HSPACE_RE = re.compile(r"[^\S\n]+") # horizontal whitespace runs → one space
|
||||
_NEWLINE_RE = re.compile(r"\n{3,}") # blank-line floods → one blank line
|
||||
|
||||
|
||||
def _safety_filters(text: str) -> str:
|
||||
out = _ZW_CONTROL_RE.sub("", text)
|
||||
out = _ENTITY_RE.sub(lambda m: _ENTITIES.get(m.group(0), "&"), out)
|
||||
out = _REPEAT_RE.sub(lambda m: m.group(1) * 3, out)
|
||||
out = _HSPACE_RE.sub(" ", out)
|
||||
out = _NEWLINE_RE.sub("\n\n", out)
|
||||
return out.strip()
|
||||
|
||||
|
||||
# ── Bracket masking ──────────────────────────────────────────────────────────
|
||||
#
|
||||
# Language passes must never rewrite `[…]` spans: `[pause 300ms]` /
|
||||
# `[rate 0.9]` / `[voice:NAME]` are grammar, and `[[term|replacement]]`
|
||||
# belongs to the pronunciation layer. Bounded repetition keeps it linear.
|
||||
|
||||
_BRACKET_SPAN_RE = re.compile(r"\[[^\][\n]{0,128}\]")
|
||||
|
||||
|
||||
def _outside_brackets(text: str, fn: Callable[[str], str]) -> str:
|
||||
if "[" not in text:
|
||||
return fn(text)
|
||||
parts: list[str] = []
|
||||
last = 0
|
||||
for m in _BRACKET_SPAN_RE.finditer(text):
|
||||
parts.append(fn(text[last:m.start()]))
|
||||
parts.append(m.group(0))
|
||||
last = m.end()
|
||||
parts.append(fn(text[last:]))
|
||||
return "".join(parts)
|
||||
|
||||
|
||||
# ── Abbreviation expansion ────────────────────────────────────────────────────
|
||||
#
|
||||
# Per-language (key, expansion, guard) triples. Matching is case-sensitive
|
||||
# (a lowercase "st." is NOT the title "St."); lowercase connective keys
|
||||
# ("e.g.") get an auto-added sentence-initial variant. Guards:
|
||||
# "cap" — only before a capitalized word (titles precede names; leaves
|
||||
# street-suffix "Elm St." / "Elm Dr." untouched).
|
||||
# "digit" — only before a number ("No. 5"; leaves the word "No." alone).
|
||||
|
||||
_ABBREVIATIONS: dict[str, list[tuple[str, str, Optional[str]]]] = {
|
||||
"en": [
|
||||
("Dr.", "Doctor", "cap"),
|
||||
("Mr.", "Mister", "cap"),
|
||||
("Mrs.", "Missus", "cap"),
|
||||
("Prof.", "Professor", "cap"),
|
||||
("St.", "Saint", "cap"),
|
||||
("Mt.", "Mount", "cap"),
|
||||
("Jr.", "Junior", None),
|
||||
("Sr.", "Senior", None),
|
||||
("vs.", "versus", None),
|
||||
("etc.", "et cetera", None),
|
||||
("e.g.", "for example", None),
|
||||
("i.e.", "that is", None),
|
||||
("approx.", "approximately", None),
|
||||
("No.", "number", "digit"),
|
||||
],
|
||||
"de": [
|
||||
("Dr.", "Doktor", "cap"),
|
||||
("Prof.", "Professor", "cap"),
|
||||
("Nr.", "Nummer", "digit"),
|
||||
("z.B.", "zum Beispiel", None),
|
||||
("z. B.", "zum Beispiel", None),
|
||||
("d.h.", "das heißt", None),
|
||||
("d. h.", "das heißt", None),
|
||||
("usw.", "und so weiter", None),
|
||||
("bzw.", "beziehungsweise", None),
|
||||
("ca.", "circa", None),
|
||||
],
|
||||
"es": [
|
||||
("Sr.", "Señor", "cap"),
|
||||
("Sra.", "Señora", "cap"),
|
||||
("Srta.", "Señorita", "cap"),
|
||||
("Dr.", "Doctor", "cap"),
|
||||
("Dra.", "Doctora", "cap"),
|
||||
("Ud.", "usted", None),
|
||||
("Uds.", "ustedes", None),
|
||||
("etc.", "etcétera", None),
|
||||
("núm.", "número", "digit"),
|
||||
],
|
||||
"fr": [
|
||||
# "M." is deliberately absent: indistinguishable from a middle initial.
|
||||
("Mme", "Madame", "cap"),
|
||||
("Mmes", "Mesdames", "cap"),
|
||||
("Mlle", "Mademoiselle", "cap"),
|
||||
("Mlles", "Mesdemoiselles", "cap"),
|
||||
("etc.", "et cetera", None),
|
||||
("n°", "numéro", "digit"),
|
||||
("N°", "Numéro", "digit"),
|
||||
],
|
||||
}
|
||||
|
||||
_GUARD_LOOKAHEAD = {
|
||||
None: "",
|
||||
"cap": r"(?=\s+[A-ZÀ-ÖØ-Þ])",
|
||||
"digit": r"(?=\s*\d)",
|
||||
}
|
||||
|
||||
|
||||
def _compile_abbreviations() -> dict[str, tuple[re.Pattern, dict[str, str]]]:
|
||||
compiled: dict[str, tuple[re.Pattern, dict[str, str]]] = {}
|
||||
for lang, entries in _ABBREVIATIONS.items():
|
||||
entries = list(entries)
|
||||
# Sentence-initial variants for lowercase connectives ("E.g." → …).
|
||||
for key, expansion, guard in list(entries):
|
||||
if key[:1].islower():
|
||||
cap_key = key[0].upper() + key[1:]
|
||||
if not any(k == cap_key for k, _, _ in entries):
|
||||
entries.append((cap_key, expansion[0].upper() + expansion[1:], guard))
|
||||
entries.sort(key=lambda e: len(e[0]), reverse=True) # longest key wins
|
||||
lookup = {key: expansion for key, expansion, _ in entries}
|
||||
alts = []
|
||||
for key, _, guard in entries:
|
||||
suffix = r"(?!\w)" if key[-1:].isalnum() else ""
|
||||
alts.append(f"{re.escape(key)}{suffix}{_GUARD_LOOKAHEAD[guard]}")
|
||||
# Literal alternation with per-key guards; no nested quantifiers.
|
||||
pattern = re.compile(r"(?<![\w.])(?:" + "|".join(alts) + ")")
|
||||
compiled[lang] = (pattern, lookup)
|
||||
return compiled
|
||||
|
||||
|
||||
_ABBREV_COMPILED = _compile_abbreviations()
|
||||
|
||||
|
||||
def _expand_abbreviations(text: str, lang: str) -> str:
|
||||
entry = _ABBREV_COMPILED.get(lang)
|
||||
if entry is None:
|
||||
return text
|
||||
pattern, lookup = entry
|
||||
|
||||
def _repl(m: re.Match) -> str:
|
||||
return lookup.get(m.group(0), m.group(0))
|
||||
|
||||
return pattern.sub(_repl, text)
|
||||
|
||||
|
||||
# ── Numbers → words ──────────────────────────────────────────────────────────
|
||||
#
|
||||
# Every pattern requires clean word boundaries: digits glued to letters
|
||||
# ("MP3", "v2"), separators ("1,000", "3-5", "1/2", "12:34:56"), leading
|
||||
# zeros ("007") or 7+ digits (IDs, phone numbers) are all left alone.
|
||||
|
||||
# EN-only clock time: H:MM, 0-23 hours. Rejects H:MM:SS (durations).
|
||||
_TIME_RE = re.compile(r"(?<![\d:.,])([01]?\d|2[0-3]):([0-5]\d)(?![\d:])")
|
||||
|
||||
# EN-only ordinal, suffix verified in the callback ("2th" stays as-is).
|
||||
_ORDINAL_RE = re.compile(r"(?<![\w.,])(\d{1,4})(st|nd|rd|th)\b")
|
||||
|
||||
# EN-only dollars: $N or $N.CC. "$1,000" is blocked by the lookahead.
|
||||
_CURRENCY_RE = re.compile(r"(?<!\w)\$(\d{1,6})(?:\.(\d{2}))?(?![\d.,])")
|
||||
|
||||
_PERCENT_RE = re.compile(r"(?<![\w.,])(\d{1,6}(?:\.\d{1,4})?)\s?%")
|
||||
|
||||
_DECIMAL_RE = re.compile(
|
||||
r"(?<![\w.,:/$%-])(\d{1,6})\.(\d{1,6})(?![\w:/%-])(?![.,]\d)"
|
||||
)
|
||||
|
||||
_INTEGER_RE = re.compile(
|
||||
r"(?<![\w.,:/$%-])(?!0\d)(\d{1,6})(?![\w:/%-])(?![.,]\d)"
|
||||
)
|
||||
|
||||
_ORDINAL_SUFFIX = {1: "st", 2: "nd", 3: "rd"}
|
||||
|
||||
|
||||
def _correct_ordinal_suffix(n: int) -> str:
|
||||
if 10 <= n % 100 <= 13:
|
||||
return "th"
|
||||
return _ORDINAL_SUFFIX.get(n % 10, "th")
|
||||
|
||||
|
||||
def _numbers_to_words(text: str, lang: str) -> str:
|
||||
try:
|
||||
from num2words import num2words
|
||||
except ImportError: # pragma: no cover — direct dependency; belt & braces
|
||||
return text
|
||||
|
||||
def _safe(m: re.Match, render: Callable[[re.Match], str]) -> str:
|
||||
# Any num2words hiccup leaves this occurrence untouched.
|
||||
try:
|
||||
return render(m)
|
||||
except Exception: # noqa: BLE001 — conservative: never mangle
|
||||
return m.group(0)
|
||||
|
||||
if lang == "en":
|
||||
def _time(m: re.Match) -> str:
|
||||
h, mm = int(m.group(1)), int(m.group(2))
|
||||
hw = num2words(h, lang="en")
|
||||
if mm == 0:
|
||||
return f"{hw} o'clock"
|
||||
if mm < 10:
|
||||
return f"{hw} oh {num2words(mm, lang='en')}"
|
||||
return f"{hw} {num2words(mm, lang='en')}"
|
||||
|
||||
text = _TIME_RE.sub(lambda m: _safe(m, _time), text)
|
||||
|
||||
def _ordinal(m: re.Match) -> str:
|
||||
n = int(m.group(1))
|
||||
if m.group(2) != _correct_ordinal_suffix(n):
|
||||
return m.group(0)
|
||||
return num2words(n, lang="en", to="ordinal")
|
||||
|
||||
text = _ORDINAL_RE.sub(lambda m: _safe(m, _ordinal), text)
|
||||
|
||||
def _currency(m: re.Match) -> str:
|
||||
dollars = int(m.group(1))
|
||||
if m.group(2) is not None:
|
||||
amount = float(f"{m.group(1)}.{m.group(2)}")
|
||||
return num2words(amount, lang="en", to="currency", currency="USD")
|
||||
unit = "dollar" if dollars == 1 else "dollars"
|
||||
return f"{num2words(dollars, lang='en')} {unit}"
|
||||
|
||||
text = _CURRENCY_RE.sub(lambda m: _safe(m, _currency), text)
|
||||
|
||||
percent_word = _PERCENT_WORD.get(lang)
|
||||
if percent_word:
|
||||
def _percent(m: re.Match) -> str:
|
||||
raw = m.group(1)
|
||||
if "." in raw:
|
||||
if lang not in _DECIMAL_LANGS:
|
||||
return m.group(0)
|
||||
value: object = float(raw)
|
||||
else:
|
||||
value = int(raw)
|
||||
return f"{num2words(value, lang=lang)} {percent_word}"
|
||||
|
||||
text = _PERCENT_RE.sub(lambda m: _safe(m, _percent), text)
|
||||
|
||||
if lang in _DECIMAL_LANGS:
|
||||
def _decimal(m: re.Match) -> str:
|
||||
return num2words(float(f"{m.group(1)}.{m.group(2)}"), lang=lang)
|
||||
|
||||
text = _DECIMAL_RE.sub(lambda m: _safe(m, _decimal), text)
|
||||
|
||||
def _integer(m: re.Match) -> str:
|
||||
raw = m.group(1)
|
||||
n = int(raw)
|
||||
if len(raw) == 4 and 1500 <= n <= 2099:
|
||||
# Bare 4-digit numbers in this range read as years
|
||||
# ("nineteen eighty-four"); fall back to cardinal where the
|
||||
# locale has no year form (sv, vi).
|
||||
try:
|
||||
return num2words(n, lang=lang, to="year")
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
return num2words(n, lang=lang)
|
||||
|
||||
return _INTEGER_RE.sub(lambda m: _safe(m, _integer), text)
|
||||
|
||||
|
||||
# ── Public API ───────────────────────────────────────────────────────────────
|
||||
|
||||
def normalize_text(text: str, language: Optional[str] = None) -> str:
|
||||
"""Pure, idempotent normalization pass (no pref gate — see
|
||||
:func:`normalize_for_tts` for the gated entry point pipelines call)."""
|
||||
if not text:
|
||||
return text or ""
|
||||
out = _safety_filters(text)
|
||||
lang = _num2words_lang(language)
|
||||
if lang:
|
||||
if lang in _ABBREV_COMPILED:
|
||||
out = _outside_brackets(out, lambda t: _expand_abbreviations(t, lang))
|
||||
out = _outside_brackets(out, lambda t: _numbers_to_words(t, lang))
|
||||
return out
|
||||
|
||||
|
||||
def normalization_enabled() -> bool:
|
||||
"""Env wins (power-user override, mirrors OMNIVOICE_PRONUNCIATION);
|
||||
otherwise the ``text_normalization_enabled`` pref, default ON."""
|
||||
env = os.environ.get(ENV_VAR)
|
||||
if env is not None:
|
||||
return env.strip().lower() not in ("0", "false", "no", "off", "")
|
||||
try:
|
||||
from core import prefs
|
||||
return bool(prefs.get(PREF_KEY, True))
|
||||
except Exception: # noqa: BLE001 — prefs unreadable → default ON
|
||||
return True
|
||||
|
||||
|
||||
def normalize_for_tts(text: str, language: Optional[str] = None) -> str:
|
||||
"""Gated + hardened entry point: pref/env toggle, never raises.
|
||||
|
||||
Every TTS pipeline calls this exactly once, at its text→engine choke
|
||||
point, BEFORE the pronunciation dictionary (see module docstring).
|
||||
"""
|
||||
if not text:
|
||||
return text or ""
|
||||
if not normalization_enabled():
|
||||
return text
|
||||
try:
|
||||
return normalize_text(text, language)
|
||||
except Exception: # noqa: BLE001 — normalization must never break synth
|
||||
logger.warning("text normalization failed; using raw text", exc_info=True)
|
||||
return text
|
||||
@@ -30,7 +30,7 @@ logger = logging.getLogger("omnivoice.token_resolver")
|
||||
Source = Literal["app", "env", "hf-cli"]
|
||||
_PRIORITY: tuple[Source, ...] = ("app", "env", "hf-cli")
|
||||
|
||||
_CACHE_TTL_SECONDS = 300.0 # See Open Question #4 — UI "Test now" calls invalidate.
|
||||
_CACHE_TTL_SECONDS = 300.0 # UI "Test now" busts it via GET /hf-token/state?fresh=1.
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -57,7 +57,8 @@ _CACHE_LOCK = threading.Lock()
|
||||
|
||||
def invalidate_cache() -> None:
|
||||
"""Drop the whoami validation cache. Called by the Settings UI "Test now"
|
||||
button (Plan 01-02) and by save/clear API endpoints (Task 3)."""
|
||||
button (GET /api/settings/hf-token/state?fresh=1), by save/clear API
|
||||
endpoints, and by on_401()."""
|
||||
with _CACHE_LOCK:
|
||||
_VALIDATION_CACHE.clear()
|
||||
|
||||
|
||||
@@ -0,0 +1,282 @@
|
||||
"""
|
||||
Two-stage translation quality for the LLM dub engine (provider="openai").
|
||||
|
||||
Stage 1 — auto-glossary. ONE up-front LLM pass over the full transcript
|
||||
extracts a short theme summary plus a source→target terminology map for the
|
||||
target language. The caller merges it with the user's manual glossary
|
||||
(user entries always win) and injects the result into every per-segment
|
||||
translation prompt, so recurring names/terms are rendered the same way in
|
||||
segment 3 and segment 300. The extraction result is cached on the dub job
|
||||
dict (``job["translation_context"][target_lang]``) and rides the existing
|
||||
``job_data`` JSON blob — no schema change; a transcript fingerprint keys the
|
||||
cache so edited segments re-extract.
|
||||
|
||||
Stage 2 — reflect pass. After a segment's direct LLM translation, a
|
||||
critique-then-rewrite step reviews the draft for wordiness / stiff or
|
||||
unnatural register and produces the final natural line. It runs on the SAME
|
||||
client/model the translation used (the dub_translation skill's provider).
|
||||
|
||||
Failure policy for BOTH stages: refinement must never fail a segment. Any
|
||||
error, timeout, empty output, or divergent rewrite silently keeps the direct
|
||||
translation — callers get ``None`` back and move on.
|
||||
|
||||
MT engines (argos/nllb/google/deepl/…) never reach this module: they have no
|
||||
prompts to inject into and no LLM to critique with. The Cinematic/Autofit
|
||||
refine for those engines lives in ``services/translator.py``.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import logging
|
||||
import os
|
||||
from typing import Iterable, Optional
|
||||
|
||||
logger = logging.getLogger("omnivoice.translation_quality")
|
||||
|
||||
# ── Prompts ──────────────────────────────────────────────────────────────────
|
||||
# The context pass runs ONCE per (job, target language, transcript); the
|
||||
# reflect prompts run twice per segment — keep them short, verbosity = wall time.
|
||||
|
||||
_CONTEXT_PROMPT = """\
|
||||
You are a dubbing terminology editor preparing a translation brief. The user
|
||||
gives you the full source-language transcript of one video. Reply in this
|
||||
exact plain-text format (no JSON, no code fences, no commentary):
|
||||
|
||||
THEME: one or two sentences — what the video is about, its register
|
||||
(casual / formal / technical) and audience.
|
||||
TERM: SOURCE || TARGET
|
||||
TERM: SOURCE || TARGET
|
||||
|
||||
TERM lines list proper nouns (people, places, brands, product names) and
|
||||
recurring domain terms that must be translated identically every time, each
|
||||
with your preferred {target_name} rendering. At most {max_terms} TERM lines;
|
||||
fewer is better. Skip one-off words and anything trivially consistent."""
|
||||
|
||||
_REVIEW_PROMPT = """\
|
||||
You are a dubbing script reviewer. The user gives you a source line and its
|
||||
draft {target_name} translation. In 1-2 short sentences, point out where the
|
||||
draft is wordy, stiff, or uses a register nobody would use in spoken
|
||||
dialogue, and whether recurring terms follow the brief. If the draft already
|
||||
sounds natural, say so. Reply ONLY with the critique — no headers, no lists,
|
||||
no code fences."""
|
||||
|
||||
_POLISH_PROMPT = """\
|
||||
You are a dubbing script writer. Rewrite the draft translation using the
|
||||
reviewer's notes so it reads like natural spoken {target_name}. Keep the
|
||||
meaning faithful to the source line, keep required terminology, and never add
|
||||
content that is not in the source. Prefer the same length or shorter than the
|
||||
draft. The output MUST stay in the same language and script as the draft —
|
||||
never switch language or transliterate. Reply ONLY with the final translation
|
||||
— no quotes, no notes, no commentary."""
|
||||
|
||||
|
||||
def _chat(client, model: str, timeout: float, *, system: str, user: str) -> str:
|
||||
"""One-shot chat completion on the caller's client. Raises on failure."""
|
||||
res = client.chat.completions.create(
|
||||
model=model,
|
||||
timeout=timeout,
|
||||
temperature=0.2, # pinned like the direct-translate path — 1.0 drifts
|
||||
messages=[
|
||||
{"role": "system", "content": system},
|
||||
{"role": "user", "content": user},
|
||||
],
|
||||
)
|
||||
return (res.choices[0].message.content or "").strip()
|
||||
|
||||
|
||||
# ── Stage 1: auto-glossary (theme + terminology) ────────────────────────────
|
||||
|
||||
|
||||
def transcript_fingerprint(segment_texts: Iterable[str]) -> str:
|
||||
"""Stable hash of the transcript, so the per-job context cache invalidates
|
||||
when the user edits segments between translate runs."""
|
||||
h = hashlib.sha256()
|
||||
for t in segment_texts:
|
||||
h.update((t or "").strip().encode("utf-8", errors="replace"))
|
||||
h.update(b"\x00")
|
||||
return h.hexdigest()[:16]
|
||||
|
||||
|
||||
def extract_context_sync(
|
||||
client,
|
||||
model: str,
|
||||
timeout: float,
|
||||
*,
|
||||
segment_texts: Iterable[str],
|
||||
source_lang: str,
|
||||
target_lang: str,
|
||||
source_name: Optional[str] = None,
|
||||
target_name: Optional[str] = None,
|
||||
max_terms: int = 30,
|
||||
) -> Optional[dict]:
|
||||
"""One LLM pass over the whole transcript → ``{"theme", "terms"}``.
|
||||
|
||||
``terms`` is ``[{"source", "target"}]``. Returns None on ANY failure or
|
||||
when the response yields neither a theme nor terms — the caller proceeds
|
||||
without context, never errors. Blocking; run in an executor.
|
||||
"""
|
||||
text = "\n".join(t.strip() for t in segment_texts if t and t.strip())
|
||||
if not text:
|
||||
return None
|
||||
# Same cap as the explicit glossary auto-extract endpoint — one shared
|
||||
# knob for "how much transcript may ride a single LLM context call".
|
||||
try:
|
||||
max_chars = int(os.environ.get("OMNIVOICE_GLOSSARY_MAX_CHARS", "12000"))
|
||||
except ValueError:
|
||||
max_chars = 12000
|
||||
if len(text) > max_chars:
|
||||
text = text[:max_chars] + "\n…[truncated]"
|
||||
|
||||
system = _CONTEXT_PROMPT.format(
|
||||
target_name=target_name or target_lang, max_terms=max_terms,
|
||||
)
|
||||
user = (
|
||||
f"Source language: {source_name or source_lang}\n"
|
||||
f"Target language: {target_name or target_lang}\n"
|
||||
f"Transcript:\n{text}"
|
||||
)
|
||||
try:
|
||||
body = _chat(client, model, timeout, system=system, user=user)
|
||||
except Exception as e: # noqa: BLE001 — context is an enhancement, never a gate
|
||||
logger.warning("auto-glossary context pass failed: %s", e)
|
||||
return None
|
||||
|
||||
theme = ""
|
||||
terms: list[dict] = []
|
||||
for line in body.splitlines():
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
upper = line.upper()
|
||||
if upper.startswith("THEME:"):
|
||||
theme = line[len("THEME:"):].strip()
|
||||
continue
|
||||
if upper.startswith("TERM:"):
|
||||
line = line[len("TERM:"):].strip()
|
||||
if "||" not in line:
|
||||
continue
|
||||
parts = [p.strip() for p in line.split("||")]
|
||||
if len(parts) < 2 or not parts[0] or not parts[1]:
|
||||
continue
|
||||
terms.append({"source": parts[0], "target": parts[1]})
|
||||
if len(terms) >= max_terms:
|
||||
break
|
||||
if not theme and not terms:
|
||||
logger.warning("auto-glossary context pass returned nothing parseable")
|
||||
return None
|
||||
return {"theme": theme, "terms": terms}
|
||||
|
||||
|
||||
def merge_glossary(
|
||||
user_terms: Optional[Iterable[dict]],
|
||||
auto_terms: Optional[Iterable[dict]],
|
||||
) -> list[dict]:
|
||||
"""Merge manual + auto glossaries. User entries ALWAYS win: an auto term
|
||||
whose source matches a user source (case-insensitive) is dropped."""
|
||||
merged: list[dict] = []
|
||||
seen: set[str] = set()
|
||||
for entry in user_terms or []:
|
||||
src = (entry.get("source") or "").strip()
|
||||
tgt = (entry.get("target") or "").strip()
|
||||
if not src or not tgt:
|
||||
continue
|
||||
merged.append(entry)
|
||||
seen.add(src.lower())
|
||||
for entry in auto_terms or []:
|
||||
src = (entry.get("source") or "").strip()
|
||||
tgt = (entry.get("target") or "").strip()
|
||||
if not src or not tgt or src.lower() in seen:
|
||||
continue
|
||||
merged.append({"source": src, "target": tgt})
|
||||
seen.add(src.lower())
|
||||
return merged
|
||||
|
||||
|
||||
def context_clause(theme: str, terms: Optional[Iterable[dict]]) -> str:
|
||||
"""Prompt fragment carrying the theme + merged glossary into every
|
||||
per-segment translation prompt. Empty string when there's nothing."""
|
||||
parts: list[str] = []
|
||||
theme = (theme or "").strip()
|
||||
if theme:
|
||||
parts.append(f"Video context: {theme}")
|
||||
lines = []
|
||||
for entry in terms or []:
|
||||
src = (entry.get("source") or "").strip()
|
||||
tgt = (entry.get("target") or "").strip()
|
||||
if not src or not tgt:
|
||||
continue
|
||||
note = (entry.get("note") or "").strip()
|
||||
lines.append(f"- {src} → {tgt}" + (f" (note: {note})" if note else ""))
|
||||
if lines:
|
||||
parts.append(
|
||||
"Terminology — render every occurrence of a source term exactly "
|
||||
"as its target:\n" + "\n".join(lines)
|
||||
)
|
||||
return "\n".join(parts)
|
||||
|
||||
|
||||
# ── Stage 2: reflect pass (critique → rewrite) ──────────────────────────────
|
||||
|
||||
|
||||
def reflect_translation_sync(
|
||||
client,
|
||||
model: str,
|
||||
timeout: float,
|
||||
*,
|
||||
source_text: str,
|
||||
direct_text: str,
|
||||
source_lang: str,
|
||||
target_lang: str,
|
||||
target_name: Optional[str] = None,
|
||||
extra_clause: str = "",
|
||||
) -> Optional[str]:
|
||||
"""Critique-then-rewrite the direct translation of one segment.
|
||||
|
||||
Returns the polished line, or None whenever the direct translation should
|
||||
stand: any LLM failure/timeout, an empty rewrite, or a rewrite that
|
||||
diverged from the draft (wrong script, runaway length, critique echoed
|
||||
back — the shared ``refine_output_ok`` guard). Never raises. Blocking;
|
||||
run in an executor.
|
||||
"""
|
||||
if not direct_text or not direct_text.strip():
|
||||
return None
|
||||
tgt_name = target_name or target_lang
|
||||
|
||||
def _with_clause(base: str) -> str:
|
||||
return base + "\n\n" + extra_clause if extra_clause.strip() else base
|
||||
|
||||
try:
|
||||
review_user = (
|
||||
f"Source ({source_lang}): {source_text}\n"
|
||||
f"Draft translation ({target_lang}): {direct_text}"
|
||||
)
|
||||
critique = _chat(
|
||||
client, model, timeout,
|
||||
system=_with_clause(_REVIEW_PROMPT.format(target_name=tgt_name)),
|
||||
user=review_user,
|
||||
)
|
||||
polish_user = review_user + f"\nReviewer's notes: {critique}"
|
||||
polished = _chat(
|
||||
client, model, timeout,
|
||||
system=_with_clause(_POLISH_PROMPT.format(target_name=tgt_name)),
|
||||
user=polish_user,
|
||||
)
|
||||
except Exception as e: # noqa: BLE001 — refinement must never fail a segment
|
||||
logger.warning("reflect pass failed (%s) — keeping direct translation", e)
|
||||
return None
|
||||
|
||||
polished = (polished or "").strip()
|
||||
if not polished or polished == direct_text:
|
||||
return None
|
||||
# Same divergence guard the Cinematic ADAPT step uses: wrong script,
|
||||
# runaway length, or the critique leaking through as the "translation".
|
||||
from services.translator import refine_output_ok
|
||||
|
||||
ok, reason = refine_output_ok(direct_text, polished, target_lang, critique=critique)
|
||||
if not ok:
|
||||
logger.warning(
|
||||
"reflect pass diverged for %s (%s) — keeping direct translation",
|
||||
target_lang, reason,
|
||||
)
|
||||
return None
|
||||
return polished
|
||||
+269
-10
@@ -6,7 +6,7 @@ A uniform protocol for every TTS engine. Today we ship:
|
||||
• OmniVoiceBackend — wraps the current k2-fsa/OmniVoice model. Zero
|
||||
behaviour change for existing callers.
|
||||
• VoxCPM2Backend — thin stub that raises with a clear install hint
|
||||
until `pip install voxcpm` is present and enabled.
|
||||
until `pip install "voxcpm>=2.0.3"` is present and enabled.
|
||||
|
||||
Callers should use `get_active_tts_backend()` to pick the configured engine
|
||||
instead of importing a specific class. The selection is controlled by the
|
||||
@@ -55,6 +55,27 @@ def _mask_hf_tokens(value):
|
||||
return _HF_TOKEN_MASK_RE.sub(_HF_TOKEN_MASK, value)
|
||||
|
||||
|
||||
def _available_hint(msg) -> Optional[str]:
|
||||
"""Advisory text carried by an *available* engine's ``is_available()``
|
||||
message, or None when the message is a plain readiness echo.
|
||||
|
||||
Convention (established by VoxCPM2's version-floor hint): an engine
|
||||
that is available but wants the user to know something returns
|
||||
``(True, "ready — <advice>")``. This extracts ``<advice>`` so
|
||||
:func:`list_backends` can surface it — previously the whole message
|
||||
was dropped for available rows (``reason`` is None when ok), so
|
||||
upgrade hints never reached the UI. Plain "ready" / "ready (…)"
|
||||
messages yield None. Output is token-masked like ``reason``.
|
||||
"""
|
||||
if not isinstance(msg, str):
|
||||
return None
|
||||
head, sep, advice = msg.partition(" — ")
|
||||
advice = advice.strip()
|
||||
if not sep or not advice or not head.strip().lower().startswith("ready"):
|
||||
return None
|
||||
return _mask_hf_tokens(advice)
|
||||
|
||||
|
||||
# ── HF Hub closed-client recovery (#880) ────────────────────────────────────
|
||||
#
|
||||
# huggingface_hub ≥1.x shares ONE global httpx client across every download.
|
||||
@@ -142,6 +163,26 @@ class TTSBackend(ABC):
|
||||
#: (e.g. "young female, warm tone, British accent") without reference audio.
|
||||
supports_voice_design: bool = False
|
||||
|
||||
def ensure_ready(self) -> None:
|
||||
"""Load model weights now (blocking), so callers can separate the
|
||||
LOAD budget from the GENERATE budget (#1033/#1037 class).
|
||||
|
||||
Every adapter lazily loads inside ``generate()`` via a private
|
||||
``_ensure_loaded()`` — which meant a cold first call spent its whole
|
||||
``OMNIVOICE_GENERATE_TIMEOUT_S`` window (default 300s) downloading /
|
||||
loading weights and got killed with a misleading "too heavy for the
|
||||
available compute" error (measured in the wild on a fresh install:
|
||||
multi-GB checkpoint download, 0% GPU util, #1014). Routes call this
|
||||
first under the model-load budget (``OMNIVOICE_MODEL_LOAD_TIMEOUT``,
|
||||
default 1200s), then start the generate clock on an already-warm
|
||||
engine. Default implementation dispatches to the adapter's own
|
||||
``_ensure_loaded`` when present; engines without lazy state no-op.
|
||||
Must be called on the GPU pool (it's blocking), same as generate.
|
||||
"""
|
||||
loader = getattr(self, "_ensure_loaded", None)
|
||||
if callable(loader):
|
||||
loader()
|
||||
|
||||
#: Whether this engine already emits mastered, studio-grade audio and should
|
||||
#: therefore skip the shared apply_mastering() chain (highpass + Compressor,
|
||||
#: tuned for OmniVoice's 24 kHz output). Studio engines like VoxCPM2 (native
|
||||
@@ -387,9 +428,155 @@ class OmniVoiceBackend(TTSBackend):
|
||||
|
||||
# ── VoxCPM2 adapter (optional, scaffolded) ──────────────────────────────────
|
||||
|
||||
#: Minimum recommended `voxcpm` package version. 2.0.3 fixed an audio-quality
|
||||
#: bug on Apple Silicon (low-precision dtypes on the MPS device produced
|
||||
#: degraded output). A floor, NOT a pin: newer versions are fine, and an
|
||||
#: already-installed older version keeps working — we only surface an upgrade
|
||||
#: hint (is_available reason + load-time warning), never force a reinstall.
|
||||
_VOXCPM_MIN_VERSION = "2.0.3"
|
||||
|
||||
#: Reference-clip cap for VoxCPM2 cloning (seconds). The `voxcpm` package no
|
||||
#: longer trims reference audio internally, so an unbounded user clip would
|
||||
#: condition the model on minutes of audio (slow, and past a point it stops
|
||||
#: helping voice similarity). 30 s is a conservative upper bound.
|
||||
_VOXCPM_REF_MAX_S = 30.0
|
||||
|
||||
#: Silence pad kept around the voiced region when trimming a reference clip —
|
||||
#: a hard cut exactly at the first/last voiced sample clips consonant onsets.
|
||||
_VOXCPM_REF_EDGE_PAD_S = 0.05
|
||||
|
||||
|
||||
def _version_tuple(v: str) -> Optional[tuple[int, ...]]:
|
||||
"""Parse the leading numeric components of a version string ("2.0.3" →
|
||||
(2, 0, 3), "2.1rc1" → (2, 1)). Returns None when nothing numeric parses —
|
||||
callers treat that as 'unknown, assume fine' rather than failing."""
|
||||
parts: list[int] = []
|
||||
for piece in v.split("."):
|
||||
digits = ""
|
||||
for ch in piece:
|
||||
if not ch.isdigit():
|
||||
break
|
||||
digits += ch
|
||||
if not digits:
|
||||
break
|
||||
parts.append(int(digits))
|
||||
return tuple(parts) if parts else None
|
||||
|
||||
|
||||
def _voxcpm_installed_version() -> Optional[str]:
|
||||
"""Installed `voxcpm` dist version, or None when undeterminable
|
||||
(not installed, or importable without package metadata)."""
|
||||
try:
|
||||
from importlib.metadata import version
|
||||
return version("voxcpm")
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _voxcpm_upgrade_hint() -> Optional[str]:
|
||||
"""Actionable upgrade hint when the installed `voxcpm` is older than
|
||||
:data:`_VOXCPM_MIN_VERSION`, else None. Never raises; an unparseable or
|
||||
unknown version yields None (don't nag users we can't be sure about)."""
|
||||
installed = _voxcpm_installed_version()
|
||||
if installed is None:
|
||||
return None
|
||||
have = _version_tuple(installed)
|
||||
want = _version_tuple(_VOXCPM_MIN_VERSION)
|
||||
if have is None or want is None or have >= want:
|
||||
return None
|
||||
return (
|
||||
f"installed voxcpm {installed} is older than {_VOXCPM_MIN_VERSION}, "
|
||||
"which fixed an audio-quality bug on Apple Silicon (low-precision "
|
||||
"dtypes on MPS). The engine still works, but upgrading is "
|
||||
'recommended: pip install --upgrade "voxcpm>=2.0.3"'
|
||||
)
|
||||
|
||||
|
||||
# Prepared-reference cache: (abspath, mtime_ns, size) → prepared path (which
|
||||
# may be the original path itself when no trim/cap applied). Keeps repeat
|
||||
# generations from re-reading + re-writing the same clip, and keeps the temp
|
||||
# dir from filling with one copy per generate() call.
|
||||
_VOXCPM_REF_PREP_CACHE: dict[tuple, str] = {}
|
||||
|
||||
|
||||
def _prepare_voxcpm_ref(path: str) -> str:
|
||||
"""Prepare a cloning reference clip for VoxCPM2.
|
||||
|
||||
The `voxcpm` package used to trim reference audio itself but no longer
|
||||
does — raw user clips reach the model unconditioned. This applies the
|
||||
minimal, conservative preparation the model expects:
|
||||
|
||||
• trim leading/trailing near-silence (amplitude threshold at the same
|
||||
-50 dBFS floor `audio_dsp.normalize_audio` uses, with a small
|
||||
:data:`_VOXCPM_REF_EDGE_PAD_S` pad kept on each side), and
|
||||
• cap the reference at :data:`_VOXCPM_REF_MAX_S` seconds from the
|
||||
trimmed start.
|
||||
|
||||
Returns a path to the prepared WAV. Deliberately non-destructive and
|
||||
fail-open: the ORIGINAL path is returned unchanged when the clip needs no
|
||||
meaningful trim/cap (short clean clips pass through untouched), when the
|
||||
whole clip sits below the silence floor (nothing to anchor a trim on), or
|
||||
when anything at all goes wrong — reference prep must never be the reason
|
||||
a generation fails.
|
||||
"""
|
||||
try:
|
||||
import numpy as np
|
||||
import soundfile as sf
|
||||
|
||||
abspath = os.path.abspath(path)
|
||||
st = os.stat(abspath)
|
||||
cache_key = (abspath, st.st_mtime_ns, st.st_size)
|
||||
cached = _VOXCPM_REF_PREP_CACHE.get(cache_key)
|
||||
if cached is not None and (cached == abspath or os.path.exists(cached)):
|
||||
return cached
|
||||
|
||||
audio, sr = sf.read(abspath, dtype="float32", always_2d=True) # (n, ch)
|
||||
n = audio.shape[0]
|
||||
if n == 0 or sr <= 0:
|
||||
return path
|
||||
|
||||
# Silence floor: -50 dBFS, matching audio_dsp.normalize_audio. A clip
|
||||
# that never rises above it is left alone (fail-open, see docstring).
|
||||
floor = 10 ** (-50.0 / 20.0)
|
||||
envelope = np.abs(audio).max(axis=1)
|
||||
voiced = np.flatnonzero(envelope > floor)
|
||||
if voiced.size == 0:
|
||||
_VOXCPM_REF_PREP_CACHE[cache_key] = abspath
|
||||
return path
|
||||
|
||||
pad = int(_VOXCPM_REF_EDGE_PAD_S * sr)
|
||||
start = max(0, int(voiced[0]) - pad)
|
||||
end = min(n, int(voiced[-1]) + 1 + pad)
|
||||
cap = int(_VOXCPM_REF_MAX_S * sr)
|
||||
end = min(end, start + cap)
|
||||
|
||||
# No-op path: nothing meaningful to cut (>0.1 s total) — hand the
|
||||
# original file to the model byte-identical.
|
||||
if (start + (n - end)) <= int(0.1 * sr):
|
||||
_VOXCPM_REF_PREP_CACHE[cache_key] = abspath
|
||||
return path
|
||||
|
||||
import tempfile
|
||||
fd, prepared = tempfile.mkstemp(prefix="voxcpm_ref_", suffix=".wav")
|
||||
os.close(fd)
|
||||
sf.write(prepared, audio[start:end], sr)
|
||||
_VOXCPM_REF_PREP_CACHE[cache_key] = prepared
|
||||
logger.info(
|
||||
"VoxCPM2: prepared reference clip %s → %s (%.2fs → %.2fs; "
|
||||
"silence trimmed, cap %.0fs)",
|
||||
path, prepared, n / sr, (end - start) / sr, _VOXCPM_REF_MAX_S,
|
||||
)
|
||||
return prepared
|
||||
except Exception as e: # noqa: BLE001 — prep is best-effort by contract
|
||||
logger.warning(
|
||||
"VoxCPM2: reference-clip preparation failed for %s — using the "
|
||||
"raw clip: %s", path, e,
|
||||
)
|
||||
return path
|
||||
|
||||
|
||||
class VoxCPM2Backend(TTSBackend):
|
||||
"""OpenBMB VoxCPM2 wrapper — `pip install voxcpm` required.
|
||||
"""OpenBMB VoxCPM2 wrapper — `pip install "voxcpm>=2.0.3"` required.
|
||||
|
||||
Ships as a scaffold: the class loads and reports unavailability cleanly
|
||||
when the dep isn't installed, so Settings UI can gate the engine selector
|
||||
@@ -417,10 +604,17 @@ class VoxCPM2Backend(TTSBackend):
|
||||
import voxcpm # noqa: F401
|
||||
except ImportError:
|
||||
return False, (
|
||||
"voxcpm package not installed. Install with `pip install voxcpm` "
|
||||
"voxcpm package not installed. Install with "
|
||||
'`pip install "voxcpm>=2.0.3"` '
|
||||
"(requires Python ≥3.10, PyTorch ≥2.5). CUDA ≥12 recommended "
|
||||
"for full speed; MPS (Apple Silicon) and CPU also supported."
|
||||
)
|
||||
# Version FLOOR, not pin: an older install still reports available
|
||||
# (no forced reinstall), but the reason carries the upgrade hint and
|
||||
# _ensure_loaded() logs it at load time.
|
||||
hint = _voxcpm_upgrade_hint()
|
||||
if hint:
|
||||
return True, f"ready — {hint}"
|
||||
return True, "ready"
|
||||
|
||||
@property
|
||||
@@ -442,6 +636,9 @@ class VoxCPM2Backend(TTSBackend):
|
||||
ok, msg = self.is_available()
|
||||
if not ok:
|
||||
raise RuntimeError(f"VoxCPM2 unavailable: {msg}")
|
||||
hint = _voxcpm_upgrade_hint()
|
||||
if hint:
|
||||
logger.warning("VoxCPM2: %s", hint)
|
||||
from voxcpm import VoxCPM # type: ignore[import-not-found]
|
||||
checkpoint = os.environ.get("OMNIVOICE_VOXCPM_MODEL", "openbmb/VoxCPM2")
|
||||
logger.info("Loading VoxCPM2 from %s", checkpoint)
|
||||
@@ -471,14 +668,16 @@ class VoxCPM2Backend(TTSBackend):
|
||||
cfg_value=kw.get("guidance_scale", 2.0),
|
||||
inference_timesteps=kw.get("num_step", 10),
|
||||
)
|
||||
if isinstance(wav, np.ndarray):
|
||||
wav = torch.from_numpy(wav).float()
|
||||
if wav.ndim == 1:
|
||||
wav = wav.unsqueeze(0)
|
||||
return wav
|
||||
return self._finalize(wav)
|
||||
|
||||
# ── Standard clone / instruct mode ──────────────────────────────
|
||||
# Map our instruct prop onto VoxCPM2's inline "(instruct)prompt" prefix.
|
||||
# The reference clip is prepared first (edge-silence trim + length
|
||||
# cap) — the model no longer trims it internally, so a raw user clip
|
||||
# would condition generation on dead air. Fail-open: on any prep
|
||||
# problem the raw path is used, exactly as before.
|
||||
if ref_audio:
|
||||
ref_audio = _prepare_voxcpm_ref(ref_audio)
|
||||
prompt = text
|
||||
if instruct:
|
||||
prompt = f"({instruct}){text}"
|
||||
@@ -490,11 +689,26 @@ class VoxCPM2Backend(TTSBackend):
|
||||
prompt_wav_path=ref_audio if ref_text else None,
|
||||
prompt_text=ref_text,
|
||||
)
|
||||
return self._finalize(wav)
|
||||
|
||||
def _finalize(self, wav) -> torch.Tensor:
|
||||
"""Normalize model output to a (1, n) float tensor and apply the
|
||||
trailing-silence guard.
|
||||
|
||||
The guard is a SILENCE trim only: generations often end with a long
|
||||
near-silent tail, which this cuts (keeping a short ~0.3 s natural
|
||||
tail). It deliberately does NOT attempt to detect or judge trailing
|
||||
*content* — an output that ends in audible audio, wanted or not,
|
||||
passes through unchanged, as does any output without a silent tail.
|
||||
"""
|
||||
import numpy as np
|
||||
from services.audio_dsp import trim_trailing_silence
|
||||
|
||||
if isinstance(wav, np.ndarray):
|
||||
wav = torch.from_numpy(wav).float()
|
||||
if wav.ndim == 1:
|
||||
wav = wav.unsqueeze(0)
|
||||
return wav
|
||||
return trim_trailing_silence(wav, self.sample_rate)
|
||||
|
||||
|
||||
# ── MOSS-TTS-Nano adapter (tiny, CPU-friendly, 20 langs) ────────────────────
|
||||
@@ -849,6 +1063,7 @@ class MLXAudioBackend(TTSBackend):
|
||||
|
||||
voice = kw.get("voice")
|
||||
ref_audio = kw.get("ref_audio")
|
||||
ref_text = kw.get("ref_text")
|
||||
language = kw.get("language")
|
||||
speed = float(kw.get("speed", 1.0))
|
||||
|
||||
@@ -859,6 +1074,12 @@ class MLXAudioBackend(TTSBackend):
|
||||
kwargs = {"text": text, "speed": speed}
|
||||
if voice: kwargs["voice"] = voice
|
||||
if ref_audio: kwargs["ref_audio"] = ref_audio
|
||||
# CSM (sesame.py) only builds its cloning context when BOTH ref_audio
|
||||
# AND ref_text are present — with ref_text missing, its context list
|
||||
# stays empty and indexing into it raises an opaque
|
||||
# "IndexError: list index out of range" deep inside mlx-audio,
|
||||
# instead of ever attempting the clone. Community-diagnosed (#1012).
|
||||
if ref_audio and ref_text: kwargs["ref_text"] = ref_text
|
||||
if language and language != "Auto":
|
||||
if self._model_id == self.CURATED_MODELS.get("kokoro"):
|
||||
# Kokoro's vendored pipeline hard-asserts `lang_code` against
|
||||
@@ -1432,7 +1653,7 @@ _INSTALL_HINTS: dict[str, str] = {
|
||||
"cosyvoice": "git clone --recursive FunAudioLLM/CosyVoice + pip install -r requirements.txt + SoX",
|
||||
"kittentts": "pip install kittentts (ONNX, CPU-only, ~80 MB)",
|
||||
"mlx-audio": "pip install mlx-audio (Apple Silicon only)",
|
||||
"voxcpm2": "pip install voxcpm (CPU/MPS supported; CUDA recommended for speed)",
|
||||
"voxcpm2": 'pip install "voxcpm>=2.0.3" (floor: 2.0.3 fixed Apple-Silicon audio quality; CPU/MPS supported, CUDA recommended for speed)',
|
||||
"moss-tts-nano": "git clone OpenMOSS/MOSS-TTS-Nano && pip install -e . (not on PyPI)",
|
||||
"indextts2": "git clone index-tts/index-tts && uv pip install -e . (NOT uv sync --all-extras)",
|
||||
"gpt-sovits": "External API server — start api_v2.py on port 9880",
|
||||
@@ -1477,6 +1698,23 @@ _MLX_AUDIO_MODEL_LABELS: dict[str, str] = {
|
||||
}
|
||||
|
||||
|
||||
def _sidecar_installable_ids() -> frozenset[str]:
|
||||
"""Engine ids with a one-click sidecar installer. Deferred import — the
|
||||
installer module is tiny, but keeping the import inside the function
|
||||
means a broken/absent installer can never take the engine picker down.
|
||||
|
||||
All current sidecar SPECS are TTS engines, so only this registry carries
|
||||
``one_click_install``; the first non-TTS sidecar engine will need the same
|
||||
field plumbed into asr_backend/llm_backend.list_backends and the Install
|
||||
button into their matrix rows.
|
||||
"""
|
||||
try:
|
||||
from services.sidecar_install import SPECS
|
||||
return frozenset(SPECS)
|
||||
except Exception: # pragma: no cover — defensive only
|
||||
return frozenset()
|
||||
|
||||
|
||||
def list_backends() -> list[dict]:
|
||||
"""Enumerate every registered backend with its availability state.
|
||||
|
||||
@@ -1487,11 +1725,17 @@ def list_backends() -> list[dict]:
|
||||
"display_name": str,
|
||||
"available": bool,
|
||||
"reason": Optional[str], # message when not available
|
||||
"hint": Optional[str], # advice when available-but-has-advice
|
||||
# (is_available "ready — <advice>" convention;
|
||||
# e.g. VoxCPM2's >=2.0.3 upgrade hint)
|
||||
"install_hint": Optional[str],
|
||||
"setup_snippet": Optional[str], # exact `export VAR=...` for path-gated opt-in engines
|
||||
"one_click_install": bool, # services.sidecar_install can provision it in-app
|
||||
"last_error": Optional[str], # cached most-recent failure
|
||||
"isolation_mode": "in-process" | "subprocess",
|
||||
"gpu_compat": list[str], # subset of {cuda, rocm, mps, xpu, cpu}
|
||||
"supports_cloning": Optional[bool], # True/False from the class attr; None when
|
||||
# model-dependent (property, e.g. mlx-audio)
|
||||
"effective_device": str, # device this engine uses on THIS host
|
||||
"routing_status": "accelerated" | "cpu_fallback" | "cpu_only" | "unavailable",
|
||||
"routing_reason": Optional[str], # scrubbed; null when none
|
||||
@@ -1519,6 +1763,7 @@ def list_backends() -> list[dict]:
|
||||
from core.device_caps import detect_host_caps
|
||||
from services.engine_routing import routing_fields
|
||||
caps = detect_host_caps()
|
||||
installable = _sidecar_installable_ids()
|
||||
|
||||
out: list[dict] = []
|
||||
for bid, cls in _REGISTRY.items():
|
||||
@@ -1546,14 +1791,28 @@ def list_backends() -> list[dict]:
|
||||
else:
|
||||
isolation = "in-process"
|
||||
gpu_compat = getattr(cls, "gpu_compat", ("cpu",))
|
||||
# Cloning capability: same descriptor guard as
|
||||
# cloning_capable_engine_ids() — a class-level getattr on a *property*
|
||||
# (mlx-audio: capability depends on the picked model) returns the
|
||||
# descriptor, not a bool, so report None (= model-dependent) there
|
||||
# instead of an always-truthy false positive.
|
||||
_clone = getattr(cls, "supports_cloning", True)
|
||||
out.append({
|
||||
"id": bid,
|
||||
"display_name": cls.display_name,
|
||||
"available": ok,
|
||||
"reason": None if ok else _mask_hf_tokens(msg),
|
||||
# Available-but-has-advice (e.g. VoxCPM2's ">=2.0.3 recommended"
|
||||
# upgrade hint). None unless ok and the message carries advice.
|
||||
"hint": _available_hint(msg) if ok else None,
|
||||
"supports_cloning": _clone if isinstance(_clone, bool) else None,
|
||||
"install_hint": _INSTALL_HINTS.get(bid),
|
||||
# Exact `export VAR=...` line for path-gated opt-in engines, or None.
|
||||
"setup_snippet": _SETUP_SNIPPETS.get(bid),
|
||||
# True when services.sidecar_install can provision this engine
|
||||
# in-app (Settings renders an Install button instead of leading
|
||||
# with the manual setup snippet).
|
||||
"one_click_install": bid in installable,
|
||||
"last_error": _LAST_ERRORS.get(bid),
|
||||
"isolation_mode": isolation,
|
||||
"gpu_compat": list(gpu_compat),
|
||||
|
||||
@@ -39,6 +39,27 @@ _audioseal_available: Optional[bool] = None
|
||||
# This is our signature — every OmniVoice-generated audio carries it.
|
||||
OMNI_MESSAGE = [0, 1, 0, 0, 1, 1, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1]
|
||||
|
||||
# Watermark ops run chunk-by-chunk: AudioSeal's activation memory grows
|
||||
# linearly with input length — a single multi-minute waveform demands a
|
||||
# multi-GB CPU buffer, which OOM'd a 16 GB machine mid-generate (#1045).
|
||||
# 30 s bounds each call to tens of MB; the 16-bit message repeats throughout
|
||||
# the audio, so per-chunk embedding/detection is equivalent.
|
||||
_CHUNK_SECONDS = 30
|
||||
|
||||
|
||||
def _iter_chunks(audio: torch.Tensor, sample_rate: int):
|
||||
"""Yield ≤ ~_CHUNK_SECONDS slices of (batch, channels, samples) audio
|
||||
along the time axis. A sub-second tail is folded into the previous chunk
|
||||
(AudioSeal embeds poorly on very short segments)."""
|
||||
total = audio.shape[-1]
|
||||
step = _CHUNK_SECONDS * sample_rate
|
||||
starts = list(range(0, total, step))
|
||||
if len(starts) > 1 and total - starts[-1] < sample_rate:
|
||||
starts.pop()
|
||||
for i, start in enumerate(starts):
|
||||
end = starts[i + 1] if i + 1 < len(starts) else total
|
||||
yield audio[..., start:end]
|
||||
|
||||
|
||||
def _check_available() -> bool:
|
||||
"""Check if AudioSeal is installed and importable."""
|
||||
@@ -135,7 +156,13 @@ def embed_watermark(
|
||||
|
||||
# AudioSeal operates at 16kHz internally; it handles resampling, but
|
||||
# we need to inform it of the source rate for correct embedding.
|
||||
watermarked = generator(audio, sample_rate=sample_rate, message=msg)
|
||||
watermarked = torch.cat(
|
||||
[
|
||||
generator(seg, sample_rate=sample_rate, message=msg)
|
||||
for seg in _iter_chunks(audio, sample_rate)
|
||||
],
|
||||
dim=-1,
|
||||
)
|
||||
|
||||
# Restore original shape
|
||||
if len(original_shape) == 2:
|
||||
@@ -189,11 +216,17 @@ def detect_watermark(
|
||||
else:
|
||||
audio = waveform
|
||||
|
||||
result = detector.detect_watermark(audio, sample_rate=sample_rate, message_threshold=0.5)
|
||||
|
||||
# result is (detection_confidence, decoded_message)
|
||||
confidence = float(result[0]) if isinstance(result, tuple) else 0.0
|
||||
decoded_msg = result[1] if isinstance(result, tuple) and len(result) > 1 else None
|
||||
# Detect per chunk and keep the best hit: bounds memory the same way
|
||||
# embedding does, and a splice where only part of the file is
|
||||
# OmniVoice audio still registers (a whole-file average would dilute it).
|
||||
best_conf, decoded_msg = -1.0, None
|
||||
for seg in _iter_chunks(audio, sample_rate):
|
||||
result = detector.detect_watermark(seg, sample_rate=sample_rate, message_threshold=0.5)
|
||||
seg_conf = float(result[0]) if isinstance(result, tuple) else 0.0
|
||||
if seg_conf > best_conf:
|
||||
best_conf = seg_conf
|
||||
decoded_msg = result[1] if isinstance(result, tuple) and len(result) > 1 else None
|
||||
confidence = max(best_conf, 0.0)
|
||||
|
||||
# Decode message bits
|
||||
message_bits = ""
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
"""Shared setup for backend/tests — import path + hermetic data dir.
|
||||
|
||||
Historically every module in this directory stubbed
|
||||
``sys.modules["core.config"]`` with a bare 3-4 attribute ``ModuleType``
|
||||
pointing at its own ``mkdtemp``. That stub leaked **process-wide at
|
||||
collection time**: pytest imports test modules while collecting, so in any
|
||||
mixed invocation (``pytest tests/... backend/tests/...``) every *later* lazy
|
||||
import of ``core.config`` resolved the stub instead of the real module —
|
||||
``tests/test_router_smoke.py``'s ``from main import app`` died with
|
||||
ImportError (missing config attrs), and
|
||||
``monkeypatch.setattr("core.config.X", ...)`` died with AttributeError
|
||||
(``core`` never gets a ``config`` attribute when the name is satisfied
|
||||
straight from ``sys.modules``). That was the root cause of the
|
||||
order-pollution combos around test_longform_e2e (8 AttributeErrors) and
|
||||
test_router_smoke (24 fixture ImportErrors).
|
||||
|
||||
The real ``core.config`` derives every path from ``OMNIVOICE_DATA_DIR`` at
|
||||
import time, so pointing that env var at a throwaway dir *before* any test
|
||||
module imports it gives the same hermeticity (issue #878: never touch the
|
||||
developer's real app state) with zero ``sys.modules`` surgery. This mirrors
|
||||
``tests/conftest.py``; in a mixed run whichever conftest loads first wins
|
||||
(``setdefault`` semantics) and both point at a throwaway tmpdir.
|
||||
|
||||
Do NOT reintroduce module-level ``sys.modules`` stubs in this directory —
|
||||
import the real module and rely on this conftest instead.
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
|
||||
# Backend runs with `--app-dir backend`, so tests must do the same.
|
||||
_BACKEND = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
|
||||
if _BACKEND not in sys.path:
|
||||
sys.path.insert(0, _BACKEND)
|
||||
|
||||
if not os.environ.get("OMNIVOICE_DATA_DIR"):
|
||||
os.environ["OMNIVOICE_DATA_DIR"] = tempfile.mkdtemp(prefix="omnivoice-test-data-")
|
||||
if not os.environ.get("OMNIVOICE_ENV_FILE"):
|
||||
os.environ["OMNIVOICE_ENV_FILE"] = os.path.join(
|
||||
os.environ["OMNIVOICE_DATA_DIR"], "user-env"
|
||||
)
|
||||
@@ -14,24 +14,13 @@ verified manually (spectral flatness back in the speech range + Whisper ASR).
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import types
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
|
||||
|
||||
# Stub core.config before the router imports OUTPUTS_DIR / VOICES_DIR from it.
|
||||
_TMP = tempfile.mkdtemp(prefix="omnivoice_preview_q_")
|
||||
_config = types.ModuleType("core.config")
|
||||
_config.DATA_DIR = _TMP
|
||||
_config.VOICES_DIR = str(Path(_TMP) / "voices")
|
||||
_config.OUTPUTS_DIR = str(Path(_TMP) / "outputs")
|
||||
sys.modules["core.config"] = _config
|
||||
|
||||
# conftest.py puts `backend/` on sys.path and points OMNIVOICE_DATA_DIR at a
|
||||
# throwaway tmpdir before the router imports OUTPUTS_DIR / VOICES_DIR from
|
||||
# the REAL core.config (the old sys.modules stub leaked at collection time
|
||||
# and broke later importers in mixed runs — see conftest.py).
|
||||
torch = pytest.importorskip("torch") # noqa: E402
|
||||
|
||||
from api.routers import archetypes as arch # noqa: E402
|
||||
|
||||
@@ -9,29 +9,13 @@ generation.py's proven ``_run_inference`` rather than re-implementing it.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import types
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
|
||||
|
||||
# Stub core.config before the router imports VOICES_DIR / OUTPUTS_DIR from it.
|
||||
_TMP = tempfile.mkdtemp(prefix="omnivoice_arch_test_")
|
||||
_VOICES = Path(_TMP) / "voices"
|
||||
_OUTPUTS = Path(_TMP) / "outputs"
|
||||
_VOICES.mkdir(parents=True, exist_ok=True)
|
||||
_OUTPUTS.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
_config = types.ModuleType("core.config")
|
||||
_config.DATA_DIR = _TMP
|
||||
_config.VOICES_DIR = str(_VOICES)
|
||||
_config.OUTPUTS_DIR = str(_OUTPUTS)
|
||||
sys.modules["core.config"] = _config
|
||||
|
||||
# conftest.py puts `backend/` on sys.path and points OMNIVOICE_DATA_DIR at a
|
||||
# throwaway tmpdir before the router imports VOICES_DIR / OUTPUTS_DIR from
|
||||
# the REAL core.config (the old sys.modules stub leaked at collection time).
|
||||
from fastapi import FastAPI # noqa: E402
|
||||
from fastapi.testclient import TestClient # noqa: E402
|
||||
|
||||
|
||||
@@ -8,21 +8,14 @@ OOM deterministically (no GPU needed) and asserts the device switch.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import types
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
|
||||
|
||||
_config = types.ModuleType("core.config")
|
||||
_config.DATA_DIR = tempfile.mkdtemp(prefix="omnivoice_asr_oom_")
|
||||
_config.VOICES_DIR = _config.DATA_DIR
|
||||
_config.OUTPUTS_DIR = _config.DATA_DIR
|
||||
sys.modules["core.config"] = _config
|
||||
|
||||
# conftest.py puts `backend/` on sys.path and points OMNIVOICE_DATA_DIR at a
|
||||
# throwaway tmpdir before this module imports the REAL core.config (the old
|
||||
# sys.modules stub leaked at collection time and broke mixed runs).
|
||||
whisperx = pytest.importorskip("whisperx")
|
||||
|
||||
from services.asr_backend import ( # noqa: E402
|
||||
|
||||
@@ -3,32 +3,18 @@
|
||||
is covered by the manifest round-trip (tests/test_longform_resume.py) + the
|
||||
existing render tests, not here.
|
||||
|
||||
Config-stub pattern (mounts only the audiobook router — torch-free at import) so
|
||||
it runs locally without the main+torch segfault.
|
||||
Mounts only the audiobook router (no ``main`` import) so it runs locally
|
||||
without the main+torch segfault; conftest.py provides the hermetic data dir.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import types
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
|
||||
|
||||
_TMP = tempfile.mkdtemp(prefix="omnivoice_abresume_test_")
|
||||
_config = types.ModuleType("core.config")
|
||||
_config.DATA_DIR = _TMP
|
||||
_config.VOICES_DIR = str(Path(_TMP) / "voices")
|
||||
_config.OUTPUTS_DIR = str(Path(_TMP) / "outputs")
|
||||
_config.DB_PATH = str(Path(_TMP) / "omnivoice.db")
|
||||
os.makedirs(_config.VOICES_DIR, exist_ok=True)
|
||||
os.makedirs(_config.OUTPUTS_DIR, exist_ok=True)
|
||||
sys.modules["core.config"] = _config
|
||||
|
||||
# conftest.py puts `backend/` on sys.path and points OMNIVOICE_DATA_DIR at a
|
||||
# throwaway tmpdir before this module imports the REAL core.config (the old
|
||||
# sys.modules stub leaked at collection time and broke mixed runs).
|
||||
from fastapi import FastAPI # noqa: E402
|
||||
from fastapi.testclient import TestClient # noqa: E402
|
||||
|
||||
|
||||
@@ -6,19 +6,11 @@ lightweight — it only imports os, uuid, time, asyncio, logging,
|
||||
fastapi, and pydantic at module level.
|
||||
"""
|
||||
import io
|
||||
import os
|
||||
import sys
|
||||
import pytest
|
||||
|
||||
# Add backend to path
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
|
||||
|
||||
# Stub core.config before batch imports it
|
||||
import types
|
||||
config_mod = types.ModuleType("core.config")
|
||||
config_mod.DATA_DIR = "/tmp/omnivoice_test_data"
|
||||
sys.modules["core.config"] = config_mod
|
||||
|
||||
# conftest.py puts `backend/` on sys.path and points OMNIVOICE_DATA_DIR at a
|
||||
# throwaway tmpdir before the batch router imports the REAL core.config (the
|
||||
# old sys.modules stub leaked at collection time and broke mixed runs).
|
||||
from fastapi import FastAPI
|
||||
from fastapi.testclient import TestClient
|
||||
from api.routers.batch import router, _jobs, _set_progress
|
||||
|
||||
@@ -4,17 +4,11 @@ Only tests the pure-Python helper functions (no GPU needed).
|
||||
The WebSocket endpoint itself requires the full app, which we
|
||||
skip in CI — it's integration-tested via the browser.
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
|
||||
|
||||
# Stub heavy deps
|
||||
import types
|
||||
for mod_name in ["services.model_manager", "services.asr_backend", "services.ffmpeg_utils"]:
|
||||
if mod_name not in sys.modules:
|
||||
sys.modules[mod_name] = types.ModuleType(mod_name)
|
||||
|
||||
# conftest.py puts `backend/` on sys.path. capture_ws imports its heavy deps
|
||||
# (model_manager / asr_backend / ffmpeg_utils) lazily inside handlers, so no
|
||||
# stubbing is needed — the old empty-ModuleType stubs leaked process-wide at
|
||||
# collection time and broke every later `from services.ffmpeg_utils import
|
||||
# find_ffmpeg` in mixed runs (see conftest.py).
|
||||
from api.routers.capture_ws import _chunks_to_wav, MIN_BUFFER_BYTES
|
||||
|
||||
|
||||
|
||||
@@ -9,23 +9,12 @@ are exercised at runtime.
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import types
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
|
||||
|
||||
_TMP = tempfile.mkdtemp(prefix="omnivoice_community_test_")
|
||||
_config = types.ModuleType("core.config")
|
||||
_config.DATA_DIR = _TMP
|
||||
_config.VOICES_DIR = str(Path(_TMP) / "voices")
|
||||
_config.OUTPUTS_DIR = str(Path(_TMP) / "outputs")
|
||||
sys.modules["core.config"] = _config
|
||||
|
||||
# conftest.py puts `backend/` on sys.path and points OMNIVOICE_DATA_DIR at a
|
||||
# throwaway tmpdir before this module imports the REAL core.config (the old
|
||||
# sys.modules stub leaked at collection time and broke mixed runs).
|
||||
from fastapi import FastAPI # noqa: E402
|
||||
from fastapi.testclient import TestClient # noqa: E402
|
||||
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
"""Regression guard for the collection-time sys.modules stub class.
|
||||
|
||||
Seven modules in this directory used to install bare ``types.ModuleType``
|
||||
stubs for ``core.config`` (and test_capture_ws.py for ``services.*``) at
|
||||
module level. pytest imports every test module during *collection*, so the
|
||||
stubs leaked process-wide before a single test ran: in any mixed invocation
|
||||
(``pytest tests/... backend/tests/...``) later lazy imports resolved the
|
||||
stub — tests/test_router_smoke.py's ``from main import app`` died with
|
||||
``ImportError: cannot import name 'find_ffmpeg' from 'services.ffmpeg_utils'
|
||||
(unknown location)`` and tests/test_longform_e2e.py's
|
||||
``monkeypatch.setattr("core.config.OUTPUTS_DIR", ...)`` died with
|
||||
``AttributeError: module 'core' has no attribute 'config'``.
|
||||
|
||||
This test runs after collection has imported every sibling module, so any
|
||||
reintroduced module-level stub trips it even in a backend/tests-only run.
|
||||
A stub is recognizable because a bare ModuleType has neither ``__file__``
|
||||
(real module) nor ``__path__`` (namespace package).
|
||||
|
||||
If you need a fake module in a test, use ``monkeypatch.setitem(sys.modules,
|
||||
name, fake)`` inside the test — pytest restores it. For hermetic data dirs,
|
||||
rely on conftest.py's ``OMNIVOICE_DATA_DIR`` redirect instead of stubbing
|
||||
``core.config``.
|
||||
"""
|
||||
import sys
|
||||
|
||||
# Backend packages whose identity later tests depend on. Top-level third-party
|
||||
# modules are out of scope (some legitimately lack __file__, e.g. frozen ones).
|
||||
_GUARDED_PREFIXES = ("core.", "api.", "services.", "schemas.", "utils.")
|
||||
|
||||
|
||||
def test_no_collection_time_sys_modules_stubs():
|
||||
offenders = []
|
||||
for name, mod in list(sys.modules.items()):
|
||||
if mod is None or not name.startswith(_GUARDED_PREFIXES):
|
||||
continue
|
||||
if getattr(mod, "__file__", None) is None and not hasattr(mod, "__path__"):
|
||||
offenders.append(name)
|
||||
assert not offenders, (
|
||||
"Stub module(s) found in sys.modules: "
|
||||
f"{offenders}. Some test module installed a bare ModuleType at import "
|
||||
"time; that leaks process-wide from pytest collection and breaks "
|
||||
"every later import of the real module in mixed runs. Use "
|
||||
"monkeypatch.setitem(sys.modules, ...) inside the test instead."
|
||||
)
|
||||
@@ -6,34 +6,23 @@ the model. The export path's preview generation needs torchaudio and is covered
|
||||
by the service-layer round-trip in ``tests/test_persona_bundle.py`` + CI; here we
|
||||
only assert export's 404 (which fails before any audio work).
|
||||
|
||||
Follows the config-stub pattern of ``test_archetypes_api.py`` / ``test_community.py``
|
||||
so it mounts ONLY the persona router on a bare FastAPI app (no ``main`` import,
|
||||
no torch at collection).
|
||||
Follows the pattern of ``test_archetypes_api.py`` / ``test_community.py``:
|
||||
mounts ONLY the persona router on a bare FastAPI app (no ``main`` import,
|
||||
no torch at collection); conftest.py provides the hermetic data dir.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import types
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
|
||||
|
||||
_TMP = tempfile.mkdtemp(prefix="omnivoice_personas_test_")
|
||||
_VOICES = Path(_TMP) / "voices"
|
||||
_VOICES.mkdir(parents=True, exist_ok=True)
|
||||
_config = types.ModuleType("core.config")
|
||||
_config.DATA_DIR = _TMP
|
||||
_config.VOICES_DIR = str(_VOICES)
|
||||
_config.OUTPUTS_DIR = str(Path(_TMP) / "outputs")
|
||||
_config.DB_PATH = str(Path(_TMP) / "omnivoice.db")
|
||||
sys.modules["core.config"] = _config
|
||||
# conftest.py puts `backend/` on sys.path and points OMNIVOICE_DATA_DIR at a
|
||||
# throwaway tmpdir before this module imports the REAL core.config (the old
|
||||
# sys.modules stub leaked at collection time and broke mixed runs).
|
||||
from core import config as _config # noqa: E402
|
||||
|
||||
from fastapi import FastAPI # noqa: E402
|
||||
from fastapi.testclient import TestClient # noqa: E402
|
||||
|
||||
@@ -58,6 +58,10 @@ RUN uv pip install --system --no-cache .
|
||||
# Copy application source
|
||||
COPY backend/ ./backend/
|
||||
COPY omnivoice/ ./omnivoice/
|
||||
# Alembic config so schema migrations run natively on existing volumes
|
||||
# (without it the backend fell back to the additive-column self-heal —
|
||||
# functional, but the real migration chain is the first-class path).
|
||||
COPY alembic.ini ./
|
||||
|
||||
# Copy the pre-built React frontend from the builder stage
|
||||
COPY --from=frontend-builder /app/frontend/dist ./frontend/dist
|
||||
@@ -65,6 +69,12 @@ COPY --from=frontend-builder /app/frontend/dist ./frontend/dist
|
||||
# Expose the single unified API and UI port
|
||||
EXPOSE 3900
|
||||
|
||||
# Image-level health probe (compose files define their own; this covers plain
|
||||
# `docker run`). Generous start period: first boot creates the venv-less
|
||||
# schema + may pull model metadata before /health answers.
|
||||
HEALTHCHECK --interval=30s --timeout=5s --start-period=120s --retries=5 \
|
||||
CMD curl -fsS http://127.0.0.1:3900/health || exit 1
|
||||
|
||||
# Mount points for persistent data (sqlite db, user voices, huggingface cache)
|
||||
VOLUME ["/app/omnivoice_data"]
|
||||
|
||||
|
||||
@@ -76,18 +76,33 @@ default** (set its var to `0` to disable); the rest default **off**.
|
||||
|
||||
## Restricted networks / mirrors (e.g. China)
|
||||
|
||||
If `huggingface.co` is slow or blocked, point the client at a mirror:
|
||||
**Automatic (the default).** When no endpoint is explicitly configured,
|
||||
OmniVoice picks one for you: it probes `huggingface.co` and the community
|
||||
mirror `hf-mirror.com` in parallel (short HTTPS reachability + latency
|
||||
checks — no geo-IP lookups, no third-party services; your device
|
||||
language/timezone only decides which endpoint is probed *first*), prefers the
|
||||
official endpoint unless the mirror is decisively faster, and remembers the
|
||||
winner. The decision is re-tested only on the first-run system check, after a
|
||||
network-classified download failure (the failed download retries once on the
|
||||
new winner), when it's more than 7 days old, or when you press **Test again**
|
||||
in **Settings → Models → Hugging Face mirror**. Mirror integrity is a
|
||||
non-issue: `huggingface_hub` verifies every download by checksum regardless
|
||||
of endpoint. Opt out with `OMNIVOICE_HF_ENDPOINT_MODE=manual`.
|
||||
|
||||
**Explicit (always wins).** To pin an endpoint yourself:
|
||||
|
||||
```
|
||||
HF_ENDPOINT=https://hf-mirror.com
|
||||
```
|
||||
|
||||
Set it in **Settings → Models → Hugging Face mirror** (quick-pick presets
|
||||
included), or as an environment variable before launching. On first run, the
|
||||
setup wizard offers the same mirror quick-pick right on the system-check
|
||||
screen when the endpoint is unreachable — the network check is a warning, not
|
||||
a blocker, so an offline or firewalled machine can still finish setup once
|
||||
models are available (mirror, or manual download below). Caveats:
|
||||
Set it in **Settings → Models → Hugging Face mirror** (quick-pick presets and
|
||||
a custom URL — any explicit choice switches the panel to manual mode and is
|
||||
**never** auto-switched), or as an environment variable before launching. On
|
||||
first run, the setup wizard's network check reports which endpoint the
|
||||
automatic selection picked, and still offers the mirror quick-pick when
|
||||
nothing is reachable — the check is a warning, not a blocker, so an offline
|
||||
or firewalled machine can still finish setup once models are available
|
||||
(mirror, or manual download below). Caveats:
|
||||
|
||||
- A mirror serves the **classic** download path, **not Xet** — you lose
|
||||
chunk-dedup and Xet's parallel fetch, but you gain reachability. On the
|
||||
|
||||
@@ -86,6 +86,46 @@ translation is produced:
|
||||
Cinematic and Autofit **require an LLM** (below). If none is configured, they
|
||||
fall back to Fast with a notice.
|
||||
|
||||
## Two-stage quality on the LLM engine (auto-glossary + reflect pass)
|
||||
|
||||
When the **LLM (OpenAI-compatible)** engine is the active translator, two extra
|
||||
quality stages run by default. Both have checkboxes next to the Quality control
|
||||
in the Dub tab's translation settings (they only appear for the LLM engine —
|
||||
MT engines can't run either stage):
|
||||
|
||||
- **Auto glossary** — before the per-segment translation, ONE extra LLM pass
|
||||
reads the whole transcript and extracts a short theme summary plus a
|
||||
source → target terminology map. That brief rides every segment's translation
|
||||
prompt, so character names, places, and recurring domain terms come out the
|
||||
same in segment 3 and segment 300. It's merged with your manual glossary —
|
||||
**your entries always win** on a clashing term. The result is cached with the
|
||||
dub project per target language, so re-translating an unchanged transcript
|
||||
costs zero extra calls; editing segments re-extracts.
|
||||
- **Reflect pass** — after each segment's direct translation, the LLM critiques
|
||||
the draft for wordiness and stiff/unnatural register, then rewrites it as
|
||||
natural spoken dialogue. **This uses 3 LLM calls per segment instead of 1** —
|
||||
turn it off for long videos on slow or metered providers. If any refinement
|
||||
step fails or times out, the direct translation is kept silently; refinement
|
||||
can never fail a segment.
|
||||
### Fit prediction (all quality levels)
|
||||
|
||||
Every translation additionally gets a **pre-synthesis fit check** — no LLM
|
||||
needed. For each segment, OmniVoice predicts how long the translated line will
|
||||
take to speak (self-calibrating to your voice/engine from segments already
|
||||
generated in the job, with a per-language rate table as the cold-start
|
||||
fallback) and compares it against the slot plus the silence it can borrow
|
||||
before the next line. Segments the Smart Fit caps can only absorb with an
|
||||
audible speed-up get a **Tight fit** badge; segments no fitting can save get a
|
||||
**Won't fit +Ns** badge — so you can shorten the text *before* burning GPU
|
||||
time on a line that would end up trimmed. Badges are informational only:
|
||||
generation is never blocked.
|
||||
|
||||
**Suggest shorter lines** (checkbox under Quality, off by default) goes one
|
||||
step further: for every "Won't fit" segment it asks the configured LLM for a
|
||||
meaning-preserving shorter rewrite and offers it on the row as a one-click
|
||||
**Use shorter rewrite** suggestion. It never rewrites anything automatically,
|
||||
and with no LLM configured (or on any LLM error) it simply does nothing.
|
||||
|
||||
## LLM Providers (for Cinematic / Autofit)
|
||||
|
||||
**Settings → System → LLM Providers** is the one place to set up the LLM. Pick a
|
||||
|
||||
+72
-18
@@ -8,12 +8,57 @@ pins `transformers>=5.3`. This isolation is the resolution of
|
||||
canonical `OffloadedCache` ImportError that resulted from loading
|
||||
both libraries inside one Python interpreter.
|
||||
|
||||
## Install
|
||||
## Install (one-click, recommended)
|
||||
|
||||
IndexTTS-2 is **not** bundled with OmniVoice — the model weights are
|
||||
~6 GB and the package itself pins a conflicting transformers
|
||||
version. OmniVoice ships with a sidecar runner that loads IndexTTS
|
||||
into an isolated venv on demand.
|
||||
into an isolated venv on demand, plus a guided installer that
|
||||
provisions everything for you:
|
||||
|
||||
1. Open **Settings → Engines**, expand the IndexTTS2 row
|
||||
("Why unavailable?"), and click **Install**.
|
||||
2. Watch the step-by-step progress: preflight (uv + disk space),
|
||||
source fetch, isolated venv creation, dependency install,
|
||||
verification, model-weight download (~6 GB, resumable), and
|
||||
configuration save.
|
||||
3. Done — the engine flips to `available: true` immediately, **no
|
||||
restart needed**.
|
||||
|
||||
What the installer does under the hood (all cross-platform —
|
||||
macOS / Windows / Linux):
|
||||
|
||||
* Fetches the IndexTTS source with `git clone --depth 1` (or, when
|
||||
git isn't installed, downloads the GitHub source tarball over
|
||||
HTTPS) into OmniVoice's data directory
|
||||
(`<data-dir>/engines/indextts2/index-tts`).
|
||||
* Creates a dedicated venv inside the checkout with `uv venv` and
|
||||
runs `uv pip install -e .` against it — the `transformers<5`
|
||||
isolation is preserved; the parent app's environment is never
|
||||
touched. uv is resolved from `OMNIVOICE_BUNDLED_UV`, then `PATH`.
|
||||
* Downloads the `IndexTeam/IndexTTS-2` weights into
|
||||
`checkpoints/` (where the sidecar loads them from), honouring your
|
||||
configured/auto-selected Hugging Face endpoint and HF token.
|
||||
* Persists `OMNIVOICE_INDEXTTS_DIR` for you (in-process for
|
||||
immediate use + `prefs.json` for the next launch).
|
||||
|
||||
Preflight requires roughly **12 GB free disk space** (source + venv +
|
||||
weights, checked before anything is written); the install fails early
|
||||
with an actionable message otherwise. Re-running the installer is
|
||||
always safe: it repairs partial installs and resumes interrupted
|
||||
downloads instead of starting over. An app-managed install can be
|
||||
removed again with `DELETE /engines/sidecar/indextts2/install` (a
|
||||
user-managed clone is never touched).
|
||||
|
||||
If you already installed IndexTTS manually (any OmniVoice version),
|
||||
the installer detects it via `OMNIVOICE_INDEXTTS_DIR` and reports
|
||||
`already_installed` — nothing is re-downloaded or moved.
|
||||
|
||||
## Manual install (fallback)
|
||||
|
||||
The manual flow still works and is what the installer automates. Use
|
||||
it if you want the clone somewhere specific, share one clone across
|
||||
tools, or can't use the in-app installer:
|
||||
|
||||
1. Clone the IndexTTS repo on disk:
|
||||
|
||||
@@ -32,16 +77,12 @@ into an isolated venv on demand.
|
||||
uv pip install -e .
|
||||
```
|
||||
|
||||
3. Download the model weights (~6 GB). Either:
|
||||
3. Download the model weights (~6 GB):
|
||||
|
||||
```bash
|
||||
hf download IndexTeam/IndexTTS-2 --local-dir=checkpoints
|
||||
```
|
||||
|
||||
or let HuggingFace cache them on first synthesize call (the parent
|
||||
forwards `HF_HOME` / `HF_HUB_CACHE` to the sidecar so the cache is
|
||||
shared with the rest of OmniVoice's downloads).
|
||||
|
||||
4. Set the `OMNIVOICE_INDEXTTS_DIR` environment variable to the repo
|
||||
root (the directory that contains `checkpoints/` and
|
||||
`pyproject.toml`):
|
||||
@@ -65,12 +106,14 @@ into an isolated venv on demand.
|
||||
OmniVoice probes for a usable IndexTTS Python interpreter in this
|
||||
priority order (see `backend/engines/indextts/bootstrap.py`):
|
||||
|
||||
1. **`${OMNIVOICE_INDEXTTS_DIR}/.venv/`** — your existing clone's
|
||||
venv. Highest priority, so v0.2.7 users who already ran
|
||||
`uv pip install -e .` get zero migration cost on the upgrade to
|
||||
v0.3.x.
|
||||
1. **`${OMNIVOICE_INDEXTTS_DIR}/.venv/`** — the install dir's own
|
||||
venv. This is what BOTH the one-click installer (which sets
|
||||
`OMNIVOICE_INDEXTTS_DIR` to its managed checkout) and a manual
|
||||
clone resolve to. Highest priority, so v0.2.7 users who already
|
||||
ran `uv pip install -e .` get zero migration cost on the upgrade
|
||||
to v0.3.x.
|
||||
2. **`backend/engines/indextts/.venv/`** — OmniVoice's own venv,
|
||||
created on demand by step 3.
|
||||
created on demand by the lazy bootstrap below.
|
||||
3. **Lazy bootstrap** — if neither venv exists, OmniVoice runs
|
||||
`uv venv backend/engines/indextts/.venv` and
|
||||
`uv pip install --python <python> -e ${OMNIVOICE_INDEXTTS_DIR}`
|
||||
@@ -87,14 +130,25 @@ weights survive the upgrade byte-for-byte.
|
||||
|
||||
### `IndexTTS-2 venv not found. Set OMNIVOICE_INDEXTTS_DIR ...`
|
||||
|
||||
You haven't pointed OmniVoice at an IndexTTS clone yet. Follow the
|
||||
**Install** steps above.
|
||||
You haven't installed IndexTTS yet. Click **Install** on the
|
||||
IndexTTS2 row in **Settings → Engines** (recommended), or follow the
|
||||
**Manual install** steps above.
|
||||
|
||||
### `uv is required to bootstrap the IndexTTS-2 venv but was not found on PATH`
|
||||
### `uv was not found` / `uv is required to bootstrap the IndexTTS-2 venv but was not found on PATH`
|
||||
|
||||
The bootstrap path needs a working `uv` binary. Either install `uv`
|
||||
into your `PATH` (https://docs.astral.sh/uv/) or pre-create the venv
|
||||
manually with `uv venv` and `uv pip install -e` as in step 2.
|
||||
Both the one-click installer and the bootstrap path need a working
|
||||
`uv` binary (resolved from `OMNIVOICE_BUNDLED_UV`, then `PATH`).
|
||||
Either install `uv` into your `PATH` (https://docs.astral.sh/uv/) or
|
||||
pre-create the venv manually with `uv venv` and `uv pip install -e`
|
||||
as in the manual steps.
|
||||
|
||||
### `Not enough disk space to install IndexTTS-2 ...`
|
||||
|
||||
The installer's preflight found less free space than the estimated
|
||||
source + venv + weights footprint (plus headroom). The message names
|
||||
the exact numbers; free up space (or move OmniVoice's data directory
|
||||
to a larger volume) and click Install again — it resumes where it
|
||||
stopped.
|
||||
|
||||
### `IndexTTS bootstrap completed but `import indextts.infer_v2` still fails`
|
||||
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
# OmniVoice Studio — OpenAI-Compatible Remote ASR
|
||||
|
||||
A path to Qwen3-ASR, a self-hosted FunASR/SenseVoice server, or OpenAI's own
|
||||
Whisper API — today, without waiting on `transformers` to ship a direct
|
||||
Qwen3-ASR integration (tracked separately). Unlike every other ASR engine,
|
||||
this one runs no model locally: it's a pure network client that calls any
|
||||
server exposing an OpenAI-compatible `POST /v1/audio/transcriptions`
|
||||
endpoint.
|
||||
|
||||
## Setup
|
||||
|
||||
No install step — configure it directly:
|
||||
|
||||
1. Open **Settings → Models** and find **OpenAI-compatible ASR (remote
|
||||
server)**.
|
||||
2. Set **Server URL** to your server's base URL (e.g.
|
||||
`http://localhost:8000/v1` for a local Qwen3-ASR/FunASR server, or
|
||||
`https://api.openai.com/v1` for OpenAI's own API).
|
||||
3. Set **Model** to whatever your server expects (`whisper-1` for OpenAI's
|
||||
API; check your self-hosted server's docs otherwise).
|
||||
4. **API key** is optional — many self-hosted servers accept requests
|
||||
without one. Set it if your server requires auth, or if you're using
|
||||
OpenAI's own API.
|
||||
5. Activate the engine in **Settings → Engines** — click **Use** on
|
||||
**OpenAI-compatible ASR** in the ASR Engines table (the same picker TTS
|
||||
engines have). Power users can pin it instead by setting
|
||||
`OMNIVOICE_ASR_BACKEND=openai-compat-asr` before launching — the env var
|
||||
always wins over the Settings pick.
|
||||
|
||||
## Response format
|
||||
|
||||
The backend prefers `response_format=verbose_json` for real per-segment
|
||||
timestamps (OpenAI's API and most compatible servers support it) and falls
|
||||
back to plain text automatically if your server rejects that format. Neither
|
||||
path returns word-level timestamps — that's not part of this API.
|
||||
|
||||
## Privacy note
|
||||
|
||||
Unlike every other ASR engine in OmniVoice, audio sent through this backend
|
||||
leaves your machine — to whatever server you configured. If that's a
|
||||
self-hosted server on your own network, nothing leaves your control; if
|
||||
it's a third-party API (OpenAI's, or someone else's), review their data
|
||||
handling before sending anything sensitive.
|
||||
@@ -74,6 +74,8 @@ asr_engines:
|
||||
readme: FunASR
|
||||
- id: sherpa-onnx-asr
|
||||
readme: "**sherpa-onnx** (live dictation)"
|
||||
- id: openai-compat-asr
|
||||
readme: "**OpenAI-compatible** ⚠️ remote"
|
||||
|
||||
# Doc files that must exist (the install path users are sent to).
|
||||
docs:
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
# Verified Tesla T4 (16GB) inference notes
|
||||
|
||||
Measured on a real NVIDIA Tesla T4 (16GB, Turing/sm_75), driver 550.163.01 (CUDA 12.8), torch
|
||||
2.8.0+cu128, transformers 5.3.0, Python 3.11.15 (uv-managed). Engine under test: the default
|
||||
`omnivoice` TTS backend (`OMNIVOICE_TTS_BACKEND=omnivoice`).
|
||||
|
||||
## Cold-cache first call can time out at 300s
|
||||
|
||||
The first `generate()` call lazily downloads the ~2.3GB `k2-fsa/OmniVoice` checkpoint, and that
|
||||
download happens *inside* the `OMNIVOICE_GENERATE_TIMEOUT_S` budget (default 300s). On a fresh
|
||||
install, the very first `POST /v1/audio/speech` can fail like this even though the GPU isn't
|
||||
actually short on memory:
|
||||
|
||||
```
|
||||
ERROR [omnivoice.openai_compat] OpenAI TTS failed: OpenAI TTS generate exceeded 300s and was
|
||||
abandoned — the backend is running, but the job was too heavy for the available compute.
|
||||
... most often the GPU is VRAM-starved ...
|
||||
```
|
||||
|
||||
VRAM sampling during the failure showed a flat ~2GB with 0% GPU utilization for the whole 300s —
|
||||
consistent with waiting on a download, not compute. Once the checkpoint is cached, the identical
|
||||
request succeeds in ~1s (reproduced 5x: 1.574s / 1.034s / 1.065s / 0.995s / 0.911s).
|
||||
|
||||
**Workaround (no code change needed, both already exist):**
|
||||
- For headless/API-only setups, pre-fetch the checkpoint before your first real TTS request:
|
||||
```bash
|
||||
curl -X POST http://localhost:3900/models/install \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"repo_id": "k2-fsa/OmniVoice"}'
|
||||
```
|
||||
(`repo_id` is required — `InstallModelRequest` in `backend/api/schemas.py` rejects a bare/empty
|
||||
body — and must match one of the entries in `KNOWN_MODELS`, e.g. the default engine's
|
||||
`k2-fsa/OmniVoice`.) Progress streams over the existing `/setup/download-stream` SSE feed.
|
||||
- Or raise `OMNIVOICE_GENERATE_TIMEOUT_S` for the first request.
|
||||
|
||||
## OpenAI-compatible endpoint doesn't expose `num_step` / `guidance_scale`
|
||||
|
||||
`POST /v1/audio/speech`'s request schema doesn't declare `num_step` or `guidance_scale` fields —
|
||||
sending them in the JSON body returns `200 OK` but they're silently discarded (pydantic's default
|
||||
`extra=ignore` behavior). The native multipart `POST /generate` endpoint *does* expose both as
|
||||
explicit form fields, so use that endpoint if you need to control them.
|
||||
|
||||
Separately: the app's own default for `num_step` is 16 — half of the model's documented default of
|
||||
32 (see `docs/generation-parameters.md`, "Use 16 for faster inference"). Not a bug, just not stated
|
||||
that the app already runs the "fast" preset unless you override it via `/generate`.
|
||||
|
||||
## T4 acceleration checklist
|
||||
|
||||
| Option | Status |
|
||||
|---|---|
|
||||
| dtype | `torch.float16` hardcoded for the `omnivoice` engine (`model_manager.py`) — correct for Turing (no bf16 tensor cores this generation). No env var override for this engine specifically (ASR engines have `ASR_COMPUTE_TYPE`; `dots_tts`/`indextts` have their own precision vars; `omnivoice` doesn't). |
|
||||
| Attention | `sdpa`, selected automatically since `flash_attn` isn't installed (`_supports_flash_attn_2=True` is declared but the package itself is absent) — safe on T4. |
|
||||
| int8 | No int8 path for this engine (ASR's CTranslate2 `int8` and `sherpa-onnx`'s int8 ONNX models are separate/unrelated). |
|
||||
| CUDA Graphs | No direct API usage in the app. Reachable indirectly via `torch.compile(mode="reduce-overhead")`, which the app attempts **by default** on this GPU (T4/sm_75 isn't in the framework's compile-exclusion list, unlike newer/Blackwell GPUs). The numbers above were measured with `TORCH_COMPILE_DISABLE=1` for a clean eager baseline. |
|
||||
| torch.compile | Attempted by default on T4 (see above) — not evaluated further here. |
|
||||
|
||||
## VRAM
|
||||
|
||||
Peak measured: 2487 MiB (`nvidia-smi`) / 2.050 GB (`torch.cuda.max_memory_allocated()`) for the
|
||||
default `omnivoice` engine — comfortably fits even the README's stated "minimum" (4GB) tier.
|
||||
@@ -13,7 +13,7 @@ and [`palashdeb/omnivoice-studio` on Docker Hub](https://hub.docker.com/r/palash
|
||||
> |-----|--------------|
|
||||
> | `:latest` | **Rolling preview** — latest commit on `main` (always one patch ahead of the last release). This is the preview channel; pin `:stable` for production. |
|
||||
> | `:stable` | Most recent versioned release (updated on every `v*` git tag) |
|
||||
> | `:0.3.6` | Exact release version |
|
||||
> | `:0.3.17` | Exact release version |
|
||||
> | `:0.3` | Latest patch within the 0.3 minor |
|
||||
> | `:main` | Alias of the same rolling `main` build as `:latest` |
|
||||
> | `:sha-xxxxxxx` | Specific commit (produced by manual workflow dispatch) |
|
||||
|
||||
@@ -12,13 +12,11 @@ working OmniVoice Studio install on a Debian / Ubuntu / Fedora / Arch host.
|
||||
- **~10 GB free disk** for the app, its Python environment, and model weights.
|
||||
- Optional: an **NVIDIA driver** for CUDA GPU acceleration — the app runs
|
||||
CPU-only without one. For AMD GPUs see [AMD GPU (ROCm)](#amd-gpu-rocm).
|
||||
- Optional: **yt-dlp** for downloading YouTube/video clips directly in the
|
||||
Voice Gallery and Dub tabs — `sudo apt install yt-dlp` (Debian/Ubuntu),
|
||||
`sudo dnf install yt-dlp` (Fedora), or `sudo pacman -S yt-dlp` (Arch).
|
||||
Without it those downloads fail; everything else works fine.
|
||||
|
||||
That's it — Python, FFmpeg, and the model weights are bundled or bootstrapped
|
||||
by the app itself on first launch. No toolchain needed.
|
||||
That's it — Python, FFmpeg/FFprobe, yt-dlp, and the model weights are bundled
|
||||
or bootstrapped by the app itself on first launch. No toolchain needed. (If no
|
||||
FFmpeg resolves anywhere, the app downloads its own checksummed static build
|
||||
in the background during setup; **Settings → Audio tools** shows exactly which
|
||||
binaries are in use and lets you override them or update yt-dlp.)
|
||||
|
||||
### Building from source
|
||||
|
||||
@@ -29,7 +27,6 @@ Everything above, plus the toolchain:
|
||||
- **Python 3.11+** — typically `sudo apt install python3.11` on Debian/Ubuntu,
|
||||
`sudo dnf install python3.11` on Fedora, or already installed on Arch.
|
||||
- **Bun** — `curl -fsSL https://bun.sh/install | bash`.
|
||||
- **FFmpeg** — `sudo apt install ffmpeg` (Debian/Ubuntu), `sudo dnf install ffmpeg-free` (Fedora), or `sudo pacman -S ffmpeg` (Arch).
|
||||
- **Rust / Cargo** — `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh` or via your package manager (e.g., `sudo apt install rustc cargo`).
|
||||
If you use rustup, reopen the shell or source `"$HOME/.cargo/env"` before running `bun run desktop-prod`.
|
||||
- **GTK/WebKit deps** for the Tauri shell:
|
||||
|
||||
@@ -35,10 +35,15 @@ Everything above, plus the toolchain:
|
||||
and the C toolchain; `curl` ships with macOS).
|
||||
- **Python 3.11+** — `brew install python@3.11` (or use `pyenv` / the system Python if you already have ≥3.11).
|
||||
- **Bun** — `curl -fsSL https://bun.sh/install | bash`.
|
||||
- **FFmpeg** (used by the dubbing + capture pipelines) — `brew install ffmpeg`.
|
||||
- **Rust / Cargo** — `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh` or `brew install rust`.
|
||||
If you use rustup, reopen the terminal or source `"$HOME/.cargo/env"` before running `bun run desktop-prod`.
|
||||
|
||||
FFmpeg/FFprobe and yt-dlp are **not** prerequisites on any install path: the
|
||||
app resolves them itself (a static build ships with the Python environment;
|
||||
if nothing resolves, the app downloads its own checksummed build on first
|
||||
run). Power users can inspect or override the binaries in
|
||||
**Settings → Audio tools** — including pointing at a Homebrew copy.
|
||||
|
||||
Optional but recommended:
|
||||
|
||||
- **A Hugging Face account** for diarization and the larger TTS models. See
|
||||
|
||||
@@ -167,6 +167,32 @@ that rely on `/usr/bin/ffprobe`.
|
||||
|
||||
**Fix:** see [linux.md#deb-ffprobe-conflict](linux.md#deb-ffprobe-conflict).
|
||||
|
||||
## 7b. "Media engine unavailable" / FFmpeg questions
|
||||
|
||||
FFmpeg, FFprobe, and yt-dlp are **not** things you install for OmniVoice.
|
||||
The app resolves them itself, in order: a path provided by the desktop shell →
|
||||
the static build shipped with the Python environment → the app's own
|
||||
downloaded build → whatever is on your PATH. When nothing resolves at all
|
||||
(some source installs on a fresh machine), the Setup Wizard downloads a
|
||||
pinned, checksum-verified static build in the background — you'll see a
|
||||
one-line "Preparing media engine…" progress and, only if that download fails,
|
||||
a card with **Retry** and **Use a system copy**.
|
||||
|
||||
If a running install ever reports "Media engine unavailable":
|
||||
|
||||
1. Open **Settings → Audio tools**. Each row shows the binary actually in use
|
||||
(version, path, and origin — Bundled / System / Custom).
|
||||
2. Press **Restore bundled** to re-fetch the app's own build (needs network
|
||||
once), or **Use system copy** / **Choose file…** to point at an FFmpeg you
|
||||
already have. Installing via a package manager (`brew install ffmpeg`,
|
||||
`sudo apt install ffmpeg`, `winget install ffmpeg`) also works — press
|
||||
**Use system copy** afterwards.
|
||||
|
||||
The same panel updates **yt-dlp** (video imports): site support changes
|
||||
faster than app releases, so when video-URL imports start failing, press
|
||||
**Update** there — the new version survives app updates, and **Restore tested
|
||||
version** reverts to the build the app shipped with.
|
||||
|
||||
## 8. Docker LAN access — media preview 404
|
||||
|
||||
**Symptom:** OmniVoice loads on `http://<lan-ip>:3900` but the audio preview
|
||||
@@ -323,10 +349,16 @@ order:
|
||||
files are a common false-positive quarantine), then re-enable.
|
||||
- **Connection** — use a stable, direct connection; pause any VPN; avoid
|
||||
corporate/school networks.
|
||||
- **Region mirror** — if `huggingface.co` is slow/blocked where you are, pick a
|
||||
mirror in-app (**Settings → Models → Hugging Face mirror**, or the quick-pick
|
||||
the first-run system check offers when the endpoint is unreachable), or set
|
||||
it as an env var before launching and relaunch:
|
||||
- **Region mirror** — if `huggingface.co` is slow/blocked where you are,
|
||||
OmniVoice normally handles this automatically: with no endpoint explicitly
|
||||
configured it probes both the official endpoint and the `hf-mirror.com`
|
||||
community mirror and downloads from whichever works (downloads are
|
||||
checksum-verified either way; see
|
||||
[downloading-models.md](../downloading-models.md)). To check or re-test the
|
||||
automatic pick, use **Settings → Models → Hugging Face mirror → Test
|
||||
again**. To pin a mirror yourself, pick one in the same panel (or the
|
||||
quick-pick the first-run system check offers when nothing is reachable), or
|
||||
set it as an env var before launching and relaunch:
|
||||
- macOS/Linux: `export HF_ENDPOINT=https://hf-mirror.com`
|
||||
- Windows (PowerShell): `[Environment]::SetEnvironmentVariable("HF_ENDPOINT","https://hf-mirror.com","User")`
|
||||
|
||||
@@ -445,6 +477,41 @@ quit OmniVoice Studio, delete the folder below, then start the app again.
|
||||
Remove-Item -Recurse -Force "$env:LOCALAPPDATA\com.debpalash.omnivoice-studio\EBWebView"
|
||||
```
|
||||
|
||||
## 16. macOS: microphone permission never prompts, OmniVoice never appears in System Settings
|
||||
|
||||
**Symptom:** clicking record shows "Microphone access denied. macOS: open
|
||||
System Settings → Privacy & Security → Microphone and enable OmniVoice" —
|
||||
but OmniVoice never appears in that list, so there's nothing to enable.
|
||||
`NSMicrophoneUsageDescription` is present in the app's `Info.plist`, and
|
||||
resetting the permission (`tccutil reset Microphone
|
||||
com.debpalash.omnivoice-studio`) followed by a relaunch changes nothing — no
|
||||
system prompt ever appears.
|
||||
|
||||
**Cause:** the app bundle was missing the Hardened Runtime *entitlement* for
|
||||
microphone access. An earlier revision of this section blamed an upstream
|
||||
Tauri/WebKit limitation — that was wrong (a community contributor,
|
||||
[@MahdiHedhli](https://github.com/MahdiHedhli), read the sources more
|
||||
carefully and found the real gap). wry's `WKUIDelegate` already grants the
|
||||
WebKit-layer media-capture request; but Tauri's macOS bundler enables
|
||||
Hardened Runtime by default, and Hardened Runtime blocks microphone hardware
|
||||
access unless `com.apple.security.device.audio-input` is present in the
|
||||
signed binary's entitlements — regardless of `Info.plist`'s
|
||||
`NSMicrophoneUsageDescription` (that only supplies the prompt *text*).
|
||||
Without the entitlement, macOS's TCC layer never registers a request, which
|
||||
is exactly why the app never appears in the System Settings list.
|
||||
|
||||
**Fix:** ships in the release after v0.3.12 (the bundle now carries
|
||||
`src-tauri/entitlements.plist` — [#1016](https://github.com/debpalash/OmniVoice-Studio/pull/1016),
|
||||
contributed by the same person who diagnosed it). Update and live recording
|
||||
works, with a normal macOS permission prompt on first use.
|
||||
|
||||
**Workaround on older builds (≤ v0.3.12):** record your voice sample in any
|
||||
other app (Voice Memos, QuickTime, etc.) and upload the resulting file in
|
||||
OmniVoice instead of using live recording — upload-based cloning is
|
||||
unaffected and works normally.
|
||||
|
||||
**Linked issue:** [#1013](https://github.com/debpalash/OmniVoice-Studio/issues/1013)
|
||||
|
||||
## Dub: "translation engine needs the optional … package"
|
||||
|
||||
**Symptom:** in the Dub tab, translating fails with e.g. *"The 'google'
|
||||
|
||||
@@ -77,6 +77,28 @@ Download the latest MSI from the
|
||||
run it, follow the wizard. The shortcut lands in the Start menu as
|
||||
**OmniVoice Studio**.
|
||||
|
||||
### Installing to a different drive
|
||||
|
||||
<a id="install-other-drive"></a>
|
||||
|
||||
The wizard's **directory picker** lets you install the app to any **local**
|
||||
drive (D:, E:, …). Two caveats:
|
||||
|
||||
- **Mapped network drives (Z: → a share) are not supported** — this is a
|
||||
Windows Installer limitation, not an OmniVoice bug: MSI custom actions run
|
||||
as a service account that doesn't see per-user drive mappings, so the
|
||||
install fails or rolls back. Install to a local drive instead.
|
||||
- The install location only moves the ~200 MB app itself. The big data
|
||||
(models, voices, projects — tens of GB) lives in the **data directory**,
|
||||
which you move independently: **Settings → Storage → Models directory**
|
||||
in-app, or `OMNIVOICE_DATA_DIR` / [Portable mode](#portable-install) for
|
||||
the whole data tree.
|
||||
|
||||
If an install to a local non-C: drive fails anyway, capture a log with
|
||||
`msiexec /i OmniVoice*.msi /L*V install.log` and
|
||||
[open an issue](https://github.com/debpalash/OmniVoice-Studio/issues) with it
|
||||
— that log shows exactly which step rolled back.
|
||||
|
||||
## Portable install (Windows)
|
||||
|
||||
<a id="portable-install"></a>
|
||||
|
||||
+4
-1
@@ -76,7 +76,10 @@ Settings → Sharing → **Remote backend**:
|
||||
- **Test connection** hits `{url}/health` and shows the remote's version and
|
||||
device.
|
||||
- **Save & reload** stores both in this browser/app and restarts the UI
|
||||
against the remote.
|
||||
against the remote. The URL must be a full `http://` or `https://` URL
|
||||
(`gpu-box:3900` alone is rejected), and saving a URL that hasn't passed
|
||||
**Test connection** asks for confirmation first — a wrong base would leave
|
||||
the app unable to reach any backend until you change it back here.
|
||||
|
||||
Leave the URL empty to go back to the local backend.
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "omnivoice-studio",
|
||||
"version": "0.3.12",
|
||||
"version": "0.3.18",
|
||||
"private": true,
|
||||
"license": "AGPL-3.0-only",
|
||||
"type": "module",
|
||||
|
||||
Generated
+1
-1
@@ -2941,7 +2941,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "omnivoice-studio"
|
||||
version = "0.3.12"
|
||||
version = "0.3.18"
|
||||
dependencies = [
|
||||
"arboard",
|
||||
"dirs-next",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "omnivoice-studio"
|
||||
version = "0.3.12"
|
||||
version = "0.3.18"
|
||||
description = "OmniVoice Studio – AI voice cloning & dubbing desktop app"
|
||||
authors = ["Debpalash"]
|
||||
license = "AGPL-3.0-only"
|
||||
|
||||
@@ -22,9 +22,27 @@ HERE="$(dirname -- "$(readlink -f -- "$0")")"
|
||||
# Sourced by AppRun.test.sh — keep this function pure so unit tests can stub
|
||||
# `pkg-config`, source the file, call _detect_webkit_workaround, and inspect
|
||||
# the resulting environment without exec'ing the binary.
|
||||
#
|
||||
# Version source (#961 follow-up): the WebKitGTK that actually RUNS is the
|
||||
# BUNDLED copy (LD_LIBRARY_PATH below puts $HERE/usr/lib first) — NOT the
|
||||
# host's. Asking the host's pkg-config therefore reads the wrong number
|
||||
# whenever host and bundle diverge (e.g. a user who builds from source has
|
||||
# dev packages installed, so pkg-config answers with their system's healthy
|
||||
# 2.48 while the bundle runs an older lib — skipping a workaround the running
|
||||
# library needs). inject-apprun.sh stamps the bundled version into
|
||||
# .bundled-webkitgtk-version at build time, where it is knowable by
|
||||
# construction; the host pkg-config path survives only as a fallback for
|
||||
# bundles predating the stamp. OMNIVOICE_APPRUN_WK_MARKER exists for the
|
||||
# unit tests to point at a fixture marker.
|
||||
_detect_webkit_workaround() {
|
||||
local wk_version="0.0"
|
||||
if command -v pkg-config >/dev/null 2>&1; then
|
||||
local marker="${OMNIVOICE_APPRUN_WK_MARKER:-$HERE/.bundled-webkitgtk-version}"
|
||||
if [ -r "$marker" ]; then
|
||||
# Empty/unreadable marker content → "0.0" (unknown) → fail-safe workaround,
|
||||
# same philosophy as the missing-pkg-config branch below.
|
||||
wk_version="$(cat "$marker" 2>/dev/null | tr -d '[:space:]')"
|
||||
[ -n "$wk_version" ] || wk_version="0.0"
|
||||
elif command -v pkg-config >/dev/null 2>&1; then
|
||||
wk_version="$(pkg-config --modversion webkit2gtk-4.1 2>/dev/null \
|
||||
|| pkg-config --modversion webkit2gtk-4.0 2>/dev/null \
|
||||
|| echo "0.0")"
|
||||
|
||||
@@ -72,6 +72,56 @@ run_case "2.46 (broken)" "2.46.1" "1"
|
||||
run_case "2.48 (healthy)" "2.48.0" "unset"
|
||||
run_case "pkg-config absent" "0.0" "1" "no"
|
||||
|
||||
# ── Bundled-version marker cases (#961 follow-up) ───────────────────────────
|
||||
# inject-apprun.sh stamps the bundle's actual WebKitGTK version into
|
||||
# .bundled-webkitgtk-version at build time; AppRun must prefer that marker
|
||||
# over the host's pkg-config (which reports the SYSTEM version — wrong
|
||||
# whenever it diverges from the bundled copy, e.g. on a machine with newer
|
||||
# dev packages installed).
|
||||
|
||||
run_marker_case() {
|
||||
local label="$1" marker_content="$2" pkg_output="$3" expected="$4"
|
||||
local marker_file
|
||||
marker_file="$(mktemp)"
|
||||
printf '%s\n' "$marker_content" > "$marker_file"
|
||||
|
||||
local actual
|
||||
actual=$(
|
||||
bash -c '
|
||||
set +e
|
||||
pkg_output="'"$pkg_output"'"
|
||||
export OMNIVOICE_APPRUN_WK_MARKER="'"$marker_file"'"
|
||||
|
||||
pkg-config() { echo "$pkg_output"; }
|
||||
export -f pkg-config
|
||||
|
||||
exec() { :; }
|
||||
export -f exec
|
||||
|
||||
# shellcheck disable=SC1090
|
||||
source "'"$THIS_DIR"'/AppRun" >/dev/null 2>&1 || true
|
||||
echo "${WEBKIT_DISABLE_COMPOSITING_MODE:-unset}"
|
||||
'
|
||||
)
|
||||
rm -f "$marker_file"
|
||||
|
||||
if [[ "$actual" == "$expected" ]]; then
|
||||
echo "PASS [$label]"
|
||||
PASS_COUNT=$((PASS_COUNT + 1))
|
||||
else
|
||||
echo "FAIL [$label]: expected '$expected' got '$actual'" >&2
|
||||
FAIL_COUNT=$((FAIL_COUNT + 1))
|
||||
fi
|
||||
}
|
||||
|
||||
# Marker says broken → workaround applies, even though host pkg-config says healthy.
|
||||
run_marker_case "marker 2.46 beats host 2.48" "2.46.1" "2.48.0" "1"
|
||||
# Marker says healthy → no workaround, even though host pkg-config says broken
|
||||
# (the exact #961 inversion: from-source user with old system lib, new bundle).
|
||||
run_marker_case "marker 2.48 beats host 2.44" "2.48.0" "2.44.3" "unset"
|
||||
# Empty marker → treated as unknown → fail-safe workaround.
|
||||
run_marker_case "empty marker fails safe" "" "2.48.0" "1"
|
||||
|
||||
echo
|
||||
echo "─── AppRun test summary: $PASS_COUNT pass / $FAIL_COUNT fail ───"
|
||||
if [[ $FAIL_COUNT -ne 0 ]]; then
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<!--
|
||||
Tauri's macOS bundle defaults `hardenedRuntime` to true. Hardened
|
||||
Runtime blocks camera/microphone hardware access unless the matching
|
||||
entitlement is present here — regardless of Info.plist's
|
||||
NSMicrophoneUsageDescription and regardless of wry's own WKUIDelegate
|
||||
already granting the request at the WebKit/JS layer
|
||||
(WryWebViewUIDelegate::request_media_capture_permission unconditionally
|
||||
calls WKPermissionDecision::Grant). Without this entitlement, TCC
|
||||
never even registers a request for the app — nothing shows up in
|
||||
System Settings → Privacy & Security → Microphone to enable, because
|
||||
the OS never saw a legitimately-entitled process ask.
|
||||
-->
|
||||
<key>com.apple.security.device.audio-input</key>
|
||||
<true/>
|
||||
|
||||
<!--
|
||||
Matches Info.plist's forward-looking NSCameraUsageDescription — no
|
||||
current feature uses the camera, but ship the entitlement now so a
|
||||
future getUserMedia({video: true}) call doesn't hit this same bug.
|
||||
-->
|
||||
<key>com.apple.security.device.camera</key>
|
||||
<true/>
|
||||
</dict>
|
||||
</plist>
|
||||
@@ -83,7 +83,41 @@ pub fn same_app_version(running: &str) -> bool {
|
||||
!running.is_empty() && base(running) == base(env!("CARGO_PKG_VERSION"))
|
||||
}
|
||||
|
||||
/// Deep health probe for the attach-to-a-running-backend shortcut.
|
||||
///
|
||||
/// `/health` and `/system/info` keep answering from a backend whose install
|
||||
/// was deleted out from under it (files unlinked on disk, code already in
|
||||
/// memory) — that zombie passes the version check and then 500s every real
|
||||
/// route, so the UI looks alive but nothing works. Probe a DB-touching
|
||||
/// endpoint and require an actual `200` status line before attaching;
|
||||
/// anything else (500, timeout, refused) means the responder is not a
|
||||
/// backend worth keeping.
|
||||
pub fn backend_deep_healthy(port: u16) -> bool {
|
||||
let url = format!("http://127.0.0.1:{}/profiles", port);
|
||||
match raw_http_get(&url, Duration::from_millis(1500)) {
|
||||
Ok(resp) => parse_http_status(&resp) == Some(200),
|
||||
Err(_) => false,
|
||||
}
|
||||
}
|
||||
|
||||
/// Status code from a raw HTTP response ("HTTP/1.1 200 OK" → 200).
|
||||
fn parse_http_status(response: &str) -> Option<u16> {
|
||||
let line = response.lines().next()?;
|
||||
line.split_whitespace().nth(1)?.parse().ok()
|
||||
}
|
||||
|
||||
fn ureq_get_with_timeout(url: &str, timeout: Duration) -> Result<String, String> {
|
||||
let buf = raw_http_get(url, timeout)?;
|
||||
if let Some(idx) = buf.find("\r\n\r\n") {
|
||||
Ok(buf[idx + 4..].to_string())
|
||||
} else {
|
||||
Err("no body".into())
|
||||
}
|
||||
}
|
||||
|
||||
/// One raw loopback HTTP GET, returning the FULL response (status line +
|
||||
/// headers + body). Kept dependency-free on purpose — see module docs.
|
||||
fn raw_http_get(url: &str, timeout: Duration) -> Result<String, String> {
|
||||
let url = url.strip_prefix("http://").ok_or("only http:// supported")?;
|
||||
let (host_port, path) = match url.find('/') {
|
||||
Some(i) => (&url[..i], &url[i..]),
|
||||
@@ -112,11 +146,7 @@ fn ureq_get_with_timeout(url: &str, timeout: Duration) -> Result<String, String>
|
||||
stream.write_all(req.as_bytes()).map_err(|e| e.to_string())?;
|
||||
let mut buf = String::new();
|
||||
stream.read_to_string(&mut buf).map_err(|e| e.to_string())?;
|
||||
if let Some(idx) = buf.find("\r\n\r\n") {
|
||||
Ok(buf[idx + 4..].to_string())
|
||||
} else {
|
||||
Err("no body".into())
|
||||
}
|
||||
Ok(buf)
|
||||
}
|
||||
|
||||
/// Kill whatever process owns the port.
|
||||
@@ -428,6 +458,17 @@ mod tests {
|
||||
assert_eq!(parse_app_version("<html>not json</html>"), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_http_status_reads_the_status_line_only() {
|
||||
assert_eq!(super::parse_http_status("HTTP/1.1 200 OK\r\nX: 500\r\n\r\nbody"), Some(200));
|
||||
assert_eq!(
|
||||
super::parse_http_status("HTTP/1.1 500 Internal Server Error\r\n\r\nInternal Server Error"),
|
||||
Some(500)
|
||||
);
|
||||
assert_eq!(super::parse_http_status("garbage"), None);
|
||||
assert_eq!(super::parse_http_status(""), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn same_app_version_matches_current_build_and_rejects_stale() {
|
||||
let ours = env!("CARGO_PKG_VERSION");
|
||||
|
||||
@@ -148,12 +148,24 @@ pub fn retry_bootstrap(app: tauri::AppHandle, state: tauri::State<'_, BootstrapS
|
||||
}
|
||||
match crate::backend::running_backend_version(backend_port()) {
|
||||
Some(v) if crate::backend::same_app_version(&v) => {
|
||||
log::info!(
|
||||
"Port {} already serving OmniVoice backend v{} — attaching",
|
||||
if crate::backend::backend_deep_healthy(backend_port()) {
|
||||
log::info!(
|
||||
"Port {} already serving OmniVoice backend v{} — attaching",
|
||||
backend_port(), v
|
||||
);
|
||||
set_stage(&stage_handle, BootstrapStage::Ready);
|
||||
return;
|
||||
}
|
||||
// Same version but a DB-touching probe fails: a backend whose
|
||||
// install was wiped/corrupted while it kept running. Attaching
|
||||
// would look alive and 500 on everything — replace it.
|
||||
log::warn!(
|
||||
"Port {} serves OmniVoice v{} but failed the deep health probe — replacing it",
|
||||
backend_port(), v
|
||||
);
|
||||
set_stage(&stage_handle, BootstrapStage::Ready);
|
||||
return;
|
||||
set_backend_kill_intended(true); // deliberate kill, not a crash (#941)
|
||||
crate::backend::kill_orphan_on_port(backend_port());
|
||||
std::thread::sleep(Duration::from_millis(500));
|
||||
}
|
||||
Some(v) => {
|
||||
// A healthy-but-stale backend from a previous version (the
|
||||
@@ -713,6 +725,27 @@ fn sync_failure_is_torch_download(tail: &str) -> bool {
|
||||
/// distro-matched ROCm builds torch's own index doesn't carry).
|
||||
const ROCM_TORCH_INDEX: &str = "https://download.pytorch.org/whl/rocm6.4";
|
||||
|
||||
/// Args for the routine update-drift sync (#307 path) — the one that runs on
|
||||
/// every app update when `uv.lock` changed. `--inexact` is the fix for #1029:
|
||||
/// plain `uv sync` UNINSTALLS every package not in the lockfile, which
|
||||
/// silently deleted user-pip-installed optional engines (voxcpm, kittentts —
|
||||
/// packages the app's own Settings → Engines hints tell users to install
|
||||
/// into this venv) on every single update. `--inexact` still installs/
|
||||
/// upgrades everything the lockfile demands — locked deps stay exactly
|
||||
/// correct — it just stops removing extras the user added on purpose.
|
||||
///
|
||||
/// Deliberately NOT applied to the repair sync (`repair_sync_args`): repair
|
||||
/// runs when the venv is *broken*, and a user-installed extra is a plausible
|
||||
/// cause — healing must restore the known-good locked state, extras
|
||||
/// included-out. An engine lost to a repair is re-installable; a venv that
|
||||
/// repair can't actually repair is a support thread.
|
||||
const DRIFT_SYNC_ARGS: [&str; 5] = ["sync", "--frozen", "--inexact", "--no-dev", "--verbose"];
|
||||
|
||||
/// Exact-sync args for the venv-repair path — see `DRIFT_SYNC_ARGS` for why
|
||||
/// repair stays exact while the update-drift sync preserves user extras.
|
||||
const REPAIR_SYNC_ARGS_LOCKED: [&str; 4] = ["sync", "--frozen", "--no-dev", "--verbose"];
|
||||
const REPAIR_SYNC_ARGS_UNLOCKED: [&str; 3] = ["sync", "--no-dev", "--verbose"];
|
||||
|
||||
/// `uv pip install` args that replace the default CUDA torch build with the AMD
|
||||
/// ROCm wheel (#124). Opt-in (gated on OMNIVOICE_TORCH_VARIANT=rocm by the
|
||||
/// caller); the detection side (`get_best_device`) already routes ROCm through
|
||||
@@ -1254,7 +1287,7 @@ manually, then relaunch.",
|
||||
drift_cmd.env("UV_INDEX_URL", "https://mirrors.aliyun.com/pypi/simple/");
|
||||
}
|
||||
drift_cmd
|
||||
.args(["sync", "--frozen", "--no-dev", "--verbose"])
|
||||
.args(DRIFT_SYNC_ARGS)
|
||||
.current_dir(&project_dir);
|
||||
match run_streaming(app, "installing_deps", &mut drift_cmd) {
|
||||
Ok(ref s) if s.success() => {
|
||||
@@ -1329,9 +1362,9 @@ the existing venv; newly added dependencies may be missing (#307)",
|
||||
apply_uv_http_env(&mut repair_cmd);
|
||||
let has_lockfile = project_dir.join("uv.lock").is_file();
|
||||
if has_lockfile {
|
||||
repair_cmd.args(["sync", "--frozen", "--no-dev", "--verbose"]);
|
||||
repair_cmd.args(REPAIR_SYNC_ARGS_LOCKED);
|
||||
} else {
|
||||
repair_cmd.args(["sync", "--no-dev", "--verbose"]);
|
||||
repair_cmd.args(REPAIR_SYNC_ARGS_UNLOCKED);
|
||||
}
|
||||
repair_cmd.current_dir(&project_dir);
|
||||
let repair_status = run_streaming(app, "installing_deps", &mut repair_cmd);
|
||||
@@ -1712,6 +1745,29 @@ mod tests {
|
||||
use super::*;
|
||||
use std::collections::HashMap;
|
||||
|
||||
#[test]
|
||||
fn update_drift_sync_preserves_user_installed_engines() {
|
||||
// #1029: the routine update sync must carry --inexact so a
|
||||
// user-pip-installed optional engine (voxcpm, kittentts — packages
|
||||
// the app's own Settings → Engines hints tell users to install into
|
||||
// this venv) survives every update instead of being silently
|
||||
// uninstalled. --frozen must stay (lockfile is the resolution truth).
|
||||
assert!(DRIFT_SYNC_ARGS.contains(&"--inexact"),
|
||||
"update-drift sync lost --inexact — user-installed engines get wiped on every update (#1029)");
|
||||
assert!(DRIFT_SYNC_ARGS.contains(&"--frozen"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn repair_sync_stays_exact() {
|
||||
// Deliberate asymmetry with the drift sync: repair runs when the venv
|
||||
// is BROKEN and a user-installed extra is a plausible cause — healing
|
||||
// must restore the known-good locked state, extras included-out.
|
||||
assert!(!REPAIR_SYNC_ARGS_LOCKED.contains(&"--inexact"),
|
||||
"repair sync must stay exact — it's the recovery path when an extra broke the venv");
|
||||
assert!(!REPAIR_SYNC_ARGS_UNLOCKED.contains(&"--inexact"));
|
||||
assert!(REPAIR_SYNC_ARGS_LOCKED.contains(&"--frozen"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn scrub_python_env_removes_bundled_runtime_vars() {
|
||||
// #144: every uv/venv/pip subprocess must drop the AppImage's bundled
|
||||
|
||||
@@ -79,9 +79,18 @@ pub const TRAY_ICON_RECORDING: &[u8] = include_bytes!("../icons/tray-recording.p
|
||||
// applies on top.
|
||||
// - Linux (WebKitGTK): media-stream must be enabled per-WebView and the
|
||||
// permission request answered programmatically.
|
||||
// - macOS (WKWebView): nothing to do here — wry grants media-capture to the
|
||||
// app origin and the user-visible consent is the system TCC prompt driven
|
||||
// by NSMicrophoneUsageDescription in src-tauri/Info.plist.
|
||||
// - macOS (WKWebView): nothing to do here in code — wry's own WKUIDelegate
|
||||
// (WryWebViewUIDelegate::request_media_capture_permission) already grants
|
||||
// every media-capture request unconditionally at the WebKit/JS layer. But
|
||||
// that alone isn't sufficient (#1013): Tauri's macOS bundle defaults
|
||||
// `hardenedRuntime` to true, and Hardened Runtime blocks camera/microphone
|
||||
// hardware access unless the matching entitlement is present — without it,
|
||||
// TCC never even registers a request, so the app never appears in System
|
||||
// Settings → Privacy & Security → Microphone for the user to enable. See
|
||||
// src-tauri/entitlements.plist (wired in via tauri.conf.json's
|
||||
// bundle.macOS.entitlements) for the actual grant; NSMicrophoneUsageDescription
|
||||
// in Info.plist only supplies the *prompt text* TCC shows, it doesn't
|
||||
// substitute for the entitlement.
|
||||
|
||||
/// True for origins the app itself serves: the Tauri custom-protocol origin
|
||||
/// in production and the Vite dev server / loopback in `tauri dev`.
|
||||
@@ -798,12 +807,23 @@ pub fn run() {
|
||||
}
|
||||
match backend::running_backend_version(backend_port()) {
|
||||
Some(v) if backend::same_app_version(&v) => {
|
||||
log::info!(
|
||||
"Port {} already serving OmniVoice backend v{} — attaching",
|
||||
if backend::backend_deep_healthy(backend_port()) {
|
||||
log::info!(
|
||||
"Port {} already serving OmniVoice backend v{} — attaching",
|
||||
backend_port(), v
|
||||
);
|
||||
set_stage(&stage_handle, BootstrapStage::Ready);
|
||||
return;
|
||||
}
|
||||
// Same version but a DB-touching probe fails: a backend whose
|
||||
// install was wiped/corrupted while it kept running. Attaching
|
||||
// would look alive and 500 on everything — replace it.
|
||||
log::warn!(
|
||||
"Port {} serves OmniVoice v{} but failed the deep health probe — replacing it",
|
||||
backend_port(), v
|
||||
);
|
||||
set_stage(&stage_handle, BootstrapStage::Ready);
|
||||
return;
|
||||
backend::kill_orphan_on_port(backend_port());
|
||||
std::thread::sleep(Duration::from_millis(500));
|
||||
}
|
||||
Some(v) => {
|
||||
// Healthy-but-stale backend from a previous version —
|
||||
|
||||
@@ -85,7 +85,8 @@
|
||||
],
|
||||
"macOS": {
|
||||
"minimumSystemVersion": "12.0",
|
||||
"signingIdentity": "-"
|
||||
"signingIdentity": "-",
|
||||
"entitlements": "entitlements.plist"
|
||||
}
|
||||
},
|
||||
"plugins": {
|
||||
|
||||
+66
-1
@@ -42,6 +42,7 @@ import WorkspaceVoices from './components/WorkspaceVoices';
|
||||
import WorkspaceProjects from './components/WorkspaceProjects';
|
||||
import ErrorBoundary from './components/ErrorBoundary';
|
||||
import FloatingPill from './components/FloatingPill';
|
||||
import GlobalAudioPlayer from './components/GlobalAudioPlayer';
|
||||
import BackendCrashNotice from './components/BackendCrashNotice';
|
||||
// RemoteAuthGate is mounted at the true outermost provider in main-app.jsx so
|
||||
// it covers all app states (setup check / wizard / bootstrap), not just the
|
||||
@@ -83,6 +84,12 @@ import {
|
||||
renameProject as apiRenameProject,
|
||||
} from './api/projects';
|
||||
import { exportAction, exportReveal, exportRecord } from './api/exports';
|
||||
import {
|
||||
clearHistory as apiClearHistory,
|
||||
setHistoryStarred as apiSetHistoryStarred,
|
||||
audioUrlWithCacheBust,
|
||||
} from './api/generate';
|
||||
import { clearDubHistory as apiClearDubHistory } from './api/dub';
|
||||
|
||||
import { isTauri, doubleClickMaximize, fileToMediaUrl, playBlobAudio } from './utils/media';
|
||||
import { browserDownload } from './utils/download';
|
||||
@@ -589,7 +596,7 @@ function App() {
|
||||
fd.append('num_step', '16');
|
||||
const res = await apiFetch(`${API}/generate`, { method: 'POST', body: fd });
|
||||
const blob = await res.blob();
|
||||
await playBlobAudio(blob);
|
||||
await playBlobAudio(blob, { label: i18n.t('player.generated_audio') });
|
||||
toast.success(i18n.t('firstrun.first_sound_done'), { duration: 7000 });
|
||||
} catch {
|
||||
/* silent — see above */
|
||||
@@ -1135,6 +1142,33 @@ function App() {
|
||||
toast.success(i18n.t('app.toast_restored_state'));
|
||||
};
|
||||
|
||||
// Generation takes: star/unstar a take so it survives the retention cap and
|
||||
// never ages off the rail. Optimistic errors only — the WS
|
||||
// generation_history event refreshes the list on success.
|
||||
const toggleStarHistory = async (item) => {
|
||||
try {
|
||||
await apiSetHistoryStarred(item.id, !item.starred);
|
||||
loadHistory();
|
||||
} catch (err) {
|
||||
toast.error(err.message);
|
||||
}
|
||||
};
|
||||
|
||||
// Load a past take back as the active output: fetch its WAV and hand it to
|
||||
// the same global mini-player a fresh generation plays through.
|
||||
const playTakeAsOutput = async (item) => {
|
||||
try {
|
||||
const res = await apiFetch(audioUrlWithCacheBust(item.audio_path));
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
||||
const blob = await res.blob();
|
||||
await playBlobAudio(blob, {
|
||||
label: item.text || i18n.t('player.generated_audio'),
|
||||
});
|
||||
} catch (err) {
|
||||
toast.error(i18n.t('history.load_take_failed', { message: err.message || '' }));
|
||||
}
|
||||
};
|
||||
|
||||
const deleteHistory = async (id, type) => {
|
||||
if (!(await askConfirm('Delete this history item?'))) return;
|
||||
try {
|
||||
@@ -1151,6 +1185,27 @@ function App() {
|
||||
}
|
||||
};
|
||||
|
||||
// Clear-all for the workspace history panels (#1032). The control lived in
|
||||
// the old left Sidebar; the workspace UX overhaul (#374) moved history into
|
||||
// the right-side WorkspaceHistory panels and the button was dropped in the
|
||||
// move — restore it, scoped per workspace (voice = synth rows, dub = dubs).
|
||||
const clearWorkspaceHistory = async (type) => {
|
||||
const count = type === 'dub' ? dubHistory.length : history.length;
|
||||
if (!(await askConfirm(i18n.t('sidebar.clear_confirm', { count })))) return;
|
||||
try {
|
||||
if (type === 'dub') {
|
||||
await apiClearDubHistory();
|
||||
loadDubHistory();
|
||||
} else {
|
||||
await apiClearHistory();
|
||||
loadHistory();
|
||||
}
|
||||
toast.success(i18n.t('sidebar.history_cleared'));
|
||||
} catch (err) {
|
||||
toast.error(err.message);
|
||||
}
|
||||
};
|
||||
|
||||
// Install-plan screen outranks everything — both on a true first run and
|
||||
// when explicitly requested via `--setup`. Without this, a live backend
|
||||
// answering /setup/status would route straight to the model wizard and the
|
||||
@@ -1498,6 +1553,7 @@ function App() {
|
||||
dubHistory={dubHistory}
|
||||
restoreDubHistory={restoreDubHistory}
|
||||
deleteHistory={deleteHistory}
|
||||
clearHistory={() => clearWorkspaceHistory('dub')}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
@@ -1594,6 +1650,9 @@ function App() {
|
||||
handleNativeExport={handleNativeExport}
|
||||
restoreHistory={restoreHistory}
|
||||
deleteHistory={deleteHistory}
|
||||
clearHistory={() => clearWorkspaceHistory('synth')}
|
||||
toggleStarHistory={toggleStarHistory}
|
||||
playTakeAsOutput={playTakeAsOutput}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1700,6 +1759,12 @@ function App() {
|
||||
</Suspense>
|
||||
)}
|
||||
|
||||
{/* ═══ GLOBAL AUDIO MINI-PLAYER (grid row 3, above the footer) ═══
|
||||
Subsumes the #1032 PlaybackStopPill: waveform + seek + time + stop
|
||||
for every playBlobAudio playback that has no on-screen player. As a
|
||||
real grid row it can never overlap row-2 content or the footer. */}
|
||||
<GlobalAudioPlayer />
|
||||
|
||||
{/* ═══ BOTTOM LOGS PANEL (VSCode-style) ═══ */}
|
||||
<Suspense fallback={null}>
|
||||
<LogsFooter />
|
||||
|
||||
@@ -84,6 +84,64 @@ export async function selfTestEngine(engineId: string): Promise<EngineSelfTestRe
|
||||
return apiPost<EngineSelfTestResponse>(`/engines/${encodeURIComponent(engineId)}/selftest`, {});
|
||||
}
|
||||
|
||||
// ── One-click sidecar-engine install (IndexTTS-2 & friends) ─────────────
|
||||
|
||||
export type SidecarStepState = 'pending' | 'running' | 'done' | 'skipped' | 'error';
|
||||
|
||||
export interface SidecarInstallStep {
|
||||
id: string;
|
||||
state: SidecarStepState;
|
||||
detail: string | null;
|
||||
}
|
||||
|
||||
export interface SidecarInstallJob {
|
||||
engine_id: string;
|
||||
state: 'running' | 'succeeded' | 'failed';
|
||||
steps: SidecarInstallStep[];
|
||||
log: string[];
|
||||
error: string | null;
|
||||
remediation: string | null;
|
||||
weights_progress: {
|
||||
filename: string | null;
|
||||
downloaded: number | null;
|
||||
total: number | null;
|
||||
pct: number | null;
|
||||
} | null;
|
||||
started_at: number;
|
||||
finished_at: number | null;
|
||||
}
|
||||
|
||||
export interface SidecarInstallStatus {
|
||||
engine_id: string;
|
||||
installed: boolean;
|
||||
managed: boolean;
|
||||
install_dir: string | null;
|
||||
job: SidecarInstallJob | null;
|
||||
}
|
||||
|
||||
export interface SidecarInstallStartResponse {
|
||||
status: 'started' | 'already_running' | 'already_installed';
|
||||
engine: string;
|
||||
}
|
||||
|
||||
/** Start the resumable one-click install for a sidecar engine (IndexTTS-2).
|
||||
* Idempotent: re-POSTing while a job runs returns `already_running`; a
|
||||
* healthy install returns `already_installed`; a partial install repairs. */
|
||||
export async function installSidecarEngine(engineId: string): Promise<SidecarInstallStartResponse> {
|
||||
return apiPost<SidecarInstallStartResponse>(
|
||||
`/engines/sidecar/${encodeURIComponent(engineId)}/install`,
|
||||
{},
|
||||
);
|
||||
}
|
||||
|
||||
/** Poll the sidecar install job — step-by-step states + log tail + error
|
||||
* with remediation. Cheap (file probes only), safe to poll every ~1.5 s. */
|
||||
export async function getSidecarInstallStatus(engineId: string): Promise<SidecarInstallStatus> {
|
||||
return apiJson<SidecarInstallStatus>(
|
||||
`/engines/sidecar/${encodeURIComponent(engineId)}/install/status`,
|
||||
);
|
||||
}
|
||||
|
||||
export async function listTranslationEngines(): Promise<TranslationEnginesResponse> {
|
||||
return apiJson<TranslationEnginesResponse>('/engines/translation');
|
||||
}
|
||||
|
||||
@@ -16,6 +16,14 @@ export async function clearHistory(): Promise<Response> {
|
||||
return apiFetch('/history', { method: 'DELETE' });
|
||||
}
|
||||
|
||||
export async function setHistoryStarred(id: string, starred: boolean): Promise<unknown> {
|
||||
return apiJson(`/history/${id}/starred`, {
|
||||
method: 'PUT',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ starred }),
|
||||
});
|
||||
}
|
||||
|
||||
export function audioUrl(filename: string): string {
|
||||
return `${API}/audio/${filename}`;
|
||||
}
|
||||
|
||||
@@ -88,6 +88,38 @@ export async function modelStatus(): Promise<ModelStatus> {
|
||||
return apiJson<ModelStatus>('/model/status');
|
||||
}
|
||||
|
||||
// ── Loaded-model residency (MM2-04 endpoints) ────────────────────────────
|
||||
|
||||
/** One entry from GET /model/loaded — a model currently resident in memory.
|
||||
* `engine_id`/`is_active_engine` attribute TTS-family entries to an engine
|
||||
* (a model can stay resident after the user switches engines). */
|
||||
export interface LoadedModel {
|
||||
id: string; // 'tts' | 'asr' | 'diarization' | 'sidecar:<engine>'
|
||||
name: string;
|
||||
checkpoint: string;
|
||||
device: string;
|
||||
vram_mb: number;
|
||||
unloadable: boolean;
|
||||
note?: string;
|
||||
engine_id?: string;
|
||||
is_active_engine?: boolean | null;
|
||||
}
|
||||
|
||||
export interface LoadedModelsResponse {
|
||||
models: LoadedModel[];
|
||||
count: number;
|
||||
}
|
||||
|
||||
export async function listLoadedModels(): Promise<LoadedModelsResponse> {
|
||||
return apiJson<LoadedModelsResponse>('/model/loaded');
|
||||
}
|
||||
|
||||
/** Unload one resident model by its /model/loaded `id`. The model reloads
|
||||
* lazily on next use — unloading only frees memory, it never loses data. */
|
||||
export async function unloadLoadedModel(modelId: string): Promise<unknown> {
|
||||
return apiPost(`/model/unload/${encodeURIComponent(modelId)}`);
|
||||
}
|
||||
|
||||
// ── Audio cleaning ───────────────────────────────────────────────────────
|
||||
|
||||
export async function cleanAudio(formData: FormData): Promise<Response> {
|
||||
|
||||
@@ -29,10 +29,22 @@ interface EngineBackend {
|
||||
display_name: string;
|
||||
available: boolean;
|
||||
reason: string | null;
|
||||
// Available-but-has-advice: the backend's `is_available()` returned ok with
|
||||
// an advisory tail ("ready — <advice>", e.g. VoxCPM2's upgrade hint). Null
|
||||
// for plain-ready and unavailable rows; absent on legacy payloads.
|
||||
hint?: string | null;
|
||||
// Cloning capability (TTS family): true/false from the backend class, null
|
||||
// when model-dependent (mlx-audio's curated models differ). Only badge on
|
||||
// an explicit true.
|
||||
supports_cloning?: boolean | null;
|
||||
install_hint?: string | null;
|
||||
// Copy-paste-ready `export VAR=...` line for a path-gated opt-in engine
|
||||
// (IndexTTS / MOSS-v1.5 / dots.tts / Confucius4), else null/absent.
|
||||
setup_snippet?: string | null;
|
||||
// True when the backend's sidecar provisioner can install this engine
|
||||
// in-app (Settings renders an Install button; the manual snippet is
|
||||
// demoted to a collapsible fallback). Absent on legacy payloads.
|
||||
one_click_install?: boolean;
|
||||
last_error?: string | null;
|
||||
isolation_mode?: 'in-process' | 'subprocess';
|
||||
gpu_compat?: GPUTarget[];
|
||||
@@ -244,6 +256,17 @@ export interface DubTranslateResponse {
|
||||
text_original?: string;
|
||||
rate_ratio?: number;
|
||||
rate_error?: string;
|
||||
/** Pre-synthesis duration plan (backend services/duration_planner.py). */
|
||||
plan?: {
|
||||
status: 'fits' | 'tight' | 'impossible';
|
||||
est_dur_s: number;
|
||||
available_s: number;
|
||||
est_overrun_s: number;
|
||||
calibrated: boolean;
|
||||
/** Opt-in LLM condensation suggestion (request condense=true only). */
|
||||
suggested_text?: string;
|
||||
suggested_est_dur_s?: number;
|
||||
};
|
||||
}[];
|
||||
}
|
||||
|
||||
|
||||
@@ -275,6 +275,42 @@ function DubSegmentRow({
|
||||
📖 {seg.rate_ratio.toFixed(2)}×
|
||||
</span>
|
||||
)}
|
||||
{/* Pre-synthesis duration plan (backend duration_planner): warn about
|
||||
tight/impossible segments BEFORE GPU time is spent. Informational
|
||||
only — generation is never blocked. */}
|
||||
{seg.plan && (seg.plan.status === 'tight' || seg.plan.status === 'impossible') && (
|
||||
<span
|
||||
className="text-[0.48rem] mt-[1px] inline-flex items-center gap-[1px]"
|
||||
style={{ color: seg.plan.status === 'impossible' ? '#fb4934' : '#fabd2f' }}
|
||||
title={t(
|
||||
seg.plan.status === 'impossible'
|
||||
? 'segment.plan_impossible_title'
|
||||
: 'segment.plan_tight_title',
|
||||
{
|
||||
est: (seg.plan.est_dur_s || 0).toFixed(1),
|
||||
avail: (seg.plan.available_s || 0).toFixed(1),
|
||||
seconds: (seg.plan.est_overrun_s || 0).toFixed(1),
|
||||
},
|
||||
)}
|
||||
>
|
||||
<AlertCircle size={8} />{' '}
|
||||
{seg.plan.status === 'impossible'
|
||||
? t('segment.plan_impossible', {
|
||||
seconds: (seg.plan.est_overrun_s || 0).toFixed(1),
|
||||
})
|
||||
: t('segment.plan_tight')}
|
||||
</span>
|
||||
)}
|
||||
{seg.plan && seg.plan.suggested_text && seg.plan.suggested_text !== seg.text && (
|
||||
<button
|
||||
onClick={() => onEditField(seg.id, 'text', seg.plan.suggested_text)}
|
||||
disabled={disabled}
|
||||
title={t('segment.plan_apply_title', { text: seg.plan.suggested_text })}
|
||||
className="bg-transparent border-none text-[#83a598] cursor-pointer p-0 mt-[1px] text-[0.48rem] text-left"
|
||||
>
|
||||
✂ {t('segment.plan_apply')}
|
||||
</button>
|
||||
)}
|
||||
</span>
|
||||
|
||||
<input
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,62 @@
|
||||
import React from 'react';
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
/**
|
||||
* EngineMark — the per-engine identity mark for the Models & Engines
|
||||
* Settings surfaces.
|
||||
*
|
||||
* A small monogram chip whose hue is derived deterministically from the
|
||||
* engine id (the same trick as `models/format.js`'s `orgColor` for HF
|
||||
* orgs), so the same engine is instantly recognizable everywhere it
|
||||
* appears on these pages: the Engine Compatibility Matrix rows and the
|
||||
* "in memory" residency chips. Purely decorative (`aria-hidden`) — the
|
||||
* engine's name and id are always rendered as text alongside it.
|
||||
*
|
||||
* Theme-safe by construction: the hue is fixed per engine, but the fill
|
||||
* is a low-opacity `color-mix` over transparent and the glyph color is
|
||||
* mixed toward `--chrome-fg`, so it stays legible on light and dark
|
||||
* themes without per-theme overrides.
|
||||
*/
|
||||
|
||||
/** Deterministic hue (0–359) from an engine id. */
|
||||
export function engineHue(id) {
|
||||
const s = String(id || '');
|
||||
let h = 0;
|
||||
for (let i = 0; i < s.length; i++) h = (h * 31 + s.charCodeAt(i)) & 0xffff;
|
||||
return h % 360;
|
||||
}
|
||||
|
||||
/** Two-character monogram from an engine id ("mlx-audio" → "MA",
|
||||
* "voxcpm2" → "VO"). Falls back to "?" for an empty id. */
|
||||
export function engineMonogram(id) {
|
||||
const parts = String(id || '')
|
||||
.split(/[^a-z0-9]+/i)
|
||||
.filter(Boolean);
|
||||
if (parts.length === 0) return '?';
|
||||
const mono = parts.length >= 2 ? parts[0][0] + parts[1][0] : parts[0].slice(0, 2);
|
||||
return mono.toUpperCase();
|
||||
}
|
||||
|
||||
export default function EngineMark({ id, size = 20, className = '' }) {
|
||||
const accent = `hsl(${engineHue(id)} 62% 52%)`;
|
||||
return (
|
||||
<span
|
||||
aria-hidden="true"
|
||||
data-testid={`engine-mark-${id}`}
|
||||
className={cn(
|
||||
'inline-flex shrink-0 select-none items-center justify-center rounded-[5px] font-semibold tracking-[0.02em]',
|
||||
className,
|
||||
)}
|
||||
style={{
|
||||
width: size,
|
||||
height: size,
|
||||
fontSize: Math.max(8, Math.round(size * 0.42)),
|
||||
background: `color-mix(in srgb, ${accent} 15%, transparent)`,
|
||||
border: `1px solid color-mix(in srgb, ${accent} 40%, transparent)`,
|
||||
color: `color-mix(in srgb, ${accent} 55%, var(--chrome-fg, currentColor))`,
|
||||
}}
|
||||
>
|
||||
{engineMonogram(id)}
|
||||
</span>
|
||||
);
|
||||
}
|
||||
@@ -286,7 +286,7 @@ export default function ExportModal({
|
||||
|
||||
return createPortal(
|
||||
<div
|
||||
className="pointer-events-none fixed inset-x-0 bottom-[var(--logs-footer-height,28px)] z-[90] flex justify-center"
|
||||
className="pointer-events-none fixed inset-x-0 bottom-[calc(var(--logs-footer-height,28px)+var(--audio-dock-height,0px))] z-[90] flex justify-center"
|
||||
role="dialog"
|
||||
aria-modal="false"
|
||||
aria-label={t('exportModal.export_options')}
|
||||
|
||||
@@ -0,0 +1,223 @@
|
||||
/**
|
||||
* GlobalAudioPlayer — persistent bottom mini-player for "invisible" audio.
|
||||
*
|
||||
* `playBlobAudio` (playback source 'output') plays the generate auto-play,
|
||||
* profile & dub-segment previews, story lines, gallery voices and Projects
|
||||
* renders through a bare Audio()/AudioContext with no on-screen player. Its
|
||||
* only global affordance used to be the stop-only PlaybackStopPill (#1032) —
|
||||
* this bar subsumes it: waveform (peaks decoded once from the blob already in
|
||||
* hand), click/drag/keyboard seek, play/pause, elapsed/total time, a source
|
||||
* label and a stop button, on every page (mounted once in App.jsx).
|
||||
*
|
||||
* Exclusion semantics are the pill's, unchanged: ONLY source 'output'
|
||||
* renders here. Sources with their own visible player UI (WaveformPlayer
|
||||
* instances, 'design-preview', 'demo-output') stay in-place.
|
||||
*
|
||||
* Layout: a real grid row of .app-container (row 3, directly above the
|
||||
* LogsFooter — see index.css). Content in row 2 physically ends at the bar's
|
||||
* top edge, so the fixed-overlay overlap class the pill had at 1440×900
|
||||
* (covering the studio's Production Overrides row) is impossible by
|
||||
* construction. While visible it publishes --audio-dock-height so the fixed
|
||||
* overlays that anchor above the footer (FloatingPill, VoicePreview,
|
||||
* ExportModal, compare drawer) ride above the bar too.
|
||||
*/
|
||||
import React, { useEffect, useRef, useState } from 'react';
|
||||
import { Pause, Play, Square } from 'lucide-react';
|
||||
import { useTranslation } from 'react-i18next';
|
||||
import {
|
||||
pauseActivePlayback,
|
||||
resumeActivePlayback,
|
||||
seekActivePlayback,
|
||||
stopActivePlayback,
|
||||
usePlaybackTrack,
|
||||
} from '../utils/playback';
|
||||
|
||||
const DOCK_H = 44; // collapsed-chrome scale: header/footer bars are 28px, player needs touch room
|
||||
|
||||
const fmt = (s) => {
|
||||
if (!isFinite(s) || s < 0) s = 0;
|
||||
const m = Math.floor(s / 60);
|
||||
const sec = Math.floor(s % 60);
|
||||
return `${m}:${String(sec).padStart(2, '0')}`;
|
||||
};
|
||||
|
||||
// Same visual language as WaveformPlayer's wavesurfer config (bar width 2,
|
||||
// gap 1, wave/progress colors), just hand-drawn on a canvas — the peaks are
|
||||
// precomputed in utils/media.js, so no wavesurfer instance (and no second
|
||||
// decode/fetch) is needed here.
|
||||
const WAVE_COLOR = 'rgba(168,153,132,0.45)';
|
||||
const PROGRESS_COLOR = 'rgba(211,134,155,0.75)';
|
||||
const CURSOR_COLOR = '#d3869b';
|
||||
|
||||
function WaveCanvas({ peaks, progress }) {
|
||||
const wrapRef = useRef(null);
|
||||
const canvasRef = useRef(null);
|
||||
const [width, setWidth] = useState(0);
|
||||
|
||||
useEffect(() => {
|
||||
const el = wrapRef.current;
|
||||
if (!el || typeof ResizeObserver === 'undefined') return undefined;
|
||||
const ro = new ResizeObserver(() => setWidth(el.clientWidth));
|
||||
ro.observe(el);
|
||||
setWidth(el.clientWidth);
|
||||
return () => ro.disconnect();
|
||||
}, []);
|
||||
|
||||
useEffect(() => {
|
||||
const canvas = canvasRef.current;
|
||||
const ctx = canvas?.getContext?.('2d');
|
||||
if (!ctx) return; // jsdom / very old engines — seek + time still work
|
||||
const w = width || canvas.clientWidth;
|
||||
const h = canvas.clientHeight || 28;
|
||||
if (!w || !h) return;
|
||||
const dpr = window.devicePixelRatio || 1;
|
||||
canvas.width = w * dpr;
|
||||
canvas.height = h * dpr;
|
||||
ctx.scale(dpr, dpr);
|
||||
ctx.clearRect(0, 0, w, h);
|
||||
const playedX = Math.max(0, Math.min(1, progress)) * w;
|
||||
if (peaks && peaks.length) {
|
||||
const barW = 2;
|
||||
const gap = 1;
|
||||
const count = Math.max(1, Math.floor(w / (barW + gap)));
|
||||
for (let i = 0; i < count; i++) {
|
||||
const x = i * (barW + gap);
|
||||
const peak = peaks[Math.floor((i / count) * peaks.length)] || 0;
|
||||
const barH = Math.max(2, peak * (h - 2));
|
||||
ctx.fillStyle = x + barW <= playedX ? PROGRESS_COLOR : WAVE_COLOR;
|
||||
ctx.fillRect(x, (h - barH) / 2, barW, barH);
|
||||
}
|
||||
} else {
|
||||
// No peaks (decode unavailable — e.g. the Tauri streamed fallback):
|
||||
// a plain progress track, same colors.
|
||||
ctx.fillStyle = WAVE_COLOR;
|
||||
ctx.fillRect(0, h / 2 - 1.5, w, 3);
|
||||
ctx.fillStyle = PROGRESS_COLOR;
|
||||
ctx.fillRect(0, h / 2 - 1.5, playedX, 3);
|
||||
}
|
||||
// Playhead cursor.
|
||||
ctx.fillStyle = CURSOR_COLOR;
|
||||
ctx.fillRect(Math.min(playedX, w - 1), 0, 1.5, h);
|
||||
}, [peaks, progress, width]);
|
||||
|
||||
return (
|
||||
<div ref={wrapRef} className="w-full h-full">
|
||||
<canvas ref={canvasRef} className="block w-full h-full" aria-hidden="true" />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function PlayerBar({ track }) {
|
||||
const { t } = useTranslation();
|
||||
const { label, paused, currentTime, duration, peaks, canSeek, canPause } = track;
|
||||
const scrubbingRef = useRef(false);
|
||||
|
||||
const seekable = canSeek && duration > 0;
|
||||
const seekToClientX = (target, clientX) => {
|
||||
const rect = target.getBoundingClientRect();
|
||||
if (!rect.width) return;
|
||||
const frac = Math.max(0, Math.min(1, (clientX - rect.left) / rect.width));
|
||||
seekActivePlayback(frac * duration);
|
||||
};
|
||||
|
||||
const onPointerDown = (e) => {
|
||||
if (!seekable) return;
|
||||
scrubbingRef.current = true;
|
||||
e.currentTarget.setPointerCapture?.(e.pointerId);
|
||||
seekToClientX(e.currentTarget, e.clientX);
|
||||
};
|
||||
const onPointerMove = (e) => {
|
||||
if (!seekable || !scrubbingRef.current) return;
|
||||
seekToClientX(e.currentTarget, e.clientX);
|
||||
};
|
||||
const endScrub = () => {
|
||||
scrubbingRef.current = false;
|
||||
};
|
||||
const onKeyDown = (e) => {
|
||||
if (!seekable) return;
|
||||
if (e.key === 'ArrowRight') seekActivePlayback(Math.min(duration, currentTime + 5));
|
||||
else if (e.key === 'ArrowLeft') seekActivePlayback(Math.max(0, currentTime - 5));
|
||||
else if (e.key === 'Home') seekActivePlayback(0);
|
||||
else if (e.key === 'End') seekActivePlayback(duration);
|
||||
else return;
|
||||
e.preventDefault();
|
||||
};
|
||||
|
||||
return (
|
||||
<div
|
||||
className="global-audio-dock flex items-center gap-[10px] px-[10px] [background:var(--chrome-bg)] [border-top:1px_solid_var(--chrome-border)] [color:var(--chrome-fg)] select-none"
|
||||
style={{ height: DOCK_H }}
|
||||
role="region"
|
||||
aria-label={t('player.now_playing')}
|
||||
data-testid="global-audio-player"
|
||||
>
|
||||
{canPause && (
|
||||
<button
|
||||
type="button"
|
||||
className="wf-player__btn shrink-0 inline-flex items-center justify-center w-[28px] h-[28px] border-none rounded-full cursor-pointer text-[color:var(--color-fg-inverse)] bg-[var(--color-brand)] [transition:background_0.15s_ease,transform_0.1s_ease] hover:bg-[var(--color-brand-hover)] active:scale-[0.94]"
|
||||
onClick={paused ? resumeActivePlayback : pauseActivePlayback}
|
||||
aria-label={paused ? t('player.play') : t('player.pause')}
|
||||
>
|
||||
{paused ? <Play size={14} /> : <Pause size={14} />}
|
||||
</button>
|
||||
)}
|
||||
<span
|
||||
className="shrink-0 max-w-[220px] truncate text-[11.5px] [color:var(--chrome-fg-muted)]"
|
||||
title={label || t('player.untitled')}
|
||||
>
|
||||
{label || t('player.untitled')}
|
||||
</span>
|
||||
<div
|
||||
className={`flex-1 min-w-0 h-[28px] ${seekable ? 'cursor-pointer' : 'cursor-default'}`}
|
||||
role="slider"
|
||||
tabIndex={seekable ? 0 : -1}
|
||||
aria-label={t('player.seek')}
|
||||
aria-valuemin={0}
|
||||
aria-valuemax={Math.round(duration)}
|
||||
aria-valuenow={Math.round(currentTime)}
|
||||
aria-valuetext={`${fmt(currentTime)} / ${fmt(duration)}`}
|
||||
aria-disabled={!seekable}
|
||||
onPointerDown={onPointerDown}
|
||||
onPointerMove={onPointerMove}
|
||||
onPointerUp={endScrub}
|
||||
onPointerCancel={endScrub}
|
||||
onKeyDown={onKeyDown}
|
||||
>
|
||||
<WaveCanvas peaks={peaks} progress={duration > 0 ? currentTime / duration : 0} />
|
||||
</div>
|
||||
<span className="shrink-0 [font-variant-numeric:tabular-nums] text-[11px] [color:var(--chrome-fg-muted)] whitespace-nowrap">
|
||||
{fmt(currentTime)} / {fmt(duration)}
|
||||
</span>
|
||||
<button
|
||||
type="button"
|
||||
className="shrink-0 flex items-center justify-center w-[var(--chrome-icon-btn)] h-[var(--chrome-icon-btn)] rounded-[3px] bg-transparent border-0 cursor-pointer [color:var(--chrome-fg-muted)] hover:[color:var(--chrome-fg)] hover:[background:var(--chrome-hover-bg)] focus-visible:[outline:2px_solid_var(--chrome-accent)] focus-visible:[outline-offset:1px]"
|
||||
onClick={stopActivePlayback}
|
||||
title={t('player.stop')}
|
||||
aria-label={t('player.stop')}
|
||||
>
|
||||
<Square size={12} />
|
||||
</button>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
export default function GlobalAudioPlayer() {
|
||||
const track = usePlaybackTrack();
|
||||
// Exact PlaybackStopPill routing: only bare 'output' playback docks here.
|
||||
const visible = track?.source === 'output';
|
||||
|
||||
// Publish the dock height so fixed overlays anchored above the LogsFooter
|
||||
// (--logs-footer-height consumers) stack above the bar instead of over it.
|
||||
useEffect(() => {
|
||||
document.documentElement.style.setProperty(
|
||||
'--audio-dock-height',
|
||||
visible ? `${DOCK_H}px` : '0px',
|
||||
);
|
||||
return () => {
|
||||
document.documentElement.style.setProperty('--audio-dock-height', '0px');
|
||||
};
|
||||
}, [visible]);
|
||||
|
||||
if (!visible) return null;
|
||||
return <PlayerBar track={track} />;
|
||||
}
|
||||
@@ -798,6 +798,9 @@ export default function LogsFooter() {
|
||||
if (notif.action.type === 'navigate') {
|
||||
useAppStore.getState().setMode?.(notif.action.target);
|
||||
setCollapsed(true);
|
||||
} else if (notif.action.type === 'settings-tab') {
|
||||
useAppStore.getState().openSettingsTab?.(notif.action.target);
|
||||
setCollapsed(true);
|
||||
} else if (notif.action.type === 'link') {
|
||||
import('../api/external').then((m) => m.openExternal(notif.action.target));
|
||||
}
|
||||
|
||||
@@ -0,0 +1,181 @@
|
||||
/**
|
||||
* Media engine — invisible unless it needs help.
|
||||
*
|
||||
* The media engine (ffmpeg/ffprobe) is an internal dependency, not a system
|
||||
* requirement: when the backend's resolution chain finds nothing, preflight
|
||||
* already kicked a background download of the app's own pinned static build.
|
||||
* This renders NOTHING when the engine is ready (the ideal outcome), a quiet
|
||||
* one-line progress while acquiring, and an actionable card only on failure
|
||||
* (Retry / use a copy already on the machine). yt-dlp never appears here —
|
||||
* it's an importable module, not a user task.
|
||||
*/
|
||||
import { useCallback, useEffect, useState } from 'react';
|
||||
import { useTranslation } from 'react-i18next';
|
||||
import { Loader } from 'lucide-react';
|
||||
import { apiJson, apiFetch } from '../api/client';
|
||||
import { Button } from '../ui';
|
||||
|
||||
export default function MediaEngineCard() {
|
||||
const { t } = useTranslation();
|
||||
const [status, setStatus] = useState(null);
|
||||
const [detectError, setDetectError] = useState(null);
|
||||
const [customPath, setCustomPath] = useState('');
|
||||
const [showPathInput, setShowPathInput] = useState(false);
|
||||
const [busy, setBusy] = useState(false);
|
||||
|
||||
const refresh = useCallback(async () => {
|
||||
try {
|
||||
const st = await apiJson('/media-tools/status');
|
||||
setStatus(st);
|
||||
return st;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}, []);
|
||||
|
||||
useEffect(() => {
|
||||
refresh();
|
||||
}, [refresh]);
|
||||
|
||||
const acquiring = status?.ops?.acquire?.state === 'running';
|
||||
useEffect(() => {
|
||||
if (!acquiring) return undefined;
|
||||
const iv = setInterval(refresh, 1500);
|
||||
return () => clearInterval(iv);
|
||||
}, [acquiring, refresh]);
|
||||
|
||||
const post = async (path, body) => {
|
||||
setBusy(true);
|
||||
setDetectError(null);
|
||||
try {
|
||||
const res = await apiFetch(path, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: body ? JSON.stringify(body) : undefined,
|
||||
});
|
||||
if (!res.ok) {
|
||||
let detail = `HTTP ${res.status}`;
|
||||
try {
|
||||
detail = (await res.json())?.detail || detail;
|
||||
} catch {
|
||||
/* non-JSON body */
|
||||
}
|
||||
throw new Error(detail);
|
||||
}
|
||||
return true;
|
||||
} catch (e) {
|
||||
setDetectError(e?.message || String(e));
|
||||
return false;
|
||||
} finally {
|
||||
setBusy(false);
|
||||
refresh();
|
||||
}
|
||||
};
|
||||
|
||||
const useSystemCopy = async () => {
|
||||
// ffprobe rides along: the resolver derives the sibling ffprobe from a
|
||||
// resolved ffmpeg, so pinning ffmpeg is enough in the common case.
|
||||
await post('/media-tools/ffmpeg/use-system');
|
||||
};
|
||||
|
||||
const chooseFile = async () => {
|
||||
try {
|
||||
if ('__TAURI_INTERNALS__' in window) {
|
||||
const { open } = await import('@tauri-apps/plugin-dialog');
|
||||
const picked = await open({ multiple: false, directory: false, title: 'FFmpeg' });
|
||||
if (typeof picked === 'string') {
|
||||
await post('/media-tools/ffmpeg/custom-path', { path: picked });
|
||||
return;
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
/* picker unavailable — fall through to the inline input */
|
||||
}
|
||||
setShowPathInput(true);
|
||||
};
|
||||
|
||||
if (!status || status.ready) return null; // the ideal outcome: nothing.
|
||||
|
||||
const op = status.ops?.acquire || {};
|
||||
if (op.state === 'running' || op.state === 'idle') {
|
||||
// idle-and-not-ready = preflight is about to kick the download (or a
|
||||
// recheck is in flight) — show the quiet line, never flash the card.
|
||||
return (
|
||||
<div
|
||||
className="mt-3 flex items-center gap-2 text-xs text-fg-muted"
|
||||
data-testid="media-engine-progress"
|
||||
>
|
||||
<Loader className="animate-spin" size={12} aria-hidden="true" />
|
||||
{t('setup.media_engine_preparing', { defaultValue: 'Preparing media engine…' })}
|
||||
{op.state === 'running' && ` ${Math.round((op.progress || 0) * 100)}%`}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<div
|
||||
className="mt-3 flex flex-col gap-1.5 rounded-md border border-border px-3 py-2.5"
|
||||
data-testid="media-engine-card"
|
||||
>
|
||||
<span className="text-sm font-semibold">
|
||||
{t('setup.media_engine_failed_title', { defaultValue: 'Media engine download failed' })}
|
||||
</span>
|
||||
<span className="text-xs leading-snug text-fg-muted">
|
||||
{t('setup.media_engine_failed_desc', {
|
||||
defaultValue:
|
||||
"The app couldn't fetch its bundled audio/video engine (FFmpeg). Retry, or point it at a copy already on this computer.",
|
||||
})}
|
||||
</span>
|
||||
{(op.error || detectError) && (
|
||||
<span className="text-xs text-danger" role="alert" data-testid="media-engine-error">
|
||||
{detectError || op.error}
|
||||
</span>
|
||||
)}
|
||||
<div className="mt-1 flex flex-wrap items-center gap-2">
|
||||
<Button
|
||||
variant="subtle"
|
||||
size="sm"
|
||||
loading={busy}
|
||||
disabled={busy}
|
||||
onClick={() => post('/media-tools/acquire')}
|
||||
data-testid="media-engine-retry"
|
||||
>
|
||||
{t('setup.media_engine_retry', { defaultValue: 'Retry' })}
|
||||
</Button>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
disabled={busy}
|
||||
onClick={useSystemCopy}
|
||||
data-testid="media-engine-use-system"
|
||||
>
|
||||
{t('setup.media_engine_use_system', { defaultValue: 'Use a system copy' })}
|
||||
</Button>
|
||||
<Button variant="ghost" size="sm" disabled={busy} onClick={chooseFile}>
|
||||
{t('setup.media_engine_choose_file', { defaultValue: 'Choose file…' })}
|
||||
</Button>
|
||||
{showPathInput && (
|
||||
<>
|
||||
<input
|
||||
type="text"
|
||||
value={customPath}
|
||||
onChange={(e) => setCustomPath(e.target.value)}
|
||||
placeholder="/usr/bin/ffmpeg"
|
||||
className="min-w-[220px] flex-1 rounded border border-border bg-transparent px-2 py-1 font-mono text-xs text-fg"
|
||||
aria-label={t('settings.ffmpeg_input_aria', { defaultValue: 'FFmpeg path' })}
|
||||
data-testid="media-engine-path"
|
||||
/>
|
||||
<Button
|
||||
variant="subtle"
|
||||
size="sm"
|
||||
disabled={busy || !customPath.trim()}
|
||||
onClick={() => post('/media-tools/ffmpeg/custom-path', { path: customPath.trim() })}
|
||||
>
|
||||
{t('credentials.save', { defaultValue: 'Save' })}
|
||||
</Button>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,100 @@
|
||||
import React from 'react';
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest';
|
||||
import { fireEvent, render, screen, waitFor } from '@testing-library/react';
|
||||
|
||||
vi.mock('../api/client', () => ({
|
||||
apiJson: vi.fn(),
|
||||
apiFetch: vi.fn(),
|
||||
}));
|
||||
|
||||
import { apiJson, apiFetch } from '../api/client';
|
||||
import MediaEngineCard from './MediaEngineCard';
|
||||
|
||||
const statusWith = (ready, acquire) => ({
|
||||
ready,
|
||||
tools: {},
|
||||
ops: { acquire: acquire || { state: 'idle', progress: 0, error: null } },
|
||||
});
|
||||
|
||||
describe('MediaEngineCard — invisible-by-default media engine', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
apiFetch.mockResolvedValue({ ok: true, json: async () => ({}) });
|
||||
});
|
||||
|
||||
it('renders NOTHING when the media engine is resolved (the ideal outcome)', async () => {
|
||||
apiJson.mockResolvedValue(statusWith(true));
|
||||
const { container } = render(<MediaEngineCard />);
|
||||
await waitFor(() => expect(apiJson).toHaveBeenCalledWith('/media-tools/status'));
|
||||
expect(container).toBeEmptyDOMElement();
|
||||
expect(screen.queryByTestId('media-engine-card')).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('shows only a quiet progress line while the bundled build downloads', async () => {
|
||||
apiJson.mockResolvedValue(statusWith(false, { state: 'running', progress: 0.42, error: null }));
|
||||
render(<MediaEngineCard />);
|
||||
const line = await screen.findByTestId('media-engine-progress');
|
||||
expect(line).toHaveTextContent('Preparing media engine…');
|
||||
expect(line).toHaveTextContent('42%');
|
||||
// No requirements-style card, no mention of package managers.
|
||||
expect(screen.queryByTestId('media-engine-card')).not.toBeInTheDocument();
|
||||
expect(document.body.textContent).not.toMatch(/brew|apt|choco/i);
|
||||
});
|
||||
|
||||
it('shows the actionable failure card only when acquisition failed', async () => {
|
||||
apiJson.mockResolvedValue(
|
||||
statusWith(false, { state: 'error', progress: 0, error: 'download checksum mismatch' }),
|
||||
);
|
||||
render(<MediaEngineCard />);
|
||||
const card = await screen.findByTestId('media-engine-card');
|
||||
expect(card).toHaveTextContent('Media engine download failed');
|
||||
expect(screen.getByTestId('media-engine-error')).toHaveTextContent(
|
||||
'download checksum mismatch',
|
||||
);
|
||||
expect(screen.getByTestId('media-engine-retry')).toBeInTheDocument();
|
||||
expect(screen.getByTestId('media-engine-use-system')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('Retry re-posts the acquisition endpoint', async () => {
|
||||
apiJson.mockResolvedValue(statusWith(false, { state: 'error', error: 'boom' }));
|
||||
render(<MediaEngineCard />);
|
||||
fireEvent.click(await screen.findByTestId('media-engine-retry'));
|
||||
await waitFor(() =>
|
||||
expect(apiFetch).toHaveBeenCalledWith('/media-tools/acquire', expect.anything()),
|
||||
);
|
||||
});
|
||||
|
||||
it('Use a system copy posts use-system and surfaces a not-found detail', async () => {
|
||||
apiJson.mockResolvedValue(statusWith(false, { state: 'error', error: 'boom' }));
|
||||
apiFetch.mockResolvedValue({
|
||||
ok: false,
|
||||
status: 404,
|
||||
json: async () => ({
|
||||
detail: 'No system ffmpeg found on PATH or in the usual install locations.',
|
||||
}),
|
||||
});
|
||||
render(<MediaEngineCard />);
|
||||
fireEvent.click(await screen.findByTestId('media-engine-use-system'));
|
||||
await waitFor(() =>
|
||||
expect(apiFetch).toHaveBeenCalledWith('/media-tools/ffmpeg/use-system', expect.anything()),
|
||||
);
|
||||
expect(await screen.findByTestId('media-engine-error')).toHaveTextContent(
|
||||
'No system ffmpeg found',
|
||||
);
|
||||
});
|
||||
|
||||
it('Choose file… falls back to an inline path input outside Tauri and saves it', async () => {
|
||||
apiJson.mockResolvedValue(statusWith(false, { state: 'error', error: 'boom' }));
|
||||
render(<MediaEngineCard />);
|
||||
fireEvent.click(await screen.findByText('Choose file…'));
|
||||
const input = await screen.findByTestId('media-engine-path');
|
||||
fireEvent.change(input, { target: { value: '/usr/local/bin/ffmpeg' } });
|
||||
fireEvent.click(screen.getByText('Save'));
|
||||
await waitFor(() =>
|
||||
expect(apiFetch).toHaveBeenCalledWith(
|
||||
'/media-tools/ffmpeg/custom-path',
|
||||
expect.objectContaining({ body: JSON.stringify({ path: '/usr/local/bin/ffmpeg' }) }),
|
||||
),
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -73,9 +73,6 @@ export default function NavRail({ mode, setMode, side = 'left', onFlipSide }) {
|
||||
[t],
|
||||
);
|
||||
|
||||
const donateLabel = t('donate.pill', { defaultValue: 'Support OmniVoice' });
|
||||
const donateActive = mode === 'donate';
|
||||
|
||||
// `nav-rail` is retained purely as the layout hook the (out-of-scope)
|
||||
// `.app-container > .nav-rail` grid rules position by; all visual styling now
|
||||
// lives in the utilities below. Border flips to the inner edge when on the right.
|
||||
@@ -84,17 +81,6 @@ export default function NavRail({ mode, setMode, side = 'left', onFlipSide }) {
|
||||
? '[border-left:1px_solid_var(--chrome-border)]'
|
||||
: '[border-right:1px_solid_var(--chrome-border)]';
|
||||
|
||||
// Quiet "Support" pill (was `.rail-btn.donate-pill`): neutral at rest, warms to
|
||||
// the accent on hover/active.
|
||||
const donateState = donateActive
|
||||
? 'text-[var(--chrome-accent)] bg-[var(--chrome-accent-bg)] [border:1px_solid_var(--chrome-accent-border)]'
|
||||
: 'bg-transparent text-[var(--chrome-fg-dim)] [border:1px_solid_transparent] hover:bg-[color-mix(in_srgb,var(--chrome-accent)_10%,transparent)] hover:text-[var(--chrome-accent)]';
|
||||
const heartBase =
|
||||
'text-[16px] leading-none [transition:filter_0.16s,opacity_0.16s,transform_0.16s] group-hover:[transform:scale(1.1)] motion-reduce:[transition:none] motion-reduce:group-hover:[transform:none]';
|
||||
const heartState = donateActive
|
||||
? 'opacity-100 [filter:grayscale(0)]'
|
||||
: 'opacity-75 [filter:grayscale(0.55)] group-hover:opacity-100 group-hover:[filter:grayscale(0)]';
|
||||
|
||||
return (
|
||||
<aside
|
||||
className={`nav-rail z-50 flex select-none flex-col items-center gap-[6px] bg-[var(--chrome-bg)] py-[8px] ${asideBorder}`}
|
||||
@@ -111,19 +97,6 @@ export default function NavRail({ mode, setMode, side = 'left', onFlipSide }) {
|
||||
))}
|
||||
</div>
|
||||
<div className="flex flex-col items-center gap-[4px]">
|
||||
{/* Quiet "Support" pill — warms to the accent on hover, opens the
|
||||
donate page. Sits with the footer nav (Settings / flip). (#007) */}
|
||||
<button
|
||||
onClick={() => setMode('donate')}
|
||||
title={donateLabel}
|
||||
aria-label={donateLabel}
|
||||
className={`${RAIL_BTN_BASE} ${donateState}`}
|
||||
>
|
||||
<span className={`${heartBase} ${heartState}`} aria-hidden="true">
|
||||
🩷
|
||||
</span>
|
||||
<span className={railLabelCls(side)}>{donateLabel}</span>
|
||||
</button>
|
||||
{footerItems.map((it) => (
|
||||
<RailBtn
|
||||
key={it.id}
|
||||
|
||||
@@ -54,6 +54,7 @@ import {
|
||||
import { parseScript } from '../utils/parseScript';
|
||||
import { importToText } from '../utils/importStory';
|
||||
import { generateSpeech, audioUrl } from '../api/generate';
|
||||
import { playBlobAudio } from '../utils/media';
|
||||
import { encodeAudio } from '../api/stories';
|
||||
import { longformRender } from '../api/audiobook';
|
||||
import { exportStems } from '../utils/storyExport';
|
||||
@@ -423,14 +424,6 @@ export default function StoriesEditor({ profiles = [] }) {
|
||||
return res.blob();
|
||||
}, []);
|
||||
|
||||
const fetchChunkAudio = useCallback(
|
||||
async (text, profileId, speed = 1.0) => {
|
||||
const blob = await fetchChunkBlob(text, profileId, speed);
|
||||
return URL.createObjectURL(blob);
|
||||
},
|
||||
[fetchChunkBlob],
|
||||
);
|
||||
|
||||
const previewTrack = useCallback(
|
||||
async (track) => {
|
||||
const raw = (track.text || '').trim();
|
||||
@@ -443,14 +436,18 @@ export default function StoriesEditor({ profiles = [] }) {
|
||||
|
||||
if (!hasStoryMarkers(raw)) {
|
||||
try {
|
||||
const url = await fetchChunkAudio(raw, pid, spd);
|
||||
const blob = await fetchChunkBlob(raw, pid, spd);
|
||||
const url = URL.createObjectURL(blob);
|
||||
setTracks((prev) =>
|
||||
prev.map((tk) =>
|
||||
tk.id === track.id ? { ...tk, audioUrl: url, generating: false } : tk,
|
||||
),
|
||||
);
|
||||
const audio = new Audio(url);
|
||||
audio.play().catch(() => {});
|
||||
// Shared playback path (labelled with the line text): registers with
|
||||
// the single-playback manager + global mini-player, and — unlike the
|
||||
// old bare `new Audio(blobUrl)` — actually plays under Tauri's
|
||||
// WebKit, where blob: URLs are dead in media elements.
|
||||
playBlobAudio(blob, { label: raw }).catch(() => {});
|
||||
} catch (err) {
|
||||
console.warn('Stories preview failed:', err);
|
||||
setTracks((prev) =>
|
||||
@@ -462,17 +459,15 @@ export default function StoriesEditor({ profiles = [] }) {
|
||||
|
||||
const parsed = parseStoryText(raw, pid);
|
||||
try {
|
||||
const audioUrls = await Promise.all(
|
||||
const chunkBlobs = await Promise.all(
|
||||
parsed.map((seg) =>
|
||||
seg.type === 'chunk'
|
||||
? fetchChunkAudio(seg.text, seg.profileId, spd)
|
||||
? fetchChunkBlob(seg.text, seg.profileId, spd)
|
||||
: Promise.resolve(null),
|
||||
),
|
||||
);
|
||||
let cursor = 0;
|
||||
const finish = () => {
|
||||
for (let i = cursor; i < audioUrls.length; i++)
|
||||
if (audioUrls[i]) URL.revokeObjectURL(audioUrls[i]);
|
||||
setTracks((prev) =>
|
||||
prev.map((tk) =>
|
||||
tk.id === track.id ? { ...tk, generating: false, audioUrl: null } : tk,
|
||||
@@ -482,26 +477,21 @@ export default function StoriesEditor({ profiles = [] }) {
|
||||
const step = () => {
|
||||
while (cursor < parsed.length) {
|
||||
const seg = parsed[cursor];
|
||||
const url = audioUrls[cursor];
|
||||
const blob = chunkBlobs[cursor];
|
||||
cursor++;
|
||||
if (seg.type === 'pause') {
|
||||
setTimeout(step, seg.seconds * 1000);
|
||||
return;
|
||||
}
|
||||
if (seg.type === 'chunk' && url) {
|
||||
const audio = new Audio(url);
|
||||
audio.onended = () => {
|
||||
URL.revokeObjectURL(url);
|
||||
step();
|
||||
};
|
||||
audio.onerror = () => {
|
||||
URL.revokeObjectURL(url);
|
||||
step();
|
||||
};
|
||||
audio.play().catch(() => {
|
||||
URL.revokeObjectURL(url);
|
||||
step();
|
||||
});
|
||||
if (seg.type === 'chunk' && blob) {
|
||||
// Chained through the shared playback path: each chunk claims
|
||||
// the global manager (mini-player shows the line), a natural
|
||||
// end (or a broken chunk) advances the chain, and stopping from
|
||||
// the player/another claim cancels the rest of the chain.
|
||||
playBlobAudio(blob, {
|
||||
label: raw,
|
||||
onDone: (reason) => (reason === 'stopped' ? finish() : step()),
|
||||
}).catch(() => step());
|
||||
return;
|
||||
}
|
||||
}
|
||||
@@ -515,7 +505,7 @@ export default function StoriesEditor({ profiles = [] }) {
|
||||
);
|
||||
}
|
||||
},
|
||||
[fetchChunkAudio, cast, globalSpeed, setTracks],
|
||||
[fetchChunkBlob, cast, globalSpeed, setTracks],
|
||||
);
|
||||
|
||||
// Deliver a stitched WAV in the chosen format. MP3 routes through the backend
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user