Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
67789fb31c | ||
|
|
da4bef8e42 | ||
|
|
0076d0067e | ||
|
|
faf34348c8 | ||
|
|
69ce697ee5 | ||
|
|
93a3cb260a | ||
|
|
33379890ad | ||
|
|
7f8a42ce51 | ||
|
|
4de3c824d0 | ||
|
|
11fbbd3f6d | ||
|
|
f546f04c8e | ||
|
|
270b3b1c4c | ||
|
|
5a7d9cc05c |
@@ -149,7 +149,7 @@ jobs:
|
||||
- os: windows-2022
|
||||
label: Windows
|
||||
rust_target: x86_64-pc-windows-msvc
|
||||
- os: ubuntu-22.04
|
||||
- os: ubuntu-24.04
|
||||
label: Linux
|
||||
rust_target: x86_64-unknown-linux-gnu
|
||||
runs-on: ${{ matrix.os }}
|
||||
|
||||
@@ -213,13 +213,24 @@ jobs:
|
||||
bundles: "msi,updater"
|
||||
|
||||
# Linux: ship .AppImage only. AppImage is universal (no distro
|
||||
# package-manager dep), runs on any glibc-2.31+ host, and is the
|
||||
# package-manager dep), runs on any glibc-2.39+ host, and is the
|
||||
# Linux auto-update target. The .deb target was dropped: tauri-bundler
|
||||
# fails it with "Failed to create control scripts: No such file or
|
||||
# directory" (no custom deb config of ours is at fault) — revisit on a
|
||||
# tauri-cli bump. FUSE unavailability on GH runners is handled via
|
||||
# APPIMAGE_EXTRACT_AND_RUN=1.
|
||||
- os: ubuntu-22.04
|
||||
#
|
||||
# Bumped from ubuntu-22.04 → ubuntu-24.04 (#961): the AppImage
|
||||
# bundles whatever `libwebkit2gtk-4.1-dev` the build runner's apt
|
||||
# repos resolve (see the "Linux system deps" step below) — 22.04's
|
||||
# was meaningfully stale relative to what current Ubuntu/Fedora
|
||||
# ship, and AppRun's LD_LIBRARY_PATH makes that bundled, stale copy
|
||||
# take priority over a healthy system WebKitGTK at runtime. Raises
|
||||
# the AppImage's glibc floor from 2.35 to 2.39 — pre-2022 distros
|
||||
# (Ubuntu <22.04, Debian <12) lose support; no report of anyone on
|
||||
# something that old has come in, and the project's own install
|
||||
# docs already assume Debian 12 / Ubuntu 22.04+.
|
||||
- os: ubuntu-24.04
|
||||
arch: x86_64-unknown-linux-gnu
|
||||
label: "Linux x64"
|
||||
rust_target: x86_64-unknown-linux-gnu
|
||||
|
||||
@@ -8,6 +8,34 @@ The bundled TTS model package (`pyproject.toml`) is versioned independently.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [0.3.13] — 2026-07-09
|
||||
|
||||
The community-fixes release. Two contributors didn't just report bugs — they diagnosed them to the exact line and submitted the fixes that shipped: **voice cloning on mlx-audio's CSM model works for the first time**, and **macOS live recording finally gets its microphone permission prompt** (both @MahdiHedhli). A third reporter's A/B analysis fixed **cross-language dubs speaking the wrong language**. On top of that: a backend shutdown race that produced confusing crash-on-quit reports is fixed, the Linux AppImage stops shipping a stale WebKitGTK that white-screened current distros, and a new OpenAI-compatible transcription backend opens a path to Qwen3-ASR today. Thank you to everyone who filed, diagnosed, and contributed — this release is mostly yours.
|
||||
|
||||
### Added
|
||||
|
||||
- **A path to Qwen3-ASR today: generic OpenAI-compatible transcription.** The direct integration is still blocked on `transformers>=5.13` stabilizing upstream, but a community member proposed splitting the work — add a backend that talks to any OpenAI-compatible transcription server right now. Point OmniVoice at a self-hosted Qwen3-ASR/FunASR/SenseVoice server, or OpenAI's own API, configured in Settings → Models. No install; audio does leave your machine to whichever server you configure, unlike every other ASR engine. (#877)
|
||||
|
||||
### Fixed
|
||||
|
||||
- **The Linux AppImage no longer white-screens on current distros with a healthy system WebKitGTK.** The release build ran on an older CI base image, and the resulting AppImage bundles whatever `libwebkit2gtk` that image's apt repos resolve — which the AppImage's own `LD_LIBRARY_PATH` then prioritizes over your system's newer, healthy copy at runtime. A from-source build (which links straight against your system library) worked fine on the exact same machine where the shipped AppImage didn't — that split was the tell. Bumped the release build to a current Ubuntu LTS. Raises the AppImage's minimum host to glibc 2.39 (Ubuntu 24.04+); no reports from anyone on an older distro. (#961)
|
||||
- **Backend shutdown no longer races a still-loading model, surfacing a confusing crash on restart.** Quitting the app while a model was still loading in the background let shutdown report itself "done" while a background thread was still mid-import; tearing the process down under that thread produced a misleading error (a generic transformers import-failure message, unrelated to the real cause) that looked like a real crash rather than a timing issue. All background tasks are now properly cancelled and awaited before shutdown proceeds. (#1000, likely the same class behind #941 and #979)
|
||||
- **Cross-language dub no longer speaks the source-language reference line verbatim.** Auto-generated speaker clones pair an audio slice with the ASR segment's own text field, assuming the two agree — but ASR segment text and its timestamps routinely drift (a trailing word audible in the clip but missing from the text, or vice versa). A mismatched (reference audio, reference text) pair breaks zero-shot TTS prompt priming badly enough that the clone can emit the reference text itself instead of the target-language line it was asked to speak. Each reference clip is now re-transcribed after it's written, so the pair matches by construction — reported with an exceptionally clear root-cause diagnosis and a working A/B repro. (#1004)
|
||||
- **Voice Gallery errors now say what actually went wrong.** "Use voice", "Preview", search, upload, save, delete, and trim in the Gallery all showed the same hardcoded guess ("the engine may be loading") on ANY failure — a 500, a validation error, a genuinely unrelated bug — discarding the real, already-clean backend error message in the process. Every one of those now shows the actual error.
|
||||
- **Voice cloning on mlx-audio's CSM model no longer crashes with an opaque "list index out of range".** `MLXAudioBackend.generate()` read `voice`/`ref_audio`/`language`/`speed` from its kwargs but silently dropped `ref_text` — CSM only builds its cloning context when both `ref_audio` and `ref_text` are present, so cloning on this engine could never have worked as shipped. Reported with the exact root cause and a working fix. (#1012, #1013)
|
||||
- **A dub segment's free-text style tags no longer 400 the segment preview.** A validator-safe instruct builder already keeps Studio and Clone generation from round-tripping a 400 on unsupported free-text (a preset's raw attrs, an old profile's stray descriptive phrase) — but the Dub tab's segment preview, and saving a profile from a clone or from history, built their instruct strings directly and skipped it. Same guard now applies everywhere an instruct string is sent. (#1010)
|
||||
- **The dub editor's play button no longer sticks permanently disabled after an audio-decode hiccup.** When the initial WaveSurfer decode fails, the timeline falls back to loading pre-computed peaks — the waveform draws fine, but the button's enabled state only relied on the `ready` event firing again for that recovery load, which it didn't reliably do. Each fallback path now confirms readiness explicitly once it settles.
|
||||
- **macOS: live recording finally works — the microphone permission prompt now actually appears.** The app never showed up in System Settings → Privacy & Security → Microphone because macOS never saw a legitimate request: Tauri enables Hardened Runtime by default, which blocks microphone hardware access unless the matching entitlement is in the signed bundle — and it wasn't. Diagnosed to the exact mechanism and fixed by a community contributor (@MahdiHedhli), who also corrected our initial mis-read of this as an upstream WebKit limitation. (#1013, #1016)
|
||||
- **Quitting during a slow model load waits longer before giving up.** A post-merge code review of the shutdown-race fix flagged that its 3-second wait could still be outrun by a cold model import on a slow disk, reproducing the original confusing-crash-on-quit in rare cases. The wait is now 20 seconds — imperceptible on a normal quit (tasks finish or cancel in milliseconds), only felt in the exact case it protects. (#1020)
|
||||
|
||||
### Changed
|
||||
|
||||
- **Removed the donate heart from the nav rail.** Support OmniVoice is still one click away from Settings and the Contact page.
|
||||
|
||||
### CI
|
||||
|
||||
- **The "flaky trio" is root-caused and neutralized.** Three tests failed intermittently on CI — never locally — across unrelated PRs, costing a re-run each time. Cause: a leaked half-precision torch default from some earlier test in CI's ordering (the giveaway: a failing assertion's observed value was exactly float16(0.1)). An autouse test-suite guard now resets the leak between tests and names the offending test in CI output when it fires. (#1021)
|
||||
|
||||
## [0.3.12] — 2026-07-08
|
||||
|
||||
A community-issue sweep — nineteen open reports triaged in one pass, most fixed same-day. The through-line: **your active engine selection is now honored everywhere** (dubbing, batch, and — new in this release — MLX-Audio's own curated models are finally selectable instead of always silently defaulting to Kokoro), **first-run stops dead-ending users on restricted networks or behind corporate TLS proxies**, and a run of sharp community diagnoses (a one-line ROCm index fix, a Windows-only focus-stealing bug, a genuine crash regression) got fixed largely because reporters did the hard diagnostic work themselves. Thank you.
|
||||
|
||||
@@ -266,7 +266,7 @@ Professional-grade voice AI, minus the subscription and the cloud.
|
||||
|
||||
| | **Minimum** | **Recommended** |
|
||||
|---|---|---|
|
||||
| **OS** | Windows 10, macOS 12+ (Apple Silicon), Ubuntu 20.04+ | Any modern 64-bit OS |
|
||||
| **OS** | Windows 10, macOS 12+ (Apple Silicon), Ubuntu 24.04+ (glibc 2.39+) | Any modern 64-bit OS |
|
||||
| **RAM** | 8 GB | 16 GB+ |
|
||||
| **VRAM (GPU)** | 4 GB (auto-offloads TTS to CPU) | 8 GB+ (NVIDIA RTX 3060+) |
|
||||
| **Disk** | 10 GB free (models + cache) | 20 GB+ SSD |
|
||||
@@ -322,10 +322,10 @@ Professional-grade voice AI, minus the subscription and the cloud.
|
||||
|
||||
### 🎧 ASR Engines
|
||||
|
||||
**9 engines, all fully local** — they power dictation, video dubbing, and subtitles. **WhisperX** is the cross-platform default (~100 languages, word-level timing); the rest are opt-in and auto-detected. Switch in **Settings → ASR Engine** or via the `OMNIVOICE_ASR_BACKEND` env var.
|
||||
**10 engines** — they power dictation, video dubbing, and subtitles. **WhisperX** is the cross-platform default (~100 languages, word-level timing); the rest are opt-in and auto-detected. Switch in **Settings → ASR Engine** or via the `OMNIVOICE_ASR_BACKEND` env var. Nine run fully on-device; one (OpenAI-compatible) is an optional remote client for pointing at Qwen3-ASR or another compatible server — see below.
|
||||
|
||||
<details>
|
||||
<summary><b>📊 The full lineup</b> — 9 engines, what each is best at, and compute-type notes</summary>
|
||||
<summary><b>📊 The full lineup</b> — 10 engines, what each is best at, and compute-type notes</summary>
|
||||
|
||||
<br/>
|
||||
|
||||
@@ -340,6 +340,7 @@ Professional-grade voice AI, minus the subscription and the cloud.
|
||||
| **Moonshine** | `moonshine` | English | Edge / low-latency, ONNX |
|
||||
| **FunASR** | `funasr` | 50+ | All-in-one multilingual — built-in VAD + inline speaker diarization (SenseVoice) |
|
||||
| **sherpa-onnx** (live dictation) | `sherpa-onnx-asr` | 25 EU + 90+ | Live, faster-than-real-time dictation — small streaming/offline ONNX models (Parakeet TDT v3/v2, streaming Zipformer & Paraformer, Whisper Tiny), CPU, identical on macOS / Windows / Linux. Picked per-model in **Settings → Voice**. |
|
||||
| **OpenAI-compatible** ⚠️ remote | `openai-compat-asr` | Server-dependent | A path to **Qwen3-ASR** today (self-hosted server, no transformers wait), any OpenAI-compatible transcription endpoint, or OpenAI's own API — no install, configure in **Settings → Models**. Audio leaves your machine to whatever server you point it at; see [docs/engines/openai-compatible-asr.md](docs/engines/openai-compatible-asr.md). |
|
||||
|
||||
> Whisper-family engines cover ~100 languages; **FunASR / SenseVoice** adds an all-in-one multilingual path with built-in voice-activity detection and inline speaker diarization. **sherpa-onnx** powers the live dictation model picker — you talk and text appears as you speak. Every engine runs on-device — no API keys, no cloud.
|
||||
|
||||
|
||||
@@ -1006,6 +1006,11 @@ async def dub_transcribe_stream(
|
||||
clones = done.pop().result()
|
||||
break
|
||||
yield _sse_event("ping", {})
|
||||
if clones:
|
||||
from services.speaker_clone import refine_ref_texts
|
||||
clones = await loop.run_in_executor(
|
||||
_gpu_pool, lambda: refine_ref_texts(clones, _asr_backend),
|
||||
)
|
||||
# Wave 3.2: per-segment clone refs. Cut each long-enough segment's
|
||||
# own reference from the vocals so the dub of each line matches the
|
||||
# prosody of its source line. Short lines fall back to the
|
||||
@@ -1025,6 +1030,10 @@ async def dub_transcribe_stream(
|
||||
),
|
||||
)
|
||||
if seg_clones:
|
||||
from services.speaker_clone import refine_ref_texts
|
||||
seg_clones = await loop.run_in_executor(
|
||||
_gpu_pool, lambda: refine_ref_texts(seg_clones, _asr_backend),
|
||||
)
|
||||
job["segment_clones"] = seg_clones
|
||||
except Exception as e:
|
||||
logger.warning("per-segment clone refs skipped: %s", e)
|
||||
|
||||
@@ -750,6 +750,51 @@ def set_hf_mirror(body: _HFMirrorBody):
|
||||
return {"configured": url, "restart_required": changed, "presets": _HF_MIRROR_PRESETS}
|
||||
|
||||
|
||||
# ── OpenAI-compatible remote ASR (#877) ─────────────────────────────────────
|
||||
# A path to Qwen3-ASR/FunASR/SenseVoice — or OpenAI's own Whisper API — today,
|
||||
# without waiting on transformers to ship a direct Qwen3-ASR integration.
|
||||
# base_url/model are plain settings_store text rows; the key is encrypted via
|
||||
# settings_store.set_secret — same convention as /llm-providers, never
|
||||
# returned to the client, '' clears it, omitted/None leaves it unchanged.
|
||||
|
||||
|
||||
class _ASROpenAICompatBody(BaseModel):
|
||||
base_url: str | None = None
|
||||
model: str | None = None
|
||||
api_key: str | None = Field(None, description="'' clears it, None leaves unchanged")
|
||||
|
||||
|
||||
@router.get("/asr-openai-compat")
|
||||
def get_asr_openai_compat():
|
||||
from services import asr_backend
|
||||
|
||||
return {
|
||||
"base_url": asr_backend.resolve_openai_compat_asr_base_url(),
|
||||
"model": asr_backend.resolve_openai_compat_asr_model(),
|
||||
"has_key": asr_backend.openai_compat_asr_has_key(),
|
||||
}
|
||||
|
||||
|
||||
@router.put("/asr-openai-compat")
|
||||
def set_asr_openai_compat(body: _ASROpenAICompatBody):
|
||||
from services import asr_backend, settings_store
|
||||
|
||||
if body.base_url is not None:
|
||||
url = body.base_url.strip().rstrip("/")
|
||||
if url and not url.startswith(("http://", "https://")):
|
||||
raise HTTPException(status_code=400, detail="Base URL must start with http(s)://")
|
||||
settings_store.set_text(asr_backend._ASR_OPENAI_COMPAT_BASE_URL_KEY, url)
|
||||
if body.model is not None:
|
||||
settings_store.set_text(
|
||||
asr_backend._ASR_OPENAI_COMPAT_MODEL_KEY, body.model.strip() or "whisper-1"
|
||||
)
|
||||
if body.api_key is not None:
|
||||
settings_store.set_secret(
|
||||
asr_backend._ASR_OPENAI_COMPAT_SECRET_NAME, body.api_key.strip()
|
||||
)
|
||||
return get_asr_openai_compat()
|
||||
|
||||
|
||||
# ── Updates panel: shipped changelog + pre-migration DB backup state ────────
|
||||
# (feat/safe-updates). Both are read-only, local-first surfaces for
|
||||
# Settings → Updates: the "What's new" viewer reads the CHANGELOG.md that
|
||||
|
||||
@@ -24,7 +24,7 @@ from pathlib import Path
|
||||
# tests/test_app_version.py::test_all_version_files_in_lockstep and bumped by
|
||||
# release.yml's version-bump job, so it stays equal to
|
||||
# pyproject/tauri.conf/Cargo/package.json.
|
||||
_FALLBACK_VERSION = "0.3.12"
|
||||
_FALLBACK_VERSION = "0.3.13"
|
||||
|
||||
|
||||
def _fallback_version() -> str:
|
||||
|
||||
+61
-8
@@ -504,6 +504,35 @@ async def _start_mcp_session_manager(session_manager, *, timeout: float):
|
||||
return task, stop, mounted
|
||||
|
||||
|
||||
async def _cancel_and_await_tasks(*tasks, timeout: float = 3.0) -> None:
|
||||
"""Cancel each background task and give it a bounded chance to actually
|
||||
finish before shutdown proceeds — ``None`` entries are skipped (a task
|
||||
that's conditionally created, e.g. ``capture_preload_task``, may not
|
||||
exist).
|
||||
|
||||
``task.cancel()`` alone is not enough for a task awaiting
|
||||
``run_in_executor()``: once the underlying OS thread is inside blocking
|
||||
native/import work, cancellation can't stop it, so cancel-and-move-on lets
|
||||
shutdown finish while that thread is still running — invisible to
|
||||
asyncio, but very much alive when the interpreter starts tearing down
|
||||
module state under it (#1000 class). Awaiting with a bound (instead of
|
||||
just cancelling) gives an early-stage task a real chance to exit cleanly
|
||||
first; a task that's genuinely still deep in blocking work times out here
|
||||
same as before, and the caller's own GPU-pool reset handles that case.
|
||||
"""
|
||||
for t in tasks:
|
||||
if t is None:
|
||||
continue
|
||||
t.cancel()
|
||||
for t in tasks:
|
||||
if t is None:
|
||||
continue
|
||||
try:
|
||||
await asyncio.wait_for(t, timeout=timeout)
|
||||
except (asyncio.CancelledError, asyncio.TimeoutError):
|
||||
pass
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
# Startup watchdog (#632): a silent hang during startup (e.g. a model-load /
|
||||
@@ -578,6 +607,7 @@ async def lifespan(app: FastAPI):
|
||||
# lean and the first dictation is instant instead of a cold model load.
|
||||
# OMNIVOICE_PRELOAD_CAPTURE_ASR=0 opts out; the warm-up is also skipped
|
||||
# under 4 GB free RAM (checked at warm time, not boot time).
|
||||
capture_preload_task = None # only assigned when the preload actually runs (#1000 class)
|
||||
if _env_flag("OMNIVOICE_PRELOAD_CAPTURE_ASR", default=True):
|
||||
async def _preload_capture_asr():
|
||||
await asyncio.sleep(_capture_preload_delay_s())
|
||||
@@ -646,14 +676,33 @@ async def lifespan(app: FastAPI):
|
||||
pass
|
||||
except Exception:
|
||||
pass
|
||||
idle_task.cancel()
|
||||
worker_task.cancel()
|
||||
# Wait for tasks to finish their current iteration
|
||||
for t in (idle_task, worker_task):
|
||||
try:
|
||||
await asyncio.wait_for(t, timeout=3.0)
|
||||
except (asyncio.CancelledError, asyncio.TimeoutError):
|
||||
pass
|
||||
# preload_task/capture_preload_task matter most here (#1000 class): a quit
|
||||
# mid-preload used to fall straight through to "Shutdown: done." while the
|
||||
# model load was still running on a GPU-pool thread — cancel() can't stop
|
||||
# a thread already inside blocking import/load work, so the process
|
||||
# reported a clean exit while that background thread was still mid-
|
||||
# `import transformers`, and got torn down by interpreter finalization
|
||||
# instead. That surfaced as a misleading "Could not import module
|
||||
# 'AutoFeatureExtractor'" — transformers' own generic lazy-import wrapper,
|
||||
# not a real dependency problem. Awaiting here lets an early-stage load
|
||||
# (still importing, not yet mid weight-download) finish cleanly before we
|
||||
# report done; a load that's genuinely deep into a multi-GB download still
|
||||
# times out — _reset_gpu_pool() below abandons it either way.
|
||||
#
|
||||
# 20s, not the original 3s (code-review finding post-merge): a cold
|
||||
# transformers import alone can take longer than 3s on a slow disk or a
|
||||
# first-ever launch, so the original bound left a real residual window —
|
||||
# cancellation detaches the asyncio task, but the underlying OS thread
|
||||
# keeps running past it, and shutdown could still report "done" while
|
||||
# that thread was alive. Python cannot forcibly kill a running thread, so
|
||||
# no finite bound eliminates this outright — 20s just shrinks the window
|
||||
# from "any preload" to "an unusually slow cold-import," which is the
|
||||
# practical ceiling before a longer shutdown itself becomes the
|
||||
# complaint. A thread that's still running past 20s was never going to
|
||||
# finish in a shutdown-appropriate timeframe regardless.
|
||||
await _cancel_and_await_tasks(
|
||||
idle_task, worker_task, preload_task, capture_preload_task, timeout=20.0,
|
||||
)
|
||||
# Unload the model and free GPU memory
|
||||
try:
|
||||
import services.model_manager as mm
|
||||
@@ -661,6 +710,10 @@ async def lifespan(app: FastAPI):
|
||||
mm.model = None
|
||||
logger.info("Shutdown: model unloaded.")
|
||||
mm.free_vram()
|
||||
# Abandon a still-running preload's GPU-pool thread (Python can't kill
|
||||
# a thread mid blocking call) so it can't outlive this shutdown block
|
||||
# holding a reference into module state that's about to be torn down.
|
||||
mm._reset_gpu_pool()
|
||||
except Exception:
|
||||
pass
|
||||
# Run GC to release any remaining references
|
||||
|
||||
@@ -29,6 +29,7 @@ import os
|
||||
import re
|
||||
import threading
|
||||
from abc import ABC, abstractmethod
|
||||
from typing import Optional
|
||||
|
||||
logger = logging.getLogger("omnivoice.asr")
|
||||
|
||||
@@ -1634,6 +1635,161 @@ class FunASRBackend(ASRBackend):
|
||||
pass
|
||||
|
||||
|
||||
# ── OpenAI-compatible remote transcription (#877 — Qwen3-ASR / FunASR / any
|
||||
# compatible server, today, without waiting on transformers to catch up) ──
|
||||
#
|
||||
# transformers doesn't yet ship a stable Qwen3-ASR integration (issue #877),
|
||||
# but a self-hosted Qwen3-ASR/FunASR/SenseVoice server exposing an
|
||||
# OpenAI-compatible `POST /v1/audio/transcriptions` endpoint — or OpenAI's own
|
||||
# Whisper API — is usable right now. This backend is a pure network client:
|
||||
# no model runs locally, so it needs no install and claims no GPU.
|
||||
#
|
||||
# Settings mirror the LLM-providers convention exactly (services/
|
||||
# llm_providers.py): base_url/model are plain settings_store text rows; the
|
||||
# API key is Fernet-encrypted via settings_store.set_secret/get_secret — never
|
||||
# a .env row, never echoed back to the client. Optional: some self-hosted
|
||||
# servers (vLLM, LM Studio-style) don't check the key at all.
|
||||
|
||||
_ASR_OPENAI_COMPAT_BASE_URL_KEY = "asr.openai_compat.base_url"
|
||||
_ASR_OPENAI_COMPAT_MODEL_KEY = "asr.openai_compat.model"
|
||||
_ASR_OPENAI_COMPAT_SECRET_NAME = "asr_openai_compat_key"
|
||||
|
||||
|
||||
def resolve_openai_compat_asr_base_url() -> str:
|
||||
from services import settings_store
|
||||
return (
|
||||
os.environ.get("ASR_OPENAI_COMPAT_BASE_URL")
|
||||
or settings_store.get_text(_ASR_OPENAI_COMPAT_BASE_URL_KEY)
|
||||
or ""
|
||||
)
|
||||
|
||||
|
||||
def resolve_openai_compat_asr_model() -> str:
|
||||
from services import settings_store
|
||||
return (
|
||||
os.environ.get("ASR_OPENAI_COMPAT_MODEL")
|
||||
or settings_store.get_text(_ASR_OPENAI_COMPAT_MODEL_KEY)
|
||||
or "whisper-1"
|
||||
)
|
||||
|
||||
|
||||
def resolve_openai_compat_asr_api_key() -> Optional[str]:
|
||||
"""Env → encrypted stored key → None. Unlike LLM providers, no 'local'
|
||||
sentinel: many self-hosted transcription servers accept an empty/omitted
|
||||
Authorization header outright, so the OpenAI SDK is constructed with
|
||||
``api_key="not-needed"`` (a non-empty placeholder the SDK requires) when
|
||||
this returns None, rather than treating a keyless server as unconfigured.
|
||||
"""
|
||||
from services import settings_store
|
||||
return os.environ.get("ASR_OPENAI_COMPAT_API_KEY") or settings_store.get_secret(
|
||||
_ASR_OPENAI_COMPAT_SECRET_NAME
|
||||
)
|
||||
|
||||
|
||||
def openai_compat_asr_has_key() -> bool:
|
||||
"""Whether a key is configured, without ever decrypting it — mirrors
|
||||
llm_providers.has_key()'s no-plaintext-round-trip contract."""
|
||||
from services import settings_store
|
||||
if os.environ.get("ASR_OPENAI_COMPAT_API_KEY"):
|
||||
return True
|
||||
return _ASR_OPENAI_COMPAT_SECRET_NAME in settings_store.list_secret_names()
|
||||
|
||||
|
||||
class OpenAICompatASRBackend(ASRBackend):
|
||||
"""Remote transcription via any OpenAI-compatible server.
|
||||
|
||||
Adapts whatever the server returns into this module's expected shape.
|
||||
Prefers `response_format="verbose_json"` for real per-segment timestamps
|
||||
(OpenAI's own API and most compatible servers support it); falls back to
|
||||
plain text with rough single-segment bounds — mirroring
|
||||
MoonshineASRBackend's degraded shape — for minimal servers that reject it.
|
||||
"""
|
||||
id = "openai-compat-asr"
|
||||
display_name = "OpenAI-compatible (remote server)"
|
||||
gpu_compat = ("cpu",) # network client only — no local compute
|
||||
|
||||
def __init__(self):
|
||||
self._base_url = resolve_openai_compat_asr_base_url()
|
||||
self._model = resolve_openai_compat_asr_model()
|
||||
|
||||
@classmethod
|
||||
def is_available(cls) -> tuple[bool, str]:
|
||||
if not resolve_openai_compat_asr_base_url():
|
||||
return False, "Configure a server endpoint in Settings → Engines"
|
||||
try:
|
||||
import openai # noqa: F401
|
||||
except ImportError:
|
||||
return False, "openai package not installed. Install with: uv pip install openai"
|
||||
return True, "ready"
|
||||
|
||||
def _client(self):
|
||||
from openai import OpenAI
|
||||
api_key = resolve_openai_compat_asr_api_key() or "not-needed"
|
||||
# max_retries=0: mirrors llm_skills.resolve_skill_client — a
|
||||
# rate-limited/slow server retrying inside the SDK would blow past
|
||||
# whatever bounded timeout the caller (dub transcribe, dictation)
|
||||
# expects from a single call.
|
||||
return OpenAI(base_url=self._base_url, api_key=api_key, max_retries=0)
|
||||
|
||||
def transcribe(self, audio_path: str, *, word_timestamps: bool = True) -> dict:
|
||||
logger.info(
|
||||
"OpenAI-compat ASR transcribing %s (base_url=%s, model=%s)",
|
||||
audio_path, self._base_url, self._model,
|
||||
)
|
||||
client = self._client()
|
||||
try:
|
||||
with open(audio_path, "rb") as f:
|
||||
try:
|
||||
resp = client.audio.transcriptions.create(
|
||||
file=f, model=self._model, response_format="verbose_json",
|
||||
)
|
||||
except Exception:
|
||||
# Minimal/older compatible servers reject verbose_json
|
||||
# outright — retry plain before treating it as a real
|
||||
# failure. Re-open: the SDK may have partially consumed
|
||||
# the file handle on the first attempt.
|
||||
f.seek(0)
|
||||
resp = client.audio.transcriptions.create(
|
||||
file=f, model=self._model, response_format="json",
|
||||
)
|
||||
except Exception as exc:
|
||||
# Never leak a raw SDK/httpx exception object (auth headers,
|
||||
# connection internals) straight into a user-facing message —
|
||||
# same convention as generation.py's _safe_exc_text (#977 class).
|
||||
raise RuntimeError(
|
||||
f"OpenAI-compatible ASR server at {self._base_url!r} failed: "
|
||||
f"{type(exc).__name__}: {exc}"
|
||||
) from exc
|
||||
return self._adapt_response(resp)
|
||||
|
||||
@staticmethod
|
||||
def _adapt_response(resp) -> dict:
|
||||
segments_out = []
|
||||
# verbose_json: resp.segments is a list of objects with start/end/text.
|
||||
raw_segments = getattr(resp, "segments", None)
|
||||
if raw_segments:
|
||||
for seg in raw_segments:
|
||||
seg_dict = seg if isinstance(seg, dict) else seg.model_dump()
|
||||
segments_out.append({
|
||||
"text": (seg_dict.get("text") or "").strip(),
|
||||
"start": seg_dict.get("start", 0.0),
|
||||
"end": seg_dict.get("end", 0.0),
|
||||
"words": [], # word-level timing isn't part of this API
|
||||
})
|
||||
else:
|
||||
# Plain text response (json/text format) — single-segment shape,
|
||||
# matching MoonshineASRBackend's degraded fallback exactly.
|
||||
text = (getattr(resp, "text", None) or "").strip()
|
||||
if text:
|
||||
segments_out.append({"text": text, "start": 0.0, "end": None, "words": []})
|
||||
chunks = [
|
||||
{"text": seg["text"], "timestamp": (seg["start"], seg["end"])}
|
||||
for seg in segments_out
|
||||
]
|
||||
language = getattr(resp, "language", None) or "en"
|
||||
return {"chunks": chunks, "segments": segments_out, "language": language}
|
||||
|
||||
|
||||
def _isolated_faster_whisper():
|
||||
"""Lazy import so the subprocess_asr → subprocess_backend chain isn't
|
||||
pulled in at registry definition time."""
|
||||
@@ -1689,6 +1845,7 @@ _REGISTRY: dict[str, type[ASRBackend]] = _LazyASRRegistry({
|
||||
"moonshine": MoonshineASRBackend,
|
||||
"funasr": FunASRBackend,
|
||||
"sherpa-onnx-asr": SherpaDictationBackend,
|
||||
"openai-compat-asr": OpenAICompatASRBackend,
|
||||
# "faster-whisper-isolated": resolved lazily (crash-isolated subprocess).
|
||||
})
|
||||
|
||||
@@ -1713,6 +1870,13 @@ _INSTALL_HINTS: dict[str, str] = {
|
||||
"moonshine": "pip install useful-moonshine (edge/CPU-optimized ASR)",
|
||||
"funasr": "pip install funasr (SenseVoiceSmall + FSMN-VAD; CUDA or CPU)",
|
||||
"sherpa-onnx-asr": "uv add sherpa-onnx (ONNX live dictation; CPU, cross-platform)",
|
||||
"openai-compat-asr": (
|
||||
"No install needed — configure a server endpoint in Settings → "
|
||||
"Engines. Points OmniVoice at any OpenAI-compatible transcription "
|
||||
"server (a self-hosted Qwen3-ASR/FunASR/SenseVoice server, OpenAI's "
|
||||
"own Whisper API, or similar) — a path to Qwen3-ASR today, without "
|
||||
"waiting on a direct transformers integration."
|
||||
),
|
||||
"faster-whisper-isolated": (
|
||||
"No extra install (reuses faster-whisper). Escape hatch for hanging "
|
||||
"transcribes: runs ASR in a separate process that can be force-killed "
|
||||
|
||||
@@ -957,7 +957,13 @@ def _load_model_sync():
|
||||
except Exception: # never let failure-formatting mask the real error
|
||||
err_msg = str(exc)
|
||||
_set_loading("error", "Model loading failed", error=err_msg)
|
||||
logger.error("Model loading failed: %s", str(exc))
|
||||
# #1000 class: transformers' lazy-import machinery wraps ANY disruption
|
||||
# to an inner import (including one interrupted by process teardown)
|
||||
# in a generic "Could not import module X. Are this object's
|
||||
# requirements defined correctly?" — logging only str(exc) discarded
|
||||
# the real cause in __cause__/__context__ and made a shutdown race
|
||||
# look like a broken install. exc_info surfaces the full chain.
|
||||
logger.error("Model loading failed: %s", str(exc), exc_info=exc)
|
||||
raise
|
||||
finally:
|
||||
unregister_listener(lid)
|
||||
@@ -1089,7 +1095,11 @@ async def preload_model():
|
||||
model = await _load_model_with_timeout()
|
||||
logger.info("Preload complete — model ready.")
|
||||
except Exception as e:
|
||||
logger.warning("Model preload failed (non-fatal): %s", e)
|
||||
# See the matching exc_info note on the _load_model_sync handler above
|
||||
# (#1000 class) — the full chain, not just str(e), is what actually
|
||||
# distinguishes a real dependency problem from a shutdown-interrupted
|
||||
# import.
|
||||
logger.warning("Model preload failed (non-fatal): %s", e, exc_info=e)
|
||||
|
||||
def get_model_status():
|
||||
is_loaded = model is not None
|
||||
|
||||
@@ -223,6 +223,60 @@ def extract_segment_refs(
|
||||
return out
|
||||
|
||||
|
||||
def refine_ref_text(ref_audio_path: str, asr_backend, fallback_text: str) -> str:
|
||||
"""Re-transcribe a written reference clip and return that transcript.
|
||||
|
||||
`extract_speaker_clones`/`extract_segment_refs` pair each audio slice with
|
||||
the ASR segment's OWN text field, on the assumption that the segment's
|
||||
timestamps and its transcribed text agree. They routinely don't — Whisper
|
||||
(and friends) frequently drift on segment boundaries: a trailing word
|
||||
audible in `[start, end]` but missing from `text`, or vice versa. When the
|
||||
(ref_audio, ref_text) pair disagrees, zero-shot TTS prompt-priming breaks
|
||||
down and the clone can speak the mismatched reference text itself instead
|
||||
of the target-language text it was given to synthesize (issue #1004).
|
||||
|
||||
Re-transcribing the *actual written clip* guarantees the pair matches by
|
||||
construction — the model doesn't care whether the original ASR text was
|
||||
right, only that ref_text is what's really in ref_audio. `asr_backend` is
|
||||
the caller's already-loaded active backend (duck-typed:
|
||||
`.transcribe(path, word_timestamps=...) -> dict` with a `chunks` list of
|
||||
`{"text": ...}`); the model is already warm, so this costs one more short
|
||||
transcribe call, not a fresh load. Falls back to `fallback_text` — never
|
||||
raises — so a re-transcribe failure is a strict no-op, never a regression
|
||||
from the original (matching) behavior.
|
||||
"""
|
||||
if asr_backend is None:
|
||||
return fallback_text
|
||||
try:
|
||||
result = asr_backend.transcribe(ref_audio_path, word_timestamps=False)
|
||||
text = " ".join(
|
||||
(c.get("text") or "").strip() for c in (result.get("chunks") or [])
|
||||
).strip()
|
||||
return text or fallback_text
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
"speaker_clone: re-transcribe of %s failed, keeping original ref_text: %s",
|
||||
ref_audio_path, e,
|
||||
)
|
||||
return fallback_text
|
||||
|
||||
|
||||
def refine_ref_texts(clones: dict[str, dict], asr_backend) -> dict[str, dict]:
|
||||
"""Apply `refine_ref_text` to every entry's `ref_text` in place.
|
||||
|
||||
Batches the whole dict (per-speaker `clones` from `extract_speaker_clones`
|
||||
or per-segment `seg_clones` from `extract_segment_refs`) into the single
|
||||
executor round-trip the caller submits to the GPU pool, rather than one
|
||||
dispatch per reference. Mutates and returns `clones` for a convenient
|
||||
call-and-reassign at the call site.
|
||||
"""
|
||||
for entry in clones.values():
|
||||
entry["ref_text"] = refine_ref_text(
|
||||
entry["ref_audio"], asr_backend, entry.get("ref_text", "")
|
||||
)
|
||||
return clones
|
||||
|
||||
|
||||
# ── Internals ───────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
|
||||
@@ -849,6 +849,7 @@ class MLXAudioBackend(TTSBackend):
|
||||
|
||||
voice = kw.get("voice")
|
||||
ref_audio = kw.get("ref_audio")
|
||||
ref_text = kw.get("ref_text")
|
||||
language = kw.get("language")
|
||||
speed = float(kw.get("speed", 1.0))
|
||||
|
||||
@@ -859,6 +860,12 @@ class MLXAudioBackend(TTSBackend):
|
||||
kwargs = {"text": text, "speed": speed}
|
||||
if voice: kwargs["voice"] = voice
|
||||
if ref_audio: kwargs["ref_audio"] = ref_audio
|
||||
# CSM (sesame.py) only builds its cloning context when BOTH ref_audio
|
||||
# AND ref_text are present — with ref_text missing, its context list
|
||||
# stays empty and indexing into it raises an opaque
|
||||
# "IndexError: list index out of range" deep inside mlx-audio,
|
||||
# instead of ever attempting the clone. Community-diagnosed (#1012).
|
||||
if ref_audio and ref_text: kwargs["ref_text"] = ref_text
|
||||
if language and language != "Auto":
|
||||
if self._model_id == self.CURATED_MODELS.get("kokoro"):
|
||||
# Kokoro's vendored pipeline hard-asserts `lang_code` against
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
# OmniVoice Studio — OpenAI-Compatible Remote ASR
|
||||
|
||||
A path to Qwen3-ASR, a self-hosted FunASR/SenseVoice server, or OpenAI's own
|
||||
Whisper API — today, without waiting on `transformers` to ship a direct
|
||||
Qwen3-ASR integration (tracked separately). Unlike every other ASR engine,
|
||||
this one runs no model locally: it's a pure network client that calls any
|
||||
server exposing an OpenAI-compatible `POST /v1/audio/transcriptions`
|
||||
endpoint.
|
||||
|
||||
## Setup
|
||||
|
||||
No install step — configure it directly:
|
||||
|
||||
1. Open **Settings → Models** and find **OpenAI-compatible ASR (remote
|
||||
server)**.
|
||||
2. Set **Server URL** to your server's base URL (e.g.
|
||||
`http://localhost:8000/v1` for a local Qwen3-ASR/FunASR server, or
|
||||
`https://api.openai.com/v1` for OpenAI's own API).
|
||||
3. Set **Model** to whatever your server expects (`whisper-1` for OpenAI's
|
||||
API; check your self-hosted server's docs otherwise).
|
||||
4. **API key** is optional — many self-hosted servers accept requests
|
||||
without one. Set it if your server requires auth, or if you're using
|
||||
OpenAI's own API.
|
||||
5. Activate the engine by setting `OMNIVOICE_ASR_BACKEND=openai-compat-asr`
|
||||
before launching. There's no in-app ASR engine picker yet (only TTS
|
||||
engines have one today) — this is the one manual step until that ships.
|
||||
|
||||
## Response format
|
||||
|
||||
The backend prefers `response_format=verbose_json` for real per-segment
|
||||
timestamps (OpenAI's API and most compatible servers support it) and falls
|
||||
back to plain text automatically if your server rejects that format. Neither
|
||||
path returns word-level timestamps — that's not part of this API.
|
||||
|
||||
## Privacy note
|
||||
|
||||
Unlike every other ASR engine in OmniVoice, audio sent through this backend
|
||||
leaves your machine — to whatever server you configured. If that's a
|
||||
self-hosted server on your own network, nothing leaves your control; if
|
||||
it's a third-party API (OpenAI's, or someone else's), review their data
|
||||
handling before sending anything sensitive.
|
||||
@@ -74,6 +74,8 @@ asr_engines:
|
||||
readme: FunASR
|
||||
- id: sherpa-onnx-asr
|
||||
readme: "**sherpa-onnx** (live dictation)"
|
||||
- id: openai-compat-asr
|
||||
readme: "**OpenAI-compatible** ⚠️ remote"
|
||||
|
||||
# Doc files that must exist (the install path users are sent to).
|
||||
docs:
|
||||
|
||||
@@ -445,6 +445,41 @@ quit OmniVoice Studio, delete the folder below, then start the app again.
|
||||
Remove-Item -Recurse -Force "$env:LOCALAPPDATA\com.debpalash.omnivoice-studio\EBWebView"
|
||||
```
|
||||
|
||||
## 16. macOS: microphone permission never prompts, OmniVoice never appears in System Settings
|
||||
|
||||
**Symptom:** clicking record shows "Microphone access denied. macOS: open
|
||||
System Settings → Privacy & Security → Microphone and enable OmniVoice" —
|
||||
but OmniVoice never appears in that list, so there's nothing to enable.
|
||||
`NSMicrophoneUsageDescription` is present in the app's `Info.plist`, and
|
||||
resetting the permission (`tccutil reset Microphone
|
||||
com.debpalash.omnivoice-studio`) followed by a relaunch changes nothing — no
|
||||
system prompt ever appears.
|
||||
|
||||
**Cause:** the app bundle was missing the Hardened Runtime *entitlement* for
|
||||
microphone access. An earlier revision of this section blamed an upstream
|
||||
Tauri/WebKit limitation — that was wrong (a community contributor,
|
||||
[@MahdiHedhli](https://github.com/MahdiHedhli), read the sources more
|
||||
carefully and found the real gap). wry's `WKUIDelegate` already grants the
|
||||
WebKit-layer media-capture request; but Tauri's macOS bundler enables
|
||||
Hardened Runtime by default, and Hardened Runtime blocks microphone hardware
|
||||
access unless `com.apple.security.device.audio-input` is present in the
|
||||
signed binary's entitlements — regardless of `Info.plist`'s
|
||||
`NSMicrophoneUsageDescription` (that only supplies the prompt *text*).
|
||||
Without the entitlement, macOS's TCC layer never registers a request, which
|
||||
is exactly why the app never appears in the System Settings list.
|
||||
|
||||
**Fix:** ships in the release after v0.3.12 (the bundle now carries
|
||||
`src-tauri/entitlements.plist` — [#1016](https://github.com/debpalash/OmniVoice-Studio/pull/1016),
|
||||
contributed by the same person who diagnosed it). Update and live recording
|
||||
works, with a normal macOS permission prompt on first use.
|
||||
|
||||
**Workaround on older builds (≤ v0.3.12):** record your voice sample in any
|
||||
other app (Voice Memos, QuickTime, etc.) and upload the resulting file in
|
||||
OmniVoice instead of using live recording — upload-based cloning is
|
||||
unaffected and works normally.
|
||||
|
||||
**Linked issue:** [#1013](https://github.com/debpalash/OmniVoice-Studio/issues/1013)
|
||||
|
||||
## Dub: "translation engine needs the optional … package"
|
||||
|
||||
**Symptom:** in the Dub tab, translating fails with e.g. *"The 'google'
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "omnivoice-studio",
|
||||
"version": "0.3.12",
|
||||
"version": "0.3.13",
|
||||
"private": true,
|
||||
"license": "AGPL-3.0-only",
|
||||
"type": "module",
|
||||
|
||||
Generated
+1
-1
@@ -2941,7 +2941,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "omnivoice-studio"
|
||||
version = "0.3.12"
|
||||
version = "0.3.13"
|
||||
dependencies = [
|
||||
"arboard",
|
||||
"dirs-next",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "omnivoice-studio"
|
||||
version = "0.3.12"
|
||||
version = "0.3.13"
|
||||
description = "OmniVoice Studio – AI voice cloning & dubbing desktop app"
|
||||
authors = ["Debpalash"]
|
||||
license = "AGPL-3.0-only"
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<!--
|
||||
Tauri's macOS bundle defaults `hardenedRuntime` to true. Hardened
|
||||
Runtime blocks camera/microphone hardware access unless the matching
|
||||
entitlement is present here — regardless of Info.plist's
|
||||
NSMicrophoneUsageDescription and regardless of wry's own WKUIDelegate
|
||||
already granting the request at the WebKit/JS layer
|
||||
(WryWebViewUIDelegate::request_media_capture_permission unconditionally
|
||||
calls WKPermissionDecision::Grant). Without this entitlement, TCC
|
||||
never even registers a request for the app — nothing shows up in
|
||||
System Settings → Privacy & Security → Microphone to enable, because
|
||||
the OS never saw a legitimately-entitled process ask.
|
||||
-->
|
||||
<key>com.apple.security.device.audio-input</key>
|
||||
<true/>
|
||||
|
||||
<!--
|
||||
Matches Info.plist's forward-looking NSCameraUsageDescription — no
|
||||
current feature uses the camera, but ship the entitlement now so a
|
||||
future getUserMedia({video: true}) call doesn't hit this same bug.
|
||||
-->
|
||||
<key>com.apple.security.device.camera</key>
|
||||
<true/>
|
||||
</dict>
|
||||
</plist>
|
||||
@@ -79,9 +79,18 @@ pub const TRAY_ICON_RECORDING: &[u8] = include_bytes!("../icons/tray-recording.p
|
||||
// applies on top.
|
||||
// - Linux (WebKitGTK): media-stream must be enabled per-WebView and the
|
||||
// permission request answered programmatically.
|
||||
// - macOS (WKWebView): nothing to do here — wry grants media-capture to the
|
||||
// app origin and the user-visible consent is the system TCC prompt driven
|
||||
// by NSMicrophoneUsageDescription in src-tauri/Info.plist.
|
||||
// - macOS (WKWebView): nothing to do here in code — wry's own WKUIDelegate
|
||||
// (WryWebViewUIDelegate::request_media_capture_permission) already grants
|
||||
// every media-capture request unconditionally at the WebKit/JS layer. But
|
||||
// that alone isn't sufficient (#1013): Tauri's macOS bundle defaults
|
||||
// `hardenedRuntime` to true, and Hardened Runtime blocks camera/microphone
|
||||
// hardware access unless the matching entitlement is present — without it,
|
||||
// TCC never even registers a request, so the app never appears in System
|
||||
// Settings → Privacy & Security → Microphone for the user to enable. See
|
||||
// src-tauri/entitlements.plist (wired in via tauri.conf.json's
|
||||
// bundle.macOS.entitlements) for the actual grant; NSMicrophoneUsageDescription
|
||||
// in Info.plist only supplies the *prompt text* TCC shows, it doesn't
|
||||
// substitute for the entitlement.
|
||||
|
||||
/// True for origins the app itself serves: the Tauri custom-protocol origin
|
||||
/// in production and the Vite dev server / loopback in `tauri dev`.
|
||||
|
||||
@@ -85,7 +85,8 @@
|
||||
],
|
||||
"macOS": {
|
||||
"minimumSystemVersion": "12.0",
|
||||
"signingIdentity": "-"
|
||||
"signingIdentity": "-",
|
||||
"entitlements": "entitlements.plist"
|
||||
}
|
||||
},
|
||||
"plugins": {
|
||||
|
||||
@@ -73,9 +73,6 @@ export default function NavRail({ mode, setMode, side = 'left', onFlipSide }) {
|
||||
[t],
|
||||
);
|
||||
|
||||
const donateLabel = t('donate.pill', { defaultValue: 'Support OmniVoice' });
|
||||
const donateActive = mode === 'donate';
|
||||
|
||||
// `nav-rail` is retained purely as the layout hook the (out-of-scope)
|
||||
// `.app-container > .nav-rail` grid rules position by; all visual styling now
|
||||
// lives in the utilities below. Border flips to the inner edge when on the right.
|
||||
@@ -84,17 +81,6 @@ export default function NavRail({ mode, setMode, side = 'left', onFlipSide }) {
|
||||
? '[border-left:1px_solid_var(--chrome-border)]'
|
||||
: '[border-right:1px_solid_var(--chrome-border)]';
|
||||
|
||||
// Quiet "Support" pill (was `.rail-btn.donate-pill`): neutral at rest, warms to
|
||||
// the accent on hover/active.
|
||||
const donateState = donateActive
|
||||
? 'text-[var(--chrome-accent)] bg-[var(--chrome-accent-bg)] [border:1px_solid_var(--chrome-accent-border)]'
|
||||
: 'bg-transparent text-[var(--chrome-fg-dim)] [border:1px_solid_transparent] hover:bg-[color-mix(in_srgb,var(--chrome-accent)_10%,transparent)] hover:text-[var(--chrome-accent)]';
|
||||
const heartBase =
|
||||
'text-[16px] leading-none [transition:filter_0.16s,opacity_0.16s,transform_0.16s] group-hover:[transform:scale(1.1)] motion-reduce:[transition:none] motion-reduce:group-hover:[transform:none]';
|
||||
const heartState = donateActive
|
||||
? 'opacity-100 [filter:grayscale(0)]'
|
||||
: 'opacity-75 [filter:grayscale(0.55)] group-hover:opacity-100 group-hover:[filter:grayscale(0)]';
|
||||
|
||||
return (
|
||||
<aside
|
||||
className={`nav-rail z-50 flex select-none flex-col items-center gap-[6px] bg-[var(--chrome-bg)] py-[8px] ${asideBorder}`}
|
||||
@@ -111,19 +97,6 @@ export default function NavRail({ mode, setMode, side = 'left', onFlipSide }) {
|
||||
))}
|
||||
</div>
|
||||
<div className="flex flex-col items-center gap-[4px]">
|
||||
{/* Quiet "Support" pill — warms to the accent on hover, opens the
|
||||
donate page. Sits with the footer nav (Settings / flip). (#007) */}
|
||||
<button
|
||||
onClick={() => setMode('donate')}
|
||||
title={donateLabel}
|
||||
aria-label={donateLabel}
|
||||
className={`${RAIL_BTN_BASE} ${donateState}`}
|
||||
>
|
||||
<span className={`${heartBase} ${heartState}`} aria-hidden="true">
|
||||
🩷
|
||||
</span>
|
||||
<span className={railLabelCls(side)}>{donateLabel}</span>
|
||||
</button>
|
||||
{footerItems.map((it) => (
|
||||
<RailBtn
|
||||
key={it.id}
|
||||
|
||||
@@ -351,7 +351,14 @@ function WaveformTimeline(
|
||||
console.warn('WebKit audio decode not supported, using media element directly');
|
||||
try {
|
||||
const emptyPeaks = new Float32Array(1000).fill(0);
|
||||
ws.load(undefined, [emptyPeaks], mediaEl.duration || 60);
|
||||
// Don't rely solely on the 'ready' event firing again for this
|
||||
// recovery load — the play button stayed permanently disabled
|
||||
// when it didn't (the waveform still rendered from the peaks, so
|
||||
// there was no visible sign anything was wrong). Confirm
|
||||
// readiness explicitly once this load settles either way.
|
||||
Promise.resolve(ws.load(undefined, [emptyPeaks], mediaEl.duration || 60))
|
||||
.then(() => setReady(true))
|
||||
.catch(() => setReady(true));
|
||||
} catch (_) {
|
||||
setReady(true);
|
||||
}
|
||||
@@ -372,7 +379,12 @@ function WaveformTimeline(
|
||||
})
|
||||
.then((audioBuffer) => {
|
||||
const channelData = audioBuffer.getChannelData(0);
|
||||
ws.load(undefined, [channelData], audioBuffer.duration);
|
||||
// Same explicit-readiness guard as the NotSupportedError branch
|
||||
// above — don't depend on the 'ready' event re-firing for this
|
||||
// manually-decoded recovery load.
|
||||
Promise.resolve(ws.load(undefined, [channelData], audioBuffer.duration))
|
||||
.then(() => setReady(true))
|
||||
.catch(() => setReady(true));
|
||||
})
|
||||
.catch((decodeErr) => {
|
||||
// HTTP 404 on the companion audio means the source file is
|
||||
@@ -391,7 +403,9 @@ function WaveformTimeline(
|
||||
console.warn('Audio decode fallback failed, loading with empty peaks:', decodeErr);
|
||||
try {
|
||||
const emptyPeaks = new Float32Array(1000).fill(0);
|
||||
ws.load(undefined, [emptyPeaks], mediaEl.duration || 60);
|
||||
Promise.resolve(ws.load(undefined, [emptyPeaks], mediaEl.duration || 60))
|
||||
.then(() => setReady(true))
|
||||
.catch(() => setReady(true));
|
||||
} catch (_) {
|
||||
setLoadError(true);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import path from 'node:path';
|
||||
|
||||
// Regression guard: the dub editor's play button stayed permanently disabled
|
||||
// (disabled={!ready}) whenever the initial WaveSurfer decode failed and the
|
||||
// component fell back to a peaks-only ws.load(undefined, [peaks], duration)
|
||||
// call — the waveform still rendered from those peaks (so nothing looked
|
||||
// visibly broken), but `ready` was only ever set from the 'ready' event
|
||||
// re-firing on that recovery load, which this component's own error-handling
|
||||
// code never actually confirmed. Each fallback load must now explicitly
|
||||
// confirm readiness once it settles, instead of assuming the event fires.
|
||||
//
|
||||
// Driving WaveSurfer + a real decode-failure/recovery sequence through jsdom
|
||||
// is brittle (see WaveformTimeline.unlock.test.js), so this is a
|
||||
// source-level contract guard, same house pattern: every `ws.load(undefined,
|
||||
// ...)` recovery call inside the `ws.on('error', ...)` handler must be
|
||||
// followed by an explicit setReady(true) confirmation.
|
||||
|
||||
const src = readFileSync(
|
||||
path.resolve(process.cwd(), 'src/components/WaveformTimeline.jsx'),
|
||||
'utf8',
|
||||
);
|
||||
|
||||
describe('WaveformTimeline error-recovery ready confirmation', () => {
|
||||
it("confirms readiness explicitly after every fallback ws.load() call, not just via the 'ready' event", () => {
|
||||
const errorHandler = /ws\.on\('error', \(err\) => \{([\s\S]*?)\n \}\);/.exec(src)?.[1];
|
||||
expect(errorHandler, "ws.on('error', ...) handler not found").toBeTruthy();
|
||||
|
||||
// Every recovery load in this handler passes peaks explicitly
|
||||
// (`ws.load(undefined, [...], ...)`) — each occurrence must be
|
||||
// immediately confirmed ready via a .then()/.catch() pair (or an
|
||||
// unconditional setReady in a synchronous catch), not left to hope the
|
||||
// 'ready' event re-fires on its own.
|
||||
const loadCalls = [...errorHandler.matchAll(/ws\.load\(undefined, \[[^\]]*\][^)]*\)/g)];
|
||||
expect(loadCalls.length).toBeGreaterThanOrEqual(3);
|
||||
|
||||
for (const match of loadCalls) {
|
||||
const tail = errorHandler.slice(match.index, match.index + 220);
|
||||
expect(tail, `no readiness confirmation after: ${match[0]}`).toMatch(/setReady\(true\)/);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -91,8 +91,13 @@ export default function CommunityZone({
|
||||
name: r.name,
|
||||
}),
|
||||
);
|
||||
} catch {
|
||||
flash(t('gallery.use_failed', { defaultValue: 'Could not add that voice.' }));
|
||||
} catch (e) {
|
||||
flash(
|
||||
t('gallery.use_failed', {
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Could not create that voice: {{message}}',
|
||||
}),
|
||||
);
|
||||
}
|
||||
}}
|
||||
onDesign={(item) =>
|
||||
|
||||
@@ -110,7 +110,12 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
|
||||
const r = await searchYoutube(q, 'import', 10);
|
||||
setResults(r.results || []);
|
||||
} catch (e) {
|
||||
flash(t('gallery.search_failed', { defaultValue: 'Search failed.' }));
|
||||
flash(
|
||||
t('gallery.search_failed', {
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Search failed: {{message}}',
|
||||
}),
|
||||
);
|
||||
} finally {
|
||||
setIsSearching(false);
|
||||
}
|
||||
@@ -149,7 +154,12 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
|
||||
await uploadVoiceClip(fd);
|
||||
reload();
|
||||
} catch (err) {
|
||||
flash(t('gallery.upload_failed', { defaultValue: 'Upload failed.' }));
|
||||
flash(
|
||||
t('gallery.upload_failed', {
|
||||
message: err?.message || String(err),
|
||||
defaultValue: 'Upload failed: {{message}}',
|
||||
}),
|
||||
);
|
||||
} finally {
|
||||
if (fileRef.current) fileRef.current.value = '';
|
||||
}
|
||||
@@ -165,7 +175,12 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
|
||||
}),
|
||||
);
|
||||
} catch (e) {
|
||||
flash(t('gallery.save_failed', { defaultValue: 'Could not save profile.' }));
|
||||
flash(
|
||||
t('gallery.save_failed', {
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Could not save profile: {{message}}',
|
||||
}),
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -179,8 +194,13 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
|
||||
try {
|
||||
await deleteGalleryVoice(v.id);
|
||||
reload();
|
||||
} catch {
|
||||
/* noop */
|
||||
} catch (e) {
|
||||
flash(
|
||||
t('gallery.delete_failed', {
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Could not delete: {{message}}',
|
||||
}),
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -191,7 +211,12 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
|
||||
const file = new File([blob], `${v.name}.wav`, { type: 'audio/wav' });
|
||||
setTrimming({ voice: v, file });
|
||||
} catch (e) {
|
||||
flash(t('gallery.trim_load_failed', { defaultValue: 'Could not load audio for trimming.' }));
|
||||
flash(
|
||||
t('gallery.trim_load_failed', {
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Could not load audio for trimming: {{message}}',
|
||||
}),
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -209,7 +234,12 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
|
||||
reload();
|
||||
setTrimming(null);
|
||||
} catch (e) {
|
||||
flash(t('gallery.upload_failed', { defaultValue: 'Upload failed.' }));
|
||||
flash(
|
||||
t('gallery.upload_failed', {
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Upload failed: {{message}}',
|
||||
}),
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
@@ -0,0 +1,152 @@
|
||||
/**
|
||||
* Settings → Models tab → OpenAI-compatible remote ASR panel (#877).
|
||||
*
|
||||
* A path to Qwen3-ASR, a self-hosted FunASR/SenseVoice server, or OpenAI's
|
||||
* own Whisper API — today, without waiting on transformers to ship a direct
|
||||
* Qwen3-ASR integration. Configures the `openai-compat-asr` backend's
|
||||
* base_url/model/api_key; activating it as the active ASR engine still needs
|
||||
* `OMNIVOICE_ASR_BACKEND=openai-compat-asr` (no in-app ASR engine picker
|
||||
* exists yet for any ASR backend — this panel only configures this one).
|
||||
*
|
||||
* Endpoints (loopback-only):
|
||||
* GET /api/settings/asr-openai-compat → {base_url, model, has_key}
|
||||
* PUT /api/settings/asr-openai-compat body {base_url?, model?, api_key?}
|
||||
* ('' clears api_key; omitted/null leaves it unchanged — never returned)
|
||||
*/
|
||||
import React, { useCallback, useEffect, useState } from 'react';
|
||||
import { Mic } from 'lucide-react';
|
||||
import { useTranslation } from 'react-i18next';
|
||||
import { apiJson, apiFetch } from '../../api/client';
|
||||
import { SettingsSection, SettingRow, SettingsInput } from './primitives';
|
||||
import { Button } from '../../ui';
|
||||
|
||||
export default function AsrOpenAICompatPanel() {
|
||||
const { t } = useTranslation();
|
||||
const [baseUrl, setBaseUrl] = useState('');
|
||||
const [model, setModel] = useState('');
|
||||
const [apiKey, setApiKey] = useState('');
|
||||
const [hasKey, setHasKey] = useState(false);
|
||||
const [saving, setSaving] = useState(false);
|
||||
const [error, setError] = useState(null);
|
||||
|
||||
const refresh = useCallback(async () => {
|
||||
setError(null);
|
||||
try {
|
||||
const d = await apiJson('/api/settings/asr-openai-compat');
|
||||
setBaseUrl(d?.base_url || '');
|
||||
setModel(d?.model || '');
|
||||
setHasKey(Boolean(d?.has_key));
|
||||
setApiKey(''); // the key is never returned — the field always starts blank
|
||||
} catch (e) {
|
||||
setError(e?.message || t('models.asrOpenAICompatLoadError'));
|
||||
}
|
||||
}, [t]);
|
||||
|
||||
useEffect(() => {
|
||||
refresh();
|
||||
}, [refresh]);
|
||||
|
||||
const save = async () => {
|
||||
setSaving(true);
|
||||
setError(null);
|
||||
try {
|
||||
const res = await apiFetch('/api/settings/asr-openai-compat', {
|
||||
method: 'PUT',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
base_url: baseUrl,
|
||||
model,
|
||||
// Only send api_key when the user actually typed something —
|
||||
// an untouched field must leave the stored key unchanged, not
|
||||
// clear it (the field is always blank on load, so "unchanged"
|
||||
// and "empty" would otherwise be indistinguishable).
|
||||
...(apiKey ? { api_key: apiKey } : {}),
|
||||
}),
|
||||
});
|
||||
const d = await res.json();
|
||||
setBaseUrl(d.base_url || '');
|
||||
setModel(d.model || '');
|
||||
setHasKey(Boolean(d.has_key));
|
||||
setApiKey('');
|
||||
} catch (e) {
|
||||
setError(e?.message || t('models.asrOpenAICompatSaveError'));
|
||||
} finally {
|
||||
setSaving(false);
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<SettingsSection
|
||||
icon={Mic}
|
||||
title={t('models.asrOpenAICompatTitle')}
|
||||
description={t('models.asrOpenAICompatDescription')}
|
||||
>
|
||||
{error && (
|
||||
<div className="perfpanel__error" role="alert">
|
||||
{error}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<SettingRow
|
||||
stack
|
||||
title={t('models.asrOpenAICompatBaseUrlTitle')}
|
||||
hint={t('models.asrOpenAICompatBaseUrlHint')}
|
||||
control={
|
||||
<SettingsInput
|
||||
mono
|
||||
type="text"
|
||||
value={baseUrl}
|
||||
onChange={(e) => setBaseUrl(e.target.value)}
|
||||
placeholder="http://localhost:8000/v1"
|
||||
data-testid="asr-openai-compat-base-url"
|
||||
/>
|
||||
}
|
||||
/>
|
||||
|
||||
<SettingRow
|
||||
stack
|
||||
title={t('models.asrOpenAICompatModelTitle')}
|
||||
control={
|
||||
<SettingsInput
|
||||
mono
|
||||
type="text"
|
||||
value={model}
|
||||
onChange={(e) => setModel(e.target.value)}
|
||||
placeholder="whisper-1"
|
||||
data-testid="asr-openai-compat-model"
|
||||
/>
|
||||
}
|
||||
/>
|
||||
|
||||
<SettingRow
|
||||
stack
|
||||
title={t('models.asrOpenAICompatApiKeyTitle')}
|
||||
hint={
|
||||
hasKey ? t('models.asrOpenAICompatKeyConfigured') : t('models.asrOpenAICompatApiKeyHint')
|
||||
}
|
||||
control={
|
||||
<>
|
||||
<SettingsInput
|
||||
mono
|
||||
type="password"
|
||||
value={apiKey}
|
||||
onChange={(e) => setApiKey(e.target.value)}
|
||||
placeholder={hasKey ? '••••••••' : t('models.asrOpenAICompatApiKeyOptional')}
|
||||
data-testid="asr-openai-compat-api-key"
|
||||
/>
|
||||
<Button
|
||||
variant="subtle"
|
||||
size="sm"
|
||||
onClick={save}
|
||||
loading={saving}
|
||||
disabled={saving}
|
||||
data-testid="asr-openai-compat-save"
|
||||
>
|
||||
{t('common.save')}
|
||||
</Button>
|
||||
</>
|
||||
}
|
||||
/>
|
||||
</SettingsSection>
|
||||
);
|
||||
}
|
||||
@@ -11,7 +11,11 @@ import { generateSpeech, audioUrlWithCacheBust } from '../api/generate';
|
||||
import { apiFetch } from '../api/client';
|
||||
import { playBlobAudio } from '../utils/media';
|
||||
import { PRESETS } from '../utils/constants';
|
||||
import { instructToFormValue, mergeDescribedAttrs } from '../utils/voiceInstruct';
|
||||
import {
|
||||
instructToFormValue,
|
||||
mergeDescribedAttrs,
|
||||
buildDesignInstruct,
|
||||
} from '../utils/voiceInstruct';
|
||||
import { askConfirm } from '../utils/dialog';
|
||||
import { toast } from 'react-hot-toast';
|
||||
import { recordValueMoment } from '../utils/donationMoments';
|
||||
@@ -55,7 +59,12 @@ export default function useProfiles({ loadHistory, loadProfiles }) {
|
||||
const safeBlob = new Blob([arrBuf], { type: refAudio.type });
|
||||
formData.append('ref_audio', safeBlob, refAudio.name || 'profile.wav');
|
||||
formData.append('ref_text', refText);
|
||||
formData.append('instruct', instruct);
|
||||
// #1010: the backend only sanitizes instruct on save for kind='design'
|
||||
// profiles — a clone profile (this call always creates kind='clone')
|
||||
// would silently persist an unsupported free-text instruct and then
|
||||
// 400 every single time it's used to generate. Filter here too.
|
||||
const { instruct: safeInst } = buildDesignInstruct({}, instruct);
|
||||
formData.append('instruct', safeInst);
|
||||
formData.append('language', language);
|
||||
try {
|
||||
await createProfile(formData);
|
||||
@@ -204,6 +213,25 @@ export default function useProfiles({ loadHistory, loadProfiles }) {
|
||||
fin_prof = '';
|
||||
}
|
||||
|
||||
// #1010: this instruct string comes straight from segment/preset data,
|
||||
// never through the validator-safe builder — a preset's raw attrs or a
|
||||
// free-text style field can carry phrases outside the active engine's
|
||||
// supported instruct vocabulary, 400ing instead of previewing. Same
|
||||
// client-side guard useTTS.js already applies to the clone path.
|
||||
if (fin_inst) {
|
||||
const { instruct: safeInst, unsupported, duplicates } = buildDesignInstruct({}, fin_inst);
|
||||
if (unsupported.length) {
|
||||
toast(t('tts_errors.ignored_unsupported', { items: unsupported.join(', ') }), {
|
||||
icon: '⚠️',
|
||||
});
|
||||
}
|
||||
if (duplicates.length) {
|
||||
toast(t('tts_errors.ignored_duplicate', { items: duplicates.join(', ') }), {
|
||||
icon: '⚠️',
|
||||
});
|
||||
}
|
||||
fin_inst = safeInst;
|
||||
}
|
||||
if (fin_prof) formData.append('profile_id', fin_prof);
|
||||
if (fin_inst) formData.append('instruct', fin_inst);
|
||||
const fin_lang = seg.target_lang || dubLang;
|
||||
@@ -245,7 +273,10 @@ export default function useProfiles({ loadHistory, loadProfiles }) {
|
||||
: item.text
|
||||
: '';
|
||||
formData.append('ref_text', extractedText);
|
||||
formData.append('instruct', item.instruct || '');
|
||||
// #1010: same guard as handleSaveProfile — this always creates a
|
||||
// kind='clone' profile, which the backend never sanitizes on save.
|
||||
const { instruct: safeHistInst } = buildDesignInstruct({}, item.instruct || '');
|
||||
formData.append('instruct', safeHistInst);
|
||||
formData.append('language', item.language || 'Auto');
|
||||
if (item.seed !== undefined && item.seed !== null) {
|
||||
formData.append('seed', item.seed);
|
||||
|
||||
@@ -1188,8 +1188,8 @@
|
||||
"no_matches": "No voices match these filters.",
|
||||
"load_more": "Load more",
|
||||
"saved_as_profile": "Added \"{{name}}\" to your voices.",
|
||||
"use_failed": "Could not create that voice — the engine may be loading.",
|
||||
"preview_failed": "Preview unavailable — the voice engine may still be loading.",
|
||||
"use_failed": "Could not create that voice: {{message}}",
|
||||
"preview_failed": "Preview unavailable: {{message}}",
|
||||
"import_explainer": "Paste a URL you have the rights to (or upload a file), trim the part you need, and save it as a voice. You are responsible for the licensing of anything you import.",
|
||||
"import_placeholder": "Paste a video/audio URL, or type to search…",
|
||||
"imported_clip": "Imported clip",
|
||||
@@ -1199,11 +1199,12 @@
|
||||
"no_imports": "Nothing imported yet. Paste a URL above to get started.",
|
||||
"search_results": "{{count}} results",
|
||||
"download_failed": "Download failed: {{msg}}",
|
||||
"search_failed": "Search failed.",
|
||||
"upload_failed": "Upload failed.",
|
||||
"save_failed": "Could not save profile.",
|
||||
"search_failed": "Search failed: {{message}}",
|
||||
"upload_failed": "Upload failed: {{message}}",
|
||||
"save_failed": "Could not save profile: {{message}}",
|
||||
"confirm_delete": "Delete \"{{name}}\"?",
|
||||
"trim_load_failed": "Could not load audio for trimming.",
|
||||
"delete_failed": "Could not delete: {{message}}",
|
||||
"trim_load_failed": "Could not load audio for trimming: {{message}}",
|
||||
"delete": "Delete",
|
||||
"community_empty": "No community voices loaded yet — connect to the internet and reopen, or be the first to submit one.",
|
||||
"community_explainer": "Designed presets and recorded voices shared by the community, loaded from the omnivoice-gallery.",
|
||||
@@ -2014,7 +2015,18 @@
|
||||
"mirror_preset_hint": "On a restricted network, route model downloads through a mirror. Leave empty for the official endpoint.",
|
||||
"mirror_restart_note": "Model Store downloads use the new mirror immediately. Only model loads (transformers) pick it up after a restart.",
|
||||
"mirror_load_error": "Failed to load mirror setting",
|
||||
"mirror_save_error": "Failed to save"
|
||||
"mirror_save_error": "Failed to save",
|
||||
"asrOpenAICompatTitle": "OpenAI-compatible ASR (remote server)",
|
||||
"asrOpenAICompatDescription": "Point transcription at Qwen3-ASR, a self-hosted FunASR/SenseVoice server, or OpenAI's own API.",
|
||||
"asrOpenAICompatBaseUrlTitle": "Server URL",
|
||||
"asrOpenAICompatBaseUrlHint": "The base URL of an OpenAI-compatible transcription server. To use this engine, also set OMNIVOICE_ASR_BACKEND=openai-compat-asr — there's no in-app engine picker for ASR yet.",
|
||||
"asrOpenAICompatModelTitle": "Model",
|
||||
"asrOpenAICompatApiKeyTitle": "API key",
|
||||
"asrOpenAICompatApiKeyHint": "Optional — many self-hosted servers don't require one.",
|
||||
"asrOpenAICompatApiKeyOptional": "optional",
|
||||
"asrOpenAICompatKeyConfigured": "A key is saved. Leave blank to keep it, or type a new one to replace it.",
|
||||
"asrOpenAICompatLoadError": "Failed to load ASR server setting",
|
||||
"asrOpenAICompatSaveError": "Failed to save"
|
||||
},
|
||||
"enterprise_faq": {
|
||||
"q_internal_tools": "Do I need a license for internal tools?",
|
||||
|
||||
@@ -22,6 +22,7 @@ import StoragePanel from '../components/settings/StoragePanel';
|
||||
import StorageTab from '../components/settings/StorageTab';
|
||||
import StorageUsagePanel from '../components/settings/StorageUsagePanel';
|
||||
import HFMirrorPanel from '../components/settings/HFMirrorPanel';
|
||||
import AsrOpenAICompatPanel from '../components/settings/AsrOpenAICompatPanel';
|
||||
import SharingPanel from '../components/settings/SharingPanel';
|
||||
import RemoteBackendPanel from '../components/settings/RemoteBackendPanel';
|
||||
import MCPBindingsPanel from '../components/settings/MCPBindingsPanel';
|
||||
@@ -367,6 +368,7 @@ export default function Settings() {
|
||||
<>
|
||||
<StoragePanel />
|
||||
<HFMirrorPanel />
|
||||
<AsrOpenAICompatPanel />
|
||||
<ModelStoreTab info={info} modelBadge={modelBadge} />
|
||||
</>
|
||||
);
|
||||
|
||||
@@ -138,7 +138,8 @@ export default function VoiceGallery() {
|
||||
stopPlayback();
|
||||
flash(
|
||||
t('gallery.preview_failed', {
|
||||
defaultValue: 'Preview unavailable — the voice engine may still be loading.',
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Preview unavailable: {{message}}',
|
||||
}),
|
||||
);
|
||||
} finally {
|
||||
@@ -222,7 +223,8 @@ export default function VoiceGallery() {
|
||||
} catch (e) {
|
||||
flash(
|
||||
t('gallery.use_failed', {
|
||||
defaultValue: 'Could not create that voice — the engine may be loading.',
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Could not create that voice: {{message}}',
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
// Regression guard: VoiceGallery/CommunityZone/ImportsZone catch blocks used
|
||||
// to discard the real error and show a hardcoded, often-wrong generic guess
|
||||
// (e.g. "the engine may be loading" on ANY failure, including ones that had
|
||||
// nothing to do with loading). Fixed to interpolate the real `e.message`
|
||||
// (already a clean, user-facing string from api/client.js's ApiError),
|
||||
// matching the `{{message}}` convention used everywhere else in this file.
|
||||
// This test only pins the i18n keys, not the call sites, deliberately: it's
|
||||
// a cheap net against reverting to a hardcoded string, not a full behavior test.
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import en from '../i18n/locales/en.json';
|
||||
|
||||
describe('gallery error messages interpolate the real error', () => {
|
||||
const keys = [
|
||||
'use_failed',
|
||||
'preview_failed',
|
||||
'search_failed',
|
||||
'upload_failed',
|
||||
'save_failed',
|
||||
'delete_failed',
|
||||
'trim_load_failed',
|
||||
];
|
||||
|
||||
it.each(keys)('gallery.%s contains {{message}}', (key) => {
|
||||
expect(en.gallery[key]).toContain('{{message}}');
|
||||
});
|
||||
});
|
||||
+1
-1
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
||||
|
||||
[project]
|
||||
name = "omnivoice"
|
||||
version = "0.3.12"
|
||||
version = "0.3.13"
|
||||
description = "OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models"
|
||||
readme = "README.md"
|
||||
# Free and open-source under the GNU Affero General Public License v3 (see
|
||||
|
||||
@@ -37,6 +37,38 @@ if not os.environ.get("OMNIVOICE_ENV_FILE"):
|
||||
|
||||
|
||||
import pytest
|
||||
import warnings as _warnings
|
||||
|
||||
|
||||
# ── torch default-dtype isolation (CI flaky trio) ───────────────────────────
|
||||
# Three tests (test_effects_chain / test_generation_audio_guard /
|
||||
# test_persona_bundle) fail intermittently on CI — never locally — with
|
||||
# signatures that all trace to one cause: a leaked
|
||||
# `torch.set_default_dtype(torch.float16)` from some earlier test. The
|
||||
# smoking gun is test_generation_audio_guard's observed value
|
||||
# 0.0999755859375, which is exactly float16(0.1): `torch.tensor([0.1, …])`
|
||||
# built under a leaked fp16 default. The same leak collapses
|
||||
# test_effects_chain's preset differences into identical quantized outputs,
|
||||
# and hands test_persona_bundle's soundfile writer fp16 data libsndfile
|
||||
# can't encode. The polluter only executes on CI-Linux (it never reproduces
|
||||
# on macOS), so rather than chase it blind, this guard makes the whole leak
|
||||
# class impossible — same philosophy as the LLM-state guard below — and
|
||||
# names the offender in CI output when it fires, so it CAN be chased.
|
||||
@pytest.fixture(autouse=True)
|
||||
def _torch_default_dtype_guard(request):
|
||||
yield
|
||||
torch = sys.modules.get("torch")
|
||||
if torch is None:
|
||||
return
|
||||
if torch.get_default_dtype() is not torch.float32:
|
||||
_warnings.warn(
|
||||
f"{request.node.nodeid} leaked torch default dtype "
|
||||
f"{torch.get_default_dtype()} — resetting to float32. This is "
|
||||
f"the polluter behind the CI flaky trio; fix it at the source.",
|
||||
stacklevel=1,
|
||||
)
|
||||
torch.set_default_dtype(torch.float32)
|
||||
|
||||
|
||||
# ── LLM-provider state isolation (issue #878) ──────────────────────────────
|
||||
# LLM provider selection is process-global three ways: env vars (the
|
||||
|
||||
Vendored
+2
@@ -18,6 +18,7 @@ DELETE /profiles/{profile_id}/consent
|
||||
DELETE /projects/{project_id}
|
||||
DELETE /pronunciation/{entry_id}
|
||||
GET /api/mcp/bindings
|
||||
GET /api/settings/asr-openai-compat
|
||||
GET /api/settings/changelog
|
||||
GET /api/settings/db-backup
|
||||
GET /api/settings/dictation-refinement
|
||||
@@ -216,6 +217,7 @@ POST /v1/audio/transcriptions
|
||||
POST /watermark/detect
|
||||
POST /watermark/settings
|
||||
PUT /api/mcp/bindings
|
||||
PUT /api/settings/asr-openai-compat
|
||||
PUT /api/settings/dictation-refinement
|
||||
PUT /api/settings/hf-mirror
|
||||
PUT /api/settings/llm-endpoint
|
||||
|
||||
@@ -0,0 +1,239 @@
|
||||
"""Generic OpenAI-compatible ASR backend (#877) — a path to Qwen3-ASR,
|
||||
FunASR/SenseVoice self-hosted servers, or OpenAI's own Whisper API, today,
|
||||
without waiting on transformers to ship a direct Qwen3-ASR integration.
|
||||
|
||||
settings_store backed by in-memory dicts, OpenAI client faked at the SDK
|
||||
boundary (no network) — house convention, same as test_llm_providers_router.py:
|
||||
direct handler calls, no TestClient, so the loopback auth guard isn't in play.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
import types
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "backend"))
|
||||
|
||||
os.environ.setdefault("OMNIVOICE_MODEL", "test")
|
||||
os.environ.setdefault("OMNIVOICE_DISABLE_FILE_LOG", "1")
|
||||
|
||||
_HAS_OPENAI = __import__("importlib").util.find_spec("openai") is not None
|
||||
pytestmark = pytest.mark.skipif(not _HAS_OPENAI, reason="openai package not installed")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def ss(monkeypatch):
|
||||
"""services.settings_store, resolved fresh (no module-level import — see
|
||||
asr_mod's docstring for why staleness across sys.modules reimports is a
|
||||
real risk in this suite) and patched to in-memory dicts (no SQLite)."""
|
||||
from services import settings_store as _ss
|
||||
|
||||
text: dict[str, str] = {}
|
||||
secrets: dict[str, str] = {}
|
||||
monkeypatch.setattr(_ss, "get_text", lambda k, default=None: text.get(k, default))
|
||||
monkeypatch.setattr(_ss, "set_text", lambda k, v: text.__setitem__(k, v))
|
||||
monkeypatch.setattr(_ss, "get_secret", lambda n: secrets.get(n))
|
||||
monkeypatch.setattr(
|
||||
_ss, "set_secret", lambda n, v: secrets.__setitem__(n, v) if v else secrets.pop(n, None)
|
||||
)
|
||||
monkeypatch.setattr(_ss, "list_secret_names", lambda: list(secrets))
|
||||
return _ss
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def asr_mod(ss, monkeypatch):
|
||||
"""services.asr_backend with settings_store in-memory (no SQLite).
|
||||
|
||||
Resolved via importlib.import_module INSIDE the fixture (not a top-level
|
||||
`import` in this file) so it's the module object actually live in
|
||||
sys.modules at test-run time — other test files in this ~2400-test suite
|
||||
pop+reimport shared service modules (services.model_manager,
|
||||
services.tts_backend), and a module-level import captured once at file
|
||||
COLLECTION time can go stale by the time an individual test in this file
|
||||
finally runs, hours of test-order later. A collection-time reference
|
||||
calling .set_text() and a fixture-time reference reading via .get_text()
|
||||
can silently be two different module objects — the write and the read
|
||||
land in different in-memory dicts, and the test fails with no obvious
|
||||
cause. Every test below takes `ss` as a fixture (not a module-level
|
||||
`from services import settings_store`) for the same reason.
|
||||
"""
|
||||
for var in ("ASR_OPENAI_COMPAT_BASE_URL", "ASR_OPENAI_COMPAT_MODEL", "ASR_OPENAI_COMPAT_API_KEY"):
|
||||
monkeypatch.delenv(var, raising=False)
|
||||
import importlib
|
||||
return importlib.import_module("services.asr_backend")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def settings_mod(asr_mod):
|
||||
"""api.routers.settings sharing the same monkeypatched settings_store."""
|
||||
import importlib
|
||||
return importlib.import_module("api.routers.settings")
|
||||
|
||||
|
||||
def _fake_openai_transcribe(monkeypatch, *, verbose_ok=True, response=None, raise_exc=None):
|
||||
"""Fake openai.OpenAI whose audio.transcriptions.create() either returns
|
||||
a canned response or raises. verbose_ok=False simulates a minimal server
|
||||
that rejects response_format="verbose_json" on the first call, forcing
|
||||
the plain-json fallback."""
|
||||
captured_kwargs = []
|
||||
calls = []
|
||||
|
||||
class _FakeClient:
|
||||
def __init__(self, **kwargs):
|
||||
captured_kwargs.append(kwargs)
|
||||
self.audio = types.SimpleNamespace(
|
||||
transcriptions=types.SimpleNamespace(create=self._create)
|
||||
)
|
||||
|
||||
def _create(self, **kw):
|
||||
calls.append(kw)
|
||||
if raise_exc is not None:
|
||||
raise raise_exc
|
||||
if kw.get("response_format") == "verbose_json" and not verbose_ok:
|
||||
raise RuntimeError("response_format not supported")
|
||||
return response
|
||||
|
||||
import openai
|
||||
monkeypatch.setattr(openai, "OpenAI", _FakeClient)
|
||||
return captured_kwargs, calls
|
||||
|
||||
|
||||
# ── is_available() gating ───────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_unavailable_without_base_url(asr_mod):
|
||||
ok, msg = asr_mod.OpenAICompatASRBackend.is_available()
|
||||
assert ok is False
|
||||
assert "Settings" in msg
|
||||
|
||||
|
||||
def test_available_once_base_url_configured(asr_mod, ss):
|
||||
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://localhost:8080/v1")
|
||||
ok, _ = asr_mod.OpenAICompatASRBackend.is_available()
|
||||
assert ok is True
|
||||
|
||||
|
||||
# ── response adaptation ─────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_transcribe_adapts_verbose_json_segments(asr_mod, ss, monkeypatch, tmp_path):
|
||||
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://localhost:8080/v1")
|
||||
|
||||
class _Seg:
|
||||
def model_dump(self):
|
||||
return {"text": "hello world", "start": 0.0, "end": 1.5}
|
||||
|
||||
resp = types.SimpleNamespace(segments=[_Seg()], language="en")
|
||||
_fake_openai_transcribe(monkeypatch, response=resp)
|
||||
|
||||
audio = tmp_path / "seg.wav"
|
||||
audio.write_bytes(b"RIFF....WAVEfmt ") # content is never read by the fake client
|
||||
out = asr_mod.OpenAICompatASRBackend().transcribe(str(audio))
|
||||
assert out["language"] == "en"
|
||||
assert out["segments"] == [{"text": "hello world", "start": 0.0, "end": 1.5, "words": []}]
|
||||
assert out["chunks"] == [{"text": "hello world", "timestamp": (0.0, 1.5)}]
|
||||
|
||||
|
||||
def test_transcribe_falls_back_to_plain_text_when_verbose_json_rejected(asr_mod, ss, monkeypatch, tmp_path):
|
||||
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://localhost:8080/v1")
|
||||
resp = types.SimpleNamespace(text="plain text only", segments=None, language=None)
|
||||
_captured, calls = _fake_openai_transcribe(monkeypatch, verbose_ok=False, response=resp)
|
||||
|
||||
audio = tmp_path / "seg.wav"
|
||||
audio.write_bytes(b"RIFF....WAVEfmt ")
|
||||
out = asr_mod.OpenAICompatASRBackend().transcribe(str(audio))
|
||||
assert len(calls) == 2 # verbose_json attempt, then the plain fallback
|
||||
assert calls[0]["response_format"] == "verbose_json"
|
||||
assert calls[1]["response_format"] == "json"
|
||||
assert out["segments"] == [{"text": "plain text only", "start": 0.0, "end": None, "words": []}]
|
||||
assert out["language"] == "en" # default when the server doesn't report one
|
||||
|
||||
|
||||
def test_transcribe_network_failure_does_not_leak_raw_exception(asr_mod, ss, monkeypatch, tmp_path):
|
||||
"""Mirrors the #977 convention: a raw SDK/httpx exception must never reach
|
||||
the caller unformatted — only a clean, actionable RuntimeError."""
|
||||
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://localhost:8080/v1")
|
||||
_fake_openai_transcribe(monkeypatch, raise_exc=ConnectionError("connection refused"))
|
||||
|
||||
audio = tmp_path / "seg.wav"
|
||||
audio.write_bytes(b"RIFF....WAVEfmt ")
|
||||
with pytest.raises(RuntimeError) as ei:
|
||||
asr_mod.OpenAICompatASRBackend().transcribe(str(audio))
|
||||
msg = str(ei.value)
|
||||
assert "localhost:8080" in msg
|
||||
assert "ConnectionError" in msg
|
||||
|
||||
|
||||
def test_client_disables_sdk_retries(asr_mod, ss, monkeypatch, tmp_path):
|
||||
"""max_retries=0 — mirrors llm_skills.resolve_skill_client: a slow/rate-
|
||||
limited server retrying inside the SDK would blow past the caller's own
|
||||
bounded timeout expectation for a single transcribe call."""
|
||||
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://localhost:8080/v1")
|
||||
resp = types.SimpleNamespace(text="ok", segments=None, language="en")
|
||||
captured_kwargs, _ = _fake_openai_transcribe(monkeypatch, response=resp)
|
||||
|
||||
audio = tmp_path / "seg.wav"
|
||||
audio.write_bytes(b"RIFF....WAVEfmt ")
|
||||
asr_mod.OpenAICompatASRBackend().transcribe(str(audio))
|
||||
assert captured_kwargs[0]["max_retries"] == 0
|
||||
|
||||
|
||||
# ── settings endpoints ───────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_get_default_empty(settings_mod):
|
||||
st = settings_mod.get_asr_openai_compat()
|
||||
assert st == {"base_url": "", "model": "whisper-1", "has_key": False}
|
||||
|
||||
|
||||
def test_put_persists_and_never_echoes_the_key(settings_mod):
|
||||
st = settings_mod.set_asr_openai_compat(
|
||||
settings_mod._ASROpenAICompatBody(
|
||||
base_url="http://localhost:8080/v1/", model="qwen3-asr", api_key="sk-test-123",
|
||||
)
|
||||
)
|
||||
assert st["base_url"] == "http://localhost:8080/v1" # trailing slash trimmed
|
||||
assert st["model"] == "qwen3-asr"
|
||||
assert st["has_key"] is True
|
||||
assert "sk-test-123" not in str(st) # the key never round-trips
|
||||
|
||||
st2 = settings_mod.get_asr_openai_compat()
|
||||
assert st2 == st
|
||||
|
||||
|
||||
def test_empty_api_key_clears_it(settings_mod):
|
||||
settings_mod.set_asr_openai_compat(
|
||||
settings_mod._ASROpenAICompatBody(api_key="sk-test-123")
|
||||
)
|
||||
assert settings_mod.get_asr_openai_compat()["has_key"] is True
|
||||
|
||||
settings_mod.set_asr_openai_compat(settings_mod._ASROpenAICompatBody(api_key=""))
|
||||
assert settings_mod.get_asr_openai_compat()["has_key"] is False
|
||||
|
||||
|
||||
def test_none_fields_leave_existing_values_unchanged(settings_mod):
|
||||
settings_mod.set_asr_openai_compat(
|
||||
settings_mod._ASROpenAICompatBody(base_url="http://localhost:8080/v1", model="qwen3-asr")
|
||||
)
|
||||
# A save that only touches api_key must not clobber base_url/model.
|
||||
settings_mod.set_asr_openai_compat(settings_mod._ASROpenAICompatBody(api_key="sk-abc"))
|
||||
st = settings_mod.get_asr_openai_compat()
|
||||
assert st["base_url"] == "http://localhost:8080/v1"
|
||||
assert st["model"] == "qwen3-asr"
|
||||
assert st["has_key"] is True
|
||||
|
||||
|
||||
def test_rejects_a_base_url_without_scheme(settings_mod):
|
||||
from fastapi import HTTPException
|
||||
with pytest.raises(HTTPException):
|
||||
settings_mod.set_asr_openai_compat(
|
||||
settings_mod._ASROpenAICompatBody(base_url="localhost:8080/v1")
|
||||
)
|
||||
|
||||
|
||||
def test_registered_in_backend_list(asr_mod):
|
||||
assert "openai-compat-asr" in asr_mod._REGISTRY
|
||||
assert asr_mod._REGISTRY["openai-compat-asr"] is asr_mod.OpenAICompatASRBackend
|
||||
assert "openai-compat-asr" in asr_mod._INSTALL_HINTS
|
||||
@@ -357,6 +357,70 @@ def test_mlx_audio_generate_rejects_unsupported_kokoro_language_before_calling_m
|
||||
backend.generate("hello", language="Dutch")
|
||||
|
||||
|
||||
def test_mlx_audio_generate_passes_ref_text_through_for_cloning():
|
||||
# #1012/#1013: MLXAudioBackend.generate() read voice/ref_audio/language/
|
||||
# speed from kwargs but silently dropped ref_text — CSM (sesame.py) only
|
||||
# builds its cloning context when BOTH ref_audio and ref_text are
|
||||
# present, so cloning on CSM always raised an opaque
|
||||
# "IndexError: list index out of range" deep inside mlx-audio instead of
|
||||
# ever attempting the clone. Community-diagnosed with the exact fix.
|
||||
pytest.importorskip("mlx_audio", reason="mlx-audio is Apple-Silicon-only")
|
||||
backend = tts_backend.MLXAudioBackend()
|
||||
backend._ensure_loaded = lambda: None
|
||||
|
||||
captured = {}
|
||||
|
||||
def _fake_generate(**kw):
|
||||
captured.update(kw)
|
||||
return iter([types.SimpleNamespace(audio=__import__("numpy").zeros(4))])
|
||||
|
||||
backend._model = types.SimpleNamespace(generate=_fake_generate)
|
||||
backend.generate("hello", ref_audio="/tmp/ref.wav", ref_text="the reference line")
|
||||
|
||||
assert captured.get("ref_text") == "the reference line"
|
||||
assert captured.get("ref_audio") == "/tmp/ref.wav"
|
||||
|
||||
|
||||
def test_mlx_audio_generate_omits_ref_text_without_ref_audio():
|
||||
# ref_text alone (no ref_audio) means nothing to CSM's context builder —
|
||||
# don't pass a stray kwarg an engine that isn't cloning doesn't expect.
|
||||
pytest.importorskip("mlx_audio", reason="mlx-audio is Apple-Silicon-only")
|
||||
backend = tts_backend.MLXAudioBackend()
|
||||
backend._ensure_loaded = lambda: None
|
||||
|
||||
captured = {}
|
||||
|
||||
def _fake_generate(**kw):
|
||||
captured.update(kw)
|
||||
return iter([types.SimpleNamespace(audio=__import__("numpy").zeros(4))])
|
||||
|
||||
backend._model = types.SimpleNamespace(generate=_fake_generate)
|
||||
backend.generate("hello", ref_text="orphaned text, no audio")
|
||||
|
||||
assert "ref_text" not in captured
|
||||
|
||||
|
||||
def test_mlx_audio_generate_design_path_unaffected_without_any_ref():
|
||||
# Absorbed from community PR #1015 (MahdiHedhli) — the design/instruct
|
||||
# path (no ref_audio, no ref_text at all) must stay untouched by the
|
||||
# ref_text forwarding fix; neither kwarg may leak into the model call.
|
||||
pytest.importorskip("mlx_audio", reason="mlx-audio is Apple-Silicon-only")
|
||||
backend = tts_backend.MLXAudioBackend()
|
||||
backend._ensure_loaded = lambda: None
|
||||
|
||||
captured = {}
|
||||
|
||||
def _fake_generate(**kw):
|
||||
captured.update(kw)
|
||||
return iter([types.SimpleNamespace(audio=__import__("numpy").zeros(4))])
|
||||
|
||||
backend._model = types.SimpleNamespace(generate=_fake_generate)
|
||||
backend.generate("hello")
|
||||
|
||||
assert "ref_text" not in captured
|
||||
assert "ref_audio" not in captured
|
||||
|
||||
|
||||
def test_mlx_audio_generate_auto_language_skips_lang_code_entirely():
|
||||
# Matches the "Auto" convention other engines in this file use
|
||||
# (OmniVoiceBackend.generate(), _run_backend_inference) — never resolved,
|
||||
|
||||
@@ -0,0 +1,143 @@
|
||||
"""A quit mid-preload must not report a clean shutdown while a GPU-pool
|
||||
thread is still running (#1000 class).
|
||||
|
||||
Field report: a backend log showed three rapid restart cycles, each ending
|
||||
with "Shutdown: done." immediately followed by a "Model loading failed:
|
||||
Could not import module 'AutoFeatureExtractor'" error — transformers' own
|
||||
generic lazy-import wrapper, not a real dependency problem. The real cause:
|
||||
`preload_task` (and the optional `capture_preload_task`) were created at
|
||||
startup but never referenced in the shutdown block, so `idle_task`/
|
||||
`worker_task` got cancelled-and-awaited while the preload task was simply
|
||||
abandoned — the process declared "done" while a background GPU-pool thread
|
||||
was still mid-`import`, and got torn down by interpreter finalization under
|
||||
it.
|
||||
|
||||
`_cancel_and_await_tasks` is the extracted, directly-testable shutdown
|
||||
helper — the full `lifespan()` context manager touches too much startup
|
||||
machinery (DB init, gallery init, MCP session manager) to drive directly in
|
||||
a unit test (this suite's own test_mcp_mount.py notes exactly this: running
|
||||
the full lifespan contaminates other tests' event loops).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "backend"))
|
||||
|
||||
from main import _cancel_and_await_tasks # noqa: E402
|
||||
|
||||
|
||||
def _run(coro):
|
||||
return asyncio.run(coro)
|
||||
|
||||
|
||||
def test_a_task_that_finished_before_cancel_keeps_its_result():
|
||||
"""An early-stage task (mirrors preload still importing, not yet deep in
|
||||
blocking weight-load work) that completes on its own before the shutdown
|
||||
helper even reaches it must not be treated as an error — `.cancel()` on
|
||||
an already-done task is a no-op, and its real result survives. This is
|
||||
the fix: previously preload_task was never referenced in shutdown at
|
||||
all, so this case (the common one — most quits don't land mid-import)
|
||||
was never even checked."""
|
||||
finished = []
|
||||
|
||||
async def _quick():
|
||||
await asyncio.sleep(0.01)
|
||||
finished.append("done")
|
||||
|
||||
async def _scenario():
|
||||
t = asyncio.create_task(_quick())
|
||||
await asyncio.sleep(0.05) # long enough for _quick() to fully finish
|
||||
assert t.done()
|
||||
await _cancel_and_await_tasks(t, timeout=1.0) # must not raise on a done task
|
||||
|
||||
_run(_scenario())
|
||||
assert finished == ["done"]
|
||||
|
||||
|
||||
def test_none_entries_are_skipped_without_error():
|
||||
"""capture_preload_task is None when OMNIVOICE_PRELOAD_CAPTURE_ASR=0 —
|
||||
the helper must not crash on a mix of real tasks and None."""
|
||||
async def _noop():
|
||||
return None
|
||||
|
||||
async def _scenario():
|
||||
t = asyncio.create_task(_noop())
|
||||
await _cancel_and_await_tasks(t, None, timeout=1.0)
|
||||
|
||||
_run(_scenario()) # must not raise
|
||||
|
||||
|
||||
def test_a_task_stuck_past_the_bound_times_out_without_hanging():
|
||||
"""A task that never yields back (mirroring a GPU-pool thread stuck in a
|
||||
blocking native call) must not hang shutdown forever — the bound is the
|
||||
backstop, same as the pre-existing idle_task/worker_task pattern."""
|
||||
async def _wedged():
|
||||
await asyncio.sleep(10.0)
|
||||
|
||||
async def _scenario():
|
||||
t = asyncio.create_task(_wedged())
|
||||
await asyncio.sleep(0.01)
|
||||
await _cancel_and_await_tasks(t, timeout=0.2)
|
||||
|
||||
import time
|
||||
start = time.monotonic()
|
||||
_run(_scenario())
|
||||
elapsed = time.monotonic() - start
|
||||
assert elapsed < 2.0, f"shutdown helper did not bound its wait: took {elapsed:.2f}s"
|
||||
|
||||
|
||||
def test_multiple_tasks_are_all_cancelled_before_any_await():
|
||||
"""Cancel-then-await (not cancel-then-immediately-await-one-at-a-time) —
|
||||
every task gets its cancellation requested up front, so a slow task
|
||||
earlier in the list can't delay a later task's cancel signal."""
|
||||
cancelled_order = []
|
||||
|
||||
async def _tracked(name, delay):
|
||||
try:
|
||||
await asyncio.sleep(delay)
|
||||
except asyncio.CancelledError:
|
||||
cancelled_order.append(name)
|
||||
raise
|
||||
|
||||
async def _scenario():
|
||||
t1 = asyncio.create_task(_tracked("slow", 5.0))
|
||||
t2 = asyncio.create_task(_tracked("fast", 5.0))
|
||||
await asyncio.sleep(0.01)
|
||||
await _cancel_and_await_tasks(t1, t2, timeout=0.5)
|
||||
|
||||
_run(_scenario())
|
||||
assert set(cancelled_order) == {"slow", "fast"}
|
||||
|
||||
|
||||
def test_production_shutdown_wait_is_generous_enough_for_a_cold_import():
|
||||
"""Post-merge code-review finding (Greptile, PR #1002): the original 3s
|
||||
bound left a real residual window — cancelling the asyncio task doesn't
|
||||
stop the underlying OS thread, so a cold transformers import taking
|
||||
longer than the bound could still let shutdown report "done" while that
|
||||
thread was alive, the exact #1000 class again just with lower odds.
|
||||
Python can't forcibly kill a running thread, so no finite bound
|
||||
eliminates this outright — this pins the production call site to a
|
||||
materially more generous wait (20s, not 3s) rather than letting a future
|
||||
edit quietly shrink it back down without deliberate consideration.
|
||||
|
||||
Source-level guard, not a live-timing test: driving an actual >3s cold
|
||||
import through this suite would make it slow and environment-dependent
|
||||
for no real benefit.
|
||||
"""
|
||||
import re
|
||||
|
||||
src = open(os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
|
||||
"backend", "main.py")).read()
|
||||
call = re.search(
|
||||
r"await _cancel_and_await_tasks\(\s*idle_task,\s*worker_task,\s*preload_task,"
|
||||
r"\s*capture_preload_task,\s*timeout=([\d.]+),?\s*\)",
|
||||
src,
|
||||
)
|
||||
assert call, "production shutdown call site not found in main.py"
|
||||
assert float(call.group(1)) >= 15.0, (
|
||||
f"shutdown wait bound regressed to {call.group(1)}s — see PR #1002 review history "
|
||||
"before shrinking this"
|
||||
)
|
||||
@@ -21,6 +21,8 @@ from services.speaker_clone import (
|
||||
MIN_SLICE_DURATION_S,
|
||||
_pick_reference_slices,
|
||||
extract_speaker_clones,
|
||||
refine_ref_text,
|
||||
refine_ref_texts,
|
||||
)
|
||||
|
||||
SR = 16000
|
||||
@@ -124,3 +126,83 @@ class TestExtractSpeakerClones:
|
||||
# or every real turn boundary would be flagged.
|
||||
from services.segmentation import SPEAKER_GAP
|
||||
assert 0 < ADJACENT_TURN_GUARD_S < SPEAKER_GAP
|
||||
|
||||
|
||||
class _FakeASR:
|
||||
"""Stands in for the active ASR backend's .transcribe() — no model, no
|
||||
network. `chunks_by_path` maps a ref_audio path to the canned chunk list
|
||||
that path's re-transcription should return."""
|
||||
def __init__(self, chunks_by_path=None, raises_for=()):
|
||||
self.chunks_by_path = chunks_by_path or {}
|
||||
self.raises_for = set(raises_for)
|
||||
self.calls = []
|
||||
|
||||
def transcribe(self, path, *, word_timestamps=True):
|
||||
self.calls.append(path)
|
||||
if path in self.raises_for:
|
||||
raise RuntimeError("simulated ASR failure")
|
||||
return {"chunks": self.chunks_by_path.get(path, []), "language": "es"}
|
||||
|
||||
|
||||
class TestRefineRefText:
|
||||
# Issue #1004: the ASR segment's `text` field and its `[start, end]`
|
||||
# timestamps routinely drift (a trailing word audible in the slice but
|
||||
# missing from the text, or vice versa) — pairing a mismatched (ref_audio,
|
||||
# ref_text) breaks zero-shot TTS prompt priming badly enough that the
|
||||
# clone can speak the reference text verbatim instead of the target text.
|
||||
# Re-transcribing the actual written clip guarantees the pair matches.
|
||||
|
||||
def test_replaces_mismatched_text_with_the_actual_clip_transcript(self):
|
||||
asr = _FakeASR(chunks_by_path={
|
||||
"/tmp/ref.wav": [{"text": "hola"}, {"text": "que tal"}],
|
||||
})
|
||||
out = refine_ref_text("/tmp/ref.wav", asr, fallback_text="mismatched source text")
|
||||
assert out == "hola que tal"
|
||||
assert asr.calls == ["/tmp/ref.wav"]
|
||||
|
||||
def test_falls_back_to_original_text_on_asr_failure(self):
|
||||
asr = _FakeASR(raises_for={"/tmp/ref.wav"})
|
||||
out = refine_ref_text("/tmp/ref.wav", asr, fallback_text="original text")
|
||||
assert out == "original text"
|
||||
|
||||
def test_falls_back_to_original_text_on_empty_transcript(self):
|
||||
# A clip ASR can't get any text out of (e.g. near-silent) shouldn't
|
||||
# wipe out a usable original — empty is worse than stale.
|
||||
asr = _FakeASR(chunks_by_path={"/tmp/ref.wav": []})
|
||||
out = refine_ref_text("/tmp/ref.wav", asr, fallback_text="original text")
|
||||
assert out == "original text"
|
||||
|
||||
def test_no_asr_backend_is_a_strict_no_op(self):
|
||||
# Preflight ASR load failure, or any other reason the caller has no
|
||||
# backend to hand in — never a crash, never blocks the original path.
|
||||
out = refine_ref_text("/tmp/ref.wav", None, fallback_text="original text")
|
||||
assert out == "original text"
|
||||
|
||||
|
||||
class TestRefineRefTexts:
|
||||
def test_refines_every_entry_in_place_and_returns_the_dict(self):
|
||||
asr = _FakeASR(chunks_by_path={
|
||||
"/tmp/spk1.wav": [{"text": "hola amigo"}],
|
||||
"/tmp/spk2.wav": [{"text": "buenos dias"}],
|
||||
})
|
||||
clones = {
|
||||
"Speaker 1": {"ref_audio": "/tmp/spk1.wav", "ref_text": "stale 1"},
|
||||
"Speaker 2": {"ref_audio": "/tmp/spk2.wav", "ref_text": "stale 2"},
|
||||
}
|
||||
out = refine_ref_texts(clones, asr)
|
||||
assert out is clones # mutated in place, returned for call-and-reassign
|
||||
assert clones["Speaker 1"]["ref_text"] == "hola amigo"
|
||||
assert clones["Speaker 2"]["ref_text"] == "buenos dias"
|
||||
|
||||
def test_a_failing_entry_does_not_affect_the_others(self):
|
||||
asr = _FakeASR(
|
||||
chunks_by_path={"/tmp/spk2.wav": [{"text": "buenos dias"}]},
|
||||
raises_for={"/tmp/spk1.wav"},
|
||||
)
|
||||
clones = {
|
||||
"Speaker 1": {"ref_audio": "/tmp/spk1.wav", "ref_text": "kept on failure"},
|
||||
"Speaker 2": {"ref_audio": "/tmp/spk2.wav", "ref_text": "stale 2"},
|
||||
}
|
||||
refine_ref_texts(clones, asr)
|
||||
assert clones["Speaker 1"]["ref_text"] == "kept on failure"
|
||||
assert clones["Speaker 2"]["ref_text"] == "buenos dias"
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
"""The conftest torch-dtype guard resets a leaked default dtype between tests.
|
||||
|
||||
The CI "flaky trio" (test_effects_chain / test_generation_audio_guard /
|
||||
test_persona_bundle) failed intermittently on CI-Linux with signatures that
|
||||
all trace to one leak: some earlier test leaves
|
||||
``torch.set_default_dtype(torch.float16)`` behind. Reproduced locally with a
|
||||
simulated polluter — ``torch.tensor([0.1, …])`` under fp16 yields exactly the
|
||||
0.0999755859375 CI observed, and Pedalboard refuses fp16 audio outright
|
||||
("only supports 32-bit and 64-bit floating point"), silently returning
|
||||
unmodified audio for every preset so their outputs compare identical.
|
||||
|
||||
These two tests are order-dependent BY DESIGN (pytest runs tests within a
|
||||
file in definition order): the first leaks, the second proves the autouse
|
||||
guard in conftest.py reset the leak before the next test began.
|
||||
"""
|
||||
import torch
|
||||
|
||||
|
||||
def test_a_deliberate_dtype_leak():
|
||||
# Simulates the CI polluter. The conftest guard must clean this up (and
|
||||
# emit a UserWarning naming this exact test as the offender).
|
||||
torch.set_default_dtype(torch.float16)
|
||||
assert torch.get_default_dtype() is torch.float16
|
||||
|
||||
|
||||
def test_b_next_test_starts_back_at_float32():
|
||||
# If the guard is ever removed/broken, this fails — and so, eventually,
|
||||
# does the flaky trio on CI, much less legibly.
|
||||
assert torch.get_default_dtype() is torch.float32
|
||||
# The exact fp16 signature the trio's CI failures showed, as documentation:
|
||||
assert torch.tensor([0.1]).item() != 0.0999755859375
|
||||
Reference in New Issue
Block a user