Compare commits

..
Author SHA1 Message Date
debpalash 553d91d0e3 docs(changelog): re-merge [Unreleased] after the main merge
Each PR appends to the same [Unreleased] block, so every merge after the first
conflicts there. Rebuilt from main's version with this branch's entries
re-inserted, rather than resolving the diff — which mangles the section
structure the changelog linter enforces.
2026-07-23 05:26:46 +05:30
2342 changed files with 26889 additions and 399520 deletions
-60
View File
@@ -1,60 +0,0 @@
---
name: fastapi-python
description: Expert in FastAPI Python development with best practices for APIs and async operations
---
# FastAPI Python
You are an expert in FastAPI and Python backend development.
## Key Principles
- Write concise, technical responses with accurate Python examples
- Favor functional, declarative programming over class-based approaches
- Prioritize modularization to eliminate code duplication
- Use descriptive variable names with auxiliary verbs (e.g., `is_active`, `has_permission`)
- Employ lowercase with underscores for file/directory naming (e.g., `routers/user_routes.py`)
- Export routes and utilities explicitly
- Follow the RORO (Receive an Object, Return an Object) pattern
## Python/FastAPI Standards
- Use `def` for pure functions, `async def` for asynchronous operations
- Use type hints for all function signatures. Prefer Pydantic models over raw dictionaries
- Structure: exported router, sub-routes, utilities, static content, types (models, schemas)
- Use ordinary Python control flow; prefer readability over compressed one-line conditionals
## Error Handling
- Handle edge cases at function entry points
- Employ early returns for error conditions
- Place happy path logic last
- Avoid unnecessary else statements; use if-return patterns
- Implement guard clauses for preconditions
- Provide proper error logging and user-friendly messaging
## FastAPI-Specific Guidelines
- Use functional components (plain functions) and Pydantic models for input validation
- Declare routes with clear return type annotations
- Prefer lifespan context managers for managing startup and shutdown events
- Leverage middleware for logging, error monitoring, and optimization
- Use HTTPException for expected errors and model them as specific HTTP responses
- Apply Pydantic's BaseModel consistently for validation
## Performance Optimization
- Minimize blocking I/O. In `async def` handlers, use awaitable database/API clients; put synchronous SQLite or other blocking work in synchronous routes or explicitly offload it
- Implement caching with Redis or in-memory stores
- Optimize Pydantic serialization/deserialization
- Use lazy loading for large datasets
## Key Conventions
1. Rely on FastAPI's dependency injection system
2. Prioritize API performance metrics (response time, latency, throughput)
3. Structure routes and dependencies for readability and maintainability
## Dependencies
FastAPI, Pydantic v2, asyncpg/aiomysql, SQLAlchemy 2.0
-357
View File
@@ -1,357 +0,0 @@
---
name: vite
description: Expert guidance for Vite development with modern build tooling, HMR, framework integrations, and performance optimization
---
# Vite Development
You are an expert in Vite, modern JavaScript/TypeScript build tooling, and frontend development.
## Key Principles
- Leverage native ES modules for fast development
- Use Vite's opinionated defaults when possible
- Configure only what needs customization
- Understand the dev/build differences
- Optimize for both development speed and production performance
## Project Setup
### Basic Configuration
```typescript
// vite.config.ts
import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react';
export default defineConfig({
plugins: [react()],
server: {
port: 3000,
open: true,
},
build: {
outDir: 'dist',
sourcemap: true,
},
});
```
### Path Aliases
```typescript
import { defineConfig } from 'vite';
export default defineConfig({
resolve: {
alias: {
'@': new URL('./src', import.meta.url).pathname,
'@components': new URL('./src/components', import.meta.url).pathname,
'@utils': new URL('./src/utils', import.meta.url).pathname,
},
},
});
```
## Environment Variables
### Usage
```typescript
// .env
VITE_API_URL=https://api.example.com
VITE_APP_TITLE=My App
// In code
const apiUrl = import.meta.env.VITE_API_URL;
const isDev = import.meta.env.DEV;
const isProd = import.meta.env.PROD;
const mode = import.meta.env.MODE;
```
### Type Definitions
```typescript
// src/vite-env.d.ts
/// <reference types="vite/client" />
interface ImportMetaEnv {
readonly VITE_API_URL: string;
readonly VITE_APP_TITLE: string;
}
interface ImportMeta {
readonly env: ImportMetaEnv;
}
```
## Hot Module Replacement
### Manual HMR
```typescript
// For libraries without HMR support
if (import.meta.hot) {
import.meta.hot.accept('./module.ts', (newModule) => {
// Handle the updated module
console.log('Module updated:', newModule);
});
import.meta.hot.dispose(() => {
// Cleanup before module is replaced
});
}
```
## Asset Handling
### Static Assets
```typescript
// Import as URL
import imageUrl from './image.png';
// <img src={imageUrl} />
// Import as string (raw)
import shaderCode from './shader.glsl?raw';
// Import as worker
import Worker from './worker.ts?worker';
const worker = new Worker();
```
### Public Directory
```
public/
├── favicon.ico # Served at /favicon.ico
├── robots.txt # Served at /robots.txt
└── images/ # Served at /images/
```
## Framework Integrations
### React
```typescript
import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react';
export default defineConfig({
plugins: [
react({
// Babel plugins
babel: {
plugins: ['@emotion/babel-plugin'],
},
}),
],
});
```
### Vue
```typescript
import { defineConfig } from 'vite';
import vue from '@vitejs/plugin-vue';
export default defineConfig({
plugins: [vue()],
});
```
### Svelte
```typescript
import { defineConfig } from 'vite';
import { svelte } from '@sveltejs/vite-plugin-svelte';
export default defineConfig({
plugins: [svelte()],
});
```
## Build Optimization
### Code Splitting
```typescript
// Dynamic imports create separate chunks
const AdminPanel = lazy(() => import('./AdminPanel'));
// Manual chunks
export default defineConfig({
build: {
rollupOptions: {
output: {
manualChunks: {
vendor: ['react', 'react-dom'],
utils: ['lodash', 'date-fns'],
},
},
},
},
});
```
### Chunk Size Optimization
```typescript
export default defineConfig({
build: {
chunkSizeWarningLimit: 500,
rollupOptions: {
output: {
manualChunks(id) {
if (id.includes('node_modules')) {
return id.split('node_modules/')[1].split('/')[0];
}
},
},
},
},
});
```
## CSS Handling
### CSS Modules
```typescript
// styles.module.css is auto-detected
import styles from './styles.module.css';
// <div className={styles.container}>
```
### PostCSS
```javascript
// postcss.config.js
export default {
plugins: {
tailwindcss: {},
autoprefixer: {},
},
};
```
### Preprocessors
```typescript
// Automatically handled with package installed
// npm install -D sass
import './styles.scss';
```
## Proxy Configuration
```typescript
export default defineConfig({
server: {
proxy: {
'/api': {
target: 'http://localhost:4000',
changeOrigin: true,
rewrite: (path) => path.replace(/^\/api/, ''),
},
'/socket.io': {
target: 'ws://localhost:4000',
ws: true,
},
},
},
});
```
## Plugin Development
```typescript
// my-vite-plugin.ts
import type { Plugin } from 'vite';
export function myPlugin(): Plugin {
return {
name: 'my-plugin',
// Hook: modify config
config(config, { mode }) {
return {
define: {
__BUILD_TIME__: JSON.stringify(new Date().toISOString()),
},
};
},
// Hook: transform code
transform(code, id) {
if (id.endsWith('.md')) {
return {
code: `export default ${JSON.stringify(code)}`,
map: null,
};
}
},
// Hook: configure dev server
configureServer(server) {
server.middlewares.use((req, res, next) => {
// Custom middleware
next();
});
},
};
}
```
## Testing with Vitest
```typescript
// vitest.config.ts
import { defineConfig } from 'vitest/config';
export default defineConfig({
test: {
globals: true,
environment: 'jsdom',
setupFiles: './src/test/setup.ts',
coverage: {
provider: 'v8',
reporter: ['text', 'json', 'html'],
},
},
});
```
## SSR Configuration
```typescript
export default defineConfig({
build: {
ssr: true,
rollupOptions: {
input: './src/entry-server.ts',
},
},
ssr: {
external: ['express'],
noExternal: ['my-ui-library'],
},
});
```
## Library Mode
```typescript
export default defineConfig({
build: {
lib: {
entry: './src/index.ts',
name: 'MyLib',
fileName: (format) => `my-lib.${format}.js`,
},
rollupOptions: {
external: ['react', 'react-dom'],
output: {
globals: {
react: 'React',
'react-dom': 'ReactDOM',
},
},
},
},
});
```
## Best Practices
- Use `vite preview` to test production builds locally
- Keep dependencies that support ESM in regular deps
- Use `optimizeDeps.include` for CommonJS dependencies
- Enable `build.sourcemap` for debugging production
- Use `server.warmup` for faster dev server starts
-133
View File
@@ -1,133 +0,0 @@
---
name: owner-judge
description: Reviews proposed changes to VoiceStudio against the owner's documented standards. Use before merging any PR, before tagging a release, and whenever another agent reports work as finished. Returns a verdict with blocking findings — it judges work, it does not authorise publishing.
model: opus
tools: Bash, Read, Grep, Glob, WebFetch
---
# The owner's standing review
You review changes to **VoiceStudio** the way its owner would. You are a
**critic**, not an approver.
## What you are, precisely
You carry the owner's documented standards and apply them without flinching.
You are not the owner, and you cannot consent on their behalf. Two things
follow, and they matter:
- **You never authorise an irreversible or outward-facing action.** Publishing a
release, posting to users, deleting data, pushing to `main` — you can say
"this meets the bar" but you cannot say "go ahead". A judgement that a change
is *sound* is not permission to *ship* it. If asked to approve one of those,
say so plainly and give your technical verdict instead.
- **Your job is to find what's wrong.** A review that returns "looks good" has
usually not been done. Assume the author — human or agent — has a blind spot,
and go looking for it. Reviews that agreed with the author have already cost
this project real bugs: a fix for the Linux blank window shipped that was
**completely inert**, and a dub-pipeline fix left a resurrection race, both
caught only because a reviewer attacked them instead of agreeing.
Be fair, not hostile. A finding you cannot substantiate is noise, and noise
trains people to ignore you. Every finding needs a concrete failure: specific
input or state, and the wrong result it produces.
## The standards (from CLAUDE.md — these are load-bearing)
**Core value: a first-run that actually works.** A user who downloads the
installer should reach a working output without hitting a wall, and when
something breaks, the error or docs should say exactly what to do. Weigh
findings against this. An unactionable error message reaching a user is a real
defect here, not a nitpick.
**Fix quality.** Root-cause fully; fix the whole *class*, not the reported
instance; add a regression test that genuinely fails before and passes after;
harden against recurrence. Ask of every fix:
- Does it address the cause, or the symptom?
- Are there other instances of this same bug in the codebase, unfixed?
- Would the test actually fail without the fix? Source-text assertions
(`assert "foo(" in inspect.getsource(...)`) usually would not — they pass
when the call is unreachable or its result discarded. This project has been
bitten by exactly that.
- Is the test tautological? An assertion that holds for reasons unrelated to
the fix proves nothing.
**Cross-platform parity (strict).** A feature shipping in default mode must
behave identically on macOS, Windows, and Linux. Platform-specific
*implementation* is fine; divergent user-visible *default behaviour* is a P0 —
fix it on the missing platform or move it behind explicit opt-in. There is no
third option. Check: does this change assume a POSIX path, a shell, a
case-sensitive filesystem, an evergreen browser engine, or a GPU that some
supported platform lacks?
**Compatibility.** Existing engines must not need reinstalling. Existing
`omnivoice_data/` must keep working with no manual migration; schema changes go
through alembic with a tested upgrade path.
**Local-first.** Nothing leaves the machine without an explicit yes, and the app
stays fully functional with everything declined. No third-party endpoints for
bug reporting or crash dumps. No PAT/token-based GitHub posting from the app.
The single sanctioned external endpoint is the opt-in, consent-gated PostHog EU
analytics, which must never grow exception or DOM autocapture.
**Keep main green.** A merge must never break CI. Dependency, lockfile, and
config changes must be validated against *every* consumer — `frontend/` is a bun
workspace monorepo whose lockfile is the repo-root `bun.lock`, and
`deploy/Dockerfile` runs `bun install --frozen-lockfile`, so a `package.json`
change without a regenerated root lockfile is CI-green and Docker-red.
**Versioning.** `frontend/package.json` is the single source of truth. Three
mirrors stay in lockstep: `frontend/src-tauri/Cargo.toml`, `pyproject.toml`, and
`_FALLBACK_VERSION` in `backend/core/version.py`. Never hand-edit a mirror or
re-hardcode a literal in `tauri.conf.json`. `Cargo.lock` must match the manifest
or `cargo build --locked` fails.
**Docs-sync.** A change that alters what README, `.github/*`, or `docs/**`
describe must update those docs in the *same* change. Stale docs are bugs.
**Changelog.** Quiet and scannable: a short `**Highlights**` list in plain
words, then `### Changed` / `### Added` / `### Docs` / `### Fixed` / `### CI`
subsections where each entry is a one-liner ending in its `(#NNN)` ref with
contributor credit where due. Highlights bullets do **not** carry refs — the
`###` entries do. Never edit an already-published version's section.
**Localisation.** No hardcoded non-English user-facing text outside
`frontend/src/i18n/`. Functional CJK is allowed via the allowlist in
`tests/test_no_hardcoded_cjk.py`, with a justification.
**Mechanical rules belong in tests, not in review.** Changelog style, locale
parity, version lockstep and CJK are already enforced by pytest. Do not spend
findings on them — spend findings on what a test cannot judge: architecture,
cross-file semantics, product intent, and whether the fix is actually a fix.
## How to review
1. **Read the actual change.** `git diff origin/main...HEAD`, or the PR diff.
Never review from a description alone — the description is the author's
belief about the change, which is precisely what may be wrong.
2. **Reproduce the reasoning.** For a bug fix, find the original defect in the
code and confirm the change actually removes it. For the Linux fix mentioned
above, the give-away was that nothing in the diff could alter the search
order it claimed to alter.
3. **Run what you can.** Targeted tests, the linter, a syntax check. Verify the
regression test fails without the fix — revert the source hunk, run the test,
restore it. A test that passes both ways is not a regression test.
4. **Hunt the rest of the class.** Grep for the same idiom elsewhere. If the fix
is real and the pattern repeats, those are unfixed instances of a known bug.
5. **Check the platforms the author could not.** Most work here is done on
macOS. Windows path handling, Linux packaging, and older WebView engines are
where unverified assumptions accumulate.
## What to return
A verdict — `BLOCK`, `CONCERNS`, or `PASS` — then the findings, most severe
first. For each: the file and line, what breaks, and the concrete input or state
that breaks it. If you could not verify something important, say which and why,
rather than implying coverage you do not have.
`PASS` means "I attacked this and it held", not "I read it and nothing jumped
out". If you did not try to break it, do not return `PASS`.
State clearly when the remaining decision is the owner's — anything that
publishes to users, or any change you could not verify on the platform it
affects. Naming that boundary *is* part of the review.
-29
View File
@@ -1,29 +0,0 @@
# VoiceStudio compatibility entry
The current cross-agent package is [voicestudio](../../../../skills/voicestudio/SKILL.md).
For new installations use `npx skills add debpalash/VoiceStudio --skill voicestudio`.
Use the running backend at the user's configured address (default
`http://localhost:3900`). Check `/health`, discover `/openapi.json` and
`/v1/audio/voices`, then use the installed schema for speech, transcription,
profiles, and jobs. The HTTP MCP endpoint is `/mcp`; discover tools from the
connected server instead of assuming this older package's tool inventory.
Launch the installed Electron app if the backend is unavailable. For source
development follow the checkout's Electron README. Existing helpers in
`scripts/` support legacy source installations; inspect their environment
and dependency assumptions before running them.
Model downloads and remote services require the user's choice. Never silently
install models, promise fixed latency, or treat compatibility voice names as
real provider voices. Validate saved audio and asynchronous job completion
before reporting success. Protected backends require configured credentials;
never disable authentication to make an example work.
Source and current setup documentation:
https://github.com/debpalash/VoiceStudio
This archived entry is not an installable skill. Existing installations should
remove the old `omnivoice` / `oss-maintainer` entries and install `voicestudio` /
`voicestudio-maintainer` from the canonical repository. Legacy helpers remain
for existing users; the Electron supervisor is the preferred launcher.
+169
View File
@@ -0,0 +1,169 @@
---
name: omnivoice
description: "Local TTS, voice cloning, voice design, and video dubbing via the OmniVoice Studio MCP server (open-source ElevenLabs alternative; nothing leaves the machine, runs on MPS/CUDA/CPU). Use when: (1) generating speech from text in any of 646 languages, (2) cloning a voice from a 3-second reference clip, (3) designing a voice by gender/age/accent/pitch/style, (4) dubbing a video into another language, (5) listing voice profiles or personality presets, (6) producing narration where privacy, cost, or absent API keys matter, (7) non-English narration where Edge TTS/kokoro fall short, (8) batch audio for blog posts or content pipelines. Triggers: 'omnivoice', 'voice clone', 'clone this voice', 'tts', 'narrate', 'generate speech', 'voice synthesis', 'dub video', 'voice design', 'local tts', 'multilingual voice', 'narrate this post', 'elevenlabs alternative'."
---
# OmniVoice
## Overview
Generate audio locally via the OmniVoice Studio MCP server. Tools: `generate_speech`, `list_voices`, `list_personalities`, `list_languages`, `check_health`. Resources: `voice://{id}`, `history://recent`.
## Prerequisites — Backend Must Be Running
The MCP tools all hit `$OMNIVOICE_API_URL` (default `http://localhost:3900`). If the backend is down, every tool returns a connection error. Install + boot:
```bash
git clone https://github.com/debpalash/OmniVoice-Studio.git "$OMNIVOICE_HOME"
cd "$OMNIVOICE_HOME"
uv sync
VIRTUAL_ENV="$(pwd)/.venv" uv pip install 'mcp[cli]'
```
Then:
```bash
scripts/check-health.sh # exit 0 if up
scripts/start-backend.sh # boot in background (MPS/CUDA auto-detected)
```
First synthesis call lazy-downloads the `k2-fsa/OmniVoice` model (~2.4 GB) from HuggingFace — cached on subsequent boots.
## Task Index — Pick the Right Tool
| Task | Tool | Notes |
|---|---|---|
| Verify backend is up | `check_health` | Returns `{"status":"ok","device":"mps|cuda|cpu"}` |
| Text → audio with a saved voice | `generate_speech(text, profile_id)` | Returns base64 WAV. `profile_id="demo0001"` is the bundled demo voice |
| Text → audio without a clone (voice design) | `generate_speech(text, instruct="…")` | Omit `profile_id`; pass an `instruct` like `"warm middle-aged female narrator, calm pace"` |
| Multilingual narration | `generate_speech(text, language="es")` | Any ISO 639 code or `"Auto"` |
| List existing voices | `list_voices` | Returns id, name, type, personality |
| List personality presets | `list_personalities` | Returns narrator / casual / news-anchor / etc. with their `instruct` strings |
| List supported languages | `list_languages` | 646 total; returns 20 popular + the full count |
For non-trivial decisions (which engine to use, when to pick OmniVoice over kokoro / Edge TTS / ElevenLabs), see [references/engines-comparison.md](references/engines-comparison.md).
For MCP wiring details, backend lifecycle, troubleshooting, and a clean teardown, see [references/mcp-setup.md](references/mcp-setup.md).
## Common Workflows
### 1. One-shot narration with the demo voice
```python
# As called through the MCP client (your agent will do this for you):
result = generate_speech(
text="Hello — this is OmniVoice generating speech locally.",
profile_id="demo0001",
language="English",
steps=16, # 8 = fast/draft · 16 = balanced · 32 = quality
)
# result is JSON with audio_id, generation_time_s, audio_duration_s, format, wav_base64
```
Benchmark: 4.2 s of audio in ~24 s server-side on Apple Silicon MPS at 16 diffusion steps.
### 2. Save the WAV to disk and play
Tool returns base64 PCM WAV (16-bit, mono, 24 kHz). Decode + write:
```python
import base64, json
payload = json.loads(result_text) # parse JSON the tool returns
open("out.wav","wb").write(base64.b64decode(payload["wav_base64"]))
```
On macOS: `afplay out.wav`. Convert to MP3 with `ffmpeg -i out.wav -codec:a libmp3lame -b:a 128k out.mp3`.
### 3. Voice clone — end-to-end recipe
Cloning needs a 3-10 second reference clip the model will use as a speaker embedding. The MCP server does NOT expose profile creation — it only reads existing profiles. Two paths to create one:
**Path A — bundled helper (macOS, recommended for fresh clones):**
```bash
scripts/record-reference.sh ~/Downloads/my-ref.wav 12 1
# args: output_path raw_duration_sec mic_index
# Default mic_index=1 (MacBook built-in); list devices via:
# ffmpeg -f avfoundation -list_devices true -i ""
```
The script gives **audible** countdown + start/stop cues via macOS `say` + `/System/Library/Sounds/Ping.aiff` so the user knows when to speak (terminal stdout is buffered — text "speak now" prompts arrive too late). It records a longer raw window, then trims to ~10 seconds of speech via `silenceremove + atrim`, plays back for verification, and prints the next-step `curl` command.
**Path B — manual:**
```bash
# 1. Record (mono, 24 kHz native — matches model's internal rate)
ffmpeg -f avfoundation -i ":1" -t 12 -ac 1 -ar 24000 raw.wav
# 2. Trim leading silence + take first 10 sec of speech
ffmpeg -i raw.wav \
-af "silenceremove=start_periods=1:start_silence=0.05:start_threshold=-40dB,atrim=end=10" \
-ac 1 -ar 24000 ref.wav
# 3. Verify
ffmpeg -i ref.wav -af volumedetect -f null - 2>&1 | grep volume # max should be > -20 dB
afplay ref.wav
```
**POST to /profiles** (multipart/form-data — required fields: `name`, `ref_audio`):
```bash
curl -X POST http://127.0.0.1:3900/profiles \
-F "name=carlos-clone" \
-F "ref_audio=@ref.wav" \
-F "ref_text=The exact text spoken in the clip" \
-F "language=English" \
| python3 -m json.tool
# returns { "id": "abc12345", "name": "carlos-clone" }
```
Once created, pass `profile_id` to `generate_speech` (via MCP) or directly via `POST /generate`. Profiles persist in SQLite + reference-audio files at `~/Library/Application Support/OmniVoice/voices/<id>.<ext>` (the backend preserves the uploaded extension — `.wav` if you uploaded a WAV, `.mp3` if MP3, etc.). State persists across backend restarts.
**Reference clip tips that materially affect quality:**
| Factor | Why it matters |
|---|---|
| Single speaker | Mixed speakers blur the embedding |
| Clean speech, no music/noise | Model embeds the noise too |
| Natural prosody (avoid pangrams) | Diffusion samples replicate prosody, not just timbre |
| 3-10 sec is the sweet spot | < 3 s lacks information; > 10 s adds compute without quality gain |
| Match `ref_text` to what's spoken | Improves alignment, especially on noisy refs |
| `language` correct | Wrong language → cross-lingual transfer artifacts |
| Loudness peak ≥ -15 dB | Quiet refs work but normalize poorly |
### 4. Voice design (no reference clip)
Skip `profile_id`; provide an `instruct` string describing the desired voice:
```python
generate_speech(
text="Welcome to the future of agentic systems.",
instruct="warm middle-aged female narrator, calm authoritative pace, documentary style",
)
```
Get pre-made instructs via `list_personalities` and copy the one matching the brief (narrator, casual, news-anchor, etc.).
### 5. Video dubbing (web UI only)
The MCP server does not expose the dubbing endpoint. The full transcribe → translate → re-voice → mux pipeline lives behind the desktop UI (`bun run desktop` in `$OMNIVOICE_HOME`) and the `/dub/*` REST routes. When the user asks to dub a video, point them to the UI; surface this skill only for the synthesis primitives above.
## When NOT to use OmniVoice
- **Fast English-only narration on weak hardware** → `kokoro-tts` is ~10× smaller and 2× realtime on CPU (see [references/engines-comparison.md](references/engines-comparison.md))
- **Lowest-friction one-off TTS** → Edge TTS needs no install or backend
- **Highest possible quality regardless of cost** → ElevenLabs still wins on English narration polish; OmniVoice ties or wins on multilingual + cloning
- **Real-time streaming dictation** → use the OmniVoice desktop widget (`⌘+⇧+Space`), not the MCP server
## Resources
- [references/engines-comparison.md](references/engines-comparison.md) — Decision tree across OmniVoice / kokoro / Voicebox / Edge TTS / ElevenLabs / cloud APIs
- [references/mcp-setup.md](references/mcp-setup.md) — MCP wiring, backend lifecycle, env vars, troubleshooting
- [scripts/check-health.sh](scripts/check-health.sh) — `curl /health`, exit 0/1
- [scripts/start-backend.sh](scripts/start-backend.sh) — Start uvicorn on 127.0.0.1:3900 with health probe
- [scripts/stop-backend.sh](scripts/stop-backend.sh) — Clean shutdown via `kill -TERM` on the bound PID
- [scripts/record-reference.sh](scripts/record-reference.sh) — macOS-only: record + trim + verify a reference clip for cloning, with audible cues (`say` + system beeps) that bypass terminal output buffering
Backend Swagger / OpenAPI: `http://127.0.0.1:3900/docs` (when backend is up).
Upstream: github.com/debpalash/OmniVoice-Studio — FSL-1.1-ALv2 (free for personal/internal/non-commercial; auto-converts to Apache-2.0 two years after each release).
@@ -1,21 +1,21 @@
# TTS Engine Selection — Decision Tree
When to pick VoiceStudio vs other engines available in this workspace. Match the user's constraint to the right column.
When to pick OmniVoice vs other engines available in this workspace. Match the user's constraint to the right column.
## Decision tree
```
Is voice cloning required?
├─ yes → VoiceStudio (3-sec ref clip, zero-shot, 646 langs)
├─ yes → OmniVoice (3-sec ref clip, zero-shot, 646 langs)
└─ no →
Is the language non-English?
├─ yes → VoiceStudio (646 langs) or Edge TTS (subset, cloud)
├─ yes → OmniVoice (646 langs) or Edge TTS (subset, cloud)
└─ no (English) →
Is privacy required (no cloud)?
├─ yes →
│ Is GPU available?
│ ├─ yes (CUDA/MPS) → VoiceStudio (best quality) or Voicebox
│ └─ no (CPU only) → kokoro-tts (2× realtime CPU) or VoiceStudio on CPU (slow)
│ ├─ yes (CUDA/MPS) → OmniVoice (best quality) or Voicebox
│ └─ no (CPU only) → kokoro-tts (2× realtime CPU) or OmniVoice on CPU (slow)
└─ no (cloud OK) →
Is cost-no-object?
├─ yes → ElevenLabs (best polish), then OpenAI TTS
@@ -26,9 +26,9 @@ Is voice cloning required?
| Engine | Quality | Clone | Multilingual | Cost | Privacy | Setup | Best for |
|---|---|---|---|---|---|---|---|
| **VoiceStudio** | 8-9/10 | ✅ 3-sec ref | 646 langs | Free | Local | Bun + uv install | Multilingual, cloning, privacy-critical |
| **OmniVoice** | 8-9/10 | ✅ 3-sec ref | 646 langs | Free | Local | Bun + uv install | Multilingual, cloning, privacy-critical |
| ElevenLabs | 9-10/10 | ✅ 3-sec ref | 32 langs | $5-330/mo | Cloud | API key | Best English polish, fastest cloud TTS |
| Voicebox (Qwen3-TTS) | 8-9/10 | ✅ | Multi | Free | Local | Docker | Self-hosted alternative to VoiceStudio |
| Voicebox (Qwen3-TTS) | 8-9/10 | ✅ | Multi | Free | Local | Docker | Self-hosted alternative to OmniVoice |
| Voicebox (LuxTTS) | 7/10 | ❌ | Multi | Free | Local | Docker | CPU at 150× realtime |
| kokoro-tts | 7-8/10 | ❌ | Multi (limited) | Free | Local | pip | Fast English narration on CPU |
| mlx-audio | 7-8/10 | varies | Multi | Free | Local | pip | Apple Silicon native, 14+ sub-engines |
@@ -38,34 +38,34 @@ Is voice cloning required?
*Edge TTS is unofficial. Microsoft could block it at any time.
## When VoiceStudio wins decisively
## When OmniVoice wins decisively
1. **Voice cloning** — 3-sec reference clip, zero-shot, no fine-tuning. ElevenLabs is the only competitor; VoiceStudio is free and local.
1. **Voice cloning** — 3-sec reference clip, zero-shot, no fine-tuning. ElevenLabs is the only competitor; OmniVoice is free and local.
2. **Long-tail languages** — 646 supported. ElevenLabs covers 32; everything else fewer.
3. **Privacy / regulatory** — Nothing leaves the machine. ElevenLabs and OpenAI ship audio to their servers.
4. **No-API-key constraint** — Local-first. No accounts.
5. **Bulk generation without metered cost** — ElevenLabs bills per character. VoiceStudio is free at any volume.
5. **Bulk generation without metered cost** — ElevenLabs bills per character. OmniVoice is free at any volume.
## When VoiceStudio loses
## When OmniVoice loses
1. **Lowest-friction one-off TTS** — Backend install + ~3 GB model + uvicorn boot. Edge TTS or OpenAI TTS is one command.
2. **Fast English narration on weak hardware** — kokoro-tts is ~30 MB vs VoiceStudio's 2.4 GB and runs 2× realtime on CPU. Use kokoro for blog-narration batch jobs unless you need cloning.
3. **Streaming real-time TTS** — VoiceStudio is diffusion-based and not streaming. Use Edge TTS or cloud APIs for true streaming.
2. **Fast English narration on weak hardware** — kokoro-tts is ~30 MB vs OmniVoice's 2.4 GB and runs 2× realtime on CPU. Use kokoro for blog-narration batch jobs unless you need cloning.
3. **Streaming real-time TTS**OmniVoice is diffusion-based and not streaming. Use Edge TTS or cloud APIs for true streaming.
4. **Apple Silicon-only specialized voices**`mlx-audio` ships 14 engines (Kokoro, CSM, Dia, Qwen3-TTS, etc.) that may match a specific voice better.
## Composition with content pipelines
VoiceStudio fits between visual asset generation and video assembly:
OmniVoice fits between visual asset generation and video assembly:
```
research → narrative → visual assets → AUDIO (VoiceStudio) → video assembly → distribution
research → narrative → visual assets → AUDIO (OmniVoice) → video assembly → distribution
```
Default for blog-post audio narration:
- **English, no cloning needed, fast** → kokoro-tts (cheap CPU)
- **English, want a specific cloned voice** → VoiceStudio with a saved profile
- **Non-English** → VoiceStudio
- **English, want a specific cloned voice** → OmniVoice with a saved profile
- **Non-English** → OmniVoice
- **One-time, no install** → Edge TTS
For Remotion-based video pipelines that previously required ElevenLabs, VoiceStudio closes the last cloud dependency — pair it with any local image/video generator for a fully self-hosted multimedia stack.
For Remotion-based video pipelines that previously required ElevenLabs, OmniVoice closes the last cloud dependency — pair it with any local image/video generator for a fully self-hosted multimedia stack.
@@ -1,13 +1,13 @@
# VoiceStudio MCP Setup, Lifecycle, Troubleshooting
# OmniVoice MCP Setup, Lifecycle, Troubleshooting
## Install
```bash
# Pick any location. The scripts in this skill default to ~/VoiceStudio if
# Pick any location. The scripts in this skill default to ~/OmniVoice-Studio if
# $OMNIVOICE_HOME is unset.
export OMNIVOICE_HOME="${HOME}/VoiceStudio"
export OMNIVOICE_HOME="${HOME}/OmniVoice-Studio"
git clone https://github.com/debpalash/VoiceStudio.git "$OMNIVOICE_HOME"
git clone https://github.com/debpalash/OmniVoice-Studio.git "$OMNIVOICE_HOME"
cd "$OMNIVOICE_HOME"
uv sync # ~1.6 GB venv on darwin arm64
VIRTUAL_ENV="$(pwd)/.venv" uv pip install 'mcp[cli]' # SDK not in their lockfile yet
@@ -37,7 +37,7 @@ Drop into your MCP client config (Claude Desktop, Claude Code at `~/.claude.json
Restart the MCP client. The server only starts at client launch — in-session edits do not hot-reload.
> **Note (mcp SDK ≥ 1.10):** If you see `TypeError: FastMCP.__init__() got an unexpected keyword argument 'version'`, your `VoiceStudio` checkout is older than [debpalash/VoiceStudio#112](https://github.com/debpalash/VoiceStudio/pull/112). Either `git pull` once that PR lands, or apply the 3-line patch manually: replace `version="…", description=(…)` with `instructions=(…)` in `backend/mcp_server.py`.
> **Note (mcp SDK ≥ 1.10):** If you see `TypeError: FastMCP.__init__() got an unexpected keyword argument 'version'`, your `OmniVoice-Studio` checkout is older than [debpalash/OmniVoice-Studio#112](https://github.com/debpalash/OmniVoice-Studio/pull/112). Either `git pull` once that PR lands, or apply the 3-line patch manually: replace `version="…", description=(…)` with `instructions=(…)` in `backend/mcp_server.py`.
## Backend Lifecycle
@@ -61,7 +61,7 @@ First boot runs alembic migrations on the SQLite settings DB at `<data_dir>/omni
First synthesis call lazy-downloads the `k2-fsa/OmniVoice` model (~2.4 GB) into the HuggingFace cache. Path varies by OS:
- **macOS / Linux**: `~/.cache/huggingface/hub/`
- **Windows**: `%LOCALAPPDATA%\OmniVoice\hf_cache` (VoiceStudio redirects via `backend/core/config.py` to keep the cache off the system drive root)
- **Windows**: `%LOCALAPPDATA%\OmniVoice\hf_cache` (OmniVoice redirects via `backend/core/config.py` to keep the cache off the system drive root)
Cached on subsequent boots.
@@ -73,7 +73,7 @@ Cached on subsequent boots.
| Var | Default | Purpose |
|---|---|---|
| `OMNIVOICE_HOME` | `~/VoiceStudio` | Where the VoiceStudio repo is cloned (used by scripts in this skill) |
| `OMNIVOICE_HOME` | `~/OmniVoice-Studio` | Where the OmniVoice Studio repo is cloned (used by scripts in this skill) |
| `OMNIVOICE_API_URL` | `http://localhost:3900` | MCP server's target backend URL |
| `OMNIVOICE_TTS_BACKEND` | `omnivoice` | Switch engine: `cosyvoice`, `mlx-audio`, `voxcpm2`, `moss-tts-nano`, `kittentts` |
| `HF_TOKEN` | (none) | Only needed for gated pyannote diarization models — basic TTS does not require one |
@@ -84,7 +84,7 @@ Cached on subsequent boots.
|---|---|---|
| MCP tool returns connection error | Backend not running | `scripts/start-backend.sh` |
| `address already in use` | Stale uvicorn on 3900 | `lsof -nP -iTCP:3900 -sTCP:LISTEN``kill -TERM <pid>` |
| `FastMCP.__init__() got unexpected keyword argument 'version'` | mcp SDK ≥ 1.10 dropped `version`/`description`, checkout pre-dates [#112](https://github.com/debpalash/VoiceStudio/pull/112) | Update the checkout or apply the 3-line patch manually |
| `FastMCP.__init__() got unexpected keyword argument 'version'` | mcp SDK ≥ 1.10 dropped `version`/`description`, checkout pre-dates [#112](https://github.com/debpalash/OmniVoice-Studio/pull/112) | Update the checkout or apply the 3-line patch manually |
| First call hangs 5-10 min | Model download from HuggingFace | Watch `~/.cache/huggingface/hub/models--k2-fsa--OmniVoice/` grow |
| `/health` returns 500 | Alembic migration failed | Inspect `<data_dir>/crash_log.txt` |
| Voice profile not found | `profile_id` invalid or profile not yet created | `list_voices` first to get valid IDs |
@@ -99,4 +99,4 @@ scripts/stop-backend.sh # graceful shutdown
# Remove the `omnivoice` entry from your MCP client config
```
User profiles + history live in the platform data dir (`~/Library/Application Support/OmniVoice/` on macOS; `~/.local/share/VoiceStudio/` on Linux). Preserve across reinstalls if you want to keep your saved voice profiles.
User profiles + history live in the platform data dir (`~/Library/Application Support/OmniVoice/` on macOS; `~/.local/share/OmniVoice/` on Linux). Preserve across reinstalls if you want to keep your saved voice profiles.
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# Start the OmniVoice FastAPI backend on 127.0.0.1:3900, detached, idempotent.
# Honors $OMNIVOICE_HOME (default ~/VoiceStudio).
# Honors $OMNIVOICE_HOME (default ~/OmniVoice-Studio).
#
# Exit codes:
# 0 success (already running, or freshly started + healthy within 60s)
@@ -11,7 +11,7 @@
set -euo pipefail
HOME_DIR="${OMNIVOICE_HOME:-$HOME/VoiceStudio}"
HOME_DIR="${OMNIVOICE_HOME:-$HOME/OmniVoice-Studio}"
URL="${OMNIVOICE_API_URL:-http://127.0.0.1:3900}"
LOG="$HOME_DIR/backend.log"
+1 -5
View File
@@ -98,11 +98,7 @@ reviews:
access accordingly; window and webview lifecycle on all three OSes;
child-process spawn/exit-code/stderr handling; no unwrap/expect on
user-controlled input; platform cfg blocks keep user-visible defaults
identical across macOS/Windows/Linux. The parity rule covers BEHAVIOUR,
not PERFORMANCE: hardware acceleration is host-dependent by design
(CUDA/MPS/DirectML, Triton availability, torch.compile), so an
optimization skipped where it cannot work is NOT a parity violation and
must not be reported as one.
identical across macOS/Windows/Linux.
- path: "tests/**/*.py"
instructions: >-
Review as a test-infrastructure engineer. Check: the test would fail
+1 -1
View File
@@ -46,7 +46,7 @@ an individual is officially representing the community in public spaces.
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
**VoiceStudio@palash.dev**.
**OmniVoice@palash.dev**.
All complaints will be reviewed and investigated promptly and fairly.
+20 -72
View File
@@ -1,24 +1,18 @@
# Contributing to VoiceStudio
# Contributing to OmniVoice Studio
Thanks for your interest in improving VoiceStudio! This guide covers everything you need to get started.
Thanks for your interest in improving OmniVoice Studio! This guide covers everything you need to get started.
## Quick Links
| | |
|---|---|
| 💬 **Chat** | [Discord](https://discord.gg/bzQavDfVV9) |
| 🐛 **Bugs** | [GitHub Issues](https://github.com/debpalash/VoiceStudio/issues) |
| 🏷️ **Good First Issues** | [Filtered list](https://github.com/debpalash/VoiceStudio/labels/good%20first%20issue) |
| 🐛 **Bugs** | [GitHub Issues](https://github.com/debpalash/OmniVoice-Studio/issues) |
| 🏷️ **Good First Issues** | [Filtered list](https://github.com/debpalash/OmniVoice-Studio/labels/good%20first%20issue) |
| 📋 **Roadmap** | [README → Roadmap](README.md#roadmap) |
---
## Adding a TTS or ASR engine
New engines are hired for a **named job**, not added to a list — the bar, the current job map,
and the out-of-tree path are in [docs/engine-acceptance.md](../docs/engine-acceptance.md).
Read it before opening a proposal; the licence check in particular ends most of them.
## Development Setup
### Prerequisites
@@ -28,45 +22,18 @@ Read it before opening a proposal; the licence check in particular ends most of
- [Bun](https://bun.sh/) (frontend package manager)
- [uv](https://docs.astral.sh/uv/) (Python environment manager)
- [ffmpeg](https://ffmpeg.org/) (audio/video processing)
- [Rust / Cargo](https://rustup.rs/) (desktop shell only)
- Python 3.10+ (managed automatically by `uv`)
Linux desktop development also needs WebKitGTK/GTK development libraries. On
Debian or Ubuntu, install the same packages used by CI:
```bash
sudo apt-get update
sudo apt-get install -y \
libwebkit2gtk-4.1-dev libgtk-3-dev libpango1.0-dev libcairo2-dev \
libsoup-3.0-dev libgdk-pixbuf-2.0-dev \
libayatana-appindicator3-dev librsvg2-dev libssl-dev libxdo-dev \
gstreamer1.0-plugins-good \
libasound2-dev build-essential curl wget file
```
See the [Linux source-build guide](../docs/install/linux.md#building-from-source)
for Fedora and Arch packages.
### Clone & Run
```bash
git clone https://github.com/debpalash/VoiceStudio.git
cd VoiceStudio
git clone https://github.com/debpalash/OmniVoice-Studio.git
cd OmniVoice-Studio
bun install
bun run dev
```
This launches Electron with hot reload. Its runtime supervisor manages backend setup
and startup; do not launch a second backend. See [Electron setup](../electron/README.md).
```bash
bun run build # build Electron
bun run start # launch the built Electron app
bun run dist # package locally without publishing
bun run dev:web # legacy browser UI + backend
```
The legacy browser command starts both services:
This starts both services:
| Service | URL | What it does |
|---------|-----|---|
@@ -81,38 +48,20 @@ cause doesn't scroll away with the terminal. The same death is also reported
as a crash notice in the UI the next time the backend starts (see
[docs/install/troubleshooting.md §14c](docs/install/troubleshooting.md)).
### Legacy Desktop App (Tauri)
### Desktop App (Tauri)
```bash
bun run tauri # legacy dev: hot-reload Tauri shell + backend
bun run tauri:desktop-prod # legacy production: builds, bundles the backend, then launches
bun run desktop # dev: hot-reload Tauri shell + backend
bun run desktop-prod # production: builds, bundles the backend, then launches
```
Both run `uv sync` first (so the Python backend env is set up) and start the
backend automatically — you do **not** start it separately. Use the exact script
names: there is no `desktop=prod` (note the **hyphen** in `tauri:desktop-prod`).
`tauri:desktop-prod` is Windows-aware (auto-detects bash/git; see `scripts/desktop-prod.mjs`).
names: there is no `desktop=prod` (note the **hyphen** in `desktop-prod`).
`desktop-prod` is Windows-aware (auto-detects bash/git; see `scripts/desktop-prod.mjs`).
Requires [Rust](https://rustup.rs/) and platform-specific Tauri dependencies — see the [Tauri prerequisites](https://v2.tauri.app/start/prerequisites/).
After installing Rust with rustup (or `uv` with its installer), a terminal that
was already open still has the old `PATH`. The desktop launchers (`bun tauri`,
`bun tauri:desktop-prod`, `bun tauri:desktop-fresh`) detect this and add `~/.cargo/bin` /
`~/.local/bin` for that run, printing a one-line note; to make it permanent,
open a new terminal, or on macOS/Linux load Cargo into the current one:
```bash
source "$HOME/.cargo/env"
bun run tauri
```
If Rust is genuinely not installed, the launchers stop up front with the
install command instead of failing later inside `cargo metadata`.
On Linux, errors such as `Package gdk-3.0 was not found`, `pango.pc` missing,
or `javascriptcoregtk-4.1` missing mean the native packages above were not
installed; changing `PKG_CONFIG_PATH` does not fix libraries that are absent.
If the app opens but stays on the **setup splash with no buttons**, the Python
backend didn't finish starting — the splash surfaces the stall reason, a log
panel, and a **Retry** button (and Settings → Logs → Backend has the full trace).
@@ -123,7 +72,7 @@ The most common from-source cause is `uv` or Python not being on your PATH.
## Project Structure
```
VoiceStudio/
OmniVoice-Studio/
├── backend/ # Python FastAPI server
│ ├── api/ # Route handlers
│ ├── core/ # Config, prefs, constants
@@ -147,7 +96,7 @@ VoiceStudio/
### Bug Reports
Open an [issue](https://github.com/debpalash/VoiceStudio/issues/new) with:
Open an [issue](https://github.com/debpalash/OmniVoice-Studio/issues/new) with:
1. **What happened** vs **what you expected**
2. **Steps to reproduce**
@@ -171,7 +120,7 @@ Open an [issue](https://github.com/debpalash/VoiceStudio/issues/new) with:
### Adding a New TTS Engine
VoiceStudio's TTS backend is a plugin registry. Adding a new engine takes ~50 lines:
OmniVoice's TTS backend is a plugin registry. Adding a new engine takes ~50 lines:
1. Open `backend/services/tts_backend.py`
2. Create a class extending `TTSBackend`:
@@ -219,8 +168,7 @@ class MyEngineBackend(TTSBackend):
- **Components**: Functional components with hooks
- **State**: Zustand stores in `src/stores/`, organized by slice
- **Brand assets**: Reuse the canonical mark, palette, naming, and compatibility rules in [`docs/branding.md`](../docs/branding.md); do not redraw or rename runtime identifiers ad hoc
- **CSS**: **Utilities-first + shadcn/ui, one stylesheet.** UI is built on the shadcn/ui primitives in `src/components/ui/` (wrapped by the `src/ui/` barrel, themed to the VoiceStudio palette), composed with Tailwind v4 utility classes. **All styling now lives in a single file — `src/index.css`**: the `@theme` / `[data-theme]` token foundation plus the irreducible set utilities can't express (`@keyframes`, glassmorphism/`backdrop-filter`, pseudo-elements, `:has()`, unlayered cascade overrides, and styling hooks on library-generated DOM like virtualized rows / WaveSurfer). The per-component `.css` files were eliminated in the CSS→Tailwind/shadcn migration — **do not create new ones.** Reach for shadcn primitives + utilities; if a rule is genuinely irreducible, add it to `src/index.css` with a provenance comment. (The only other `.css` is the test-only visual harness. See `docs/shadcn-migration.md`.)
- **CSS**: **Utilities-first + shadcn/ui, one stylesheet.** UI is built on the shadcn/ui primitives in `src/components/ui/` (wrapped by the `src/ui/` barrel, themed to the OmniVoice palette), composed with Tailwind v4 utility classes. **All styling now lives in a single file — `src/index.css`**: the `@theme` / `[data-theme]` token foundation plus the irreducible set utilities can't express (`@keyframes`, glassmorphism/`backdrop-filter`, pseudo-elements, `:has()`, unlayered cascade overrides, and styling hooks on library-generated DOM like virtualized rows / WaveSurfer). The per-component `.css` files were eliminated in the CSS→Tailwind/shadcn migration — **do not create new ones.** Reach for shadcn primitives + utilities; if a rule is genuinely irreducible, add it to `src/index.css` with a provenance comment. (The only other `.css` is the test-only visual harness. See `docs/shadcn-migration.md`.)
- **Naming**: `PascalCase` for components, `camelCase` for hooks and utils
### Rust (Tauri)
@@ -320,7 +268,7 @@ that — the agent recalls the architecture, conventions, and your past findings
instead of re-reading the tree each time. [**memxt**](https://github.com/debpalash/memxt)
(100% local, MCP-based, built by this project's maintainer) exists for exactly
this; any MCP memory server works. Pair it with the repo's agent skill —
`npx skills add debpalash/VoiceStudio` — so your agent knows the project's
`npx skills add debpalash/omnivoice-studio` — so your agent knows the project's
hard rules from the first prompt.
## Quality gates your PR must pass
@@ -349,7 +297,7 @@ hard rules from the first prompt.
## Contribution licensing
VoiceStudio is **AGPL-3.0-only**, and the maintainer also offers a
OmniVoice Studio is **AGPL-3.0-only**, and the maintainer also offers a
**commercial license** (see [LICENSE](LICENSE)). By submitting a contribution
you agree that:
@@ -369,7 +317,7 @@ appreciated but not required.
## Need Help?
- **Stuck on setup?** Ask in [Discord #help](https://discord.gg/bzQavDfVV9)
- **Not sure where to start?** Check [good first issues](https://github.com/debpalash/VoiceStudio/labels/good%20first%20issue)
- **Want to discuss a big change?** Open a [discussion](https://github.com/debpalash/VoiceStudio/discussions) or Discord thread before coding
- **Not sure where to start?** Check [good first issues](https://github.com/debpalash/OmniVoice-Studio/labels/good%20first%20issue)
- **Want to discuss a big change?** Open a [discussion](https://github.com/debpalash/OmniVoice-Studio/discussions) or Discord thread before coding
Thank you for contributing! 🎙️
+1 -1
View File
@@ -4,6 +4,6 @@
ko_fi: debpalash
custom:
- "https://paypal.me/palashCoder"
- "https://github.com/debpalash/VoiceStudio/blob/main/SPONSORS.md"
- "https://github.com/debpalash/OmniVoice-Studio/blob/main/SPONSORS.md"
# github: [debpalash] # not available
# open_collective: omnivoice-studio
+2 -2
View File
@@ -6,7 +6,7 @@ body:
- type: markdown
attributes:
value: |
Thanks for helping improve VoiceStudio! 🎙️
Thanks for helping improve OmniVoice Studio! 🎙️
**Fastest path to a fix:** **Settings → About → "Save diagnostic bundle"** makes a
zip (self-check + recent errors + scrubbed log tails) — drag it onto this issue and
@@ -17,7 +17,7 @@ body:
attributes:
label: Before filing
options:
- label: I searched [existing issues](https://github.com/debpalash/VoiceStudio/issues?q=is%3Aissue) and this isn't a duplicate.
- label: I searched [existing issues](https://github.com/debpalash/OmniVoice-Studio/issues?q=is%3Aissue) and this isn't a duplicate.
required: true
- label: I'm on the latest release (or `main`) — older builds may already be fixed.
required: false
+2 -2
View File
@@ -4,8 +4,8 @@ contact_links:
url: https://discord.gg/bzQavDfVV9
about: Usage questions, setup help, and chat. Faster than an issue for "how do I…".
- name: 🗣️ GitHub Discussions
url: https://github.com/debpalash/VoiceStudio/discussions
url: https://github.com/debpalash/OmniVoice-Studio/discussions
about: Ideas, show-and-tell, and open-ended Q&A that isn't a bug or a specific feature ask.
- name: 🔒 Security vulnerability
url: https://github.com/debpalash/VoiceStudio/security/policy
url: https://github.com/debpalash/OmniVoice-Studio/security/policy
about: Please report security issues privately — do NOT open a public issue.
+2 -2
View File
@@ -8,7 +8,7 @@ body:
attributes:
label: Before filing
options:
- label: I searched [existing issues](https://github.com/debpalash/VoiceStudio/issues?q=is%3Aissue) and [discussions](https://github.com/debpalash/VoiceStudio/discussions) for this idea.
- label: I searched [existing issues](https://github.com/debpalash/OmniVoice-Studio/issues?q=is%3Aissue) and [discussions](https://github.com/debpalash/OmniVoice-Studio/discussions) for this idea.
required: true
- type: textarea
id: problem
@@ -45,6 +45,6 @@ body:
- type: markdown
attributes:
value: |
> VoiceStudio is **local-first** — core features work offline without an account,
> OmniVoice is **local-first** — features must work fully offline with no accounts,
API keys, or cloud calls, and behave identically on macOS/Windows/Linux. Proposals
that fit those constraints are easiest to land.
+6 -6
View File
@@ -1,15 +1,15 @@
name: 🤝 Sponsorship inquiry
description: Support VoiceStudio and (optionally) claim a logo slot. Not for bugs or feature requests.
description: Support OmniVoice and (optionally) claim a logo slot. Not for bugs or feature requests.
title: "Sponsorship inquiry: "
labels: ["sponsor"]
body:
- type: markdown
attributes:
value: |
Thanks for considering sponsoring **VoiceStudio** 💛
Thanks for considering sponsoring **OmniVoice Studio** 💛
VoiceStudio is free, local-first, and AGPL-3.0 — sponsorship keeps development going.
See **[SPONSORS.md](https://github.com/debpalash/VoiceStudio/blob/main/SPONSORS.md)** for tiers, placements, and logo guidelines.
OmniVoice is free, local-first, and AGPL-3.0 — sponsorship keeps development going.
See **[SPONSORS.md](https://github.com/debpalash/OmniVoice-Studio/blob/main/SPONSORS.md)** for tiers, placements, and logo guidelines.
Prefer to just donate? [Ko-fi](https://ko-fi.com/debpalash) (recurring) or [PayPal](https://paypal.me/palashCoder) (one-time) — you don't need this form for that.
- type: input
id: name
@@ -72,7 +72,7 @@ body:
attributes:
label: Acknowledgements
options:
- label: I understand sponsorship is a thank-you, not a paywall — VoiceStudio stays fully free and AGPL-3.0, and sponsors don't get gated features.
- label: I understand sponsorship is a thank-you, not a paywall — OmniVoice stays fully free and AGPL-3.0, and sponsors don't get gated features.
required: true
- label: If I provide a logo, I have the right to use it and grant VoiceStudio permission to display it in the README, the app, and the project website.
- label: If I provide a logo, I have the right to use it and grant OmniVoice permission to display it in the README, the app, and the project website.
required: false
+5 -5
View File
@@ -4,13 +4,13 @@
| Version | Supported |
|---------|-----------|
| 0.5.x (latest release + `main` previews) | ✅ Current — all fixes land here |
| 0.3.x (latest release + `main` previews) | ✅ Current — all fixes land here |
| 0.2.7 | ⚠️ Legacy stable — security fixes only, upgrade recommended |
| < 0.2.7 | ❌ No longer supported |
## Model supply chain
VoiceStudio supports models from **public, verifiable sources only** (Hugging
OmniVoice supports models from **public, verifiable sources only** (Hugging
Face repos, official project releases). Privately sold or gated model files
are not supported: an archive from a private source can carry anything
(bundled executables, modified configs), and nobody else can verify or
@@ -23,7 +23,7 @@ download, and never run executables bundled with model archives.
Instead, report them privately via one of these channels:
1. **GitHub Security Advisories** (preferred) — [Report a vulnerability](https://github.com/debpalash/VoiceStudio/security/advisories/new)
1. **GitHub Security Advisories** (preferred) — [Report a vulnerability](https://github.com/debpalash/OmniVoice-Studio/security/advisories/new)
2. **Email** — Send details to **security@palash.dev**
### What to include
@@ -44,7 +44,7 @@ Instead, report them privately via one of these channels:
### Scope
VoiceStudio runs **100% locally** by default. The primary attack surface is:
OmniVoice Studio runs **100% locally** by default. The primary attack surface is:
- **Network exposure** — if the user binds to `0.0.0.0` without a reverse proxy
- **Model downloads** — fetched from Hugging Face Hub over HTTPS
@@ -71,6 +71,6 @@ GitHub Apps on creation.
## Security Best Practices for Users
- **Do not expose VoiceStudio to the internet without authentication.** The API has no built-in auth. Use a reverse proxy (Caddy, nginx, Tailscale) if you need remote access.
- **Do not expose OmniVoice to the internet without authentication.** The API has no built-in auth. Use a reverse proxy (Caddy, nginx, Tailscale) if you need remote access.
- **Keep your installation updated.** The desktop app auto-checks for updates via the built-in updater.
- **Review model sources.** Only download models from trusted Hugging Face repositories.
+5 -5
View File
@@ -5,21 +5,21 @@
| Channel | Best for |
|---|---|
| [Discord](https://discord.gg/bzQavDfVV9) — `#help` | Setup problems, quick questions, sharing results |
| [GitHub Issues](https://github.com/debpalash/VoiceStudio/issues) | Bugs and feature requests — use the templates; attach the diagnostic bundle (Settings → About → "Save diagnostic bundle") |
| [GitHub Discussions](https://github.com/debpalash/VoiceStudio/discussions) | Design questions, ideas, show & tell |
| [GitHub Issues](https://github.com/debpalash/OmniVoice-Studio/issues) | Bugs and feature requests — use the templates; attach the diagnostic bundle (Settings → About → "Save diagnostic bundle") |
| [GitHub Discussions](https://github.com/debpalash/OmniVoice-Studio/discussions) | Design questions, ideas, show & tell |
| Security issues | **Never a public issue** — see [SECURITY.md](SECURITY.md) for private reporting |
## Uninstalling / removing all data
VoiceStudio is fully local — there's nothing to deactivate, just folders to
OmniVoice is fully local — there's nothing to deactivate, just folders to
delete. `scripts/uninstall.sh` (macOS/Linux) or `scripts\uninstall.ps1`
(Windows) lists every VoiceStudio folder with its size (dry-run first) and
(Windows) lists every OmniVoice folder with its size (dry-run first) and
removes them on `--yes`. The complete per-platform path list is in
[docs/install/uninstall.md](docs/install/uninstall.md).
## Model sources we support
VoiceStudio is built on the idea that everything it runs is **open and available
OmniVoice is built on the idea that everything it runs is **open and available
to everyone**: free, public models with verifiable sources and licenses
(Hugging Face repos, official project releases), so the whole community can
use, test, and debug the same thing.
+1 -1
View File
@@ -36,7 +36,7 @@
## Release cadence
VoiceStudio ships **continuous-to-main** — no release candidates, no soak windows.
OmniVoice ships **continuous-to-main** — no release candidates, no soak windows.
Every merged PR is immediately part of the rolling preview (`main`, Docker
`:latest`, the desktop Preview channel). Versioned releases are tagged from
`main` when it's ready; `main` then bumps to the next patch automatically.
+5 -20
View File
@@ -51,21 +51,15 @@ jobs:
- os: ubuntu-latest
platform: linux-x86_64
experimental: false
- os: ubuntu-24.04-arm
platform: linux-aarch64
# Apple Silicon under Asahi Linux. Experimental: the Vulkan
# (Honeykrisp GPU) build path is new and the hosted arm64
# runner has no GPU — it validates that the binary builds;
# on-host Vulkan acceleration is exercised by users.
experimental: true
- os: windows-latest
platform: windows-x86_64
experimental: false
- os: macos-14
platform: darwin-arm64
# Apple Silicon Metal build compiles cleanly with -DGGML_METAL=ON
# at the pinned SHA (#2105); non-experimental to catch regressions.
experimental: false
# Metal build path is unpublished upstream (Pitfall 1 in
# 04-RESEARCH.md); experimental so a failed Metal build doesn't
# block — the SPIKE-01 ADR records the in-process fallback.
experimental: true
steps:
- uses: actions/checkout@v4
with:
@@ -86,18 +80,11 @@ jobs:
# Linux-only: upstream `buildcpu.sh` enables `-DGGML_BLAS=ON` which
# requires a system BLAS implementation at cmake configure time.
- name: Linux system deps (BLAS for ggml-blas backend)
if: startsWith(matrix.platform, 'linux')
if: matrix.platform == 'linux-x86_64'
run: |
sudo apt-get update
sudo apt-get install -y libopenblas-dev pkg-config
# linux-aarch64: let the build script's Vulkan path (Honeykrisp GPU
# on Asahi) engage instead of silently falling back to CPU.
- name: Vulkan dev deps (linux-aarch64 GPU backend)
if: matrix.platform == 'linux-aarch64'
run: |
sudo apt-get install -y glslc libvulkan-dev spirv-headers
- name: Build omnivoice-tts
shell: bash
# Pass values through env (quoted) rather than ${{ }} interpolation
@@ -115,6 +102,4 @@ jobs:
name: omnivoice-tts-${{ matrix.platform }}
path: |
bin/omnivoice-tts-${{ matrix.platform }}*
bin/libggml*
bin/ggml*.dll
bin/checksums.sha256
+16 -224
View File
@@ -11,11 +11,6 @@ on:
push:
branches: [main]
workflow_dispatch:
inputs:
windows_wix_diagnostic:
description: Run only the tiny nonpublishing Windows MSI authoring diagnostic
type: boolean
default: false
permissions:
contents: read
@@ -26,15 +21,8 @@ env:
jobs:
test:
if: ${{ !inputs.windows_wix_diagnostic }}
name: Tests (backend + frontend)
runs-on: ubuntu-22.04
env:
# Same restricted-network resilience the smoke matrix already sets. This
# job resolves the same direct-URL dependency and had none of it, which
# is why it was the one that kept dying (see scripts/uv-sync-retry.sh).
UV_HTTP_TIMEOUT: "120"
UV_HTTP_RETRIES: "5"
steps:
- uses: actions/checkout@v4
@@ -63,7 +51,7 @@ jobs:
# apt install ffmpeg is ~30 s every run; cache the resolved .debs.
- name: System deps (ffmpeg)
uses: awalsh128/cache-apt-pkgs-action@v1.6.3
uses: awalsh128/cache-apt-pkgs-action@latest
with:
packages: ffmpeg
version: 1.0
@@ -73,14 +61,7 @@ jobs:
# so their tests can exercise the real import path, not the
# "package not installed" fallback. Smoke job below stays on bare
# `uv sync` because smoke only hits /health + fixture profiles.
#
# Retried because one dependency — en-core-web-sm — resolves to a
# direct GitHub release URL, and github.com intermittently answers
# `http2 error: refused stream before processing any application
# logic`. uv's own 3 retries all land inside the same few seconds and
# fail together, which has cost otherwise-green runs (#1517, #1518).
# Backing off between whole attempts is what actually clears it.
run: bash scripts/uv-sync-retry.sh --all-extras
run: uv sync --all-extras
# HF_HUB_OFFLINE=1 is a recurrence guard, not an optimization: a test
# that reaches huggingface.co fails fast and loud instead of silently
@@ -90,7 +71,7 @@ jobs:
# interactions in tests are stubbed; anything that trips this is a
# test-isolation bug.
- name: Run pytest
run: uv run --no-sync pytest tests/ -q --tb=short
run: uv run pytest tests/ -q --tb=short
env:
HF_HUB_OFFLINE: "1"
@@ -102,23 +83,12 @@ jobs:
- name: Validate install docs against desktop-prod.sh
run: python scripts/validate-install-docs.py
# The AppImage launcher decides which WebKitGTK actually runs — the wrong
# answer is a permanently blank window on Linux (#56, #961, #1258), and
# the only place that logic is exercised is this shell harness. It had
# never been wired into CI, so its cases were a regression test nothing
# ran. Cheap (pure bash, stubs pkg-config) and it gates the class.
- name: AppImage launcher (AppRun) unit tests
run: |
bash frontend/src-tauri/appimage/AppRun.test.sh
bash scripts/inject-apprun.test.sh
bash scripts/verify-apprun-bundle.test.sh
# `backend/tests/` mounts routers on bare FastAPI apps (no heavy main
# import chain) with a hermetic data dir from its conftest.py. It no
# longer stubs sys.modules, so mixed sessions with tests/ are safe;
# the separate session is kept for cheaper, clearer CI output.
- name: Run pytest (backend/tests, isolated)
run: uv run --no-sync pytest backend/tests/ -q --tb=short
run: uv run pytest backend/tests/ -q --tb=short
env:
HF_HUB_OFFLINE: "1" # same no-silent-downloads guard as tests/
@@ -172,12 +142,6 @@ jobs:
working-directory: frontend
run: node --experimental-strip-types --no-warnings --test ../tests/frontend/*.test.mjs
# Electron used to be built only after a release started, so renderer,
# preload and packaging regressions could pass the required PR gate.
# Keep this command shared with release.yml through the root script.
- name: Electron typecheck, tests and production contract
run: bun run check:electron
# Production-bundle blank-screen gate. Everything above runs UN-minified
# (dev server + Vitest/jsdom), so a crash that exists ONLY in the minified
# release bundle — a TDZ reorder that throws before React mounts — passes
@@ -192,28 +156,6 @@ jobs:
- name: Production-bundle smoke — no blank screen
working-directory: frontend
run: bun run test:prod-bundle
- name: Electron renderer workflow smokes
shell: bash
run: |
set -euo pipefail
export OMNIVOICE_PORT=3999
export VOICESTUDIO_UI_URL=http://localhost:3912
export PLAYWRIGHT_CHANNEL=chromium
bun run --cwd electron smoke:server > /tmp/voicestudio-electron-smoke.log 2>&1 &
server_pid=$!
trap 'kill "$server_pid" 2>/dev/null || true' EXIT
for _ in {1..60}; do
if curl --fail --silent --show-error "$VOICESTUDIO_UI_URL" >/dev/null; then
break
fi
sleep 0.25
done
curl --fail --silent --show-error "$VOICESTUDIO_UI_URL" >/dev/null || {
cat /tmp/voicestudio-electron-smoke.log
exit 1
}
node electron/tests/playback-smoke.mjs
node electron/tests/dub-smoke.mjs
# ── Cross-platform Tauri shell check ────────────────────────────────────
# Catches platform-specific Rust regressions on PR (cfg(target_os=...)
@@ -223,7 +165,6 @@ jobs:
# and `cargo test --lib` runs the shell's unit tests natively on each OS.
# Full bundling stays in release.yml on tag push.
tauri-cross-platform:
if: ${{ !inputs.windows_wix_diagnostic }}
name: Tauri shell check (${{ matrix.label }})
needs: test
strategy:
@@ -306,17 +247,6 @@ jobs:
working-directory: frontend/src-tauri
run: cargo test --lib --target ${{ matrix.rust_target }} --message-format=short
# Backend-lifecycle fault-injection harness: real child processes die
# scripted deaths through the OMNIVOICE_BACKEND_CMD seam, and each
# scenario asserts the user-visible diagnosis names the actual cause
# (port conflict / traceback root cause / spawn failure / timeout /
# crash-loop exhaustion / signal 9 / deliberate replace / deferred-
# startup step). Serial: the scenarios share process-global state
# (env vars, crash store, kill-intended flag) by design.
- name: Cargo test (backend lifecycle harness)
working-directory: frontend/src-tauri
run: cargo test --test backend_lifecycle --target ${{ matrix.rust_target }} --message-format=short -- --test-threads=1
# ── Cross-platform Python runtime smoke (Phase 0 GATE-02) ───────────────
# Loads the frozen tests/fixtures/omnivoice_data/ fixture and boots the
# FastAPI app in-process via TestClient on macOS/Windows/Linux. Catches
@@ -324,7 +254,6 @@ jobs:
# job above misses. Narrow scope (tests/smoke/ only) — full pytest stays
# on Linux until Phase 1's INST-01 lands setuptools for WhisperX.
smoke-matrix:
if: ${{ !inputs.windows_wix_diagnostic }}
name: Smoke (${{ matrix.label }})
needs: test
strategy:
@@ -333,33 +262,12 @@ jobs:
include:
- os: macos-14
label: macOS
backend_supported: true
- os: macos-15-intel
label: macOS Intel
backend_supported: false
- os: windows-2022
label: Windows
backend_supported: true
- os: ubuntu-22.04
label: Linux
backend_supported: true
runs-on: ${{ matrix.os }}
# Priced for a COLD `uv sync`, on every platform.
#
# The previous split (Windows 25, Linux/macOS 10) came from a warm-cache
# measurement — Linux and macOS finish in ~65 s when setup-uv restores its
# cache, so 10 looked generous. Then run 30439640107 hit
# "Failed to restore: Cache service responded with 400", Linux installed
# torch from scratch, and the leg was killed at 10m17s. The 65 s was the
# cache, not the platform.
#
# A cache miss is not rare enough to treat as an outage (GitHub's cache
# service 400s, a lockfile change invalidates the key, a new runner image
# starts empty), and a timeout here is self-perpetuating: the leg dies
# before the post-step saves the cache, so the next run is cold too.
# 25 everywhere is still bounded — a genuinely wedged job is caught in
# minutes, not hours — and warm runs land nowhere near it.
timeout-minutes: 25
timeout-minutes: 10
env:
# Restricted-network resilience (RESEARCH Pitfall #6) — keeps uv from
# giving up on the first slow PyPI / python-build-standalone fetch.
@@ -383,149 +291,33 @@ jobs:
# though the silence WAV doesn't decode anything heavy — keeps test
# collection from import-erroring on optional audio modules.
- name: System deps (macOS)
if: runner.os == 'macOS' && matrix.backend_supported
if: runner.os == 'macOS'
run: brew install ffmpeg libsndfile || true
- name: System deps (Windows)
if: runner.os == 'Windows' && matrix.backend_supported
if: runner.os == 'Windows'
shell: bash
run: |
# The community chocolatey feed 50x's intermittently (broke PR runs on
# 2026-07-20 and 2026-07-28) — retry with backoff before failing.
#
# Test the OUTCOME, not choco's exit code. On 2026-07-28 the feed
# returned 503, choco reported "Unable to find package 'ffmpeg'" and
# "installed 0/0 packages" — and still exited 0. The `&& break` that
# was supposed to guard this fired on the first attempt, no retry ran,
# and the job died one line later on `ffmpeg: command not found`.
# A retry that trusts a lying exit code is not a retry.
# The community chocolatey feed 504s intermittently (broke a PR run
# on 2026-07-20) — retry with backoff before failing the job.
for i in 1 2 3; do
choco install ffmpeg -y --no-progress || true
hash -r 2>/dev/null || true
if command -v ffmpeg >/dev/null 2>&1; then break; fi
# No backoff after the last attempt — there is no fourth try to
# wait for, and sleeping 90s only delays an already-doomed job.
if [ "$i" -eq 3 ]; then
echo "choco failed to produce ffmpeg after 3 attempts"
break
fi
echo "choco attempt $i did not produce ffmpeg — retrying in $((i * 30))s"
choco install ffmpeg -y --no-progress && break
echo "choco attempt $i failed — retrying in $((i * 30))s"
sleep $((i * 30))
done
# Chocolatey is one distribution channel, not the dependency. When
# its feed is down across every retry (2026-08-13: three attempts,
# three 'installed 0/1'), fall back to the static gyan.dev release
# build GitHub mirror — the same binary, no feed in the path.
if ! command -v ffmpeg >/dev/null 2>&1; then
echo "::warning::choco feed down — falling back to static ffmpeg build"
curl -fsSL --retry 3 -o /tmp/ffmpeg.zip \
https://github.com/GyanD/codexffmpeg/releases/download/7.1/ffmpeg-7.1-essentials_build.zip
unzip -q /tmp/ffmpeg.zip -d /tmp/ffmpeg
bindir=$(dirname "$(find /tmp/ffmpeg -name ffmpeg.exe | head -1)")
echo "$bindir" >> "$GITHUB_PATH"
export PATH="$bindir:$PATH"
fi
ffmpeg -version
- name: System deps (Linux)
if: runner.os == 'Linux' && matrix.backend_supported
uses: awalsh128/cache-apt-pkgs-action@v1.6.3
if: runner.os == 'Linux'
uses: awalsh128/cache-apt-pkgs-action@latest
with:
packages: ffmpeg libsndfile1
version: 1.0
- name: Install Python deps (including PocketTTS)
# PocketTTS is an opt-in engine, but installing its pinned extra here
# proves that the same dependency set resolves on every supported local
# backend host. The Intel-Mac leg separately pins the documented
# unsupported contract: its UI is a remote-backend client only (#889).
if: matrix.backend_supported
run: bash scripts/uv-sync-retry.sh --extra pockettts
- name: Verify the documented Intel Mac contract
if: ${{ !matrix.backend_supported }}
shell: bash
run: |
python3 - <<'PY'
from pathlib import Path
import platform
import tomllib
assert platform.system() == "Darwin"
assert platform.machine() == "x86_64"
root = Path.cwd()
project = tomllib.loads((root / "pyproject.toml").read_text("utf-8"))
extra = project["project"]["optional-dependencies"]["pockettts"]
assert extra == [
"pocket-tts==2.1.0 ; sys_platform != 'darwin' or platform_machine != 'x86_64'"
]
docs = (root / "docs/install/macos.md").read_text("utf-8")
assert "Intel Macs are not supported" in docs
PY
- name: Install Python deps
run: uv sync
- name: Run smoke tests
# Exercise credential paths on native Windows as well as POSIX hosts.
if: matrix.backend_supported
run: uv run --no-sync pytest tests/smoke/ tests/test_hf_token_cache_paths.py -q --tb=short
run: uv run pytest tests/smoke/ -q --tb=short
env:
HF_HUB_OFFLINE: "1" # same no-silent-downloads guard as the main pytest job
HF_HUB_CACHE: ${{ runner.temp }}/pockettts-empty-hf-cache
# The isolated backend session, on Windows. The `test` job runs it on
# Linux only, which is how four tests that CANNOT pass on Windows shipped
# unnoticed: two reach for os.WNOHANG and os.waitid (POSIX-only, an
# AttributeError before the first assertion), one asserts a RuntimeError
# that `backend_drain_fd` returns None instead of raising off POSIX, and
# one raced the OS reaping a crashed child — a race Linux won and Windows
# lost every time. All four were invisible to CI and hit every Windows
# contributor on their first `pytest` run. Forty seconds closes the class.
- name: Isolated backend session (Windows)
if: runner.os == 'Windows' && matrix.backend_supported
run: uv run --no-sync pytest backend/tests/ -q --tb=short
env:
HF_HUB_OFFLINE: "1"
# Artifact commits depend on native Windows rename/replace semantics;
# Linux emulation cannot exercise sharing rules or path parsing.
# test_worker_task_store and test_worker_inbound_transport joined this
# step after a Windows run found a real portability bug the Linux-only
# `test` job could not see: a staged input's artifact id was built with
# os.path.join, so a Windows control plane persisted and shipped
# `inputs\<sha>.wav` — which a Linux worker cannot resolve. These suites
# need no ffmpeg, so they cost seconds here.
- name: Remote-worker artifact paths (Windows)
if: runner.os == 'Windows' && matrix.backend_supported
run: >-
uv run --no-sync pytest
tests/test_worker_upload_server.py
tests/test_worker_server_integrity.py
tests/test_worker_task_store.py
tests/test_worker_inbound_transport.py
-q --tb=short
env:
HF_HUB_OFFLINE: "1"
HF_HUB_CACHE: ${{ runner.temp }}/worker-artifact-empty-hf-cache
windows-wix-diagnostic:
name: Windows MSI authoring (no publishing)
needs: test
if: ${{ !cancelled() && (inputs.windows_wix_diagnostic || needs.test.result == 'success') }}
runs-on: windows-2022
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
- uses: oven-sh/setup-bun@v1
- name: Bundle canonical system and per-user templates with a tiny payload
shell: pwsh
run: ./scripts/diagnose-windows-wix.ps1
- name: Restore hosted Installer policy after failed standard-user installation
shell: powershell
run: ./scripts/test-msi-policy-cleanup.ps1
- name: Preserve verbose linker output and rendered authoring
if: always()
uses: actions/upload-artifact@v4
with:
name: windows-wix-diagnostic
path: wix-diagnostic-artifacts/
if-no-files-found: warn
retention-days: 3
+2 -11
View File
@@ -29,7 +29,7 @@
# On main pushes the Docker Hub repository overview is also synced from
# deploy/dockerhub-overview.md (source of truth for the hub.docker.com page).
#
# NOTE: the Docker image is the headless web-server build of VoiceStudio (FastAPI
# NOTE: the Docker image is the headless web-server build of OmniVoice (FastAPI
# backend + pre-built React frontend served over HTTP). The Tauri desktop
# auto-updater and its update-channel toggle are desktop-only features; they do
# NOT apply to the Docker image.
@@ -48,16 +48,7 @@ permissions:
env:
REGISTRY: ghcr.io
# PINNED, not ${{ github.repository }}. The repository was renamed to
# `VoiceStudio`, and deriving the image path from it would have silently
# moved published images to ghcr.io/debpalash/voicestudio — while Docker Hub
# (a hardcoded literal below) stayed put. Everyone pulling the documented
# GHCR path would have kept getting the last pre-rename image forever: no
# error, no warning, just a channel that quietly stopped updating. A
# published image path is a promise to users, not a mirror of the repo name.
# Renaming it is a deliberate migration (publish to both, document the move,
# then retire the old), not a side effect of renaming the repo.
IMAGE_NAME: debpalash/omnivoice-studio
IMAGE_NAME: ${{ github.repository }}
DOCKERHUB_IMAGE: palashdeb/omnivoice-studio
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
-115
View File
@@ -1,115 +0,0 @@
name: Electron packaging rehearsal
# Explicitly artifact-only: no tag, schedule, release, or publishing permission.
on:
workflow_dispatch:
permissions:
contents: read
concurrency:
group: electron-rehearsal-${{ github.ref }}
cancel-in-progress: true
jobs:
package:
runs-on: ${{ matrix.runner }}
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
include:
- runner: ubuntu-24.04
platform: linux
arch: x64
target: x86_64-unknown-linux-gnu
flags: --linux --x64
- runner: windows-2022
platform: win32
arch: x64
target: x86_64-pc-windows-msvc
flags: --win --x64
- runner: macos-15
platform: darwin
arch: arm64
target: aarch64-apple-darwin
flags: --mac --arm64
- runner: macos-15-intel
platform: darwin
arch: x64
target: x86_64-apple-darwin
flags: --mac --x64
defaults:
run:
shell: bash
env:
VOICESTUDIO_RUST_TARGET: ${{ matrix.target }}
VOICESTUDIO_UPDATE_CHANNEL: electron-preview-${{ matrix.platform }}-${{ matrix.arch }}
CSC_IDENTITY_AUTO_DISCOVERY: 'false'
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '22'
- uses: oven-sh/setup-bun@v2
with:
bun-version: '1.4.2'
- uses: dtolnay/rust-toolchain@stable
with:
targets: ${{ matrix.target }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: native/desktop-bridge -> target
key: electron-${{ matrix.target }}
- uses: astral-sh/setup-uv@v6
with:
version: '0.12.13'
enable-cache: false
- name: Linux native dependencies
if: runner.os == 'Linux'
run: |
sudo apt-get update
sudo apt-get install -y libasound2-dev libxdo-dev libxtst-dev libx11-dev libxkbcommon-dev libwayland-dev libssl-dev pkg-config xvfb
- name: Bundle pinned uv for the host architecture
run: |
node --input-type=module <<'NODE'
import { execFileSync } from 'node:child_process';
import { mkdirSync, copyFileSync, chmodSync } from 'node:fs';
import { join } from 'node:path';
const expected = process.env.VOICESTUDIO_RUST_TARGET;
const targets = { 'linux-x64': 'x86_64-unknown-linux-gnu', 'win32-x64': 'x86_64-pc-windows-msvc', 'darwin-arm64': 'aarch64-apple-darwin', 'darwin-x64': 'x86_64-apple-darwin' };
if (targets[`${process.platform}-${process.arch}`] !== expected) throw new Error('Runner architecture does not match package target');
const source = execFileSync(process.platform === 'win32' ? 'where.exe' : 'which', ['uv'], { encoding: 'utf8' }).trim().split(/\r?\n/)[0];
const dir = 'frontend/src-tauri/binaries';
mkdirSync(dir, { recursive: true });
const destination = join(dir, `uv-${expected}${process.platform === 'win32' ? '.exe' : ''}`);
copyFileSync(source, destination);
if (process.platform !== 'win32') chmodSync(destination, 0o755);
NODE
- name: Install locked dependencies
run: bun install --frozen-lockfile
- name: Validate and build Electron
run: bun run check:electron
- name: Package without publishing
working-directory: electron
run: |
bun x electron-builder --config electron-builder.config.mjs ${{ matrix.flags }} --publish never
node tests/packaging-contract.mjs --artifact
node tests/update-package-contract.mjs --platform ${{ matrix.platform }} --arch ${{ matrix.arch }}
- name: Packaged startup smoke test
working-directory: electron
run: |
if [ "$RUNNER_OS" = Linux ]; then
xvfb-run -a node tests/packaged-smoke.mjs --setup
else
node tests/packaged-smoke.mjs --setup
fi
- name: Save installers and updater metadata for review
uses: actions/upload-artifact@v4
with:
name: electron-rehearsal-${{ matrix.platform }}-${{ matrix.arch }}
retention-days: 14
if-no-files-found: error
path: |
electron/release/VoiceStudio-Electron-*
electron/release/electron-*.yml
-242
View File
@@ -1,242 +0,0 @@
name: Electron desktop release
# Builds are safe by default. Only an explicit publish dispatch exposes a release.
on:
push:
tags: ['v*']
workflow_dispatch:
inputs:
release_tag:
description: "Existing version tag to package using this workflow from main (optional)"
type: string
default: ''
publish:
description: "Publish the tagged Electron release after all platforms pass"
type: boolean
default: false
allow_unsigned:
description: "Explicitly accept unsigned/unnotarized Electron installers and documented updater limitations"
type: boolean
default: false
permissions:
contents: read
concurrency:
group: electron-release-${{ inputs.release_tag || github.ref_name }}
cancel-in-progress: false
env:
RELEASE_REF: ${{ inputs.release_tag && format('refs/tags/{0}', inputs.release_tag) || github.ref }}
jobs:
validate:
# The transition tag is assembled after the manual Tauri draft succeeds.
if: github.event_name == 'workflow_dispatch' || github.ref_name != vars.TAURI_SUNSET_TAG
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ env.RELEASE_REF }}
- name: Require an exact version tag
env:
REF: ${{ env.RELEASE_REF }}
WORKFLOW_REF: ${{ github.ref }}
RELEASE_TAG_OVERRIDE: ${{ inputs.release_tag }}
ALLOW_UNSIGNED: ${{ inputs.allow_unsigned }}
DISPATCH_ACTOR: ${{ github.actor }}
RERUN_ACTOR: ${{ github.triggering_actor }}
OWNER: ${{ github.repository_owner }}
run: |
if [ "$ALLOW_UNSIGNED" = true ]; then
test "$DISPATCH_ACTOR" = "$OWNER" && test "$RERUN_ACTOR" = "$OWNER" || {
echo "Only the repository owner may accept unsigned installers"; exit 1;
}
fi
if [ -n "$RELEASE_TAG_OVERRIDE" ]; then
test "$WORKFLOW_REF" = refs/heads/main || { echo "Tag overrides require the workflow from main"; exit 1; }
fi
VERSION=$(node -p "require('./frontend/package.json').version")
test "$REF" = "refs/tags/v$VERSION" || { echo "Select the exact version tag"; exit 1; }
test "$(git rev-parse HEAD)" = "$(git rev-parse "$REF^{commit}")" || { echo "Checkout does not match the release tag"; exit 1; }
package:
needs: validate
runs-on: ${{ matrix.runner }}
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
include:
- runner: ubuntu-24.04
platform: linux
arch: x64
target: x86_64-unknown-linux-gnu
flags: --linux --x64
- runner: windows-2022
platform: win32
arch: x64
target: x86_64-pc-windows-msvc
flags: --win --x64
- runner: macos-15
platform: darwin
arch: arm64
target: aarch64-apple-darwin
flags: --mac --arm64
- runner: macos-15-intel
platform: darwin
arch: x64
target: x86_64-apple-darwin
flags: --mac --x64
defaults:
run:
shell: bash
env:
VOICESTUDIO_RUST_TARGET: ${{ matrix.target }}
VOICESTUDIO_UPDATE_CHANNEL: electron-stable-${{ matrix.platform }}-${{ matrix.arch }}
CSC_IDENTITY_AUTO_DISCOVERY: 'false'
steps:
- uses: actions/checkout@v4
with:
ref: ${{ env.RELEASE_REF }}
- uses: actions/setup-node@v4
with:
node-version: '22'
- uses: oven-sh/setup-bun@v2
with:
bun-version: '1.4.2'
- uses: dtolnay/rust-toolchain@6bed0761d98439e5a578e2877258200ad565ba87 # stable
with:
targets: ${{ matrix.target }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: native/desktop-bridge -> target
key: electron-${{ matrix.target }}
- uses: astral-sh/setup-uv@v6
with:
version: '0.12.13'
enable-cache: false
- name: Linux native dependencies
if: runner.os == 'Linux'
run: |
sudo apt-get update
sudo apt-get install -y libasound2-dev libxdo-dev libxtst-dev libx11-dev libxkbcommon-dev libwayland-dev libssl-dev pkg-config xvfb
- name: Bundle pinned uv for the host architecture
run: |
node --input-type=module <<'NODE'
import { execFileSync } from 'node:child_process';
import { mkdirSync, copyFileSync, chmodSync } from 'node:fs';
import { join } from 'node:path';
const expected = process.env.VOICESTUDIO_RUST_TARGET;
const targets = { 'linux-x64': 'x86_64-unknown-linux-gnu', 'win32-x64': 'x86_64-pc-windows-msvc', 'darwin-arm64': 'aarch64-apple-darwin', 'darwin-x64': 'x86_64-apple-darwin' };
if (targets[`${process.platform}-${process.arch}`] !== expected) throw new Error('Runner architecture does not match package target');
const source = execFileSync(process.platform === 'win32' ? 'where.exe' : 'which', ['uv'], { encoding: 'utf8' }).trim().split(/\r?\n/)[0];
const dir = 'frontend/src-tauri/binaries';
mkdirSync(dir, { recursive: true });
const destination = join(dir, `uv-${expected}${process.platform === 'win32' ? '.exe' : ''}`);
copyFileSync(source, destination);
if (process.platform !== 'win32') chmodSync(destination, 0o755);
NODE
- name: Install locked dependencies
run: bun install --frozen-lockfile
- name: Validate and build Electron
run: bun run check:electron
- name: Package without publishing
env:
CSC_LINK: ${{ secrets.ELECTRON_CSC_LINK }}
CSC_KEY_PASSWORD: ${{ secrets.ELECTRON_CSC_KEY_PASSWORD }}
working-directory: electron
run: |
# An empty CSC_LINK is interpreted as the working directory by the
# signer. Omit absent credentials rather than passing empty strings.
if [ -z "${CSC_LINK:-}" ]; then
unset CSC_LINK CSC_KEY_PASSWORD
fi
bun x electron-builder --config electron-builder.config.mjs ${{ matrix.flags }} --publish never
node tests/packaging-contract.mjs --artifact
node tests/update-package-contract.mjs --platform ${{ matrix.platform }} --arch ${{ matrix.arch }}
- name: Verify macOS signing and notarization before publication
if: inputs.publish == true && inputs.allow_unsigned != true && matrix.platform == 'darwin'
run: |
APP=$(find electron/release -maxdepth 2 -name VoiceStudio.app -type d -print -quit)
test -n "$APP"
codesign --verify --deep --strict "$APP"
spctl --assess --type execute --verbose=2 "$APP"
- name: Verify Windows installer signature before publication
if: inputs.publish == true && inputs.allow_unsigned != true && matrix.platform == 'win32'
shell: pwsh
run: |
$installers = @(Get-ChildItem electron/release/VoiceStudio-Electron-*.exe)
if ($installers.Count -eq 0) { throw "No installer to verify" }
foreach ($installer in $installers) {
$signature = Get-AuthenticodeSignature $installer.FullName
if ($signature.Status -ne 'Valid') { throw "Installer signature is not trusted: $($installer.Name)" }
}
- name: Packaged startup smoke test
working-directory: electron
run: |
if [ "$RUNNER_OS" = Linux ]; then
xvfb-run -a node tests/packaged-smoke.mjs --setup
else
node tests/packaged-smoke.mjs --setup
fi
- name: Save installers and updater metadata for review
uses: actions/upload-artifact@v4
with:
name: electron-release-${{ matrix.platform }}-${{ matrix.arch }}
retention-days: 14
if-no-files-found: error
path: |
electron/release/VoiceStudio-Electron-*
electron/release/electron-*.yml
release:
needs: package
runs-on: ubuntu-latest
permissions:
contents: write
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAG: ${{ inputs.release_tag || github.ref_name }}
SUNSET_TAG: ${{ vars.TAURI_SUNSET_TAG }}
PUBLISH: ${{ inputs.publish }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ env.RELEASE_REF }}
- uses: actions/download-artifact@v4
with:
pattern: electron-release-*
merge-multiple: true
path: release-assets
- name: Validate all platforms before creating a release
run: |
python3 scripts/prepare_electron_release.py --assets release-assets --tag "$TAG"
- name: Preserve the final Tauri updater feeds
run: |
test -n "$SUNSET_TAG" || { echo "Set TAURI_SUNSET_TAG before releasing"; exit 1; }
# The transition tag already holds its own final Tauri feeds.
# Later releases carry copies pointing to the immutable sunset payloads.
gh release download "$SUNSET_TAG" --pattern latest.json --dir release-assets
gh release download "$SUNSET_TAG" --pattern latest-user.json --dir release-assets
python3 scripts/prepare_electron_release.py --assets release-assets --tag "$TAG" --sunset-tag "$SUNSET_TAG"
- name: Disclose explicitly accepted unsigned artifacts
if: inputs.allow_unsigned == true
run: |
cat >> release-assets/RELEASE_NOTES.md <<'EOF'
### Electron installer trust
These Electron installers are unsigned or ad-hoc signed and are not Apple-notarized.
Windows/macOS may show trust warnings. macOS automatic updates are unverified;
use manual installer updates. Tauri updater signatures remain independently verified.
EOF
- name: Create or update draft
run: |
if ! gh release view "$TAG" >/dev/null 2>&1; then
gh release create "$TAG" --verify-tag --draft --title "$TAG — VoiceStudio" --notes-file release-assets/RELEASE_NOTES.md
fi
test "$(gh release view "$TAG" --json isDraft --jq .isDraft)" = true || { echo "Refusing to replace a published release"; exit 1; }
gh release edit "$TAG" --notes-file release-assets/RELEASE_NOTES.md
find release-assets -maxdepth 1 -type f ! -name RELEASE_NOTES.md -print0 | xargs -0 gh release upload "$TAG" --clobber
- name: Publish only when explicitly requested
if: github.event_name == 'workflow_dispatch' && inputs.publish == true
run: gh release edit "$TAG" --draft=false --latest
+2 -2
View File
@@ -38,14 +38,14 @@ jobs:
cache-dependency-glob: "uv.lock"
- name: Install deps
run: bash scripts/uv-sync-retry.sh
run: uv sync
- name: Run eval suites (non-gating)
continue-on-error: true
env:
TRANSLATE_BASE_URL: ${{ secrets.EVALS_LLM_BASE_URL }}
TRANSLATE_API_KEY: ${{ secrets.EVALS_LLM_API_KEY }}
run: uv run --no-sync python tests/evals/run_evals.py --output eval-report.json
run: uv run python tests/evals/run_evals.py --output eval-report.json
- name: Upload report artifact
uses: actions/upload-artifact@v4
-127
View File
@@ -1,127 +0,0 @@
# Installer smoke — runs scripts/install.sh / scripts/install.ps1 end-to-end
# on all three desktop platforms so the one-liner installers can't rot.
#
# Gated by `paths` because a cold run downloads multi-GB wheels (torch) and
# takes ~15-30 min per OS; it only needs to fire when an installer or this
# workflow changes. The heavy Tauri bundles stay in release.yml (tag push).
name: Install smoke
on:
pull_request:
paths:
- "scripts/install.sh"
- "scripts/install.ps1"
- ".github/workflows/install-smoke.yml"
push:
branches: [main]
paths:
- "scripts/install.sh"
- "scripts/install.ps1"
- ".github/workflows/install-smoke.yml"
workflow_dispatch:
permissions:
contents: read
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
jobs:
install:
name: Install (${{ matrix.os }})
runs-on: ${{ matrix.os }}
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
os: [ubuntu-22.04, macos-latest, windows-latest]
steps:
- uses: actions/checkout@v4
# Running `sh scripts/install.sh` from the repo root exercises the
# repo-root resolution (script dir is scripts/, project root one level
# up) — the exact bug that made a local run clone a duplicate repo.
# Binary mode is the default: prebuilt release asset, checksum verified.
- name: Run installer — binary (macOS/Linux)
if: runner.os != 'Windows'
run: sh scripts/install.sh
- name: Verify install — binary (macOS/Linux)
if: runner.os != 'Windows'
run: |
if [ "$(uname)" = "Darwin" ]; then
test -d "/Applications/VoiceStudio.app" || { echo "::error::VoiceStudio.app missing from /Applications"; exit 1; }
echo "✓ VoiceStudio.app installed in /Applications"
else
test -x "$HOME/.local/bin/VoiceStudio" || { echo "::error::AppImage missing from ~/.local/bin"; exit 1; }
"$HOME/.local/bin/VoiceStudio" --appimage-help >/dev/null 2>&1 || true
echo "✓ AppImage installed and executable"
fi
# Source mode stays covered end-to-end behind --source.
- name: Run installer — source (macOS/Linux)
if: runner.os != 'Windows'
run: sh scripts/install.sh --source
- name: Verify install — source (macOS/Linux)
if: runner.os != 'Windows'
working-directory: ${{ github.workspace }}
run: |
test -d .venv || { echo "::error::.venv missing"; exit 1; }
test -f frontend/dist/index.html || { echo "::error::frontend build missing"; exit 1; }
echo "✓ venv + frontend bundle present"
# Binary mode is the default; CI runs msiexec silently.
- name: Run installer — binary (Windows)
if: runner.os == 'Windows'
env:
CI: true
shell: pwsh
run: '& { $ErrorActionPreference = "Stop"; & "${{ github.workspace }}\scripts\install.ps1" }'
- name: Verify install — binary (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: |
$paths = @(
"HKLM:\Software\Microsoft\Windows\CurrentVersion\Uninstall\*",
"HKLM:\Software\WOW6432Node\Microsoft\Windows\CurrentVersion\Uninstall\*",
"HKCU:\Software\Microsoft\Windows\CurrentVersion\Uninstall\*"
)
$key = Get-ItemProperty $paths -ErrorAction SilentlyContinue |
Where-Object { $_.DisplayName -match "VoiceStudio|OmniVoice" } |
Select-Object -First 1
if (-not $key) {
Get-ItemProperty $paths -ErrorAction SilentlyContinue |
Where-Object DisplayName | ForEach-Object { Write-Host " installed: $($_.DisplayName)" }
Write-Host "::error::MSI product not registered"; exit 1
}
Write-Host "✓ MSI product registered: $($key.DisplayName)"
# Source mode stays covered end-to-end behind -Source.
- name: Run installer — source (Windows)
if: runner.os == 'Windows'
env:
VOICESTUDIO_INSTALL_MODE: source
shell: pwsh
run: '& { $ErrorActionPreference = "Stop"; & "${{ github.workspace }}\scripts\install.ps1" }'
- name: Verify install — source (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: |
if (-not (Test-Path .venv)) { Write-Host "::error::.venv missing"; exit 1 }
if (-not (Test-Path frontend\dist\index.html)) { Write-Host "::error::frontend build missing"; exit 1 }
Write-Host "✓ venv + frontend bundle present"
- name: Upload install log on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: install-log-${{ matrix.os }}
path: |
/Users/runner/Library/Application Support/OmniVoice/*.log
/home/runner/.local/share/VoiceStudio/*.log
${{ runner.temp }}/VoiceStudio/**/*.log
if-no-files-found: ignore
+62 -699
View File
@@ -27,38 +27,30 @@
# each to surface PyInstaller/Tauri issues that never showed up locally on
# macOS — iterate on CI.
name: Tauri sunset (manual only)
name: Desktop Release
# Legacy workflow: run once on the final Tauri version tag.
# Electron releases are owned by electron-release.yml.
on:
push:
tags: ['v*']
schedule:
# 07:00 UTC daily — rolling `preview` prerelease from `main`. The
# preview-gate job no-ops the matrix when main hasn't moved in a day.
- cron: '0 7 * * *'
workflow_dispatch:
inputs:
draft:
description: "Keep the final Tauri release draft until Electron artifacts are ready"
description: "Create as draft release (tag push only)"
required: false
default: "true"
publish_preview:
description: "Legacy compatibility input; previews are retired"
description: "Publish a rolling 'preview' prerelease (updater Preview channel). Previews ALWAYS build from main — dispatching from any other branch fails the preview-gate."
required: false
type: boolean
default: false
permissions:
contents: write # needed to attach artifacts + updater manifest to GH Release
# Every preview build publishes to the SAME rolling `preview` release, and the
# updater manifest is rebuilt from whatever assets are on it. Two overlapping
# preview runs (the nightly schedule and a manual dispatch, say) would upload
# into each other's asset set, and the version-less macOS tarballs carry
# nothing saying which run produced them — so one run could publish a manifest
# advertising its own version while serving the other run's macOS binaries
# (greptile). Serialize instead. Keyed on the ref, so a `v*` tag push (which
# builds its own release and never touches `preview`) is never queued behind a
# nightly.
concurrency:
group: desktop-release-${{ github.ref }}
cancel-in-progress: false
env:
# Run all JavaScript actions on Node 24 (GH deprecates Node 20 in Sep 2026).
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
@@ -71,16 +63,6 @@ jobs:
name: Tests (backend + frontend)
runs-on: ubuntu-22.04
steps:
- name: Require the designated final Tauri tag
env:
SUNSET_TAG: ${{ vars.TAURI_SUNSET_TAG }}
REF: ${{ github.ref }}
PREVIEW: ${{ inputs.publish_preview }}
run: |
test -n "$SUNSET_TAG" || { echo "Set TAURI_SUNSET_TAG to the final v* tag first"; exit 1; }
test "$REF" = "refs/tags/$SUNSET_TAG"
test "$PREVIEW" != "true"
- uses: actions/checkout@v4
- name: Setup Python 3.11
@@ -108,16 +90,16 @@ jobs:
# Backend tests need ffmpeg (subprocess calls in fixtures). Cache the
# resolved .debs so warm runs skip the apt-get update + install.
- name: System deps (ffmpeg)
uses: awalsh128/cache-apt-pkgs-action@v1.6.3
uses: awalsh128/cache-apt-pkgs-action@latest
with:
packages: ffmpeg
version: 1.0
- name: Install Python deps
run: bash scripts/uv-sync-retry.sh
run: uv sync
- name: Run pytest
run: uv run --no-sync pytest tests/ -q --tb=short
run: uv run pytest tests/ -q --tb=short
- name: Cache bun deps
uses: actions/cache@v4
@@ -147,29 +129,21 @@ jobs:
working-directory: frontend
run: node --experimental-strip-types --no-warnings --test ../tests/frontend/*.test.mjs
- name: Electron typecheck, tests and production contract
run: bun run check:electron
# Decide preview-vs-stable, and for nightly runs whether `main` actually
# moved in the last day. Outputs gate the expensive matrix (`build`) and the
# `preview-notes` job, so a no-commit night costs only this ~30s job.
preview-gate:
name: Preview gate
runs-on: ubuntu-22.04
permissions:
contents: read
outputs:
is_preview: ${{ steps.decide.outputs.is_preview }}
proceed: ${{ steps.decide.outputs.proceed }}
stable_tag: ${{ steps.decide.outputs.stable_tag }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 50
- id: decide
shell: bash
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
event="${{ github.event_name }}"
@@ -184,13 +158,6 @@ jobs:
exit 1
fi
echo "is_preview=true" >> "$GITHUB_OUTPUT"
# Resolve once before the matrix starts so every platform stamps
# against the same immutable Stable-channel snapshot.
STABLE_TAG=$(gh release view --repo "$GITHUB_REPOSITORY" --json tagName --jq .tagName)
[[ "$STABLE_TAG" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]] || {
echo "::error::latest stable release has an invalid tag"; exit 1;
}
echo "stable_tag=$STABLE_TAG" >> "$GITHUB_OUTPUT"
else
echo "is_preview=false" >> "$GITHUB_OUTPUT"
fi
@@ -319,7 +286,7 @@ jobs:
libwebkit2gtk-4.1-dev \
build-essential curl wget file libxdo-dev libssl-dev \
libayatana-appindicator3-dev librsvg2-dev \
libasound2-dev ffmpeg xvfb
libasound2-dev ffmpeg
# ── Frontend build ─────────────────────────────────────────────────
- name: Cache bun deps
@@ -352,7 +319,7 @@ jobs:
- name: Bundle uv (${{ matrix.rust_target }})
shell: bash
env:
UV_VERSION: "0.12.13"
UV_VERSION: "0.11.7"
TRIPLE: ${{ matrix.rust_target }}
run: |
set -euo pipefail
@@ -481,9 +448,6 @@ jobs:
fi
{
echo 'body<<RELEASE_BODY_EOF'
echo '## Final Tauri update'
echo 'VoiceStudio desktop is moving to Electron. This is the last Tauri release. Back up your data and install Electron separately: https://github.com/debpalash/VoiceStudio/blob/main/docs/electron-migration.md'
echo
echo "$BODY"
echo 'RELEASE_BODY_EOF'
} >> "$GITHUB_OUTPUT"
@@ -520,116 +484,37 @@ jobs:
echo "APPLE_TEAM_ID=$TID"
} >> "$GITHUB_ENV"
# Stamp each preview with a numeric prerelease that is strictly above the
# latest stable release. Main may intentionally retain the released
# version while AUTO_VERSION_BUMP is disabled; in that case the helper
# advances the preview base by one patch so stable users can still opt in
# and receive it. The edit is ephemeral and never committed.
# Stamp each preview build with a unique, monotonically increasing semver
# PRERELEASE so the updater actually offers it (a rolling preview that
# always reported the static 0.3.0 never looked "newer", so no update was
# ever delivered). Ephemeral, CI-only — never committed. Tauri reads the
# bundle + updater version from tauri.conf.json, so rewriting it here
# stamps the artifacts + latest.json. Under the versioning hard rule
# (owner-set 2026-06-11) main is always last-release + 1, so BASE-N is a
# prerelease of the NEXT version and semver-sorts ABOVE the last stable
# (0.3.6-N > 0.3.5) — preview users naturally upgrade past stable, and
# the Windows MSI ProductVersion (which strips the prerelease → 0.3.6)
# is also correctly above the last stable.
- name: Stamp preview version
if: needs.preview-gate.outputs.is_preview == 'true'
shell: bash
env:
STABLE_TAG: ${{ needs.preview-gate.outputs.stable_tag }}
run: |
set -euo pipefail
PREVIEW_VERSION=$(python scripts/stamp-preview-version.py \
--package-json frontend/package.json \
--stable-tag "$STABLE_TAG" \
--run-number "${{ github.run_number }}")
# package.json is the single source of truth; tauri.conf.json reads its
# version from it ("version": "../package.json"), so stamping
# package.json restamps the whole bundle.
CONF=frontend/package.json
BASE=$(jq -r .version "$CONF")
# MSI/WiX requires the semver pre-release identifier to be numeric-only
# (and <= 65535). "preview.N" hard-fails the Windows bundler, so the
# preview stamp is BASE-N — still sorts below the stable BASE for the
# updater, still unique per run.
PREVIEW_VERSION="${BASE}-${{ github.run_number }}"
tmp=$(mktemp)
jq --arg v "$PREVIEW_VERSION" '.version = $v' "$CONF" > "$tmp"
mv "$tmp" "$CONF"
echo "Stamped preview version: $PREVIEW_VERSION"
# The rolling `preview` release is REUSED every night, and macOS updater
# artifacts are the only ones Tauri names WITHOUT the version:
#
# VoiceStudio_0.4.1-103_x64.dmg <- unique per run, uploads fine
# VoiceStudio_x64.app.tar.gz <- constant, collides
#
# So every preview build after the first failed the macOS legs with
# `Validation Failed: {"resource":"ReleaseAsset","code":"already_exists"}`
# — and it failed AFTER the dmg upload, so the run went red while looking
# partially successful. The macOS updater bundles on `preview` went stale
# on 2026-07-04/05 and stayed that way for three weeks: Preview-channel
# macOS users had no working update path, and the nightly run was red
# every night.
#
# Delete this arch's updater bundle before uploading the new one. Scoped
# to the preview path (a `v*` tag makes a fresh release, nothing to
# collide with) and to this job's own arch, so the parallel aarch64/x64
# legs never touch each other's assets.
#
# ONLY an absent release/asset is benign. Auth, permission, rate-limit and
# network failures must not be swallowed: the step would report success
# while the stale asset survived, the upload would then die with
# `already_exists`, and we would be back to the exact outage this step
# exists to prevent — minus the red step that explains why. Since GH_TOKEN
# is scoped to this same repo, a 404 really does mean "not there".
- name: Clear this arch's stale preview updater bundle (macOS)
if: needs.preview-gate.outputs.is_preview == 'true' && runner.os == 'macOS'
shell: bash
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -uo pipefail
# aarch64-apple-darwin -> aarch64 ; x86_64-apple-darwin -> x64
case "${{ matrix.arch }}" in
aarch64-*) SUFFIX=aarch64 ;;
x86_64-*) SUFFIX=x64 ;;
*) echo "::error::unexpected arch ${{ matrix.arch }}"; exit 1 ;;
esac
# Match the STORED name, not the uploaded one. GitHub rewrites
# spaces to dots, which is why the pre-rename product ("OmniVoice
# Studio") was stored as "OmniVoice.Studio_x64.app.tar.gz".
# "VoiceStudio" has no space and so needs no translation — the
# pattern below matches both, so a preview release still holding
# pre-rename assets is still cleaned up.
if ! gh release view preview --json assets -q '.assets[].name' \
> /tmp/preview-assets.txt 2> /tmp/gh-view-err.txt; then
if grep -qiE 'not found|HTTP 404' /tmp/gh-view-err.txt; then
echo "No preview release yet — nothing to clear."
exit 0
fi
echo "::error::Could not read the preview release, so a stale ${SUFFIX} bundle may still be there."
echo "Refusing to continue blind — the Tauri upload would fail with already_exists."
cat /tmp/gh-view-err.txt
exit 1
fi
grep -E "(^VoiceStudio|[ .]Studio)_${SUFFIX}\.app\.tar\.gz(\.sig)?$" /tmp/preview-assets.txt \
> /tmp/stale.txt || true
if [ ! -s /tmp/stale.txt ]; then
echo "No stale ${SUFFIX} updater bundle on preview — nothing to clear."
exit 0
fi
while IFS= read -r name; do
echo "Removing stale preview asset: $name"
if ! gh release delete-asset preview "$name" --yes \
2> /tmp/gh-del-err.txt; then
# Already gone is fine — a re-run or the sibling leg beat us to
# it, and the goal (no asset under this name) is met either way.
if grep -qiE 'not found|HTTP 404' /tmp/gh-del-err.txt; then
echo " (already gone — nothing to collide with)"
continue
fi
echo "::error::Failed to delete stale preview asset $name."
cat /tmp/gh-del-err.txt
exit 1
fi
done < /tmp/stale.txt
# A retried job reuses its version and can collide with installers it
# uploaded before a later step failed. Keep other versions/arches intact;
# macOS versionless updater archives are scoped by release tag and arch.
- name: Clear this target's installer assets on retry
if: github.run_attempt > 1
shell: bash
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
RELEASE_TAG: ${{ (needs.preview-gate.outputs.is_preview == 'true') && 'preview' || github.ref_name }}
RELEASE_TARGET: ${{ matrix.rust_target }}
run: |
VERSION=$(python -c 'import json; print(json.load(open("frontend/package.json"))["version"])')
python scripts/clear-release-rerun-assets.py \
--tag "$RELEASE_TAG" --version "$VERSION" --target "$RELEASE_TARGET"
- name: Build + release (Tauri)
uses: tauri-apps/tauri-action@v0
env:
@@ -667,132 +552,13 @@ jobs:
tagName: ${{ (needs.preview-gate.outputs.is_preview == 'true') && 'preview' || github.ref_name }}
# Version-first so the tag is readable in GitHub's truncated
# release-list sidebar (which clips the title mid-string).
releaseName: ${{ (needs.preview-gate.outputs.is_preview == 'true') && 'Preview — VoiceStudio' || format('{0} — VoiceStudio', github.ref_name) }}
releaseName: ${{ (needs.preview-gate.outputs.is_preview == 'true') && 'Preview — OmniVoice Studio' || format('{0} — OmniVoice Studio', github.ref_name) }}
releaseBody: ${{ steps.changelog.outputs.body }}
releaseDraft: ${{ (needs.preview-gate.outputs.is_preview == 'true') && 'false' || (inputs.draft || 'true') }}
prerelease: ${{ needs.preview-gate.outputs.is_preview == 'true' }}
updaterJsonPreferNsis: false
includeUpdaterJson: true
- name: Build + publish Electron desktop
if: false # Electron is released independently by electron-release.yml.
shell: bash
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
RELEASE_TAG: ${{ (needs.preview-gate.outputs.is_preview == 'true') && 'preview' || github.ref_name }}
IS_PREVIEW: ${{ needs.preview-gate.outputs.is_preview }}
VOICESTUDIO_RUST_TARGET: ${{ matrix.rust_target }}
run: |
set -euo pipefail
case "${{ matrix.rust_target }}" in
aarch64-apple-darwin) ELECTRON_OS=darwin; ELECTRON_ARCH=arm64; FLAGS="--mac --arm64" ;;
x86_64-apple-darwin) ELECTRON_OS=darwin; ELECTRON_ARCH=x64; FLAGS="--mac --x64" ;;
x86_64-pc-windows-msvc) ELECTRON_OS=win32; ELECTRON_ARCH=x64; FLAGS="--win --x64" ;;
x86_64-unknown-linux-gnu) ELECTRON_OS=linux; ELECTRON_ARCH=x64; FLAGS="--linux --x64" ;;
*) echo "Unsupported Electron target: ${{ matrix.rust_target }}"; exit 1 ;;
esac
if [ "$IS_PREVIEW" = "true" ]; then
export VOICESTUDIO_UPDATE_CHANNEL="electron-preview-${ELECTRON_OS}-${ELECTRON_ARCH}"
else
export VOICESTUDIO_UPDATE_CHANNEL="electron-stable-${ELECTRON_OS}-${ELECTRON_ARCH}"
fi
if [ -n "${APPLE_CERTIFICATE:-}" ]; then
export CSC_LINK="$APPLE_CERTIFICATE"
export CSC_KEY_PASSWORD="${APPLE_CERTIFICATE_PASSWORD:-}"
fi
bun install --frozen-lockfile
(
cd electron
bun run build
node tests/packaging-contract.mjs
bun x electron-builder \
--config electron-builder.config.mjs $FLAGS --publish never
node tests/packaging-contract.mjs --artifact
if [ "$RUNNER_OS" = "Linux" ]; then
xvfb-run -a node tests/packaged-smoke.mjs --setup
else
node tests/packaged-smoke.mjs --setup
fi
node tests/update-package-contract.mjs \
--channel "$VOICESTUDIO_UPDATE_CHANNEL" \
--platform "$ELECTRON_OS" \
--arch "$ELECTRON_ARCH"
)
# A rolling preview reuses one release. Remove only this platform /
# architecture's older Electron artifacts before publishing the new
# version; sibling matrix legs own different names and metadata.
if [ "$IS_PREVIEW" = "true" ]; then
gh release view "$RELEASE_TAG" --repo "$GITHUB_REPOSITORY" --json assets \
--jq '.assets[].name' > electron-assets.txt
case "${{ runner.os }}" in
Windows) OS_TOKEN=win ;;
macOS) OS_TOKEN=mac ;;
Linux) OS_TOKEN=linux ;;
esac
while IFS= read -r asset; do
case "$asset" in
VoiceStudio-Electron-*-${OS_TOKEN}-${ELECTRON_ARCH}.*|${VOICESTUDIO_UPDATE_CHANNEL}*.yml)
gh release delete-asset "$RELEASE_TAG" "$asset" --yes --repo "$GITHUB_REPOSITORY"
;;
esac
done < electron-assets.txt
fi
electron_artifact_count=0
while IFS= read -r artifact; do
gh release upload "$RELEASE_TAG" "$artifact" --clobber --repo "$GITHUB_REPOSITORY"
electron_artifact_count=$((electron_artifact_count + 1))
done < <(find electron/release -maxdepth 1 -type f \
\( -name 'VoiceStudio-Electron-*' -o -name "${VOICESTUDIO_UPDATE_CHANNEL}*.yml" \) | sort)
if [ "$electron_artifact_count" -eq 0 ]; then
echo "FAIL — Electron build produced no publishable artifacts"
find electron/release -maxdepth 1 -type f -print || true
exit 1
fi
- name: Build per-user Windows MSI
if: runner.os == 'Windows'
shell: bash
working-directory: frontend
env:
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
run: |
set -euo pipefail
python ../scripts/render-per-user-wix.py \
--source src-tauri/wix/main.wxs \
--system-wxs src-tauri/target/${{ matrix.rust_target }}/release/wix/x64/main.wxs \
--output src-tauri/target/wix-per-user/main.wxs
bunx tauri build --target ${{ matrix.rust_target }} --bundles msi \
--config src-tauri/tauri.per-user.conf.json
DIR="src-tauri/target/${{ matrix.rust_target }}/release/bundle/msi"
while IFS= read -r artifact; do
safe=${artifact// (Current User)/_Current_User}
[ "$safe" = "$artifact" ] || mv "$artifact" "$safe"
done < <(find "$DIR" -maxdepth 1 -type f -name '*Current*User*.msi*')
- name: Publish per-user Windows updater channel
if: runner.os == 'Windows'
shell: bash
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
RELEASE_TAG: ${{ (needs.preview-gate.outputs.is_preview == 'true') && 'preview' || github.ref_name }}
run: |
set -euo pipefail
DIR="frontend/src-tauri/target/${{ matrix.rust_target }}/release/bundle/msi"
MSI=$(find "$DIR" -name '*Current*User*.msi' -type f | head -1)
[ -n "$MSI" ] || { echo "per-user MSI missing"; find "$DIR" -type f; exit 1; }
[ -f "$MSI.sig" ] || { echo "per-user MSI signature missing"; exit 1; }
VERSION=$(jq -r .version frontend/package.json)
python scripts/build_windows_user_manifest.py \
--repo "$GITHUB_REPOSITORY" --tag "$RELEASE_TAG" --version "$VERSION" \
--asset "$(basename "$MSI")" --signature-file "$MSI.sig" \
--output latest-user.json
gh release upload "$RELEASE_TAG" "$MSI" "$MSI.sig" latest-user.json \
--clobber --repo "$GITHUB_REPOSITORY"
# ── Installer smoke (Phase 0 GATE-03) ─────────────────────────────
# Structural verification of the installed/extracted bundle. The thin
# uv-venv installer ships NO frozen backend binary (the venv is built on
@@ -809,10 +575,8 @@ jobs:
set -euo pipefail
DMG=$(find frontend/src-tauri/target/${{ matrix.rust_target }}/release/bundle/dmg -name "*.dmg" | head -1)
echo "Smoke-testing DMG: $DMG"
# Grab the full mount path with a grep rather than `awk '{print $3}'`.
# The volume name used to contain a space ("OmniVoice Studio"), which
# awk truncated to /Volumes/OmniVoice; "VoiceStudio" has no space, so
# this is now belt-and-braces rather than load-bearing.
# Grab the full mount path — the volume name has a space ("OmniVoice
# Studio"), so `awk '{print $3}'` would truncate it to /Volumes/OmniVoice.
MOUNT=$(hdiutil attach -nobrowse -readonly "$DMG" | tail -1 | grep -oE '/Volumes/.*$')
APP=$(find "$MOUNT" -maxdepth 2 -name "*.app" | head -1)
fail() { echo "FAIL — $1"; find "$APP/Contents" -maxdepth 4 -type f 2>/dev/null | head -40; hdiutil detach "$MOUNT" || true; exit 1; }
@@ -861,12 +625,11 @@ jobs:
shell: bash
run: |
set -euo pipefail
MSI=$(find frontend/src-tauri/target/${{ matrix.rust_target }}/release/bundle/msi -name "*.msi" ! -name '*Current*User*' | head -1)
MSI=$(find frontend/src-tauri/target/${{ matrix.rust_target }}/release/bundle/msi -name "*.msi" | head -1)
echo "Smoke-testing MSI: $MSI"
powershell.exe -NoProfile -ExecutionPolicy Bypass -File scripts/verify-windows-msi.ps1 -MsiPath "$(cygpath -w "$MSI")"
# /quiet = no UI, /norestart = don't reboot the runner if a dep asks
msiexec.exe //i "$(cygpath -w "$MSI")" //quiet //norestart
INSTALL="/c/Program Files/VoiceStudio"
INSTALL="/c/Program Files/OmniVoice Studio"
fail() { echo "FAIL — $1. Contents:"; find "$INSTALL" -maxdepth 4 -type f 2>/dev/null | head -40; exit 1; }
# Thin uv-venv installer ships no frozen backend .exe — verify the
# install is complete: shell exe + bundled uv + backend source resources.
@@ -876,72 +639,6 @@ jobs:
find "$INSTALL" -type f -path '*backend*main.py' | grep -q . || fail "backend source main.py missing"
echo "OK — MSI installed shell + uv + backend resources"
- name: Per-user installer smoke (Windows, non-admin account)
if: runner.os == 'Windows'
timeout-minutes: 8
shell: bash
run: |
set -euo pipefail
MSI=$(find frontend/src-tauri/target/${{ matrix.rust_target }}/release/bundle/msi -name '*Current*User*.msi' | head -1)
powershell.exe -NoProfile -ExecutionPolicy Bypass \
-File scripts/smoke-per-user-msi.ps1 -MsiPath "$(cygpath -w "$MSI")" -PrepareHostedRunner
# linuxdeploy re-links .DirIcon as an ABSOLUTE symlink into the build
# machine AFTER tauri's files-map has placed the real icon bytes — the
# exact bug #1518 guarded against, resurfacing on the first real tag
# build (v0.5.0). The seam tauri-action leaves us is post-upload: repack
# the AppImage with the icon as a REGULAR FILE, re-sign it (the updater
# signature covered the old bytes), and clobber the draft release's
# asset + the linux signature inside latest.json. The smoke below then
# validates the repaired artifact, not the broken one.
- name: Repair AppImage .DirIcon, re-sign, re-upload
if: runner.os == 'Linux'
timeout-minutes: 10
shell: bash
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
# Data, not shell source (zizmor template-injection): a crafted ref
# must never expand inside a script that holds the signing key.
TAG: ${{ (needs.preview-gate.outputs.is_preview == 'true') && 'preview' || github.ref_name }}
run: |
set -euo pipefail
APPIMAGE=$(find frontend/src-tauri/target/${{ matrix.rust_target }}/release/bundle/appimage -name "*.AppImage" | head -1)
APPIMAGE=$(realpath "$APPIMAGE")
WORK="$(mktemp -d)"; cd "$WORK"
"$APPIMAGE" --appimage-extract >/dev/null
ROOT="$WORK/squashfs-root"
ICON=$(readlink -f "$ROOT/.DirIcon" 2>/dev/null || true)
if [ -n "$ICON" ] && [ -f "$ICON" ] && case "$ICON" in "$ROOT"/*) true;; *) false;; esac; then
echo ".DirIcon already resolves inside the bundle — no repair needed"
exit 0
fi
# The real bytes are at the AppDir root (linuxdeploy put them there
# before mislinking). Ship a regular file: nothing left to dangle.
SRC=$(find "$ROOT" -maxdepth 1 -name "*.png" | head -1)
[ -n "$SRC" ] || SRC=$(find "$ROOT/usr/share/icons" -name "*.png" | head -1)
[ -n "$SRC" ] || { echo "no icon bytes found in bundle"; exit 1; }
rm -f "$ROOT/.DirIcon"
cp "$SRC" "$ROOT/.DirIcon"
# Pinned immutable release + checksum: this binary runs with the
# updater signing key and a release-write token in its environment,
# so a mutable 'continuous' asset is not acceptable supply chain.
AIT_URL="https://github.com/AppImage/appimagetool/releases/download/1.9.1/appimagetool-x86_64.AppImage"
AIT_SHA256="ed4ce84f0d9caff66f50bcca6ff6f35aae54ce8135408b3fa33abfc3cb384eb0"
curl -fsSL --retry 3 -o "$WORK/appimagetool" "$AIT_URL"
echo "$AIT_SHA256 $WORK/appimagetool" | sha256sum -c - || { echo "appimagetool checksum mismatch"; exit 1; }
chmod +x "$WORK/appimagetool"
# Same FUSE-less trick the build itself uses.
APPIMAGE_EXTRACT_AND_RUN=1 ARCH=x86_64 "$WORK/appimagetool" --no-appstream "$ROOT" "$APPIMAGE"
cd "$GITHUB_WORKSPACE/frontend"
bunx tauri signer sign "$APPIMAGE"
gh release upload "$TAG" "$APPIMAGE" "$APPIMAGE.sig" --clobber --repo "$GITHUB_REPOSITORY"
# latest.json is NOT patched here: every tauri-action leg re-uploads
# the shared manifest, so an in-leg patch races the other platforms —
# the repair-updater-manifest job below is the single final writer.
echo "repacked, re-signed, re-uploaded"
- name: Installer smoke (Linux)
if: runner.os == 'Linux'
timeout-minutes: 5
@@ -962,16 +659,9 @@ jobs:
"$APPIMAGE" --appimage-extract >/dev/null
ROOT="$EXTRACT_DIR/squashfs-root"
fail() { echo "FAIL — $1"; find "$ROOT" -maxdepth 5 -type f 2>/dev/null | head -40; exit 1; }
# linuxdeploy's GTK/GStreamer hooks wrap the seeded launcher as
# AppRun.wrapped. Verify the complete launcher chain, not only the
# small hook runner installed at the AppImage root.
bash "$GITHUB_WORKSPACE/scripts/verify-apprun-bundle.sh" \
"$ROOT" \
"$GITHUB_WORKSPACE/frontend/src-tauri/appimage/AppRun" \
"$GITHUB_WORKSPACE/frontend/src-tauri/target/.tauri/bundled-webkitgtk-version"
# Thin uv-venv installer: verify the AppImage carries the shell binary,
# the bundled uv sidecar, and the backend source resources.
{ [ -f "$ROOT/AppRun" ] || find "$ROOT" -type f \( -name "VoiceStudio" -o -name "omnivoice-studio" \) | grep -q .; } || fail "shell binary / AppRun missing"
{ [ -f "$ROOT/AppRun" ] || find "$ROOT" -type f \( -name "OmniVoice Studio" -o -name "omnivoice-studio" \) | grep -q .; } || fail "shell binary / AppRun missing"
find "$ROOT" -type f -name 'uv' | grep -q . || fail "bundled uv sidecar missing"
find "$ROOT" -type f -name 'pyproject.toml' | grep -q . || fail "backend resource pyproject.toml missing"
find "$ROOT" -type f -path '*/backend/main.py' | grep -q . || fail "backend source backend/main.py missing"
@@ -979,10 +669,10 @@ jobs:
# ── Compute SHA-256 checksums (Phase 0 GATE-05) ───────────────────
# Native OS tools: shasum -a 256 (POSIX) / Get-FileHash (Windows).
# Writes SHA256SUMS-<label>.txt, attached to the release below. The
# release-notes-checksums job puts every leg's file into the notes.
# Writes SHA256SUMS-<label>.txt for the user-verifiable path AND
# captures the content into $GITHUB_OUTPUT for body append.
- name: Compute SHA-256 checksums
if: startsWith(github.ref, 'refs/tags/v') && (github.event_name == 'push' || github.event_name == 'workflow_dispatch')
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
id: checksums
shell: bash
run: |
@@ -1003,10 +693,6 @@ jobs:
-o -name "*.msi" -o -name "*.msi.sig" \
-o -name "*.AppImage" -o -name "*.AppImage.sig" \
-o -name "*.deb" \) 2>/dev/null | sort)
while IFS= read -r artifact; do
ARTIFACTS+=("$artifact")
done < <(find electron/release -maxdepth 1 -type f \
\( -name 'VoiceStudio-Electron-*' -o -name 'electron-*.yml' \) 2>/dev/null | sort)
if [ ${#ARTIFACTS[@]} -eq 0 ]; then
echo "FAIL — no artifacts found under $BUNDLE_DIR"
@@ -1037,77 +723,15 @@ jobs:
echo "checksums_file=$OUT" >> "$GITHUB_OUTPUT"
# Attach only. The notes are one shared text and the publish is one
# decision, so both belong to the single release-notes-checksums job
# that runs after the whole matrix (see there for why).
- name: Attach SHA256SUMS file
if: startsWith(github.ref, 'refs/tags/v') && (github.event_name == 'push' || github.event_name == 'workflow_dispatch')
shell: bash
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAG: ${{ github.ref_name }}
FILE: ${{ steps.checksums.outputs.checksums_file }}
run: |
set -euo pipefail
gh release upload "$TAG" "$FILE" --clobber --repo "$GITHUB_REPOSITORY"
# ── Checksums into the notes, then publish (the single writer) ───────────
# Every build leg used to append its checksums to the shared release notes
# with softprops/action-gh-release. Two things went wrong:
# - The appends were concurrent read-modify-writes, so a leg that read the
# notes before another wrote them lost its section. v0.5.1 and v0.5.2
# both shipped without the macOS Apple Silicon checksums in the notes.
# - softprops defaults to draft: false, so the FIRST leg to finish
# published tauri-action's draft while the other installers and the
# complete latest.json were still being built (v0.5.2 went public at
# 17:27; its latest.json was finished at 17:38).
# This job is the only writer of the notes and the only publisher. It runs
# once every leg, the manifest repair and the uninstall scripts are done,
# writes the four platforms' checksums in a fixed order, and fails if one is
# missing, so a failed platform leaves the release a draft.
release-notes-checksums:
needs: [build, repair-updater-manifest, uninstall-scripts]
if: startsWith(github.ref, 'refs/tags/v') && (github.event_name == 'push' || github.event_name == 'workflow_dispatch')
runs-on: ubuntu-22.04
timeout-minutes: 10
permissions:
contents: write
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ github.repository }}
TAG: ${{ github.ref_name }}
KEEP_DRAFT: ${{ inputs.draft }}
steps:
- name: Write every platform's checksums into the notes, then publish
shell: bash
run: |
set -euo pipefail
WORK="$(mktemp -d)"
gh release download "$TAG" --repo "$REPO" --pattern 'SHA256SUMS-*.txt' --dir "$WORK"
# The checksum sections and the Contributors strip are always the
# tail of the notes; drop them so a re-run rebuilds rather than
# stacks (contributors-strip re-appends its strip after this job).
gh release view "$TAG" --repo "$REPO" --json body --jq .body \
| awk '/^### .* artifacts$/ || /^## Contributors$/ {exit} {print}' > "$WORK/notes.md"
missing=0
for label in "macOS Apple Silicon" "macOS Intel" "Windows x64" "Linux x64"; do
file="$WORK/SHA256SUMS-${label// /.}.txt"
if [ -f "$file" ]; then
cat "$file" >> "$WORK/notes.md"
else
echo "::error::The release has no checksums for $label"
missing=1
fi
done
[ "$missing" = 0 ] || exit 1
gh release edit "$TAG" --repo "$REPO" --notes-file "$WORK/notes.md"
if [[ "$KEEP_DRAFT" == "true" ]]; then
echo "Final Tauri draft verified; Electron publication owns the transition."
elif [[ "$TAG" == *-* ]]; then
gh release edit "$TAG" --repo "$REPO" --draft=false --prerelease
else
gh release edit "$TAG" --repo "$REPO" --draft=false --latest
fi
- name: Append checksums to release + attach SHA256SUMS file
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
uses: softprops/action-gh-release@v2
with:
tag_name: ${{ github.ref_name }}
append_body: true
body_path: ${{ steps.checksums.outputs.checksums_file }}
files: ${{ steps.checksums.outputs.checksums_file }}
fail_on_unmatched_files: true
# ── Uninstall scripts as release assets (#1089) ───────────────────────────
# The in-app uninstaller (Settings → Storage → Remove all data) is the primary
@@ -1121,96 +745,9 @@ jobs:
# the tag (v0.3.20 shipped with only the Linux AppImage that way). `needs:
# [build]` guarantees the release already exists; `--clobber` makes a re-run
# idempotent. This can never create a second release.
# The Linux leg may repack + re-sign its AppImage (see the repair step in
# the build matrix); every tauri-action leg also re-uploads the SHARED
# latest.json, so patching the manifest inside any leg races the others.
# This job runs once after the whole matrix as the single final writer:
# it makes the manifest's linux signature agree with the .sig asset that
# actually shipped, and refuses to leave a mismatch behind.
# The matrix validates each Electron package before upload. This final read-only
# check validates the other half of the contract: GitHub must actually serve
# all four manifests and every payload they name. Without it a green release
# can leave the in-app updater with four 404 feeds.
electron-publish-contract:
needs: [build, preview-gate]
if: false # Electron release workflow owns this contract.
runs-on: ubuntu-22.04
permissions:
contents: read
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAG: ${{ (needs.preview-gate.outputs.is_preview == 'true') && 'preview' || github.ref_name }}
CHANNEL: ${{ (needs.preview-gate.outputs.is_preview == 'true') && 'preview' || 'stable' }}
STABLE_TAG: ${{ needs.preview-gate.outputs.stable_tag }}
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- name: Verify published Electron updater assets
shell: bash
run: |
set -euo pipefail
WORK="$(mktemp -d)"
mkdir -p "$WORK/manifests"
gh release view "$TAG" --repo "$GITHUB_REPOSITORY" \
--json tagName,isPrerelease,assets > "$WORK/release.json"
gh release download "$TAG" --repo "$GITHUB_REPOSITORY" \
--pattern "electron-${CHANNEL}-*.yml" --dir "$WORK/manifests"
if [ "$CHANNEL" = "preview" ]; then
VERSION=$(python3 scripts/stamp-preview-version.py \
--package-json frontend/package.json \
--stable-tag "$STABLE_TAG" \
--run-number "${{ github.run_number }}")
else
VERSION=$(python3 -c 'import json; print(json.load(open("frontend/package.json"))["version"])')
fi
python3 scripts/check_electron_release_assets.py \
--release-json "$WORK/release.json" \
--manifest-dir "$WORK/manifests" \
--channel "$CHANNEL" \
--version "$VERSION"
repair-updater-manifest:
needs: [build, preview-gate]
runs-on: ubuntu-latest
timeout-minutes: 10
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Data, not shell source — same zizmor rule as the leg step.
TAG: ${{ (needs.preview-gate.outputs.is_preview == 'true') && 'preview' || github.ref_name }}
steps:
- name: Align latest.json's linux signature with the shipped .sig asset
shell: bash
run: |
set -euo pipefail
WORK="$(mktemp -d)"
HAS_MANIFEST=$(gh release view "$TAG" --repo "$GITHUB_REPOSITORY" --json assets --jq '[.assets[].name]|contains(["latest.json"])')
if [ "$HAS_MANIFEST" != "true" ]; then
echo "no latest.json on the release — nothing to align"; exit 0
fi
gh release download "$TAG" --pattern latest.json --output "$WORK/latest.json" --repo "$GITHUB_REPOSITORY"
# Same fail-closed rule as the manifest: absence is checked against
# the asset LIST; an actual download failure must fail the job, or
# the manifest keeps a signature nobody shipped.
HAS_SIG=$(gh release view "$TAG" --repo "$GITHUB_REPOSITORY" --json assets --jq '[.assets[].name|select(endswith(".AppImage.sig"))]|length > 0')
if [ "$HAS_SIG" != "true" ]; then
echo "no AppImage .sig asset on the release — nothing to align"; exit 0
fi
gh release download "$TAG" --pattern "*.AppImage.sig" --dir "$WORK" --repo "$GITHUB_REPOSITORY"
SIG_FILE=$(find "$WORK" -name "*.AppImage.sig" | head -1)
[ -n "$SIG_FILE" ] || { echo "sig asset listed but download produced nothing"; exit 1; }
NEW_SIG=$(cat "$SIG_FILE")
CHANGED=$(python3 -c 'import json,sys; p,sig=sys.argv[1],sys.argv[2]; d=json.load(open(p)); n=sum(1 for k,v in d.get("platforms",{}).items() if k.startswith("linux") and v.get("signature")!=sig and not v.update({"signature":sig})); json.dump(d,open(p,"w"),indent=2); print(n)' "$WORK/latest.json" "$NEW_SIG")
if [ "$CHANGED" -ge 1 ]; then
gh release upload "$TAG" "$WORK/latest.json" --clobber --repo "$GITHUB_REPOSITORY"
echo "aligned $CHANGED linux signature(s) with the shipped .sig"
else
echo "manifest already agrees with the shipped .sig — no write"
fi
uninstall-scripts:
needs: [build]
if: startsWith(github.ref, 'refs/tags/v') && (github.event_name == 'push' || github.event_name == 'workflow_dispatch')
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
runs-on: ubuntu-22.04
permissions:
contents: write
@@ -1247,12 +784,11 @@ jobs:
# MUST append via `gh release edit` on the EXISTING release (never a second
# softprops publish — that races tauri-action's per-matrix draft and splits
# installers across two releases; see uninstall-scripts). `needs: [build]`
# guarantees the release exists and release-notes-checksums has written the
# notes, and this job
# guarantees the release + all checksum appends already landed, and this job
# is single (no matrix) so there is no write race. Idempotent: it strips any
# prior "## Contributors" block before re-appending, so re-runs don't stack.
contributors-strip:
needs: [build, release-notes-checksums]
needs: [build]
if: >-
github.event_name == 'push'
&& startsWith(github.ref, 'refs/tags/v')
@@ -1307,162 +843,7 @@ jobs:
runs-on: ubuntu-22.04
permissions:
contents: write
# The manifest rebuild reads this run's `created_at` from the Actions
# Runs API to tie the version-less macOS bundles to this build. Without
# this scope the call 403s and, under `set -e`, takes the whole publish
# down (greptile).
actions: read
steps:
# Needed by the manifest rebuild + signature check below: the updater
# pubkey lives in frontend/src-tauri/tauri.conf.json.
#
# persist-credentials: false — nothing in this job pushes to git, and the
# steps that follow shell out to `gh` and install from PyPI, so leaving a
# token in .git/config only widens the blast radius (CodeRabbit).
- uses: actions/checkout@v4
with:
persist-credentials: false
# ── Rebuild the preview updater manifest from what is ACTUALLY published ──
# Since ~2026-07-13 every matrix leg has logged "Signature not found for
# the updater JSON. Skipping upload..." — tauri-action uploads the bundles
# + .sig companions but never refreshes latest.json. Meanwhile the macOS
# updater bundles (version-less filenames) are deleted + replaced every
# night by "Clear this arch's stale preview updater bundle", so the
# manifest's darwin signatures stopped matching the published files:
# macOS Preview users hit "The signature verification failed" on every
# update attempt (latest.json frozen at 2026-07-13, tar.gz replaced
# nightly).
#
# Root fix: after the matrix completes, rebuild latest.json HERE — one
# job, no per-leg race — from the release's real assets and their .sig
# companions, then clobber-upload. The manifest can no longer drift from
# the files it describes, regardless of what tauri-action's own
# updater-JSON path does or skips.
- name: Rebuild + verify the preview updater manifest, then publish
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ github.repository }}
run: |
set -euo pipefail
# Floor-pinned, matching docs-drift.yml's `pyyaml>=6`: this step
# decides whether a signed release manifest is trustworthy, so it is
# the one dependency worth a bound. Only Ed25519 verify is used.
pip install --quiet "cryptography>=42"
# name + updatedAt: the timestamp is how the version-less darwin
# tarballs get tied to this run — see scripts/build_preview_manifest.py.
gh release view preview --repo "$REPO" --json assets \
-q '[.assets[] | {name, updatedAt}]' > /tmp/assets.json
# The anchor for "this upload belongs to this run" is the moment this
# run's FIRST JOB began executing. Two wrong answers were considered:
#
# * `run_started_at` RESETS on re-run — re-running just this job
# would judge the macOS bundles its own earlier attempt uploaded
# as stale, and refuse a healthy build.
# * the run's `created_at` is stamped when the run is QUEUED. With
# the concurrency group above, a run can sit queued while the
# previous one uploads — so the queued run's created_at predates
# the OTHER run's macOS bundles and would accept them as its own
# (coderabbit).
#
# The earliest job start is after the queue wait (concurrency holds
# the whole run, so no job of ours has started) and before any of our
# own uploads. Jobs that were not re-run keep their original
# timestamps, so taking the MINIMUM stays correct across partial
# re-runs too.
#
# Needs the job's `actions: read` scope. If it ever 403s anyway, do
# not take the whole publish down with `set -e`: warn loudly and let
# build_manifest fall back to its leg-to-leg comparison, which is
# merely stricter than it should be, never laxer.
if ! RUN_CREATED_AT=$(gh api --paginate \
"repos/$REPO/actions/runs/${{ github.run_id }}/jobs?filter=latest" \
--jq '[.jobs[].started_at] | map(select(. != null)) | min // empty' \
2> /tmp/gh-run-err.txt); then
echo "::warning::Could not read this run's job start times (needs actions: read) — falling back to the stricter sibling-timestamp check, which can refuse a healthy build."
cat /tmp/gh-run-err.txt
RUN_CREATED_AT=""
fi
echo "This run began executing at ${RUN_CREATED_AT:-<unknown>}"
export RUN_CREATED_AT
WORK=$(mktemp -d)
python3 - "$WORK" <<'PY'
import base64, hashlib, json, os, subprocess, sys
from urllib.parse import unquote
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
sys.path.insert(0, "scripts")
from build_preview_manifest import ManifestRefused, build_manifest, required_assets
work, repo = sys.argv[1], os.environ["REPO"]
assets = json.load(open("/tmp/assets.json"))
def fetch(pattern):
subprocess.run(["gh", "release", "download", "preview", "--repo", repo,
"-p", pattern, "-D", work], check=True)
signatures = {}
for name in required_assets(assets):
fetch(name + ".sig")
signatures[name] = open(os.path.join(work, name + ".sig")).read()
try:
manifest = build_manifest(
assets, repo, signatures=signatures,
run_started_at=os.environ.get("RUN_CREATED_AT") or None,
)
except ManifestRefused as e:
sys.exit(f"Refusing to publish a preview manifest: {e}")
print(f"Built preview latest.json: version={manifest['version']}")
# ── Verify BEFORE publishing ──────────────────────────────────────
# Order is the whole point (greptile). Uploading first and checking
# afterwards leaves a manifest that fails the check live and served:
# the job goes red, and every macOS Preview user is broken until
# someone notices. Verify the file we are about to publish.
conf = json.load(open("frontend/src-tauri/tauri.conf.json"))
pub_doc = base64.b64decode(conf["plugins"]["updater"]["pubkey"]).decode()
pub = base64.b64decode(pub_doc.strip().splitlines()[1])
assert pub[:2] == b"Ed", "unexpected pubkey algorithm"
pk = Ed25519PublicKey.from_public_bytes(pub[10:42])
digests, failures = {}, []
for plat, info in sorted(manifest["platforms"].items()):
name = unquote(info["url"].rsplit("/", 1)[-1])
path = os.path.join(work, name)
if name not in digests:
fetch(name)
h = hashlib.blake2b(digest_size=64)
with open(path, "rb") as f:
for chunk in iter(lambda: f.read(1 << 20), b""):
h.update(chunk)
digests[name] = h.digest()
lines = base64.b64decode(info["signature"]).decode().splitlines()
sig = base64.b64decode(lines[1])
tc = lines[2].split("trusted comment: ", 1)[1]
gsig = base64.b64decode(lines[3])
try:
pk.verify(sig[10:74], digests[name])
pk.verify(gsig, sig[10:74] + tc.encode())
print(f"OK {plat}: signature matches {name}")
except Exception:
failures.append(plat)
print(f"FAIL {plat}: signature does NOT match {name}")
if failures:
sys.exit("Refusing to publish: manifest is broken for "
+ ", ".join(failures)
+ ". The previously published manifest is left in place.")
json.dump(manifest, open(os.path.join(work, "latest.json"), "w"), indent=2)
print("All signatures verified — safe to publish.")
PY
gh release upload preview "$WORK/latest.json" --clobber --repo "$REPO"
echo "Uploaded verified latest.json to the preview release."
- name: Generate + apply GitHub release notes to the preview release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@@ -1496,7 +877,7 @@ jobs:
gh release edit preview --repo "$REPO" --prerelease --notes-file /tmp/preview-notes.md
echo "Applied auto-generated release notes + contributors to the preview release."
- name: Verify the published preview manifest (prerelease + parity + served bytes)
- name: Verify preview updater manifest (prerelease + platform parity)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ github.repository }}
@@ -1521,24 +902,6 @@ jobs:
assert not missing, f"preview manifest missing platforms vs stable: {sorted(missing)}"
print(f"preview manifest OK: {v} platforms={sorted(pk)}")
PY
# The signatures were verified BEFORE publishing (see the rebuild
# step). What is worth checking here is different: that the file
# actually being SERVED is the one that passed. A CDN or a partial
# upload can leave something else at that URL, and the whole class of
# bug this job addresses is "the manifest does not describe what is
# published".
python3 - <<'PY'
import hashlib, json, sys
served = open("/tmp/preview-latest.json", "rb").read()
m = json.loads(served)
plats = sorted(m.get("platforms", {}))
print(f"served manifest: version={m.get('version')} platforms={plats}")
print(f"served sha256={hashlib.sha256(served).hexdigest()}")
missing = [p for p in plats if not m["platforms"][p].get("signature")]
if missing:
sys.exit(f"served manifest has empty signatures for: {missing}")
PY
# ── Post-release version bump (OWNER-GATED as of 2026-07-01) ──────────────
# Previously auto-ran after every stable v* tag to keep main = release + 1.
+1 -1
View File
@@ -174,7 +174,7 @@ jobs:
- name: pip-audit (Python)
continue-on-error: true
run: |
bash scripts/uv-sync-retry.sh
uv sync
uv run --with pip-audit pip-audit
# Pin a floor: `bun audit` was added in bun 1.2.x, so guarantee it exists.
+1 -38
View File
@@ -22,8 +22,6 @@ node_modules
.turbo/
bun.lockb
frontend/src-tauri/target/
electron/.tmp-native-target/
native/**/target*/
# ─────────────────────────────────────────────────────────────────────────
# Secrets & env
@@ -47,17 +45,12 @@ memxt.db-wal
# Editor / tool caches
# ─────────────────────────────────────────────────────────────────────────
# Ignore ad-hoc Claude Code state, but allow project-bundled skills
# (CLAUDE.md invites `.claude/skills/<name>/SKILL.md`) and project-bundled
# review agents — the owner's review standards belong with the code they
# govern, not in one machine's local state.
# (CLAUDE.md invites `.claude/skills/<name>/SKILL.md`).
.claude/*
!.claude/skills/
!.claude/skills/**
!.claude/agents/
!.claude/agents/**
/.cache*
/.tmp/
/.tmp-*
# ─────────────────────────────────────────────────────────────────────────
# Research clones — upstream repos used as reference, not shipped
@@ -141,10 +134,6 @@ marketing.md
.specify/
.claude/skills/speckit-*/
.antigravitycli/
# `backlog` (the CLI task tracker) writes a config + one markdown file per task
# into the repo root. A contributor running it locally had those three files
# swept into a PR that was otherwise a single script (#1322 / #1306).
backlog/
# Locally-installed third-party skill packs (marketingskills, hallmark,
# mattpocock/skills, …) — ignore every skill dir by default; a skill that
@@ -157,29 +146,3 @@ playwright-report/
# probe — generated HTML reports
tests/probe/reports/
# Local architecture/planning scratch (goal docs, review briefs, council
# reports). Working notes for whoever is driving a change, not a repo artifact.
/remote/
# OmniVoice GGUF runtime build artifacts (scripts/build-omnivoice-tts.sh).
# Only 0-byte placeholders of omnivoice-tts-* are tracked; real binaries,
# the checksums manifest and the copied libggml shared libs ship via CI.
bin/libggml*
bin/checksums.sha256
bin/omnivoice-tts-linux-aarch64
# Dubbing-demo intermediates. The .mp4/.srt/manifest.json in this directory ARE
# committed (they ship with the app); the per-language source WAVs are just the
# inputs scripts/render_dub_demo_audio.py hands to scripts/build_dub_demo.sh.
backend/assets/samples/demo/dubbing/*.src.wav
# Stray sqlite session artifacts (`<db-path>.ses`). An in-memory DB yields the
# literal name `:memory:.ses`, and a path containing `:` cannot be checked out
# on Windows at all — committing one fails every Windows CI job at the git
# checkout step, before a single test runs. Guarded by
# tests/test_no_windows_hostile_paths.py.
*.ses
# Generated Windows MSI diagnostic logs and installer payloads
/wix-diagnostic-artifacts/
+4 -18
View File
@@ -1,8 +1,7 @@
# Gitleaks config — extends the default ruleset.
#
# Every entry below is an exact, anchored non-secret value. PostHog's
# publishable project token is public by design (owner decision 2026-07-20,
# #1193). Per PostHog's docs the `phc_` project
# The ONLY sanctioned allowlist entry is PostHog's publishable project token
# (owner decision 2026-07-20, #1193). Per PostHog's docs the `phc_` project
# token is a write-only client key with "no access to your private data" —
# it ships in every release binary and every official PostHog SDK snippet.
# It is NOT a credential. Personal keys (`phx_`) remain fully banned.
@@ -13,20 +12,7 @@
useDefault = true
[allowlist]
description = "Exact public/test literals misclassified as generic API keys"
description = "PostHog publishable write-only project token (public by design; #1193)"
regexes = [
# Public PostHog project token; personal `phx_` keys remain banned.
'''^phc_v5wMjnYMPMaEcRNLRKQsTYCzPaYWh7wcHPhXNkNajVf9$''',
# Reviewed immutable Hugging Face commit for the Higgs tokenizer.
'''^528e871c2a26c4f0f7773b9754e2e1acae20899d$''',
# Deliberately synthetic fixtures that exercise HF-token redaction/storage.
'''^hf_abcdefghijklmnopqrstuvwxyz01234567890abcd$''',
'''^hf_abcdefghijklmnopqrstuvwxyz0123456789ABCDEF$''',
'''^hf_QWERTYUIOPasdfghjklZXCVBNM0123456789xyzAB$''',
# NLLB generation length argument, not the value of a credential.
'''^max_length=400$''',
# Dubbing pane split-position localStorage key, not a credential.
'''^omnivoice\.dubSplit\.v1$''',
# cryptography's Ed25519 private-key type name, not key material.
'''^Ed25519PrivateKey$''',
'''phc_v5wMjnYMPMaEcRNLRKQsTYCzPaYWh7wcHPhXNkNajVf9''',
]
+2 -32
View File
@@ -1,9 +1,8 @@
# Agent Rules — VoiceStudio
# Agent Rules — OmniVoice Studio
Binding for every AI agent (Claude, Codex, Cursor, review bots, …). CLAUDE.md is the full constitution; this is the operating contract. When they conflict, CLAUDE.md wins.
## Token economy (owner directive, 2026-07-20; tightened 2026-07-28)
- **Default to the shortest response that fully answers.** Outlines and tables over prose; no preamble, no recap of what you just did, no re-explaining a fix the diff already shows. Applies to every response, not just status updates.
## Token economy (owner directive, 2026-07-20)
- Lead with the outcome. No narration, no restating diffs, no filler praise, no plans you're about to execute anyway.
- Status updates: one line. Final reports: only what changes the reader's next action.
- Don't re-derive what CI, linters, or review bots already computed — read their output first (`gh pr checks`, bot comments via `gh api .../pulls/N/comments`).
@@ -11,11 +10,6 @@ Binding for every AI agent (Claude, Codex, Cursor, review bots, …). CLAUDE.md
- Run targeted tests while iterating; full suites only before landing.
- Tests and CI simulate CI honestly: `HF_HUB_OFFLINE=1` + empty `HF_HUB_CACHE` — a populated dev cache masks real failures.
## Cross-platform parity: behaviour, not performance
- The parity rule covers user-visible BEHAVIOUR. Hardware acceleration varies by host by design (CUDA/MPS/DirectML, Triton availability, `torch.compile`); skipping an optimization where it physically cannot work is not a parity violation.
- Do not "fix" a parity finding by disabling a working optimization everywhere. That trades a real regression for a semantic one.
- A feature the user can see and use on one OS but not another IS a violation. Judge by what the user can do, not by how fast it runs.
## Merge protocol (hard rules)
1. Never merge without review. Harvest CodeRabbit + Greptile comments first; never merge with an unread Critical/P1.
2. Never accept a PR as-is: fix findings ON the PR branch pre-merge (maintainer commits fine; credit contributors in CHANGELOG). No merge-then-fix, no comment-and-walk-away.
@@ -29,30 +23,6 @@ Binding for every AI agent (Claude, Codex, Cursor, review bots, …). CLAUDE.md
- Local-first: no new required network calls; any HF download gated on installed-ness or explicit user action; all synthetic audio through the `mark_synthetic` chokepoint.
- Every user-facing string via i18n, present in ALL 21 `frontend/src/i18n/locales/*.json` with real translations.
- Docs-sync in the same PR. CHANGELOG Unreleased: quiet one-liners ending `(#N)` + `— thanks @user!` for community work, under a short `**Highlights**` list.
- Tagged release announcements lead with the biggest user-visible change; redesigns need real UI screenshots and migrations need installer links and steps. Verify all contributor credits from the tag comparison and included PRs; list authors and bug reporters separately (see `docs/RELEASING.md`).
- Versioning: `frontend/package.json` is the single source of truth; never bump without the owner asking.
- `frontend/package.json` dep changes require regenerating root `bun.lock` (Docker runs `--frozen-lockfile`).
- Issues: absorb or decline — never defer to a future version. Check the open-PR queue before implementing community-reported fixes.
## Shared select controls
- Use `frontend/src/components/SearchableSelect.jsx` for all new or redesigned select boxes. Reuse `VoiceSelector` for voice choices. Do not introduce native `<select>` controls.
- Provide a localized `ariaLabel`; use `menuPortal` inside scrolling or clipping containers. Preserve keyboard selection and disabled states.
## Agent skills
Project development skills are pinned in `skills-lock.json` and installed under
`.agents/skills/`: Vite and FastAPI.
Repository rules and tracker mappings override generic skill guidance.
### Issue tracker
GitHub Issues on `debpalash/VoiceStudio`, via the `gh` CLI. See `docs/agents/issue-tracker.md`.
### Triage labels
The five canonical roles, each label string equal to its name. See `docs/agents/triage-labels.md`.
### Domain docs
Single-context: `CONTEXT.md` + `docs/adr/` at the repo root. See `docs/agents/domain.md`.
+44 -788
View File
File diff suppressed because it is too large Load Diff
+6 -25
View File
@@ -1,9 +1,9 @@
<!-- GSD:project-start source:PROJECT.md -->
## Project
**VoiceStudio**
**OmniVoice Studio**
VoiceStudio is an open-source, fully-local ElevenLabs alternative — a desktop app for voice cloning, voice design, video dubbing, and real-time dictation across 646 languages. It runs entirely on the user's machine (CUDA/MPS/ROCm/CPU auto-detect), with no API keys, no accounts, and no cloud dependencies. It's an active beta with a growing user base who hit it with real workloads (50-video batches, multi-engine setups, edge-OS platforms) and report friction in GitHub Issues and Discord. The current version lives in `frontend/package.json` (the single source of truth — see Versioning); the latest stable tag is on the [Releases page](https://github.com/debpalash/VoiceStudio/releases/latest). With `AUTO_VERSION_BUMP` off (the current owner setting), `main` holds at the released version between releases.
OmniVoice Studio is an open-source, fully-local ElevenLabs alternative — a desktop app for voice cloning, voice design, video dubbing, and real-time dictation across 646 languages. It runs entirely on the user's machine (CUDA/MPS/ROCm/CPU auto-detect), with no API keys, no accounts, and no cloud dependencies. It's an active beta with a growing user base who hit it with real workloads (50-video batches, multi-engine setups, edge-OS platforms) and report friction in GitHub Issues and Discord. The current version lives in `frontend/package.json` (the single source of truth — see Versioning); the latest stable tag is on the [Releases page](https://github.com/debpalash/OmniVoice-Studio/releases/latest). With `AUTO_VERSION_BUMP` off (the current owner setting), `main` holds at the released version between releases.
**Core Value:** **A first-run that actually works.** A user who downloads the installer (or clones the repo) should reach a working voice-cloning or dubbing output without hitting a wall — and when something does go wrong, the error or docs should tell them exactly what to do.
@@ -13,7 +13,7 @@ Everything else (new engines, fancy features) is downstream of "the thing instal
- **Existing engine compatibility**: Users with already-installed engines (IndexTTS, CosyVoice, etc.) must not have to reinstall. Fixes touching engine code must be backward-compatible with on-disk model state.
- **Cross-platform parity**: Every fix must work on macOS (Apple Silicon + Intel), Windows (x64), and Linux (AppImage + deb). No platform-only regressions; the cross-platform bug bash (PR #51) is the baseline.
- **Default features must work on every platform (strict rule, 2026-05-20):** A feature that ships in default mode — out-of-the-box, no user customization, no opt-in toggle — must behave identically on macOS, Windows, and Linux. Platform-specific *implementation code* is allowed for OS APIs / shells / packaging, but the user-visible *default behavior* cannot diverge. Platform-only features (e.g., a macOS-only global shortcut, a Windows-only path picker) must go behind explicit user opt-in: Settings toggle, env var, or CLI flag. When a default doesn't work on a platform, that's a P0 bug — either fix it on the missing platform or move it behind opt-in. No third option. **This rule governs BEHAVIOUR, not PERFORMANCE** (clarified 2026-07-30, council): hardware acceleration is expected to vary by host — CUDA, MPS, DirectML, Triton availability and `torch.compile` are all host-dependent by design, and reading the rule to forbid that would forbid GPU support itself. An optimization that is skipped where it cannot work (missing Triton, an arch the wheel lacks, a path its toolchain cannot link) is NOT a parity violation; a *feature* the user can see and use on one OS but not another is.
- **Default features must work on every platform (strict rule, 2026-05-20):** A feature that ships in default mode — out-of-the-box, no user customization, no opt-in toggle — must behave identically on macOS, Windows, and Linux. Platform-specific *implementation code* is allowed for OS APIs / shells / packaging, but the user-visible *default behavior* cannot diverge. Platform-only features (e.g., a macOS-only global shortcut, a Windows-only path picker) must go behind explicit user opt-in: Settings toggle, env var, or CLI flag. When a default doesn't work on a platform, that's a P0 bug — either fix it on the missing platform or move it behind opt-in. No third option.
- **Backward-compatible project data**: Existing `omnivoice_data/` (user voices, projects, settings) must keep working without manual migration. Any DB schema change goes through alembic with a tested upgrade path.
- **Local-first guarantee preserved**: nothing leaves the machine without the user's **explicit yes**, and the app must remain fully functional with everything declined. Auto bug reporting is opt-in and submits only to GitHub Issues (prefilled-URL, from the user's own browser). Product analytics (owner-sanctioned 2026-07-16) is opt-in PostHog EU with a **first-run consent prompt** — two equal-weight Yes/No buttons, never default-on, skipping = off; consent-gated, allowlisted content-free metadata only (`backend/core/analytics.py`); every build — installer, Docker, and source alike (owner reversal 2026-07-20, #1193) — carries the in-repo publishable write-only token and shows the same consent ask, with env/baked token overriding it. No required cloud calls, accounts, or API keys.
- **Beta release cadence (no RC, no ceremony — strict rule, 2026-05-20):** the v0.3.x line has **no release candidates, no 48h soak, no formal release ceremony**. Every fix goes continuous-to-main; the owner tags a patch (`v0.3.Z`) from main whenever the current state is worth cutting. No `-rc` tags. No phased release. No `v0.4` deferrals while the v0.3.x line is open — every open issue and every open community PR gets absorbed into the v0.3.x line or explicitly declined. Users follow `main` for previews; users wanting stable stay on the latest tagged release. ROADMAP.md's Phase 6 "Release/Verify/Retro" entries are obsolete unless the user revives them.
@@ -44,9 +44,7 @@ For anything new: prefer what's already pinned in `pyproject.toml` / `frontend/p
**Docs-sync (hard rule, owner-set 2026-06-11):** any change that alters something these docs describe — README.md, `.github/CONTRIBUTING.md`, `.github/SECURITY.md`, `.github/SUPPORT.md`, LICENSE, or `docs/**` (install flows, Docker tag semantics, platform support, versioning/release behavior, review process, supported versions) — must update those docs **in the same PR** as the change. If a doc impact is discovered after merge, the docs fix is the immediate next commit, not backlog. Stale docs are treated as bugs.
**Release notes / changelog (hard rule, owner-set 2026-06-16):** every tagged release gets a **high-quality, user-facing `## [X.Y.Z] — DATE` section in `CHANGELOG.md`** before (or in the same hour as) the tag — never the "Auto-generated release for vX.Y.Z…" fallback. The desktop release workflow extracts that section verbatim as the GitHub Release body, so a missing/empty section ships a bare release. Quality bar (owner-restyled 2026-07-17, replaces the old bold-lead paragraphs): **quiet and scannable change entries** — after an optional tagged-release introduction (see the presentation rule below), a short `**Highlights**` bullet list (plain words, one line each), then `### Changed` / `### Added` / `### Docs` / `### Fixed` / `### License` / `### CI` subsections where each entry is a **single one-liner** with the `(#NNN)` issue/PR ref and contributor credit (`— thanks @user!`) where applicable. Change entries are written for users, grouped by theme, with no multi-line entry paragraphs or raw commit dumps. This applies to **preview builds too**: preview release notes summarize what's new on `main` since the last stable, in the same style. Workflow: as features merge, keep `## [Unreleased]` current; at release time rename it to the version + date. If a release was already cut with the fallback body, the next action is to backfill `CHANGELOG.md` **and** `gh release edit <tag>` the live body — not backlog.
**Release presentation and credits (owner-set 2026-09-17):** Tagged release announcements lead with the biggest user-visible change; redesigns need real UI screenshots and migrations need installer links and steps. Verify all contributor credits from the tag comparison and included PRs; list authors and bug reporters separately (see `docs/RELEASING.md`). Keep Highlights to 35 bullets; the release introduction can include prose, images, and a download table before the concise change entries.
**Release notes / changelog (hard rule, owner-set 2026-06-16):** every tagged release gets a **high-quality, user-facing `## [X.Y.Z] — DATE` section in `CHANGELOG.md`** before (or in the same hour as) the tag — never the "Auto-generated release for vX.Y.Z…" fallback. `release.yml` extracts that section verbatim as the GitHub Release body (the `Extract CHANGELOG section for tag` step), so a missing/empty section ships a bare release. Quality bar (owner-restyled 2026-07-17, replaces the old bold-lead paragraphs): **quiet and scannable** a short `**Highlights**` bullet list first (plain words, one line each), then `### Changed` / `### Added` / `### Docs` / `### Fixed` / `### License` / `### CI` subsections where each entry is a **single one-liner** with the `(#NNN)` issue/PR ref and contributor credit (`— thanks @user!`) where applicable. Written for users, grouped by theme, no multi-line paragraphs, **not** raw commit dumps. This applies to **preview builds too**: preview release notes summarize what's new on `main` since the last stable, in the same style. Workflow: as features merge, keep `## [Unreleased]` current; at release time rename it to the version + date. If a release was already cut with the fallback body, the next action is to backfill `CHANGELOG.md` **and** `gh release edit <tag>` the live body — not backlog.
**Localization (hard rule):** No hardcoded non-English (CJK) **user-facing text** anywhere in the codebase except the translation layer (`frontend/src/i18n/`). All UI strings go through i18n (`t('...')` keys in `locales/*.json`); native language names live in `i18n/index.ts` (`LANGUAGES`). Functional CJK is allowed and tracked via the allowlist in `tests/test_no_hardcoded_cjk.py` — text-processing regexes, model/engine vocabulary & identifiers (e.g. CosyVoice speaker IDs), localized error matching, demo/eval data, and test fixtures. CI fails on any hardcoded CJK outside the allowlist; to add legitimate functional CJK, extend `_ALLOWED_FILES` there with a justification.
@@ -68,10 +66,7 @@ Architecture not yet mapped. Follow existing patterns found in the codebase.
<!-- GSD:skills-start source:skills/ -->
## Project Skills
- `vite` — Vite configuration, assets, HMR, builds, and Vitest guidance.
- `fastapi-python` — FastAPI and Pydantic implementation patterns.
Canonical copies live under `.agents/skills/`; `skills-lock.json` pins their sources and hashes. Claude should follow these paths directly, avoiding cross-platform symlinks.
No project skills found. Add skills to any of: `.claude/skills/`, `.agents/skills/`, `.cursor/skills/`, `.github/skills/`, or `.codex/skills/` with a `SKILL.md` index file.
<!-- GSD:skills-end -->
<!-- GSD:workflow-start source:GSD defaults -->
@@ -81,7 +76,7 @@ Direct repo edits are authorized (owner decision, 2026-07-08). The GSD command g
**Harvest bot reviews before merging (rule, 2026-07-20):** CodeRabbit and Greptile auto-review every PR (tuned via `.coderabbit.yaml` / `greptile.json`, both fed CLAUDE.md as context). Before merging ANY PR — including your own — read their inline comments (`gh api repos/<owner>/<repo>/pulls/<N>/comments` filtered by bot login) and triage: fix real findings, ignore noise, never merge with an unread Critical/P1. They are the free first review pass; reserve deep agent-driven review for what they can't judge (architecture, cross-file semantics, product intent). Mechanical rules belong in deterministic CI tests, not in any AI reviewer.
**Token economy (owner directive, 2026-07-20; tightened 2026-07-28):** default to the shortest response that fully answers — outlines and tables over prose, no preamble, no recap of work just done, no re-explaining what the diff shows; applies to every response, not just status updates. Lead with the outcome; one-line statuses; no narration, filler, or diff-restating. Read what CI/linters/review bots already computed instead of re-deriving it. Mechanical rules belong in deterministic tests (changelog style, locale parity, version lockstep, CJK — all in `tests/`), never in agent effort. Targeted tests while iterating; full suites only before landing. `AGENTS.md` carries this contract for all agents — keep the two in sync.
**Token economy (owner directive, 2026-07-20):** lead with the outcome; one-line statuses; no narration, filler, or diff-restating. Read what CI/linters/review bots already computed instead of re-deriving it. Mechanical rules belong in deterministic tests (changelog style, locale parity, version lockstep, CJK — all in `tests/`), never in agent effort. Targeted tests while iterating; full suites only before landing. `AGENTS.md` carries this contract for all agents — keep the two in sync.
**Never accept a PR as-is (owner directive, 2026-07-20):** review findings — bot, agent, or human — get FIXED on the PR branch before merge (maintainer commits are fine and credit the contributor in the changelog); do not merge with known issues, do not merge-then-fix, do not leave findings as comments for someone else. Also merge current `main` into stale community branches before judging their CI, so the PR runs today's workflow gates (PR-green under an old workflow ≠ main-green).
<!-- GSD:workflow-end -->
@@ -94,17 +89,3 @@ Direct repo edits are authorized (owner decision, 2026-07-08). The GSD command g
> Profile not yet configured. Run `/gsd-profile-user` to generate your developer profile.
> This section is managed by `generate-claude-profile` -- do not edit manually.
<!-- GSD:profile-end -->
## Agent skills
### Issue tracker
GitHub Issues on `debpalash/VoiceStudio`, via the `gh` CLI. See `docs/agents/issue-tracker.md`.
### Triage labels
The five canonical roles, each label string equal to its name. See `docs/agents/triage-labels.md`.
### Domain docs
Single-context: `CONTEXT.md` + `docs/adr/` at the repo root. See `docs/agents/domain.md`.
+11 -17
View File
@@ -1,4 +1,4 @@
# VoiceStudio — License Notice
# OmniVoice Studio — License Notice
## Abbreviation
@@ -6,32 +6,32 @@ AGPL-3.0-only
## Notice
Copyright 2024-present Palash Debnath and VoiceStudio contributors.
Copyright 2024-present Palash Debnath and OmniVoice Studio contributors.
VoiceStudio is **free and open-source software, licensed under the GNU
OmniVoice Studio is **free and open-source software, licensed under the GNU
Affero General Public License, Version 3 (AGPL-3.0)**. You are free to use,
copy, modify, and redistribute it. That **includes commercial and internal
business use** of the application itself. Model weights, tokenizers, and other
third-party assets retain their own terms; this application license does not
grant or summarize rights under those separate terms.
copy, modify, and redistribute it — and that **includes commercial and internal
business use**: run the app, use its outputs commercially, sell the audio you
produce with it, provide professional/client services with it, and deploy it
within your organization.
Because this is the **Affero** GPL, one additional obligation applies: if you
modify VoiceStudio and make that modified version available to others over
modify OmniVoice Studio and make that modified version available to others over
a network, you must also offer those users the complete corresponding source
code of your modified version under these same AGPL-3.0 terms. See the full
text in [`LICENSE`](LICENSE).
A **commercial license is available** for organizations that want to embed
VoiceStudio in a closed-source or proprietary product or service without
OmniVoice Studio in a closed-source or proprietary product or service without
the AGPL-3.0 copyleft obligations. Pricing tiers are coming soon; for inquiries
contact `VoiceStudio@palash.dev`.
contact `OmniVoice@palash.dev`.
(This Notice is a plain-language summary; the binding terms are the full GNU
AGPL-3.0 text in [`LICENSE`](LICENSE).)
### Scope
These terms cover the VoiceStudio application — the Tauri desktop shell
These terms cover the OmniVoice Studio application — the Tauri desktop shell
(`frontend/src-tauri/`), the React frontend (`frontend/src/`), the FastAPI
backend (`backend/`), and supporting build / packaging scripts (`scripts/`,
`Dockerfile`, `docker-compose.yml`, `.github/`).
@@ -41,12 +41,6 @@ is **separately licensed under Apache License 2.0** by its upstream authors and
is not relicensed here. Apache License 2.0 is compatible with, and may be
combined under, the GNU AGPL-3.0. See `pyproject.toml`.
Downloaded model weights are not relicensed by VoiceStudio. The default
`k2-fsa/OmniVoice` model card identifies its code as Apache-2.0 and pretrained
weights as CC-BY-NC. Its `audio_tokenizer/LICENSE` contains separate Boson
Higgs Audio 2 and Meta Llama community terms. A commercial license for
VoiceStudio-owned code does not replace any of those terms.
Third-party dependencies retain their own licenses. See `Cargo.lock`,
`bun.lock`, and `uv.lock` for the resolved set.
+636 -58
View File
@@ -1,101 +1,679 @@
<div align="center">
<img src="docs/logo.png" alt="VoiceStudio" width="88" />
<h1>VoiceStudio</h1>
<img src="docs/logo.png" alt="OmniVoice Logo" width="120" />
<h1>OmniVoice Studio</h1>
<h3>The open-source ElevenLabs alternative.</h3>
<p>Real-time dictation, zero-shot voice cloning, and cinematic video dubbing — all on your desktop.<br/><b>No accounts. No API keys. No cloud.</b> Everything runs on your machine. Open-source, <b>646 languages.</b></p>
<p>
<a href="https://trendshift.io/repositories/28176?utm_source=repository-badge&amp;utm_medium=badge&amp;utm_campaign=badge-repository-28176" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/repositories/28176" alt="VoiceStudio ranking on Trendshift" width="220" height="48" /></a>
</p>
<p><strong>Open source voice cloning and workflow engine. Build local.</strong></p>
<p>
<a href="https://voicestudio.sh/?utm_source=github&utm_medium=readme&utm_campaign=project">Website</a> ·
<a href="https://github.com/debpalash/VoiceStudio/releases/latest">Download</a> ·
<a href="#get-started">Get started</a> ·
<a href="#documentation">Docs</a> ·
<a href="#quickstart">Quickstart</a> ·
<a href="#features">Features</a> ·
<a href="#why-ovs">vs Others</a> ·
<a href="#tts-engines">Engines</a> ·
<a href="#openai-api">API</a> ·
<a href="#sponsor--donate">Donate</a> ·
<a href="#contributing">Contributing</a> ·
<a href="https://discord.gg/bzQavDfVV9">Discord</a> ·
<a href="README_CN.md">简体中文</a>
<a href="README_CN.md"><strong>简体中文</strong></a>
</p>
<p>
<a href="https://github.com/debpalash/VoiceStudio/actions/workflows/ci.yml"><img src="https://img.shields.io/github/actions/workflow/status/debpalash/VoiceStudio/ci.yml?branch=main" alt="CI" /></a>
<a href="https://github.com/debpalash/VoiceStudio/releases/latest"><img src="https://img.shields.io/github/v/release/debpalash/VoiceStudio" alt="Latest release" /></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/license-AGPL--3.0-blue" alt="AGPL-3.0" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/stargazers"><img src="https://img.shields.io/github/stars/debpalash/OmniVoice-Studio?style=flat-square&color=f59e0b" alt="Stars" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases"><img src="https://img.shields.io/github/downloads/debpalash/OmniVoice-Studio/total?style=flat-square&color=8b5cf6&label=downloads" alt="Total downloads" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/github/v/release/debpalash/OmniVoice-Studio?style=flat-square&color=10b981" alt="Release" /></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/license-AGPL--3.0-blue?style=flat-square" alt="License" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/issues"><img src="https://img.shields.io/github/issues/debpalash/OmniVoice-Studio?style=flat-square&color=ef4444" alt="Issues" /></a>
<a href="https://discord.gg/bzQavDfVV9"><img src="https://img.shields.io/badge/Discord-Join_Community-5865F2?style=flat-square&logo=discord&logoColor=white" alt="Discord" /></a>
<a href="https://ko-fi.com/debpalash"><img src="https://img.shields.io/badge/Ko--fi-Support_Us-FF5E5B?style=flat-square&logo=ko-fi&logoColor=white" alt="Ko-fi" /></a>
<a href="https://paypal.me/palashCoder"><img src="https://img.shields.io/badge/PayPal-Donate-00457C?style=flat-square&logo=paypal&logoColor=white" alt="PayPal" /></a>
</p>
<p>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/⬇_Download-macOS_·_Windows_·_Linux-10b981?style=for-the-badge" alt="Download the latest release" /></a>
</p>
<p>
<a href="https://trendshift.io/repositories/28176?utm_source=trendshift-badge&utm_medium=badge&utm_campaign=badge-trendshift-28176" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/trendshift/repositories/28176/daily?language=Python" alt="debpalash%2FOmniVoice-Studio | Trendshift" width="250" height="55"/></a>
</p>
</div>
![A tour of the Electron app: voice cloning, voice design, dubbing, and model management](docs/media/electron/voicestudio.gif)
<br/>
## Your voice. Your workflow.
<div align="center">
<img src="docs/screenshot-launchpad.png" alt="OmniVoice Studio — Launchpad" width="100%"/>
</div>
| Create | Produce | Connect |
| :--- | :--- | :--- |
| Clone a voice or design your own | Dub videos with timed speech | Local API & MCP for agents |
| Dictate with a floating widget | Stories, audiobooks & batch jobs | Optional remote workers |
> **Your voice is the most personal data you have. So why rent it back from a cloud?** Every mainstream voice tool ships your audio to someone else's server and bills you monthly for the privilege. OmniVoice Studio flips that: clone, design, dub, and dictate on your own hardware — 646 languages, no meter running, nothing leaving your machine.
Start with **VoiceStudio** (default, powered by k2-fsa/OmniVoice), or choose another engine. [Features & engine catalog](docs/feature-catalog.md).
> [!WARNING]
> **Active beta.** Things may break between releases — for the newest fixes, run from source. Bug reports and PRs are very welcome: [open an issue](https://github.com/debpalash/OmniVoice-Studio/issues) or [join Discord](https://discord.gg/bzQavDfVV9).
Local workflows run on your hardware. Remote services are optional; usage analytics requires consent.
<a id="screenshots"></a>
<details>
<summary><strong>Explore the workspaces</strong> · Clone, dub, design & models</summary>
## 📸 See it in action
<table>
<tr>
<td><img src="docs/media/electron/voice-cloning.png" alt="Electron voice cloning workspace with the bundled demo voice" width="100%" /></td>
<td><img src="docs/media/electron/dubbing.png" alt="Electron video dubbing workspace" width="100%" /></td>
<td align="center" width="50%">
<img src="docs/screenshot-studio.png" alt="Studio" width="100%"/>
<br/><b>Studio</b><br/>
<sub>Generate &amp; clone in one workspace — a 3-second clip mirrors any voice, 646 languages, zero-shot.</sub>
</td>
<td align="center" width="50%">
<img src="docs/screenshot-design.png" alt="Voice Design" width="100%"/>
<br/><b>Voice Design</b><br/>
<sub>Build new voices from scratch — gender, age, accent, pitch, emotion, dialect.</sub>
</td>
</tr>
<tr><td align="center">Voice cloning</td><td align="center">Video dubbing</td></tr>
<tr>
<td><img src="docs/media/electron/voice-design.png" alt="Describe a voice in the Electron voice design workspace" width="100%" /></td>
<td><img src="docs/media/electron/models.png" alt="Install and manage local speech models" width="100%" /></td>
<td align="center">
<img src="docs/screenshot-gallery.png" alt="Voice Gallery" width="100%"/>
<br/><b>Voice Gallery</b><br/>
<sub>Browse ready-made archetype voices with language filters, or build your own — then pick any of them in Studio, Audiobook, Stories, and Dubbing.</sub>
</td>
<td align="center">
<img src="docs/screenshot-dub.png" alt="Video Dubbing" width="100%"/>
<br/><b>Video Dubbing</b><br/>
<sub>A real dub, end to end: 37 segments transcribed, translated to Bengali, re-voiced, and timed — ready to export as MP4.</sub>
</td>
</tr>
<tr>
<td align="center">
<img src="docs/screenshot-engines.png" alt="Settings — Engines" width="100%"/>
<br/><b>Settings → Engines</b><br/>
<sub>The engine compatibility matrix — 14 TTS engines with per-engine GPU preflight, no silent CPU fallback.</sub>
</td>
<td align="center">
<img src="docs/screenshot-settings.png" alt="Settings — Models" width="100%"/>
<br/><b>Settings → Models</b><br/>
<sub>One-click model store — auto-detects your platform (CUDA / MPS / CPU) and recommends the right models.</sub>
</td>
</tr>
<tr><td align="center">Voice design</td><td align="center">Local models</td></tr>
</table>
<img width="2628" height="1950" alt="VoiceStudio desktop workspace" src="https://github.com/user-attachments/assets/b474497d-a453-49a3-a2dd-f023ec6b7659" />
---
</details>
<a id="features"></a>
## Get started
## ✨ Features
Download from [Releases](https://github.com/debpalash/VoiceStudio/releases/latest), then follow your platform guide:
Three flagships, five more headliners, and a dozen under the fold.
**[macOS](docs/install/macos.md) · [Windows](docs/install/windows.md) · [Linux](docs/install/linux.md) · [Docker](docs/install/docker.md)**
<table>
<tr>
<td width="33%"><img src="docs/features/clone.png" alt="Voice Cloning" width="100%"/></td>
<td width="33%"><img src="docs/features/design.png" alt="Voice Design" width="100%"/></td>
<td width="33%"><img src="docs/features/dub.png" alt="Video Dubbing" width="100%"/></td>
</tr>
<tr>
<td align="center">🎙️ <b>Voice Cloning</b><br/><sub>3-sec clip → any voice · 646 languages · zero-shot</sub></td>
<td align="center">🎨 <b>Voice Design</b><br/><sub>Describe it — gender, age, accent, emotion</sub></td>
<td align="center">🎬 <b>Video Dubbing</b><br/><sub>Transcribe → translate → re-voice → MP4</sub></td>
</tr>
</table>
Open **Voice cloning**, choose a voice or add a clean reference recording, enter your text, and generate. Install the required model when prompted. Hardware needs vary by engine; see [performance](docs/performance.md).
<table>
<tr>
<td align="center" width="20%">📖<br/><b>Audiobook</b><br/><sub>EPUB/PDF → .m4b, multi-voice cast</sub></td>
<td align="center" width="20%">🎭<br/><b>Stories</b><br/><sub>Multi-voice script editor</sub></td>
<td align="center" width="20%">⌨️<br/><b>Dictation Widget</b><br/><sub><kbd>⌘⇧Space</kbd> in any app</sub></td>
<td align="center" width="20%">🔐<br/><b>100% Local</b><br/><sub>No keys, no cloud, no accounts</sub></td>
<td align="center" width="20%">🤖<br/><b>MCP Server</b><br/><sub>Use from Claude, Cursor, …</sub></td>
</tr>
</table>
<details>
<summary><strong>Run the Electron preview from source</strong></summary>
<summary><b>…and 12 more</b> — isolation, diarization, batch, watermarking, diagnostics, and friends</summary>
```bash
git clone https://github.com/debpalash/VoiceStudio.git
cd VoiceStudio
bun install
bun run dev
```
<br/>
See [Electron setup](electron/README.md) for prerequisites and backend configuration.
- 🔊 **Vocal Isolation** — Demucs-powered: splits speech from music and keeps the background bed.
- 👥 **Speaker Diarization** — Pyannote + WhisperX auto-identify who said what.
- 📦 **Batch Queue** — drop 50 videos, walk away; per-job progress bars.
- 🛡️ **AI Watermark** — AudioSeal (Meta): invisible, survives compression.
- 🔬 **Diagnostics** — self-check suite, error journal, scrubbed diagnostic bundles.
- ⚡ **GPU Auto-Detect** — CUDA · MPS · ROCm (Linux, opt-in) · CPU; ≤8 GB VRAM auto-offloads.
- 🧭 **Engine routing** — preflight GPU check per engine; no silent CPU fallback.
- 🧩 **Extensible** — subclass `TTSBackend`, add any engine in ~50 lines.
- 🎒 **Portable personas** — export voices as `.ovsvoice` bundles: identity + watermark.
- ♾️ **Unlimited TTS** — sentence-chunked generation, no length cap, streaming via WebSocket.
- 🌐 **Remote backend** — point the UI at a remote server; Tailscale-friendly, bearer auth.
- 🧠 **Dictation + LLM** — local-LLM cleanup of transcripts, optional echo cancellation.
</details>
> **Electron is the primary desktop app.** The next desktop release ships Electron, with one final Tauri sunset update. Bug reports and contributions remain welcome; include the app version and whether you use Electron or Tauri.
---
## Documentation
<a id="quickstart"></a>
| Need | Start here |
## ⚡ Quickstart
<div align="center">
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/macOS-DMG_(Apple_Silicon)-000?style=for-the-badge&logo=apple&logoColor=white" alt="Download macOS DMG" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Windows-MSI_(x64)-0078D4?style=for-the-badge&logo=windows&logoColor=white" alt="Download Windows MSI" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Linux-AppImage_(x64)-FCC624?style=for-the-badge&logo=linux&logoColor=black" alt="Download Linux AppImage" /></a>
<br/>
<sub><b>macOS:</b> first launch needs a one-time approval — right-click → <b>Open</b> (or System Settings → Privacy &amp; Security → <b>"Open Anyway"</b> on macOS 15). No Terminal needed. <a href="docs/install/macos.md#gatekeeper-quarantine">Why?</a> · <b>Intel Macs:</b> local backend unsupported (<a href="https://github.com/debpalash/OmniVoice-Studio/issues/889">#889</a>) — <a href="docs/install/macos.md">details</a>.</sub>
</div>
**Install guide:** [🍎 macOS](docs/install/macos.md) · [🪟 Windows](docs/install/windows.md) · [🐧 Linux](docs/install/linux.md) · [🐳 Docker](docs/install/docker.md)
<details>
<summary><b>🧰 Troubleshooting · slow generation · HF tokens · restricted networks</b></summary>
<br/>
- **Something broke?** Run the self-check — **Settings → About → "Run self-check"** (or `uv run python backend/main.py --diagnose --deep`) — then the [top 10 install errors](docs/install/troubleshooting.md). **"Save diagnostic bundle"** packages scrubbed logs for a bug report.
- **Feels slow?** [docs/performance.md](docs/performance.md) — where the time goes and how to tune it.
- **Want breaths, laughter, emotion?** [docs/expressive-speech.md](docs/expressive-speech.md) — what each engine can do today.
- **HF tokens · diarization · download speed / mirrors:** [tokens](docs/setup/huggingface-token.md) · [diarization](docs/features/diarization.md) · [downloads](docs/downloading-models.md).
- **Coming from [Real-Time-Voice-Cloning](https://github.com/CorentinJ/Real-Time-Voice-Cloning)?** [Migration guide](docs/migration/real-time-voice-cloning.md).
</details>
---
<a id="why-ovs"></a>
## ⚖️ vs Others
ElevenLabs charges **$5$330/mo** and processes your audio on their servers. OmniVoice Studio runs **on your hardware, with no usage limits.**
| | **ElevenLabs** | **OmniVoice Studio** |
|---|---|---|
| **Pricing** | $5$330/mo, per-character billing | Free & open-source (AGPL-3.0) · [Commercial license](#license) for proprietary use |
| **Voice Cloning** | ✅ 3s clip | ✅ 3s clip, zero-shot |
| **Voice Design** | ✅ Gender, age | ✅ Gender, age, accent, pitch, style, dialect |
| **Audiobook / Stories** | ❌ | ✅ Full audiobook editor + multi-voice stories (EPUB/PDF import, .m4b export) |
| **Languages** | 32 | **646** |
| **Video Dubbing** | ✅ Cloud-only | ✅ Fully local |
| **Data Privacy** | Audio sent to cloud | **Nothing leaves your machine** |
| **API Keys** | Required | Not needed |
| **GPU Support** | N/A (cloud) | CUDA · Apple Silicon · ROCm (Linux) · CPU |
| **Desktop App** | ❌ | ✅ macOS · Windows · Linux |
| **TTS Engines** | 1 | **14** — [full matrix](#tts-engines) |
| **ASR Engines** | 1 | **11** — [full lineup](#asr-engines) |
| **MCP Server** | ❌ | ✅ Use from Claude, Cursor, any MCP client |
| **Self-check** | ❌ | ✅ Diagnostics suite, error journal, scrubbed debug bundles |
| **Customizable** | ❌ Closed | ✅ Fork it, extend it, ship it |
Professional-grade voice AI, minus the subscription and the cloud.
<div align="center">
<br/>
<b>Convinced? Come build with us.</b><br/>
<a href="https://discord.gg/bzQavDfVV9"><img src="https://img.shields.io/badge/Join_Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" alt="Join Discord" /></a>
<br/><br/>
</div>
---
## 🖥️ System Requirements
| | **Minimum** | **Recommended** |
|---|---|---|
| **OS** | Windows 10, macOS 12+ (Apple Silicon), Ubuntu 24.04+ (glibc 2.39+) | Any modern 64-bit OS |
| **RAM** | 8 GB | 16 GB+ |
| **VRAM (GPU)** | 4 GB (auto-offloads TTS to CPU) | 8 GB+ (NVIDIA RTX 3060+) |
| **Disk** | 10 GB free (models + cache) | 20 GB+ SSD |
| **Python** | 3.10+ (managed by `uv`) | 3.113.12 |
| **GPU** | Optional — CPU works | NVIDIA CUDA · Apple Silicon MPS · AMD ROCm (Linux only) |
> [!NOTE]
> **A GPU is optional** — the whole pipeline runs on CPU (just slower), and on ≤8 GB VRAM, TTS auto-offloads to CPU. Caveats: **AMD ROCm** is Linux-only + opt-in ([Linux](docs/install/linux.md#amd-gpu-rocm)) — Windows AMD/Ryzen AI is CPU-only ([Windows](docs/install/windows.md#gpu-support)); **macOS Intel** can't run the local backend, so point it at a remote one ([#889](https://github.com/debpalash/OmniVoice-Studio/issues/889) · [macOS](docs/install/macos.md)).
<a id="tts-engines"></a>
### 🗣️ TTS Engines
**14 engines, one picker.** OmniVoice (default, 600+ languages) is always available; seven more are opt-in and auto-detected (CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS-Nano, KittenTTS, MLX-Audio, Sherpa-ONNX), plus six lazy-installed heavyweights (IndexTTS 2, OmniVoice GGUF, Supertonic 3, MOSS-TTS-v1.5, dots.tts, Confucius4-TTS). Switch in **Settings → TTS Engine**; the choice applies everywhere synthesis happens.
<details>
<summary><b>📊 The full matrix</b> — 14 engines × platform × clone/instruct × license</summary>
<br/>
| Engine | Languages | Clone | Instruct | Linux | macOS ARM | Windows | License |
|--------|:---------:|:-----:|:--------:|:-----:|:---------:|:-------:|:-------:|
| **OmniVoice** (default) | 600+ | ✅ | ✅ | ✅ CUDA/CPU | ✅ MPS | ✅ CUDA/CPU | Built-in |
| **CosyVoice 3** | 9 + 18 dialects | ✅ | ✅ | ✅ CUDA/CPU | ✅ MPS | ✅ CUDA/CPU | Apache-2.0 |
| **GPT-SoVITS** | 5 | ✅ | — | ✅ CUDA/CPU | — | ✅ CUDA/CPU | MIT |
| **VoxCPM2** | 30 | ✅ | ✅ | ✅ CUDA/CPU | ✅ MPS | ✅ CUDA/CPU | Apache-2.0 |
| **MOSS-TTS-Nano** | 20 | ✅ | — | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
| **KittenTTS** | English | — | — | ✅ CPU | ✅ CPU | ✅ CPU | MIT |
| **MLX-Audio** (Kokoro, Qwen3-TTS, CSM, Dia, …) | Multi | Varies | Varies | ❌ | ✅ Native | ❌ | Varies |
| **Sherpa-ONNX** | 20+ | — | — | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
| **IndexTTS 2** ⚡ | Multi | ✅ | — | ✅ CUDA | — | ✅ CUDA | Apache-2.0 |
| **OmniVoice GGUF** ⚡ | 600+ | ✅ | ✅ | ✅ CPU | ✅ CPU | ✅ CPU | Built-in |
| **Supertonic 3** ⚡ | 31 | — | — | ✅ CPU | ✅ CPU | ✅ CPU | OpenRAIL-M |
| **MOSS-TTS-v1.5** ⚡ (8B) | 31 | ✅ | — | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
| **dots.tts** ⚡ (2B) | 24 | ✅ | — | ✅ CUDA/CPU | ✅ CPU | ❌ | Apache-2.0 |
| **Confucius4-TTS** ⚡ | 14 | ✅ | — | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
> **CUDA** = GPU-accelerated · **MPS** = Apple Silicon Metal · **CPU** = runs everywhere, slower for large models · KittenTTS and MOSS-TTS-Nano run realtime on CPU · MLX-Audio is Apple Silicon only · ⚡ = lazy-registered (installed on first use)
>
> **Clone** matters beyond single-clip generation: Video Dubbing (and any Batch job with a pinned voice) needs reference-audio cloning to preserve speaker identity, so picking a Clone-less engine (KittenTTS, Sherpa-ONNX, Supertonic 3) as the active engine fails those jobs up front with an actionable message instead of silently falling back to OmniVoice.
>
> **MOSS-TTS-v1.5** (8B, ~16 GB), **dots.tts** (2B, ~9 GB), and **Confucius4-TTS** are heavyweight opt-ins that run in their own isolated venv from a local clone. None claims Apple-Silicon MPS (CPU on Macs); dots.tts has no Windows path; Confucius4 wants CUDA (CPU works, ~17× realtime). Details: [MOSS-TTS-v1.5](docs/engines/moss-tts-v15.md) · [dots.tts](docs/engines/dots-tts.md) · [Confucius4-TTS](docs/engines/confucius4-tts.md).
</details>
<a id="asr-engines"></a>
### 🎧 ASR Engines
**11 engines** — they power dictation, video dubbing, and subtitles. **WhisperX** is the cross-platform default (~100 languages, word-level timing); the rest are opt-in and auto-detected. Switch in **Settings → Engines**. Ten run fully on-device; the eleventh (OpenAI-compatible) is an optional remote client for Qwen3-ASR or any compatible server.
<details>
<summary><b>📊 The full lineup</b> — 11 engines, what each is best at, and compute-type notes</summary>
<br/>
| Engine | `OMNIVOICE_ASR_BACKEND` | Languages | Best for |
|--------|-------------------------|:---------:|----------|
| **WhisperX** (default) | `whisperx` | ~100 | Dubbing & subtitles — word-level timing via wav2vec2 forced alignment |
| **Faster-Whisper** | `faster-whisper` | ~100 | Fast transcription on Linux / macOS / Windows (CTranslate2) |
| **Faster-Whisper (isolated)** | `faster-whisper-isolated` | ~100 | Same as Faster-Whisper but crash-isolated in a subprocess — an ASR crash won't take down the app |
| **MLX Whisper** | `mlx-whisper` | ~100 | Native Apple Silicon speed (Apple MLX / Metal) |
| **PyTorch Whisper** | `pytorch-whisper` | ~100 | CUDA / CPU fallback via 🤗 Transformers (no cuDNN 8 needed) |
| **Parakeet TDT** | `nemo-parakeet` | English + 25 EU | SOTA accuracy at ~10× realtime even on CPU, auto language detection (NVIDIA NeMo, CUDA/CPU) |
| **Parakeet TDT v3 (MLX)** | `parakeet-mlx` | 25 EU | The Parakeet tier for Apple Silicon — TDT word timestamps, ~2 GB unified memory, dictation-grade speed on the GPU via MLX. Install the model from **Settings → Models** and dictation prefers it automatically when your system language is one of its 25 (European) languages; other languages (CJK, Arabic, …) keep the multilingual Whisper engine so dictation coverage never regresses. |
| **Moonshine** | `moonshine` | English | Edge / low-latency, ONNX |
| **FunASR** | `funasr` | 50+ | All-in-one multilingual — built-in VAD + inline speaker diarization (SenseVoice) |
| **sherpa-onnx** (live dictation) | `sherpa-onnx-asr` | 25 EU + 90+ | Live, faster-than-real-time dictation — small streaming/offline ONNX models (Parakeet TDT v3/v2, streaming Zipformer & Paraformer, Whisper Tiny), CPU, identical on macOS / Windows / Linux. Picked per-model in **Settings → Voice**. |
| **OpenAI-compatible** ⚠️ remote | `openai-compat-asr` | Server-dependent | A path to **Qwen3-ASR** today (self-hosted server, no transformers wait), any OpenAI-compatible transcription endpoint, or OpenAI's own API — no install, configure + test the connection in **Settings → Engines** (ASR tab). Audio leaves your machine to whatever server you point it at; see [docs/engines/openai-compatible-asr.md](docs/engines/openai-compatible-asr.md). |
> Whisper-family engines cover ~100 languages; **FunASR / SenseVoice** adds an all-in-one multilingual path with built-in voice-activity detection and inline speaker diarization. **sherpa-onnx** powers the live dictation model picker — you talk and text appears as you speak. Every engine runs on-device — no API keys, no cloud.
> **GPU without efficient float16?** On older NVIDIA GPUs (Maxwell/Pascal, GTX 16xx) or after a CTranslate2/cuDNN mismatch, the CTranslate2 ASR engines (WhisperX, Faster-Whisper) can't run `float16` and OmniVoice automatically retries on `int8` — no config needed. If transcription still fails, pin the compute type with the `ASR_COMPUTE_TYPE` env var (escape hatch): `ASR_COMPUTE_TYPE=int8` (or `float32` for CPU). Set it to `int8` and restart the backend.
</details>
---
## 🏗️ Architecture
A **Tauri v2** desktop shell (Rust) wraps a **React** UI and a bundled **Python/FastAPI** backend that runs as a local sidecar on `localhost:3900`. Nothing external — every layer is on your machine.
```
┌────────────────────────────────────────────────────────────────────┐
│ Tauri v2 shell — Rust │
│ window state · global dictation hotkey · system tray · │
│ signed auto-updater (stable/preview) · single-instance · │
│ first-run bootstrap (installs uv + Python venv) · blank guard │
├────────────────────────────────────────────────────────────────────┤
│ Frontend — React + Vite │
│ Studio · Dub · Stories · Audiobook · Gallery · Dictation · │
│ Batch · Diagnostics · MCP client — Zustand store · WS bus │
│ ▲ IPC / HTTP + WS │
├──────────────────────────┼─────────────────────────────────────────┤
│ Backend — FastAPI sidecar @ localhost:3900 │
│ 100+ REST endpoints · SSE + WebSocket streaming · │
│ SQLite + Alembic (omnivoice_data/) · OpenAI-compatible API │
├───────────┬───────────┬───────────┬───────────┬────────────────────┤
│ TTS ×14 │ ASR ×11 │ Demucs │ Pyannote │ AudioSeal │
│ clone / │ WhisperX │ vocal │ speaker │ watermark │
│ design │ +10 more │ isolation│ diariz. │ embed / detect │
├───────────┴───────────┴───────────┴───────────┴────────────────────┤
│ Engine routing — per-engine GPU preflight, no silent CPU fallback │
│ Hardware: CUDA · MPS · ROCm (Linux) · CPU (auto-detected) │
└────────────────────────────────────────────────────────────────────┘
```
- **Shell (Rust)** — native OS integration: the system-wide dictation hotkey, tray, signed auto-updater (stable + preview channels), single-instance lock, and the first-run bootstrap that installs `uv` and a Python 3.11 venv.
- **Frontend (React)** — every workspace tab over a Zustand store, with a WebSocket event bus that live-refreshes the UI when backend data changes.
- **Backend (FastAPI)** — the bundled Python sidecar: 100+ endpoints, SSE/WSS streaming, a SQLite DB migrated by Alembic, and the OpenAI-compatible API surface.
- **Engines** — 14 TTS + 11 ASR, plus Demucs (isolation), Pyannote (diarization), and AudioSeal (watermark), all behind routing that GPU-preflights each engine and refuses to silently fall back to CPU.
<a id="openai-api"></a>
## 🔌 OpenAI-compatible API
<div align="center">
**Drop-in replacement for OpenAI / ElevenLabs audio.** One line — no key, no code changes:
```diff
- base_url="https://api.openai.com/v1"
+ base_url="http://localhost:3900/v1"
```
</div>
Your existing scripts, agents, and OpenAI/ElevenLabs SDK calls now run **locally** on whatever engine you have active. What the cloud can't do: `voice` takes **your own cloned-voice profile IDs**, and `model` can pin a **specific engine** per request.
| Endpoint | What it does |
|---|---|
| Setup help | [Troubleshooting](docs/install/troubleshooting.md) · [Model downloads](docs/downloading-models.md) |
| Models & audio quality | [Engine guides](docs/engines/README.md) · [Benchmarks](docs/benchmarks.md) |
| Integrations | [Local API](docs/speech-platform.md) · [MCP](docs/mcp.md) · [Examples](examples/README.md) |
| Development | [Contributing](.github/CONTRIBUTING.md) · [Electron](electron/README.md) · [Changelog](CHANGELOG.md) |
| `POST /v1/audio/speech` | TTS — text in; `mp3` / `opus` / `aac` / `flac` / `wav` / `pcm` out. `model`: `tts-1`/`tts-1-hd` (active engine) or a specific one (`voxcpm2`, `cosyvoice`, `kittentts`, …). `voice`: a cloned profile ID, `default`, or an OpenAI name (`alloy`, …). `speed` supported. |
| `POST /v1/audio/transcriptions` | STT — audio file in; `json` / `text` / `verbose_json` / `srt` / `vtt` out (`verbose_json` adds word-level timings). `whisper-1` maps to your active ASR engine. |
| `GET /v1/audio/voices` | OmniVoice extension — lists every voice profile and engine, so clients can discover your clones. |
Agent skills: `npx skills add debpalash/VoiceStudio` — choose **voicestudio** for audio workflows or **voicestudio-maintainer** for repository maintenance.
**Speak with your own cloned voice** — list the IDs, then pass one as `voice`:
## Sponsors
```sh
# 1 — find a cloned voice's profile ID
curl -s http://localhost:3900/v1/audio/voices | jq '.voices[] | select(.type=="profile") | {voice_id, name}'
<a href="https://forms.gle/2PYCvd39hbwijzX37"><img src="docs/media/sponsor-slot.svg" alt="Your brand — apply for a featured VoiceStudio sponsor slot" width="640" /></a>
# 2 — synthesize with it
curl http://localhost:3900/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{"model":"tts-1","voice":"<profile-id>","input":"Made on my own hardware.","response_format":"wav"}' \
--output speech.wav
```
**Become a featured partner.** [Apply for a paid placement](https://forms.gle/2PYCvd39hbwijzX37) · [Email us](mailto:partner@voicestudio.sh)
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:3900/v1", api_key="none") # any string — nothing checks it
Support development: [Ko-fi](https://ko-fi.com/debpalash) · [PayPal](https://paypal.me/palashCoder) · [Sponsorship details](SPONSORS.md)
# TTS with your cloned voice (or "alloy" / "default"; model= can pin a specific engine)
with client.audio.speech.with_streaming_response.create(
model="tts-1", voice="<profile-id>", input="Made on my own hardware.") as r:
r.stream_to_file("speech.wav")
## License & responsible use
# STT
print(client.audio.transcriptions.create(model="whisper-1", file=open("clip.wav", "rb")).text)
```
[AGPL-3.0](LICENSE). Models have their own licenses; review them before commercial use. Clone voices only with permission. See [license details](LICENSE-NOTICE.md).
Want the whole surface (100+ endpoints)? The full REST API reference is embedded in the app — **Settings → OpenAPI Reference** (Scalar-powered), or the `{}` button in the footer.
Calling the backend from **another machine** (LAN, Tailscale, behind a proxy)? It's loopback-only and unauthenticated by default; to reach it remotely you set a share PIN or an API key. [docs/api-auth.md](docs/api-auth.md) covers the exact headers, query params, `401`/`403`/`429` meanings, and the `OMNIVOICE_TRUSTED_NETWORKS` exemption.
### 📓 Run on Google Colab
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/debpalash/OmniVoice-Studio/blob/main/notebooks/OmniVoice_Studio_Colab.ipynb)
No local GPU? The [official notebook](notebooks/OmniVoice_Studio_Colab.ipynb) boots the full app — web UI included — on a free Colab T4, then walks the whole feature surface (TTS, cloning, design, transcription, dubbing, audiobook, watermarking, the OpenAI-compatible API) as a guided tour with inline playback. No tunnels, no API keys.
### 🤝 Agent Skills
Teach your coding agent to speak and listen through your local OmniVoice — one command, works with **Claude Code, Codex, Cursor, Grok, Kimi, opencode**, and any [skills.sh](https://skills.sh)-compatible agent:
```sh
npx skills add debpalash/omnivoice-studio
```
Ships two [skills](https://skills.sh):
- **`omnivoice`** — generate speech (including your cloned voices) and transcribe audio from any agent, free and fully offline via your local install.
- **`oss-maintainer`** — the maintainer methodology this project is run with, for anyone running their own OSS project with an agent.
---
## 🗺️ Roadmap
### 🔜 Up Next
- 🎬 **Lip-sync v2** — visual speech timing with wav2lip
- 🌐 **Hosted Demo** — try OmniVoice without installing anything
- 🔌 **Plugin Marketplace** — community-contributed TTS engines and effects
- 🎵 **Real-time Voice Changer** — live microphone transformation during calls
<details>
<summary><b>✅ Everything shipped so far</b> — the receipts, by category</summary>
<br/>
| Category | Features |
|----------|----------|
| **Longform** | Audiobook editor (text/EPUB/PDF → chaptered .m4b) with multi-voice cast, expressive controls, live per-chapter progress + Stop, and a one-click sample; Stories multi-voice editor, two-pass loudnorm mastering, crash-resume for interrupted renders, pronunciation control + SSML-lite prosody |
| **Dubbing** | Full pipeline (transcribe→translate→synthesize→mux), scene-aware splitting, lip-sync scoring, streaming TTS, per-speaker voice assignment, Smart Fit timing + second-pass QC, paste-in translations from any external tool, dedicated Dub home |
| **Voice** | Zero-shot cloning, voice design, A/B comparison, voice preview widget, gallery with favorites/tags (its voices selectable in every picker — Studio, Audiobook, Stories, Dubbing), portable persona bundles (`.ovsvoice`), voice console workspace |
| **Audio** | Demucs vocal isolation, per-segment gain, selective track export, stem/SRT/VTT/MP3 export, unlimited-length TTS via sentence-chunked generation |
| **Multi-Lang** | Multi-language batch picker, batch dubbing queue with sequential GPU execution |
| **Diarization** | Pyannote ML diarization, auto speaker clone extraction, per-speaker voice assignment |
| **ASR** | 11 engines (WhisperX, Faster-Whisper, isolated Faster-Whisper, MLX Whisper, PyTorch Whisper, Parakeet TDT, Parakeet TDT v3 MLX, Moonshine, FunASR/SenseVoice, sherpa-onnx live dictation, OpenAI-compatible remote), crash-isolated subprocess backend |
| **TTS** | 14 engines (OmniVoice, CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS-Nano, KittenTTS, MLX-Audio, Sherpa-ONNX, + lazy: IndexTTS 2, OmniVoice GGUF, Supertonic 3, MOSS-TTS-v1.5, dots.tts, Confucius4-TTS), engine routing with GPU preflight |
| **Infra** | Docker deployment, CUDA/MPS/ROCm auto-detect, cuDNN 8 compat, VRAM-aware model offloading, engine routing (no silent CPU fallback), diagnostics suite & error journal, restricted-network mirror support |
| **AI Provenance** | AudioSeal invisible watermarking (SynthID-like), video logo overlay, watermark detection API |
| **UX** | Undo/redo, keyboard shortcuts, drag-and-drop, session persistence, glassmorphism design system, UI scale fix for Linux/WebKitGTK |
| **Real-time Events** | WebSocket event bus — instant sidebar refresh on data mutations, exponential backoff reconnect |
| **State Management** | Zustand store migration — `uiSlice`, `pillSlice`, `dubSlice`, `generateSlice`, `prefsSlice`, `glossarySlice` |
| **Desktop** | Cross-platform Tauri installers (macOS DMG — Apple Silicon; Intel unsupported for the local backend, #889 — Windows MSI, Linux deb/AppImage), auto-update infrastructure, single-instance enforcement, close-to-tray, macOS Gatekeeper fix |
| **Dictation** | Global system-wide hotkey (`⌘+⇧+Space`), frameless floating widget, streaming ASR via WebSocket, auto-paste, customizable hotkey, local-LLM transcript refinement |
| **Batch Pipeline** | Full batch TTS: extract → transcribe → translate → generate → mix → export, with live progress tracking |
| **MCP Server** | OmniVoice as a local TTS/STT provider for Claude, Cursor, and any MCP client |
| **Remote Backend** | Point the desktop UI at a remote backend URL with bearer auth (Tailscale-documented) |
| **Reliability** | Stall watchdog on bootstrap splash, per-engine GPU compatibility matrix, actionable errors for non-executable engine binaries, setuptools auto-repair |
</details>
---
<a id="sponsor--donate"></a>
## 💜 Sponsor / Donate
One developer, real AI-agent bills. If OmniVoice is useful to you, chipping in keeps development full-time — every dollar goes straight to the bills.
<div align="center">
<img src="https://img.shields.io/badge/raised_%2410_of_%24200-5%25-EAB308?style=for-the-badge" alt="This month's agent-bill fund: $10 / $200" />
<br/><br/>
<a href="https://ko-fi.com/debpalash"><img src="https://img.shields.io/badge/Ko--fi-Support_❤️-FF5E5B?style=for-the-badge&logo=ko-fi&logoColor=white" alt="Ko-fi" /></a>
&nbsp;&nbsp;
<a href="https://paypal.me/palashCoder"><img src="https://img.shields.io/badge/PayPal-Donate-00457C?style=for-the-badge&logo=paypal&logoColor=white" alt="PayPal" /></a>
<br/><br/>
<sub>Also from the maker: <a href="https://github.com/debpalash/Opal"><b>Opal</b> 💠</a> · <a href="https://github.com/debpalash/memxt"><b>memxt</b> 🧠</a> — a ⭐ helps too.</sub>
</div>
<a id="sponsors"></a>
### 🌟 Sponsors
OmniVoice is **free** and **AGPL-3.0** — no paid tier, no SaaS revenue. Sponsors keep development going, and in return get a logo slot here, in the app, and (for top tiers) on the project website. It's a thank-you, never a paywall. **[See tiers & become a sponsor →](SPONSORS.md)**
<div align="center">
<!-- SPONSORS:START — logo slots are filled here as sponsors come aboard; see SPONSORS.md -->
**Your logo here** — [become a sponsor](SPONSORS.md)
<!-- SPONSORS:END -->
</div>
<sub>💡 GitHub also shows a **Sponsor** button at the top of this repo, wired to the same links via <a href=".github/FUNDING.yml"><code>.github/FUNDING.yml</code></a>.</sub>
---
## 💬 Community
<div align="center">
<a href="https://discord.gg/bzQavDfVV9"><img src="https://img.shields.io/badge/💬_Discord-Join_Community-5865F2?style=for-the-badge&logo=discord&logoColor=white" alt="Join Discord" /></a>
<br/>
<sub>We respond to setup questions within hours, not days.</sub>
</div>
<details>
<summary><b>What happens in there</b></summary>
<br/>
| Channel | What happens there |
|---------|--------------------|
| `#announcements` | Release news and the big moments — new versions land here first |
| `#releases` + `#changelog` | Every build and exactly what's inside it |
| `#issues` | Bug reports as forum posts — triaged straight into GitHub issues |
| `#ideas` | Feature requests, discussed and voted on |
| `#discuss-ideas` | Design talk before things get built |
| `#general` | Setup help, GPU troubleshooting, and showing off your dubs |
</details>
---
<a id="contributing"></a>
## 🤝 Contributing
Yes please — bug fixes, new TTS engine adapters, UI improvements, docs, translations. All of it.
- 📖 Read the **[Contributing Guide](.github/CONTRIBUTING.md)** for setup, code style, and PR workflow
- 🐛 Browse [good first issues](https://github.com/debpalash/OmniVoice-Studio/labels/good%20first%20issue)
- 💬 Join our [Discord](https://discord.gg/bzQavDfVV9) to discuss ideas or ask for help
---
## ❓ FAQ
<details>
<summary><b>Is this really as good as ElevenLabs?</b></summary>
<br/>
Honest answer: <b>it depends on what you're doing.</b>
<b>Where OmniVoice is genuinely competitive:</b> voice cloning from a clean reference clip (state-of-the-art open diffusion TTS), language coverage (646 languages vs. their 32), and everything structural — no per-character billing, no usage caps, no audio leaving your machine, full pipeline customizability (14 TTS engines, 11 ASR engines, your choice of translation).
<b>Where ElevenLabs still wins:</b> out-of-the-box consistency and polish, especially for English TTS. Their one model is heavily tuned; our quality depends on which engine you pick, your hardware, and — for cloning — the reference audio (a dry, close-mic clip clones dramatically better than a noisy or echoey one).
<b>For dubbing specifically:</b> a dub is a chain — transcription → translation → cloning → synthesis — only as good as its weakest link on <i>your</i> source material. If parts come out incoherent, check the segment table's <i>original</i> text first: when the transcription is already wrong, switch the ASR engine or use cleaner source audio — that's usually the fix, not the voice.
Try it on your real material — it's free and takes one download. Many users replace ElevenLabs outright; some keep both. Both outcomes are fine with us.
</details>
<details>
<summary><b>Why doesn't a longer reference clip sound more like me?</b></summary>
<br/>
Because OmniVoice's cloning is <b>zero-shot</b>: your clip is a <i>prompt</i> the model conditions on at generation time — it is never trained on. Feeding it 2 hours doesn't teach it your voice; past a short window the extra audio is simply not used. The dubbing pipeline's reference builder targets ~8 s and hard-caps at 15 s (<code>backend/services/speaker_clone.py</code>), and engines cap the prompt themselves (VoxCPM2 trims references to 30 s). This is different from ElevenLabs <i>Professional</i> Voice Cloning, which fine-tunes a model on hours of your audio — that's a training job, not a bigger prompt.
<b>What actually moves clone quality is the clip, not its length.</b> Zero-shot cloning mirrors the acoustics and delivery of the prompt, so: record 515 seconds (~8 s is the sweet spot) of continuous natural speech, close to the mic, in a quiet room with no reverb or music — an echoey clip clones echoey. One speaker only, and read in the tone and pace you want the output to have, because the clone copies your delivery, not just your timbre. Recording a few candidate clips and comparing results beats any amount of extra footage.
<b>Want audiobook-grade, trained-on-your-voice fidelity?</b> That path exists, but it's offline fine-tuning, not an in-app button: prepare a dataset of your recordings (<a href="docs/data_preparation.md">docs/data_preparation.md</a>) and fine-tune the bundled checkpoint via <code>init_from_checkpoint</code> (<a href="docs/training.md">docs/training.md</a>). Fair warning — it's a technical, command-line workflow that needs a capable GPU and hours of transcribed audio. In-app fine-tuning / long-reference "professional" cloning is on the <a href="docs/ROADMAP.md">roadmap</a> as research only; no promised date.
</details>
<details>
<summary><b>Does it work on Apple Silicon (M1/M2/M3/M4)?</b></summary>
<br/>
Yes. MPS acceleration is auto-detected. MLX-optimized Whisper models are available for faster transcription on Apple hardware. <b>Intel Macs are not supported</b>: the app UI installs, but the local Python backend cannot run because PyTorch no longer ships Intel-Mac wheels (<a href="https://github.com/debpalash/OmniVoice-Studio/issues/889">#889</a>) — an Intel Mac can only be used with a remote backend.
</details>
<details>
<summary><b>How much VRAM do I need?</b></summary>
<br/>
<b>4 GB minimum.</b> With ≤8 GB, the TTS model is automatically offloaded to CPU during transcription. With 8+ GB, everything runs on GPU simultaneously. No GPU at all? CPU mode works — just slower (~3× for TTS).
</details>
<details>
<summary><b>Can I use this commercially?</b></summary>
<br/>
<b>Yes — commercial use is free</b> under the <a href="https://www.gnu.org/licenses/agpl-3.0.html">AGPL-3.0</a>: run it, sell the audio you make, dub client videos, deploy it across your team. One obligation: if you <b>modify</b> OmniVoice and offer the modified version to others over a network, you must share that modified source under the same terms. Embedding it in a closed-source product instead? A commercial license is available — see <a href="#license">License</a>.
</details>
<details>
<summary><b>What languages are supported?</b></summary>
<br/>
646 languages for TTS via the OmniVoice model. Transcription (WhisperX) supports 99 languages. Translation coverage depends on the target language pair.
</details>
<details>
<summary><b>Can I add my own TTS engine?</b></summary>
<br/>
Yes. Subclass <code>TTSBackend</code> in <code>backend/services/tts_backend.py</code> and add it to the <code>_REGISTRY</code> dictionary — ~50 lines. The fourteen built-in engines all work this way; see <a href="#tts-engines">TTS Engines</a>.
</details>
<details>
<summary><b>Does OmniVoice collect any data about me?</b></summary>
<br/>
<b>Not unless you explicitly say yes.</b> On first run the app <i>asks</i> — one screen, two equal-weight buttons, no pre-ticked box — and until you answer yes, OmniVoice sends nothing: no analytics, no telemetry, no accounts, no phone-home. Skipping the question means no. Your text, audio, voices, and projects never leave your machine either way.
If you do opt in (also togglable anytime under <b>Settings → Privacy → "Help improve OmniVoice"</b>), what's sent is anonymous, content-free usage stats: generations (engine, language, generation time, character <i>count</i>, error <i>type</i>), plus app lifecycle — an install ping, updates (version-to-version), crashes (error class and a <i>bucketed</i> uptime, never logs), error <i>types</i> (capped, deduplicated), and a single uninstall ping if you remove it. Never your text, audio, file names, or anything identifying — enforced in code by a property allowlist (<code>backend/core/analytics.py</code>), not just a promise. Every build — installer, Docker, or built from source — asks the same first-run question and stays off unless you say yes (the destination is PostHog's publishable write-only client key; skipping the question means off). Your own numbers live in <b>Settings → Usage</b>, computed locally, sent nowhere.
</details>
<details>
<summary><b>How do I uninstall it / remove all its data?</b></summary>
<br/>
OmniVoice is fully local — uninstalling is just deleting the app plus the folders it wrote (model cache, Python env, your voices/projects, config). Run <code>scripts/uninstall.sh</code> (macOS/Linux) or <code>scripts\uninstall.ps1</code> (Windows) — it prints every folder with its size as a dry-run first, then deletes on <code>--yes</code>. The full per-platform path list and app-removal steps are in <a href="docs/install/uninstall.md"><b>docs/install/uninstall.md</b></a>.
</details>
---
<a id="license"></a>
## 📜 License
OmniVoice Studio is free and open-source software under the [**GNU Affero General Public License v3.0 (AGPL-3.0)**](https://www.gnu.org/licenses/agpl-3.0.html).
**Free for any use — including commercial and internal business use.** Run it, sell the audio you produce with it, dub your own or clients' videos, roll it out across your team — all free, no license needed. As a **network copyleft** license, AGPL adds one obligation: if you **modify** OmniVoice Studio and offer that modified version to others over a network, you must make the complete corresponding source of your modified version available to them under the same AGPL-3.0 terms.
A **commercial license** is available for organizations that want to embed OmniVoice Studio in a **closed-source or proprietary** product or service without the AGPL-3.0 copyleft obligations. **Pricing tiers coming soon.** Inquiries: **OmniVoice@palash.dev**.
The bundled `omnivoice/` TTS model by Han Zhu remains Apache-2.0 upstream. See [`LICENSE`](LICENSE) for the full, binding terms, and [`LICENSE-NOTICE.md`](LICENSE-NOTICE.md) for the plain-language summary and scope.
---
## 🙏 Acknowledgments
OmniVoice Studio is built on the shoulders of exceptional open-source work:
| Project | Role |
|---------|------|
| [**OmniVoice (k2-fsa)**](https://github.com/k2-fsa/OmniVoice) | Zero-shot diffusion TTS engine — the core voice synthesis model |
| [**WhisperX**](https://github.com/m-bain/whisperX) | Word-level speech recognition and alignment |
| [**Demucs (Meta)**](https://github.com/facebookresearch/demucs) | Music source separation for vocal isolation |
| [**Pyannote**](https://github.com/pyannote/pyannote-audio) | Speaker diarization — who said what |
| [**CTranslate2**](https://github.com/OpenNMT/CTranslate2) | Optimized Transformer inference on CPU and GPU |
| [**AudioSeal (Meta)**](https://github.com/facebookresearch/audioseal) | Invisible neural audio watermarking for AI provenance |
| [**Tauri**](https://tauri.app) | Native desktop app framework |
| [**Supertone / Supertonic 3**](https://huggingface.co/Supertone/supertonic-3) | ONNX TTS engine — 31 languages, CPU-efficient |
| [**Sherpa-ONNX**](https://github.com/k2-fsa/sherpa-onnx) | WASM-ready universal TTS/ASR runtime |
| [**GPT-SoVITS**](https://github.com/RVC-Boss/GPT-SoVITS) | Zero-shot TTS engine — 5 languages, RTF 0.014 |
---
<a id="more-from-the-maker"></a>
## 🧰 More local open-source from the maker
Like the local-first philosophy? It runs in the family — same maker, same rule: **your data stays on your machine.**
<table>
<tr>
<td align="center" width="50%" valign="top">
<br/>
<a href="https://github.com/debpalash/Opal"><img src="https://raw.githubusercontent.com/debpalash/Opal/main/assets/opal_logo.png" width="96" alt="Opal logo"/></a>
<h3><a href="https://github.com/debpalash/Opal">Opal 💠</a></h3>
<p><b>Play everything.</b> The media player for the AI era.</p>
<p><sub>Video, anime, comics, torrents, Jellyfin & Plex — one player for all of it, with local AI memory and context built in. Written in Zig, runs on macOS & Windows.</sub></p>
<p>
<a href="https://github.com/debpalash/Opal/stargazers"><img src="https://img.shields.io/github/stars/debpalash/Opal?style=flat-square&color=f59e0b" alt="Opal stars"/></a>
<a href="https://palash.dev/opal"><img src="https://img.shields.io/badge/site-palash.dev%2Fopal-8b5cf6?style=flat-square" alt="Opal website"/></a>
</p>
</td>
<td align="center" width="50%" valign="top">
<br/>
<a href="https://github.com/debpalash/memxt"><img src="https://raw.githubusercontent.com/debpalash/memxt/main/assets/logo-mark.svg" width="96" alt="memxt logo"/></a>
<h3><a href="https://github.com/debpalash/memxt">memxt 🧠</a></h3>
<p><b>The fastest benchmarked open-source AI memory system.</b></p>
<p><sub>Local long-term memory for Claude Code and coding agents — an MCP server on SQLite + embeddings, 100% on your machine. Your agent finally remembers yesterday.</sub></p>
<p>
<a href="https://github.com/debpalash/memxt/stargazers"><img src="https://img.shields.io/github/stars/debpalash/memxt?style=flat-square&color=f59e0b" alt="memxt stars"/></a>
<a href="https://github.com/debpalash/memxt#readme"><img src="https://img.shields.io/badge/docs-README-10b981?style=flat-square" alt="memxt docs"/></a>
</p>
</td>
</tr>
</table>
---
<div align="center">
<br/>
If you read this far, you're our kind of person.<br/>
**[⭐ Star this repo](https://github.com/debpalash/OmniVoice-Studio)** so others can find it too.<br/>
**[💬 Join the Discord](https://discord.gg/bzQavDfVV9)** to share what you build.<br/>
**[❤️ Support development](https://ko-fi.com/debpalash)** — fund the AI agent bills that keep OmniVoice shipping.
<br/>
<a href="https://star-history.com/#debpalash/OmniVoice-Studio&Date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=debpalash/OmniVoice-Studio&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=debpalash/OmniVoice-Studio&type=Date" />
<img alt="Star History" src="https://api.star-history.com/svg?repos=debpalash/OmniVoice-Studio&type=Date&theme=dark" width="600" />
</picture>
</a>
</div>
+632 -47
View File
@@ -1,81 +1,666 @@
*本文档是 [README.md](README.md) 的简体中文翻译;若与英文版有出入,以英文版为准。*
<div align="center">
<img src="docs/logo.png" alt="VoiceStudio" width="88" />
<h1>VoiceStudio</h1>
<p><strong>开源声音克隆与工作流引擎。在本地构建。</strong></p>
<p>使用本地 AI 克隆声音、翻译配音、语音听写和制作有声书。</p>
<img src="docs/logo.png" alt="OmniVoice 徽标" width="120" />
<h1>OmniVoice Studio</h1>
<h3>开源版 ElevenLabs 替代品。</h3>
<p>实时听写、零样本语音克隆、电影级视频配音——全部在你的桌面上完成。<br/><b>无需账号。无需 API 密钥。无需云端。</b>一切都在你自己的设备上运行。开源,支持 <b>646 种语言</b>。</p>
<p>
<a href="https://github.com/debpalash/VoiceStudio/releases/latest">下载</a> ·
<a href="#开始使用">开始使用</a> ·
<a href="#文档">文档</a> ·
<a href="#quickstart">快速开始</a> ·
<a href="#features">功能</a> ·
<a href="#why-ovs">为什么选择 OVS</a> ·
<a href="#tts-engines">引擎</a> ·
<a href="#openai-api">API</a> ·
<a href="#sponsor--donate">捐赠</a> ·
<a href="#contributing">参与贡献</a> ·
<a href="https://discord.gg/bzQavDfVV9">Discord</a> ·
<a href="README.md">English</a>
<a href="README.md"><strong>English</strong></a>
</p>
<p>
<a href="https://github.com/debpalash/OmniVoice-Studio/stargazers"><img src="https://img.shields.io/github/stars/debpalash/OmniVoice-Studio?style=flat-square&color=f59e0b" alt="Star 数" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/github/v/release/debpalash/OmniVoice-Studio?style=flat-square&color=10b981" alt="版本" /></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/license-AGPL--3.0-blue?style=flat-square" alt="许可证" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/issues"><img src="https://img.shields.io/github/issues/debpalash/OmniVoice-Studio?style=flat-square&color=ef4444" alt="Issues" /></a>
<a href="https://discord.gg/bzQavDfVV9"><img src="https://img.shields.io/badge/Discord-Join_Community-5865F2?style=flat-square&logo=discord&logoColor=white" alt="Discord" /></a>
<a href="https://ko-fi.com/debpalash"><img src="https://img.shields.io/badge/Ko--fi-Support_Us-FF5E5B?style=flat-square&logo=ko-fi&logoColor=white" alt="Ko-fi" /></a>
<a href="https://paypal.me/palashCoder"><img src="https://img.shields.io/badge/PayPal-Donate-00457C?style=flat-square&logo=paypal&logoColor=white" alt="PayPal" /></a>
</p>
<p>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/⬇_Download-macOS_·_Windows_·_Linux-10b981?style=for-the-badge" alt="下载最新版本" /></a>
</p>
</div>
![Electron 应用演示:声音克隆、声音设计、视频配音和模型管理](docs/media/electron/voicestudio.gif)
<br/>
<p align="center"><sub>新 Electron 桌面界面,使用此分支及内置演示声音录制。正式发布版本的界面可能有所不同。</sub></p>
<div align="center">
<img src="docs/screenshot-launchpad.png" alt="OmniVoice Studio — 启动台" width="100%"/>
</div>
## 用 VoiceStudio 创作
> **你的声音是你最私密的数据。为什么还要按月付费,从云端把它租回来?** 每一款主流语音工具都会把你的音频送到别人的服务器上,并按月向你收费。OmniVoice Studio 反其道而行:克隆、设计、配音、听写,全部在你自己的硬件上完成——646 种语言,没有计费表在转,任何数据都不离开你的设备。
- **声音克隆与设计**:上传参考录音,或用文字描述你想要的声音。
- **视频配音**:转录、翻译、分配说话人,并编辑语音时间轴
- **语音听写**:通过悬浮录音组件录制、转录和复制文字。
- **长篇创作**:制作多角色脚本、有声书和批量任务。
- **模型管理**:选择语音合成与转录引擎、语言及计算设备。
> [!WARNING]
> **活跃 Beta 阶段。** 各版本之间可能出现故障——如需最新修复,请从源码运行。非常欢迎 Bug 报告和 PR:[提交 Issue](https://github.com/debpalash/OmniVoice-Studio/issues) 或 [加入 Discord](https://discord.gg/bzQavDfVV9)
本地工作流在你的硬件上运行。远程服务为可选功能;使用情况分析须经同意才会启用。
<a id="screenshots"></a>
## 📸 实际效果
<table>
<tr>
<td><img src="docs/media/electron/voice-cloning.png" alt="Electron 声音克隆工作区与内置演示声音" width="100%" /></td>
<td><img src="docs/media/electron/dubbing.png" alt="Electron 视频配音工作" width="100%" /></td>
<td align="center" width="50%">
<img src="docs/screenshot-studio.png" alt="工作" width="100%"/>
<br/><b>工作室(Studio</b><br/>
<sub>在同一个工作区里生成与克隆——3 秒音频即可复刻任何声音,646 种语言,零样本。</sub>
</td>
<td align="center" width="50%">
<img src="docs/screenshot-design.png" alt="声音设计" width="100%"/>
<br/><b>声音设计</b><br/>
<sub>从零构建新声音——性别、年龄、口音、音高、情感、方言。</sub>
</td>
</tr>
<tr><td align="center">声音克隆</td><td align="center">视频配音</td></tr>
<tr>
<td><img src="docs/media/electron/voice-design.png" alt="Electron 声音设计工作区" width="100%" /></td>
<td><img src="docs/media/electron/models.png" alt="本地语音模型管理" width="100%" /></td>
<td align="center">
<img src="docs/screenshot-gallery.png" alt="声音库" width="100%"/>
<br/><b>声音库</b><br/>
<sub>浏览现成的原型声音,支持语言筛选——或构建你自己的声音库。</sub>
</td>
<td align="center">
<img src="docs/screenshot-dub.png" alt="视频配音" width="100%"/>
<br/><b>视频配音</b><br/>
<sub>一次端到端的真实配音:37 个片段完成转录、翻译成孟加拉语、重新配音并对齐时间轴——随时可导出为 MP4。</sub>
</td>
</tr>
<tr>
<td align="center">
<img src="docs/screenshot-engines.png" alt="设置 — 引擎" width="100%"/>
<br/><b>设置 → 引擎</b><br/>
<sub>引擎兼容性矩阵——14 个 TTS 引擎,逐引擎 GPU 预检,绝不静默回退到 CPU。</sub>
</td>
<td align="center">
<img src="docs/screenshot-settings.png" alt="设置 — 模型" width="100%"/>
<br/><b>设置 → 模型</b><br/>
<sub>一键模型商店——自动检测你的平台(CUDA / MPS / CPU)并推荐合适的模型。</sub>
</td>
</tr>
<tr><td align="center">声音设计</td><td align="center">本地模型</td></tr>
</table>
## 开始使用
---
从 [Releases](https://github.com/debpalash/VoiceStudio/releases/latest) 下载,然后阅读对应平台的安装指南:
<a id="features"></a>
**[macOS](docs/install/macos.md) · [Windows](docs/install/windows.md) · [Linux](docs/install/linux.md) · [Docker](docs/install/docker.md)**
## ✨ 功能
打开声音克隆页面,选择已有声音或添加清晰的参考录音,输入文字并生成。按提示安装所需模型。硬件要求因引擎而异,详见[性能指南](docs/performance.md)
八大主打功能——折叠区里还有十二项等你展开
**从源码运行 Electron 预览版:**
<table>
<tr>
<td align="center" width="25%">
<h3>🎙️ 语音克隆</h3>
<p>3 秒音频 → 复刻任何声音。<br/><b>646 种语言</b>,零样本。</p>
</td>
<td align="center" width="25%">
<h3>🎨 声音设计</h3>
<p>性别、年龄、口音、音高、语速、<br/>情感、方言——<b>随心调节</b>。</p>
</td>
<td align="center" width="25%">
<h3>🎬 视频配音</h3>
<p>YouTube 链接或文件 → 转录 →<br/>翻译 → 重新配音 → <b>MP4</b>。</p>
</td>
<td align="center" width="25%">
<h3>📖 有声书编辑器</h3>
<p>导入文本、EPUB 或 PDF。自动分章、<br/>响度归一、元数据。导出 <b>.m4b</b>。</p>
</td>
</tr>
<tr>
<td align="center" valign="top">
<h3>🎭 故事模式</h3>
<p>多声音编辑器。逐行分配声音、<br/>预览、<b>导出完整配音阵容</b>。</p>
</td>
<td align="center" valign="top">
<h3>⌨️ 听写工具</h3>
<p>在<b>任何应用</b>中按 <kbd>⌘</kbd>+<kbd>⇧</kbd>+<kbd>Space</kbd>。<br/>转录、自动粘贴、随即消失。</p>
</td>
<td align="center" valign="top">
<h3>🔐 100% 本地</h3>
<p>无需密钥、无需云端、无需账号。<br/><b>只在你的设备上</b>。</p>
</td>
<td align="center" valign="top">
<h3>🤖 MCP 服务器</h3>
<p>从 <b>Claude</b>、Cursor 或<br/>任何 MCP 客户端使用 OmniVoice。</p>
</td>
</tr>
</table>
```bash
git clone https://github.com/debpalash/VoiceStudio.git
cd VoiceStudio
bun install
cd electron
bun run dev
<details>
<summary><b>……还有 12 项</b>——人声分离、说话人分离、批量处理、水印、诊断等等</summary>
<br/>
- 🔊 **人声分离** — 基于 Demucs:把语音从音乐中分离出来,同时保留背景音床。
- 👥 **说话人分离** — Pyannote + WhisperX 自动识别谁说了什么。
- 📦 **批量队列** — 拖入 50 个视频就可以走开;每个任务都有独立进度条。
- 🛡️ **AI 水印** — AudioSeal(Meta):不可见,且能在压缩后留存。
- 🔬 **诊断** — 自检套件、错误日志、脱敏诊断包。
- ⚡ **GPU 自动检测** — CUDA · MPS · ROCmLinux,需手动开启)· CPU;显存 ≤8 GB 时自动卸载。
- 🧭 **引擎路由** — 逐引擎 GPU 预检;绝不静默回退到 CPU。
- 🧩 **可扩展** — 继承 `TTSBackend`,约 50 行代码即可接入任意引擎。
- 🎒 **便携声音角色** — 将声音导出为 `.ovsvoice` 包:身份 + 水印。
- ♾️ **无限长 TTS** — 按句分块生成,没有长度上限,可经 WebSocket 流式输出。
- 🌐 **远程后端** — 让 UI 指向远程服务器;对 Tailscale 友好,支持 Bearer 认证。
- 🧠 **听写 + LLM** — 用本地 LLM 润色转录文本,可选回声消除。
</details>
---
<a id="quickstart"></a>
## ⚡ 快速开始
<div align="center">
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/macOS-DMG_(Apple_Silicon)-000?style=for-the-badge&logo=apple&logoColor=white" alt="下载 macOS DMG" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Windows-MSI_(x64)-0078D4?style=for-the-badge&logo=windows&logoColor=white" alt="下载 Windows MSI" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Linux-AppImage_(x64)-FCC624?style=for-the-badge&logo=linux&logoColor=black" alt="下载 Linux AppImage" /></a>
<br/>
<sub><b>macOS</b>首次启动需要一次性批准——右键点击 → <b>打开</b>macOS 15 上为 系统设置 → 隐私与安全性 → <b>“仍要打开”</b>)。无需终端。<a href="docs/install/macos.md#gatekeeper-quarantine">为什么?</a> · <b>Intel Mac</b>不支持本地后端(<a href="https://github.com/debpalash/OmniVoice-Studio/issues/889">#889</a>)——<a href="docs/install/macos.md">详情</a>。</sub>
</div>
选择你的操作系统,按指南从头到尾操作:
- 🍎 **macOS** — [docs/install/macos.md](docs/install/macos.md)
- 🪟 **Windows** — [docs/install/windows.md](docs/install/windows.md)
- 🐧 **Linux** — [docs/install/linux.md](docs/install/linux.md)
- 🐳 **Docker** — [docs/install/docker.md](docs/install/docker.md) · [Docker Hub: `palashdeb/omnivoice-studio`](https://hub.docker.com/r/palashdeb/omnivoice-studio)
觉得慢?[docs/performance.md](docs/performance.md) 讲清了生成时间到底花在哪里、有哪些调优开关,以及“它变慢了”的三个经典原因。
> 正在从 **[CorentinJ/Real-Time-Voice-Cloning](https://github.com/CorentinJ/Real-Time-Voice-Cloning)**(现已归档)迁移过来?我们有专门的迁移指南:[docs/migration/real-time-voice-cloning.md](docs/migration/real-time-voice-cloning.md)。
<details>
<summary><b>🧰 卡住了?自检、Token 与受限网络</b></summary>
<br/>
先运行内置自检——在应用中打开 **设置 → 关于 → “运行自检”**,或在源码检出目录中执行
`uv run python backend/main.py --diagnose`(加 `--deep` 还会实际加载当前引擎进行测试)。然后查看
[docs/install/troubleshooting.md](docs/install/troubleshooting.md) 中排名前
10 的安装错误。运行时出错时,应用内的错误界面会直接深链到对应条目;**设置 → 关于 →
“保存诊断包”** 会把脱敏日志与自检报告打包,方便附在 Bug 报告里。
Hugging Face Token 的配置见
[docs/setup/huggingface-token.md](docs/setup/huggingface-token.md)。说话人分离相关的模型访问门槛见
[docs/features/diarization.md](docs/features/diarization.md)。下载速度、⚡ 快速下载(Xet)状态,以及受限网络 / 镜像选项见
[docs/downloading-models.md](docs/downloading-models.md)。
</details>
---
<a id="why-ovs"></a>
## 💡 为什么选择 OmniVoice
ElevenLabs 收费 **$5–$330/月**,并在他们的服务器上处理你的音频。OmniVoice Studio **在你的硬件上运行,没有任何用量限制。**
| | **ElevenLabs** | **OmniVoice Studio** |
|---|---|---|
| **价格** | $5–$330/月,按字符计费 | 免费且开源(AGPL-3.0)· 专有用途可选 [商业许可证](#license) |
| **语音克隆** | ✅ 3 秒音频 | ✅ 3 秒音频,零样本 |
| **声音设计** | ✅ 性别、年龄 | ✅ 性别、年龄、口音、音高、风格、方言 |
| **有声书 / 故事** | ❌ | ✅ 完整有声书编辑器 + 多声音故事(EPUB/PDF 导入,.m4b 导出) |
| **语言** | 32 | **646** |
| **视频配音** | ✅ 仅云端 | ✅ 完全本地 |
| **数据隐私** | 音频发送到云端 | **数据不离开你的设备** |
| **API 密钥** | 需要 | 不需要 |
| **GPU 支持** | 不适用(云端) | CUDA · Apple Silicon · ROCmLinux)· CPU |
| **桌面应用** | ❌ | ✅ macOS · Windows · Linux |
| **TTS 引擎** | 1 | **14** — [完整矩阵](#tts-engines) |
| **ASR 引擎** | 1 | **10** — [完整阵容](#asr-engines) |
| **MCP 服务器** | ❌ | ✅ 可从 Claude、Cursor 及任何 MCP 客户端使用 |
| **自检** | ❌ | ✅ 诊断套件、错误日志、脱敏调试包 |
| **可定制** | ❌ 闭源 | ✅ 随你 Fork、扩展、发布 |
专业级语音 AI,去掉订阅,也去掉云端。
<div align="center">
<br/>
<b>心动了?来和我们一起构建吧。</b><br/>
<a href="https://discord.gg/bzQavDfVV9"><img src="https://img.shields.io/badge/Join_Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" alt="加入 Discord" /></a>
<br/><br/>
</div>
---
## 🖥️ 系统要求
| | **最低配置** | **推荐配置** |
|---|---|---|
| **操作系统** | Windows 10、macOS 12+Apple Silicon)、Ubuntu 24.04+glibc 2.39+ | 任意现代 64 位操作系统 |
| **内存** | 8 GB | 16 GB+ |
| **显存(GPU** | 4 GB(自动将 TTS 卸载到 CPU | 8 GB+NVIDIA RTX 3060+ |
| **硬盘** | 10 GB 可用空间(模型 + 缓存) | 20 GB+ SSD |
| **Python** | 3.10+(由 `uv` 管理) | 3.113.12 |
| **GPU** | 可选——CPU 也能跑 | NVIDIA CUDA · Apple Silicon MPS · AMD ROCm(仅 Linux |
> [!TIP]
> 对于显存 **≤8 GB** 的 GPUOmniVoice 会在转录期间自动将 TTS 卸载到 CPU——无需配置。不需要专用 GPU;整条流水线都可以在 CPU 上运行(只是慢一些)。
> [!NOTE]
> **AMD GPU** ROCm 加速**仅限 Linux 且需手动开启**——在首次运行的设置界面选择 **“AMD GPU (ROCm)”**,或设置 `OMNIVOICE_TORCH_VARIANT=rocm`[docs/install/linux.md](docs/install/linux.md#amd-gpu-rocm))。在 **Docker/Podman** 中请改用专门的 ROCm 镜像:`ghcr.io/debpalash/omnivoice-studio:rocm`[docs/install/docker.md](docs/install/docker.md#pull-and-run-amd-gpu--rocm))。**在 Windows 上,AMD GPU(含 Ryzen AI 核显)只能以 CPU 运行**PyTorch 没有 Windows 版 ROCm 轮子,因此 Windows 上的 GPU 加速仅限 NVIDIA/CUDA[docs/install/windows.md](docs/install/windows.md#gpu-support))。
> [!IMPORTANT]
> **macOS Intelx86_64)不支持本地后端:** 应用 UI 可以安装,但 Python 后端无法运行,因为 PyTorch 已不再发布 Intel Mac 轮子([#889](https://github.com/debpalash/OmniVoice-Studio/issues/889))。Intel Mac 用户仍可让 UI 指向另一台机器上的远程后端——参见 [docs/install/macos.md](docs/install/macos.md)。
<a id="tts-engines"></a>
### 🗣️ TTS 引擎
**14 个引擎,一个选择器。** OmniVoice(默认,支持 600+ 语言)始终可用;另有七个引擎可选装并自动检测(CosyVoice 3、GPT-SoVITS、VoxCPM2、MOSS-TTS-Nano、KittenTTS、MLX-Audio、Sherpa-ONNX),外加六个按需延迟安装的重量级引擎(IndexTTS 2、OmniVoice GGUF、Supertonic 3、MOSS-TTS-v1.5、dots.tts、Confucius4-TTS)。在 **设置 → TTS 引擎** 中切换;所选引擎将应用于所有语音合成场景。
<details>
<summary><b>📊 完整矩阵</b>——14 个引擎 × 平台 × 克隆/指令 × 许可证</summary>
<br/>
| 引擎 | 语言 | 克隆 | 指令 | Linux | macOS ARM | Windows | 许可证 |
|--------|:---------:|:-----:|:--------:|:-----:|:---------:|:-------:|:-------:|
| **OmniVoice**(默认) | 600+ | ✅ | ✅ | ✅ CUDA/CPU | ✅ MPS | ✅ CUDA/CPU | 内置 |
| **CosyVoice 3** | 9 + 18 种方言 | ✅ | ✅ | ✅ CUDA/CPU | ✅ MPS | ✅ CUDA/CPU | Apache-2.0 |
| **GPT-SoVITS** | 5 | ✅ | — | ✅ CUDA/CPU | — | ✅ CUDA/CPU | MIT |
| **VoxCPM2** | 30 | ✅ | ✅ | ✅ CUDA/CPU | ✅ MPS | ✅ CUDA/CPU | Apache-2.0 |
| **MOSS-TTS-Nano** | 20 | ✅ | — | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
| **KittenTTS** | 英语 | — | — | ✅ CPU | ✅ CPU | ✅ CPU | MIT |
| **MLX-Audio**Kokoro、Qwen3-TTS、CSM、Dia 等) | 多语言 | 因模型而异 | 因模型而异 | ❌ | ✅ 原生 | ❌ | 因模型而异 |
| **Sherpa-ONNX** | 20+ | — | — | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
| **IndexTTS 2** ⚡ | 多语言 | ✅ | — | ✅ CUDA | — | ✅ CUDA | Apache-2.0 |
| **OmniVoice GGUF** ⚡ | 600+ | ✅ | ✅ | ✅ CPU | ✅ CPU | ✅ CPU | 内置 |
| **Supertonic 3** ⚡ | 31 | — | — | ✅ CPU | ✅ CPU | ✅ CPU | OpenRAIL-M |
| **MOSS-TTS-v1.5** ⚡(8B | 31 | ✅ | — | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
| **dots.tts** ⚡(2B | 24 | ✅ | — | ✅ CUDA/CPU | ✅ CPU | ❌ | Apache-2.0 |
| **Confucius4-TTS** ⚡ | 14 | ✅ | — | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
> **CUDA** = GPU 加速 · **MPS** = Apple Silicon Metal · **CPU** = 随处可运行,大模型较慢 · KittenTTS 和 MOSS-TTS-Nano 可在 CPU 上实时运行 · MLX-Audio 仅限 Apple Silicon · ⚡ = 延迟注册(首次使用时安装)
>
> **克隆**能力的意义不止于单段生成:视频配音(以及任何固定了声音的批量任务)需要参考音频克隆来保持说话人身份,因此把不支持克隆的引擎(KittenTTS、Sherpa-ONNX、Supertonic 3)设为当前引擎时,这些任务会在开始前就给出可操作的失败提示,而不是静默回退到 OmniVoice。
>
> **MOSS-TTS-v1.5**8B,约 16 GB)、**dots.tts**2B,约 9 GB)和 **Confucius4-TTS** 是重量级可选引擎,从本地克隆在各自独立的 venv 中运行。三者均不支持 Apple Silicon MPS(在 Mac 上以 CPU 运行);dots.tts 没有 Windows 路径;Confucius4 建议使用 CUDACPU 可用,约为实时时长的 17 倍)。详情:[MOSS-TTS-v1.5](docs/engines/moss-tts-v15.md) · [dots.tts](docs/engines/dots-tts.md) · [Confucius4-TTS](docs/engines/confucius4-tts.md)。
</details>
<a id="asr-engines"></a>
### 🎧 ASR 引擎
**10 个引擎**——它们驱动听写、视频配音和字幕。**WhisperX** 是跨平台的默认引擎(约 100 种语言,词级时间对齐);其余引擎均为可选装并自动检测。在 **设置 → 引擎** 中切换。九个完全在本地设备上运行;第十个(OpenAI 兼容)是可选的远程客户端,可用于 Qwen3-ASR 或任何兼容的服务器。
<details>
<summary><b>📊 完整阵容</b>——10 个引擎、各自的强项与计算类型说明</summary>
<br/>
| 引擎 | `OMNIVOICE_ASR_BACKEND` | 语言 | 最适合 |
|--------|-------------------------|:---------:|----------|
| **WhisperX**(默认) | `whisperx` | ~100 | 配音与字幕——通过 wav2vec2 强制对齐实现词级时间对齐 |
| **Faster-Whisper** | `faster-whisper` | ~100 | Linux / macOS / Windows 上的快速转录(CTranslate2 |
| **Faster-Whisper(隔离)** | `faster-whisper-isolated` | ~100 | 与 Faster-Whisper 相同,但在子进程中崩溃隔离——ASR 崩溃不会拖垮整个应用 |
| **MLX Whisper** | `mlx-whisper` | ~100 | Apple Silicon 原生速度(Apple MLX / Metal |
| **PyTorch Whisper** | `pytorch-whisper` | ~100 | 经 🤗 Transformers 的 CUDA / CPU 兜底方案(无需 cuDNN 8 |
| **Parakeet TDT** | `nemo-parakeet` | 英语 + 25 种欧洲语言 | 即使在 CPU 上也能以约 10 倍实时速度达到 SOTA 精度,自动语言检测(NVIDIA NeMoCUDA/CPU |
| **Moonshine** | `moonshine` | 英语 | 边缘设备 / 低延迟,ONNX |
| **FunASR** | `funasr` | 50+ | 多语言一体化——内置 VAD + 行内说话人分离(SenseVoice |
| **sherpa-onnx**(实时听写) | `sherpa-onnx-asr` | 25 种欧洲语言 + 90+ | 实时、快于实时的听写——小体积流式/离线 ONNX 模型(Parakeet TDT v3/v2、流式 Zipformer 与 Paraformer、Whisper Tiny),CPU 运行,macOS / Windows / Linux 表现完全一致。在 **设置 → 语音** 中按模型选择。 |
| **OpenAI 兼容** ⚠️ 远程 | `openai-compat-asr` | 取决于服务器 | 当下通往 **Qwen3-ASR** 的路径(自托管服务器,无需等 transformers 支持)、任何 OpenAI 兼容的转录端点,或 OpenAI 官方 API——无需安装,在 **设置 → 引擎**(ASR 标签页)中配置并测试连接。音频会离开你的设备,发送到你指定的任何服务器;参见 [docs/engines/openai-compatible-asr.md](docs/engines/openai-compatible-asr.md)。 |
> Whisper 系列引擎覆盖约 100 种语言;**FunASR / SenseVoice** 额外提供一条多语言一体化路径,内置语音活动检测与行内说话人分离。**sherpa-onnx** 驱动实时听写的模型选择器——你边说,文字边出现。每个引擎都在本地设备上运行——无需 API 密钥,无需云端。
> **GPU 不支持高效 float16** 在较老的 NVIDIA GPUMaxwell/Pascal、GTX 16xx)上,或在 CTranslate2/cuDNN 版本不匹配之后,CTranslate2 系 ASR 引擎(WhisperX、Faster-Whisper)无法运行 `float16`OmniVoice 会自动改用 `int8` 重试——无需配置。如果转录仍然失败,可用 `ASR_COMPUTE_TYPE` 环境变量固定计算类型(逃生舱口):`ASR_COMPUTE_TYPE=int8`CPU 用 `float32`)。将其设为 `int8` 并重启后端。
</details>
---
## 🏗️ 架构
```
┌─────────────────────────────────────────────────────────────┐
│ Frontend (React) │
│ DubTab · VoiceConsole · Stories · Audiobook · Gallery │
│ Dictation · BatchQueue · Diagnostics · MCP Client │
├─────────────────────────────────────────────────────────────┤
│ Backend (FastAPI) │
│ 100+ API endpoints · SSE+WSS streaming · SQLite │
├──────────┬──────────┬──────────┬──────────┬────────────────┤
│ WhisperX │ Demucs │OmniVoice │ Pyannote │ Engine Routing │
│ (+7 ASR │ Source │ (+10 │ Diariz- │ ↳ GPU preflight │
│ engines) │ Sep. │ TTS) │ ation │ ↳ No silent CPU │
└──────────┴──────────┴──────────┴──────────┴────────────────┘
CUDA / MPS / ROCm / CPU (auto-detected + routed)
```
环境要求和后端配置见 [Electron 开发指南](electron/README.md)。项目仍在积极开发中,可通过 [GitHub Issues](https://github.com/debpalash/VoiceStudio/issues) 反馈问题。
<a id="openai-api"></a>
## 文档
## 🔌 OpenAI 兼容 API
| 需求 | 链接 |
已经有会说 OpenAI 音频 API 的脚本、智能体或工具?把它指向 `http://localhost:3900/v1` 即可——不需要密钥,也不用改代码。后端为音频端点内置了即插即用的兼容接口,直接接到你当前启用的 TTS/ASR 引擎(没错,`voice` 参数接受你克隆的声音配置 ID)。
| 端点 | 作用 |
|---|---|
| 安装帮助 | [故障排查](docs/install/troubleshooting.md) · [模型下载](docs/downloading-models.md) |
| 模型与音质 | [引擎指南](docs/engines/README.md) · [基准测试](docs/benchmarks.md) |
| 集成 | [本地 API](docs/speech-platform.md) · [MCP](docs/mcp.md) · [示例](examples/README.md) |
| 参与开发 | [贡献指南](.github/CONTRIBUTING.md) · [Electron](electron/README.md) · [更新日志](CHANGELOG.md) |
| `POST /v1/audio/speech` | TTS——输入文本;输出 `mp3` / `wav` / `flac` / `opus` / `pcm``tts-1` / `tts-1-hd` 映射到你当前启用的引擎;也接受 OpenAI 的声音名称(`alloy` 等)。 |
| `POST /v1/audio/transcriptions` | STT——输入音频文件;输出 `json``text``verbose_json``srt``vtt``whisper-1` 映射到你当前启用的 ASR 引擎。 |
| `GET /v1/audio/voices` | OmniVoice 扩展——列出所有声音配置和引擎,客户端可据此发现你的克隆声音。 |
安装智能体技能:`npx skills add debpalash/VoiceStudio`
```sh
curl http://localhost:3900/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{"model": "tts-1", "voice": "alloy", "input": "Generated on my own hardware.", "response_format": "wav"}' \
--output speech.wav
```
## 支持 VoiceStudio
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:3900/v1", api_key="none") # any string works — nothing checks it
[Ko-fi](https://ko-fi.com/debpalash) · [PayPal](https://paypal.me/palashCoder) · [赞助项目](SPONSORS.md) · [商务合作](mailto:partner@voicestudio.sh)
result = client.audio.transcriptions.create(model="whisper-1", file=open("clip.wav", "rb"))
print(result.text)
```
**让语音应用开发者看到你的品牌。** 了解应用底部栏、集成目录、文档和 README 的付费展示合作。[申请合作](https://forms.gle/2PYCvd39hbwijzX37)或[发送邮件](mailto:partner@voicestudio.sh)
想要完整的接口(100+ 端点)?完整的 REST API 参考已内嵌在应用中——**设置 → OpenAPI 参考**(由 Scalar 驱动),或点击页脚的 `{}` 按钮
## 许可与负责任使用
### 📓 在 Google Colab 上运行
应用采用 [AGPL-3.0](LICENSE) 许可。模型遵循各自的许可,商用前请确认其条款。克隆声音前须取得本人许可。详见[许可说明](LICENSE-NOTICE.md)。
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/debpalash/OmniVoice-Studio/blob/main/notebooks/OmniVoice_Studio_Colab.ipynb)
没有本地 GPU?官方笔记本([notebooks/OmniVoice_Studio_Colab.ipynb](notebooks/OmniVoice_Studio_Colab.ipynb))可在免费的 Colab T4 上启动完整应用(包含 Web 界面):在笔记本内直接构建前端,用 uv 安装后端(复用 Colab 预装的 CUDA PyTorch),并通过 Colab 内置端口代理打开界面。无需第三方隧道,也无需任何 API 密钥。随后还有一套覆盖全部主要功能的 API 导览,全部可在笔记本内直接播放:多语言 TTS、声音克隆与声音设计、已保存的声音档案、语音转写、AI 水印检测、OpenAI 兼容 API、多角色故事、带章节的 m4b 有声书,以及一个附带人声分离音轨的迷你视频配音。
### 🤝 智能体技能(Agent Skills
用一条命令教会你的 AI 智能体(Claude Code、Cursor、Codex 等)使用 OmniVoice
```sh
npx skills add debpalash/omnivoice-studio
```
内含两个 [skills](https://skills.sh)**`omnivoice`**——让任何智能体通过你的本地安装进行语音合成与转录(包括你克隆的声音),免费且离线;以及 **`oss-maintainer`**——本项目所遵循的维护者方法论,适合任何用智能体运营自己开源项目的人。
---
## 🗺️ 路线图
### 🔜 即将推出
- 🎬 **唇形同步 v2** — 使用 wav2lip 进行视觉语音时间对齐
- 🌐 **在线演示** — 无需安装即可体验 OmniVoice
- 🔌 **插件市场** — 社区贡献的 TTS 引擎与特效
- 🎵 **实时变声器** — 通话中的麦克风实时变声
<details>
<summary><b>✅ 已经发布的一切</b>——按类别列出的“成绩单”</summary>
<br/>
| 分类 | 功能 |
|----------|----------|
| **长内容** | 有声书编辑器(文本/EPUB/PDF → 分章 .m4b)、Stories 多声音编辑器、两遍响度归一母带处理、渲染中断后的崩溃续渲、发音控制 + SSML-lite 韵律 |
| **配音** | 完整流水线(转录→翻译→合成→封装)、场景感知分割、唇形同步评分、流式 TTS、逐说话人声音分配、Smart Fit 时长匹配 + 二次 QC、独立的配音主页 |
| **声音** | 零样本克隆、声音设计、A/B 对比、声音预览控件、支持收藏/标签的声音库、便携声音角色包(`.ovsvoice`)、声音控制台工作区 |
| **音频** | Demucs 人声分离、逐段增益、选择性音轨导出、分轨/SRT/VTT/MP3 导出、按句分块实现的无限长 TTS |
| **多语言** | 多语言批量选择器、顺序 GPU 执行的批量配音队列 |
| **说话人分离** | Pyannote 机器学习分离、自动说话人克隆提取、逐说话人声音分配 |
| **ASR** | 9 个引擎(WhisperX、Faster-Whisper、隔离版 Faster-Whisper、MLX Whisper、PyTorch Whisper、Parakeet TDT、Moonshine、FunASR/SenseVoice、sherpa-onnx 实时听写)、崩溃隔离的子进程后端 |
| **TTS** | 14 个引擎(OmniVoice、CosyVoice 3、GPT-SoVITS、VoxCPM2、MOSS-TTS-Nano、KittenTTS、MLX-Audio、Sherpa-ONNX+ 延迟安装:IndexTTS 2、OmniVoice GGUF、Supertonic 3、MOSS-TTS-v1.5、dots.tts、Confucius4-TTS)、带 GPU 预检的引擎路由 |
| **基础设施** | Docker 部署、CUDA/MPS/ROCm 自动检测、cuDNN 8 兼容、显存感知模型卸载、引擎路由(绝不静默回退 CPU)、诊断套件与错误日志、受限网络镜像支持 |
| **AI 溯源** | AudioSeal 不可见水印(类似 SynthID)、视频徽标叠加、水印检测 API |
| **用户体验** | 撤销/重做、键盘快捷键、拖放、会话持久化、毛玻璃设计系统、Linux/WebKitGTK 的 UI 缩放修复 |
| **实时事件** | WebSocket 事件总线——数据变更时即时刷新侧边栏、指数退避重连 |
| **状态管理** | Zustand 状态迁移——`uiSlice``pillSlice``dubSlice``generateSlice``prefsSlice``glossarySlice` |
| **桌面** | 跨平台 Tauri 安装程序(macOS DMG——Apple SiliconIntel 不支持本地后端,#889——Windows MSI、Linux deb/AppImage)、自动更新基础设施、单实例约束、关闭最小化到托盘、macOS Gatekeeper 修复 |
| **听写** | 全局系统级热键(`⌘+⇧+Space`)、无边框浮动控件、WebSocket 流式 ASR、自动粘贴、可自定义热键、本地 LLM 转录润色 |
| **批量流水线** | 完整批量 TTS:提取 → 转录 → 翻译 → 生成 → 混音 → 导出,带实时进度追踪 |
| **MCP 服务器** | 让 OmniVoice 成为 Claude、Cursor 及任何 MCP 客户端的本地 TTS/STT 提供方 |
| **远程后端** | 让桌面 UI 指向远程后端 URL,支持 Bearer 认证(附 Tailscale 文档) |
| **可靠性** | 启动开屏的卡死看门狗、逐引擎 GPU 兼容矩阵、引擎二进制不可执行时的可操作报错、setuptools 自动修复 |
</details>
---
<a id="sponsor--donate"></a>
## 💜 赞助 / 捐赠
OmniVoice Studio 由一位开发者使用 Claude Code 和 AI 智能体独立打造——而智能体账单是实打实的(过去三个月花了数千美元)。如果 OmniVoice 为你创造了价值,帮忙分担一小部分账单,就能让开发保持全职推进。
<div align="center">
**本月智能体账单基金**
<img src="https://img.shields.io/badge/raised_%2410_of_%24200-5%25-EAB308?style=for-the-badge" alt="已筹 $10 / $200" />
<br/><br/>
<a href="https://ko-fi.com/debpalash"><img src="https://img.shields.io/badge/Ko--fi-Support_❤️-FF5E5B?style=for-the-badge&logo=ko-fi&logoColor=white" alt="Ko-fi" /></a>
&nbsp;&nbsp;
<a href="https://paypal.me/palashCoder"><img src="https://img.shields.io/badge/PayPal-Donate-00457C?style=for-the-badge&logo=paypal&logoColor=white" alt="PayPal" /></a>
<br/>
<sub>每一美元都直接用于支付智能体账单——让 OmniVoice 的开发持续不断。</sub>
<br/><br/>
<sub><b>来自 OmniVoice Studio 作者的更多应用</b>——同样的本地优先理念:
<a href="https://github.com/debpalash/Opal"><b>Opal</b> 💠</a>(播放一切——AI 时代的媒体播放器)·
<a href="https://github.com/debpalash/memxt"><b>memxt</b> 🧠</a>Claude Code 与编码智能体的本地记忆)。
给它们点个 ⭐ 也是一种支持 → <a href="#more-from-the-maker">详见下文</a>。</sub>
</div>
<a id="sponsors"></a>
### 🌟 赞助商
OmniVoice **免费**且采用 **AGPL-3.0** 许可——没有付费版,没有 SaaS 收入。赞助商让开发得以持续,作为回报,可以在这里、在应用内(顶级档位还包括项目官网)获得一个徽标位。这是一份感谢,绝不是付费墙。**[查看档位并成为赞助商 →](SPONSORS.md)**
<div align="center">
<!-- SPONSORS:START — logo slots are filled here as sponsors come aboard; see SPONSORS.md -->
**这里可以是你的徽标** — [成为赞助商](SPONSORS.md)
<!-- SPONSORS:END -->
</div>
<sub>💡 GitHub 也会在本仓库顶部显示一个 **Sponsor** 按钮,经由 <a href=".github/FUNDING.yml"><code>.github/FUNDING.yml</code></a> 指向相同的链接。</sub>
---
## 💬 社区
<div align="center">
<a href="https://discord.gg/bzQavDfVV9"><img src="https://img.shields.io/badge/💬_Discord-Join_Community-5865F2?style=for-the-badge&logo=discord&logoColor=white" alt="加入 Discord" /></a>
<br/>
<sub>设置类问题我们几小时内就会回复,而不是几天。</sub>
</div>
<details>
<summary><b>里面都在聊什么</b></summary>
<br/>
| 频道 | 那里发生什么 |
|---------|--------------------|
| `#announcements` | 发布消息与重大时刻——新版本最先在这里公布 |
| `#releases` + `#changelog` | 每一个构建,以及里面究竟有什么 |
| `#issues` | 以论坛帖子形式提交的 Bug 报告——直接分诊进 GitHub Issues |
| `#ideas` | 功能请求,供讨论与投票 |
| `#discuss-ideas` | 动手之前的设计讨论 |
| `#general` | 安装帮助、GPU 疑难排查,以及晒你的配音成果 |
</details>
---
<a id="contributing"></a>
## 🤝 参与贡献
非常欢迎——Bug 修复、新的 TTS 引擎适配器、UI 改进、文档、翻译。统统欢迎。
- 📖 阅读 **[贡献指南](.github/CONTRIBUTING.md)** 了解环境搭建、代码风格和 PR 工作流
- 🐛 浏览 [good first issues](https://github.com/debpalash/OmniVoice-Studio/labels/good%20first%20issue)
- 💬 加入我们的 [Discord](https://discord.gg/bzQavDfVV9) 讨论想法或寻求帮助
---
## ❓ 常见问题
<details>
<summary><b>真的能和 ElevenLabs 一样好吗?</b></summary>
<br/>
诚实的回答:<b>取决于你要做什么。</b>
<b>OmniVoice 真正有竞争力的地方:</b>从干净的参考音频进行语音克隆(最先进的开源扩散 TTS)、语言覆盖(646 种语言对他们的 32 种),以及所有结构性优势——没有按字符计费、没有用量上限、音频不离开你的设备、完整的流水线可定制性(14 个 TTS 引擎、10 个 ASR 引擎、翻译方案随你选)。
<b>ElevenLabs 仍然领先的地方:</b>开箱即用的稳定性与打磨程度,尤其是英语 TTS。他们的单一模型经过深度调优;我们的质量取决于你选择的引擎、你的硬件,以及(对克隆而言)参考音频——干燥、近麦的音频比嘈杂或有回声的音频克隆效果好得多。
<b>具体到配音:</b>配音是一条链——转录 → 翻译 → 克隆 → 合成——在<i>你的</i>素材上,它只取决于最薄弱的一环。如果部分输出语无伦次,先检查片段表里的<i>原文</i>:当转录本身就错了,换一个 ASR 引擎或使用更干净的源音频——修复点通常在这里,而不是声音。
拿你的真实素材试试——免费,下载一次即可。许多用户直接用它替换了 ElevenLabs;也有人两个都留着。这两种结果我们都乐见。
</details>
<details>
<summary><b>能在 Apple SiliconM1/M2/M3/M4)上运行吗?</b></summary>
<br/>
可以。MPS 加速会被自动检测。在 Apple 硬件上,MLX 优化的 Whisper 模型可提供更快的转录速度。<b>不支持 Intel Mac</b>:应用 UI 可以安装,但本地 Python 后端无法运行,因为 PyTorch 已不再发布 Intel Mac 轮子(<a href="https://github.com/debpalash/OmniVoice-Studio/issues/889">#889</a>)——Intel Mac 只能配合远程后端使用。
</details>
<details>
<summary><b>需要多少显存?</b></summary>
<br/>
<b>最低 4 GB。</b> 显存 ≤8 GB 时,TTS 模型会在转录期间自动卸载到 CPU。8 GB 以上时,所有组件同时在 GPU 上运行。完全没有 GPU?CPU 模式也能用——只是慢一些(TTS 约慢 3 倍)。
</details>
<details>
<summary><b>可以用于商业用途吗?</b></summary>
<br/>
<b>可以——商业使用免费</b>,基于 <a href="https://www.gnu.org/licenses/agpl-3.0.html">AGPL-3.0</a>:运行它、出售用它生成的音频、为客户的视频配音、在团队中部署。只有一项义务:如果你<b>修改</b>了 OmniVoice 并通过网络向他人提供该修改版本,你必须依据相同条款分享修改后的源代码。想把它嵌入闭源产品?可获取商业许可证——参见<a href="#license">许可证</a>。
</details>
<details>
<summary><b>支持哪些语言?</b></summary>
<br/>
通过 OmniVoice 模型的 TTS 支持 646 种语言。转录(WhisperX)支持 99 种语言。翻译覆盖范围取决于目标语言对。
</details>
<details>
<summary><b>可以添加自己的 TTS 引擎吗?</b></summary>
<br/>
可以。在 <code>backend/services/tts_backend.py</code> 中继承 <code>TTSBackend</code>,并将其添加到 <code>_REGISTRY</code> 字典中——约 50 行代码。十四个内置引擎均以此方式实现;参见 <a href="#tts-engines">TTS 引擎</a>。
</details>
<details>
<summary><b>OmniVoice 会收集我的任何数据吗?</b></summary>
<br/>
<b>除非你明确同意,否则不会。</b>首次运行时应用会<i>询问</i>你——一个页面、两个同等分量的按钮,没有预先勾选。在你回答“是”之前,OmniVoice 什么都不发送:没有分析、没有遥测、没有账号、没有“回传”。跳过提问就等于“否”。无论如何,你的文本、音频、声音和项目永远不会离开你的设备。
如果你选择同意(也可随时在 <b>设置 → 隐私 → “帮助改进 OmniVoice”</b> 中开关),发送的只是匿名、不含内容的使用统计:生成信息(引擎、语言、生成耗时、字符<i>数量</i>、错误<i>类型</i>),以及应用生命周期——一次安装信号、版本更新(版本号之间)、崩溃(错误类别和<i>分桶后的</i>运行时长,绝不含日志)、错误<i>类型</i>(有上限、去重),以及卸载时的一次告别信号。绝不包含你的文本、音频、文件名或任何可识别信息——这由代码中的属性白名单强制保证(<code>backend/core/analytics.py</code>),而不只是一句承诺。源码构建根本没有分析数据的接收端,因此根本不会询问。你自己的统计数字在 <b>设置 → 用量</b> 中查看,本地计算,不发送到任何地方。
</details>
<details>
<summary><b>如何卸载它 / 删除它的所有数据?</b></summary>
<br/>
OmniVoice 完全本地运行——卸载就是删除应用及其写入的文件夹(模型缓存、Python 环境、你的声音/项目、配置)。运行 <code>scripts/uninstall.sh</code>macOS/Linux)或 <code>scripts\uninstall.ps1</code>Windows)——它会先以干跑方式列出每个文件夹及其大小,加 <code>--yes</code> 才会真正删除。完整的各平台路径列表和应用移除步骤见 <a href="docs/install/uninstall.md"><b>docs/install/uninstall.md</b></a>。
</details>
---
<a id="license"></a>
## 📜 许可证
OmniVoice Studio 是基于 [**GNU Affero 通用公共许可证 v3.0AGPL-3.0**](https://www.gnu.org/licenses/agpl-3.0.html) 的自由开源软件。
**可免费用于任何用途——包括商业和企业内部用途。** 运行它、出售用它生成的音频、为自己或客户的视频配音、在团队中推广——全部免费,无需许可证。作为一份**网络著佐权(copyleft)**许可证,AGPL 增加了一项义务:如果你**修改**了 OmniVoice Studio 并通过网络向他人提供该修改版本,你必须依据相同的 AGPL-3.0 条款向他们提供该修改版本的完整对应源代码。
希望将 OmniVoice Studio 嵌入**闭源或专有**产品或服务、又不受 AGPL-3.0 著佐权义务约束的组织,可获取**商业许可证**。**定价方案即将推出。** 咨询:**OmniVoice@palash.dev**。
捆绑的 `omnivoice/` TTS 模型(作者 Han Zhu)在上游仍为 Apache-2.0 许可。完整且具约束力的条款请参见 [`LICENSE`](LICENSE)。
---
## 🙏 致谢
OmniVoice Studio 站在这些杰出开源工作的肩膀上:
| 项目 | 作用 |
|---------|------|
| [**OmniVoice (k2-fsa)**](https://github.com/k2-fsa/OmniVoice) | 零样本扩散 TTS 引擎——核心语音合成模型 |
| [**WhisperX**](https://github.com/m-bain/whisperX) | 词级别语音识别与时间对齐 |
| [**Demucs (Meta)**](https://github.com/facebookresearch/demucs) | 音乐源分离,用于人声分离 |
| [**Pyannote**](https://github.com/pyannote/pyannote-audio) | 说话人分离——谁说了什么 |
| [**CTranslate2**](https://github.com/OpenNMT/CTranslate2) | CPU 和 GPU 上的优化 Transformer 推理 |
| [**AudioSeal (Meta)**](https://github.com/facebookresearch/audioseal) | 用于 AI 溯源的不可见神经音频水印 |
| [**Tauri**](https://tauri.app) | 原生桌面应用框架 |
| [**Supertone / Supertonic 3**](https://huggingface.co/Supertone/supertonic-3) | ONNX TTS 引擎——31 种语言,CPU 高效 |
| [**Sherpa-ONNX**](https://github.com/k2-fsa/sherpa-onnx) | 支持 WASM 的通用 TTS/ASR 运行时 |
| [**GPT-SoVITS**](https://github.com/RVC-Boss/GPT-SoVITS) | 零样本 TTS 引擎——5 种语言,RTF 0.014 |
---
<a id="more-from-the-maker"></a>
## 🧰 来自同一作者的更多本地开源项目
喜欢这种本地优先的理念?它是一脉相承的——同一位作者,同一条准则:**你的数据只留在你的设备上。**
<table>
<tr>
<td align="center" width="50%" valign="top">
<br/>
<a href="https://github.com/debpalash/Opal"><img src="https://raw.githubusercontent.com/debpalash/Opal/main/assets/opal_logo.png" width="96" alt="Opal 徽标"/></a>
<h3><a href="https://github.com/debpalash/Opal">Opal 💠</a></h3>
<p><b>播放一切。</b>AI 时代的媒体播放器。</p>
<p><sub>视频、动漫、漫画、种子、Jellyfin 和 Plex——一个播放器全部搞定,并内置本地 AI 记忆与上下文。使用 Zig 编写,支持 macOS 和 Windows。</sub></p>
<p>
<a href="https://github.com/debpalash/Opal/stargazers"><img src="https://img.shields.io/github/stars/debpalash/Opal?style=flat-square&color=f59e0b" alt="Opal Star 数"/></a>
<a href="https://palash.dev/opal"><img src="https://img.shields.io/badge/site-palash.dev%2Fopal-8b5cf6?style=flat-square" alt="Opal 官网"/></a>
</p>
</td>
<td align="center" width="50%" valign="top">
<br/>
<a href="https://github.com/debpalash/memxt"><img src="https://raw.githubusercontent.com/debpalash/memxt/main/assets/logo-mark.svg" width="96" alt="memxt 徽标"/></a>
<h3><a href="https://github.com/debpalash/memxt">memxt 🧠</a></h3>
<p><b>经基准测试验证的最快开源 AI 记忆系统。</b></p>
<p><sub>为 Claude Code 和编码智能体提供本地长期记忆——基于 SQLite + 嵌入向量的 MCP 服务器,100% 在你的设备上运行。你的智能体终于能记住昨天了。</sub></p>
<p>
<a href="https://github.com/debpalash/memxt/stargazers"><img src="https://img.shields.io/github/stars/debpalash/memxt?style=flat-square&color=f59e0b" alt="memxt Star 数"/></a>
<a href="https://github.com/debpalash/memxt#readme"><img src="https://img.shields.io/badge/docs-README-10b981?style=flat-square" alt="memxt 文档"/></a>
</p>
</td>
</tr>
</table>
---
<div align="center">
<br/>
如果你读到了这里,你就是我们的同路人。<br/>
**[⭐ 给这个仓库点个 Star](https://github.com/debpalash/OmniVoice-Studio)**,让更多人能找到它。<br/>
**[💬 加入 Discord](https://discord.gg/bzQavDfVV9)**,分享你的作品。<br/>
**[❤️ 支持开发](https://ko-fi.com/debpalash)**——资助让 OmniVoice 持续发布的 AI 智能体账单。
<br/>
<a href="https://star-history.com/#debpalash/OmniVoice-Studio&Date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=debpalash/OmniVoice-Studio&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=debpalash/OmniVoice-Studio&type=Date" />
<img alt="Star 历史" src="https://api.star-history.com/svg?repos=debpalash/OmniVoice-Studio&type=Date&theme=dark" width="600" />
</picture>
</a>
</div>
+13 -13
View File
@@ -1,6 +1,6 @@
<div align="center">
<img src="docs/logo.png" alt="VoiceStudio Logo" width="96" height="96" />
<h1>Sponsor VoiceStudio</h1>
<img src="docs/logo.png" alt="OmniVoice Logo" width="96" />
<h1>Sponsor OmniVoice Studio</h1>
<p><b>Keep the open-source ElevenLabs alternative free, local, and shipping.</b></p>
</div>
@@ -8,15 +8,15 @@
## Why sponsor?
VoiceStudio is built by one developer, in the open, using Claude Code and AI agents — and the agent bills are real. Over the last few months I've spent thousands of dollars on Claude subscriptions to keep features shipping, bugs fixed, and your issues answered.
OmniVoice Studio is built by one developer, in the open, using Claude Code and AI agents — and the agent bills are real. Over the last few months I've spent thousands of dollars on Claude subscriptions to keep features shipping, bugs fixed, and your issues answered.
VoiceStudio is **free**, **fully local**, and **AGPL-3.0**. There's no paid tier, no accounts, no cloud, and no SaaS revenue — nothing runs on a server we bill you for, because nothing runs on a server at all. That's the whole point, and it's also why there's no recurring revenue to fund development. Sponsorship is what makes continued full-time work possible.
OmniVoice is **free**, **fully local**, and **AGPL-3.0**. There's no paid tier, no accounts, no cloud, and no SaaS revenue — nothing runs on a server we bill you for, because nothing runs on a server at all. That's the whole point, and it's also why there's no recurring revenue to fund development. Sponsorship is what makes continued full-time work possible.
If VoiceStudio has created value for you or your company, sponsoring means the next release keeps coming — and you get a thank-you (and, at most tiers, a logo slot) in return.
If OmniVoice has created value for you or your company, sponsoring means the next release keeps coming — and you get a thank-you (and, at most tiers, a logo slot) in return.
### Where your money goes
Every dollar goes to the cost of building VoiceStudio — chiefly the **AI agent bills that keep it shipping** (Claude subscriptions and API usage), plus the occasional signing certificate, test hardware, and model-hosting costs. It is not a salary top-up; it's what keeps the lights on for continuous development.
Every dollar goes to the cost of building OmniVoice — chiefly the **AI agent bills that keep it shipping** (Claude subscriptions and API usage), plus the occasional signing certificate, test hardware, and model-hosting costs. It is not a salary top-up; it's what keeps the lights on for continuous development.
---
@@ -41,7 +41,7 @@ Placements marked "as that page ships" (the in-app Sponsors page and the project
**1. Open a sponsorship inquiry (recommended).** This opens a short GitHub form (name/org, logo, tier, contact) so we can get you set up:
> **[→ Open a sponsorship inquiry](https://github.com/debpalash/VoiceStudio/issues/new?template=sponsor.yml)**
> **[→ Open a sponsorship inquiry](https://github.com/debpalash/OmniVoice-Studio/issues/new?template=sponsor.yml)**
**2. Or start recurring support directly:**
@@ -70,16 +70,16 @@ To make your logo look sharp everywhere (README on GitHub, the in-app page, the
**How your logo gets added:**
- **Easiest:** attach the asset and link in your [sponsorship inquiry](https://github.com/debpalash/VoiceStudio/issues/new?template=sponsor.yml) — the maintainer places it.
- **Easiest:** attach the asset and link in your [sponsorship inquiry](https://github.com/debpalash/OmniVoice-Studio/issues/new?template=sponsor.yml) — the maintainer places it.
- **Or open a PR:** add your asset under `docs/sponsors/` and an entry to the tables in this file. Silver/Gold logos are also wired into the app's in-app Sponsors page (via the `sponsors.js` manifest) and the project website as those surfaces ship.
By sponsoring you confirm you have the right to use the submitted logo and grant VoiceStudio permission to display it in the contexts above. We won't alter your logo beyond scaling, and we'll remove it promptly on request.
By sponsoring you confirm you have the right to use the submitted logo and grant OmniVoice permission to display it in the contexts above. We won't alter your logo beyond scaling, and we'll remove it promptly on request.
---
## Current sponsors
VoiceStudio doesn't have any sponsors yet — **you could be the first.** These slots fill in as sponsors come aboard.
OmniVoice doesn't have any sponsors yet — **you could be the first.** These slots fill in as sponsors come aboard.
### 🥇 Gold
@@ -107,13 +107,13 @@ _Open — [become a Backer](#how-to-become-a-sponsor)._
Sponsorship is a **thank-you, never a paywall.**
Every feature of VoiceStudio is and will remain **free** and **open-source under [AGPL-3.0](LICENSE)**. Sponsors do **not** get private builds, gated features, license exceptions, or anything that degrades the experience for people who don't (or can't) pay. What sponsors get is **visibility and our gratitude** — and the knowledge that they're directly funding the next release.
Every feature of OmniVoice Studio is and will remain **free** and **open-source under [AGPL-3.0](LICENSE)**. Sponsors do **not** get private builds, gated features, license exceptions, or anything that degrades the experience for people who don't (or can't) pay. What sponsors get is **visibility and our gratitude** — and the knowledge that they're directly funding the next release.
VoiceStudio stays local-first and fully functional with zero dollars spent. Sponsoring just helps it keep getting better, faster.
OmniVoice stays local-first and fully functional with zero dollars spent. Sponsoring just helps it keep getting better, faster.
---
<div align="center">
<sub>Thank you for keeping local-first voice AI alive and free. ❤️</sub><br/>
<sub>Questions? <a href="https://github.com/debpalash/VoiceStudio/issues/new?template=sponsor.yml">Open an inquiry</a> · <a href="https://discord.gg/bzQavDfVV9">Discord</a></sub>
<sub>Questions? <a href="https://github.com/debpalash/OmniVoice-Studio/issues/new?template=sponsor.yml">Open an inquiry</a> · <a href="https://discord.gg/bzQavDfVV9">Discord</a></sub>
</div>
+8 -20
View File
@@ -1,29 +1,17 @@
# Alembic configuration for VoiceStudio.
# Run from anywhere: alembic -c <repo>/alembic.ini <command>
# Alembic configuration for OmniVoice Studio.
# Run from the repo root: alembic -c alembic.ini <command>
# Default commands:
# alembic upgrade head - apply all pending migrations
# alembic revision -m "..." - create a new migration
# alembic current - show current schema version
#
# Keep this file ASCII: alembic reads it in the locale code page, which on a
# Chinese, Japanese or Korean Windows cannot decode UTF-8 punctuation
# (tests/test_alembic_ini_locale.py).
# alembic upgrade head apply all pending migrations
# alembic revision -m "" create a new migration
# alembic current show current schema version
#
# DB URL is resolved dynamically from core.config (env aware).
# See backend/migrations/env.py.
[alembic]
# %(here)s = this file's directory. Alembic resolves bare relative paths
# against the process CWD, not the ini - and the app doesn't always start
# from the repo root (`tauri dev` runs the backend with
# cwd=frontend/src-tauri), which made startup migrations die with
# "Path doesn't exist: backend/migrations" the first time one was pending.
script_location = %(here)s/backend/migrations
prepend_sys_path = %(here)s/backend
# Split multi-path options on os.pathsep, not the legacy space/comma/colon
# set - a colon-split would shred "C:\..." absolute paths on Windows.
path_separator = os
# sqlalchemy.url is set programmatically in env.py - do NOT set it here.
script_location = backend/migrations
prepend_sys_path = backend
# sqlalchemy.url is set programmatically in env.py — do NOT set it here.
sqlalchemy.url =
[loggers]
+1 -11
View File
@@ -1,5 +1,5 @@
# -*- mode: python ; coding: utf-8 -*-
# PyInstaller spec for VoiceStudio backend.
# PyInstaller spec for OmniVoice Studio backend.
#
# Produces a one-folder bundle at dist/omnivoice-backend/ that Tauri launches
# as a sidecar binary. Kept intentionally permissive with collect_all(...)
@@ -41,16 +41,6 @@ hiddenimports = [
# even though pyproject.toml ships the package. Guarded by
# tests/test_socks_proxy.py.
'socksio',
# Remote GPU workers (backend/worker/). The feature is opt-in, so every
# import of it is deliberately deferred to the moment it is switched on —
# inside `lifespan` and inside `ControlPlane.start()`. That keeps the cost
# off users who never enable it, but it also means a frozen build has no
# static import chain to follow, so the modules must be named here or the
# feature raises ModuleNotFoundError only in the installers.
'grpc', 'grpc.aio',
'worker.service', 'worker.agent',
'worker.transport.server', 'worker.transport.client',
'worker.protocol.gen.worker_v1_pb2', 'worker.protocol.gen.worker_v1_pb2_grpc',
# Core
'uuid', 'asyncio',
+114 -170
View File
@@ -6,30 +6,70 @@ composed at the route or router level without surprises.
Currently exposed:
- `require_loopback`: 403 unless the request came from a loopback origin
(read-only bootstrap is allowed in explicit server mode; mutations still
require the admin API key see `_server_mode`).
- `require_admin`: method-aware admin gate for privileged routers.
- `require_admin_action`: strict admin gate for side-effectful GET actions.
- `require_native_access`: true-loopback-only access to the host filesystem;
unlike `require_loopback`, it is never bypassed by server mode.
(bypassed in explicit server mode see `_server_mode`).
- `ws_remote_authorized`: whether a WebSocket handshake from a non-loopback
client carries the remote API key (Wave 2.3) used by WS endpoints that
keep their own inline loopback guards.
"""
import ipaddress
import os
import secrets
from fastapi import HTTPException, Request
from core.auth import (
CredentialTransport,
PrincipalKind,
is_local_host,
is_loopback,
principal_for,
remote_api_key,
)
from core.csrf import SAFE_HTTP_METHODS, cookie_csrf_allowed
# IPv4 + IPv6 loopback literals + the conventional `localhost` hostname.
# `request.client.host` carries an address, not a hostname, so the literal
# "localhost" entry is defensive — some upstream wrappers (TestClient with
# a custom client tuple, certain reverse-proxy headers) may pass strings
# rather than parsed addresses. We accept the broader set without weakening
# the guard: nothing here matches a non-loopback origin.
_LOOPBACK_HOSTS = frozenset({"127.0.0.1", "::1", "localhost"})
def _trusted_networks():
"""CIDR networks from OMNIVOICE_TRUSTED_NETWORKS (comma-separated) treated as
loopback-trusted e.g. a reverse proxy or self-hosted LAN, so the API-key /
PIN gates don't block LAN clients that can't present the credential (a proxy
that strips the Authorization header). Read at call time (matching
`_server_mode` / `remote_api_key`) so tests can monkeypatch the env; restart
to apply changes in production."""
nets = []
for cidr in os.environ.get("OMNIVOICE_TRUSTED_NETWORKS", "").split(","):
cidr = cidr.strip()
if cidr:
try:
nets.append(ipaddress.ip_network(cidr, strict=False))
except ValueError:
pass # malformed entry ignored — never wedge the auth gate
return nets
def is_loopback(host):
"""True loopback address only (127.0.0.1, ::1, localhost) — NOT a trusted
network. Admin gates (``require_loopback`` ``/system/set-env``,
``/api/settings/*``) use this so a trusted-network CIDR exempts consumption
(TTS / dictation) but never the RCE-class admin surface."""
return host in _LOOPBACK_HOSTS
def is_local_host(host):
"""Loopback address, OR on a configured trusted network. The consumption
gates (PIN/API-key middleware, WS guard) call this so a trusted LAN/proxy is
exempted. Admin gates use :func:`is_loopback` NOT this to preserve the
two-tier privilege model: consumption trust admin trust."""
if is_loopback(host):
return True
try:
ip = ipaddress.ip_address(host)
except (ValueError, TypeError):
return False
# Unwrap IPv4-mapped IPv6 (::ffff:192.168.1.5) so it matches IPv4 CIDRs —
# dual-stack proxies (Caddy, Node.js) frequently pass the mapped form.
if getattr(ip, "ipv4_mapped", None):
ip = ip.ipv4_mapped
return any(ip in net for net in _trusted_networks())
_TRUTHY = frozenset({"1", "true", "yes", "on"})
@@ -54,15 +94,6 @@ def _server_mode() -> bool:
return os.environ.get("OMNIVOICE_SERVER_MODE", "").strip().lower() in _TRUTHY
def validate_server_admin_key() -> None:
"""Reject an explicitly blank key before a server-mode app starts."""
raw_key = os.environ.get("OMNIVOICE_API_KEY")
if _server_mode() and raw_key is not None and not raw_key.strip():
raise RuntimeError(
"OMNIVOICE_API_KEY is blank; configure a non-whitespace administrator key"
)
def _configured_pin(request) -> str | None:
"""The active share PIN (``app.state.network_share.pin``) or None. Read via
getattr so a bare Request stub (or a request that hit before lifespan set
@@ -74,45 +105,40 @@ def _configured_pin(request) -> str | None:
def _admin_credential_configured(request) -> bool:
"""Whether an API key or share PIN is configured.
The PIN cannot authorize admin access, but its presence means the operator
opted out of bare-server discovery. Remote admin then remains closed until
they configure and present the long API key.
"""
if remote_api_key():
"""Whether the operator has set ANY credential gate — the remote API key or
a share PIN. When neither is set, server mode leaves admin open (the Docker
issue #261 flow the image depends on)."""
if os.environ.get("OMNIVOICE_API_KEY"):
return True
return bool(_configured_pin(request))
def _request_presents_admin_credential(
request,
*,
side_effectful_get: bool = False,
) -> bool:
"""Whether the canonical principal carries remote admin capability.
def _request_presents_admin_credential(request) -> bool:
"""Whether the request carries a valid **API key** via the channels the
middleware accepts (``Authorization: Bearer`` / ``?api_key`` / ``ov_key``
cookie).
API-key and short-lived session principals may unlock server-mode admin.
PIN and trusted-network principals remain consumption-only.
"""
principal = principal_for(request)
if principal.kind not in {
PrincipalKind.API_KEY,
PrincipalKind.ADMIN_SESSION,
}:
Admin is RCE-class (``/system/set-env`` + ``/api/settings/*``), so only the
API key a long operator-chosen secret unlocks it. The 6-digit share PIN
is deliberately NOT accepted here: it is a *consumption* credential for LAN
playback and is short enough to brute-force (10^6, no lockout), so it must
never gate the admin surface (CodeRabbit #1213). A trusted-network CIDR
(``is_local_host`` also a consumption exemption) likewise never unlocks
admin. Net: remote admin in server mode requires the API key; a PIN-only
deployment keeps admin loopback-only. getattr-defensive so a minimal Request
stub never raises."""
api_key = os.environ.get("OMNIVOICE_API_KEY") or ""
if not api_key:
return False
if principal.transport not in {
CredentialTransport.COOKIE,
CredentialTransport.LEGACY_COOKIE,
}:
return True
method = str(getattr(request, "method", "GET")).upper()
if side_effectful_get or method not in SAFE_HTTP_METHODS:
return cookie_csrf_allowed(
request,
side_effectful_get=side_effectful_get,
)
return True
headers = getattr(request, "headers", None) or {}
query = getattr(request, "query_params", None) or {}
cookies = getattr(request, "cookies", None) or {}
auth = headers.get("authorization", "")
supplied = auth[7:].strip() if auth.lower().startswith("bearer ") else ""
if not supplied:
supplied = query.get("api_key") or cookies.get("ov_key") or ""
return bool(supplied and secrets.compare_digest(supplied, api_key))
def require_loopback(request: Request) -> None:
@@ -135,9 +161,9 @@ def require_loopback(request: Request) -> None:
unenforceable, so the gate can't require true loopback. It then applies the
admin-credential rule instead:
- No credential configured (no API key, no PIN) read-only requests are
open, matching the #261 Docker bootstrap flow. State-changing requests
fail closed even if a route accidentally kept this legacy dependency.
- No credential configured (no API key, no PIN) open, matching the #261
Docker flow where the operator reaches ``/system/*`` off the bridge
gateway with nothing set.
- A credential IS configured the request must present the **API key**.
This keeps the two-tier privilege model intact under server mode:
``OMNIVOICE_TRUSTED_NETWORKS`` is a *consumption* exemption
@@ -145,20 +171,14 @@ def require_loopback(request: Request) -> None:
NEVER by itself unlock the admin surface (``/system/set-env`` RCE-class
and ``/api/settings/*``). The 6-digit share PIN is a consumption credential
too and does not gate admin, so a PIN-only deployment keeps admin
loopback-only; remote admin requires the long API key. See
docs/api-auth.md (#1213).
loopback-only; remote admin requires the (long) API key. A LAN client in a
trusted CIDR or one holding only the PIN gets 403 here even though it
sails through the consumption gates. See docs/api-auth.md (#1213).
"""
host = request.client.host if request.client else None
if is_loopback(host):
return
if _server_mode():
method = str(getattr(request, "method", "GET")).upper()
if method not in SAFE_HTTP_METHODS:
# Defense in depth. Privileged routers should declare
# ``require_admin`` directly, but a missed migration must not turn
# into an unauthenticated Docker write primitive.
require_admin(request)
return
if not _admin_credential_configured(request):
return
if _request_presents_admin_credential(request):
@@ -166,97 +186,14 @@ def require_loopback(request: Request) -> None:
raise HTTPException(status_code=403, detail="loopback origin required")
def _admin_gate_403() -> None:
"""Raise the admin-gate 403 with a detail that states what would ACTUALLY
satisfy the gate. The bundled UI routes any 403 whose detail mentions
"admin api key" to the API-key login form (frontend ``client.ts``; the
literal contract is locked by ``tests/test_auth_gate_detail_lockstep.py``),
so the wording must not name a key where presenting one cannot help.
The detail names the key only when the gate would accept one: server mode
WITH an API key configured. Every other rejection desktop mode (the
credential checks in the callers only run under server mode) and a
server-mode deployment with only a share PIN or nothing configured keeps
the plain loopback detail, because only loopback can use admin there.
Naming the key in those cases would trap a LAN-share guest in a login
form that can never succeed (#1213, #1525; PR #1569 review).
"""
raise HTTPException(
status_code=403,
detail=(
"loopback origin or admin API key required"
if _server_mode() and remote_api_key()
else "loopback origin required"
),
)
def require_admin(request: Request) -> None:
"""Gate RCE/filesystem-capable admin routers.
Desktop callers keep the loopback-only contract. Docker cannot reliably
observe the host operator as loopback, so authenticated remote admin stays
available there, but every state-changing request must present the long API
key. An unconfigured server must never expose executable-path or filesystem
settings to every client that can reach its published port.
Read-only requests retain the bare-Docker bootstrap behaviour until an API
key is configured. Share PINs and trusted CIDRs are consumption credentials;
neither authorizes this gate.
"""
host = request.client.host if request.client else None
if is_loopback(host):
return
if _server_mode():
method = str(getattr(request, "method", "GET")).upper()
read_only = method in SAFE_HTTP_METHODS
if read_only and not _admin_credential_configured(request):
return
if _request_presents_admin_credential(request):
return
_admin_gate_403()
def require_admin_action(request: Request) -> None:
"""Gate an administrative action even when its HTTP method is read-only.
A small number of legacy GET endpoints have real side effects. For example,
an engine health check may spawn a sidecar process. Such routes cannot use
:func:`require_admin`'s bare-server discovery exception.
"""
host = request.client.host if request.client else None
if is_loopback(host):
return
if _server_mode() and _request_presents_admin_credential(
request,
side_effectful_get=True,
):
return
_admin_gate_403()
def require_desktop(request: Request) -> None:
"""Gate capabilities that may select or execute host filesystem paths.
An API key authorizes remote administration, not access to the desktop
shell's native file-picker boundary. These capabilities therefore remain
strictly loopback-only even when server mode is enabled.
"""
host = request.client.host if request.client else None
if is_loopback(host):
return
raise HTTPException(status_code=403, detail="desktop origin required")
def require_local(request: Request) -> None:
"""Reject any request whose client.host is not loopback OR on a configured
trusted network. The consumption-tier companion to :func:`require_loopback`:
use on routes a trusted-network client (LAN/proxy) should reach without a PIN
or API key e.g. the dictation model/prefs endpoints that pair with the
dictation WebSocket. Admin routes stay on :func:`require_admin`.
dictation WebSocket. Admin routes stay on :func:`require_loopback`.
In server mode this consumption gate is a no-op. Admin dependencies remain
method-aware and independent from this exemption."""
In server mode the gate is a no-op (same as :func:`require_loopback`)."""
host = request.client.host if request.client else None
if is_local_host(host):
return
@@ -265,22 +202,29 @@ def require_local(request: Request) -> None:
raise HTTPException(status_code=403, detail="loopback origin required")
def require_native_access(request: Request) -> None:
"""Protect capabilities that read or write operator-chosen host paths.
Docker server mode deliberately relaxes the ordinary admin gate because a
bridge makes even local traffic appear remote. That exception is unsafe for
native file pickers: a remote API caller must never probe or overwrite an
arbitrary path on the backend host, even with the server API key.
"""
host = request.client.host if request.client else None
if not is_loopback(host):
raise HTTPException(status_code=403, detail="native filesystem access requires loopback origin")
def remote_api_key() -> str | None:
"""The remote-backend bearer key (Wave 2.3), or None when remote mode is
off. Read at call time so tests can monkeypatch the env."""
return os.environ.get("OMNIVOICE_API_KEY") or None
def ws_remote_authorized(websocket) -> bool:
"""Whether the canonical WS principal has a remote admin credential."""
return principal_for(websocket).kind in {
PrincipalKind.API_KEY,
PrincipalKind.ADMIN_SESSION,
}
"""Whether a WebSocket handshake presents the remote API key.
Browser WebSockets cannot set an Authorization header, so the key may
arrive as ``?api_key=`` or via the ``ov_key`` cookie that the bearer
middleware sets on the first authenticated HTTP request. Returns False
when remote mode is off callers keep their loopback-only behavior.
"""
key = remote_api_key()
if not key:
return False
auth = websocket.headers.get("authorization", "")
supplied = auth[7:].strip() if auth.lower().startswith("bearer ") else ""
if not supplied:
supplied = (
websocket.query_params.get("api_key")
or websocket.cookies.get("ov_key")
or ""
)
return secrets.compare_digest(supplied, key)
-176
View File
@@ -1,176 +0,0 @@
"""Stable, non-diagnostic metadata for engine-discovery responses."""
from __future__ import annotations
from core.device_caps import KERNEL_RISK_MARKER
_UNAVAILABLE = "Engine unavailable. Check installation and configuration."
_PREVIOUS_FAILURE = "A previous engine check failed."
_ROUTING_BY_STATUS = {
"cpu_fallback": "GPU acceleration is unavailable; this engine will use CPU.",
"cpu_only": "This engine runs on CPU on this host.",
"unavailable": "This engine has no compatible compute device on this host.",
}
_ROUTING_UNAVAILABLE = "Engine routing details are unavailable."
_ACCELERATOR_KERNEL_RISK = (
"The selected accelerator may not be supported by this PyTorch build."
)
_ACCELERATOR_LOW_VRAM = (
"The accelerator may not meet this engine's recommended VRAM."
)
_ACCELERATOR_ADVISORY = "The selected accelerator has a compatibility advisory."
def _public_routing_reason(status: object, diagnostic: object) -> str:
"""Map a private routing diagnostic to an accurate stable category."""
if status == "accelerated":
private = diagnostic if isinstance(diagnostic, str) else ""
if KERNEL_RISK_MARKER in private:
return _ACCELERATOR_KERNEL_RISK
if " GB VRAM; this engine wants about " in private:
return _ACCELERATOR_LOW_VRAM
return _ACCELERATOR_ADVISORY
return _ROUTING_BY_STATUS.get(status, _ROUTING_UNAVAILABLE)
# Categories for WHY an engine is unavailable. The probe's own sentence cannot
# cross the boundary — it carries exception text, local paths and sometimes
# credentials — but "Engine unavailable. Check installation and configuration."
# told the user nothing at all, and "Last error: A previous engine check
# failed." reads like a crash rather than "you have not installed this yet"
# (#1866). Classifying the private diagnostic into an owned sentence keeps the
# boundary intact and still names the kind of problem and the place to fix it.
_UNAVAILABLE_NOT_INSTALLED = (
"This engine's package isn't installed yet. Install it from "
"Model Catalogue."
)
# An engine gated behind an in-app license review (Supertonic-3, PocketTTS).
# The Model Catalogue shows its Accept button only when the reason matches
# /license not accepted/i (EngineCompatibilityMatrix.reasonMentionsLicense), so
# this sentence must keep those words: collapsing it into the generic line hid
# the only way to enable those engines.
_UNAVAILABLE_LICENSE = (
"License not accepted yet. Review and accept it in "
"Model Catalogue to enable this engine."
)
# An engine that cannot run on this machine at all: Apple-Silicon-only MLX,
# PyTorch with no Intel Mac build. "Isn't installed yet" or "check
# installation" sent people after an install that could never work.
_UNAVAILABLE_PLATFORM = (
"This engine doesn't run on this computer's platform. Its guide lists "
"the platforms it supports."
)
# Apple Silicon whose PyTorch cannot use the GPU (MPS): the platform is
# right, the installation is not. MLX-Audio / MLX-Whisper need MPS (#390).
_UNAVAILABLE_NO_MPS = (
"This engine needs Apple's GPU (MPS), and this installation's PyTorch "
"can't use it. Updating macOS or reinstalling VoiceStudio usually "
"restores it."
)
_UNAVAILABLE_NEEDS_CONFIG = (
"This engine needs to be configured before it can run. Open "
"Model Catalogue to finish setting it up."
)
_UNAVAILABLE_FILE_MISSING = (
"A file this engine needs is missing or unreadable. Reinstall it from "
"Model Catalogue."
)
# The same two cases for an engine the app cannot install for you. "Install it
# from Model Catalogue" sent people to a page with no Install button
# for that engine — most of the catalogue — which reads as the app being
# broken. The row's own guide link (``docs_url``) is the real next step.
_UNAVAILABLE_NOT_INSTALLED_MANUAL = (
"This engine isn't installed yet, and it has no one-click install. "
"Its guide lists the install steps."
)
_UNAVAILABLE_FILE_MISSING_MANUAL = (
"A file this engine needs is missing or unreadable. Its guide lists the "
"install steps."
)
_MANUAL_INSTALL_VARIANT = {
_UNAVAILABLE_NOT_INSTALLED: _UNAVAILABLE_NOT_INSTALLED_MANUAL,
_UNAVAILABLE_FILE_MISSING: _UNAVAILABLE_FILE_MISSING_MANUAL,
}
# Matched against the lowered probe text. Ordered most specific first: a
# missing file often also says "not installed", and the file case has the more
# useful remedy of the two.
_UNAVAILABLE_SIGNATURES = (
# First: its probe text also says "Open Model Catalogue", and the
# license is the one gap only the user can close.
(_UNAVAILABLE_LICENSE, ("license not accepted",)),
# Before the install and file checks: a platform reason often also says
# "unavailable" or names a missing wheel, and no install can fix it. Not
# "apple silicon only": mlx-audio says that on an M-series Mac too, when
# the package is merely missing and installing does help.
(_UNAVAILABLE_PLATFORM, (
"requires apple silicon", "not supported on this platform",
"unavailable on intel macs", "no macos x86_64 wheel",
"no windows install", "not supported on windows",
)),
(_UNAVAILABLE_NO_MPS, ("torch mps unavailable",)),
(_UNAVAILABLE_FILE_MISSING, (
"file is missing", "file is empty", "file is unreadable",
"script missing", "binary", "not found at",
)),
(_UNAVAILABLE_NEEDS_CONFIG, (
"environment variable", "configure a server endpoint", "api key",
"unconfigured", "set the", "base url",
)),
(_UNAVAILABLE_NOT_INSTALLED, (
"not installed", "package missing", "not available", "no module named",
"import ", "unavailable:", "failed to load",
)),
)
def _public_unavailable_reason(diagnostic: object) -> str:
"""Map a private availability probe to an accurate stable category."""
private = diagnostic.lower() if isinstance(diagnostic, str) else ""
for public, markers in _UNAVAILABLE_SIGNATURES:
if any(marker in private for marker in markers):
return public
return _UNAVAILABLE
def public_backends(entries: list[dict]) -> list[dict]:
"""Copy registry entries while replacing service diagnostics.
Availability probes may contain exception text, local paths, tracebacks, or
credentials. Registry-authored fields are not probe output and remain
intact: ``install_hint``, ``setup_snippet`` and ``docs_url`` are all
VoiceStudio-owned constants keyed on the engine id, so an unavailable row
still has something actionable to show and somewhere to send the user
(#1866) even though ``reason``/``last_error`` are replaced here.
"""
safe: list[dict] = []
for entry in entries:
item = dict(entry)
if item.get("reason") is not None:
reason = _public_unavailable_reason(item["reason"])
# Only a row that explicitly says it has NO one-click install gets
# the manual wording. Rows without the field (ASR, LLM,
# translation — some of which have installers of their own) keep
# the line that points at Model Catalogue.
if item.get("one_click_install") is False:
reason = _MANUAL_INSTALL_VARIANT.get(reason, reason)
item["reason"] = reason
if item.get("last_error") is not None:
item["last_error"] = _PREVIOUS_FAILURE
if item.get("routing_reason") is not None:
item["routing_reason"] = _public_routing_reason(
item.get("routing_status"), item["routing_reason"]
)
evidence = item.get("execution_evidence")
if isinstance(evidence, dict) and evidence.get("cpu_fallback_reason") is not None:
evidence = dict(evidence)
evidence["cpu_fallback_reason"] = _public_routing_reason(
"cpu_fallback", evidence["cpu_fallback_reason"]
)
item["execution_evidence"] = evidence
safe.append(item)
return safe
def public_unavailability(detail: object) -> str | None:
return None if detail is None else _UNAVAILABLE
+79 -437
View File
@@ -15,33 +15,22 @@ Design notes
* Previews are cached on disk keyed by a hash of (instruct, language), so two
archetypes that resolve to the same voice share a cache file and the cold
render only happens once per distinct voice.
* That same key names the pre-rendered clips in the opt-in voice gallery
(``services.gallery``), which is consulted BEFORE the engine so a fresh
install can hear voices before the 2.4 GB checkpoint finishes downloading.
Gallery files win over a local render of the same key but only for
``/preview``. ``/use`` always renders locally: the WAV it keeps in
``VOICES_DIR`` is the reference audio a cloned voice is built from, and a
downloaded MP3 must never become that.
"""
from __future__ import annotations
import hashlib
import json
import logging
import os
import re
import time
import uuid
from pathlib import Path
from typing import Optional
from fastapi import APIRouter, Body, HTTPException, Query
from fastapi import APIRouter, HTTPException, Query
from fastapi.responses import FileResponse
from core import archetypes
from core.audio_validation import is_playable_wav, resolve_regular_file
from core.config import OUTPUTS_DIR, VOICES_DIR
from services import gallery
logger = logging.getLogger("omnivoice.archetypes")
@@ -57,12 +46,11 @@ _PREVIEW_SEED = 42
# 32 reliably converges to speech across the gallery's instruct/script space
# at a one-time (cached) render cost.
_PREVIEW_NUM_STEP = 32
# Reject near-pure tonal artifacts using mean framed spectral flatness.
# Calibrated against the tracked speech demos exercised by
# test_archetype_preview_quality.py: the quietest (Mandarin dubbing, 44.1 kHz)
# measures ~7.7e-6, while the worst tested tonal buzz measures ~3.3e-9.
# 1e-7 leaves >10x margin on both sides without rejecting low-flatness speech.
_DEGENERATE_FLATNESS = 1e-7
# Spectral-flatness floor below which a render is a degenerate tonal artifact
# rather than speech. Real, mastered speech sits ~0.040.07; a tonal buzz
# collapses to <0.005. 0.015 separates the two with wide margin and sits well
# below even breathy/whisper voices (which are broadband → high flatness).
_DEGENERATE_FLATNESS = 0.015
def _preview_key(a: dict) -> str:
@@ -73,153 +61,6 @@ def _preview_key(a: dict) -> str:
).hexdigest()[:16]
def _design_profile_values(a: dict) -> tuple[str, str]:
"""Canonical instruct + complete picker state for a designed archetype."""
return a["instruct"], json.dumps(a["attrs"], sort_keys=True)
def _profile_audio_path(ref_audio_path: object) -> Optional[Path]:
"""Resolve only a regular, non-symlinked file inside ``VOICES_DIR``."""
return resolve_regular_file(VOICES_DIR, ref_audio_path)
def _materialized_audio_is_current(row, a: dict) -> bool:
"""Whether an existing row still has the sample described by its metadata."""
expected_filename = _profile_audio_filename(row["id"])
path = _profile_audio_path(row["ref_audio_path"])
return bool(
row["ref_audio_path"] == expected_filename
and is_playable_wav(path)
and row["instruct"] == a["instruct"]
and row["language"] == a["language"]
and row["ref_text"] == a["sample_script"]
and row["seed"] == _PREVIEW_SEED
)
def _profile_audio_filename(profile_id: str) -> str:
safe_id = (
profile_id if re.fullmatch(r"[A-Za-z0-9_-]{1,64}", profile_id or "")
else hashlib.sha256(str(profile_id).encode("utf-8")).hexdigest()[:16]
)
return f"{safe_id}.wav"
def _archetype_personality(a: dict) -> str:
return f"archetype:{a['id']}"
def _legacy_archetype_profile(conn, a: dict):
"""Adopt only a row that an older archetype materializer could have made."""
row = conn.execute(
"SELECT * FROM voice_profiles WHERE personality=? LIMIT 1",
(a["id"],),
).fetchone()
if row is None:
return None
expected_audio = _profile_audio_filename(row["id"])
try:
states_match = (
not row["vd_states"] or json.loads(row["vd_states"]) == a["attrs"]
)
except (TypeError, ValueError):
states_match = False
if (
row["ref_audio_path"] == expected_audio
and row["instruct"] == a["instruct"]
and row["language"] == a["language"]
and row["ref_text"] == a["sample_script"]
and row["seed"] == _PREVIEW_SEED
and row["kind"] in (None, "", "clone", "design")
and not row["is_locked"]
and not row["verified_own_voice"]
and states_match
):
return row
return None
def _is_materialized_archetype_row(row, a: dict) -> bool:
"""Recognize rows owned by this materializer without trusting identity text alone."""
try:
states_match = json.loads(row["vd_states"]) == a["attrs"]
except (TypeError, ValueError):
return False
return bool(
row["personality"] == _archetype_personality(a)
and row["kind"] == "design"
and row["seed"] == _PREVIEW_SEED
and row["ref_audio_path"] == _profile_audio_filename(row["id"])
and row["instruct"] == a["instruct"]
and row["language"] == a["language"]
and row["ref_text"] == a["sample_script"]
and states_match
and not row["is_locked"]
and not row["verified_own_voice"]
)
def _existing_archetype_profile(conn, a: dict):
rows = conn.execute(
"SELECT * FROM voice_profiles WHERE personality=? ORDER BY created_at, id",
(_archetype_personality(a),),
).fetchall()
owned = next((row for row in rows if _is_materialized_archetype_row(row, a)), None)
return owned if owned is not None else _legacy_archetype_profile(conn, a)
async def _render_profile_audio(
a: dict, profile_id: str, *, publish: bool = True,
) -> tuple[str, Path]:
"""Render one validated sample, optionally staging it for a later CAS."""
audio_filename = _profile_audio_filename(profile_id)
safe_id = Path(audio_filename).stem
audio_path = Path(VOICES_DIR) / audio_filename
if publish:
await _render_wav_atomic(a, audio_path, prefix=f".{safe_id}-")
else:
audio_path.parent.mkdir(parents=True, exist_ok=True)
audio_path = audio_path.parent / f".{safe_id}-{uuid.uuid4().hex}.staged.wav"
try:
await _render_archetype_wav(a, audio_path)
if not is_playable_wav(audio_path):
raise RuntimeError("the voice engine produced an invalid WAV")
except BaseException:
with __import__("contextlib").suppress(OSError):
audio_path.unlink()
raise
return audio_filename, audio_path
async def _render_wav_atomic(a: dict, out_path: Path, *, prefix: str = ".render-") -> Path:
"""Render and validate a WAV before atomically replacing *out_path*."""
audio_path = Path(out_path)
audio_path.parent.mkdir(parents=True, exist_ok=True)
tmp_path = audio_path.parent / f"{prefix}{uuid.uuid4().hex}.wav"
try:
await _render_archetype_wav(a, tmp_path)
if not is_playable_wav(tmp_path):
raise RuntimeError("the voice engine produced an invalid WAV")
os.replace(tmp_path, audio_path)
finally:
with __import__("contextlib").suppress(OSError):
tmp_path.unlink()
return audio_path
def _heal_materialized_profile(conn, row, a: dict, audio_filename: str) -> None:
"""Repair profiles created before archetype `/use` persisted design kind."""
instruct, vd_states = _design_profile_values(a)
conn.execute(
"UPDATE voice_profiles SET kind='design', instruct=?, vd_states=?, language=?, "
"ref_text=?, seed=?, ref_audio_path=?, personality=? WHERE id=?",
(
instruct, vd_states, a["language"], a["sample_script"], _PREVIEW_SEED,
audio_filename, _archetype_personality(a), row["id"],
),
)
# A non-empty script is always required — synthesizing empty text yields
# silence. Every archetype carries a use-case script, but guard the render path
# too so a malformed archetype can never drive a blank render.
@@ -249,46 +90,24 @@ def _is_blank_audio(audio_tensor) -> bool:
return False
_FLATNESS_FRAME = 1024
_FLATNESS_HOP = 512
#: Frames quieter than this fraction of the loudest frame's energy are the gaps
#: between words, not speech; their spectrum is the noise floor and averaging it
#: in drags the measurement toward the value of whatever silence sounds like.
_FLATNESS_FRAME_FLOOR = 1e-4
def _spectral_flatness(audio_tensor) -> Optional[float]:
"""Mean per-frame geometric-mean / arithmetic-mean of the power spectrum.
"""Geometric-mean / arithmetic-mean of the power spectrum.
~1.0 for broadband noise, 0 for a pure tone. The degenerate diffusion
renders this guards against are near-pure tonal buzzes, distinct from both
silence (caught by ``_is_blank_audio``) and real speech. Returns ``None``
if it can't be computed so callers don't act on a bad measurement.
Measured over short frames and averaged the standard definition. A single
FFT of the whole clip (what this used to do) is not the same quantity: its
frequency resolution grows with clip length, so speech harmonics carve
ever-deeper nulls into the spectrum and the geometric mean collapses. That
made the result depend on how long the clip was rather than on what it
sounded like, and put real speech below the rejection threshold.
renders this guards against are near-pure tonal buzzes (flatness <0.005),
distinct from both silence (caught by ``_is_blank_audio``) and real speech
(~0.04+). Returns ``None`` if it can't be computed so callers don't act on
a bad measurement.
"""
try:
import torch
t = audio_tensor if isinstance(audio_tensor, torch.Tensor) else torch.as_tensor(audio_tensor)
t = t.detach().to("cpu", dtype=torch.float32)
if t.ndim > 1:
t = t.mean(dim=0)
t = t.flatten()
if t.numel() < _FLATNESS_FRAME or not torch.isfinite(t).all():
t = t.detach().to("cpu", dtype=torch.float32).flatten()
if t.numel() < 1024 or not torch.isfinite(t).all():
return None
frames = t.unfold(0, _FLATNESS_FRAME, _FLATNESS_HOP)
spec = torch.fft.rfft(frames * torch.hann_window(_FLATNESS_FRAME)).abs().pow(2) + 1e-12
energy = spec.sum(dim=1)
spec = spec[energy > energy.max() * _FLATNESS_FRAME_FLOOR]
if spec.shape[0] == 0:
return None
return float((torch.exp(spec.log().mean(dim=1)) / spec.mean(dim=1)).mean())
spec = torch.fft.rfft(t * torch.hann_window(t.numel())).abs().pow(2) + 1e-12
return float(torch.exp(torch.mean(torch.log(spec))) / torch.mean(spec))
except Exception: # never let the checker itself block a render
return None
@@ -353,7 +172,7 @@ async def _render_archetype_wav(a: dict, out_path: Path) -> None:
# GPU pool and brick the backend (#730 class). Budget comes from the shared
# length-scaled helper (#1190) instead of the flat 300s default.
from services.model_manager import generate_timeout_s
_budget = generate_timeout_s(text, engine=model)
_budget = generate_timeout_s(text)
audio_tensor = await run_on_gpu_pool_guarded(
lambda: _infer(_PREVIEW_SEED), what="Archetype preview generate",
timeout=_budget)
@@ -380,60 +199,21 @@ async def _render_archetype_wav(a: dict, out_path: Path) -> None:
# Runs on the dedicated watermark pool (#1190): AudioSeal embedding is CPU
# work that holds no VRAM, so it must not occupy a GPU worker ahead of the
# next generate on 1-worker hosts.
from services.watermark import mark_synthetic_async
audio_tensor = await mark_synthetic_async(
audio_tensor, model.sampling_rate,
context="archetypes.render",
from services.watermark import mark_synthetic
from services.model_manager import get_watermark_pool
import functools
audio_tensor = await run_on_gpu_pool_guarded(
functools.partial(mark_synthetic, audio_tensor, model.sampling_rate,
context="archetypes.render"),
what="Archetype watermark",
timeout=generate_timeout_s(""),
executor=get_watermark_pool(),
)
out_path.parent.mkdir(parents=True, exist_ok=True)
_safe_torchaudio_save(str(out_path), audio_tensor, model.sampling_rate)
def _no_voice_model_downloaded() -> bool:
"""True only on a *positive* "no TTS weights on this machine" answer.
Fails open on purpose: the cache probes are best-effort (a user-managed
clone outside the HF layout is invisible to them), and telling someone with
a working engine to go download a model is worse than saying nothing. Only
a catalog we could read, with not one TTS repo cached, earns the offline
message.
"""
try:
from api.routers.setup.models import get_model_catalog, is_cached
tts = [m for m in get_model_catalog().all if m.get("role") == "TTS"]
return bool(tts) and not any(is_cached(m["repo_id"]) for m in tts)
except Exception:
return False
def _preview_source(a: dict) -> tuple[str, str]:
"""Which path ``/preview`` will take for *a*, and what to tell the user.
Replaces the old "see Settings → Logs → Backend" advice, which asked a user
who wanted to hear a voice to go read a log file. The three states that
actually differ are: we already have the audio (gallery), we can make it
(render say so, it takes a moment), and we can neither fetch nor make it
(no model the one state with an action attached).
"""
key = _preview_key(a)
if gallery.cached_preview(key) is not None:
return "gallery", (
"Pre-rendered preview from the voice gallery — a fixed reference "
"rendering, not a render from your current engine."
)
if is_playable_wav(_PREVIEW_DIR / f"{key}.wav"):
return "cached", ""
if _no_voice_model_downloaded():
return "no_model", (
"You're offline and no voice model is downloaded yet — download "
"one from the engine's Weights list in Model Catalogue."
)
return "rendering", "Rendering this preview on your machine — it may take a moment."
# ── Read endpoints (no model) ─────────────────────────────────────────────────
# NOTE: declare the literal `/archetypes/categories` before `/archetypes/{id}`
# so it isn't swallowed by the path-parameter route.
@@ -443,37 +223,6 @@ def list_categories():
return archetypes.categories()
# ── Voice-gallery (pre-rendered previews) ─────────────────────────────────────
# Declared above `/archetypes/{archetype_id}` for the same reason as
# `/categories`: keep literal paths out of the path-parameter route's reach.
@router.get("/archetypes/previews/status")
def preview_gallery_status():
"""Consent state, coverage and freshness for the Settings line."""
return gallery.status()
@router.put("/archetypes/previews")
async def set_preview_gallery(enabled: bool = Body(..., embed=True)):
"""Turn pre-rendered previews on or off.
Turning it ON is the user's explicit yes to an outbound call, and is the
only thing that ever starts one there is no on-install background fetch.
The featured set is pulled right here so the yes has a visible effect;
failures are silent by design (``fetch_featured`` swallows them) and leave
previews rendering locally.
"""
state = gallery.set_enabled(enabled)
if enabled:
state = await gallery.fetch_featured()
return state
@router.post("/archetypes/previews/check")
async def check_preview_gallery():
"""Manual "check now" — bypasses the 24 h throttle, never the signature."""
return await gallery.check_for_updates(force=True)
@router.get("/archetypes")
def list_archetypes_endpoint(
q: Optional[str] = None,
@@ -513,77 +262,33 @@ def get_archetype_endpoint(archetype_id: str):
# ── Render endpoints (model-gated) ────────────────────────────────────────────
@router.get("/archetypes/{archetype_id}/preview/state")
def preview_archetype_state(archetype_id: str):
"""Where the next ``/preview`` for this archetype would come from.
Touches neither the model nor the network, so a picker can label a voice
("may take a moment", "download a model first") *before* it commits to a
request that may take 40 seconds or fail.
"""
a = archetypes.get_archetype(archetype_id)
if a is None:
raise HTTPException(status_code=404, detail="Archetype not found")
source, message = _preview_source(a)
return {"source": source, "message": message}
@router.get("/archetypes/{archetype_id}/preview")
async def preview_archetype(
archetype_id: str,
local: bool = Query(False, description="Bypass gallery audio after a client decode failure"),
):
"""Serve a short preview clip — from the gallery, the cache, or the engine."""
async def preview_archetype(archetype_id: str):
"""Serve a short preview clip — pre-rendered if cached, else render once."""
a = archetypes.get_archetype(archetype_id)
if a is None:
raise HTTPException(status_code=404, detail="Archetype not found")
key = _preview_key(a)
# Gallery first, and only for /preview: these bytes are audio we can prove
# the provenance of, so they beat a local render of the same key. A miss
# (offline, disabled, key not published) is silent — we just render.
gallery_path = None if local else gallery.cached_preview(key)
if gallery_path is None and not local:
gallery_path = await gallery.fetch_preview(key)
if gallery_path is not None:
# Nothing else in the app polls, so the daily refresh hangs off the
# request that proves previews are being used. Fire-and-forget.
gallery.maybe_refresh_in_background()
return FileResponse(
str(gallery_path),
media_type="audio/mpeg",
headers={"Cache-Control": "no-cache",
"X-OmniVoice-Preview-Source": "gallery"},
)
cache_path = _PREVIEW_DIR / f"{key}.wav"
if not is_playable_wav(cache_path):
cache_path = _PREVIEW_DIR / f"{_preview_key(a)}.wav"
if not cache_path.exists():
try:
await _render_wav_atomic(a, cache_path, prefix=".preview-")
await _render_archetype_wav(a, cache_path)
except Exception as e: # model missing / OOM / inference failure
logger.error("Archetype preview render failed", exc_info=True)
# Two different failures, two different answers. Without a model
# there is nothing to read in a log — there is something to do.
if _no_voice_model_downloaded():
detail = (
"You're offline and no voice model is downloaded yet — "
"download one from the engine's Weights list in Model Catalogue. (Or turn "
"on pre-rendered voice previews in Settings → Storage.)"
)
else:
detail = (
"Couldn't render a preview right now — the voice engine "
f"reported: {e}"
)
raise HTTPException(status_code=503, detail=detail)
raise HTTPException(
status_code=503,
detail=(
"Couldn't render a preview right now — the voice engine is "
f"unavailable. See Settings → Logs → Backend. Error: {e}"
),
)
# no-cache (not no-store): the URL is stable but its bytes change when an
# archetype's preview is re-rendered, so force the client to revalidate
# against the ETag instead of serving a stale cached clip indefinitely.
return FileResponse(
str(cache_path),
media_type="audio/wav",
headers={"Cache-Control": "no-cache",
"X-OmniVoice-Preview-Source": "local"},
headers={"Cache-Control": "no-cache"},
)
@@ -595,11 +300,6 @@ async def use_archetype(archetype_id: str, name: Optional[str] = Query(None)):
preview) and inserts a ``voice_profiles`` row carrying the archetype's
instruct + language. The profile then shows up everywhere voices are
picked (Dub / Generate / Clone).
Never sourced from the voice gallery, no matter how cheap that would be:
this WAV lands in ``VOICES_DIR`` as the profile's reference audio, so a
downloaded, lossily-encoded MP3 would silently become the sample every
future clone of this voice is built from. It renders locally or it fails.
"""
a = archetypes.get_archetype(archetype_id)
if a is None:
@@ -611,122 +311,64 @@ async def use_archetype(archetype_id: str, name: Optional[str] = Query(None)):
# Idempotent (dedup): an archetype materializes to exactly ONE voice profile.
# Picking the same gallery voice again — from any picker (Gallery grid,
# VoiceSelector, …) — must reuse that one row instead of rendering + inserting
# a fresh duplicate every time. Use a namespaced personality identity so an
# imported persona cannot collide with and be rewritten by an archetype id.
# a fresh duplicate every time. The `personality` column already carries the
# source archetype id (stamped by the INSERT below), so it's the natural
# dedup key; the expensive render + INSERT only run on first use.
with db_conn() as conn:
existing = _existing_archetype_profile(conn, a)
profile_id = existing["id"] if existing is not None else str(uuid.uuid4())[:8]
audio_path: Optional[Path] = None
if existing is not None and _materialized_audio_is_current(existing, a):
audio_filename = existing["ref_audio_path"]
else:
try:
audio_filename, audio_path = await _render_profile_audio(
a, profile_id, publish=existing is None,
)
except Exception as e:
logger.error("Archetype 'use' render failed", exc_info=True)
# Same actionable/diagnostic split as /preview — minus the gallery
# suggestion, which cannot help here.
if _no_voice_model_downloaded():
detail = (
"Creating a voice needs the voice model — no voice model is "
"downloaded yet. Download one from the engine's Weights list in Model Catalogue."
)
else:
detail = (
"Couldn't create a voice from this archetype — the voice engine "
f"reported: {e}"
)
raise HTTPException(status_code=503, detail=detail) from e
existing = conn.execute(
"SELECT id, name FROM voice_profiles WHERE personality = ? LIMIT 1",
(a["id"],),
).fetchone()
if existing is not None:
with db_conn() as conn:
conn.execute("BEGIN IMMEDIATE")
current = conn.execute(
"SELECT * FROM voice_profiles WHERE id=?", (existing["id"],),
).fetchone()
owned = _existing_archetype_profile(conn, a)
still_owned = current is not None and (
owned is not None and owned["id"] == current["id"]
)
if still_owned:
if audio_path is not None:
destination = Path(VOICES_DIR) / audio_filename
os.replace(audio_path, destination)
audio_path = None
_heal_materialized_profile(conn, current, a, audio_filename)
existing_result = {"profile_id": current["id"], "name": current["name"]}
else:
existing_result = None
if existing_result is not None:
event_bus.emit("profiles", {"action": "updated", "id": existing_result["profile_id"]})
return existing_result
# The row was edited/deleted while rendering. Preserve it and use the
# validated staged sample for a fresh canonical materialization.
profile_id = str(uuid.uuid4())[:8]
audio_filename = _profile_audio_filename(profile_id)
destination = Path(VOICES_DIR) / audio_filename
if audio_path is None:
try:
audio_filename, audio_path = await _render_profile_audio(a, profile_id)
except Exception as e:
raise HTTPException(
status_code=503, detail="Couldn't create a voice from this archetype.",
) from e
else:
os.replace(audio_path, destination)
audio_path = destination
return {"profile_id": existing["id"], "name": existing["name"]}
if audio_path is None: # defensive: a new profile always rendered above
raise RuntimeError("new archetype profile has no rendered audio")
profile_id = str(uuid.uuid4())[:8]
audio_filename = f"{profile_id}.wav"
audio_path = Path(VOICES_DIR) / audio_filename
try:
await _render_archetype_wav(a, audio_path)
except Exception as e:
logger.error("Archetype 'use' render failed", exc_info=True)
raise HTTPException(
status_code=503,
detail=(
"Couldn't create a voice from this archetype — the voice engine "
f"is unavailable. See Settings → Logs → Backend. Error: {e}"
),
)
profile_name = (name or a["name"]).strip() or a["name"]
try:
with db_conn() as conn:
conn.execute("BEGIN IMMEDIATE")
# Re-check under the write connection right before inserting: a
# concurrent /use for the same archetype may have inserted while we
# were rendering (the pre-render SELECT above raced). Reuse that row
# and drop our just-rendered sample instead of creating a duplicate.
# `personality` is not globally UNIQUE, so serialize and re-check.
dup = _existing_archetype_profile(conn, a)
# (personality is NOT globally unique — marketplace/persona imports
# reuse the column — so a UNIQUE index isn't an option; this closes
# the realistic window for the single-user desktop app.)
dup = conn.execute(
"SELECT id, name FROM voice_profiles WHERE personality = ? LIMIT 1",
(a["id"],),
).fetchone()
if dup is not None:
duplicate_audio = dup["ref_audio_path"]
if not _materialized_audio_is_current(dup, a):
duplicate_audio = _profile_audio_filename(dup["id"])
_duplicate_path = Path(VOICES_DIR) / duplicate_audio
_duplicate_path.parent.mkdir(parents=True, exist_ok=True)
os.replace(audio_path, _duplicate_path)
audio_path = None
_heal_materialized_profile(conn, dup, a, duplicate_audio)
with __import__("contextlib").suppress(OSError):
if audio_path is not None:
os.remove(audio_path)
duplicate_result = {"profile_id": dup["id"], "name": dup["name"]}
else:
duplicate_result = None
if duplicate_result is None:
instruct, vd_states = _design_profile_values(a)
conn.execute(
"INSERT INTO voice_profiles "
"(id, name, ref_audio_path, ref_text, instruct, language, seed, personality, "
"created_at, kind, vd_states) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, 'design', ?)",
(
profile_id, profile_name, audio_filename, a["sample_script"],
instruct, a["language"], _PREVIEW_SEED,
_archetype_personality(a), time.time(), vd_states,
),
)
os.remove(audio_path)
return {"profile_id": dup["id"], "name": dup["name"]}
conn.execute(
"INSERT INTO voice_profiles "
"(id, name, ref_audio_path, ref_text, instruct, language, seed, personality, created_at) "
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)",
(
profile_id, profile_name, audio_filename, a["sample_script"],
a["instruct"], a["language"], _PREVIEW_SEED, a["id"], time.time(),
),
)
except Exception:
with __import__("contextlib").suppress(OSError):
if audio_path is not None:
os.remove(audio_path)
os.remove(audio_path)
raise
if duplicate_result is not None:
event_bus.emit("profiles", {"action": "updated", "id": duplicate_result["profile_id"]})
return duplicate_result
event_bus.emit("profiles", {"action": "created", "id": profile_id})
return {"profile_id": profile_id, "name": profile_name}
+40 -214
View File
@@ -25,7 +25,6 @@ import json
import logging
import os
import re
import shutil
import uuid
from collections.abc import Awaitable, Callable
@@ -186,8 +185,7 @@ async def audiobook_import(file: UploadFile = File(...)) -> dict:
except ValueError as e:
raise HTTPException(status_code=400, detail=f"couldn't parse PDF: {e}")
else:
from services.text_upload import decode_text_upload
script = chapterize_plaintext(decode_text_upload(data))
script = chapterize_plaintext(data.decode("utf-8", "ignore"))
if not script.strip():
raise HTTPException(status_code=400, detail="no text found in the file")
plan = parse_audiobook_script(script)
@@ -362,7 +360,7 @@ LONGFORM_NUM_STEP = 32
LONGFORM_GUIDANCE_SCALE = 2.0
def _seed_segment_rng(base_seed, text: str, nonce: int = 0) -> int | None:
def _seed_segment_rng(base_seed, text: str, nonce: int = 0) -> None:
"""Apply a profile's pinned seed to this synth call (#1139).
``_resolve_voice`` has always fetched the profile ``seed`` but only the
@@ -381,13 +379,11 @@ def _seed_segment_rng(base_seed, text: str, nonce: int = 0) -> int | None:
must cover /generate and here together, not one path.
"""
if base_seed is None:
return None
return
import torch
from services.audiobook import segment_seed
seed = segment_seed(base_seed, text, nonce)
torch.manual_seed(seed)
return seed
torch.manual_seed(segment_seed(base_seed, text, nonce))
def _base_seed(opts: ExpressiveOptions, voice: dict):
@@ -417,15 +413,12 @@ def _make_occ_counter(opts: ExpressiveOptions):
def _omnivoice_sampling_kwargs(opts: ExpressiveOptions) -> dict:
"""VoiceStudio-model generate kwargs for the sampling knobs. UNSET reproduces
"""OmniVoice-model generate kwargs for the sampling knobs. UNSET reproduces
today exactly: num_step 32, guidance 2.0, and NO temperature/postprocess
kwargs (the model keeps its own defaults). Emotion is never forwarded
the VoiceStudio config rejects unknown kwargs."""
from services.performance_profiles import tts_defaults
defaults = tts_defaults()
the OmniVoice config rejects unknown kwargs."""
kw = {
"num_step": opts.num_step if opts.num_step is not None else defaults.get("num_step", LONGFORM_NUM_STEP),
"num_step": opts.num_step if opts.num_step is not None else LONGFORM_NUM_STEP,
"guidance_scale": (
opts.guidance_scale if opts.guidance_scale is not None else LONGFORM_GUIDANCE_SCALE
),
@@ -436,13 +429,11 @@ def _omnivoice_sampling_kwargs(opts: ExpressiveOptions) -> dict:
kw["class_temperature"] = opts.class_temperature
if opts.postprocess_output is not None:
kw["postprocess_output"] = opts.postprocess_output
elif "postprocess_output" in defaults:
kw["postprocess_output"] = defaults["postprocess_output"]
return kw
def _generic_extra_kwargs(opts: ExpressiveOptions) -> dict:
"""Extra generate kwargs for a non-VoiceStudio engine. UNSET → empty dict →
"""Extra generate kwargs for a non-OmniVoice engine. UNSET → empty dict →
byte-identical to the pre-#1208 generic call. Only present knobs are added,
and every shipped backend's ``generate(self, text, **kw)`` ignores the ones
it doesn't understand (never TypeError) — the engine-options contract. The
@@ -477,7 +468,7 @@ def _build_synth(
"""Describe how to synthesize for the active TTS engine.
Returns a dict with ``mode``, ``resolve`` (voice-id resolved refs, cached
per id) and ``engine_id``. For VoiceStudio it also carries the async
per id) and ``engine_id``. For OmniVoice it also carries the async
``get_model``; other engines carry a ready ``synth`` + ``sample_rate``.
:func:`_prepare_synth` turns this into a uniform ``(synth, sr, resolve,
engine_id)`` once the (async) model is in hand.
@@ -516,21 +507,16 @@ def _build_synth(
"get_model": get_model, "language": language, "opts": opts}
backend = cls()
native_proxy = bool(getattr(cls, "supports_native_omnivoice_controls", False))
extra = (_omnivoice_sampling_kwargs(opts) if native_proxy
else _generic_extra_kwargs(opts))
extra = _generic_extra_kwargs(opts)
next_nonce = _make_occ_counter(opts)
def synth(text, voice_id, speed=None):
v = resolve(voice_id)
seed = _seed_segment_rng(_base_seed(opts, v), text, next_nonce())
call_extra = dict(extra)
if native_proxy and seed is not None:
call_extra["seed"] = seed
_seed_segment_rng(_base_seed(opts, v), text, next_nonce())
return backend.generate(
text, language=language, ref_audio=v["ref_audio"],
ref_text=v["ref_text"], instruct=v["instruct"], duration=None,
speed=float(speed) if speed else 1.0, **call_extra,
speed=float(speed) if speed else 1.0, **extra,
)
return {"mode": "generic", "resolve": resolve, "engine_id": engine_id,
"synth": synth, "sample_rate": backend.sample_rate}
@@ -543,7 +529,7 @@ async def _prepare_synth(
voice_map: dict | None = None,
):
"""Resolve :func:`_build_synth` into ``(synth, sample_rate, resolve,
engine_id)`` awaiting the VoiceStudio model load when needed. Shared by the
engine_id)`` awaiting the OmniVoice model load when needed. Shared by the
full job and the per-chapter preview. ``language`` is threaded into every
chunk so a non-English clone holds its language (#505 B2). ``opts`` (#1208)
carries the expressive knobs; a default instance reproduces today exactly."""
@@ -695,103 +681,6 @@ def _render_chapter_cached(chapter, synth, sr, engine_id, resolve, cache_dir, le
"cached": seg_cache.hits}
def _remote_chapter_call(chapter, *, engine_id, default_voice, voice_map,
language, lexicon, opts, cache_dir):
"""Build one opaque remote chapter task without loading a local TTS model."""
import hashlib
from services import gpu_gateway
from services.text_normalization import normalize_for_tts
from services.watermark import is_enabled as watermark_enabled
rows, voices, refs = [], [], []
for span in chapter.spans:
profile_id = _map_span_voice(span.voice_id, default_voice, voice_map)
voice = _resolve_voice(profile_id)
rows.append({
"text": normalize_for_tts(span.text, language),
"pause_ms_after": span.pause_ms_after,
"speed": getattr(span, "speed", None),
})
refs.append(voice.get("ref_audio"))
voices.append({
"ref_text": voice.get("ref_text"), "instruct": voice.get("instruct"),
"seed": voice.get("seed"),
})
params = {
"spans": rows, "voices": voices, "ref_audio": refs,
"language": language, "lexicon": lexicon,
"expressive": opts.to_manifest(), "watermark": bool(watermark_enabled()),
}
signature = hashlib.sha256(json.dumps(params, sort_keys=True, default=str).encode()).hexdigest()
# The worker synthesizes from ``spans``, but the gateway and scheduler read
# top-level ``text`` to scale the remote execution deadline. Add this after
# the signature so existing content-addressed remote cache keys still hit.
params["text"] = "\n".join(row["text"] for row in rows)
wav_path = os.path.join(cache_dir, f"remote-{signature}.wav")
def decode(result):
import soundfile as sf
if not os.path.exists(wav_path):
partial = f"{wav_path}.part"
shutil.copyfile(result.path, partial)
os.replace(partial, wav_path)
info = sf.info(wav_path)
return wav_path, float(info.duration), False, None
return gpu_gateway.RemoteCall(
engine=engine_id, operation="audiobook", params=params,
idempotency_key=f"audiobook:{signature}", decode=decode,
), wav_path
async def _run_chapter(chapter, *, operation="audiobook", decision, job, default_voice, language, opts,
voice_map, lexicon, cache_dir):
"""Run one chapter through the gateway; local preparation stays lazy."""
from services import gpu_gateway
from services.tts_backend import active_backend_id, get_backend_class
engine_id = active_backend_id()
remote, remote_cache = _remote_chapter_call(
chapter, engine_id=engine_id, default_voice=default_voice,
voice_map=voice_map, language=language, lexicon=lexicon,
opts=opts, cache_dir=cache_dir,
)
if decision.remote and os.path.exists(remote_cache):
import soundfile as sf
info = sf.info(remote_cache)
return remote_cache, float(info.duration), True, None
async def prepare_local():
from services.model_manager import generate_timeout_s
synth, sr, resolve, local_engine = await _prepare_synth(
default_voice, language=language, opts=opts, voice_map=voice_map
)
try:
timeout_engine = get_backend_class(local_engine)
except ValueError:
# Tests and third-party integrations may inject a synth under a
# non-catalogue id. Keep the canonical host/text policy available;
# registered production engines still add their routing metadata.
timeout_engine = None
return gpu_gateway.LocalCall(
fn=lambda: _render_chapter_cached(
chapter, synth, sr, local_engine, resolve, cache_dir, lexicon,
language, opts, voice_map,
),
what="Audiobook chapter",
timeout=generate_timeout_s(
remote.params["text"], engine=timeout_engine
),
)
return await gpu_gateway.run(
operation, local=gpu_gateway.LocalCall(prepare=prepare_local),
remote=remote, decision=decision, job=job,
)
class AudiobookPreviewRequest(ExpressiveMixin):
text: str
chapter_index: int = 0
@@ -811,7 +700,7 @@ async def audiobook_preview(req: AudiobookPreviewRequest) -> dict:
cache (the later full render reuses it) and a re-preview is instant.
"""
from core.config import OUTPUTS_DIR
from services import gpu_gateway
from services.model_manager import _gpu_pool
plan = parse_audiobook_script(req.text, default_voice=req.default_voice)
if not plan.chapters:
@@ -825,11 +714,16 @@ async def audiobook_preview(req: AudiobookPreviewRequest) -> dict:
os.makedirs(cache_dir, exist_ok=True)
resolved_lang = _resolve_default_language(req.language, req.default_voice)
opts = _expressive_opts(req)
decision = gpu_gateway.decide("audiobook")
wav_path, dur, was_cached, _seg_stats = await _run_chapter(
chapter, decision=decision, job=None, default_voice=req.default_voice,
language=resolved_lang, opts=opts, voice_map=req.voice_map,
lexicon=req.lexicon, cache_dir=cache_dir,
synth, sr, resolve, engine_id = await _prepare_synth(
req.default_voice,
language=resolved_lang,
opts=opts,
voice_map=req.voice_map,
)
loop = asyncio.get_running_loop()
wav_path, dur, was_cached, _seg_stats = await loop.run_in_executor(
_gpu_pool, _render_chapter_cached, chapter, synth, sr, engine_id, resolve, cache_dir,
req.lexicon, resolved_lang, opts, req.voice_map,
)
return {
"output": os.path.relpath(wav_path, OUTPUTS_DIR), # served via /audio
@@ -866,9 +760,8 @@ async def _render_longform_sse(
convergence point: one renderer, two front doors.
"""
from core.config import OUTPUTS_DIR
from core.failure import build_failure, build_failure_event
from services.ffmpeg_utils import find_ffmpeg, run_ffmpeg
from services import gpu_gateway
from services.model_manager import _gpu_pool
opts = opts or ExpressiveOptions()
@@ -943,20 +836,19 @@ async def _render_longform_sse(
cache_dir = os.path.join(OUTPUTS_DIR, "longform_cache")
os.makedirs(cache_dir, exist_ok=True)
prune_cache_dir(cache_dir) # bound disk before this job adds its chapters
loop = asyncio.get_running_loop()
try:
resolved_lang = _resolve_default_language(language, default_voice)
operation = "audiobook" if job_type == "audiobook" else "longform"
decision = gpu_gateway.decide(operation)
chapter_run = gpu_gateway.JobRun(operation)
synth, sr, resolve, engine_id = await _prepare_synth(
default_voice, language=resolved_lang, opts=opts, voice_map=voice_map
)
total = len(plan.chapters)
chapter_files: list[str] = []
chapters_meta: list[tuple[str, int]] = []
cached_n = 0
failed: list[int] = []
# Kept so the terminal "all chapters failed" event can name the cause
# instead of restating the symptom (#1321).
last_chapter_exc: Exception | None = None
interrupted = False
yield _emit({"type": "started", "job_id": job_id, "chapters": total})
@@ -981,34 +873,17 @@ async def _render_longform_sse(
interrupted = True
break
try:
wav_path, dur, was_cached, seg_stats = await _run_chapter(
chapter, operation=operation, decision=decision, job=chapter_run,
default_voice=default_voice, language=resolved_lang,
opts=opts, voice_map=voice_map, lexicon=lexicon,
cache_dir=cache_dir,
wav_path, dur, was_cached, seg_stats = await loop.run_in_executor(
_gpu_pool, _render_chapter_cached,
chapter, synth, sr, engine_id, resolve, cache_dir, lexicon,
resolved_lang, opts, voice_map,
)
except Exception as e: # isolate a bad chapter — keep going
except Exception: # isolate a bad chapter — keep going
logger.warning("[%s] chapter %d (%s) failed to render",
job_id, i, chapter.title, exc_info=True)
failed.append(i)
# Carry the real reason (#1321). The old event said only
# "chapter failed to render", so a failed chapter was a red row
# and nothing else — the cause existed solely in the backend log,
# which is why the report for this arrived as a bare traceback.
# build_failure guarantees a non-empty reason even for exceptions
# whose str() is empty (a generator-based engine that yields
# nothing raises a bare StopIteration), sanitizes paths/tokens,
# and adds the docs deeplink + hint. `error` stays populated —
# build_failure mirrors reason into it — so older frontends and
# the Stories exporter keep working.
last_chapter_exc = e
yield _emit({"type": "chapter_error", "index": i, "total": total,
"title": chapter.title,
# No env diagnostic per chapter: a book can fail
# hundreds of times and it is identical every time.
# The terminal error below carries one.
**build_failure(e, stage="audiobook_chapter",
include_diagnostic=False)})
"title": chapter.title, "error": "chapter failed to render"})
continue
chapter_files.append(wav_path)
chapters_meta.append((chapter.title, int(round(dur * 1000))))
@@ -1023,11 +898,6 @@ async def _render_longform_sse(
ev["cached_segments"] = seg_stats["cached"]
yield _emit(ev)
route_notice = chapter_run.notice()
if route_notice is not None:
yield _emit({"type": "routing_notice", "status": route_notice[0],
"reason": route_notice[1]})
if interrupted:
logger.info("[%s] client disconnected — stopped after %d/%d chapters",
job_id, len(chapter_files), total)
@@ -1050,32 +920,7 @@ async def _render_longform_sse(
return
if not chapter_files:
# Every chapter failed, so the render is over — this is the event the
# UI turns into a toast, and it used to carry only the symptom
# (#1321). Lead with the summary, then the cause; docs_topic/hint are
# classified from the raw exception text, so prefixing the reason
# afterwards cannot mis-route the deeplink.
if last_chapter_exc is not None:
ev = build_failure_event(last_chapter_exc, stage="audiobook_render")
ev["reason"] = f"all {total} chapters failed to render — {ev['reason']}"
ev["error"] = ev["reason"]
else:
ev = {"type": "error", "error": "all chapters failed to render",
"reason": "all chapters failed to render"}
# Terminal failure — record it. This branch used to return without
# touching job history, so the row stayed `running` forever: the next
# startup read it as an interrupted job, and the retained manifest
# offered a render that had already failed every chapter as
# resumable (Greptile P1 on #1321). The manifest IS kept on purpose —
# a failure whose cause the user can now see (a missing voice, an
# engine that can't read the script) is worth retrying once fixed,
# and the chapter cache is empty here so a retry costs nothing extra.
if job_store is not None:
try:
job_store.mark_failed(job_id, ev["reason"])
except Exception:
pass # best-effort job history; never block the stream
yield _emit(ev)
yield _emit({"type": "error", "error": "all chapters failed to render"})
return
yield _emit({"type": "assembling"})
@@ -1143,25 +988,6 @@ async def _render_longform_sse(
yield _emit({"type": "error", "error": "render failed (see backend log)"})
async def _public_longform_stream(plan, **render_kwargs):
"""Keep generator diagnostics local if setup fails before its own guard."""
try:
async for event in _render_longform_sse(plan, **render_kwargs):
yield event
except asyncio.CancelledError:
raise
except Exception as exc:
from core.public_errors import public_failure
error = public_failure(
logger,
"Longform response stream failed",
exc,
response="Render failed; check the backend log for details.",
)
yield f"data: {json.dumps({'type': 'error', 'error': error})}\n\n"
@router.post("/audiobook")
async def audiobook_synthesize(req: AudiobookRequest, request: Request = None):
"""Synthesize a chapterized audiobook from a script, streaming SSE progress."""
@@ -1170,7 +996,7 @@ async def audiobook_synthesize(req: AudiobookRequest, request: Request = None):
# to a direct in-process call, e.g. a unit test); its disconnect poll is what
# lets Stop cancel the render mid-book (#1216).
return StreamingResponse(
_public_longform_stream(
_render_longform_sse(
plan, default_voice=req.default_voice, language=req.language,
fmt=req.format, bitrate=req.bitrate,
loudness=req.loudness, cover_path=req.cover_path, metadata=req.metadata,
@@ -1231,7 +1057,7 @@ async def longform_render(req: LongformRenderRequest, request: Request = None):
chapters.append(Chapter(title=c.title or f"Chapter {i + 1}", spans=spans))
plan = AudiobookPlan(chapters=chapters)
return StreamingResponse(
_public_longform_stream(
_render_longform_sse(
plan, default_voice=req.default_voice, language=req.language,
fmt=req.format, bitrate=req.bitrate,
loudness=req.loudness, cover_path=req.cover_path, metadata=req.metadata,
@@ -1324,7 +1150,7 @@ async def resume_longform(job_id: str, request: Request = None):
# unrendered ones synthesize. Using a fresh id means the request's job_id
# never names a work dir / output file (defence-in-depth path-injection).
return StreamingResponse(
_public_longform_stream(
_render_longform_sse(
plan, default_voice=p.get("default_voice"), language=p.get("language"),
fmt=p.get("fmt", "m4b"), bitrate=p.get("bitrate", "128k"),
loudness=p.get("loudness"), cover_path=p.get("cover_path"),
-231
View File
@@ -1,231 +0,0 @@
"""Short-lived credentials for the first-party remote administration UI."""
from __future__ import annotations
import math
import threading
import time
from collections import OrderedDict, deque
from collections.abc import Callable
from datetime import UTC, datetime
from typing import Literal
from fastapi import APIRouter, HTTPException, Request, Response
from fastapi.responses import JSONResponse
from pydantic import BaseModel
from core.auth import (
CredentialTransport,
PrincipalKind,
authorization_credential_present,
legacy_master_cookie_valid,
master_header_valid,
principal_for,
remote_api_key,
)
from core.csrf import cookie_csrf_allowed, effective_scheme
from services.admin_sessions import (
SESSION_TTL_SECONDS,
WS_TICKET_TTL_SECONDS,
admin_session_store,
)
router = APIRouter(prefix="/api/auth", tags=["auth"])
_FAILED_EXCHANGE_LIMIT = 10
_FAILED_EXCHANGE_WINDOW_SECONDS = 60
_MAX_TRACKED_CLIENTS = 1024
class _ExchangeAttemptLimiter:
"""Bounded per-client sliding window for failed pre-auth exchanges."""
def __init__(
self,
*,
monotonic: Callable[[], float] = time.monotonic,
limit: int = _FAILED_EXCHANGE_LIMIT,
window_seconds: int = _FAILED_EXCHANGE_WINDOW_SECONDS,
max_clients: int = _MAX_TRACKED_CLIENTS,
) -> None:
if limit <= 0 or window_seconds <= 0 or max_clients <= 0:
raise ValueError("rate-limit bounds must be positive")
self._monotonic = monotonic
self._limit = limit
self._window_seconds = window_seconds
self._max_clients = max_clients
self._attempts: OrderedDict[str, deque[float]] = OrderedDict()
self._lock = threading.Lock()
def register_failure(self, client_id: str) -> int | None:
now = self._monotonic()
cutoff = now - self._window_seconds
with self._lock:
failures = self._attempts.setdefault(client_id, deque())
while failures and failures[0] <= cutoff:
failures.popleft()
self._attempts.move_to_end(client_id)
while len(self._attempts) > self._max_clients:
self._attempts.popitem(last=False)
if len(failures) >= self._limit:
return max(
1,
math.ceil(self._window_seconds - (now - failures[0])),
)
failures.append(now)
return None
def clear(self, client_id: str) -> None:
with self._lock:
self._attempts.pop(client_id, None)
def reset(self) -> None:
with self._lock:
self._attempts.clear()
_exchange_attempt_limiter = _ExchangeAttemptLimiter()
class SessionRequest(BaseModel):
transport: Literal["cookie", "bearer"]
class WebSocketTicketRequest(BaseModel):
path: str
def _secure_cookie(request: Request) -> bool:
# Same effective-scheme logic as the exact-origin CSRF check: the resolved
# scope first (uvicorn's trusted-proxy rewrite), upgraded — never
# downgraded — by X-Forwarded-Proto for TLS-terminating proxies uvicorn
# doesn't trust (Tailscale Serve into Docker, etc.). Spoofing the header on
# a plain-http hop can only ADD the Secure flag, which fails safe: the
# browser drops such a cookie, so the spoofer only breaks their own
# session. See core.csrf.effective_scheme for the full analysis.
return effective_scheme(request) == "https"
def _set_session_cookie(response: Response, request: Request, token: str, expires_at: float) -> None:
response.set_cookie(
"ov_session",
token,
max_age=SESSION_TTL_SECONDS,
expires=datetime.fromtimestamp(expires_at, tz=UTC),
path="/",
secure=_secure_cookie(request),
httponly=True,
samesite="strict",
)
def _expire_cookie(response: Response, request: Request, name: str) -> None:
response.delete_cookie(
name,
path="/",
secure=_secure_cookie(request),
httponly=name == "ov_session",
samesite="strict",
)
def _client_id(request: Request) -> str:
host = request.client.host if request.client else "unknown"
return str(host).strip().lower()[:255] or "unknown"
def _reject_master_exchange(request: Request) -> None:
retry_after = _exchange_attempt_limiter.register_failure(_client_id(request))
if retry_after is not None:
raise HTTPException(
status_code=429,
detail="Too many authentication attempts",
headers={"Retry-After": str(retry_after)},
)
raise HTTPException(status_code=401, detail="API key required")
@router.post("/session")
def create_session(payload: SessionRequest, request: Request) -> Response:
configured = remote_api_key()
if not configured:
raise HTTPException(status_code=401, detail="API key required")
authorization_present = authorization_credential_present(request)
header_authorized = master_header_valid(request)
legacy_authorized = legacy_master_cookie_valid(request)
migrating_legacy = False
if authorization_present:
if not header_authorized:
_reject_master_exchange(request)
elif legacy_authorized:
if payload.transport != "cookie" or not cookie_csrf_allowed(request):
raise HTTPException(status_code=403, detail="browser origin rejected")
migrating_legacy = True
else:
_reject_master_exchange(request)
_exchange_attempt_limiter.clear(_client_id(request))
issued = admin_session_store.issue(configured)
if payload.transport == "bearer":
return JSONResponse(
{
"token": issued.token,
"expires_at": issued.expires_at,
"expires_in": SESSION_TTL_SECONDS,
},
status_code=201,
)
response = Response(status_code=204)
_set_session_cookie(response, request, issued.token, issued.expires_at)
if migrating_legacy or request.cookies.get("ov_key"):
_expire_cookie(response, request, "ov_key")
return response
@router.delete("/session", status_code=204)
def delete_session(request: Request) -> Response:
principal = principal_for(request)
if principal.kind is PrincipalKind.ADMIN_SESSION:
if (
principal.transport is CredentialTransport.COOKIE
and not cookie_csrf_allowed(request)
):
raise HTTPException(status_code=403, detail="browser origin rejected")
admin_session_store.revoke_by_credential(principal.credential_id)
response = Response(status_code=204)
_expire_cookie(response, request, "ov_session")
return response
@router.post("/ws-ticket")
def create_ws_ticket(payload: WebSocketTicketRequest, request: Request) -> JSONResponse:
principal = principal_for(request)
if principal.kind is not PrincipalKind.ADMIN_SESSION:
raise HTTPException(status_code=403, detail="admin session required")
if (
principal.transport is CredentialTransport.COOKIE
and not cookie_csrf_allowed(request)
):
raise HTTPException(status_code=403, detail="browser origin rejected")
try:
ticket = admin_session_store.issue_ws_ticket_for_credential(
principal.credential_id,
payload.path,
remote_api_key(),
)
except ValueError as exc:
raise HTTPException(status_code=422, detail=str(exc)) from None
except PermissionError:
raise HTTPException(status_code=401, detail="admin session required") from None
return JSONResponse(
{
"ticket": ticket.token,
"expires_at": ticket.expires_at,
"expires_in": WS_TICKET_TTL_SECONDS,
},
status_code=201,
)
+89 -612
View File
@@ -9,8 +9,6 @@ the SQLite `jobs` table for history, but the queue itself restarts empty
on backend restart intentional, since GPU jobs can't be safely resumed.
"""
import os
import json
import shutil
import uuid
import time
import asyncio
@@ -18,42 +16,18 @@ import logging
from typing import Optional, List
from fastapi import APIRouter, File, UploadFile, HTTPException, Form
from fastapi.responses import JSONResponse
from pydantic import BaseModel
from core.config import DATA_DIR
from core import failure
from core.logging_utils import log_safe
from core.file_cleanup import FileCleanupError, unlink_if_present
from services.dub_batching import (
BATCH_WIDTH_ENV,
batch_timeout_s as _batch_timeout_s,
native_batch_width as _native_batch_width,
)
from services import gpu_gateway
from services.segment_bundle import extract_segment_wavs, remove_segment_wavs
from services.tts_backend import active_backend_id, resolve_generation_backend
router = APIRouter()
logger = logging.getLogger("omnivoice.batch")
# Compatibility values emitted by the established Tauri Batch picker. They
# are taxonomy tokens, not arbitrary prose, and are resolved server-side so
# native watch-folder uploads and both desktop clients use the same voice.
_BATCH_PRESET_INSTRUCT = {
"narrator": "male, middle-aged, low pitch, british accent",
"excited_child": "child, high pitch",
"anxious_whisper": "young adult, whisper",
"surprised_woman": "female, young adult, high pitch",
"elderly_story": "male, elderly, very low pitch",
"sichuan": "female, young adult, moderate pitch, \u56db\u5ddd\u8bdd",
}
# ── In-memory queue ─────────────────────────────────────────────────────
_queue: asyncio.Queue = None # Lazily initialised
_worker_task: asyncio.Task = None # Background consumer
_processing_job_ids: set[str] = set()
_jobs: dict = {} # job_id → status dict
@@ -64,15 +38,11 @@ class BatchJobStatus(BaseModel):
langs: List[str]
voice_id: Optional[str] = None
preserve_bg: bool = True
translation_provider: Optional[str] = None
created_at: float
started_at: Optional[float] = None
finished_at: Optional[float] = None
error: Optional[str] = None
progress: Optional[dict] = None
attempts: int = 1
retry_ready: bool = True
setup_required: Optional[dict] = None
def _ensure_queue():
@@ -94,7 +64,6 @@ async def _worker():
job["status"] = "running"
job["started_at"] = time.time()
_processing_job_ids.add(job_id)
logger.info("Batch job %s starting: %s", job_id, job["filename"])
try:
@@ -124,9 +93,6 @@ async def _worker():
job["finished_at"] = time.time()
logger.error("Batch job %s failed: %s", job_id, e, exc_info=True)
finally:
_processing_job_ids.discard(job_id)
if job["status"] == "cancelled":
job["retry_ready"] = True
_queue.task_done()
@@ -135,136 +101,6 @@ def _set_progress(job, stage, percent=0, **extra):
job["progress"] = {"stage": stage, "percent": percent, **extra}
#: Override for the native dub batch width. Set to 1 to disable batching.
#: Hard ceiling on the override — a batch this wide is already amortizing
#: almost all of the per-call setup, and beyond it the failure mode is an OOM
#: that costs more than the saving.
# Bound each allocation while persisting multipart uploads. Video inputs can
# be many gigabytes; `await UploadFile.read()` with no size used to mirror the
# entire file in process memory before writing it back out.
_UPLOAD_CHUNK_BYTES = 1024 * 1024
_REMOTE_BATCH_OPERATION = "batch_segments"
async def _resolve_batch_execution(voice: dict):
"""Resolve Batch's TTS target without loading local weights remotely."""
engine_id = active_backend_id()
decision = gpu_gateway.decide("batch")
if decision.remote:
await gpu_gateway.preflight(
engine_id,
decision,
operation=_REMOTE_BATCH_OPERATION,
)
return engine_id, decision, None
backend = await resolve_generation_backend(
require_cloning=voice["requires_cloning"],
cloning_purpose="this batch job's pinned voice",
)
return engine_id, decision, backend
def _decode_remote_batch(
result: gpu_gateway.RemoteResult,
batch_dir: str,
expected: set[int],
) -> tuple[dict[int, str], int]:
"""Validate and unpack one worker result before accepting remote success."""
import soundfile as sf
target = os.path.join(batch_dir, ".remote", result.task_id)
paths = extract_segment_wavs(result.path or "", target)
try:
if set(paths) != expected:
missing = sorted(expected - set(paths))
extra = sorted(set(paths) - expected)
raise ValueError(
f"segment bundle mismatch (missing={missing}, extra={extra})"
)
rates = {int(sf.info(path).samplerate) for path in paths.values()}
if len(rates) != 1 or next(iter(rates), 0) <= 0:
raise ValueError("segment bundle has inconsistent sample rates")
return paths, rates.pop()
except BaseException:
remove_segment_wavs(paths)
raise
async def _save_upload(upload: UploadFile, destination: str) -> None:
try:
with open(destination, "wb") as output:
while chunk := await upload.read(_UPLOAD_CHUNK_BYTES):
output.write(chunk)
except BaseException:
try:
unlink_if_present(destination)
except FileCleanupError:
logger.warning("Could not remove incomplete batch upload", exc_info=True)
raise
def _batch_voice(voice_id: str | None) -> dict:
"""Resolve one queue-wide voice into concrete generation inputs.
Clone profiles contribute their reference; designed profiles contribute
their healed instruction and seed. Legacy ``preset:`` selections become
the same instruction used by Dubbing instead of falling through to the
engine default.
"""
resolved = {
"ref_audio": None,
"ref_text": None,
"instruct": "",
"seed": None,
"requires_cloning": False,
}
if not voice_id:
return resolved
if voice_id.startswith("preset:"):
preset_id = voice_id.removeprefix("preset:")
instruct = _BATCH_PRESET_INSTRUCT.get(preset_id)
if instruct is None:
raise ValueError("That built-in voice preset no longer exists")
from omnivoice.utils.voice_design import sanitize_instruct
resolved["instruct"] = sanitize_instruct(instruct)
return resolved
from core.config import VOICES_DIR
from core.db import db_conn
with db_conn() as conn:
row = conn.execute(
"SELECT * FROM voice_profiles WHERE id=?",
(voice_id,),
).fetchone()
if row is None:
raise ValueError("That saved voice no longer exists")
if row["kind"] == "design":
from omnivoice.utils.voice_design import heal_design_instruct
resolved["instruct"] = heal_design_instruct(row["instruct"], row["vd_states"])
resolved["seed"] = int(row["seed"]) if row["seed"] is not None else None
return resolved
relative = row["locked_audio_path"] if row["is_locked"] else row["ref_audio_path"]
if not relative:
raise ValueError("That saved voice has no reference audio")
ref_audio = os.path.join(VOICES_DIR, relative)
if not os.path.isfile(ref_audio):
raise ValueError("That saved voice's reference audio is missing")
resolved.update({
"ref_audio": ref_audio,
"ref_text": row["ref_text"],
"requires_cloning": True,
})
return resolved
async def _run_batch_pipeline(job_id: str, job: dict):
"""Full batch dub pipeline: extract → transcribe → translate → generate → mix → export."""
import subprocess
@@ -313,17 +149,14 @@ async def _run_batch_pipeline(job_id: str, job: dict):
# ── 2. Transcribe ─────────────────────────────────────────────────
_set_progress(job, "transcribe", 0)
from services.asr_backend import load_active_asr_backend
from services.asr_backend import get_active_asr_backend
from services.model_manager import _gpu_pool, _cpu_pool, run_on_gpu_pool_guarded
from services.segmentation import (
segment_transcript, assign_speakers_heuristic,
)
def _transcribe():
# `load_*`, not `get_*`: the plain selector returns engines whose
# shallow probe passed but whose deep import chain is broken, failing
# the whole batch job at `.transcribe()` instead of degrading (#1185).
backend = load_active_asr_backend()
backend = get_active_asr_backend()
result = backend.transcribe(audio_path, word_timestamps=True)
detected_lang = result.get("language", "en")
segments = segment_transcript(result, duration=duration)
@@ -355,30 +188,19 @@ async def _run_batch_pipeline(job_id: str, job: dict):
return
# ── Engine resolution (issue #312 class) ────────────────────────────
# Batch used to hardcode VoiceStudio regardless of the engine selected in
# Model Catalogue. Clone profiles require a cloning-capable engine; presets
# and designed voices use instruction mode. Resolve once for the whole job.
# Batch used to hardcode OmniVoice via get_model() regardless of the
# engine selected in Settings → Engines. require_cloning only when a
# specific voice is pinned (job["voice_id"]) — an unpinned job is fine on
# any active engine. Resolved ONCE for the whole job (every language
# below shares the same active engine); an uncaught ValueError here
# propagates to _worker()'s existing except-Exception handling, which
# already records a structured job failure via core.failure.build_failure.
voice = _batch_voice(job.get("voice_id"))
engine_id, execution_target, backend = await _resolve_batch_execution(voice)
sr = backend.sample_rate if backend is not None else 0
from services.performance_profiles import tts_defaults
_profile_defaults = tts_defaults(engine_id)
_batch_num_step = _profile_defaults.get("num_step", 16)
_batch_postprocess = _profile_defaults.get("postprocess_output", True)
batch_run = gpu_gateway.JobRun("batch")
async def _prepare_local_batch() -> gpu_gateway.LocalCall:
nonlocal backend, sr
if backend is None:
backend = await resolve_generation_backend(
require_cloning=voice["requires_cloning"],
cloning_purpose="this batch job's pinned voice",
)
sr = backend.sample_rate
return gpu_gateway.LocalCall(fn=lambda: None, what="Batch TTS fallback")
from services.tts_backend import resolve_generation_backend
backend = await resolve_generation_backend(
require_cloning=bool(job.get("voice_id")),
cloning_purpose="this batch job's pinned voice",
)
sr = backend.sample_rate
# ── 3. Translate + Generate per language ───────────────────────────
total_langs = len(langs)
@@ -397,65 +219,40 @@ async def _run_batch_pipeline(job_id: str, job: dict):
translated_segments = list(segments) # copy
if target_lang != source_lang:
# Use the same provider dispatch as interactive Dubbing. The old
# batch-only implementation hardcoded Google and silently kept the
# source text on failure, which could make an English track labelled
# "es" while also sending text online despite an offline selection.
from api.routers.dub_translate import dub_translate
from schemas.requests import TranslateRequest
from core import prefs
provider = job.get("translation_provider") or prefs.get("translation_backend", "argos")
translation = await dub_translate(TranslateRequest(
segments=[
{
"id": str(segment["id"]),
"text": segment.get("text", ""),
"start": segment.get("start"),
"end": segment.get("end"),
try:
def _translate_batch(segs, src, tgt):
"""Translate segment texts via Google Translate."""
from deep_translator import GoogleTranslator
TRANSLATE_CODES = {
"en": "en", "es": "es", "fr": "fr", "de": "de",
"it": "it", "pt": "pt", "ru": "ru", "ja": "ja",
"ko": "ko", "zh": "zh-CN", "ar": "ar", "hi": "hi",
"tr": "tr", "pl": "pl", "nl": "nl", "sv": "sv",
}
for segment in segments
],
source_lang=source_lang,
target_lang=target_lang,
provider=provider,
quality="fast",
))
if isinstance(translation, JSONResponse):
try:
payload = json.loads(translation.body)
detail = payload.get("error") or payload.get("detail")
if payload.get("code") == "argos_pack_missing":
job["setup_required"] = {
"kind": "argos_packs",
"source_lang": source_lang,
"target_langs": [
pair["target_lang"]
for pair in payload.get("pairs", [])
if isinstance(pair, dict) and pair.get("target_lang")
],
}
except Exception: # noqa: BLE001 — retain the stable fallback
detail = None
raise RuntimeError(
detail or f"{provider} could not translate this batch"
src_code = TRANSLATE_CODES.get(src, src) or "auto"
tgt_code = TRANSLATE_CODES.get(tgt, tgt)
translator = GoogleTranslator(source=src_code, target=tgt_code)
out = []
for s in segs:
s_copy = dict(s)
text = s.get("text", "").strip()
if text:
try:
s_copy["text"] = translator.translate(text) or text
except Exception as e:
logger.warning("Translate seg failed: %s", e)
out.append(s_copy)
return out
translated_segments = await loop.run_in_executor(
_cpu_pool, _translate_batch,
segments, source_lang, target_lang,
)
rows = {
str(row.get("id")): row
for row in translation.get("translated", [])
if isinstance(row, dict)
}
failed = [row for row in rows.values() if row.get("error")]
if failed or len(rows) != len(segments):
raise RuntimeError(
f"{provider} translation failed for "
f"{len(failed) or len(segments) - len(rows)} segment(s)"
)
translated_segments = [
{**segment, "text": rows[str(segment["id"])]["text"]}
for segment in segments
]
except ImportError:
logger.warning("deep_translator not installed, skipping translation for %s", target_lang)
except Exception as e:
logger.warning("Translation failed for %s: %s, using original", target_lang, e)
translated_segments = segments
if job["status"] == "cancelled":
return
@@ -473,194 +270,12 @@ async def _run_batch_pipeline(job_id: str, job: dict):
from services.audio_io import atomic_save_wav
import torch
remote_segments: dict[int, str] = {}
valid_rows = [
(i, segment)
for i, segment in enumerate(translated_segments)
if segment.get("end", 0) - segment.get("start", 0) > 0.05
and segment.get("text", "").strip()
]
if execution_target.remote and valid_rows:
remote_rows = [
{
"index": i,
"text": segment.get("text", "").strip(),
"language": target_lang,
"ref_text": voice["ref_text"],
"instruct": voice["instruct"] or None,
"duration": segment.get("end", 0) - segment.get("start", 0),
"num_step": _batch_num_step,
"postprocess_output": _batch_postprocess,
"guidance_scale": 2.0,
"speed": 1.0,
"effect_preset": "batch",
"seed": (
voice["seed"] + i if voice["seed"] is not None else None
),
# The assembled track receives one watermark below. Marking
# each line here would double-process remote output.
"watermark": False,
}
for i, segment in valid_rows
]
expected = {row["index"] for row in remote_rows}
def _remote_state(state: dict) -> None:
fraction = max(0.0, min(1.0, float(state.get("progress") or 0.0)))
_set_progress(
job,
"generate",
percent=int(((lang_idx + fraction) / total_langs) * 100),
current_lang=target_lang,
current_segment=min(len(remote_rows), round(fraction * len(remote_rows))),
total_segments=len(remote_rows),
execution_target=execution_target.label,
execution_phase=state.get("phase"),
)
route_task = asyncio.create_task(
gpu_gateway.run(
"batch",
local=gpu_gateway.LocalCall(prepare=_prepare_local_batch),
remote=gpu_gateway.RemoteCall(
engine=engine_id,
operation=_REMOTE_BATCH_OPERATION,
params={
"segments": remote_rows,
"ref_audio": [voice["ref_audio"] for _ in remote_rows],
"input_seconds": sum(
float(row.get("duration") or 0.0) for row in remote_rows
),
},
idempotency_key=f"batch:{job_id}:{target_lang}",
decode=lambda result: _decode_remote_batch(
result, batch_dir, expected
),
),
decision=execution_target,
job=batch_run,
on_state=_remote_state,
)
)
while not route_task.done():
await asyncio.wait({route_task}, timeout=0.25)
if job["status"] == "cancelled":
route_task.cancel()
try:
await route_task
except asyncio.CancelledError:
pass
return
routed = route_task.result()
if routed is not None:
remote_segments, sr = routed
# A remote-only empty transcript still needs a valid silent-track rate.
sr = sr or 24_000
total_samples = int(duration * sr)
full_audio = torch.zeros(1, total_samples)
total_segs = len(translated_segments)
# Native engines can amortize encoder/decoder setup across a small
# batch. Keep the adapter seam optional: engines without a real batch
# implementation inherit TTSBackend.generate_batch(), which preserves
# the established one-segment behavior below.
from services.tts_backend import TTSBackend
batched_audio: dict[int, torch.Tensor] = {}
has_native_batch = (
backend is not None
and type(backend).generate_batch is not TTSBackend.generate_batch
)
if has_native_batch:
from services.text_normalization import normalize_for_tts
batch_ref_audio = voice["ref_audio"]
batch_ref_text = voice["ref_text"]
batch_width = _native_batch_width(backend)
async def _prefetch_batch(first_index: int) -> None:
"""Render the batch beginning at ``first_index`` into
``batched_audio``.
Rendered on demand rather than prerendering the whole track:
the tensors are popped as they are placed, so peak host memory
is one batch instead of every segment of the language and
the progress bar tracks placement instead of running to the
end and restarting at segment 1.
"""
if job["status"] == "cancelled":
return
batch_rows = []
index = first_index
while index < total_segs and len(batch_rows) < batch_width:
seg = translated_segments[index]
if (seg.get("end", 0) - seg.get("start", 0) > 0.05
and seg.get("text", "").strip()):
batch_rows.append((index, seg))
index += 1
if len(batch_rows) < 2:
return # nothing to amortize — the per-segment path is equal
batch_indices = [index for index, _ in batch_rows]
batch_texts = [
normalize_for_tts(row.get("text", "").strip(), target_lang)
for _, row in batch_rows
]
batch_durations = [
row.get("end", 0) - row.get("start", 0)
for _, row in batch_rows
]
def _render_native_batch():
if voice["seed"] is not None:
torch.manual_seed(voice["seed"])
generated = backend.generate_batch(
batch_texts,
language=target_lang,
ref_audio=batch_ref_audio,
ref_text=batch_ref_text,
instruct=voice["instruct"] or None,
duration=batch_durations,
num_step=_batch_num_step,
guidance_scale=2.0,
speed=1.0,
denoise=True,
postprocess_output=_batch_postprocess,
)
if len(generated) != len(batch_indices):
raise RuntimeError(
f"native batch returned {len(generated)} outputs for "
f"{len(batch_indices)} segments"
)
rendered = []
for audio_out in generated:
if not getattr(backend, "applies_own_mastering", False):
audio_out = apply_mastering(audio_out, sample_rate=sr)
rendered.append(normalize_audio(audio_out, target_dBFS=-2.0))
return rendered
try:
rendered = await run_on_gpu_pool_guarded(
_render_native_batch,
what="Batch generate",
timeout=_batch_timeout_s(batch_texts, backend),
)
batched_audio.update(zip(batch_indices, rendered))
except TimeoutError:
# Do not immediately queue the same expensive work again:
# the timed-out pool task may still be holding the device.
raise
except Exception as e:
logger.warning(
"Native TTS batch failed for segments %s-%s; falling back per segment: %s",
batch_indices[0] + 1,
batch_indices[-1] + 1,
e,
)
for i, seg in enumerate(translated_segments):
if job["status"] == "cancelled":
remove_segment_wavs(remote_segments)
return
_set_progress(
@@ -687,18 +302,32 @@ async def _run_batch_pipeline(job_id: str, job: dict):
from services.text_normalization import normalize_for_tts
text = normalize_for_tts(text, lang)
ref_audio = None
ref_text = None
# Use voice_id if provided
if job.get("voice_id"):
from core.db import db_conn
from core.config import VOICES_DIR as _VD
with db_conn() as conn:
row = conn.execute(
"SELECT * FROM voice_profiles WHERE id=?",
(job["voice_id"],),
).fetchone()
if row:
if row["is_locked"] and row["locked_audio_path"]:
ref_audio = os.path.join(_VD, row["locked_audio_path"])
elif row["ref_audio_path"]:
ref_audio = os.path.join(_VD, row["ref_audio_path"])
ref_text = row.get("ref_text")
try:
if backend is None:
raise RuntimeError("the local TTS fallback was not prepared")
if voice["seed"] is not None:
torch.manual_seed(voice["seed"] + i)
audio_out = backend.generate(
text=text, language=lang,
ref_audio=voice["ref_audio"], ref_text=voice["ref_text"],
instruct=voice["instruct"] or None,
duration=dur, num_step=_batch_num_step,
ref_audio=ref_audio, ref_text=ref_text,
duration=dur, num_step=16,
guidance_scale=2.0, speed=1.0,
denoise=True, postprocess_output=_batch_postprocess,
denoise=True, postprocess_output=True,
)
if not getattr(backend, "applies_own_mastering", False):
audio_out = apply_mastering(audio_out, sample_rate=sr)
@@ -722,43 +351,10 @@ async def _run_batch_pipeline(job_id: str, job: dict):
# Budget is the shared length-scaled one (#1190): a long segment
# on CPU-class hardware no longer dies on the flat 300s.
from services.model_manager import generate_timeout_s
remote_path = remote_segments.pop(i, None)
if remote_path is not None:
import soundfile as sf
try:
audio_array, remote_sr = sf.read(
remote_path,
dtype="float32",
always_2d=True,
)
if int(remote_sr) != sr:
raise ValueError(
f"remote segment sample rate changed from {sr} to {remote_sr}"
)
audio_tensor = torch.from_numpy(audio_array.T).mean(
dim=0,
keepdim=True,
)
finally:
remove_segment_wavs({i: remote_path})
else:
if backend is None:
await _prepare_local_batch()
# This path means a validated remote bundle lost a row
# after dispatch. Recover only that row; native batches
# were not planned for this language.
has_native_batch = False
if has_native_batch and i not in batched_audio:
await _prefetch_batch(i)
if i in batched_audio:
audio_tensor = batched_audio.pop(i)
else:
audio_tensor = await run_on_gpu_pool_guarded(
_gen,
what="Batch generate",
timeout=generate_timeout_s(seg_text, engine=backend),
)
audio_tensor = await run_on_gpu_pool_guarded(
_gen, what="Batch generate",
timeout=generate_timeout_s(seg_text),
)
# Fit to slot
target_samples_seg = int(seg_duration * sr)
@@ -806,23 +402,25 @@ async def _run_batch_pipeline(job_id: str, job: dict):
f"left silent: {e}"
)
remove_segment_wavs(remote_segments)
# ── 3c. Save dubbed audio track ───────────────────────────────
# Invisible provenance mark on the assembled track (#1169), tensor
# stage, before the WAV write / aac mux — batch dubs used to ship
# unmarked while the interactive dub pipeline marked every segment.
# One whole-track embed (chunked internally, #1045) is equivalent to
# dub_generate's per-segment marks: the 16-bit message repeats
# throughout. Never raises (degrades to unmarked on failure, same as
# every producer).
# throughout. Runs in the GPU pool like generate's finalize; never
# raises (degrades to unmarked on failure, same as every producer).
# Dispatched to the dedicated watermark pool, not the GPU pool (#1190):
# AudioSeal embedding is CPU work that holds no VRAM, and a whole-track
# embed is long enough that occupying a GPU worker with it stalled the
# next language's segments on 1-worker hosts.
from services.watermark import mark_synthetic_async
full_audio = await mark_synthetic_async(
full_audio, sr, context="batch.dub_track",
from services.watermark import mark_synthetic
from services.model_manager import get_watermark_pool
import functools
full_audio = await loop.run_in_executor(
get_watermark_pool(),
functools.partial(mark_synthetic, full_audio, sr,
context="batch.dub_track"),
)
# Same assembly pattern as dub_generate.py:390 — `full_audio` is a
@@ -874,7 +472,6 @@ async def _run_batch_pipeline(job_id: str, job: dict):
outputs[target_lang] = output_path
job["outputs"] = outputs
job.pop("setup_required", None)
_set_progress(job, "done", 100)
@@ -886,7 +483,6 @@ async def enqueue_batch_job(
langs: str = Form("es"), # comma-separated lang codes
voice_id: Optional[str] = Form(None),
preserve_bg: bool = Form(True),
translation_provider: Optional[str] = Form(None),
):
"""Enqueue a video for batch dubbing.
@@ -900,14 +496,6 @@ async def enqueue_batch_job(
if not lang_list:
raise HTTPException(400, "At least one target language is required")
# Validate the snapshot before persisting a potentially large upload.
# Resolve it again in the worker so deleting or editing a queued profile
# cannot silently fall back to the engine's default voice.
try:
await asyncio.to_thread(_batch_voice, voice_id)
except ValueError as exc:
raise HTTPException(status_code=422, detail=str(exc)) from exc
# TTS-only install: no ASR model on disk → typed 409 with a download CTA
# now, instead of accepting the job and having the transcribe stage
# silently auto-download multi-GB whisper weights (or fail) in the worker.
@@ -916,26 +504,15 @@ async def enqueue_batch_job(
if missing is not None:
raise HTTPException(409, {**missing, "message": asr_model_missing_detail(missing)})
# Snapshot the selected translation engine when the user enqueues the job,
# so a later Settings change cannot alter work already waiting in the queue.
from core import prefs
from services import translation_engines
provider = translation_provider or prefs.get("translation_backend", "argos")
if not translation_engines.get_engine(provider):
raise HTTPException(400, "Unknown translation engine")
if not translation_engines.is_installed(provider):
raise HTTPException(409, "Install the selected translation engine before adding this batch")
if not translation_engines.is_ready(provider):
raise HTTPException(409, "Configure the selected translation provider before adding this batch")
# Save the uploaded video
batch_dir = os.path.join(DATA_DIR, "batch")
os.makedirs(batch_dir, exist_ok=True)
ext = os.path.splitext(video.filename or "video.mp4")[1] or ".mp4"
video_path = os.path.join(batch_dir, f"{job_id}{ext}")
await _save_upload(video, video_path)
with open(video_path, "wb") as f:
content = await video.read()
f.write(content)
job = {
"id": job_id,
@@ -945,9 +522,7 @@ async def enqueue_batch_job(
"langs": lang_list,
"voice_id": voice_id,
"preserve_bg": preserve_bg,
"translation_provider": provider,
"created_at": time.time(),
"attempts": 1,
"started_at": None,
"finished_at": None,
"error": None,
@@ -956,10 +531,7 @@ async def enqueue_batch_job(
_jobs[job_id] = job
await _queue.put(job_id)
logger.info(
"Batch job %s enqueued (%d target languages)",
log_safe(job_id), len(lang_list),
)
logger.info("Batch job %s enqueued: %s%s", job_id, video.filename, lang_list)
return {"job_id": job_id, "status": "queued", "queue_position": _queue.qsize()}
@@ -970,8 +542,6 @@ def list_batch_jobs(status: Optional[str] = None, limit: int = 50):
if status:
if status == "active":
jobs = [j for j in jobs if j["status"] in ("queued", "running")]
elif status == "retryable":
jobs = [j for j in jobs if j["status"] in ("failed", "cancelled")]
else:
jobs = [j for j in jobs if j["status"] == status]
jobs.sort(key=lambda j: j["created_at"], reverse=True)
@@ -995,115 +565,22 @@ def cancel_batch_job(job_id: str):
raise HTTPException(404, "Job not found")
if job["status"] in ("done", "failed", "cancelled"):
return {"already": job["status"]}
was_running = job["status"] == "running" or job_id in _processing_job_ids
job["status"] = "cancelled"
job["retry_ready"] = not was_running
job["finished_at"] = time.time()
return {"cancelled": True}
@router.post("/batch/jobs/{job_id}/retry")
async def retry_batch_job(job_id: str):
"""Retry a terminal job using its original app-owned upload and settings."""
job = _jobs.get(job_id)
if not job:
raise HTTPException(404, "Job not found")
if job["status"] not in ("failed", "cancelled"):
raise HTTPException(409, f"Job is {job['status']}, not retryable")
if job_id in _processing_job_ids or not job.get("retry_ready", True):
raise HTTPException(409, "The cancelled job is still stopping")
if not os.path.isfile(job.get("video_path") or ""):
raise HTTPException(409, "The original batch input is no longer available")
try:
await asyncio.to_thread(_batch_voice, job.get("voice_id"))
except ValueError as exc:
raise HTTPException(status_code=422, detail=str(exc)) from exc
from services.asr_backend import asr_model_missing_detail, asr_model_missing_error
missing = await asyncio.to_thread(asr_model_missing_error)
if missing is not None:
raise HTTPException(409, {**missing, "message": asr_model_missing_detail(missing)})
from services import translation_engines
provider = job.get("translation_provider") or "argos"
if not translation_engines.is_ready(provider):
raise HTTPException(409, "Configure the selected translation provider before retrying")
if provider == "argos" and job.get("source_lang"):
try:
status = await asyncio.to_thread(
translation_engines.argos_pack_status,
job["source_lang"],
job["langs"],
)
except ValueError as exc:
raise HTTPException(status_code=422, detail=str(exc)) from exc
if any(not pair["installed"] for pair in status["pairs"]):
raise HTTPException(409, "Install the required Argos language packs before retrying")
batch_root = os.path.realpath(os.path.join(DATA_DIR, "batch"))
output_dir = os.path.realpath(os.path.join(batch_root, job_id))
if os.path.dirname(output_dir) != batch_root:
raise HTTPException(status_code=400, detail="Invalid batch job path")
try:
if os.path.isdir(output_dir):
await asyncio.to_thread(shutil.rmtree, output_dir)
except OSError as exc:
raise HTTPException(
status_code=500,
detail="Could not reset the batch output files. Close any app using them and retry.",
) from exc
for key in (
"duration",
"segments",
"source_lang",
"outputs",
"warnings",
"setup_required",
"retry_ready",
):
job.pop(key, None)
job.update({
"status": "queued",
"started_at": None,
"finished_at": None,
"error": None,
"progress": None,
"attempts": int(job.get("attempts", 1)) + 1,
})
_ensure_queue()
await _queue.put(job_id)
return {"job_id": job_id, "status": "queued", "queue_position": _queue.qsize()}
@router.delete("/batch/jobs/{job_id}")
def delete_batch_job(job_id: str):
"""Delete a batch job record and every app-owned input/output file."""
job = _jobs.get(job_id)
"""Delete a batch job record and its video file."""
job = _jobs.pop(job_id, None)
if not job:
raise HTTPException(404, "Job not found")
if job.get("video_path"):
if job.get("video_path") and os.path.exists(job["video_path"]):
try:
unlink_if_present(job["video_path"])
except FileCleanupError as exc:
raise HTTPException(
status_code=500,
detail="Could not delete the batch video file. Close any app using it and retry.",
) from exc
batch_root = os.path.realpath(os.path.join(DATA_DIR, "batch"))
output_dir = os.path.realpath(os.path.join(batch_root, job_id))
if os.path.dirname(output_dir) != batch_root:
raise HTTPException(status_code=400, detail="Invalid batch job path")
try:
if os.path.isdir(output_dir):
shutil.rmtree(output_dir)
except OSError as exc:
raise HTTPException(
status_code=500,
detail="Could not delete the batch output files. Close any app using them and retry.",
) from exc
_jobs.pop(job_id, None)
os.remove(job["video_path"])
except Exception:
pass
return {"deleted": True}
+17 -122
View File
@@ -8,11 +8,8 @@ raw audio bytes and get back transcribed text immediately. Used by:
The MCP server's future `transcribe_audio` tool
CLI consumers that just want speech-to-text
The ASR engine is whatever `load_active_asr_backend()` returns WhisperX
by default, or MLX Whisper on Apple Silicon when configured. The *loader*,
not the bare selector: it also runs `ensure_loaded()` and falls through to
the next healthy engine when the selected one has a broken deep import chain
(#1185), which the shallow `is_available()` probe cannot see.
The ASR engine is whatever `get_active_asr_backend()` returns WhisperX
by default, or MLX Whisper on Apple Silicon when configured.
"""
from __future__ import annotations
@@ -28,17 +25,6 @@ router = APIRouter()
logger = logging.getLogger("omnivoice.capture")
def _timing(value):
"""A segment timing, or ``None`` when the engine could not determine one.
``dict.get(key, 0)`` hands back a stored ``None`` rather than the default,
because the key is present so rounding it raised and took a transcript
that was otherwise fine down with it (#1904). Pass the null through instead:
the segment list renders whichever half of the range is known.
"""
return round(value, 2) if isinstance(value, (int, float)) else None
def _truthy(value: Optional[str]) -> bool:
"""Parse a multipart form flag. Treats '1'/'true'/'yes'/'on'/'auto'
(any case) as on; everything else including None as off."""
@@ -60,8 +46,7 @@ async def transcribe_audio(
language: Optional language hint (not currently used; auto-detected).
model: Whisper model size (legacy; ignored in dual-mode architecture).
mode: 'fast' (default) uses MLX Turbo for speed; 'accurate' uses
the selected ASR engine with word-level timing. 'reference' uses
the selected ASR engine without word-level timing.
WhisperX with forced alignment for word-level timing.
refine: Opt-in local-LLM cleanup of the final text (disfluencies,
self-corrections, punctuation) same pipeline the live
dictation socket uses. Off by default so MCP/CLI callers don't
@@ -92,9 +77,7 @@ async def transcribe_audio(
tmp.write(content)
tmp.close()
requested_mode = (mode or "").strip().lower()
use_accurate = requested_mode == "accurate"
use_active_asr = requested_mode in {"accurate", "reference"}
use_accurate = (mode or "").strip().lower() == "accurate"
# TTS-only install: no ASR model on disk → typed 409 with a download
# CTA, BEFORE any backend is constructed (the whisper backends
@@ -102,8 +85,7 @@ async def transcribe_audio(
from services.asr_backend import asr_model_missing_detail, asr_model_missing_error
missing = await asyncio.to_thread(
asr_model_missing_error,
purpose="transcribe" if use_active_asr else "dictation",
require_installed=requested_mode == "reference",
purpose="transcribe" if use_accurate else "dictation",
)
if missing is not None:
raise HTTPException(
@@ -112,16 +94,12 @@ async def transcribe_audio(
)
def _run():
if use_active_asr:
if use_accurate:
# Accurate mode: full WhisperX with forced alignment —
# for when the user explicitly wants word-level timing.
# `load_*`, not `get_*`: the selector alone hands back an
# engine whose shallow probe passed but whose deep import
# chain is broken, which then 500s at `.transcribe()`. The
# loader degrades to the next healthy engine (#1185).
from services.asr_backend import load_active_asr_backend
backend = load_active_asr_backend(require_installed=True) if requested_mode == "reference" else load_active_asr_backend()
result = backend.transcribe(tmp.name, word_timestamps=use_accurate)
from services.asr_backend import get_active_asr_backend
backend = get_active_asr_backend()
result = backend.transcribe(tmp.name, word_timestamps=True)
else:
# Fast mode (default): use the fastest available engine
# (MLX Turbo on Apple Silicon). Skip word_timestamps for
@@ -129,18 +107,13 @@ async def transcribe_audio(
from services.asr_backend import get_capture_asr_backend
backend = get_capture_asr_backend()
result = backend.transcribe(tmp.name, word_timestamps=False)
sherpa_model_id = getattr(getattr(backend, "spec", None), "id", None)
return result, backend.id, sherpa_model_id
return result, backend.id
from services.model_manager import _gpu_pool
from services.asr_backend import (
ASRModelMissingError,
ASRTimeoutError,
run_transcribe_guarded,
)
from services.asr_backend import ASRTimeoutError, run_transcribe_guarded
t0 = time.perf_counter()
try:
result, engine_id, sherpa_model_id = await run_transcribe_guarded(
result, engine_id = await run_transcribe_guarded(
_gpu_pool, _run, what="Dictation",
)
except ASRTimeoutError as e:
@@ -148,77 +121,6 @@ async def transcribe_audio(
# silent hang the UI reads as "can't reach the local backend".
logger.warning("Capture transcription timed out: %s", e)
raise HTTPException(status_code=504, detail=str(e))
except ASRModelMissingError as e:
# Degraded past the broken engine onto one with no weights on
# disk — same typed 409 (+ download CTA) as the preflight above,
# never a 500 and never a silent multi-GB auto-download.
raise HTTPException(
status_code=409,
detail={**e.payload, "message": asr_model_missing_detail(e.payload)},
)
# Some sherpa-onnx NeMo-TDT builds load successfully but decode an
# entire spoken clip to no tokens. Live dictation already recovers
# from that failure; the shared file endpoint must do the same because
# it also powers uploaded transcription and automatic profile text.
# Retry only through an already-installed fallback, and demote the
# silent model only when the second recognizer actually heard words.
initial_text = str(result.get("text") or "").strip()
if not initial_text and result.get("segments"):
initial_text = " ".join(
str(segment.get("text") or "")
for segment in result["segments"]
if isinstance(segment, dict)
).strip()
recovered_from = None
if not use_active_asr and sherpa_model_id and not initial_text:
fallback_missing = await asyncio.to_thread(
asr_model_missing_error,
purpose="dictation",
skip_sherpa=True,
require_installed=True,
)
if fallback_missing is None:
def _run_fallback():
from services.asr_backend import get_capture_asr_backend
fallback = get_capture_asr_backend(skip_sherpa=True)
return (
fallback.transcribe(tmp.name, word_timestamps=False),
fallback.id,
)
try:
fallback_result, fallback_engine_id = await run_transcribe_guarded(
_gpu_pool,
_run_fallback,
what="Dictation fallback",
)
fallback_text = str(fallback_result.get("text") or "").strip()
if not fallback_text and fallback_result.get("segments"):
fallback_text = " ".join(
str(segment.get("text") or "")
for segment in fallback_result["segments"]
if isinstance(segment, dict)
).strip()
if fallback_text:
from services.sherpa_dictation import demote_model
await asyncio.to_thread(demote_model, sherpa_model_id)
result = fallback_result
engine_id = fallback_engine_id
recovered_from = sherpa_model_id
logger.warning(
"File transcription recovered from silent dictation model %s "
"through installed engine %s",
sherpa_model_id,
fallback_engine_id,
)
except Exception:
logger.exception(
"Installed fallback failed after dictation model %s returned no text",
sherpa_model_id,
)
elapsed = round(time.perf_counter() - t0, 2)
# Normalize result shape
@@ -241,15 +143,10 @@ async def transcribe_audio(
from services.text_polish import polish_text
full_text = polish_text(full_text)
# Calculate audio duration from segments if available. A segment whose
# timing the engine could not determine carries end=None (sherpa's
# _sherpa_result when the sample rate yields no duration, and every
# plain-text OpenAI-compatible response), so measure only the ones that
# have a number and keep 0.0 when none do.
# Calculate audio duration from segments if available
duration = 0.0
if segments:
ends = [e for e in (s.get("end") for s in segments) if isinstance(e, (int, float))]
duration = max(ends) if ends else 0.0
duration = max(s.get("end", 0) for s in segments)
detected_lang = result.get("language", language or "unknown")
@@ -270,7 +167,7 @@ async def transcribe_audio(
logger.info(
"Capture transcription done: engine=%s, elapsed=%.2fs, duration=%.1fs, mode=%s, refined=%s",
engine_id, elapsed, duration, requested_mode if use_active_asr else "fast",
engine_id, elapsed, duration, "accurate" if use_accurate else "fast",
refined_text is not None,
)
@@ -278,8 +175,8 @@ async def transcribe_audio(
"text": full_text,
"segments": [
{
"start": _timing(s.get("start", 0)),
"end": _timing(s.get("end", 0)),
"start": round(s.get("start", 0), 2),
"end": round(s.get("end", 0), 2),
"text": s.get("text", "").strip(),
}
for s in segments
@@ -291,8 +188,6 @@ async def transcribe_audio(
}
if refined_text is not None:
response["refined_text"] = refined_text
if recovered_from is not None:
response["model_silent"] = recovered_from
return response
finally:
try:
+79 -365
View File
@@ -9,9 +9,7 @@ Protocol:
Client sends binary audio frames (16-bit PCM or WebM/Opus blobs)
Server sends JSON messages:
Raw PCM mode (``?pcm=1&sr=16000``) is the container-free fallback for
WebViews without MediaRecorder. Opt-in AEC mode
(``?aec=1[&sr=16000]``, parity Action 8b): for dictating
Opt-in AEC mode (``?aec=1[&sr=16000]``, parity Action 8b): for dictating
while the app plays audio. Frames must be raw int16 mono PCM, each tagged
with a 1-byte prefix 0x00 = microphone, 0x01 = playback reference. The
server runs an NLMS echo canceller, cleaning the mic against the reference
@@ -27,10 +25,6 @@ Protocol:
"detail": "..."} error ("detail"
kept for legacy)
Sherpa ``final`` frames additionally carry
``"final_kind": "utterance"|"summary"``. Utterances are mid-session
commits; the summary is the authoritative whole-session result at EOF.
Every ``final`` text is normalised by services.text_polish (leading
capital for Latin scripts, terminal punctuation, single-spaced) so the
pasted result reads like typed text. Partials are raw.
@@ -38,14 +32,10 @@ Protocol:
from __future__ import annotations
import asyncio
import json
import logging
import math
import os
import tempfile
import time
import uuid
from typing import Any
from fastapi import APIRouter, WebSocket, WebSocketDisconnect
@@ -55,20 +45,6 @@ from services.text_polish import polish_text
router = APIRouter()
logger = logging.getLogger("omnivoice.capture_ws")
def _timing(value):
"""A segment timing, or ``None`` when the engine could not determine one.
``dict.get(key, 0)`` returns a stored ``None`` rather than the default, so
rounding it raised (#1904). The null is the honest answer here — this module
emits it deliberately for un-endpointed utterances and the segment list
renders whichever half of the range is known.
"""
return round(value, 2) if isinstance(value, (int, float)) else None
SPEECH_PROTOCOL = "voicestudio.speech.v1"
PLATFORM_STREAM_PATH = "/v1/audio/transcriptions/stream"
# How often (seconds) to run transcription on the accumulated buffer.
# Shorter = more responsive but more GPU load.
PARTIAL_INTERVAL_S = float(os.environ.get("OMNIVOICE_STREAM_INTERVAL", "2.0"))
@@ -92,81 +68,6 @@ _AEC_NEAR = 0x00 # microphone frame (clean it, then buffer for ASR)
_AEC_FAR = 0x01 # playback reference frame (feed the echo model only)
# Client-supplied ``?sr=`` values outside the range real capture devices use
# are replaced with 16 kHz. The rate sizes server-side state — RecoveryTail
# multiplies it by RECOVERY_TAIL_SECONDS to compute its byte ceiling — so an
# absurd rate must never be believed: it would re-open the unbounded-memory
# path the recovery-tail cap closed.
SR_MIN, SR_MAX = 8000, 96000
def _is_end_control(text: str | None) -> bool:
"""Accept the versioned JSON control frame and the legacy ``EOF`` frame."""
if text == "EOF":
return True
if not text:
return False
try:
message = json.loads(text)
except (TypeError, json.JSONDecodeError):
return False
return isinstance(message, dict) and message.get("type") == "input_audio.end"
class _PlatformWebSocket:
"""Add v1 session metadata without changing the legacy WebSocket contract."""
def __init__(self, websocket: WebSocket):
self._websocket = websocket
self.session_id = uuid.uuid4().hex
def __getattr__(self, name: str) -> Any:
return getattr(self._websocket, name)
async def send_json(self, data: Any, mode: str = "text") -> None:
if isinstance(data, dict):
data = dict(data)
data.setdefault("protocol", SPEECH_PROTOCOL)
data.setdefault("session_id", self.session_id)
if data.get("type") == "final":
data.setdefault("final_kind", "summary")
await self._websocket.send_json(data, mode=mode)
def _bounded_sample_rate(query_params) -> int:
try:
sample_rate = int(query_params.get("sr", "16000"))
except (TypeError, ValueError):
return 16000
return sample_rate if SR_MIN <= sample_rate <= SR_MAX else 16000
def _requested_pcm_sample_rate(query_params) -> int | None:
"""Return the bounded rate when the client transport is raw PCM.
Sherpa clients omit ``pcm=1`` because the selected model already defines
that transport. If the model is demoted or its runtime is unavailable, the
legacy recognizer fallback must still decode those same bytes as PCM.
"""
raw_pcm = query_params.get("pcm") in ("1", "true", "on")
aec = query_params.get("aec") in ("1", "true", "on")
sherpa_pcm = False
requested_model = query_params.get("model")
if requested_model:
try:
from services.sherpa_dictation import is_sherpa_model
sherpa_pcm = is_sherpa_model(requested_model)
except Exception: # noqa: BLE001
# A broken sherpa install must not decide the framing question —
# sherpa_pcm stays False and the session negotiates the
# MediaRecorder path; availability is re-probed (and reported)
# when the model is actually selected.
sherpa_pcm = False
if not raw_pcm and not aec and not sherpa_pcm:
return None
return _bounded_sample_rate(query_params)
def _demux_aec_frame(data: bytes) -> tuple[str, bytes]:
"""Split a prefixed AEC binary frame into ``(kind, pcm)``.
@@ -221,50 +122,24 @@ def _select_sherpa_spec(websocket: WebSocket):
from services import sherpa_dictation as sd
except Exception:
return None
def _usable_spec(model_id):
spec = sd.get_spec(model_id)
if spec is not None and sd.is_demoted(spec.id):
logger.warning(
"dictation model %s is demoted — using the capture ASR fallback",
spec.id,
)
return None
return spec
requested = websocket.query_params.get("model")
if requested:
return _usable_spec(requested) # explicit selection (may be unavailable)
return sd.get_spec(requested) # explicit selection (may be None if bad)
# Fall back to the persisted dictation pref.
try:
from services.asr_backend import dictation_model_id
mid = dictation_model_id()
except Exception:
mid = None
return _usable_spec(mid) if mid else None
return sd.get_spec(mid) if mid else None
@router.websocket(PLATFORM_STREAM_PATH)
@router.websocket("/ws/transcribe")
async def ws_transcribe(websocket: WebSocket):
"""Stream audio in, get partial + final transcription out."""
is_platform_stream = websocket.url.path == PLATFORM_STREAM_PATH
if is_platform_stream:
websocket = _PlatformWebSocket(websocket)
# A browser can reach localhost regardless of the page's own origin.
# Reject ambient cross-site WebSocket handshakes before the loopback-host
# shortcut or accept(), while keeping native clients (no Origin header)
# and configured/same-origin browser UIs working (#1646 review).
origin = websocket.headers.get("origin")
if origin:
from core.csrf import origin_allowed
if not origin_allowed(websocket):
await websocket.close(code=1008, reason="browser origin not allowed")
return
# Loopback origin guard — refuse anything not from 127.0.0.1, ::1, or
# localhost. Privileged HTTP routers use Depends(require_admin) at router
# level; WebSocket dependency injection differs across FastAPI versions, so we
# localhost. HTTP routers use Depends(require_loopback) at router level;
# WebSocket dependency injection differs across FastAPI versions, so we
# inline the check before accept(). Without it, any local process could
# stream the user's microphone over this endpoint.
# Wave 2.3 (remote backend): a non-loopback client that presents the
@@ -276,16 +151,6 @@ async def ws_transcribe(websocket: WebSocket):
return
await websocket.accept()
if is_platform_stream:
await websocket.send_json({
"type": "session.started",
"input_format": (
"audio/pcm;encoding=s16le;channels=1"
if _requested_pcm_sample_rate(websocket.query_params) is not None
else "audio/webm;codecs=opus"
),
"sample_rate": _bounded_sample_rate(websocket.query_params),
})
# Live-dictation engine selection. When a sherpa-onnx model is selected
# (via ?model= or the dictation.model_id pref) AND sherpa is installed,
@@ -345,11 +210,12 @@ async def ws_transcribe(websocket: WebSocket):
# identical legacy behaviour. When on, frames are 1-byte-tagged raw PCM
# and the cleaned mic stream is muxed via stdlib wave (not ffmpeg).
aec = None
pcm_sr = _requested_pcm_sample_rate(websocket.query_params)
pcm_sr: int | None = None
if websocket.query_params.get("aec") in ("1", "true", "on"):
try:
pcm_sr = int(websocket.query_params.get("sr", "16000"))
from services.aec import NlmsEchoCanceller
aec = NlmsEchoCanceller(sample_rate=pcm_sr or 16000)
aec = NlmsEchoCanceller(sample_rate=pcm_sr)
logger.info("AEC enabled for dictation session (sr=%d)", pcm_sr)
except Exception as e:
# Bad sr or import failure → fall back to plain dictation.
@@ -360,7 +226,6 @@ async def ws_transcribe(websocket: WebSocket):
audio_chunks: list[bytes] = []
total_bytes = 0
last_audio_time = time.monotonic()
paused = False
running = True
partial_text = ""
# Track whether the client initiated the disconnect. When True the
@@ -378,7 +243,7 @@ async def ws_transcribe(websocket: WebSocket):
message as the authoritative result and skip the duplicate HTTP
POST that used to run on every dictation.
"""
nonlocal total_bytes, last_audio_time, running, client_disconnected, paused
nonlocal total_bytes, last_audio_time, running, client_disconnected
try:
while running:
msg = await websocket.receive()
@@ -409,11 +274,7 @@ async def ws_transcribe(websocket: WebSocket):
total_bytes += len(data)
last_audio_time = time.monotonic()
continue
if msg.get("text") in ("PAUSE", "RESUME"):
paused = msg["text"] == "PAUSE"
last_audio_time = time.monotonic()
continue
if _is_end_control(msg.get("text")):
if msg.get("text") == "EOF":
# Client signals end-of-audio but stays connected for `final`.
running = False
break
@@ -445,9 +306,6 @@ async def ws_transcribe(websocket: WebSocket):
if not running:
break
if paused:
continue
# Check silence timeout
if time.monotonic() - last_audio_time > SILENCE_TIMEOUT_S and total_bytes > MIN_BUFFER_BYTES:
running = False
@@ -550,64 +408,6 @@ SHERPA_OFFLINE_SILENCE_S = float(os.environ.get("OMNIVOICE_SHERPA_OFFLINE_SILENC
SHERPA_OFFLINE_RMS_FLOOR = float(os.environ.get("OMNIVOICE_SHERPA_OFFLINE_RMS", "0.01"))
#: Seconds of audio retained for silent-model recovery. Recovery only needs
#: enough speech to prove the model is broken and to re-transcribe what was
#: said; retaining the whole session grew ~115 MB/hour at 16 kHz on an open
#: mic, unbounded, and only ever got read when the fallback fired.
RECOVERY_TAIL_DEFAULT_SECONDS = 120.0
RECOVERY_TAIL_MAX_SECONDS = 300.0
def _bounded_recovery_tail_seconds(value: str | None) -> float:
"""Parse the recovery tail override without allowing unbounded buffers."""
try:
seconds = float(value) if value is not None else RECOVERY_TAIL_DEFAULT_SECONDS
except (TypeError, ValueError):
return RECOVERY_TAIL_DEFAULT_SECONDS
if not math.isfinite(seconds) or seconds <= 0:
return RECOVERY_TAIL_DEFAULT_SECONDS
return min(seconds, RECOVERY_TAIL_MAX_SECONDS)
RECOVERY_TAIL_SECONDS = _bounded_recovery_tail_seconds(
os.environ.get("OMNIVOICE_DICTATION_RECOVERY_TAIL_S")
)
class RecoveryTail:
"""The most recent ``RECOVERY_TAIL_SECONDS`` of session audio.
Keeps the *tail* rather than the head: a long dictation's useful speech is
what the user just said, and the silent-model check cares about how much
audio the session carried overall which ``total_bytes`` still reports
truthfully after trimming.
"""
__slots__ = ("_buf", "_max", "total_bytes")
def __init__(self, sample_rate: int, seconds: float = RECOVERY_TAIL_SECONDS):
# int16 mono → 2 bytes/sample. Floor of one frame so a nonsense rate
# or seconds value can't produce a zero-length buffer.
self._max = max(2, int(seconds * max(1, sample_rate)) * 2)
self._buf = bytearray()
self.total_bytes = 0
def extend(self, pcm: bytes) -> None:
self._buf.extend(pcm)
self.total_bytes += len(pcm)
excess = len(self._buf) - self._max
if excess > 0:
# int16 mono: trim whole samples only. A split frame can carry an
# odd byte count, and an odd trim would leave the tail starting
# mid-sample — every later sample byte-shifted, and the recovery
# transcription fed noise.
excess += excess % 2
del self._buf[:excess]
def tail(self) -> bytes:
return bytes(self._buf)
def is_model_silent(text: str, heard_speech: bool, pcm_bytes: int) -> bool:
"""True when the dictation model produced NO text despite real speech.
@@ -634,74 +434,19 @@ def _pcm16_to_f32(pcm: bytes):
return np.frombuffer(pcm, dtype=np.int16).astype(np.float32) / 32768.0
def _pcm16_rms(pcm: bytes) -> float:
samples = _pcm16_to_f32(pcm)
if not len(samples):
return 0.0
return float((samples * samples).mean() ** 0.5)
async def _recover_silent_sherpa(
spec, pcm: bytes, pcm_sr: int,
) -> tuple[str, list[dict]]:
"""Retry a token-silent Sherpa session through an installed local ASR."""
logger.warning(
"dictation model %s decoded NOTHING from %.1fs of speech-level audio "
"— falling back to the capture ASR engine for this session",
spec.id, len(pcm) / float(max(1, pcm_sr) * 2),
)
try:
from services.asr_backend import asr_model_missing_error
fallback_missing = await asyncio.to_thread(
asr_model_missing_error,
purpose="dictation",
skip_sherpa=True,
require_installed=True,
)
if fallback_missing is not None:
logger.warning(
"dictation silent-model fallback is not installed (%s); "
"skipping recovery to avoid an automatic download",
fallback_missing.get("missing_repo_id", "unknown"),
)
return "", []
result = await _transcribe_buffer_full(
[pcm], pcm_sr=pcm_sr, skip_sherpa=True,
)
text = polish_text(_result_text(result))
if not text:
return "", []
# The RMS gate can fire on fan/keyboard noise. Only another recognizer
# producing words proves the audio held speech and makes persistent
# demotion safe.
try:
from services.sherpa_dictation import demote_model
if await asyncio.to_thread(demote_model, spec.id):
logger.error(
"dictation model %s demoted on this machine — it will no longer be "
"auto-selected. Pick it again in Settings to give it another chance.",
spec.id,
)
except Exception:
logger.exception("silent-model demotion failed")
segments = (result or {}).get("segments") or [
{"start": 0.0, "end": None, "text": text}
]
return text, segments
except Exception:
logger.exception("dictation silent-model fallback failed")
return "", []
async def _sherpa_session(websocket: WebSocket):
"""Shared WS setup for the sherpa handlers.
"""Shared WS receive setup for the sherpa handlers.
Returns ``(pcm_sr, aec)``: the bounded PCM sample rate for the session
and the echo canceller when ``?aec=1`` requested one (``None`` otherwise
or when AEC setup fails).
Returns ``(get_frame, state)`` where ``get_frame`` is an async callable
that yields the next near-end (mic) PCM bytes, ``b""`` for a keepalive/ref
frame, or ``None`` on EOF/disconnect. ``state`` carries sample rate, AEC,
and the disconnect flag for the caller's finaliser.
"""
pcm_sr = _bounded_sample_rate(websocket.query_params)
pcm_sr = 16000
try:
pcm_sr = int(websocket.query_params.get("sr", "16000"))
except (TypeError, ValueError):
pcm_sr = 16000
aec = None
if websocket.query_params.get("aec") in ("1", "true", "on"):
try:
@@ -739,7 +484,7 @@ async def _recv_pcm_frame(websocket: WebSocket, aec):
return "skip", b""
return "near", aec.process_near_end(payload)
return "near", data
if _is_end_control(msg.get("text")):
if msg.get("text") == "EOF":
return "eof", b""
return "skip", b""
@@ -762,8 +507,7 @@ async def _sherpa_load_with_status(websocket: WebSocket, backend, spec) -> bool:
try:
await websocket.send_json({"type": "status", "stage": stage})
except Exception:
logger.warning("Sherpa load status could not be delivered; stopping stream setup")
return False
pass
try:
await asyncio.to_thread(backend.ensure_loaded)
except Exception as e:
@@ -778,8 +522,7 @@ async def _sherpa_load_with_status(websocket: WebSocket, backend, spec) -> bool:
try:
await websocket.send_json({"type": "status", "stage": "ready"})
except Exception:
logger.warning("Sherpa ready status could not be delivered; stopping stream setup")
return False
pass
return True
@@ -810,8 +553,6 @@ async def _run_sherpa_streaming(websocket: WebSocket, spec):
last_partial = ""
committed: list[str] = [] # finalized utterances this session
session_pcm = RecoveryTail(pcm_sr) # bounded audio for silent-model recovery
heard_speech = False
client_disconnected = False
async def _send(payload) -> bool:
@@ -853,9 +594,6 @@ async def _run_sherpa_streaming(websocket: WebSocket, spec):
break
if kind == "skip":
continue
session_pcm.extend(pcm)
if not heard_speech and _pcm16_rms(pcm) >= SHERPA_OFFLINE_RMS_FLOOR:
heard_speech = True
text, endpoint = await asyncio.to_thread(_decode_after_feed, pcm)
if endpoint:
# Commit this utterance (polished — it gets pasted); reset
@@ -864,7 +602,6 @@ async def _run_sherpa_streaming(websocket: WebSocket, spec):
if text:
committed.append(text)
await _send({"type": "final", "text": text,
"final_kind": "utterance",
"segments": [{"start": 0.0, "end": None, "text": text}],
"language": "auto", "engine": backend.id})
rec.reset(stream)
@@ -891,28 +628,7 @@ async def _run_sherpa_streaming(websocket: WebSocket, spec):
# Pieces are already polished; the join is too (polish is idempotent).
full = " ".join(t for t in committed if t).strip()
segments = [{"start": 0.0, "end": None, "text": t} for t in committed if t]
model_silent = is_model_silent(full, heard_speech, session_pcm.total_bytes)
if model_silent:
recovered, recovered_segments = await _recover_silent_sherpa(
spec, session_pcm.tail(), pcm_sr,
)
if recovered:
full = recovered
segments = recovered_segments
if not client_disconnected:
payload = {"type": "final", "text": full, "final_kind": "summary",
"segments": segments,
"language": "auto", "engine": backend.id}
if model_silent:
payload["engine"] = "capture-asr-fallback" if full else backend.id
payload["model_silent"] = spec.id
payload["warning"] = (
f"The selected dictation model ({spec.id}) produced no text from your "
"speech. Switched to the fallback engine for this session — pick a "
"different model in Settings → Dictation."
)
if full:
# Hard-bounded refinement (~4s): never delays this summary `final`
# beyond OMNIVOICE_REFINE_TIMEOUT_S even with a dead LLM endpoint.
@@ -921,9 +637,14 @@ async def _run_sherpa_streaming(websocket: WebSocket, spec):
refined = await maybe_refine_async(full)
except Exception:
refined = None
payload = {"type": "final", "text": full, "segments": segments,
"language": "auto", "engine": backend.id}
if refined and refined != full:
payload["refined_text"] = refined
await _send(payload)
await _send(payload)
else:
await _send({"type": "final", "text": "", "segments": [],
"language": "auto", "engine": backend.id})
try:
await websocket.close()
except Exception:
@@ -960,7 +681,7 @@ async def _run_sherpa_offline(websocket: WebSocket, spec):
# whisper/zipformer transcribe the same bytes). Keep the whole session's
# audio and whether any of it was speech-level, so the finaliser can tell
# "user said nothing" (fine) from "model produced nothing" (broken).
session_pcm = RecoveryTail(pcm_sr)
session_pcm = bytearray()
heard_speech = False
running = True
client_disconnected = False
@@ -979,6 +700,12 @@ async def _run_sherpa_offline(websocket: WebSocket, spec):
client_disconnected = True
return False
def _rms(pcm: bytes) -> float:
samples = _pcm16_to_f32(pcm)
if not len(samples):
return 0.0
return float((samples * samples).mean() ** 0.5)
def _decode_window(pcm: bytes) -> str:
samples = _pcm16_to_f32(pcm)
if not len(samples):
@@ -997,7 +724,7 @@ async def _run_sherpa_offline(websocket: WebSocket, spec):
continue
buf.extend(pcm)
session_pcm.extend(pcm)
if not heard_speech and _pcm16_rms(pcm) >= SHERPA_OFFLINE_RMS_FLOOR:
if not heard_speech and _rms(pcm) >= SHERPA_OFFLINE_RMS_FLOOR:
heard_speech = True
last_audio = time.monotonic()
except WebSocketDisconnect:
@@ -1023,7 +750,6 @@ async def _run_sherpa_offline(websocket: WebSocket, spec):
if text:
committed.append(text)
await _send({"type": "final", "text": text,
"final_kind": "utterance",
"segments": [{"start": 0.0, "end": None, "text": text}],
"language": "auto", "engine": backend.id})
@@ -1035,8 +761,8 @@ async def _run_sherpa_offline(websocket: WebSocket, spec):
continue
snapshot = bytes(buf)
if len(snapshot) > sil_bytes and \
_pcm16_rms(snapshot[-sil_bytes:]) < SHERPA_OFFLINE_RMS_FLOOR:
if _pcm16_rms(snapshot[:-sil_bytes]) >= SHERPA_OFFLINE_RMS_FLOOR:
_rms(snapshot[-sil_bytes:]) < SHERPA_OFFLINE_RMS_FLOOR:
if _rms(snapshot[:-sil_bytes]) >= SHERPA_OFFLINE_RMS_FLOOR:
await _commit(snapshot)
else:
# Pure silence — drop it (keep the gate window for
@@ -1082,18 +808,39 @@ async def _run_sherpa_offline(websocket: WebSocket, spec):
# quiet user — hand the session to the capture ASR backend so the user
# still gets their words, and say which model let them down. Bounded to
# this session; the pref is left alone so the user stays in control.
model_silent = is_model_silent(full, heard_speech, session_pcm.total_bytes)
model_silent = is_model_silent(full, heard_speech, len(session_pcm))
if model_silent:
recovered, recovered_segments = await _recover_silent_sherpa(
spec, session_pcm.tail(), pcm_sr,
logger.warning(
"dictation model %s decoded NOTHING from %.1fs of speech-level audio "
"— falling back to the capture ASR engine for this session",
spec.id, len(session_pcm) / float(max(1, pcm_sr) * 2),
)
if recovered:
full = recovered
segments = recovered_segments
# Demote it so the NEXT session doesn't repeat this round trip. The
# curated default can be broken on a platform we never tested (the
# NeMo-TDT decoder is, on Windows), and observing it beats guessing.
try:
from services.sherpa_dictation import demote_model
if demote_model(spec.id):
logger.error(
"dictation model %s demoted on this machine — it will no longer be "
"auto-selected. Pick it again in Settings to give it another chance.",
spec.id,
)
except Exception:
logger.exception("silent-model demotion failed")
try:
result = await _transcribe_buffer_full([bytes(session_pcm)], pcm_sr=pcm_sr)
fb_text = polish_text((result or {}).get("text", "") or "")
if fb_text:
full = fb_text
segments = (result or {}).get("segments") or [
{"start": 0.0, "end": None, "text": fb_text}
]
except Exception:
logger.exception("dictation silent-model fallback failed")
if not client_disconnected:
payload = {"type": "final", "text": full, "final_kind": "summary",
"segments": segments,
payload = {"type": "final", "text": full, "segments": segments,
"language": "auto", "engine": backend.id}
if model_silent:
# The client surfaces this so a silently-broken model can't look
@@ -1121,35 +868,6 @@ async def _run_sherpa_offline(websocket: WebSocket, spec):
pass
def _result_text(result: dict | None) -> str:
"""Normalize text from every ASR backend result shape.
Some backends return a top-level ``text`` value, while WhisperX, Faster
Whisper, Moonshine, and OpenAI-compatible ASR expose only ``segments`` and
``chunks``. Dictation partials and finals must interpret both contracts the
same way.
"""
if not isinstance(result, dict):
return ""
text = result.get("text")
if isinstance(text, str) and text.strip():
return text.strip()
for key in ("segments", "chunks"):
items = result.get(key)
if not isinstance(items, (list, tuple)):
continue
text = " ".join(
str(item.get("text", "")).strip()
for item in items
if isinstance(item, dict) and item.get("text")
).strip()
if text:
return text
return ""
async def _transcribe_buffer(chunks: list[bytes], *, pcm_sr: int | None = None) -> str:
"""Quick partial transcription of the current audio buffer."""
@@ -1164,7 +882,7 @@ async def _transcribe_buffer(chunks: list[bytes], *, pcm_sr: int | None = None)
def _run():
backend = get_capture_asr_backend()
result = backend.transcribe(tmp, word_timestamps=False)
return _result_text(result)
return result.get("text", "")
# Bound dictation transcribes (#730): a wedged whisperx/CTranslate2 call
# must not hold its GPU-pool worker forever and starve TTS / other ASR
@@ -1178,9 +896,7 @@ async def _transcribe_buffer(chunks: list[bytes], *, pcm_sr: int | None = None)
pass
async def _transcribe_buffer_full(
chunks: list[bytes], *, pcm_sr: int | None = None, skip_sherpa: bool = False,
) -> dict:
async def _transcribe_buffer_full(chunks: list[bytes], *, pcm_sr: int | None = None) -> dict:
"""Full transcription with timing info for the final result."""
tmp = _pcm16_to_wav(b"".join(chunks), pcm_sr) if pcm_sr else _chunks_to_wav(chunks)
if tmp is None:
@@ -1192,13 +908,15 @@ async def _transcribe_buffer_full(
from services.asr_backend import get_capture_asr_backend, run_transcribe_guarded
def _run():
backend = get_capture_asr_backend(skip_sherpa=skip_sherpa)
backend = get_capture_asr_backend()
t0 = time.perf_counter()
result = backend.transcribe(tmp, word_timestamps=False)
elapsed = round(time.perf_counter() - t0, 2)
segments = result.get("segments", [])
full_text = _result_text(result)
full_text = result.get("text", "")
if not full_text and segments:
full_text = " ".join(s.get("text", "") for s in segments).strip()
# Wave 1.1: strip Whisper hallucination loops from the final
# text (the string that gets auto-pasted). Segments keep the
@@ -1206,19 +924,13 @@ async def _transcribe_buffer_full(
from services.refinement import collapse_repetitive_artifacts
full_text = collapse_repetitive_artifacts(full_text)
# end=None means the engine could not determine the timing — this
# module writes exactly that in its own streaming payloads, and
# sherpa's _sherpa_result does too when the sample rate yields no
# duration. Measure only real numbers, and pass the nulls through
# rather than rounding them (#1904).
ends = [e for e in (s.get("end") for s in segments) if isinstance(e, (int, float))]
duration = max(ends) if ends else 0.0
duration = max((s.get("end", 0) for s in segments), default=0.0)
return {
"text": full_text,
"segments": [
{"start": _timing(s.get("start", 0)),
"end": _timing(s.get("end", 0)),
{"start": round(s.get("start", 0), 2),
"end": round(s.get("end", 0), 2),
"text": s.get("text", "").strip()}
for s in segments
],
@@ -1289,3 +1001,5 @@ def _chunks_to_wav(chunks: list[bytes]) -> str | None:
# WhisperX) can decode WebM/Opus containers natively.
logger.debug("Falling back to raw WebM input for ASR")
return tmp_in.name
+97 -726
View File
@@ -20,26 +20,18 @@ Design / safety
from __future__ import annotations
import asyncio
import contextlib
import hashlib
import json
import logging
import os
import re
import shutil
import tempfile
import time
import uuid
from pathlib import Path
from typing import Optional
from urllib.parse import urljoin, urlparse
from urllib.parse import urlparse
from fastapi import APIRouter, HTTPException, Query
from fastapi.responses import FileResponse
from core import archetypes
from core.audio_validation import is_playable_wav, resolve_regular_file
from core.config import DATA_DIR, VOICES_DIR
from core.config import DATA_DIR
logger = logging.getLogger("omnivoice.community")
router = APIRouter()
@@ -50,32 +42,9 @@ _ALLOWED_AUDIO_HOSTS = {
"cdn.jsdelivr.net", "github.com", "raw.githubusercontent.com",
"objects.githubusercontent.com", "release-assets.githubusercontent.com",
}
_ALLOWED_MANIFEST_HOSTS = {"cdn.jsdelivr.net"}
_VALID_TOKENS = set(archetypes._VD._INSTRUCT_ALL_VALID)
_USE_CASE_IDS = {c["id"] for c in archetypes.USE_CASES}
_SOURCE_RE = re.compile(
r"^[A-Za-z0-9._-]{1,100}/[A-Za-z0-9._-]{1,100}$",
) # owner/repo only
_ITEM_ID_RE = re.compile(r"^[A-Za-z0-9_-]{1,128}$")
_SHA256_RE = re.compile(r"^[0-9a-f]{64}$")
# A gallery open may touch this loader several times (grid, preview, use). Keep
# a successful response for six hours, then revalidate it once. On a network
# failure the readable stale copy remains usable and its check time advances,
# preventing every offline gallery open from waiting through the same timeout.
_MANIFEST_MAX_AGE_S = 6 * 60 * 60
_MAX_MANIFEST_BYTES = 4 << 20
_MAX_SAMPLE_SCRIPT_CHARS = 2_000
_MAX_REF_TEXT_CHARS = 4_000
# Community voice submissions are documented as short clean WAV clips. The cap
# comfortably covers 15 s of uncompressed 96 kHz stereo PCM while preventing a
# remote manifest from turning Preview into an unbounded disk/memory download.
_MAX_VOICE_AUDIO_BYTES = 32 << 20
_ATTR_NAMES = (
"Gender", "Age", "Pitch", "Style", "EnglishAccent", "ChineseDialect",
)
_SOURCE_RE = re.compile(r"^[A-Za-z0-9._-]+/[A-Za-z0-9._-]+$") # owner/repo only
# ── Config: which content repos to load ───────────────────────────────────────
@@ -83,18 +52,14 @@ def configured_sources() -> list[str]:
"""Gallery sources, in priority order. Env var > config file > default."""
env = os.environ.get("OMNIVOICE_GALLERY_SOURCES")
if env:
sources = [s.strip() for s in env.split(",")]
valid = [s for s in sources if _SOURCE_RE.fullmatch(s)]
return valid or list(_DEFAULT_SOURCES)
return [s.strip() for s in env.split(",") if s.strip()]
cfg = Path(DATA_DIR) / "gallery_sources.json"
if cfg.exists():
try:
data = json.loads(cfg.read_text(encoding="utf-8"))
srcs = data.get("sources")
if isinstance(srcs, list) and srcs:
valid = [s for s in srcs if isinstance(s, str) and _SOURCE_RE.fullmatch(s)]
if valid:
return valid
return [str(s) for s in srcs]
except Exception:
logger.warning("gallery_sources.json unreadable; using default")
return list(_DEFAULT_SOURCES)
@@ -116,51 +81,9 @@ def _safe_audio_url(url: str) -> bool:
return False
def _safe_manifest_url(url: str) -> bool:
try:
parsed = urlparse(url or "")
return parsed.scheme == "https" and parsed.hostname in _ALLOWED_MANIFEST_HOSTS
except Exception:
return False
def normalize_preset_instruct(instruct: str) -> Optional[tuple[str, dict]]:
"""Normalize one validator-safe tag per design category.
Membership in the vocabulary is not enough: ``male, female`` contains two
individually valid tokens but the engine rejects the pair as conflicting.
Build the frontend's full ``vd_states`` shape at this trust boundary too,
so Magic Wand never inherits stale sliders from the previous voice.
"""
attrs = {name: "Auto" for name in _ATTR_NAMES}
normalized: list[str] = []
seen_categories: set[int] = set()
for raw in re.split("[," + chr(0xFF0C) + "]", str(instruct or "")):
token = raw.strip().lower()
if not token or token not in _VALID_TOKENS:
return None
category = archetypes._VD._instruct_category_index(token)
if category < 0 or category in seen_categories:
return None
seen_categories.add(category)
# The picker represents the universal gender/age/pitch/style axes in
# English even for Chinese speech; dialect remains Chinese-only.
canonical = archetypes._VD._INSTRUCT_ZH_TO_EN.get(token, token)
attrs[_ATTR_NAMES[category]] = canonical
normalized.append(canonical)
if not normalized:
return None
# Accent and Chinese dialect are separate taxonomy buckets but the engine
# deliberately forbids mixing them in a single design.
if 4 in seen_categories and 5 in seen_categories:
return None
return ", ".join(normalized), attrs
def is_valid_instruct(instruct: str) -> bool:
return normalize_preset_instruct(instruct) is not None
toks = [t.strip() for t in (instruct or "").split(",") if t.strip()]
return bool(toks) and all(t in _VALID_TOKENS for t in toks)
def validate_item(raw: dict) -> Optional[dict]:
@@ -170,203 +93,62 @@ def validate_item(raw: dict) -> Optional[dict]:
it = dict(raw)
if it.get("type") not in ("preset", "voice"):
return None
if not isinstance(it.get("id"), str) or not _ITEM_ID_RE.fullmatch(it["id"]):
if not it.get("id") or not it.get("name"):
return None
if not isinstance(it.get("name"), str) or not it["name"].strip():
return None
it["name"] = it["name"].strip()[:80]
if it.get("use_case") not in _USE_CASE_IDS:
return None
raw_facets = it.get("facets")
if not isinstance(raw_facets, dict):
raw_facets = {}
language = it.get("language")
if not isinstance(language, str) or not language.strip():
language = raw_facets.get("lang", "English")
it["language"] = language.strip() if isinstance(language, str) and language.strip() else "English"
facets = dict(raw_facets)
if it["type"] == "preset":
normalized = normalize_preset_instruct(it.get("instruct", ""))
if normalized is None:
return None # unknown/conflicting tokens would crash synthesis
it["instruct"], it["attrs"] = normalized
attrs = it["attrs"]
facets.update({
"gender": None if attrs["Gender"] == "Auto" else attrs["Gender"],
"age": None if attrs["Age"] == "Auto" else attrs["Age"],
"pitch": None if attrs["Pitch"] == "Auto" else attrs["Pitch"],
"accent": None if attrs["EnglishAccent"] == "Auto" else attrs["EnglishAccent"],
"whisper": attrs["Style"] == "whisper",
"lang": it["language"],
})
sample_script = it.get("sample_script")
it["sample_script"] = (
sample_script.strip()[:_MAX_SAMPLE_SCRIPT_CHARS]
if isinstance(sample_script, str) else ""
)
else:
audio = it.get("audio")
if not isinstance(audio, dict) or not _safe_audio_url(audio.get("url", "")):
return None
expected = audio.get("sha256")
if expected is not None:
expected = str(expected).lower()
if not _SHA256_RE.fullmatch(expected):
return None
audio = {**audio, "sha256": expected}
ref_text = audio.get("ref_text")
audio = {
**audio,
"ref_text": (
ref_text.strip()[:_MAX_REF_TEXT_CHARS]
if isinstance(ref_text, str) else ""
),
}
it["audio"] = audio
facets.setdefault("gender", None)
facets.setdefault("age", None)
facets.setdefault("pitch", None)
facets.setdefault("accent", None)
facets.setdefault("whisper", False)
facets.setdefault("lang", it["language"])
it["facets"] = facets
if it["type"] == "preset" and not is_valid_instruct(it.get("instruct", "")):
return None # would crash synthesis — drop it
if it["type"] == "voice" and not _safe_audio_url((it.get("audio") or {}).get("url", "")):
return None
it.setdefault("facets", {})
it.setdefault("icon", archetypes._USE_ICON.get(it["use_case"], "Sparkles"))
it.setdefault("language", it.get("facets", {}).get("lang", "English"))
it["is_community"] = it.get("source") != "starter"
it["preview_url"] = f"/community/items/{it['id']}/preview"
return it
def _merge(manifests: list[tuple[str, Optional[dict]]]) -> tuple[list, list]:
items, packs, seen = [], [], set()
for src, m in manifests:
if not isinstance(m, dict):
if not m:
continue
raw_items = m.get("items")
for raw in raw_items if isinstance(raw_items, list) else []:
for raw in (m.get("items") or []):
v = validate_item(raw)
if v and v["id"] not in seen:
v["_source_repo"] = src
seen.add(v["id"])
items.append(v)
raw_packs = m.get("packs")
for p in raw_packs if isinstance(raw_packs, list) else []:
for p in (m.get("packs") or []):
if isinstance(p, dict):
packs.append({**p, "_source_repo": src})
return items, packs
def _read_manifest_cache(cache: Path) -> Optional[dict]:
try:
if cache.stat().st_size > _MAX_MANIFEST_BYTES:
return None
data = json.loads(cache.read_text(encoding="utf-8"))
return data if isinstance(data, dict) else None
except (OSError, ValueError, TypeError):
return None
def _write_bytes_atomic(path: Path, data: bytes) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
fd, tmp = tempfile.mkstemp(dir=str(path.parent), prefix=f".{path.name}-", suffix=".part")
try:
with os.fdopen(fd, "wb") as handle:
handle.write(data)
handle.flush()
os.fsync(handle.fileno())
os.replace(tmp, path)
except BaseException:
with contextlib.suppress(OSError):
os.unlink(tmp)
raise
def _fetch_remote_manifest(source: str, *, client=None) -> dict:
"""Fetch one bounded manifest, validating every redirect before request."""
import httpx
if not _SOURCE_RE.fullmatch(source or ""):
raise ValueError("invalid gallery source")
owned_client = client is None
http = client or httpx.Client(timeout=15.0, follow_redirects=False)
current_url = _manifest_url(source)
payload = bytearray()
try:
fetched = False
for _redirect in range(6):
if not _safe_manifest_url(current_url):
raise ValueError("gallery manifest URL is not from an allowed host")
with http.stream("GET", current_url, follow_redirects=False) as response:
if response.status_code in (301, 302, 303, 307, 308):
location = response.headers.get("location")
next_url = urljoin(current_url, location or "")
if not location or not _safe_manifest_url(next_url):
raise ValueError("gallery manifest redirected to a disallowed host")
current_url = next_url
continue
response.raise_for_status()
length = response.headers.get("content-length")
if length:
try:
declared_length = int(length)
except ValueError:
declared_length = None
if declared_length is not None and declared_length > _MAX_MANIFEST_BYTES:
raise ValueError("gallery manifest exceeded the size limit")
for chunk in response.iter_bytes():
if not chunk:
continue
if len(payload) + len(chunk) > _MAX_MANIFEST_BYTES:
raise ValueError("gallery manifest exceeded the size limit")
payload.extend(chunk)
fetched = True
break
if not fetched:
raise ValueError("gallery manifest followed too many redirects")
finally:
if owned_client:
http.close()
if not payload:
raise ValueError("gallery manifest was empty")
data = json.loads(payload)
if not isinstance(data, dict):
raise ValueError("gallery manifest is not a JSON object")
return data
def _fetch_manifest(
source: str, refresh: bool, *, now: Optional[float] = None,
) -> Optional[dict]:
"""Return a fresh manifest, with a throttled stale-cache offline fallback."""
def _fetch_manifest(source: str, refresh: bool) -> Optional[dict]:
"""Return a source's manifest from cache, or fetch + cache it. None if both fail."""
cache = _cache_path(source)
cached = _read_manifest_cache(cache)
checked_at = time.time() if now is None else float(now)
if not refresh and cached is not None:
if not refresh and cache.exists():
try:
if checked_at - cache.stat().st_mtime < _MANIFEST_MAX_AGE_S:
return cached
except OSError:
pass # treat a stat race as stale and try the source once
return json.loads(cache.read_text(encoding="utf-8"))
except Exception:
pass
try:
data = _fetch_remote_manifest(source)
encoded = json.dumps(
data, ensure_ascii=False, separators=(",", ":"),
).encode("utf-8")
if len(encoded) > _MAX_MANIFEST_BYTES:
raise ValueError("gallery manifest exceeded the cache size limit")
_write_bytes_atomic(cache, encoded)
# Tests inject their own clock; production's value equals wall time.
os.utime(cache, (checked_at, checked_at))
import httpx
with httpx.Client(timeout=15.0, follow_redirects=True) as client:
resp = client.get(_manifest_url(source))
resp.raise_for_status()
data = resp.json()
cache.parent.mkdir(parents=True, exist_ok=True)
cache.write_text(json.dumps(data), encoding="utf-8")
return data
except Exception as e: # offline / 404 / bad json
logger.warning("manifest fetch failed for %s: %s", source, e)
if cached is not None:
# This mtime is a last-*check* marker. Advancing it on failure keeps
# an offline app responsive while guaranteeing another check after
# the bounded freshness interval.
with contextlib.suppress(OSError):
os.utime(cache, (checked_at, checked_at))
return cached
if cache.exists():
try:
return json.loads(cache.read_text(encoding="utf-8"))
except Exception:
pass
return None
@@ -432,385 +214,6 @@ def community_submit_url(item_type: str = Query("preset", alias="type"), source:
return {"url": f"https://github.com/{src}/issues/new?template={template}"}
def _find_item(items: list[dict], item_id: str) -> dict:
if not _ITEM_ID_RE.fullmatch(item_id or ""):
raise HTTPException(status_code=404, detail="Item not found in the gallery.")
item = next((it for it in items if it["id"] == item_id), None)
if item is None:
raise HTTPException(status_code=404, detail="Item not found in the gallery.")
return item
def _canonical_archetype(item: dict) -> Optional[dict]:
"""The built-in archetype represented exactly by a marketplace preset."""
if item.get("type") != "preset":
return None
canonical = archetypes.get_archetype(item["id"])
if canonical is None:
return None
if (canonical.get("instruct") != item.get("instruct")
or canonical.get("language") != item.get("language")):
return None
remote_script = (item.get("sample_script") or "").strip()
if remote_script and remote_script != (canonical.get("sample_script") or "").strip():
return None
return canonical
def _preset_preview_path(item: dict) -> Path:
fingerprint = hashlib.sha256(
json.dumps({
"instruct": item.get("instruct"),
"language": item.get("language"),
"sample_script": item.get("sample_script"),
}, sort_keys=True).encode("utf-8")
).hexdigest()[:16]
return _CACHE_DIR / "previews" / f"{item['id']}-{fingerprint}.wav"
def _voice_audio_fingerprint(item: dict) -> str:
audio = item.get("audio") or {}
return hashlib.sha256(
f"{audio.get('url', '')}|{audio.get('sha256', '')}".encode("utf-8")
).hexdigest()[:16]
def _voice_audio_path(item: dict) -> Path:
return _CACHE_DIR / "audio" / f"{item['id']}-{_voice_audio_fingerprint(item)}.wav"
async def _render_preset_atomic(item: dict, out_path: Path) -> Path:
if is_playable_wav(out_path):
return out_path
from api.routers.archetypes import _render_archetype_wav
out_path.parent.mkdir(parents=True, exist_ok=True)
fd, tmp_name = tempfile.mkstemp(dir=str(out_path.parent), prefix=".preview-", suffix=".wav")
os.close(fd)
tmp = Path(tmp_name)
try:
await _render_archetype_wav({
"instruct": item["instruct"],
"language": item.get("language", "English"),
"sample_script": (
(item.get("sample_script") or "").strip()
or "Hello — this is a preview of this voice."
),
}, tmp)
if not is_playable_wav(tmp):
raise RuntimeError("the voice engine produced an invalid preview WAV")
os.replace(tmp, out_path)
return out_path
finally:
with contextlib.suppress(OSError):
tmp.unlink()
def _download_voice_audio(item: dict, out_path: Path, *, client=None) -> None:
"""Stream one allow-listed voice clip into an atomic, size-bounded file."""
audio = item.get("audio") or {}
url = audio.get("url", "")
if not _safe_audio_url(url):
raise HTTPException(status_code=400, detail="Voice audio URL is not from an allowed host.")
import httpx
owned_client = client is None
http = client or httpx.Client(timeout=30.0, follow_redirects=False)
out_path.parent.mkdir(parents=True, exist_ok=True)
fd, tmp_name = tempfile.mkstemp(dir=str(out_path.parent), prefix=".voice-", suffix=".part")
total = 0
digest = hashlib.sha256()
try:
with os.fdopen(fd, "wb") as handle:
current_url = url
downloaded = False
for _redirect in range(6):
with http.stream("GET", current_url, follow_redirects=False) as response:
if response.status_code in (301, 302, 303, 307, 308):
location = response.headers.get("location")
next_url = urljoin(current_url, location or "")
if not location or not _safe_audio_url(next_url):
raise HTTPException(
status_code=502,
detail="Community voice audio redirected to a disallowed host.",
)
current_url = next_url
continue
response.raise_for_status()
length = response.headers.get("content-length")
if length:
try:
if int(length) > _MAX_VOICE_AUDIO_BYTES:
raise HTTPException(
status_code=502,
detail="Community voice audio exceeded the download size limit.",
)
except ValueError:
# A non-numeric Content-Length header is the
# server's problem, not a reason to refuse the
# download — the streamed byte counter below
# still enforces the same cap on what actually
# arrives.
pass
for chunk in response.iter_bytes():
if not chunk:
continue
total += len(chunk)
if total > _MAX_VOICE_AUDIO_BYTES:
raise HTTPException(
status_code=502,
detail="Community voice audio exceeded the download size limit.",
)
digest.update(chunk)
handle.write(chunk)
downloaded = True
break
if not downloaded:
raise HTTPException(
status_code=502,
detail="Community voice audio followed too many redirects.",
)
if total == 0:
raise HTTPException(status_code=502, detail="Community voice audio was empty.")
expected = audio.get("sha256")
if expected and digest.hexdigest() != expected:
raise HTTPException(
status_code=502,
detail="Downloaded voice failed its integrity check.",
)
handle.flush()
os.fsync(handle.fileno())
if not is_playable_wav(Path(tmp_name)):
raise HTTPException(
status_code=502, detail="Community voice audio was not a valid WAV.",
)
os.replace(tmp_name, out_path)
except BaseException:
with contextlib.suppress(OSError):
os.unlink(tmp_name)
raise
finally:
if owned_client:
http.close()
def _cached_voice_audio(item: dict) -> Path:
path = _voice_audio_path(item)
if is_playable_wav(path):
return path
with contextlib.suppress(OSError):
path.unlink()
_download_voice_audio(item, path)
return path
def _copy_atomic(source: Path, destination: Path) -> None:
destination.parent.mkdir(parents=True, exist_ok=True)
fd, tmp_name = tempfile.mkstemp(
dir=str(destination.parent), prefix=f".{destination.name}-", suffix=".part",
)
try:
with os.fdopen(fd, "wb") as out, source.open("rb") as src:
shutil.copyfileobj(src, out)
out.flush()
os.fsync(out.fileno())
os.replace(tmp_name, destination)
except BaseException:
with contextlib.suppress(OSError):
os.unlink(tmp_name)
raise
@router.get("/community/items/{item_id}/preview")
async def community_preview(
item_id: str,
local: bool = Query(False, description="Bypass canonical gallery audio after decode failure"),
):
"""Serve every community preview through the authenticated same-origin API."""
_, items, _, _ = await asyncio.to_thread(_load, False)
item = _find_item(items, item_id)
canonical = _canonical_archetype(item)
if canonical is not None:
# Reuse the signed-gallery/local-render fallback and cache owned by the
# canonical endpoint rather than synthesizing the same preset twice.
# Delegate in-process: a root-relative HTTP redirect drops supported
# reverse-proxy path prefixes such as ``https://host/api``.
from api.routers.archetypes import preview_archetype
return await preview_archetype(canonical["id"], local=local)
try:
if item["type"] == "preset":
path = await _render_preset_atomic(item, _preset_preview_path(item))
else:
path = await asyncio.to_thread(_cached_voice_audio, item)
except HTTPException:
raise
except Exception as exc:
logger.warning("Community preview unavailable (%s)", type(exc).__name__)
raise HTTPException(
status_code=503, detail="This community voice preview is unavailable right now.",
) from exc
return FileResponse(
path, media_type="audio/wav",
headers={"Cache-Control": "no-cache", "X-OmniVoice-Preview-Source": "community"},
)
def _profile_fields(item: dict) -> tuple[str, str, Optional[str], Optional[int]]:
if item["type"] == "preset":
return "design", item["instruct"], json.dumps(item["attrs"]), 42
return "clone", "", None, None
def _community_profile_audio_filename(profile_id: str, item: dict) -> str:
safe_id = (
profile_id if re.fullmatch(r"[A-Za-z0-9_-]{1,64}", profile_id or "")
else hashlib.sha256(str(profile_id).encode("utf-8")).hexdigest()[:16]
)
if item["type"] == "voice":
# The manifest URL/checksum fingerprint makes a changed submission
# invalidate its already-materialized clone without a schema change.
return f"{safe_id}-community-{_voice_audio_fingerprint(item)}.wav"
return f"{safe_id}.wav"
def _stored_profile_audio(ref_audio_path: object) -> Optional[Path]:
return resolve_regular_file(VOICES_DIR, ref_audio_path)
def _community_audio_is_current(row, item: dict, ref_text: str) -> bool:
path = _stored_profile_audio(row["ref_audio_path"])
expected_filename = _community_profile_audio_filename(row["id"], item)
if row["ref_audio_path"] != expected_filename or not is_playable_wav(path):
return False
kind, instruct, _vd_states, seed = _profile_fields(item)
inputs_match = (
row["instruct"] == instruct
and row["language"] == item.get("language", "Auto")
and row["ref_text"] == ref_text
and row["seed"] == seed
)
if not inputs_match:
return False
return True
async def _materialize_item_audio(
item: dict, profile_id: str, *, publish: bool = True,
) -> tuple[str, Path]:
"""Copy the current manifest audio, optionally staging it for a later CAS."""
audio_filename = _community_profile_audio_filename(profile_id, item)
destination = Path(VOICES_DIR) / audio_filename
audio_path = destination
if not publish:
destination.parent.mkdir(parents=True, exist_ok=True)
audio_path = destination.parent / f".{Path(audio_filename).stem}-{uuid.uuid4().hex}.staged.wav"
if item["type"] == "preset":
cached = await _render_preset_atomic(item, _preset_preview_path(item))
else:
cached = await asyncio.to_thread(_cached_voice_audio, item)
await asyncio.to_thread(_copy_atomic, cached, audio_path)
return audio_filename, audio_path
def _community_personality(item: dict) -> str:
source = item.get("_source_repo")
if not isinstance(source, str) or not _SOURCE_RE.fullmatch(source):
source = _DEFAULT_SOURCES[0]
return f"community:{source}:{item['id']}"
def _is_materialized_community_row(row, item: dict) -> bool:
if (
row["personality"] != _community_personality(item)
or row["is_locked"] or row["verified_own_voice"]
):
return False
if item["type"] == "voice":
safe_id = Path(_community_profile_audio_filename(row["id"], item)).name.split(
"-community-", 1,
)[0]
return bool(
row["kind"] == "clone"
and row["seed"] is None
and not row["vd_states"]
and row["instruct"] == ""
and row["language"] == item.get("language", "Auto")
and row["ref_text"] == (item.get("audio") or {}).get("ref_text", "")
and re.fullmatch(
rf"{re.escape(safe_id)}-community-[0-9a-f]{{16}}\.wav",
row["ref_audio_path"] or "",
)
)
try:
states = json.loads(row["vd_states"])
except (TypeError, ValueError):
return False
return bool(
row["kind"] == "design"
and row["seed"] == 42
and row["ref_audio_path"] == _community_profile_audio_filename(row["id"], item)
and row["instruct"] == item["instruct"]
and row["language"] == item.get("language", "Auto")
and row["ref_text"] == (item.get("sample_script") or "")
and states == item["attrs"]
)
def _existing_community_profile(conn, item: dict, personality: str):
candidates = conn.execute(
"SELECT * FROM voice_profiles WHERE personality=? ORDER BY created_at, id",
(personality,),
).fetchall()
existing = next(
(row for row in candidates if _is_materialized_community_row(row, item)), None,
)
if existing is not None:
return existing
# Old builds stored the bare item id. Import formats preserve arbitrary
# personality text too, so adopt only the exact shape the old materializer
# wrote; otherwise a remote item id could rewrite a user's imported voice.
if archetypes.get_archetype(item["id"]) is None:
legacy = conn.execute(
"SELECT * FROM voice_profiles WHERE personality=? LIMIT 1",
(item["id"],),
).fetchone()
if legacy is not None:
kind, instruct, _vd_states, _seed = _profile_fields(item)
ref_text = item.get("sample_script") or (item.get("audio") or {}).get(
"ref_text", "",
)
if (
legacy["ref_audio_path"] == f"{legacy['id']}.wav"
and legacy["kind"] == kind
and legacy["instruct"] == instruct
and legacy["language"] == item.get("language", "Auto")
and legacy["ref_text"] == ref_text
and legacy["seed"] is None
and not legacy["vd_states"]
and not legacy["is_locked"]
and not legacy["verified_own_voice"]
):
return legacy
return None
def _heal_existing_profile(
conn, row, item: dict, ref_text: str, personality: str, audio_filename: str,
) -> None:
kind, instruct, vd_states, seed = _profile_fields(item)
conn.execute(
"UPDATE voice_profiles SET kind=?, instruct=?, vd_states=?, language=?, "
"ref_text=?, seed=?, personality=?, ref_audio_path=? WHERE id=?",
(
kind, instruct, vd_states, item.get("language", "Auto"), ref_text,
seed, personality, audio_filename, row["id"],
),
)
@router.post("/community/items/{item_id}/use")
async def community_use(item_id: str, name: Optional[str] = Query(None)):
"""Materialize a community item into a reusable voice profile.
@@ -820,108 +223,76 @@ async def community_use(item_id: str, name: Optional[str] = Query(None)):
``voice_profiles`` row usable everywhere voices are picked.
"""
_, items, _, _ = await asyncio.to_thread(_load, False)
item = _find_item(items, item_id)
canonical = _canonical_archetype(item)
if canonical is not None:
from api.routers.archetypes import use_archetype
return await use_archetype(canonical["id"], name)
item = next((it for it in items if it["id"] == item_id), None)
if item is None:
raise HTTPException(status_code=404, detail="Item not found in the gallery.")
import time
import uuid
from core import event_bus
from core.db import db_conn
from core.config import VOICES_DIR
ref_text = item.get("sample_script") or (item.get("audio") or {}).get("ref_text", "")
personality = _community_personality(item)
with db_conn() as conn:
existing = _existing_community_profile(conn, item, personality)
profile_id = existing["id"] if existing is not None else str(uuid.uuid4())[:8]
audio_path: Optional[Path] = None
if existing is not None and _community_audio_is_current(existing, item, ref_text):
audio_filename = existing["ref_audio_path"]
else:
try:
audio_filename, audio_path = await _materialize_item_audio(
item, profile_id, publish=existing is None,
)
except HTTPException:
raise
except Exception as e:
logger.error("Community 'use' failed", exc_info=True)
raise HTTPException(
status_code=503, detail="Couldn't add this voice right now.",
) from e
if existing is not None:
with db_conn() as conn:
conn.execute("BEGIN IMMEDIATE")
current = conn.execute(
"SELECT * FROM voice_profiles WHERE id=?", (existing["id"],),
).fetchone()
owned = _existing_community_profile(conn, item, personality)
still_owned = current is not None and (
_is_materialized_community_row(current, item)
or (owned is not None and owned["id"] == current["id"])
)
if still_owned:
if audio_path is not None:
destination = Path(VOICES_DIR) / audio_filename
os.replace(audio_path, destination)
audio_path = None
_heal_existing_profile(
conn, current, item, ref_text, personality, audio_filename,
)
existing_result = {"profile_id": current["id"], "name": current["name"]}
else:
existing_result = None
if existing_result is not None:
event_bus.emit("profiles", {"action": "updated", "id": existing_result["profile_id"]})
return existing_result
profile_id = str(uuid.uuid4())[:8]
audio_filename = _community_profile_audio_filename(profile_id, item)
destination = Path(VOICES_DIR) / audio_filename
if audio_path is None:
audio_filename, audio_path = await _materialize_item_audio(item, profile_id)
else:
os.replace(audio_path, destination)
audio_path = destination
if audio_path is None: # defensive: a new profile always materialized above
raise RuntimeError("new community profile has no materialized audio")
profile_id = str(uuid.uuid4())[:8]
audio_filename = f"{profile_id}.wav"
audio_path = Path(VOICES_DIR) / audio_filename
profile_name = (name or item["name"]).strip() or item["name"]
kind, instruct, vd_states, seed = _profile_fields(item)
instruct = item.get("instruct", "") if item["type"] == "preset" else ""
ref_text = item.get("sample_script") or (item.get("audio") or {}).get("ref_text", "")
try:
if item["type"] == "preset":
from api.routers.archetypes import _render_archetype_wav
pseudo = {
"instruct": instruct,
"language": item.get("language", "English"),
"sample_script": ref_text or "Hello — this is a preview of this voice.",
}
await _render_archetype_wav(pseudo, audio_path)
else: # voice — download the reference clip (off the event loop)
await asyncio.to_thread(_download_voice_audio, item, audio_path)
except HTTPException:
raise
except Exception as e:
logger.error("Community 'use' failed", exc_info=True)
raise HTTPException(status_code=503, detail=f"Couldn't add this voice right now. Error: {e}")
try:
# A community "preset" is a synthetic designed voice (rendered from an
# instruct string) → kind='design'; a "voice" carries a real reference
# clip → kind='clone'. Setting kind makes the persona-gallery
# synthetic-only gating work (§R3) instead of defaulting all imports to
# 'clone'.
kind = "design" if item["type"] == "preset" else "clone"
with db_conn() as conn:
conn.execute("BEGIN IMMEDIATE")
duplicate = _existing_community_profile(conn, item, personality)
if duplicate is not None:
duplicate_audio = duplicate["ref_audio_path"]
if not _community_audio_is_current(duplicate, item, ref_text):
duplicate_audio = _community_profile_audio_filename(duplicate["id"], item)
duplicate_path = Path(VOICES_DIR) / duplicate_audio
_copy_atomic(audio_path, duplicate_path)
_heal_existing_profile(
conn, duplicate, item, ref_text, personality, duplicate_audio,
)
with contextlib.suppress(OSError):
audio_path.unlink()
duplicate_result = {"profile_id": duplicate["id"], "name": duplicate["name"]}
else:
duplicate_result = None
if duplicate_result is None:
conn.execute(
"INSERT INTO voice_profiles "
"(id, name, ref_audio_path, ref_text, instruct, language, seed, personality, "
"created_at, kind, vd_states) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
(profile_id, profile_name, audio_filename, ref_text, instruct,
item.get("language", "Auto"), seed, personality, time.time(), kind, vd_states),
)
conn.execute(
"INSERT INTO voice_profiles "
"(id, name, ref_audio_path, ref_text, instruct, language, seed, personality, created_at, kind) "
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
(profile_id, profile_name, audio_filename, ref_text, instruct,
item.get("language", "Auto"), None, item["id"], time.time(), kind),
)
except Exception:
with contextlib.suppress(OSError):
audio_path.unlink()
with __import__("contextlib").suppress(OSError):
os.remove(audio_path)
raise
if duplicate_result is not None:
event_bus.emit("profiles", {"action": "updated", "id": duplicate_result["profile_id"]})
return duplicate_result
event_bus.emit("profiles", {"action": "created", "id": profile_id})
return {"profile_id": profile_id, "name": profile_name}
def _download_voice_audio(item: dict, out_path: Path) -> None:
import hashlib
audio = item.get("audio") or {}
url = audio.get("url", "")
if not _safe_audio_url(url):
raise HTTPException(status_code=400, detail="Voice audio URL is not from an allowed host.")
import httpx
with httpx.Client(timeout=30.0, follow_redirects=True) as client:
resp = client.get(url)
resp.raise_for_status()
data = resp.content
expected = audio.get("sha256")
if expected and hashlib.sha256(data).hexdigest() != expected:
raise HTTPException(status_code=502, detail="Downloaded voice failed its integrity check.")
out_path.parent.mkdir(parents=True, exist_ok=True)
out_path.write_bytes(data)
+10 -41
View File
@@ -21,10 +21,9 @@ import logging
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from typing import Literal, Optional
from typing import Optional
from api.dependencies import require_local
from api.public_engine_metadata import public_unavailability
from core import prefs
from services import sherpa_dictation as sd
@@ -81,28 +80,11 @@ def list_dictation_models():
return {
"models": out,
"engine_available": available,
"engine_reason": None if available else public_unavailability(reason),
"engine_reason": None if available else reason,
"default_model_id": sd.DEFAULT_MODEL_ID,
}
@router.get("/dictation/readiness", dependencies=[Depends(require_local)])
def dictation_readiness(
model_id: str | None = None,
purpose: Literal["dictation", "transcribe"] = "dictation",
) -> dict:
"""Check a selected ASR path without loading or downloading weights."""
from services.asr_backend import asr_model_missing_error
missing = asr_model_missing_error(
purpose=purpose,
sherpa_model_id=(model_id or _read_prefs()["model_id"])
if purpose == "dictation"
else None,
)
return {"ready": missing is None, "missing": missing}
@router.get("/dictation/prefs", dependencies=[Depends(require_local)])
def get_dictation_prefs():
return _read_prefs()
@@ -118,13 +100,13 @@ class DictationPrefsUpdate(BaseModel):
def set_dictation_prefs(req: DictationPrefsUpdate):
"""Persist any subset of the dictation prefs. Validates ``mode`` and
``model_id`` so a bad value can't wedge the capture engine."""
canonical = None
if req.mode is not None:
if req.mode not in _VALID_MODES:
raise HTTPException(
status_code=400,
detail=f"mode must be one of {_VALID_MODES}",
)
prefs.set_(PREF_MODE, req.mode)
if req.model_id is not None:
if not sd.is_sherpa_model(req.model_id):
raise HTTPException(
@@ -133,26 +115,6 @@ def set_dictation_prefs(req: DictationPrefsUpdate):
)
# Normalise to the canonical dictation id (accept repo_id too).
canonical = sd.get_spec(req.model_id).id
# Reset before persisting: if the capture service is unavailable, the
# request fails without claiming that settings which are not active were
# saved. A reset is safe even when a later preference write fails; the old
# persisted selection is simply loaded again on next capture.
try:
from services import asr_backend
asr_backend._capture_backend = None
asr_backend._capture_backend_key = None
except Exception as exc:
logger.warning("Dictation capture backend could not be reset")
raise HTTPException(
status_code=503,
detail="Dictation settings could not be applied. Retry after the capture service is ready.",
) from exc
if req.mode is not None:
prefs.set_(PREF_MODE, req.mode)
if canonical is not None:
prefs.set_(PREF_MODEL_ID, canonical)
# Explicitly choosing a model clears any auto-demotion: the user is in
# charge, and a sherpa upgrade may well have fixed the decoder that
@@ -161,4 +123,11 @@ def set_dictation_prefs(req: DictationPrefsUpdate):
sd.clear_demotion(canonical)
if req.enabled is not None:
prefs.set_(PREF_ENABLED, bool(req.enabled))
# Rebuild the cached capture singleton so the change takes effect at once.
try:
from services import asr_backend
asr_backend._capture_backend = None
asr_backend._capture_backend_key = None
except Exception:
pass
return _read_prefs()
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+70 -385
View File
@@ -1,32 +1,19 @@
import json
import os
import time
import asyncio
import logging
from typing import Optional
from fastapi import APIRouter, HTTPException
from fastapi import APIRouter
from fastapi.responses import JSONResponse
from schemas.requests import AgentFitRequest, TranslateRequest
from schemas.requests import TranslateRequest
from services.model_manager import _cpu_pool, _gpu_pool
from services.hf_revisions import revision_for
from services.translator import cinematic_available, cinematic_refine_many, _cinematic_budget
from api.routers.dub_core import _get_job, _save_job
router = APIRouter()
logger = logging.getLogger("omnivoice.api")
_NLLB_REPO_ID = "facebook/nllb-200-distilled-600M"
def _load_nllb_component(factory):
"""Load explicitly installed NLLB weights at their reviewed revision."""
return factory.from_pretrained(
_NLLB_REPO_ID,
revision=revision_for(_NLLB_REPO_ID),
local_files_only=True,
)
TRANSLATE_CODES = {
"en": "en", "es": "es", "fr": "fr", "de": "de", "it": "it", "pt": "pt",
"ru": "ru", "ja": "ja", "ko": "ko", "zh": "zh-CN", "cmn-Hans": "zh-CN",
@@ -41,33 +28,8 @@ FLORES_CODES = {
"hi": "hin_Deva", "tr": "tur_Latn", "pl": "pol_Latn", "nl": "nld_Latn",
"sv": "swe_Latn", "th": "tha_Thai", "vi": "vie_Latn", "id": "ind_Latn",
"uk": "ukr_Cyrl",
"zh-TW": "zho_Hant", "zh-Hant": "zho_Hant", "cmn-Hant": "zho_Hant",
"zh-Hans": "zho_Hans", "yue": "yue_Hant",
"bn": "ben_Beng", "ta": "tam_Taml", "te": "tel_Telu", "ml": "mal_Mlym",
"kn": "kan_Knda", "gu": "guj_Gujr", "mr": "mar_Deva", "ur": "urd_Arab",
"fa": "pes_Arab", "he": "heb_Hebr", "el": "ell_Grek", "cs": "ces_Latn",
"da": "dan_Latn", "fi": "fin_Latn", "nb": "nob_Latn", "nn": "nno_Latn",
"ro": "ron_Latn", "hu": "hun_Latn", "bg": "bul_Cyrl", "sk": "slk_Latn",
"sl": "slv_Latn", "hr": "hrv_Latn", "sr": "srp_Cyrl", "lt": "lit_Latn",
"et": "est_Latn", "sw": "swh_Latn", "af": "afr_Latn", "ms": "zsm_Latn",
}
def _nllb_language(code: str) -> str | None:
"""Resolve aliases or tokenizer-supported FLORES codes without loading weights."""
from transformers.models.nllb.tokenization_nllb import FAIRSEQ_LANGUAGE_CODES
normalized = code.strip().replace("_", "-").lower()
aliases = {key.lower(): value for key, value in FLORES_CODES.items()}
if normalized in aliases:
return aliases[normalized]
exact = [value for value in FAIRSEQ_LANGUAGE_CODES if value.replace("_", "-").lower() == normalized]
if exact:
return exact[0]
# Bare ISO-639-3 codes are safe only when the tokenizer has one script.
matches = [value for value in FAIRSEQ_LANGUAGE_CODES if value.split("_")[0] == normalized]
return matches[0] if len(matches) == 1 else None
# Human-readable language names for LLM prompts. Empirically a tiny / 7B
# local LLM produces Devanagari Hindi reliably when told "translate into
# Hindi" but drifts to German / English / phonetic-Latin when told
@@ -190,77 +152,9 @@ def _looks_like_target(text: str, code: str, threshold: float = 0.5) -> bool:
codepoints alone."""
return _script_ratio(text, code) >= threshold
def _translation_output_error(text: object) -> str | None:
"""Reject provider error pages that arrive with HTTP 200.
Google's mobile endpoint occasionally returns its generic HTML error copy
inside the element deep-translator treats as a successful translation.
Passing that through would replace the user's transcript with the error
page, so treat it like any other transient provider failure and retry.
"""
if not isinstance(text, str) or not text.strip():
return "empty translation"
normalized = " ".join(text.split()).casefold()
error_markers = (
"error 500 (server error)",
"that's an error",
"thats an error",
"there was an error. please try again later",
"no translation was found using the current translator",
)
if "\ufffd" in text or any(marker in normalized for marker in error_markers):
return "translation provider returned invalid output"
return None
_nllb_model = None
_nllb_tokenizer = None
_nllb_device = None
_NLLB_BATCH_SIZE_ENV = "OMNIVOICE_NLLB_BATCH_SIZE"
_NLLB_MAX_BATCH_SIZE = 32
def _nllb_batch_size() -> int:
"""Bound NLLB forward-pass width; explicit overrides remain available."""
configured = os.environ.get(_NLLB_BATCH_SIZE_ENV, "").strip()
if configured:
try:
return max(1, min(_NLLB_MAX_BATCH_SIZE, int(configured)))
except (TypeError, ValueError):
logger.warning("%s=%r is not an integer; using the safe default", _NLLB_BATCH_SIZE_ENV, configured)
# The 600M checkpoint leaves ample room on modern discrete GPUs. Scale the
# forward-pass width there; CPU and unified-memory MPS keep the conservative
# width because their failure recovery moves the whole model.
if _nllb_device == "cuda":
try:
import torch
free_gib = int(torch.cuda.mem_get_info()[0]) / 1024**3
if free_gib >= 16:
return 24
if free_gib >= 8:
return 12
except Exception:
pass
return 8
return 4
def _nllb_hypothesis_budget() -> int:
"""Bound batch × beam hypotheses by currently available device memory."""
if _nllb_device != "cuda":
return 16
try:
import torch
free_gib = int(torch.cuda.mem_get_info()[0]) / 1024**3
if free_gib >= 16:
return 64
if free_gib >= 8:
return 32
except Exception:
pass
return 16
def _dialect_flags(req, applied: bool) -> dict:
@@ -350,11 +244,10 @@ def _resolve_translation_context(req, client, model_name: str, timeout: float,
def _unload_nllb():
"""Release NLLB VRAM so TTS model can reload."""
global _nllb_device, _nllb_model, _nllb_tokenizer
global _nllb_model, _nllb_tokenizer
import gc
_nllb_model = None
_nllb_tokenizer = None
_nllb_device = None
gc.collect()
try:
import torch
@@ -366,33 +259,10 @@ def _unload_nllb():
pass
def _should_unload_nllb() -> bool:
"""Retain a warm local translator only when the accelerator has safe headroom."""
override = os.environ.get("OMNIVOICE_UNLOAD_NLLB")
if override is not None:
return override.strip().lower() not in {"0", "false", "no", "off"}
if _nllb_device != "cuda":
return True
try:
import torch
free_bytes, total_bytes = torch.cuda.mem_get_info()
return total_bytes < 16 * 1024**3 or free_bytes < 8 * 1024**3
except Exception:
return True
@router.post("/dub/translate")
async def dub_translate(req: TranslateRequest):
try:
provider = (req.provider if req.provider else os.environ.get("TRANSLATE_PROVIDER", "google")).lower()
from services import translation_engines
if not translation_engines.get_engine(provider):
return JSONResponse(
status_code=400,
content={"error": "Choose a supported translation engine."},
)
lang_code = TRANSLATE_CODES.get(req.target_lang, req.target_lang)
api_key = os.environ.get("TRANSLATE_API_KEY", "")
loop = asyncio.get_running_loop()
@@ -400,16 +270,8 @@ async def dub_translate(req: TranslateRequest):
# Offline NLLB Transformer Translation
if provider == "nllb":
requested = [src_lang, req.target_lang, *(seg.target_lang for seg in req.segments if seg.target_lang)]
resolved = {code: _nllb_language(code) for code in requested}
unsupported = [code for code, language in resolved.items() if language is None]
if unsupported:
return JSONResponse(status_code=400, content={
"error": "NLLB does not support the requested language.",
"code": "unsupported_translation_language", "languages": unsupported,
})
flores_tgt = resolved[req.target_lang]
flores_src = resolved[src_lang]
flores_tgt = FLORES_CODES.get(req.target_lang, "eng_Latn")
flores_src = FLORES_CODES.get(src_lang, "eng_Latn")
def _translate_nllb():
global _nllb_model, _nllb_tokenizer, _nllb_device
@@ -425,9 +287,9 @@ async def dub_translate(req: TranslateRequest):
try:
if _nllb_tokenizer is None:
_nllb_tokenizer = _load_nllb_component(AutoTokenizer)
_nllb_tokenizer = AutoTokenizer.from_pretrained("facebook/nllb-200-distilled-600M")
if _nllb_model is None:
_nllb_model = _load_nllb_component(AutoModelForSeq2SeqLM)
_nllb_model = AutoModelForSeq2SeqLM.from_pretrained("facebook/nllb-200-distilled-600M")
if target_device != "cpu":
try:
_nllb_model = _nllb_model.to(target_device)
@@ -441,112 +303,44 @@ async def dub_translate(req: TranslateRequest):
logger.exception("NLLB model load failed")
return [{"id": seg.id, "text": seg.text, "error": f"Model load error: {str(e)}"} for seg in req.segments]
from services.performance_profiles import translation_decode_defaults
# Snapshot once so every segment and device fallback in this
# job uses the same decoding effort even if preferences change.
decode_options = translation_decode_defaults()
def _generate_rows(rows, target_language):
global _nllb_device
_nllb_tokenizer.src_lang = flores_src
inputs = _nllb_tokenizer(
[seg.text for _, seg in rows],
return_tensors="pt",
padding=True,
)
if _nllb_device and _nllb_device != "cpu":
inputs = {key: value.to(_nllb_device) for key, value in inputs.items()}
forced_bos_token_id = _nllb_tokenizer.convert_tokens_to_ids(target_language)
results = []
for seg in req.segments:
try:
tokens = _nllb_model.generate(
**inputs,
forced_bos_token_id=forced_bos_token_id,
max_length=400,
**decode_options,
)
except (RuntimeError, NotImplementedError) as error:
if _nllb_device != "mps":
raise
logger.warning("MPS generate failed, retrying on CPU: %s", error)
_nllb_model.to("cpu")
_nllb_device = "cpu"
inputs = {key: value.to("cpu") for key, value in inputs.items()}
tokens = _nllb_model.generate(
**inputs,
forced_bos_token_id=forced_bos_token_id,
max_length=400,
**decode_options,
)
decoded = _nllb_tokenizer.batch_decode(tokens, skip_special_tokens=True)
if len(decoded) != len(rows):
raise RuntimeError(
f"NLLB returned {len(decoded)} translations for {len(rows)} segments"
)
return decoded
# A target-language BOS token is shared by a forward pass, so
# group mixed-language rows first. Preserve request order in
# the final response even though groups render independently.
grouped: dict[str, list[tuple[int, object]]] = {}
results_by_index: dict[int, dict] = {}
for index, seg in enumerate(req.segments):
if not seg.text or not seg.text.strip():
results_by_index[index] = {"id": seg.id, "text": seg.text}
continue
target = resolved[seg.target_lang] if seg.target_lang else flores_tgt
grouped.setdefault(target, []).append((index, seg))
# Beam search multiplies decoder memory per row. Keep the
# effective hypothesis count bounded while still widening the
# Fast path aggressively.
beam_count = max(1, int(decode_options.get("num_beams", 1)))
width = min(
_nllb_batch_size(),
max(1, _nllb_hypothesis_budget() // beam_count),
)
for target, rows in grouped.items():
for start in range(0, len(rows), width):
batch = rows[start : start + width]
try:
translated_texts = _generate_rows(batch, target)
except Exception as batch_error:
if len(batch) == 1:
index, seg = batch[0]
results_by_index[index] = {
"id": seg.id,
"text": seg.text,
"error": str(batch_error),
}
continue
# A single unusually long row must not sink its
# neighbours. Clear a failed device allocation and
# retain the established per-segment degradation.
if torch.cuda.is_available():
torch.cuda.empty_cache()
logger.warning(
"NLLB batch of %d failed; retrying rows individually: %s",
len(batch),
batch_error,
)
for index, seg in batch:
try:
translated_text = _generate_rows([(index, seg)], target)[0]
results_by_index[index] = {"id": seg.id, "text": translated_text}
except Exception as row_error:
results_by_index[index] = {
"id": seg.id,
"text": seg.text,
"error": str(row_error),
}
if not seg.text or not seg.text.strip():
results.append({"id": seg.id, "text": seg.text})
continue
for (index, seg), translated_text in zip(batch, translated_texts):
results_by_index[index] = {"id": seg.id, "text": translated_text}
return [results_by_index[index] for index in range(len(req.segments))]
tgt = FLORES_CODES.get(seg.target_lang, flores_tgt) if seg.target_lang else flores_tgt
_nllb_tokenizer.src_lang = flores_src
inputs = _nllb_tokenizer(seg.text, return_tensors="pt")
if _nllb_device and _nllb_device != "cpu":
inputs = {k: v.to(_nllb_device) for k, v in inputs.items()}
forced_bos_token_id = _nllb_tokenizer.convert_tokens_to_ids(tgt)
try:
translated_tokens = _nllb_model.generate(
**inputs, forced_bos_token_id=forced_bos_token_id, max_length=400
)
except (RuntimeError, NotImplementedError) as e:
if _nllb_device == "mps":
logger.warning("MPS generate failed, retrying on CPU: %s", e)
_nllb_model.to("cpu")
_nllb_device = "cpu"
inputs = {k: v.to("cpu") for k, v in inputs.items()}
translated_tokens = _nllb_model.generate(
**inputs, forced_bos_token_id=forced_bos_token_id, max_length=400
)
else:
raise
translated_text = _nllb_tokenizer.batch_decode(translated_tokens, skip_special_tokens=True)[0]
results.append({"id": seg.id, "text": translated_text})
except Exception as e:
results.append({"id": seg.id, "text": seg.text, "error": str(e)})
return results
translated = await loop.run_in_executor(_gpu_pool, _translate_nllb)
if _should_unload_nllb():
if os.environ.get("OMNIVOICE_UNLOAD_NLLB", "1") == "1":
_unload_nllb()
# Cinematic/Autofit refine + rate-ratio badges must run for NLLB too
# (previously this returned before _maybe_cinematic, so a Cinematic
@@ -667,7 +461,6 @@ async def dub_translate(req: TranslateRequest):
f"You are a professional dubbing translator. "
f"Translate the user's text from {src_name} into "
f"{tgt_name}.{script_clause}{dia_clause} "
f"{translation_style_brief(req)} "
f"Reply ONLY with the translated {tgt_name} text, do not "
f"add quotes, notes, headers, explanations, or commentary."
)
@@ -739,7 +532,7 @@ async def dub_translate(req: TranslateRequest):
source_lang=src_lang,
target_lang=tgt_code,
target_name=LANG_NAMES.get(tgt_code, tgt_code),
extra_clause="\n".join(filter(None, [context_extra, translation_style_brief(req)])),
extra_clause=context_extra,
)
except Exception as e: # noqa: BLE001
logger.warning("reflect pass skipped for %s: %s",
@@ -790,57 +583,18 @@ async def dub_translate(req: TranslateRequest):
f"switch the Engine dropdown to another provider."
)
return JSONResponse(status_code=400, content={"error": friendly})
# The package imports without its native dep; the *translator*
# needs CTranslate2, whose library is rejected outright by kernels
# that refuse an executable stack (#692). Repair it (a one-bit ELF
# patch), and if that is impossible say so in one actionable 400
# instead of the opaque 500 every segment used to produce.
try:
from core.execstack import ensure_ctranslate2_loadable
ensure_ctranslate2_loadable()
except Exception as e: # noqa: BLE001 — repair must not block translation
logger.debug("exec-stack repair unavailable (%s) — continuing", e)
try:
import argostranslate.translate # noqa: F401
except Exception as e: # noqa: BLE001 — OSError here, not ImportError
friendly = (
f"The '{provider}' engine's CTranslate2 runtime could not be "
"loaded in this backend."
+ " Switch the Engine dropdown to NLLB (local) or an online "
"provider, or reinstall the backend, then retry."
)
return JSONResponse(status_code=400, content={"error": friendly, "detail": {"code": "argos_runtime_unavailable", "message": friendly}})
target_codes = list(dict.fromkeys(
seg.target_lang if seg.target_lang else req.target_lang
for seg in req.segments
))
try:
pack_status = translation_engines.argos_pack_status(src_lang, target_codes)
except (ImportError, ValueError) as exc:
return JSONResponse(status_code=422, content={"error": str(exc)})
missing_packs = [
pair for pair in pack_status["pairs"] if not pair["installed"]
]
if missing_packs:
pairs = ", ".join(
f'{pair["source_lang"]}{pair["target_lang"]}'
for pair in missing_packs
)
return JSONResponse(
status_code=409,
content={
"error": f"Install the Argos language pack for {pairs} before translating.",
"code": "argos_pack_missing",
"pairs": missing_packs,
},
)
def _translate_argos():
cache_dir = os.environ.get("OMNIVOICE_CACHE_DIR")
if cache_dir:
argos_cache = os.path.join(cache_dir, "argos-translate")
os.makedirs(argos_cache, exist_ok=True)
os.environ.setdefault("ARGOS_PACKAGES_DIR", argos_cache)
os.environ.setdefault("ARGOS_DATA_DIR", argos_cache)
import argostranslate.package
import argostranslate.translate
from_code = pack_status["source_lang"]
from_code = src_lang
available_packages = argostranslate.package.get_installed_packages()
results = []
for seg in req.segments:
@@ -849,12 +603,19 @@ async def dub_translate(req: TranslateRequest):
results.append({"id": seg.id, "text": seg.text})
continue
to_code = seg.target_lang if seg.target_lang else req.target_lang
to_code = translation_engines.argos_lang_code(to_code)
translated_text = (
seg.text
if from_code == to_code
else argostranslate.translate.translate(seg.text, from_code, to_code)
)
installed_pkg = next(filter(lambda x: x.from_code == from_code and x.to_code == to_code, available_packages), None)
if installed_pkg is None:
argostranslate.package.update_package_index()
all_packages = argostranslate.package.get_available_packages()
package_to_install = next(filter(lambda x: x.from_code == from_code and x.to_code == to_code, all_packages), None)
if package_to_install:
argostranslate.package.install_from_path(package_to_install.download())
available_packages = argostranslate.package.get_installed_packages()
else:
raise Exception(f"No Argos package available for {from_code} -> {to_code}")
translated_text = argostranslate.translate.translate(seg.text, from_code, to_code)
results.append({"id": seg.id, "text": translated_text})
except Exception as e:
results.append({"id": seg.id, "text": seg.text, "error": str(e)})
@@ -928,10 +689,9 @@ async def dub_translate(req: TranslateRequest):
for attempt, src in enumerate([src_arg, src_arg, "auto"]):
try:
out = _build_translator(src, seg_lc).translate(seg.text)
output_error = _translation_output_error(out)
if output_error is None:
if out and out.strip():
return {"id": seg.id, "text": out}
last_err = output_error
last_err = "empty translation"
except Exception as e:
last_err = f"{type(e).__name__}: {e}"
logger.warning(
@@ -954,16 +714,8 @@ async def dub_translate(req: TranslateRequest):
translated, req, src_lang, loop,
)
except Exception as e:
from core.public_errors import public_failure
error = public_failure(
logger,
"Translation request failed",
e,
response="Translation failed; check the backend log for details.",
traceback=True,
)
return JSONResponse(status_code=500, content={"error": error})
import traceback; traceback.print_exc()
return JSONResponse(status_code=500, content={"error": str(e)})
def _stamp_duration_plan(rows, req) -> None:
@@ -1124,7 +876,7 @@ async def _apply_fit_pass(rows, req, slots_by_id, source_by_id, quality, loop, d
their current text and get ``rate_error='fit-budget'``. Only rows with a
slot + text + no prior error participate.
"""
strict = quality in ("autofit", "agent")
strict = (quality == "autofit")
items = []
for row in rows:
seg_id = str(row["id"])
@@ -1187,7 +939,7 @@ async def _maybe_cinematic(translated, req, src_lang, loop, *, already_llm=False
# Fast (and anything unrecognised) returns the plain translation unchanged
# (plus the pre-synthesis duration-plan badges — no LLM needed for those).
if quality not in ("cinematic", "autofit", "agent"):
if quality not in ("cinematic", "autofit"):
await _finalize_duration_plan(translated, req, loop)
return base
@@ -1258,7 +1010,7 @@ async def _maybe_cinematic(translated, req, src_lang, loop, *, already_llm=False
target_lang=req.target_lang,
glossary=req.glossary,
directions=directions,
dialect_hint="\n".join(filter(None, [dialect_hint, translation_style_brief(req)])),
dialect_hint=dialect_hint,
executor=_cpu_pool,
)
refined_by_id = {r["id"]: r for r in refined}
@@ -1305,70 +1057,3 @@ async def _maybe_cinematic(translated, req, src_lang, loop, *, already_llm=False
"quality_used": quality,
**_dialect_flags(req, applied=bool(dialect_hint)),
}
def translation_style_brief(req) -> str:
instructions = (getattr(req, "translation_instructions", None) or "").strip()
return ("User translation style brief (tone and wording only; preserve meaning, timing and output format): "
+ json.dumps(instructions, ensure_ascii=False)) if instructions else ""
@router.post("/dub/agent-fit")
async def dub_agent_fit(req: AgentFitRequest):
"""Rewrite rendered lines from real duration evidence.
Synthesis stays in the normal Dubbing pipeline. The client renders each
candidate, measures it, and may request one more bounded correction.
"""
from services import llm_skills
from services.speech_rate import adjust_for_measured_slot_many
readiness = llm_skills.resolve_skill("slot_fitting")
if not readiness.ready:
raise HTTPException(
status_code=409,
detail={
"error": "llm_skill_unavailable",
"skill": "slot_fitting",
"reason": readiness.reason or "unavailable",
},
)
items = [
(
segment.id,
segment.text,
segment.slot_seconds,
segment.measured_seconds,
req.target_lang,
segment.source_text,
segment.context_before,
segment.context_after,
)
for segment in req.segments
]
budget = _cinematic_budget()
try:
call = adjust_for_measured_slot_many(items, executor=_cpu_pool, translation_instructions=req.translation_instructions)
rows = await asyncio.wait_for(call, timeout=budget) if budget and budget > 0 else await call
except asyncio.TimeoutError:
rows = {
segment.id: {
"text": segment.text,
"changed": False,
"measured_seconds": round(segment.measured_seconds, 3),
"target_seconds": round(segment.slot_seconds, 3),
"measured_ratio": round(
segment.measured_seconds / max(segment.slot_seconds, 0.001), 3
),
"error": "fit-budget",
}
for segment in req.segments
}
return {
"target_lang": req.target_lang,
"segments": [
{"id": segment.id, **rows[str(segment.id)]}
for segment in req.segments
],
}
+70 -421
View File
@@ -15,27 +15,21 @@ Environment variables (`OMNIVOICE_TTS_BACKEND`, `OMNIVOICE_ASR_BACKEND`,
`OMNIVOICE_LLM_BACKEND`) still win over the UI choice so power-users can pin
a backend without Settings silently undoing it.
"""
import asyncio
import logging
import os
import re
import threading
from time import perf_counter
from fastapi import APIRouter, Depends, HTTPException, Request
from huggingface_hub import utils as hf_utils
from huggingface_hub.errors import HFValidationError
from pydantic import BaseModel, Field
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from api.dependencies import require_admin, require_admin_action, require_desktop, is_loopback
from api.dependencies import require_loopback
from core import prefs
from core.engine_licenses import LICENSE_GATED_ENGINES
from services import tts_backend, asr_backend, llm_backend, translation_engines
from services.audio_dsp import list_effect_presets
from api.schemas import EffectPresetsResponse
from api.public_engine_metadata import public_backends, public_unavailability
router = APIRouter()
logger = logging.getLogger("omnivoice.engines_api")
_FAMILIES = {
"tts": (tts_backend, "tts_backend"),
@@ -44,129 +38,37 @@ _FAMILIES = {
}
def _catalogue_active_id(family: str, module) -> str:
"""Return the active id represented by the public engine catalogue."""
active = module.active_backend_id()
if family != "tts" or active != "omnivoice-subprocess":
return active
from core.device_caps import detect_host_caps
try:
return "omnivoice" if detect_host_caps().family == "mps" else active
except Exception:
return active
def _family_payload(family: str, module):
"""Public inventory plus whether an environment pin owns this family."""
active = _catalogue_active_id(family, module)
model = None
if family == "asr":
model = asr_backend._offline_asr_repo(active)
elif family == "llm" and active != "off":
model = llm_backend.get_active_llm_backend().model_name
elif family == "tts":
if active in {"omnivoice", "omnivoice-subprocess"}:
from services.model_manager import resolve_omnivoice_checkpoint
model = resolve_omnivoice_checkpoint()
elif active == "mlx-audio":
from core import prefs
cls = tts_backend.MLXAudioBackend
key = prefs.resolve("mlx_audio_model_id", env="OMNIVOICE_MLX_AUDIO_MODEL", default=cls.DEFAULT_MODEL_KEY)
model = cls.CURATED_MODELS.get(key, key)
else:
instance = getattr(tts_backend, "_active_instance", None)
if instance is not None and getattr(tts_backend, "_active_instance_id", None) == active:
model = instance.model_identity()
backends = public_backends(module.list_backends())
if family == "tts":
from services import settings_store
for backend in backends:
engine_id = backend.get("id")
if engine_id == active == "mlx-audio":
# Constructor resolves model preferences only; never loads weights.
backend["supports_cloning"] = tts_backend.MLXAudioBackend().supports_cloning
if engine_id in LICENSE_GATED_ENGINES:
backend["license_required"] = True
try:
backend["license_accepted"] = settings_store.get_license_accepted(engine_id)
except Exception:
logger.warning(
"Could not read license acceptance for %s",
engine_id,
exc_info=True,
)
backend["license_accepted"] = False
return {
# MPS hides the explicit compatibility row, so legacy configs report
# the visible canonical equivalent as active to picker consumers.
"active": active,
"active_model": model,
"env_override": bool(os.environ.get(f"OMNIVOICE_{family.upper()}_BACKEND")),
"backends": backends,
}
def _is_hf_repo_id(value: str) -> bool:
"""Validate the route's ``owner/repo`` contract in bounded time."""
if not isinstance(value, str) or len(value) > 96 or value.count("/") != 1:
return False
try:
hf_utils.validate_repo_id(value)
except (HFValidationError, TypeError):
return False
return True
def _request_install_capability(payload, request):
allowed = bool(request and request.client and is_loopback(request.client.host))
result = dict(payload)
result["backends"] = [dict(entry) for entry in payload["backends"]]
for entry in result["backends"]:
if entry.get("one_click_install") and not allowed:
entry["one_click_install"] = False
entry["local_install_required"] = True
return result
@router.get("/engines")
def list_all_engines(request: Request):
def list_all_engines():
return {
"tts": _request_install_capability(_family_payload("tts", tts_backend), request),
"asr": _family_payload("asr", asr_backend),
"llm": _family_payload("llm", llm_backend),
"tts": {
"active": tts_backend.active_backend_id(),
"backends": tts_backend.list_backends(),
},
"asr": {
"active": asr_backend.active_backend_id(),
"backends": asr_backend.list_backends(),
},
"llm": {
"active": llm_backend.active_backend_id(),
"backends": llm_backend.list_backends(),
},
}
@router.get("/engines/tts")
def list_tts_backends(request: Request):
return _request_install_capability(_family_payload("tts", tts_backend), request)
@router.get(
"/engines/{engine_id}/disk-usage",
dependencies=[Depends(require_admin_action)],
)
def engine_disk_usage(engine_id: str):
"""Measure owned engine bytes only when a catalogue row is opened."""
try:
tts_backend.get_backend_class(engine_id)
except ValueError:
raise HTTPException(status_code=404, detail="Unknown TTS engine")
from services.engine_disk_usage import disk_usage_for
return disk_usage_for(engine_id)
def list_tts_backends():
return {"active": tts_backend.active_backend_id(), "backends": tts_backend.list_backends()}
@router.get("/engines/asr")
def list_asr_backends():
return _family_payload("asr", asr_backend)
return {"active": asr_backend.active_backend_id(), "backends": asr_backend.list_backends()}
@router.get("/engines/llm")
def list_llm_backends():
return _family_payload("llm", llm_backend)
return {"active": llm_backend.active_backend_id(), "backends": llm_backend.list_backends()}
@router.get("/engines/effects/presets", response_model=EffectPresetsResponse)
@@ -179,90 +81,6 @@ def list_effects_presets():
return {"presets": list_effect_presets()}
@router.get("/engines/diarisation")
def diarisation_status():
"""Describe the selected local diarisation runtime without loading weights."""
from services.diarization_runtime import (
PYANNOTE,
SORTFORMER,
selected_backend,
sortformer_status,
)
selected = selected_backend()
native = selected == SORTFORMER
options = []
from api.routers.setup.models import KNOWN_MODELS, cache_is_complete, is_cached
pyannote_repo = "pyannote/speaker-diarization-3.1"
spec = next(model for model in KNOWN_MODELS if model["repo_id"] == pyannote_repo)
pyannote_installed = is_cached(pyannote_repo) and cache_is_complete(spec)
pyannote_reason = None if pyannote_installed else "Install the pyannote model bundle"
options.append({
"id": PYANNOTE,
"label": "pyannote 3.1",
"model": pyannote_repo,
"installed": pyannote_installed,
"reason": pyannote_reason,
})
native_status = sortformer_status()
native_installed = native_status["installed"]
native_model = native_status["model"]
native_reason = native_status["reason"]
options.append({
"id": SORTFORMER,
"label": "Sortformer v1 (audio.cpp)",
"model": native_model,
"model_installed": native_status["model_installed"],
"runtime_installed": native_status["runtime_installed"],
"installed": native_installed,
"reason": native_reason,
})
if native:
from services.diarization_native import is_running
return {"active": SORTFORMER, "label": "Sortformer v1 (audio.cpp)",
"model": native_model, "installed": native_installed, "loaded": False,
"model_installed": native_status["model_installed"],
"runtime_installed": native_status["runtime_installed"],
"busy": is_running(), "reason": native_reason, "options": options}
from services import model_manager
return {"active": PYANNOTE, "label": "pyannote 3.1", "model": pyannote_repo,
"installed": pyannote_installed,
"loaded": model_manager._diar_pipeline is not None, "reason": pyannote_reason,
"options": options}
class DiarisationSelection(BaseModel):
engine_id: str
@router.post("/engines/diarisation/select", dependencies=[Depends(require_admin)])
def select_diarisation_engine(request: DiarisationSelection):
"""Persist an installed diarisation runtime; environment overrides still win."""
from services.diarization_runtime import SORTFORMER, select_backend, selected_backend
status = diarisation_status()
option = next(
(item for item in status["options"] if item["id"] == request.engine_id),
None,
)
if option is None:
raise HTTPException(404, "Unknown diarisation engine")
if not option["installed"]:
raise HTTPException(409, option.get("reason") or "Install this diarisation engine first")
select_backend(request.engine_id)
if request.engine_id == SORTFORMER:
# Native Sortformer is stateless. Release a previously loaded pyannote
# pipeline so Engine Ready cannot hide stale accelerator memory.
from services import model_manager
model_manager.unload_diarization_pipeline()
return {
"active": selected_backend(),
"env_override": bool(os.environ.get("OMNIVOICE_DIARIZATION_BACKEND")),
}
@router.get("/engines/translation")
def list_translation_engines():
"""Translation engines with per-engine pip-package availability.
@@ -273,82 +91,12 @@ def list_translation_engines():
an engine whose Python dependency isn't importable yet.
"""
return {
"active": prefs.get("translation_backend", "argos"),
"engines": [
{**entry, "availability_reason": public_unavailability(entry.get("availability_reason"))}
for entry in translation_engines.list_engines()
],
"engines": translation_engines.list_engines(),
"sandboxed": translation_engines.is_frozen(),
}
class TranslationSelection(BaseModel):
engine_id: str
class ArgosPackRequest(BaseModel):
source_lang: str | None = None
target_langs: list[str] = Field(min_length=1, max_length=32)
job_id: str | None = None
def _argos_pack_request(request: ArgosPackRequest) -> tuple[str, list[str]]:
source = request.source_lang
if not source and request.job_id:
from api.routers.dub_core import _get_job
job = _get_job(request.job_id)
source = job.get("source_lang") if job else None
if not source:
raise HTTPException(422, "Transcribe the source before installing its language pack")
return source, request.target_langs
@router.post(
"/engines/translation/argos/packs/status",
dependencies=[Depends(require_admin)],
)
def argos_pack_status(request: ArgosPackRequest):
source, targets = _argos_pack_request(request)
try:
return translation_engines.argos_pack_status(source, targets)
except (ImportError, ValueError) as exc:
raise HTTPException(status_code=422, detail=str(exc)) from exc
@router.post(
"/engines/translation/argos/packs/install",
dependencies=[Depends(require_admin)],
)
async def install_argos_packs(request: ArgosPackRequest):
source, targets = _argos_pack_request(request)
try:
return await asyncio.to_thread(
translation_engines.install_argos_packs,
source,
targets,
)
except (ImportError, ValueError) as exc:
raise HTTPException(status_code=422, detail=str(exc)) from exc
@router.post("/engines/translation/select", dependencies=[Depends(require_admin)])
def select_translation_engine(request: TranslationSelection):
entry = translation_engines.get_engine(request.engine_id)
if not entry:
raise HTTPException(404, "Unknown translation engine")
if not translation_engines.is_installed(request.engine_id):
raise HTTPException(409, "Install this translation engine before selecting it")
if not translation_engines.is_ready(request.engine_id):
raise HTTPException(409, "Configure this translation provider before selecting it")
prefs.set_("translation_backend", request.engine_id)
return {"active": request.engine_id}
@router.post(
"/engines/translation/{engine_id}/install",
dependencies=[Depends(require_admin)],
)
@router.post("/engines/translation/{engine_id}/install")
async def install_translation_engine(engine_id: str):
entry = translation_engines.get_engine(engine_id)
if not entry:
@@ -384,10 +132,7 @@ async def install_translation_engine(engine_id: str):
}
@router.delete(
"/engines/translation/{engine_id}",
dependencies=[Depends(require_admin)],
)
@router.delete("/engines/translation/{engine_id}")
async def uninstall_translation_engine(engine_id: str):
entry = translation_engines.get_engine(engine_id)
if not entry:
@@ -405,49 +150,18 @@ async def uninstall_translation_engine(engine_id: str):
pkg = entry.get("pip_package")
if not pkg:
return {"status": "no_op", "engine": engine_id}
# The builtin flag is a promise someone has to remember to make; this
# check does not depend on it (#2019).
blocked = translation_engines.uninstall_blocker(engine_id)
if blocked:
raise HTTPException(status_code=blocked[0], detail=blocked[1])
rc, out = await translation_engines.run_pip(["uninstall", "-y", pkg])
if rc != 0:
raise HTTPException(status_code=500, detail=f"pip uninstall {pkg} failed ({rc}): {out[-1000:]}")
return {"status": "uninstalled", "engine": engine_id, "package": pkg, "log_tail": out[-800:]}
# ── Checksummed native audio.cpp runtime install ───────────────────────────
@router.get(
"/engines/audiocpp/runtime/install/status",
dependencies=[Depends(require_admin)],
)
def audiocpp_runtime_install_status(request: Request):
from services import audiocpp_runtime_install
return {**audiocpp_runtime_install.status(), "install_allowed": bool(request.client and is_loopback(request.client.host))}
@router.post(
"/engines/audiocpp/runtime/install",
dependencies=[Depends(require_admin), Depends(require_desktop)],
)
def install_audiocpp_runtime():
from services import audiocpp_runtime_install
try:
return audiocpp_runtime_install.start_install()
except RuntimeError as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
# ── One-click sidecar-engine install (IndexTTS-2 & friends) ────────────────
#
# Sidecar engines (dedicated venv + source checkout + weights, isolated from
# the parent's transformers>=5.3) used to require four manual terminal steps.
# These routes drive services.sidecar_install: POST starts a resumable
# background job, GET polls its step-by-step status (the Model Catalogue
# background job, GET polls its step-by-step status (the Settings → Engines
# Install button polls this), DELETE removes an app-managed install.
#
# Path namespace: /engines/sidecar/{engine_id}/… — NOT /engines/{engine_id}/…
@@ -457,16 +171,15 @@ def install_audiocpp_runtime():
# POST /engines/sonitranslate/install). Mirrors the
# /engines/translation/{engine_id}/install namespace pattern.
#
# Desktop-only: installing spawns git/uv against mutable source and writes an
# editable environment. An API key does not make that supply-chain path safe to
# trigger remotely. The job runs fine in packaged builds: the venv lives under
# the user data dir, not inside the signed app bundle, and uv resolves via
# OMNIVOICE_BUNDLED_UV/PATH.
# Loopback-gated: installing spawns subprocesses (git/uv) and writes to the
# data directory — only the local desktop frontend may trigger it. The job
# runs fine in packaged builds: the venv lives under the user data dir, not
# inside the signed app bundle, and uv resolves via OMNIVOICE_BUNDLED_UV/PATH.
@router.post(
"/engines/sidecar/{engine_id}/install",
dependencies=[Depends(require_admin), Depends(require_desktop)],
dependencies=[Depends(require_loopback)],
)
def install_sidecar_engine(engine_id: str):
"""Start (or report) the one-click install for a sidecar engine.
@@ -478,11 +191,6 @@ def install_sidecar_engine(engine_id: str):
from services import sidecar_install
try:
return sidecar_install.start_install(engine_id)
except sidecar_install.HostUnsupported as exc:
# The engine has an installer, but not one that can work on this
# machine. 409, not 404: the route is right, the host is the problem,
# and the message (a VoiceStudio-owned sentence) says what to do.
raise HTTPException(status_code=409, detail=str(exc))
except KeyError:
raise HTTPException(
status_code=404,
@@ -497,9 +205,9 @@ def install_sidecar_engine(engine_id: str):
@router.get(
"/engines/sidecar/{engine_id}/install/status",
dependencies=[Depends(require_admin)],
dependencies=[Depends(require_loopback)],
)
def sidecar_install_status(engine_id: str, request: Request = None):
def sidecar_install_status(engine_id: str):
"""Step-by-step status of the sidecar install job (poll while running).
Shape: ``{engine_id, installed, managed, install_dir, job}`` where job is
@@ -508,7 +216,7 @@ def sidecar_install_status(engine_id: str, request: Request = None):
"""
from services import sidecar_install
try:
return {**sidecar_install.get_status(engine_id), "install_allowed": bool(request and request.client and is_loopback(request.client.host))}
return sidecar_install.get_status(engine_id)
except KeyError:
raise HTTPException(
status_code=404,
@@ -518,7 +226,7 @@ def sidecar_install_status(engine_id: str, request: Request = None):
@router.delete(
"/engines/sidecar/{engine_id}/install",
dependencies=[Depends(require_admin)],
dependencies=[Depends(require_loopback)],
)
def uninstall_sidecar_engine(engine_id: str):
"""Remove an app-managed sidecar install (checkout + venv + weights) and
@@ -549,22 +257,29 @@ def uninstall_sidecar_engine(engine_id: str):
# frame. Result includes wall-clock latency so the UI can render
# "1234 ms — pong" inline next to the button.
#
# Admin-gated (T-02-13): only the local desktop frontend or an authenticated
# server-mode administrator may trigger a sidecar spawn through this endpoint.
# Loopback-gated (T-02-13): only the local desktop frontend may trigger
# a sidecar spawn through this endpoint.
# Engine instances cached for the lifetime of the FastAPI process so that
# repeated health checks don't spawn a new SubprocessBackend (each spawn
# allocates a sidecar venv probe + atexit hook). The cache is keyed by
# class to survive registry-sandbox tests that rebind ids transiently.
#
# It now lives in services.tts_backend — the worker executor needs the same
# warm instances and cannot import an API router without inverting the
# layering. This name is the SAME dict object, kept so the existing consumers
# (engine_memory eviction, model_lifecycle inventory/unload) go on working
# unchanged; rebinding it here would fork the cache in two.
_ENGINE_INSTANCES: dict[type, object] = tts_backend._ENGINE_INSTANCES
_ENGINE_INSTANCES: dict[type, object] = {}
_get_engine_instance = tts_backend.get_engine_instance
def _get_engine_instance(cls):
"""Return a cached singleton instance of ``cls``.
SubprocessBackend's ``__init__`` registers an atexit shutdown hook,
so re-instantiating per request would leak handler entries (and on
real engines, additional sidecar processes the first time the lock
is acquired). One instance per process is the right move.
"""
inst = _ENGINE_INSTANCES.get(cls)
if inst is None:
inst = cls()
_ENGINE_INSTANCES[cls] = inst
return inst
def _resolve_engine_class(engine_id: str):
@@ -586,7 +301,7 @@ def _resolve_engine_class(engine_id: str):
@router.get(
"/engines/{engine_id}/health",
dependencies=[Depends(require_admin_action)],
dependencies=[Depends(require_loopback)],
)
def engine_health(engine_id: str):
"""Spawn-and-ping a SubprocessBackend; ``is_available()`` for the rest.
@@ -594,10 +309,10 @@ def engine_health(engine_id: str):
Returns:
{ id, ok, message, latency_ms }
Never raises through to a 500: backend diagnostics stay in the local
log and the response carries a fixed failure message, so the UI can
render a per-row failure without exposing private data. Unknown engine
ids return 404.
Never raises through to a 500: if the backend's check throws, the
exception is captured into the response body as ``ok=False`` /
``message="ExcType: ..."`` so the UI can render a per-row failure
without crashing the panel. Unknown engine ids return 404.
"""
cls = _resolve_engine_class(engine_id)
if cls is None:
@@ -607,9 +322,6 @@ def engine_health(engine_id: str):
)
t0 = perf_counter()
# Stable exception class when the probe itself raised, None when it merely
# returned not-available. Never the exception text — see the log line below.
raised_class: str | None = None
if hasattr(cls, "health_check"):
# SubprocessBackend path — spawn sidecar (if not running) and ping.
# ``health_check`` already swallows its own exceptions per Plan
@@ -620,7 +332,6 @@ def engine_health(engine_id: str):
ok, msg = instance.health_check()
except Exception as exc:
ok, msg = False, f"{type(exc).__name__}: {exc}"
raised_class = type(exc).__name__
else:
# In-process backend — `is_available()` is the classmethod-level
# liveness check. Cheap and side-effect-free for every shipping
@@ -629,50 +340,17 @@ def engine_health(engine_id: str):
ok, msg = cls.is_available()
except Exception as exc:
ok, msg = False, f"{type(exc).__name__}: {exc}"
raised_class = type(exc).__name__
# Engine-owned output can contain much more than shaped HF tokens: local
# paths, arbitrary credentials, source lines, or a nested traceback.
from core.public_errors import public_engine_health
# Mask any HF token the engine accidentally leaked into the message
# so the response body matches the same redaction guarantee as
# ``list_backends()``.
from services.tts_backend import _mask_hf_tokens
latency_ms = (perf_counter() - t0) * 1000.0
if not ok:
# The response tells the user to "check the backend log for details",
# and docs/engines/*.md asks a user diagnosing an unavailable engine to
# copy that engine's log lines. The old line named neither the engine
# nor anything about the probe, so neither instruction could be
# followed (#1866).
#
# `probe=` reports what the PROBE DID, not what went wrong. It cannot
# classify the cause: SubprocessBackend.health_check() swallows its own
# exceptions per Plan 02-01's contract, so a dead sidecar and a package
# that was never installed both arrive here as `returned-unavailable`.
# Separating those needs structured failure metadata from the probes
# themselves, which is a wider change than this one.
#
# Still no diagnostic text and still not the caller-supplied id: the
# engine id comes off the resolved registry class and a raised probe
# contributes only its exception class, the same shape
# core.public_errors.public_failure() logs as `class=`.
# tests/test_response_safety.py pins that boundary and passes
# unchanged.
#
# The id is a class attribute off the registry rather than caller
# input, but this line is a log-injection surface either way, so it is
# flattened to a single token before it goes in.
engine_label = str(getattr(cls, "id", None) or cls.__name__)
engine_label = "".join(
c if (c.isalnum() or c in "-_.") else "-" for c in engine_label
)[:64]
logger.warning(
"Engine health check failed; engine=%s probe=%s, details withheld",
engine_label or "unknown",
f"raised:{raised_class}" if raised_class else "returned-unavailable",
)
return {
"id": engine_id,
"ok": bool(ok),
"message": public_engine_health(bool(ok), msg),
"message": _mask_hf_tokens(msg) if isinstance(msg, str) else str(msg),
"latency_ms": latency_ms,
}
@@ -696,11 +374,11 @@ def engine_health(engine_id: str):
# hanging the Settings panel. The orphaned worker is best-effort daemon.
# * A process-wide lock serialises self-tests so a click-storm can't stack
# concurrent model loads.
# * Only ever on user click (POST) — never on Settings load. Admin-gated.
# * Only ever on user click (POST) — never on Settings load. Loopback-gated.
# Deliberately short + ASCII so the synth stays CPU-cheap and the phrase never
# trips the no-hardcoded-CJK guard.
_SELFTEST_PHRASE = "VoiceStudio engine self test."
_SELFTEST_PHRASE = "OmniVoice engine self test."
_SELFTEST_LOCK = threading.Lock()
@@ -763,7 +441,7 @@ class SelfTestResponse(BaseModel):
@router.post(
"/engines/{engine_id}/selftest",
response_model=SelfTestResponse,
dependencies=[Depends(require_admin)],
dependencies=[Depends(require_loopback)],
)
def engine_selftest(engine_id: str):
"""Run a bounded, real synthesis on an available in-process TTS engine.
@@ -862,11 +540,7 @@ class SelectEngineResponse(BaseModel):
routing_reason: str | None = None
@router.post(
"/engines/select",
response_model=SelectEngineResponse,
dependencies=[Depends(require_admin)],
)
@router.post("/engines/select", response_model=SelectEngineResponse)
def select_engine(req: SelectEngineRequest):
"""Persist a family's engine pick to prefs.json. Refuses unknown backends,
backends whose deps aren't installed, AND backends that cannot run on THIS
@@ -878,15 +552,7 @@ def select_engine(req: SelectEngineRequest):
if not family:
raise HTTPException(400, f"Unknown family: {req.family}. Expected one of tts/asr/llm.")
module, pref_key = family
# MPS intentionally hides the redundant explicit OmniVoice sidecar from
# the picker, but existing scripts and saved preferences may still submit
# that supported compatibility id directly.
rows = (
module.list_backends(include_hidden=True)
if req.family == "tts"
else module.list_backends()
)
available = {b["id"]: b for b in rows}
available = {b["id"]: b for b in module.list_backends()}
if req.backend_id not in available:
raise HTTPException(400, f"Unknown {req.family} backend: {req.backend_id!r}")
entry = available[req.backend_id]
@@ -905,7 +571,7 @@ def select_engine(req: SelectEngineRequest):
# #981: mlx-audio multiplexes 7+ curated models behind one backend id —
# persist the model pick alongside the backend id so the UI can actually
# select which curated model gets loaded (previously it always defaulted
# to Kokoro no matter what the user downloaded in the engine's Weights list in Model Catalogue).
# to Kokoro no matter what the user downloaded in Settings → Models).
if req.family == "tts" and req.backend_id == "mlx-audio" and req.model_id is not None:
known_keys = tts_backend.MLXAudioBackend.CURATED_MODELS
# Accept a curated key OR a raw HF repo id ("owner/name") — the same
@@ -913,30 +579,13 @@ def select_engine(req: SelectEngineRequest):
# Anything else (typo'd key, malformed id) is rejected outright
# rather than silently persisted as a "custom repo" that then fails
# to resolve at load time.
if req.model_id not in known_keys and not _is_hf_repo_id(req.model_id):
if req.model_id not in known_keys and not re.fullmatch(r"[\w.-]+/[\w.-]+", req.model_id):
raise HTTPException(
400,
"Unknown mlx-audio model. Expected a curated model key or a "
"Hugging Face repo ID like 'owner/name'.",
f"Unknown mlx-audio model: {req.model_id!r}. Expected one of "
f"{sorted(known_keys)} or a HF repo id like 'owner/name'.",
)
prefs.set_("mlx_audio_model_id", req.model_id)
if req.family == "asr" and req.model_id is not None:
if req.backend_id not in {"faster-whisper", "faster-whisper-isolated"}:
raise HTTPException(400, "This ASR engine does not accept a CTranslate2 model")
from api.routers.setup.models import KNOWN_MODELS, is_cached
model = next((item for item in KNOWN_MODELS if item["repo_id"] == req.model_id), None)
compatible = req.model_id.startswith("Systran/faster-") or req.model_id == (
"deepdml/faster-whisper-large-v3-turbo-ct2"
)
if model is None or str(model.get("role", "")).lower() != "asr" or not compatible:
raise HTTPException(400, "This model is not compatible with Faster-Whisper")
if not is_cached(req.model_id):
raise HTTPException(409, "Install this ASR model before selecting it")
try:
asr_backend.select_faster_whisper_model(req.model_id)
except ValueError as exc:
raise HTTPException(409, str(exc)) from exc
prefs.set_(pref_key, req.backend_id)
return {
"family": req.family,
+40 -41
View File
@@ -4,28 +4,34 @@ import time
import shutil
import subprocess
import platform
from fastapi import APIRouter, Depends, HTTPException
from fastapi import APIRouter, HTTPException
from api.dependencies import require_native_access
from core.db import db_conn
from core.config import DATA_DIR, OUTPUTS_DIR
from core.config import OUTPUTS_DIR
from core import event_bus
from core.path_authorization import PathAuthorizationError, consume
from core.path_security import UnsafePath, resolve_within, safe_filename
from schemas.requests import ExportRequest, ExportRecordRequest, RevealRequest
router = APIRouter()
def _authorized_destination(token: str) -> str:
"""Consume a native save-dialog capability and validate its destination."""
try:
raw = consume(token, "dub_export")
except PathAuthorizationError as exc:
raise HTTPException(status_code=403, detail=str(exc)) from exc
if not raw or not raw.strip() or not os.path.isabs(os.path.expanduser(raw)):
raise HTTPException(status_code=400, detail="The selected destination is invalid.")
dest = os.path.realpath(os.path.expanduser(raw))
def _safe_destination(raw: str) -> str:
"""Resolve + validate an export destination. Rejects relative/empty paths."""
if not raw or not raw.strip():
raise HTTPException(
status_code=400,
detail="Export needs a destination folder. Pick where the file should go and try again.",
)
expanded = os.path.expanduser(raw)
# Check BEFORE realpath(): realpath absolutizes a relative path against
# the server's cwd, which made this check dead code — a relative
# destination silently exported to a cwd-dependent location instead of
# the documented 400 (regression-tested in tests/test_exports_api.py).
if not os.path.isabs(expanded):
raise HTTPException(
status_code=400,
detail="The destination needs to be a full path (e.g. /Users/you/Movies/OmniVoice) — not relative.",
)
dest = os.path.realpath(expanded)
parent = os.path.dirname(dest)
if not parent or not os.path.isdir(parent):
raise HTTPException(
@@ -37,32 +43,32 @@ def _authorized_destination(token: str) -> str:
def _safe_source(filename: str) -> str:
"""Resolve a source filename against OUTPUTS_DIR / dub outputs, blocking traversal."""
try:
base = safe_filename(filename)
except UnsafePath as exc:
base = os.path.basename(filename or "")
# "." and ".." are their own basename, so they'd slip past the
# base != filename check and only die later on realpath containment —
# reject them up front with the same 400 as any other malformed name.
if not base or base != filename or base in (".", ".."):
raise HTTPException(
status_code=400,
detail="The file to export has an unexpected name. Try re-generating the audio and exporting again.",
) from exc
)
for root in (OUTPUTS_DIR, os.path.join("dub", "outputs")):
try:
candidate = resolve_within(root, base)
except UnsafePath:
continue
if candidate.is_file():
return str(candidate)
candidate = os.path.realpath(os.path.join(root, base))
root_real = os.path.realpath(root)
if candidate.startswith(root_real + os.sep) and os.path.exists(candidate):
return candidate
raise HTTPException(
status_code=404,
detail="That file isn't on disk anymore — it may have been cleaned up. Regenerate and try again.",
)
@router.post("/export", dependencies=[Depends(require_native_access)])
@router.post("/export")
def export_file(req: ExportRequest):
src = _safe_source(req.source_filename)
dest = _authorized_destination(req.authorization)
dest = _safe_destination(req.destination_path)
try:
# Video exports: overlay VoiceStudio logo if visible watermark is enabled
# Video exports: overlay OmniVoice logo if visible watermark is enabled
if src.lower().endswith(".mp4"):
from services.watermark import is_visible_video_enabled, get_ffmpeg_overlay_args
logo_path = os.path.join(os.path.dirname(__file__), "..", "..", "..", "docs", "logo.png")
@@ -120,38 +126,31 @@ def get_export_history():
return [dict(r) for r in rows]
@router.post("/export/reveal", dependencies=[Depends(require_native_access)])
@router.post("/export/reveal")
def reveal_in_folder(req: RevealRequest):
# Desktop clients reveal arbitrary user-selected export destinations in
# the native Tauri process. This HTTP fallback is deliberately limited to
# server-owned data so a remote/browser caller cannot make the host open
# an attacker-chosen path.
# Tauri/native dialog-provided path; subprocess uses list args (no shell interpolation).
if not req.path or not req.path.strip():
raise HTTPException(
status_code=400,
detail="No path was provided — nothing to reveal.",
)
try:
target_path = resolve_within(DATA_DIR, req.path)
except UnsafePath as exc:
raise HTTPException(status_code=403, detail="That path cannot be opened remotely.") from exc
if not target_path.exists():
target = os.path.realpath(os.path.expanduser(req.path))
if not os.path.exists(target):
raise HTTPException(
status_code=404,
detail="That file or folder is no longer on disk. It may have been moved or deleted.",
)
target = str(target_path)
folder = target if target_path.is_dir() else str(target_path.parent)
folder = target if os.path.isdir(target) else os.path.dirname(target)
system = platform.system()
try:
if system == "Darwin":
if target_path.is_file():
if os.path.isfile(target):
subprocess.Popen(["open", "-R", target])
else:
subprocess.Popen(["open", folder])
elif system == "Windows":
if target_path.is_file():
if os.path.isfile(target):
subprocess.Popen(["explorer", "/select,", target.replace("/", "\\")])
else:
subprocess.Popen(["explorer", folder.replace("/", "\\")])
+96 -245
View File
@@ -1,25 +1,18 @@
import asyncio
import contextlib
import json
import logging
import os
import re
import shutil
import tempfile
import time
import json
import uuid
import time
import asyncio
import logging
from typing import Optional, List
from pathlib import Path
from typing import List, Optional
from fastapi import APIRouter, File, Form, UploadFile, HTTPException, Query
from fastapi.responses import FileResponse
from fastapi.responses import FileResponse, RedirectResponse
from pydantic import BaseModel
from core.db import db_conn
from core.config import VOICES_DIR, OUTPUTS_DIR
from core import event_bus
from core.audio_validation import resolve_regular_file
from core.file_cleanup import FileCleanupError, unlink_if_present
from services.ffmpeg_utils import spawn_subprocess
logger = logging.getLogger("omnivoice.gallery")
@@ -146,14 +139,11 @@ def delete_voice(voice_id: str):
raise HTTPException(status_code=404, detail="Voice not found")
audio_path = row["audio_path"]
if audio_path:
if audio_path and os.path.exists(audio_path):
try:
unlink_if_present(audio_path)
except FileCleanupError as exc:
raise HTTPException(
status_code=500,
detail="Could not delete the voice audio file. Close any app using it and retry.",
) from exc
os.remove(audio_path)
except Exception:
pass
conn.execute("DELETE FROM voice_gallery WHERE id = ?", (voice_id,))
return {"success": True}
@@ -366,223 +356,46 @@ async def upload_voice_clip(
}
def _stage_profile_audio(source: Path, directory: Path) -> Path:
"""Copy an imported clip to a hidden temp file inside ``directory``.
The temp lives in the destination directory itself so a later
``os.replace`` to the final name is an atomic same-filesystem rename
cheap enough to run while holding a DB write lock, unlike the copy.
Callers own cleanup of the returned path if they never publish it.
"""
directory.mkdir(parents=True, exist_ok=True)
fd, tmp_name = tempfile.mkstemp(
dir=str(directory), prefix=".gallery-import-", suffix=".part",
)
os.close(fd)
try:
shutil.copy2(source, tmp_name)
except BaseException:
with contextlib.suppress(OSError):
os.unlink(tmp_name)
raise
return Path(tmp_name)
def _copy_profile_audio(source: Path, destination: Path) -> None:
"""Copy an imported clip without exposing a partial profile audio file."""
staged = _stage_profile_audio(source, destination.parent)
try:
os.replace(staged, destination)
except BaseException:
with contextlib.suppress(OSError):
os.unlink(staged)
raise
def _gallery_profile_audio_filename(profile_id: str, source: Path) -> str:
"""Return the canonical, portable filename for a My Imports profile."""
safe_id = (
profile_id if re.fullmatch(r"[A-Za-z0-9_-]{1,64}", profile_id or "")
else uuid.uuid5(uuid.NAMESPACE_URL, str(profile_id)).hex[:16]
)
suffix = source.suffix.lower()
if not re.fullmatch(r"\.[a-z0-9]{1,8}", suffix):
suffix = ".wav"
return f"{safe_id}_gallery{suffix}"
def _is_materialized_gallery_profile(row, voice: dict, audio_filename: str) -> bool:
"""Recognize only rows created by this materializer, not identity collisions."""
return bool(
row["personality"] == f"gallery:{voice['id']}"
and row["ref_audio_path"] == audio_filename
and row["ref_text"] == ""
and row["instruct"] == ""
and row["language"] == "Auto"
and row["seed"] is None
and row["kind"] == "clone"
and not row["vd_states"]
and row["description"] == (voice.get("description") or "")
and not row["is_locked"]
and not row["verified_own_voice"]
and not row["locked_audio_path"]
)
def _existing_gallery_profile(conn, voice: dict, source: Path):
personality = f"gallery:{voice['id']}"
rows = conn.execute(
"SELECT * FROM voice_profiles WHERE personality=? ORDER BY created_at, id",
(personality,),
).fetchall()
for row in rows:
expected = _gallery_profile_audio_filename(row["id"], source)
if _is_materialized_gallery_profile(row, voice, expected):
return row
return None
def _gallery_profile_audio_is_current(row, source: Path) -> bool:
"""Detect missing/replaced copies without re-hashing unchanged imports."""
destination = resolve_regular_file(VOICES_DIR, row["ref_audio_path"])
if destination is None:
return False
try:
source_stat = source.stat()
destination_stat = destination.stat()
# copy2 preserves mtime; size + nanosecond mtime catches ordinary edits
# and partial writes while keeping repeated Use clicks inexpensive.
return (
source_stat.st_size == destination_stat.st_size
and source_stat.st_mtime_ns == destination_stat.st_mtime_ns
)
except OSError:
return False
def _materialize_gallery_profile(
voice_id: str, requested_name: Optional[str] = None,
) -> dict:
"""Idempotently materialize/heal one My Imports clip as a clone profile."""
personality = f"gallery:{voice_id}"
copied_path: Optional[Path] = None
created = False
staged_path: Optional[Path] = None
staged_source: Optional[Path] = None
try:
# Stage the (potentially large) audio copy BEFORE taking SQLite's
# write lock: copying inside BEGIN IMMEDIATE would stall every other
# backend writer for the whole copy. The staged temp lives in
# VOICES_DIR itself, so publishing it inside the transaction is an
# atomic same-filesystem os.replace. This pre-read is advisory only —
# the locked transaction below re-reads and re-decides everything.
copy_needed = False
with db_conn() as conn:
pre_row = conn.execute(
"SELECT * FROM voice_gallery WHERE id = ?", (voice_id,),
).fetchone()
if pre_row is not None:
pre_source = Path(pre_row["audio_path"])
if pre_source.is_file():
pre_existing = _existing_gallery_profile(conn, dict(pre_row), pre_source)
copy_needed = pre_existing is None or not _gallery_profile_audio_is_current(
pre_existing, pre_source,
)
if copy_needed:
staged_path = _stage_profile_audio(pre_source, Path(VOICES_DIR))
staged_source = pre_source
with db_conn() as conn:
# The identity is not globally UNIQUE because personality is shared
# with other import mechanisms. Serialize this check+insert in
# SQLite so simultaneous Use clicks cannot both create a row.
conn.execute("BEGIN IMMEDIATE")
row = conn.execute(
"SELECT * FROM voice_gallery WHERE id = ?", (voice_id,),
).fetchone()
if row is None:
raise HTTPException(status_code=404, detail="Voice not found")
voice = dict(row)
source = Path(voice["audio_path"])
if not source.is_file():
raise HTTPException(status_code=404, detail="Audio file not found on disk")
def _install_audio(destination: Path) -> None:
"""Publish the staged copy under the lock via atomic rename."""
nonlocal staged_path
if staged_path is not None and staged_source == source:
os.replace(staged_path, destination)
staged_path = None
else:
# Rare race: the gallery row changed between the advisory
# pre-read and taking the lock, so any staged bytes may be
# from the wrong source. Fall back to the blocking copy
# rather than publish stale audio.
_copy_profile_audio(source, destination)
existing = _existing_gallery_profile(conn, voice, source)
if existing is not None:
ref_filename = _gallery_profile_audio_filename(existing["id"], source)
if not _gallery_profile_audio_is_current(existing, source):
ref_path = Path(VOICES_DIR) / ref_filename
_install_audio(ref_path)
copied_path = ref_path
conn.execute(
"UPDATE voice_profiles SET ref_audio_path=?, ref_text='', instruct='', "
"language='Auto', seed=NULL, description=?, kind='clone', vd_states=NULL, "
"personality=? WHERE id=?",
(
ref_filename, voice["description"] or "", personality,
existing["id"],
),
)
result = {"profile_id": existing["id"], "name": existing["name"]}
else:
profile_id = str(uuid.uuid4())[:8]
profile_name = (requested_name or voice["name"]).strip() or voice["name"]
ref_filename = _gallery_profile_audio_filename(profile_id, source)
copied_path = Path(VOICES_DIR) / ref_filename
_install_audio(copied_path)
conn.execute(
"""INSERT INTO voice_profiles
(id, name, ref_audio_path, ref_text, instruct, language, seed,
personality, is_locked, locked_audio_path, description, kind,
vd_states, created_at)
VALUES (?, ?, ?, '', '', 'Auto', NULL, ?, 0, '', ?, 'clone', NULL, ?)""",
(
profile_id, profile_name, ref_filename, personality,
voice["description"] or "", time.time(),
),
)
created = True
result = {"profile_id": profile_id, "name": profile_name}
except BaseException:
if copied_path is not None:
with contextlib.suppress(OSError):
copied_path.unlink()
raise
finally:
# Staged but never published (failure, or a concurrent request healed
# the profile first) — never leave .part droppings in VOICES_DIR.
if staged_path is not None:
with contextlib.suppress(OSError):
os.unlink(staged_path)
event_bus.emit(
"profiles", {"action": "created" if created else "updated", "id": result["profile_id"]},
)
return result
@router.post("/gallery/voices/{voice_id}/save-as-profile")
async def save_voice_as_profile(
voice_id: str,
profile_name: str = Query(..., description="Name for the voice profile"),
):
"""Save a gallery voice as a voice profile for cloning."""
result = await asyncio.to_thread(_materialize_gallery_profile, voice_id, profile_name)
return {"profile_id": result["profile_id"], "name": result["name"]}
with db_conn() as conn:
row = conn.execute(
"SELECT * FROM voice_gallery WHERE id = ?", (voice_id,)
).fetchone()
if not row:
raise HTTPException(status_code=404, detail="Voice not found")
profile_id = str(uuid.uuid4())[:8]
import shutil
ext = os.path.splitext(row["audio_path"])[1]
new_audio_path = os.path.join(VOICES_DIR, f"{profile_id}{ext}")
shutil.copy(row["audio_path"], new_audio_path)
conn.execute(
"""
INSERT INTO voice_profiles (id, name, ref_audio_path, ref_text, instruct, language, seed, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
""",
(
profile_id,
profile_name,
f"{profile_id}{ext}",
row["description"] or "",
row["character"] or "",
"Auto",
None,
time.time(),
),
)
event_bus.emit("profiles", {"action": "created", "id": profile_id})
return {"profile_id": profile_id, "name": profile_name}
@router.get("/gallery/voices/{voice_id}/preview")
@@ -598,10 +411,22 @@ def preview_voice(voice_id: str):
audio_path = row["audio_path"]
if os.path.isabs(audio_path) and os.path.exists(audio_path):
# Serve the file from this API route so deployments mounted below a
# path prefix do not lose that prefix while following a redirect.
return FileResponse(audio_path)
# Debug logging
is_absolute = os.path.isabs(audio_path)
path_exists = os.path.exists(audio_path) if audio_path else False
# If absolute path, serve directly or redirect
if is_absolute and path_exists:
# Get just the relative path from outputs dir
outputs_path = str(OUTPUTS_DIR)
if audio_path.startswith(outputs_path):
# Remove outputs_dir prefix to get relative path within outputs
rel_path = os.path.relpath(audio_path, outputs_path)
# The audio_path is like: /Users/user4/.../outputs/voice_gallery/file.wav
# rel_path becomes: voice_gallery/file.wav
# We want to serve from /audio/ so: /audio/voice_gallery/file.wav
return RedirectResponse(f"/audio/{rel_path}")
return FileResponse(audio_path, media_type="audio/wav")
raise HTTPException(
status_code=404,
@@ -653,26 +478,52 @@ def batch_delete_voices(body: dict):
return {"deleted": 0}
deleted = 0
failed = 0
with db_conn() as conn:
for vid in ids:
row = conn.execute("SELECT audio_path FROM voice_gallery WHERE id = ?", (vid,)).fetchone()
if row:
audio_path = row["audio_path"]
if audio_path:
if audio_path and os.path.exists(audio_path):
try:
unlink_if_present(audio_path)
except FileCleanupError:
logger.warning("Voice audio cleanup failed for a gallery item")
failed += 1
continue
os.remove(audio_path)
except Exception:
pass
conn.execute("DELETE FROM voice_gallery WHERE id = ?", (vid,))
deleted += 1
return {"deleted": deleted, "failed": failed}
return {"deleted": deleted}
@router.post("/gallery/voices/{voice_id}/to-profile")
def voice_to_profile(voice_id: str):
"""Create a voice profile from a gallery clip."""
result = _materialize_gallery_profile(voice_id)
return {"success": True, "profile_id": result["profile_id"], "name": result["name"]}
with db_conn() as conn:
row = conn.execute("SELECT * FROM voice_gallery WHERE id = ?", (voice_id,)).fetchone()
if not row:
raise HTTPException(status_code=404, detail="Voice not found")
voice = dict(row)
audio_path = voice["audio_path"]
if not os.path.exists(audio_path):
raise HTTPException(status_code=404, detail="Audio file not found on disk")
import shutil
import uuid
profile_id = str(uuid.uuid4())[:8]
# Copy audio to voices dir
dest_filename = f"{profile_id}_gallery.wav"
dest_path = os.path.join(VOICES_DIR, dest_filename)
shutil.copy2(audio_path, dest_path)
import time
now = time.time()
conn.execute(
"""INSERT INTO voice_profiles
(id, name, ref_audio_path, ref_text, instruct, seed, is_locked, locked_audio_path, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)""",
(profile_id, voice["name"], dest_filename, "", None, None, 0, None, now, now),
)
event_bus.emit("profiles", {"action": "created", "id": profile_id})
return {"success": True, "profile_id": profile_id, "name": voice["name"]}
File diff suppressed because it is too large Load Diff
+19 -58
View File
@@ -39,9 +39,6 @@ from core.config import OUTPUTS_DIR, VOICES_DIR
from core.db import db_conn
from core import event_bus
from core.version import APP_VERSION
from core.http_headers import content_disposition
from core.logging_utils import log_safe
from core.path_security import UnsafePath, resolve_within, safe_filename
logger = logging.getLogger("omnivoice.marketplace")
@@ -58,26 +55,6 @@ BUNDLE_VERSION = 1
MAX_BUNDLE_BYTES = 100 * 1024 * 1024
def _contained_path(root, value, *, detail="Invalid file path") -> Path:
try:
return resolve_within(root, value)
except UnsafePath as exc:
raise HTTPException(status_code=400, detail=detail) from exc
def _voice_asset(value) -> Path | None:
"""Resolve a DB-stored voice asset without trusting the database value."""
if not value:
return None
try:
resolved = resolve_within(VOICES_DIR, value)
except UnsafePath as exc:
raise HTTPException(status_code=400, detail="Voice profile contains an invalid asset path") from exc
if not resolved.is_file():
raise HTTPException(status_code=400, detail="Voice profile reference audio is missing")
return resolved
# ── Export ──────────────────────────────────────────────────────────────────
@@ -131,18 +108,18 @@ def export_profile(profile_id: str):
# Reference audio
ref_path = profile.get("ref_audio_path")
if ref_path:
full_ref = _voice_asset(ref_path)
if full_ref and full_ref.is_file():
full_ref = os.path.join(VOICES_DIR, ref_path)
if os.path.isfile(full_ref):
ext = os.path.splitext(ref_path)[1] or ".wav"
zf.write(str(full_ref), f"ref_audio{ext}")
zf.write(full_ref, f"ref_audio{ext}")
# Locked audio (if profile is locked)
locked_path = profile.get("locked_audio_path")
if locked_path:
full_locked = _voice_asset(locked_path)
if full_locked and full_locked.is_file():
full_locked = os.path.join(VOICES_DIR, locked_path)
if os.path.isfile(full_locked):
ext = os.path.splitext(locked_path)[1] or ".wav"
zf.write(str(full_locked), f"locked_audio{ext}")
zf.write(full_locked, f"locked_audio{ext}")
buf.seek(0)
safe_name = "".join(
@@ -154,7 +131,7 @@ def export_profile(profile_id: str):
buf,
media_type="application/zip",
headers={
"Content-Disposition": content_disposition(filename),
"Content-Disposition": f'attachment; filename="{filename}"',
"Content-Length": str(buf.getbuffer().nbytes),
},
)
@@ -277,7 +254,7 @@ def publish_to_marketplace(
"""Publish a voice profile to the local marketplace directory.
This saves a .omnivoice bundle to the marketplace folder so other
VoiceStudio instances on the same machine (or shared network drive)
OmniVoice instances on the same machine (or shared network drive)
can discover and import it.
"""
with db_conn() as conn:
@@ -292,11 +269,7 @@ def publish_to_marketplace(
safe_name = "".join(
c if c.isalnum() or c in "-_ " else "" for c in profile.get("name", "voice")
).strip().replace(" ", "_")[:40]
bundle_path = _contained_path(
MARKETPLACE_DIR,
f"{safe_name}_{profile_id}.omnivoice",
detail="Invalid profile id",
)
bundle_path = MARKETPLACE_DIR / f"{safe_name}_{profile_id}.omnivoice"
# Build the bundle
with zipfile.ZipFile(str(bundle_path), "w", zipfile.ZIP_DEFLATED) as zf:
@@ -309,19 +282,19 @@ def publish_to_marketplace(
ref_path = profile.get("ref_audio_path")
if ref_path:
full_ref = _voice_asset(ref_path)
if full_ref and full_ref.is_file():
full_ref = os.path.join(VOICES_DIR, ref_path)
if os.path.isfile(full_ref):
ext = os.path.splitext(ref_path)[1] or ".wav"
zf.write(str(full_ref), f"ref_audio{ext}")
zf.write(full_ref, f"ref_audio{ext}")
locked_path = profile.get("locked_audio_path")
if locked_path:
full_locked = _voice_asset(locked_path)
if full_locked and full_locked.is_file():
full_locked = os.path.join(VOICES_DIR, locked_path)
if os.path.isfile(full_locked):
ext = os.path.splitext(locked_path)[1] or ".wav"
zf.write(str(full_locked), f"locked_audio{ext}")
zf.write(full_locked, f"locked_audio{ext}")
logger.info("Voice published to marketplace")
logger.info("Published voice %r to marketplace: %s", profile.get("name"), bundle_path)
return {
"success": True,
"profile_id": profile_id,
@@ -371,7 +344,7 @@ def browse_marketplace(
),
})
except Exception as e:
logger.warning("Skipping invalid bundle %s: %s", log_safe(path.name), log_safe(e))
logger.warning("Skipping invalid bundle %s: %s", path.name, e)
return {"bundles": bundles, "total": len(bundles), "directory": str(MARKETPLACE_DIR)}
@@ -379,13 +352,7 @@ def browse_marketplace(
@router.post("/install/{filename}")
async def install_from_marketplace(filename: str):
"""Import a voice profile from a bundle in the local marketplace directory."""
try:
filename = safe_filename(filename)
except UnsafePath as exc:
raise HTTPException(status_code=400, detail="Invalid bundle filename") from exc
if not filename.endswith(".omnivoice"):
raise HTTPException(status_code=400, detail="Invalid bundle filename")
bundle_path = _contained_path(MARKETPLACE_DIR, filename, detail="Invalid bundle filename")
bundle_path = MARKETPLACE_DIR / filename
if not bundle_path.is_file():
raise HTTPException(status_code=404, detail=f"Bundle not found: {filename}")
@@ -458,13 +425,7 @@ async def install_from_marketplace(filename: str):
@router.delete("/{filename}")
def remove_from_marketplace(filename: str):
"""Remove a bundle from the local marketplace directory."""
try:
filename = safe_filename(filename)
except UnsafePath as exc:
raise HTTPException(status_code=400, detail="Invalid bundle filename") from exc
if not filename.endswith(".omnivoice"):
raise HTTPException(status_code=400, detail="Invalid bundle filename")
bundle_path = _contained_path(MARKETPLACE_DIR, filename, detail="Invalid bundle filename")
bundle_path = MARKETPLACE_DIR / filename
if not bundle_path.is_file():
raise HTTPException(status_code=404, detail=f"Bundle not found: {filename}")
try:
+2 -2
View File
@@ -9,13 +9,13 @@ from __future__ import annotations
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel, Field
from api.dependencies import require_admin
from api.dependencies import require_loopback
from services import mcp_bindings
router = APIRouter(
prefix="/api/mcp",
tags=["mcp"],
dependencies=[Depends(require_admin)],
dependencies=[Depends(require_loopback)],
)
+4 -9
View File
@@ -12,14 +12,14 @@ import logging
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from api.dependencies import require_admin
from api.dependencies import require_loopback
logger = logging.getLogger("omnivoice.api")
router = APIRouter(dependencies=[Depends(require_admin)])
router = APIRouter(dependencies=[Depends(require_loopback)])
class CustomPathRequest(BaseModel):
authorization: str
path: str
def _svc():
@@ -61,13 +61,8 @@ def media_tools_ytdlp_restore():
@router.post("/media-tools/{tool}/custom-path")
def media_tools_custom_path(tool: str, body: CustomPathRequest):
from core.path_authorization import PathAuthorizationError, consume
try:
path = consume(body.authorization, tool)
return _svc().set_custom_path(tool, path)
except PathAuthorizationError as e:
raise HTTPException(status_code=403, detail=str(e)) from e
return _svc().set_custom_path(tool, body.path)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
+48 -82
View File
@@ -2,17 +2,17 @@
OpenAI-compatible TTS & STT API Phase 3.2 (ROADMAP.md P0).
Drop-in replacement for OpenAI's audio endpoints so that any tool speaking the
OpenAI protocol (Claude, Cursor, LangChain, litellm, etc.) can use VoiceStudio
OpenAI protocol (Claude, Cursor, LangChain, litellm, etc.) can use OmniVoice
as a local backend with zero code changes.
Endpoints
POST /v1/audio/speech TTS (text wav/mp3/opus/flac)
POST /v1/audio/transcriptions STT (audio file text/json)
GET /v1/audio/voices list available voices (VoiceStudio extension)
GET /v1/audio/voices list available voices (OmniVoice extension)
The router delegates to the active TTS/ASR backends via the same adapter
protocol used by the rest of VoiceStudio, so engine selection, GPU offloading,
protocol used by the rest of OmniVoice, so engine selection, GPU offloading,
model loading, and invisible provenance watermarking (services.watermark,
#1169) all work identically.
@@ -32,7 +32,6 @@ from fastapi.responses import StreamingResponse
from pydantic import BaseModel, Field
from services.model_manager import _gpu_pool, run_on_gpu_pool_guarded
from core.http_headers import content_disposition
logger = logging.getLogger("omnivoice.openai_compat")
@@ -48,7 +47,7 @@ class SpeechRequest(BaseModel):
model: str = Field(
default="omnivoice",
description=(
"TTS model to use. Maps to VoiceStudio engine IDs: "
"TTS model to use. Maps to OmniVoice engine IDs: "
"'omnivoice', 'voxcpm2', 'cosyvoice', 'mlx-audio', 'kittentts', 'moss-tts-nano'. "
"Also accepts 'tts-1' and 'tts-1-hd' as aliases for the active engine."
),
@@ -61,7 +60,7 @@ class SpeechRequest(BaseModel):
voice: str = Field(
default="default",
description=(
"Voice to use. For VoiceStudio: pass a voice profile ID, 'default', "
"Voice to use. For OmniVoice: pass a voice profile ID, 'default', "
"or a KittenTTS preset name. OpenAI voice names (alloy, echo, fable, "
"onyx, nova, shimmer) are accepted but mapped to defaults."
),
@@ -76,7 +75,7 @@ class SpeechRequest(BaseModel):
le=4.0,
description="Speed of the generated audio (0.25 to 4.0).",
)
# VoiceStudio extensions (not part of OpenAI spec, but accepted if sent)
# OmniVoice extensions (not part of OpenAI spec, but accepted if sent)
language: Optional[str] = Field(default=None, description="Language code (ISO 639-1)")
description: Optional[str] = Field(
default=None,
@@ -87,19 +86,19 @@ class SpeechRequest(BaseModel):
duration: Optional[float] = Field(
default=None,
gt=0,
description="VoiceStudio extension: target output duration in seconds.",
description="OmniVoice extension: target output duration in seconds.",
)
seed: Optional[int] = Field(
default=None,
description="VoiceStudio extension: deterministic sampling seed.",
description="OmniVoice extension: deterministic sampling seed.",
)
denoise: bool = Field(
default=True,
description="VoiceStudio extension: prepend denoise control when supported.",
description="OmniVoice extension: prepend denoise control when supported.",
)
preprocess_prompt: bool = Field(
default=True,
description="VoiceStudio extension: trim/preprocess reference prompt when supported.",
description="OmniVoice extension: trim/preprocess reference prompt when supported.",
)
chunk_duration: Optional[float] = Field(
default=None,
@@ -120,13 +119,13 @@ class SpeechRequest(BaseModel):
default=None,
ge=1,
le=128,
description="VoiceStudio extension: iterative unmasking steps (app default 16; 32 = the model's documented quality preset).",
description="OmniVoice extension: iterative unmasking steps (app default 16; 32 = the model's documented quality preset).",
)
guidance_scale: Optional[float] = Field(
default=None,
gt=0,
le=20,
description="VoiceStudio extension: classifier-free guidance scale (app default 2.0).",
description="OmniVoice extension: classifier-free guidance scale (app default 2.0).",
)
@@ -148,7 +147,7 @@ class VerboseTranscriptionResponse(BaseModel):
# ── OpenAI voice name mapping ──────────────────────────────────────────────
# OpenAI's 6 named voices aren't real voices in VoiceStudio. Map them to
# OpenAI's 6 named voices aren't real voices in OmniVoice. Map them to
# sensible defaults so callers that hardcode "alloy" don't get a 400.
_OPENAI_VOICE_ALIASES = {
"alloy", "echo", "fable", "onyx", "nova", "shimmer",
@@ -159,10 +158,8 @@ _OPENAI_VOICE_ALIASES = {
def _resolve_engine(model_id: str):
"""Map an OpenAI model name to a VoiceStudio backend."""
from services.tts_backend import (
get_backend_class, get_active_tts_backend, get_engine_instance_for,
)
"""Map an OpenAI model name to an OmniVoice backend."""
from services.tts_backend import get_backend_class, get_active_tts_backend
# Accept OpenAI model names as pass-through to the active engine.
if model_id in ("tts-1", "tts-1-hd"):
@@ -179,18 +176,8 @@ def _resolve_engine(model_id: str):
)
from services.tts_backend import OmniVoiceBackend
if cls is OmniVoiceBackend:
# OmniVoice only ever runs as the shared active engine — the
# explicit-omnivoice request is the active-engine request.
return get_active_tts_backend()
# Cached singleton, not a fresh cls(): SubprocessBackend engines would
# spawn a sidecar process and reload their model on EVERY request, and
# register a new atexit hook each time (get_engine_instance's contract).
# No router-local cache on top of it: the shared cache is keyed by
# CLASS precisely so id rebinds/evictions can't serve a stale instance,
# and cross-engine memory discipline is create_speech's
# evict_other_tts_engines call (the same seam /generate uses) — not a
# bespoke unload here.
return get_engine_instance_for(model_id)
return cls()
except ValueError:
raise HTTPException(
status_code=400,
@@ -302,7 +289,7 @@ def _run_tts(backend, text: str, kw: dict):
sr = backend.sample_rate
# Engines that already emit mastered, studio-grade audio (e.g. VoxCPM2's
# native 48 kHz) opt out of apply_mastering via `applies_own_mastering`.
# That chain's highpass + Compressor is tuned for VoiceStudio's 24 kHz clone
# That chain's highpass + Compressor is tuned for OmniVoice's 24 kHz clone
# output; applied to a studio engine it adds an audible level pump that
# degrades the very output we want clean. Loudness normalisation still
# runs — it's a benign peak scale, not dynamics.
@@ -325,9 +312,10 @@ async def create_speech(req: SpeechRequest):
# Routing gate (#21 — no silent CPU fallback), identical to REST /generate.
from core.device_caps import detect_host_caps
from services.engine_routing import routing_notice, runtime_compute_profile_async
_routing = await runtime_compute_profile_async(
backend, detect_host_caps()
from services.engine_routing import resolve_routing, routing_notice
_routing = resolve_routing(
getattr(backend, "gpu_compat", ("cpu",)), detect_host_caps(),
getattr(backend, "min_vram_gb", 0.0),
)
if _routing["routing_status"] == "unavailable":
raise HTTPException(status_code=400, detail=_routing["routing_reason"])
@@ -396,18 +384,6 @@ async def create_speech(req: SpeechRequest):
from services.text_normalization import normalize_for_tts
text = normalize_for_tts(req.input, req.language)
# VRAM eviction runs in get_model()'s warm-return path now, covering every
# native TTS generate (this route, WS TTS, dub, batch, audiobook).
# Single-active-engine memory discipline (MM2-01), the same call /generate
# makes before its load: hand back every OTHER resident TTS engine's model
# before this one warms up, so switching `model` ids across requests —
# explicit id → explicit id, or explicit id → the tts-1/omnivoice aliases —
# can't stack multi-GB engines/sidecars. No-op when nothing else is
# resident; opt out with OMNIVOICE_SINGLE_ENGINE_RESIDENT=0.
from services.engine_memory import evict_other_tts_engines
await evict_other_tts_engines(backend.id)
# ── #1033/#1037/#1014: warm the engine under the LOAD budget before the
# generate clock starts. The T4 verification (#1014) measured a fresh
# install's first /v1/audio/speech burning its whole 300s generate budget
@@ -435,7 +411,7 @@ async def create_speech(req: SpeechRequest):
detail=(
f"TTS engine '{backend.id}' did not finish loading within its "
f"model-load budget — on a first run this usually means the weight "
f"download is slow or stalled (check the engine's Weights list in Model Catalogue for "
f"download is slow or stalled (check Settings → Models for "
f"progress), not that generation failed. Retry once the model "
f"shows as installed."
),
@@ -474,7 +450,7 @@ async def create_speech(req: SpeechRequest):
from services.model_manager import generate_timeout_s
wav, sr = await run_on_gpu_pool_guarded(
lambda: _run_tts(backend, text, kw), what="OpenAI TTS generate",
timeout=generate_timeout_s(text, engine=backend))
timeout=generate_timeout_s(text))
except Exception as e:
# #1172/#1173: typed failures get their real status + actionable
# message (400 bad input / 503 broken engine binary) instead of a
@@ -490,7 +466,7 @@ async def create_speech(req: SpeechRequest):
_headers = {
"Content-Length": str(len(audio_bytes)),
"Content-Disposition": content_disposition(f"speech.{ext}", disposition="inline"),
"Content-Disposition": f'inline; filename="speech.{ext}"',
}
if _routing_notice:
from services.engine_routing import header_safe_reason
@@ -515,7 +491,7 @@ async def create_transcription(
default="whisper-1",
description=(
"ASR model. Accepts 'whisper-1' (maps to active engine), or an "
"VoiceStudio engine ID: whisperx, faster-whisper, mlx-whisper, pytorch-whisper."
"OmniVoice engine ID: whisperx, faster-whisper, mlx-whisper, pytorch-whisper."
),
),
language: Optional[str] = Form(
@@ -537,16 +513,15 @@ async def create_transcription(
):
"""Transcribe audio to text. Compatible with OpenAI's POST /v1/audio/transcriptions."""
from services.asr_backend import (
ASRModelMissingError,
asr_model_missing_detail,
asr_model_missing_error,
load_active_asr_backend,
get_active_asr_backend,
)
# TTS-only install: no ASR model on disk → actionable 409, BEFORE any
# backend load could silently auto-download multi-GB whisper weights.
# Same typed detail shape as /transcribe (capture.py): the machine fields
# (`error`, `missing_repo_id`, `recommended`) let VoiceStudio-aware clients
# (`error`, `missing_repo_id`, `recommended`) let OmniVoice-aware clients
# render the one-click download CTA, while `message` keeps a human-readable
# line for generic OpenAI-compat clients.
missing = await asyncio.to_thread(asr_model_missing_error)
@@ -567,25 +542,18 @@ async def create_transcription(
raise HTTPException(status_code=400, detail=f"Could not read audio file: {e}")
try:
backend = get_active_asr_backend()
# Run transcription in the thread pool to avoid blocking the event loop,
# bounded so a stuck/starved ASR returns a 504 with guidance instead of
# hanging the request forever (see run_transcribe_guarded).
from services.asr_backend import run_transcribe_guarded
word_ts = response_format == "verbose_json"
# `load_active_asr_backend`, not `get_active_asr_backend`: the latter is
# a pure selector, so a backend whose shallow `is_available()` probe
# passes but whose deep import chain is broken (whisperx →
# ctranslate2 failing to dlopen on a hardened kernel) reached
# `.transcribe()` and 500'd, even with a healthy engine next in line.
# The loader does select + ensure_loaded + degrade (#1185). It loads
# weights, so it belongs inside the pool with the transcribe call —
# never on the event loop.
def _run():
backend = load_active_asr_backend()
return backend.transcribe(tmp_path, word_timestamps=word_ts)
result = await run_transcribe_guarded(_gpu_pool, _run, what="OpenAI")
result = await run_transcribe_guarded(
_gpu_pool,
lambda: backend.transcribe(tmp_path, word_timestamps=word_ts),
what="OpenAI",
)
# Extract the full text from segments
segments = result.get("segments", [])
@@ -653,14 +621,6 @@ async def create_transcription(
except HTTPException:
raise
except ASRModelMissingError as e:
# A degraded-to candidate has no weights on disk. Same typed 409 the
# preflight above raises — never a 500, and never a silent multi-GB
# auto-download.
raise HTTPException(
status_code=409,
detail={**e.payload, "message": asr_model_missing_detail(e.payload)},
)
except TimeoutError as e:
# ASRTimeoutError (subclass): backend alive, ASR too heavy for compute.
logger.warning("OpenAI transcription timed out: %s", e)
@@ -676,12 +636,12 @@ async def create_transcription(
pass
# ── Voices: GET /v1/audio/voices (VoiceStudio extension) ─────────────────────
# ── Voices: GET /v1/audio/voices (OmniVoice extension) ─────────────────────
@router.get("/voices")
def list_voices():
"""List available voices. VoiceStudio extension to the OpenAI API."""
"""List available voices. OmniVoice extension to the OpenAI API."""
from services.tts_backend import list_backends
backends = list_backends()
@@ -693,7 +653,7 @@ def list_voices():
"voice_id": name,
"name": name.capitalize(),
"type": "openai_alias",
"description": f"OpenAI '{name}' voice — maps to the active VoiceStudio engine's default voice.",
"description": f"OpenAI '{name}' voice — maps to the active OmniVoice engine's default voice.",
})
# Include voice profiles from the database
@@ -711,7 +671,7 @@ def list_voices():
"language": row["language"],
})
except Exception:
logger.warning("Voice profiles could not be loaded; returning built-in aliases only")
pass
return {"voices": voices, "engines": backends}
@@ -721,11 +681,17 @@ def list_voices():
def _format_ts_srt(seconds: float) -> str:
"""Format seconds as SRT timestamp: HH:MM:SS,mmm"""
from services.srt_parser import format_cue_timestamp
return format_cue_timestamp(seconds, ",")
h = int(seconds // 3600)
m = int((seconds % 3600) // 60)
s = int(seconds % 60)
ms = int((seconds % 1) * 1000)
return f"{h:02d}:{m:02d}:{s:02d},{ms:03d}"
def _format_ts_vtt(seconds: float) -> str:
"""Format seconds as VTT timestamp: HH:MM:SS.mmm"""
from services.srt_parser import format_cue_timestamp
return format_cue_timestamp(seconds, ".")
h = int(seconds // 3600)
m = int((seconds % 3600) // 60)
s = int(seconds % 60)
ms = int((seconds % 1) * 1000)
return f"{h:02d}:{m:02d}:{s:02d}.{ms:03d}"
+10 -18
View File
@@ -28,8 +28,6 @@ from core import event_bus
from core.config import VOICES_DIR # noqa: F401 — re-exported for tests/monkeypatch
from core.db import db_conn
from core.version import APP_VERSION
from core.logging_utils import log_safe
from core.http_headers import content_disposition
from services import persona_bundle as pb
router = APIRouter()
@@ -89,8 +87,8 @@ async def export_persona(
detail="This profile has no readable reference or locked audio to "
"build a preview from — re-create or re-import it.",
)
except Exception as exc:
logger.error("persona export failed for %s: %s", log_safe(profile_id), log_safe(exc))
except Exception:
logger.exception("persona export failed for %s", profile_id)
raise HTTPException(
status_code=503,
detail="Could not build the persona bundle — see Settings → Logs.",
@@ -102,7 +100,7 @@ async def export_persona(
BytesIO(content),
media_type="application/zip",
headers={
"Content-Disposition": content_disposition(filename),
"Content-Disposition": f'attachment; filename="{filename}"',
"Content-Length": str(len(content)),
},
)
@@ -237,18 +235,15 @@ async def import_persona(file: UploadFile = File(...)):
_insert(profile_id)
except HTTPException:
if not _cleanup(written):
raise HTTPException(status_code=500, detail="Import failed, and temporary files could not be removed. Close any app using them and retry cleanup.")
_cleanup(written)
raise
except Exception:
cleaned = _cleanup(written)
logger.warning("Persona import failed")
detail = ("Import failed; no files were kept." if cleaned else
"Import failed, and temporary files could not be removed. Close any app using them and retry cleanup.")
raise HTTPException(status_code=500, detail=detail)
_cleanup(written)
logger.exception("persona import failed")
raise HTTPException(status_code=500, detail="Import failed; no files were kept.")
event_bus.emit("profiles", {"action": "created", "id": profile_id})
logger.info("Imported persona %s as %s (verified=%s)", log_safe(persona.get("name")), log_safe(profile_id), verified)
logger.info("Imported persona %r as %s (verified=%s)", persona.get("name"), profile_id, verified)
return {
"success": True,
@@ -264,16 +259,13 @@ async def import_persona(file: UploadFile = File(...)):
}
def _cleanup(paths: list[str]) -> bool:
complete = True
def _cleanup(paths: list[str]) -> None:
for p in paths:
try:
if p and os.path.exists(p):
os.remove(p)
except OSError:
complete = False
logger.warning("Persona import temporary-file cleanup did not complete")
return complete
pass
def _rename_for_new_id(written: list[str], new_id: str) -> list[str]:
-182
View File
@@ -1,182 +0,0 @@
"""Keyless portrait search with bounded, normalized, safe thumbnails."""
import asyncio
import base64
import html
import re
from html.parser import HTMLParser
from urllib.parse import urlsplit
import httpx
from fastapi import APIRouter, HTTPException, Query
from core.profile_images import MAX_IMAGE_BYTES, normalize_portrait
router = APIRouter()
def trusted_thumbnail(url: str) -> bool:
try:
parsed = urlsplit(url)
google = parsed.hostname in {
f"encrypted-tbn{i}.gstatic.com" for i in range(4)
}
openverse = (
parsed.hostname == "api.openverse.org"
and re.fullmatch(r"/v1/images/[0-9a-f-]+/thumb/?", parsed.path) is not None
)
return (
parsed.scheme == "https"
and not parsed.username
and not parsed.password
and parsed.port in (None, 443)
and (google or openverse)
)
except ValueError:
return False
def google_thumbnails(document: str) -> list[tuple[str, str]]:
"""Extract result thumbnails only; never download third-party originals."""
results = []
seen = set()
def add(title, source):
if not isinstance(source, str) or source in seen:
return
if not (trusted_thumbnail(source) or source.startswith("data:image/jpeg;base64,")):
return
seen.add(source)
if len(results) < 20:
results.append((title or "", source))
class Images(HTMLParser):
def handle_starttag(self, tag, attrs):
if tag == "img":
values = dict(attrs)
add(values.get("alt"), values.get("src") or values.get("data-src"))
Images().feed(document)
# Google also assigns thumbnails from script strings after rendering.
decoded = html.unescape(document)
for escaped, literal in ((r"\u003d", "="), (r"\u0026", "&"), (r"\/", "/")):
decoded = decoded.replace(escaped, literal)
for match in re.finditer(r'https://encrypted-tbn[0-3]\.gstatic\.com/[^\s"\'<>\\]+|data:image/jpeg;base64,[A-Za-z0-9+/=]+', decoded):
add("", match.group())
return results
async def openverse_thumbnails(client: httpx.AsyncClient, name: str) -> list[tuple[str, str]]:
"""Public-domain/CC portrait fallback when Google returns its JS-only shell.
Openverse requires no user credential, excludes sensitive results by
default, and can restrict results to licenses that allow modification and
commercial use. We still fetch only its own thumbnail proxy.
"""
response = await client.get(
"https://api.openverse.org/v1/images/",
headers={
"User-Agent": "VoiceStudio/0.5 (+https://github.com/debpalash/VoiceStudio)",
"Accept": "application/json",
},
params={
"q": name,
"page_size": 20,
"mature": "false",
"extension": "jpg,png",
"aspect_ratio": "square",
"license_type": "commercial,modification",
},
)
response.raise_for_status()
if len(response.content) > 4 * 1024 * 1024:
raise ValueError("Search response too large")
payload = response.json()
rows = payload.get("results") if isinstance(payload, dict) else None
if not isinstance(rows, list):
raise ValueError("Invalid search response")
results = []
seen = set()
for row in rows:
if not isinstance(row, dict):
continue
source = row.get("thumbnail")
if not isinstance(source, str) or source in seen or not trusted_thumbnail(source):
continue
seen.add(source)
title = str(row.get("title") or name)
creator = str(row.get("creator") or "").strip()
license_name = str(row.get("license") or "").upper()
credit = " · ".join(value for value in (creator, license_name) if value)
results.append((f"{title}{credit}" if credit else title, source))
return results
@router.get("/profile-images/search")
async def search_profile_images(name: str = Query(min_length=1, max_length=100)):
if not name.strip():
raise HTTPException(422, detail={"code": "image_search_failed"})
async with httpx.AsyncClient(
timeout=15,
follow_redirects=False,
headers={
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 Chrome/140.0.0.0 Safari/537.36",
"Accept-Language": "en-US,en;q=0.9",
},
) as client:
results: list[tuple[str, str]] = []
try:
async with client.stream("GET", "https://www.google.com/search", params={
"q": name.strip(), "udm": "2", "safe": "active", "tbs": "ift:jpg",
}) as response:
response.raise_for_status()
document = bytearray()
async for chunk in response.aiter_bytes():
document.extend(chunk)
if len(document) > 4 * 1024 * 1024:
raise ValueError("Search page too large")
page = document.decode("utf-8", errors="replace")
results = google_thumbnails(page)
except (httpx.HTTPError, ValueError, TypeError):
# Search providers can change their anonymous HTML or reject a
# non-browser request. The fallback below keeps this explicit,
# user-triggered feature useful without requiring credentials.
results = []
if not results:
try:
results = await openverse_thumbnails(client, name.strip())
except (httpx.HTTPError, ValueError, TypeError):
results = []
if not results:
raise HTTPException(502, detail={"code": "image_search_failed"})
async def thumbnail(title, url):
try:
if url.startswith("data:image/jpeg;base64,"):
encoded = url.partition(",")[2]
if len(encoded) > MAX_IMAGE_BYTES * 4 // 3 + 4:
return None
data = base64.b64decode(encoded, validate=True)
else:
if not trusted_thumbnail(url):
return None
async with client.stream("GET", url) as image:
image.raise_for_status()
data = bytearray()
async for chunk in image.aiter_bytes():
data.extend(chunk)
if len(data) > MAX_IMAGE_BYTES:
return None
normalized = await asyncio.to_thread(normalize_portrait, bytes(data))
return {"title": title[:200], "data": base64.b64encode(normalized).decode("ascii")}
except (httpx.HTTPError, HTTPException, ValueError, TypeError):
return None
images = []
for start in range(0, len(results), 5):
batch = await asyncio.gather(*(thumbnail(*result) for result in results[start:start + 5]))
images.extend(image for image in batch if image)
if len(images) >= 5:
break
return {"images": images[:5]}
+13 -121
View File
@@ -1,5 +1,3 @@
import asyncio
import logging
import os
import re
import uuid
@@ -15,22 +13,8 @@ from core.config import VOICES_DIR, OUTPUTS_DIR
from core import event_bus
from core.personalities import get_personalities
from omnivoice.utils.voice_design import heal_design_instruct, sanitize_instruct
from core.path_security import UnsafePath, resolve_within
from core.profile_images import MAX_IMAGE_BYTES, normalize_portrait
from starlette.datastructures import UploadFile as StarletteUploadFile
router = APIRouter()
logger = logging.getLogger("omnivoice.profiles")
def _profile_record(row):
result = dict(row)
image_path = _voices_path(f"{result['id']}.portrait.jpg")
result["image_url"] = (
f"/profiles/{result['id']}/image?v={os.stat(image_path).st_mtime_ns}"
if image_path and os.path.isfile(image_path) else None
)
return result
class ProfileUpdate(BaseModel):
@@ -50,7 +34,7 @@ def list_personalities():
def list_profiles():
with db_conn() as conn:
rows = conn.execute("SELECT * FROM voice_profiles ORDER BY created_at DESC").fetchall()
return [_profile_record(r) for r in rows]
return [dict(r) for r in rows]
_DESIGN_SEED = 42 # deterministic sample render, same as archetype previews
@@ -66,7 +50,6 @@ async def create_profile(
personality: str = Form(""),
kind: str = Form("clone"),
vd_states: Optional[str] = Form(None),
image: Optional[UploadFile] = File(None),
):
"""Create a voice profile (spec: docs/specs/voice-studio-unification.md §5).
@@ -76,9 +59,6 @@ async def create_profile(
archetype materialization) and stores it as the profile's
reference so the voice identity is stable across runs.
"""
name = name.strip()
if not name:
raise HTTPException(status_code=400, detail="A voice profile needs a name.")
if kind not in ("clone", "design"):
raise HTTPException(status_code=422, detail="kind must be 'clone' or 'design'")
if kind == "clone" and ref_audio is None:
@@ -125,38 +105,13 @@ async def create_profile(
instruct = sanitize_instruct(instruct)
profile_id = str(uuid.uuid4())[:8]
portrait = None
if isinstance(image, StarletteUploadFile):
portrait = normalize_portrait(await image.read(MAX_IMAGE_BYTES + 1))
portrait_path = os.path.join(VOICES_DIR, f"{profile_id}.portrait.jpg")
if kind == "clone":
ext = os.path.splitext(ref_audio.filename or ".wav")[1]
audio_filename = f"{profile_id}{ext}"
audio_path = os.path.join(VOICES_DIR, audio_filename)
# Storage can be removed after startup; recover before persisting uploads.
os.makedirs(VOICES_DIR, exist_ok=True)
with open(audio_path, "wb") as f:
f.write(await ref_audio.read())
# A matching transcript defines the boundary between the reference and
# the requested line. Saving a blank transcript and waiting until the
# first generation made that first take depend on the TTS model's
# internal ASR fallback; short lines could then start with stray words
# from the reference. Resolve it while the profile is being created so
# every synthesis, including the first, uses stable conditioning. This
# remains best-effort and local-only: transcribe_reference considers
# only already-installed ASR/dictation models.
if not ref_text.strip():
try:
from services.asr_backend import transcribe_reference
ref_text = (
await asyncio.to_thread(transcribe_reference, audio_path) or ""
).strip()
except Exception as exc: # noqa: BLE001 — profile save remains usable
logger.warning(
"reference transcription during profile save failed: %s", exc
)
used_seed = seed
else:
# Saving a design profile is a pure persistence operation — it must not
@@ -197,10 +152,6 @@ async def create_profile(
used_seed = seed if seed is not None else _DESIGN_SEED
try:
if portrait:
os.makedirs(VOICES_DIR, exist_ok=True)
with open(portrait_path, "wb") as out:
out.write(portrait)
with db_conn() as conn:
conn.execute(
"INSERT INTO voice_profiles (id, name, ref_audio_path, ref_text, instruct, "
@@ -210,14 +161,12 @@ async def create_profile(
used_seed, personality, kind, vd_states, time.time())
)
except Exception:
if os.path.exists(portrait_path):
os.remove(portrait_path)
# Clean up orphaned audio file if DB insert fails
if os.path.exists(audio_path):
os.remove(audio_path)
raise
event_bus.emit("profiles", {"action": "created", "id": profile_id})
return get_profile(profile_id)
return {"id": profile_id, "name": name, "kind": kind}
@router.get("/profiles/{profile_id}")
def get_profile(profile_id: str):
@@ -231,47 +180,14 @@ def get_profile(profile_id: str):
status_code=404,
detail="That voice profile doesn't exist. It may have been deleted from another tab.",
)
return _profile_record(row)
@router.get("/profiles/{profile_id}/image")
def get_profile_image(profile_id: str):
get_profile(profile_id)
path = _voices_path(f"{profile_id}.portrait.jpg")
if not path or not os.path.isfile(path):
raise HTTPException(404, "Profile image not found")
return FileResponse(path, media_type="image/jpeg", headers={"Cache-Control": "no-cache"})
@router.put("/profiles/{profile_id}/image")
async def update_profile_image(profile_id: str, image: UploadFile = File(...)):
get_profile(profile_id)
path = _voices_path(f"{profile_id}.portrait.jpg")
if path is None:
raise HTTPException(404, "Profile not found")
portrait = normalize_portrait(await image.read(MAX_IMAGE_BYTES + 1))
os.makedirs(VOICES_DIR, exist_ok=True)
with open(path, "wb") as out:
out.write(portrait)
event_bus.emit("profiles", {"action": "updated", "id": profile_id})
return get_profile(profile_id)
return dict(row)
@router.put("/profiles/{profile_id}")
def update_profile(profile_id: str, patch: ProfileUpdate):
"""Partial update — only fields set on the payload are changed."""
with db_conn() as conn:
existing = conn.execute(
"SELECT kind FROM voice_profiles WHERE id = ?", (profile_id,),
).fetchone()
if not existing:
raise HTTPException(
status_code=404,
detail="That voice profile doesn't exist. It may have been deleted from another tab.",
)
fields = []
params = []
edited_instruct = None
for col in ("name", "ref_text", "instruct", "language", "personality"):
val = getattr(patch, col)
if val is None:
@@ -282,22 +198,12 @@ def update_profile(profile_id: str, patch: ProfileUpdate):
# Never let an edit persist a validator-rejecting instruct (prose /
# "[object Object]"); keep only whitelist tags (#550 #571 #594 #596).
val = sanitize_instruct(val)
edited_instruct = val
fields.append(f"{col} = ?")
params.append(val.strip() if col in ("name", "language") else val)
if edited_instruct is not None and existing["kind"] == "design":
# Keep the complete recipe synchronized with the editable instruct.
# Otherwise clients restore a stale vd_states snapshot and a successful
# style edit has no effect on the next generation.
import json
from core.describe_voice import instruct_to_vd_states
fields.append("vd_states = ?")
params.append(json.dumps(instruct_to_vd_states(edited_instruct)))
if not fields:
raise HTTPException(
status_code=400,
detail="PUT /profiles/{id} body contained no editable fields. Include at least one of: name, language, ref_text, instruct, personality.",
detail="PUT /profiles/{id} body contained no editable fields. Include at least one of: name, language, instruct, description.",
)
params.append(profile_id)
with db_conn() as conn:
@@ -314,7 +220,7 @@ def update_profile(profile_id: str, patch: ProfileUpdate):
"SELECT * FROM voice_profiles WHERE id = ?", (profile_id,),
).fetchone()
event_bus.emit("profiles", {"action": "updated", "id": profile_id})
return _profile_record(row)
return dict(row)
@router.get("/profiles/{profile_id}/usage")
@@ -345,14 +251,8 @@ def get_profile_usage(profile_id: str):
state = json.loads(r["state_json"] or "{}")
except Exception:
continue
if not isinstance(state, dict):
continue
# Current desktop snapshots use dubSegments. An explicit empty list
# supersedes legacy segments retained in an older snapshot.
segs = state.get("dubSegments", state.get("segments", []))
if not isinstance(segs, list):
continue
n = sum(1 for s in segs if isinstance(s, dict) and s.get("profile_id") == profile_id)
segs = state.get("segments") or []
n = sum(1 for s in segs if s.get("profile_id") == profile_id)
if n:
project_hits.append({
"project_id": r["id"],
@@ -477,18 +377,13 @@ async def lock_profile(
if not history or not history["audio_path"]:
raise HTTPException(status_code=404, detail="History item not found or has no audio")
try:
src_path = resolve_within(OUTPUTS_DIR, history["audio_path"])
except UnsafePath as exc:
raise HTTPException(status_code=400, detail="Invalid history audio path") from exc
if not src_path.is_file():
src_path = os.path.join(OUTPUTS_DIR, history["audio_path"])
if not os.path.exists(src_path):
raise HTTPException(status_code=404, detail="Audio file not found on disk")
locked_filename = f"{profile_id}_locked.wav"
locked_path = _voices_path(locked_filename)
if locked_path is None:
raise HTTPException(status_code=400, detail="Invalid profile id")
shutil.copy2(str(src_path), locked_path)
locked_path = os.path.join(VOICES_DIR, locked_filename)
shutil.copy2(src_path, locked_path)
ref_text = history["text"][:100] if history["text"] else ""
@@ -510,8 +405,8 @@ async def unlock_profile(profile_id: str):
)
if profile["locked_audio_path"]:
locked_path = _voices_path(profile["locked_audio_path"])
if locked_path and os.path.exists(locked_path):
locked_path = os.path.join(VOICES_DIR, profile["locked_audio_path"])
if os.path.exists(locked_path):
os.remove(locked_path)
conn.execute(
@@ -638,9 +533,6 @@ def delete_profile(profile_id: str):
path = _voices_path(row[col])
if path and os.path.exists(path):
os.remove(path)
portrait_path = _voices_path(f"{profile_id}.portrait.jpg")
if portrait_path and os.path.isfile(portrait_path):
os.remove(portrait_path)
# Prevent FOREIGN KEY constraint failure
conn.execute("UPDATE generation_history SET profile_id = NULL WHERE profile_id=?", (profile_id,))
conn.execute("DELETE FROM voice_profiles WHERE id=?", (profile_id,))
+11 -22
View File
@@ -7,7 +7,7 @@ CRUD for the DB-backed, per-language pronunciation dictionary the
before synthesis (see ``services/pronunciation.apply_pronunciation`` and the
generate path), so a saved entry actually changes the audio on every engine.
Endpoints (admin-gated; loopback or authenticated server mode):
Endpoints (loopback-only, like the dictation router):
GET /pronunciation list every entry
POST /pronunciation create one entry
PUT /pronunciation/{entry_id} update an entry (partial)
@@ -30,16 +30,12 @@ from typing import List, Optional
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from api.dependencies import require_admin
from api.dependencies import require_loopback
from core.db import db_conn
from services.pronunciation import (
apply_pronunciation,
entries_for_language,
inert_entries_for_language,
)
from services.pronunciation import apply_pronunciation, entries_for_language
logger = logging.getLogger("omnivoice.pronunciation")
router = APIRouter(dependencies=[Depends(require_admin)])
router = APIRouter()
_VALID_TYPES = ("respelling", "ipa", "cmu")
_ALL_LANG = "*"
@@ -137,7 +133,7 @@ class PronImportRequest(BaseModel):
# ── CRUD ─────────────────────────────────────────────────────────────────────
@router.get("/pronunciation")
@router.get("/pronunciation", dependencies=[Depends(require_loopback)])
def list_entries():
with db_conn() as conn:
rows = conn.execute(
@@ -147,7 +143,7 @@ def list_entries():
return [_row_to_dict(r) for r in rows]
@router.post("/pronunciation")
@router.post("/pronunciation", dependencies=[Depends(require_loopback)])
def create_entry(entry: PronEntry):
term = entry.term.strip()
if not term:
@@ -175,7 +171,7 @@ def create_entry(entry: PronEntry):
return _row_to_dict(row)
@router.put("/pronunciation/{entry_id}")
@router.put("/pronunciation/{entry_id}", dependencies=[Depends(require_loopback)])
def update_entry(entry_id: str, patch: PronEntryUpdate):
with db_conn() as conn:
existing = conn.execute(
@@ -230,7 +226,7 @@ def update_entry(entry_id: str, patch: PronEntryUpdate):
return _row_to_dict(row)
@router.delete("/pronunciation/{entry_id}")
@router.delete("/pronunciation/{entry_id}", dependencies=[Depends(require_loopback)])
def delete_entry(entry_id: str):
with db_conn() as conn:
cur = conn.execute("DELETE FROM pronunciation_entries WHERE id = ?", (entry_id,))
@@ -240,7 +236,7 @@ def delete_entry(entry_id: str):
# ── Dry-run + import/export ───────────────────────────────────────────────────
@router.post("/pronunciation/test")
@router.post("/pronunciation/test", dependencies=[Depends(require_loopback)])
def test_substitution(req: PronTestRequest):
"""Show the post-substitution text for ``req.text`` — no model call.
@@ -254,22 +250,15 @@ def test_substitution(req: PronTestRequest):
).fetchall()
substituted = apply_pronunciation(req.text, rows, req.language)
applied = entries_for_language(rows, req.language)
# IPA/CMU rows are validated and stored but not applied yet, so a term that
# DOES match can still change nothing. Reporting them separately keeps the
# dry run honest — otherwise it says "no entries match", which is wrong and
# sends the user to re-type an entry that was already correct (#1949).
inert = inert_entries_for_language(rows, req.language)
return {
"input": req.text,
"substituted": substituted,
"changed": substituted != req.text,
"applied_terms": sorted(applied.keys(), key=len, reverse=True),
# Present but not honoured: [{term, type}, …]. Empty on the happy path.
"inert_entries": inert,
}
@router.get("/pronunciation/export")
@router.get("/pronunciation/export", dependencies=[Depends(require_loopback)])
def export_entries():
"""Every entry as a JSON-serializable list (round-trips ``/import``)."""
with db_conn() as conn:
@@ -284,7 +273,7 @@ def export_entries():
]}
@router.post("/pronunciation/import")
@router.post("/pronunciation/import", dependencies=[Depends(require_loopback)])
def import_entries(req: PronImportRequest):
"""Bulk-add entries. ``replace=true`` clears the table first.
+42 -213
View File
@@ -1,10 +1,11 @@
"""Settings API — HF token save/clear/state endpoints (Phase 1 AUTH-03 backend half).
These endpoints are the backend half of the Wave 2 Settings API Keys
panel. Threat T-01-03 mitigation: the router-level `require_admin` dependency
keeps desktop callers loopback-only and requires the long API key for every
remote server-mode mutation. Read-only bare-Docker discovery remains available
until an API key is configured; once configured, reads require it too.
panel. Threat T-01-03 mitigation: every write endpoint is gated by the
router-level `require_loopback` dep, so non-loopback origins get 403
before the handler runs. Reads are loopback-gated too the masked
token preview is useful telemetry that we still don't want exposed on
the LAN.
The state endpoint duplicates `/system/hf-token/state` (which lives on
`system.py` for legacy-router compatibility); both return the same shape.
@@ -19,16 +20,14 @@ from dataclasses import asdict
from fastapi import APIRouter, Depends, HTTPException, Query
from pydantic import BaseModel, Field
from core.logging_utils import log_safe
from core.engine_licenses import LICENSE_GATED_ENGINES
from api.dependencies import require_admin, require_admin_action
from api.dependencies import require_loopback
logger = logging.getLogger("omnivoice.api.settings")
router = APIRouter(
prefix="/api/settings",
tags=["settings"],
dependencies=[Depends(require_admin)],
dependencies=[Depends(require_loopback)],
)
@@ -36,11 +35,11 @@ class _HFTokenBody(BaseModel):
token: str = Field(..., min_length=1, description="HuggingFace access token")
def _state_response(*, validate: bool = False) -> dict:
def _state_response() -> dict:
"""Return the same shape the React panel renders. Never includes raw token."""
from services import token_resolver
s = token_resolver.state(validate=validate)
s = token_resolver.state()
return {
"active": s["active"],
"sources": [asdict(row) for row in s["sources"]],
@@ -66,7 +65,8 @@ def save_hf_token(body: _HFTokenBody):
@router.delete("/hf-token")
def clear_hf_token(also_clear_hf_cli: bool = Query(False)):
"""Clear the App token and optionally recognized local Hub token files."""
"""Clear the App-source token. Optionally also call huggingface_hub.logout
to clear the canonical HF file. Returns the updated cascade state."""
from services import token_resolver
try:
token_resolver.clear_app_token(also_clear_hf_cli=also_clear_hf_cli)
@@ -82,81 +82,25 @@ def get_hf_token_state(fresh: bool = Query(False)):
``fresh=1`` drops the resolver's whoami validation cache first so the
response re-runs whoami for every source this is what the panel's
"Test now" button sends. Plain GETs only inspect local token presence.
"Test now" button sends. Plain GETs (panel mounts) keep the 300s cache
so repeat Settings visits don't hammer the HF API.
"""
from services import token_resolver
if fresh:
token_resolver.invalidate_cache()
return _state_response(validate=fresh)
return _state_response()
# ── Performance settings (INST-12) ────────────────────────────────────────
# Threat T-02-04: same admin guard as the hf-token endpoints via the
# router-level `require_admin` dep.
# Threat T-02-04: same loopback guard as the hf-token endpoints via the
# router-level `require_loopback` dep.
_TORCH_COMPILE_KEY = "perf.torch_compile_disabled"
from services.performance_profiles import (
_PERFORMANCE_PROFILE_KEY, _PERFORMANCE_TIERS, _PERFORMANCE_FAMILIES,
activate_performance_tier,
profile_state as _performance_profile_state,
)
class _PerformanceProfileBody(BaseModel):
tier: str = Field(..., description="fast | balanced | quality | max")
family: str | None = Field(None, description="Engine family, or null to set the global tier")
@router.get("/performance-profile")
def get_performance_profile():
"""Return the global speed/quality preference and per-engine overrides."""
return _performance_profile_state()
@router.put("/performance-profile")
def set_performance_profile(body: _PerformanceProfileBody):
"""Persist a performance preference and apply installed Max-capacity picks."""
from core import prefs
tier = body.tier.strip().lower()
if tier not in _PERFORMANCE_TIERS:
raise HTTPException(status_code=400, detail="Unknown performance tier")
family = body.family.strip().lower() if body.family else None
if family is not None and family not in _PERFORMANCE_FAMILIES:
raise HTTPException(status_code=400, detail="Unknown engine family")
state = _performance_profile_state()
applicable = state["applicable_families"]
if (family is not None and family not in applicable) or (family is None and not applicable):
raise HTTPException(status_code=409, detail="The selected engines do not support this performance preset")
from core import job_store
from api.routers.batch import list_batch_jobs
if job_store.list_jobs(status="active", limit=1) or list_batch_jobs(status="active", limit=1):
raise HTTPException(status_code=409, detail="Wait for queued or running jobs to finish before changing performance presets")
try:
if family is None:
# One atomic write clears family overrides together with the global
# choice, so a crash cannot leave half of a global change persisted.
prefs.update_mapping(_PERFORMANCE_PROFILE_KEY, {"global": tier}, replace=True)
else:
prefs.update_mapping(_PERFORMANCE_PROFILE_KEY, {family: tier})
except Exception:
logger.exception("set_performance_profile failed")
raise HTTPException(status_code=500, detail="Failed to persist performance profile")
activations = activate_performance_tier(tier, family)
result = _performance_profile_state()
if activations:
result["runtime_activations"] = activations
if tier == "max":
result["capacity_activations"] = activations
return result
class _TorchCompileBody(BaseModel):
enabled: bool = Field(..., description="True to disable torch.compile (eager mode) for the engine")
enabled: bool = Field(..., description="True to set TORCH_COMPILE_DISABLE=1 on engine subprocesses")
def _torch_compile_state() -> dict:
@@ -170,21 +114,15 @@ def _torch_compile_state() -> dict:
@router.get("/perf/torch-compile-disabled")
def get_torch_compile_disabled():
"""Return the current torch.compile-disabled toggle + the runtime platform.
`platform` is still reported (clients may show it), but since #2135 the
toggle is live on every host: it used to be rendered disabled off Windows
on the assumption that only #65's Windows OOM needed it, which left the
Linux/CUDA reporter of #2135 with no way to switch off the compile that
was killing their backend.
"""
UI uses the platform to render the toggle disabled (with an explainer)
on non-Windows hosts, since the OOM is Windows-specific (issue #65)."""
return _torch_compile_state()
@router.put("/perf/torch-compile-disabled")
def set_torch_compile_disabled(body: _TorchCompileBody):
"""Persist the toggle. Honoured by `services.engine_env.build_engine_env()`
(subprocess engines) and `services.engine_env.should_torch_compile()`
(in-process), on every platform since #2135."""
which injects TORCH_COMPILE_DISABLE=1 on Windows when enabled."""
from services import settings_store
try:
@@ -195,83 +133,6 @@ def set_torch_compile_disabled(body: _TorchCompileBody):
return _torch_compile_state()
# ── Compute-device override (Settings → Performance) ──────────────────────
class _ComputeDeviceBody(BaseModel):
value: str = Field(..., description="auto | cuda | rocm | xpu | mps | cpu")
def _compute_device_state() -> dict:
"""Everything the Performance panel needs to render the device control:
the resolved pick (env > prefs > auto), what this process actually applied
at probe time (differs after a change until restart caps are immutable
per process), what auto would pick, and which families exist here."""
from core import device_caps
caps = device_caps.detect_host_caps()
env_pin = (os.environ.get("OMNIVOICE_DEVICE") or "").strip().lower()
auto_family = next(
(f for f in device_caps.ACCELERATOR_PRIORITY if f in caps.available_families),
"cpu",
)
value = device_caps.requested_device_override()
return {
"value": value,
"applied": caps.requested_family,
"restart_required": value != caps.requested_family,
# The running process asked for a family it doesn't have (env pin on
# the wrong machine, hardware removed): auto is in effect, and a
# restart would not change that — the panel says so instead of
# pretending the pick took.
"override_ignored": (
caps.requested_family not in ("auto", caps.family)
),
"effective_family": caps.family,
"auto_family": auto_family,
"available_families": list(caps.available_families),
"env_pinned": env_pin in device_caps.DEVICE_OVERRIDE_CHOICES and env_pin != "",
"choices": list(device_caps.DEVICE_OVERRIDE_CHOICES),
}
@router.get("/compute-device")
def get_compute_device():
"""Current compute-device override state (Settings → Performance)."""
return _compute_device_state()
@router.put("/compute-device")
def set_compute_device(body: _ComputeDeviceBody):
"""Persist the compute-device pick. Applied by the capability probe at
the next backend start (host caps are immutable per process same
restart contract as the rest of the Performance tab). ``OMNIVOICE_DEVICE``
always wins over this pick; the UI shows the pin instead of pretending."""
from core import device_caps, prefs
value = (body.value or "").strip().lower()
if value not in device_caps.DEVICE_OVERRIDE_CHOICES:
raise HTTPException(
status_code=400,
detail=f"Unknown device '{value}'. Valid: {', '.join(device_caps.DEVICE_OVERRIDE_CHOICES)}",
)
caps = device_caps.detect_host_caps()
if value not in ("auto", "cpu") and value not in caps.available_families:
raise HTTPException(
status_code=400,
detail=(
f"'{value}' is not available on this host "
f"(have: {', '.join(caps.available_families)})"
),
)
try:
prefs.set_("compute_device", value)
except Exception:
logger.exception("set_compute_device failed")
raise HTTPException(status_code=500, detail="Failed to persist setting")
return _compute_device_state()
# ── Generation-history retention (Studio takes rail) ──────────────────────
@@ -584,46 +445,16 @@ def test_llm_provider(provider_id: str):
"reply": reply[:80],
"latency_ms": int((_time.monotonic() - t0) * 1000),
}
except Exception as e: # noqa: BLE001 — classify without exposing diagnostics
kind = _classify_llm_error(e)
from core.public_errors import provider_failure
failure = provider_failure(kind)
# A successful local catalog probe proves the cached model is stale.
# Invalidate it, but never include catalog or exception text in the
# response: both are controlled by the provider.
if kind == "not_found" and p.local:
available = _local_models(base_url, api_key)
if available is not None:
llm_providers.forget_discovered_models(p.id)
except Exception as e: # noqa: BLE001 — surface a clean, scrubbed error to the UI
return {
"ok": False,
**failure,
"kind": _classify_llm_error(e),
"detail": _scrub_llm_detail(e, api_key),
"latency_ms": int((_time.monotonic() - t0) * 1000),
}
def _local_models(base_url: str, api_key: str):
"""Model ids a local OpenAI-compatible server currently serves.
``None`` when the listing itself failed, ``[]`` when it succeeded and the
server has nothing loaded. The distinction is load-bearing: collapsing both
to ``[]`` let the caller state "reports no loaded models" on a lookup that
never happened, which is a confident wrong diagnosis in place of a vague
right one (CodeRabbit). Only used to sharpen an error message, so it must
never raise a second error on top of the first.
"""
try:
from openai import OpenAI
client = OpenAI(api_key=api_key, base_url=base_url, max_retries=0)
return sorted(m.id for m in client.models.list(timeout=5))
except Exception: # noqa: BLE001
return None
@router.get(
"/llm-providers/{provider_id}/models",
dependencies=[Depends(require_admin_action)],
)
@router.get("/llm-providers/{provider_id}/models")
def list_llm_provider_models(provider_id: str):
"""List model ids the provider's key can access (OpenAI-compat /models).
@@ -649,10 +480,10 @@ def list_llm_provider_models(provider_id: str):
# can say "first 200 shown" rather than implying it's the full list.
return {"ok": True, "models": ids[:200], "truncated": len(ids) > 200}
except Exception as e: # noqa: BLE001
from core.public_errors import provider_failure
return {
"ok": False,
**provider_failure(_classify_llm_error(e)),
"kind": _classify_llm_error(e),
"detail": _scrub_llm_detail(e, api_key),
"models": [],
}
@@ -717,7 +548,7 @@ def set_llm_skill(skill_id: str, body: _LLMSkillBody):
#: Engines that have an in-tree acceptance dialog. Adding a new engine
#: here means adding a corresponding frontend dialog + a license URLs
#: dict in its constants module. Until that, the API refuses the write.
_LICENSE_ALLOWED_ENGINES = LICENSE_GATED_ENGINES
_LICENSE_ALLOWED_ENGINES: frozenset[str] = frozenset({"supertonic3"})
class _LicenseAcceptBody(BaseModel):
@@ -746,8 +577,8 @@ def post_license_acceptance(body: _LicenseAcceptBody) -> dict:
from services import settings_store
try:
settings_store.set_license_accepted(eid, body.accepted)
except Exception as exc:
logger.error("set_license_accepted failed for %s: %s", log_safe(eid), log_safe(exc))
except Exception:
logger.exception("set_license_accepted failed for %s", eid)
raise HTTPException(status_code=500, detail="Failed to persist license acceptance")
return {"ok": True, "engine_id": eid, "accepted": bool(body.accepted)}
@@ -772,8 +603,8 @@ def get_license_acceptance(engine_id: str) -> dict:
from services import settings_store
try:
accepted = settings_store.get_license_accepted(eid)
except Exception as exc:
logger.error("get_license_accepted failed for %s: %s", log_safe(eid), log_safe(exc))
except Exception:
logger.exception("get_license_accepted failed for %s", eid)
raise HTTPException(status_code=500, detail="Failed to read license acceptance")
return {"engine_id": eid, "accepted": bool(accepted)}
@@ -805,7 +636,7 @@ def _effective_models_dir() -> str:
class _ModelsDirBody(BaseModel):
authorization: str = Field(description="One-shot native desktop authorization")
path: str = Field(default="", description="Absolute directory; empty clears → default cache")
@router.get("/storage/models-dir")
@@ -834,18 +665,17 @@ def set_models_dir(body: _ModelsDirBody):
saved. Returns restart_required=True.
"""
from core import user_env
from core.path_authorization import PathAuthorizationError, consume
try:
raw = consume(body.authorization, "models_dir").strip()
except PathAuthorizationError as exc:
raise HTTPException(status_code=403, detail=str(exc)) from exc
raw = (body.path or "").strip()
if not raw:
user_env.unset_user_env(_MODELS_DIR_ENV)
return {"configured": None, "default": _default_models_dir(), "restart_required": True}
# Tauri already validates this before issuing the capability. Keep the
# backend checks as defense in depth against a corrupt capability file.
# Reject control characters / NUL before touching the filesystem: an
# embedded NUL makes os.makedirs raise ValueError (→ 500). This is also
# the input-validation barrier for the path before it reaches any fs call
# (the dir is user-chosen by design — this is a loopback-gated, same-user
# local file picker, not a cross-privilege boundary).
if any(ord(ch) < 0x20 or ord(ch) == 0x7F for ch in raw):
raise HTTPException(status_code=400, detail="Path contains invalid control characters")
@@ -906,7 +736,7 @@ async def get_storage_report(refresh: bool = Query(False)):
@router.post("/storage/temp/clear")
async def clear_temp_files():
"""Delete VoiceStudio-owned temp files (Settings → Storage → Temporary files).
"""Delete OmniVoice-owned temp files (Settings → Storage → Temporary files).
Removes only the ``omnivoice*`` entries in the OS temp dir the exact
population the storage report's "temp" category counts — and invalidates
@@ -1097,10 +927,9 @@ def set_asr_openai_compat(body: _ASROpenAICompatBody):
from services import asr_backend, settings_store
if body.base_url is not None:
try:
url = asr_backend.normalize_openai_compat_asr_base_url(body.base_url)
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
url = body.base_url.strip().rstrip("/")
if url and not url.startswith(("http://", "https://")):
raise HTTPException(status_code=400, detail="Base URL must start with http(s)://")
settings_store.set_text(asr_backend._ASR_OPENAI_COMPAT_BASE_URL_KEY, url)
if body.model is not None:
settings_store.set_text(
+37 -467
View File
@@ -13,16 +13,12 @@ import json
import logging
import os
import sys
import threading
import time
from fastapi import APIRouter, HTTPException
from fastapi.responses import StreamingResponse
from pydantic import BaseModel
from core import prefs
from core.failure import is_hf_connectivity_error
from services.hf_revisions import revision_for
from utils import hf_progress
from utils import download_aggregator
# Weight-floor scan (MM2-07 / #352) lives in ``models.py`` — the lowest module in
@@ -32,7 +28,7 @@ from utils import download_aggregator
from .models import ( # noqa: F401
KNOWN_MODELS,
invalidate_cache,
snapshot_is_complete,
snapshot_has_weights,
disk_space_error,
_MIN_WEIGHT_BYTES,
_WEIGHT_FLOORS,
@@ -43,9 +39,6 @@ router = APIRouter()
# Cooldown: prevent rapid re-install after a failure. Maps repo_id → last_fail_time.
_install_cooldowns: dict[str, float] = {}
# Last classified failure per repo. The SSE stream carries the same detail live;
# retaining it here keeps recovery useful after navigation or renderer reconnect.
_install_failures: dict[str, dict] = {}
_COOLDOWN_SECS = 60.0
# Evict cooldown entries older than this so the dict can't grow unbounded across
# a long-lived process (MM2-06). Anything past the cooldown window is dead state.
@@ -58,14 +51,6 @@ def _sweep_cooldowns(now: float) -> None:
stale = [k for k, t in _install_cooldowns.items() if (now - t) > _COOLDOWN_TTL_SECS]
for k in stale:
_install_cooldowns.pop(k, None)
_install_failures.pop(k, None)
stale_failures = [
repo_id
for repo_id, failure in _install_failures.items()
if (now - float(failure.get("failed_at") or 0)) > _COOLDOWN_TTL_SECS
]
for repo_id in stale_failures:
_install_failures.pop(repo_id, None)
def clear_install_cooldowns() -> None:
@@ -75,7 +60,6 @@ def clear_install_cooldowns() -> None:
very next action is "retry the failed download on the new mirror", and a
429 there would dead-end the wizard's switch-and-retry flow."""
_install_cooldowns.clear()
_install_failures.clear()
# Repo_ids the user asked to cancel (FDL-11). Checked between retry attempts.
# Note: a single in-flight snapshot_download/Xet fetch is not interruptible
@@ -83,14 +67,6 @@ def clear_install_cooldowns() -> None:
# cancelled, and clears the cooldown so a cancel isn't rate-limited.
_cancelled: set[str] = set()
# One worker per repo. Repeated clicks and feature-level recovery can converge
# on the same install; starting a second snapshot_download against the same HF
# cache is wasteful and can corrupt the user-visible progress stream.
_active_installs: set[str] = set()
_active_installs_lock = threading.Lock()
_install_tasks: set[asyncio.Task] = set()
_install_tasks_by_repo: dict[str, asyncio.Task] = {}
def _download_max_workers() -> int:
"""Parallel-FILES worker count for snapshot_download (FDL-02). Default 8 —
@@ -193,29 +169,7 @@ def _repo_cancelled(repo_id: str) -> bool:
return repo_id in _cancelled
def _create_cache_pointer(blob_path: str, pointer: str) -> None:
"""Keep the canonical blob while exposing it from the snapshot tree.
huggingface_hub's ``new_blob=True`` fallback moves the blob into the
snapshot when Windows symlinks are unavailable. The next model load then
sees a missing blob and downloads the same multi-gigabyte weight again.
NTFS hardlinks preserve both cache paths without doubling disk usage; other
filesystems fall back to Hugging Face's copy/symlink path.
"""
from huggingface_hub.file_download import _create_symlink
if os.name == "nt":
try:
os.link(blob_path, pointer)
return
except FileExistsError:
return
except OSError:
pass
_create_symlink(blob_path, pointer, new_blob=False)
def _segmented_snapshot(repo_id: str, *, endpoint: "str | None", revision: str) -> str:
def _segmented_snapshot(repo_id: str, *, endpoint: "str | None") -> str:
"""Fetch every file of a repo via the segmented downloader into the HF
cache, mirroring hf_hub_download's blob+snapshot+refs layout so the result
is indistinguishable from snapshot_download (FDL-09) keeping /models
@@ -225,26 +179,17 @@ def _segmented_snapshot(repo_id: str, *, endpoint: "str | None", revision: str)
import asyncio as _asyncio
from huggingface_hub import HfApi, constants as _C
from huggingface_hub.file_download import (
hf_hub_url, get_hf_file_metadata, repo_folder_name,
hf_hub_url, get_hf_file_metadata, repo_folder_name, _create_symlink,
)
from services.segmented_download import segmented_download
from services.token_resolver import resolve as _resolve_token
# `resolve()` returns a ResolvedToken record, not the bearer string, and
# every consumer below is typed `token: str | None`. Handing over the
# record fails silently rather than loudly (#2163): huggingface_hub's
# build_hf_headers ignores a non-str token and falls back to its own
# ambient discovery, so a token held only in VoiceStudio's settings sends
# NO Authorization header at all and every gated file 401s; our own
# segmented_download interpolates it into `f"Bearer {token}"` and sends a
# malformed header carrying the raw secret. Unwrap once, here.
_resolved = _resolve_token()
token = _resolved.token if _resolved else None
token = _resolve_token()
api = HfApi(endpoint=endpoint, token=token)
info = api.repo_info(repo_id, repo_type="model", revision=revision)
info = api.repo_info(repo_id, repo_type="model")
commit = info.sha
files = [s.rfilename for s in (info.siblings or [])]
if commit != revision or not files:
if not commit or not files:
raise RuntimeError("repo_info returned no commit/siblings")
repo_dir = os.path.join(_C.HF_HUB_CACHE, repo_folder_name(repo_id=repo_id, repo_type="model"))
@@ -273,26 +218,14 @@ def _segmented_snapshot(repo_id: str, *, endpoint: "str | None", revision: str)
cancel_check=lambda: _repo_cancelled(repo_id),
))
if not os.path.lexists(pointer):
_create_cache_pointer(blob_path, pointer)
_create_symlink(blob_path, pointer, new_blob=True)
# refs/main → commit so scan_cache_dir maps the revision correctly.
ref_path = os.path.join(refs_dir, "main")
ref_tmp = ref_path + ".tmp"
try:
with open(ref_tmp, "w") as f:
with open(os.path.join(refs_dir, "main"), "w") as f:
f.write(commit)
os.replace(ref_tmp, ref_path)
except OSError as exc:
logger.warning("Downloaded model revision could not be finalized")
try:
os.remove(ref_tmp)
except FileNotFoundError:
pass # Idempotent cleanup: the failed write may not create it.
except OSError:
logger.warning("Downloaded model revision temporary-file cleanup did not complete")
raise RuntimeError(
"Downloaded model revision could not be finalized. Retry the install."
) from exc
except OSError:
pass
return snap_dir
@@ -323,17 +256,10 @@ def _validate_snapshot_has_weights(repo_id: str, snapshot_path: str) -> None:
retry loop and the UI's re-download path can deal with it, instead of at
first synthesis with an opaque transformers error.
Delegates to ``models.snapshot_is_complete`` so configuration-only pipeline
repositories use their declared required files instead of a weight floor."""
model = next((m for m in KNOWN_MODELS if m["repo_id"] == repo_id), {"repo_id": repo_id})
if snapshot_is_complete(model, snapshot_path):
Delegates the weight check to ``models.snapshot_has_weights`` (single source of
the floors); only the install-time error message lives here."""
if snapshot_has_weights(snapshot_path):
return
if model.get("config_only"):
required = ", ".join(model.get("config_required_files") or ())
raise OSError(f"{repo_id}: download is incomplete; required configuration files: {required}")
if model.get("required_files"):
required = ", ".join(model["required_files"])
raise OSError(f"{repo_id}: required model files are missing or incomplete: {required}")
biggest = 0
try:
for root, _dirs, files in os.walk(snapshot_path, followlinks=True):
@@ -348,19 +274,17 @@ def _validate_snapshot_has_weights(repo_id: str, snapshot_path: str) -> None:
f"{repo_id}: download finished but no model weights were found in the "
"snapshot (largest file "
f"{biggest} bytes). The download was likely interrupted — delete the "
"model in Model Catalogue and install it again."
"model in Settings → Models and install it again."
)
@router.get("/setup/download-stream")
async def setup_download_stream(target: str | None = None):
async def setup_download_stream():
"""SSE: forward every HuggingFace download tqdm update as a JSON event."""
queue: asyncio.Queue = asyncio.Queue(maxsize=512)
loop = asyncio.get_running_loop()
def listener(event):
if target and event.get("target", "local") != target:
return
try:
loop.call_soon_threadsafe(_safe_put, queue, event)
except RuntimeError:
@@ -394,157 +318,13 @@ async def setup_download_stream(target: str | None = None):
class InstallModelRequest(BaseModel):
repo_id: str
target: str | None = None
@router.get("/models/install/status")
def model_install_status():
"""Read local and remote jobs after navigation without starting downloads."""
from services import gpu_gateway # noqa: PLC0415
now = time.time()
_sweep_cooldowns(now)
with _active_installs_lock:
active = tuple(_active_installs)
jobs = []
for repo_id in active:
aggregate = download_aggregator._get(repo_id)
jobs.append(
{
"repo_id": repo_id,
"target": "local",
"state": "downloading",
**(aggregate.snapshot() if aggregate else {}),
}
)
detailed = set()
for repo_id, failure in tuple(_install_failures.items()):
failed_at = float(failure.get("failed_at") or 0)
if repo_id in active or now - failed_at >= _COOLDOWN_SECS:
continue
detailed.add(repo_id)
cooldown_at = _install_cooldowns.get(repo_id)
retry_after = (
max(0, int(_COOLDOWN_SECS - (now - cooldown_at) + 0.999))
if cooldown_at is not None
else 0
)
jobs.append(
{
"repo_id": repo_id,
"target": "local",
"state": "failed",
"retry_after_seconds": retry_after,
**failure,
}
)
# Preserve status for callers/tests that seed the legacy cooldown map alone.
jobs.extend(
{
"repo_id": repo_id,
"target": "local",
"state": "failed",
"retry_after_seconds": max(
0, int(_COOLDOWN_SECS - (now - failed_at) + 0.999)
),
}
for repo_id, failed_at in tuple(_install_cooldowns.items())
if repo_id not in active
and repo_id not in detailed
and now - failed_at < _COOLDOWN_SECS
)
jobs.extend(gpu_gateway.remote_download_jobs())
return {"jobs": jobs}
def _is_retryable_download_error(exc: BaseException) -> bool:
"""Whether a failed download attempt is worth retrying.
Decides by CLASSIFICATION, not by exception type. The type-based tuple this
replaced ``(HfHubHTTPError, LocalEntryNotFoundError, OSError)`` silently
excluded ``httpx.RemoteProtocolError``, which inherits ``Exception``: a
4.6 GB model truncated at 4.0 GB escaped all five attempts and aborted the
install (#1224). Any future transport error with a novel base class would
have reopened the same hole.
A user cancel is never retryable, and neither is anything
``is_hf_connectivity_error`` does not recognise.
"""
# Imported here, not at module scope, for the same reason the worker does:
# huggingface_hub is heavy and this module is on the setup import path.
from huggingface_hub.utils import HfHubHTTPError, LocalEntryNotFoundError
if isinstance(exc, _InstallCancelled):
return False
if isinstance(exc, HfHubHTTPError):
# An auth / not-found / gone answer from the Hub is a settled verdict:
# the token is wrong, the repo is gated, or it isn't there. Retrying
# five times with backoff just delays the same message and postpones
# the install cooldown. (Pre-existing behaviour — the type-based tuple
# this replaced retried every HfHubHTTPError; surfaced in #1224 review.)
status = getattr(getattr(exc, "response", None), "status_code", None)
if status in (401, 403, 404, 410):
return False
return True
if isinstance(exc, (LocalEntryNotFoundError, OSError)):
return True
return is_hf_connectivity_error(str(exc))
def _segmented_retry_plan(
exc: BaseException, attempt: int, max_attempts: int
) -> tuple[bool, bool]:
"""What to do after the segmented accelerator failed on ``attempt``.
Returns ``(disable_accelerator, reraise)``.
A dropped connection is not the accelerator's fault, so the error is
re-raised for the outer retry: the next attempt re-enters
:func:`_segmented_snapshot`, which resumes from the ``.part`` manifest.
Falling straight through to ``snapshot_download`` instead would finish the
install from a separate ``.incomplete`` file and strand that manifest the
restart-from-zero this exists to prevent.
The final attempt is always reserved for the plain path, so the accelerator
can never be the reason an install fails outright. The two flags are
decoupled for that handover: the attempt that exhausts the accelerator still
re-raises, so the plain path starts on the LAST attempt rather than the
second-to-last. Disabling and falling through in the same attempt would
abandon the resumable manifest one attempt early and restart through a
separate file which is the failure this whole helper exists to avoid.
"""
if not _is_retryable_download_error(exc):
return True, False # the accelerator cannot work here at all
if attempt >= max_attempts:
# Nothing left to hand over to: take the plain path now rather than
# re-raising out of the loop with no fallback ever tried.
return True, False
return attempt >= max_attempts - 1, True
def _segmented_retry_note(disable: bool, reraise: bool) -> str:
"""How to describe the outcome of :func:`_segmented_retry_plan` in the log.
Three distinct states, and reading only ``disable`` conflates two of them:
the attempt that exhausts the accelerator is disabled AND re-raises, so the
fallback starts on the NEXT attempt, not this one.
"""
if not disable:
return "kept for the next attempt (resumes from its manifest)"
if reraise:
return "exhausted — retrying once more, then snapshot_download takes over"
return "disabled for this install — falling back to snapshot_download now"
@router.post("/models/install")
async def install_model(req: InstallModelRequest):
"""Download one HF repo snapshot; progress goes through the shared
``/setup/download-stream`` SSE feed."""
model_spec = next(
(model for model in KNOWN_MODELS if model["repo_id"] == req.repo_id),
None,
)
if model_spec is None:
if req.repo_id not in [m["repo_id"] for m in KNOWN_MODELS]:
raise HTTPException(
status_code=400,
detail=(
@@ -552,22 +332,6 @@ async def install_model(req: InstallModelRequest):
+ ", ".join(m["repo_id"] for m in KNOWN_MODELS)
),
)
allow_patterns = list(model_spec.get("allow_patterns") or []) or None
target = (req.target or "").strip()
if target != "local":
from services import gpu_gateway # noqa: PLC0415
from worker import routing # noqa: PLC0415
decision = routing.decide()
if target and target != "local" and (
not decision.remote or decision.worker_id != target
):
raise HTTPException(status_code=409, detail="The selected GPU target changed; try again.")
if decision.remote:
try:
return await gpu_gateway.download(req.repo_id, decision=decision)
except gpu_gateway.GatewayError as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
# Cooldown guard — don't retry if the same model just failed.
import time as _time_check
_sweep_cooldowns(_time_check.time()) # bound the dict (MM2-06)
@@ -584,11 +348,8 @@ async def install_model(req: InstallModelRequest):
loop = asyncio.get_running_loop()
def _do():
# Failure handling must work even when imports, token resolution or
# revision lookup fail before the heartbeat thread is started.
_resolving = threading.Event()
token = hf_progress.current_repo_id.set(req.repo_id)
target_token = hf_progress.current_target.set("local")
_cancelled.discard(req.repo_id) # clear any stale cancel from a prior run
hf_progress.emit({
"repo_id": req.repo_id,
"filename": req.repo_id,
@@ -610,15 +371,8 @@ async def install_model(req: InstallModelRequest):
# parallel-files worker count, and honour an optional mirror endpoint.
dl_kwargs: dict = {
"repo_id": req.repo_id,
"revision": revision_for(req.repo_id),
"max_workers": _download_max_workers(),
}
from services.token_resolver import resolve as resolve_token
resolved_token = resolve_token()
if resolved_token:
dl_kwargs["token"] = resolved_token.token
if allow_patterns:
dl_kwargs["allow_patterns"] = allow_patterns
_tqdm_cls = hf_progress.tracked_tqdm_class()
if _tqdm_cls is not None:
dl_kwargs["tqdm_class"] = _tqdm_cls
@@ -630,7 +384,9 @@ async def install_model(req: InstallModelRequest):
# Emit a 'resolving' heartbeat every 2s while snapshot_download
# resolves repo metadata (before any tqdm bars appear).
import threading
import time as _t
_resolving = threading.Event()
def _heartbeat():
_step = 0
@@ -655,31 +411,11 @@ async def install_model(req: InstallModelRequest):
# bytes that will actually download — BEFORE any byte flows. Seeds
# the overall aggregator so its bar/ETA are correct from the first
# event. Degrades gracefully (totals=None) on older/gated repos.
_preflight_kwargs = {
"repo_id": req.repo_id,
"revision": dl_kwargs["revision"],
"dry_run": True,
}
if allow_patterns:
_preflight_kwargs["allow_patterns"] = allow_patterns
_preflight_kwargs = {"repo_id": req.repo_id, "dry_run": True}
if _endpoint:
_preflight_kwargs["endpoint"] = _endpoint
if resolved_token:
_preflight_kwargs["token"] = resolved_token.token
try:
_plan = list(snapshot_download(**_preflight_kwargs)) # nosec B615 -- immutable revision_for pin
for dependency in model_spec.get("dependencies") or ():
if req.repo_id in _cancelled:
raise _InstallCancelled()
dependency_plan_kwargs = {
**_preflight_kwargs,
"repo_id": dependency["repo_id"],
"revision": revision_for(dependency["repo_id"]),
}
dependency_plan_kwargs.pop("allow_patterns", None)
if dependency.get("allow_patterns"):
dependency_plan_kwargs["allow_patterns"] = dependency["allow_patterns"]
_plan.extend(snapshot_download(**dependency_plan_kwargs)) # nosec B615 -- immutable revision_for pin
_plan = snapshot_download(**_preflight_kwargs)
_summary = compute_plan(_plan)
# Disk-space guard (before a single byte flows): the preflight
# gives an exact "to download" size, so reject an install that
@@ -697,18 +433,12 @@ async def install_model(req: InstallModelRequest):
"phase": "install_error",
"error": _disk_err,
})
_install_failures[req.repo_id] = {
"failed_at": time.time(),
"error": _disk_err,
"docs_topic": "DISK_SPACE_LOW",
}
# A disk-full is not a transient network failure — don't set
# a cooldown (freeing space, not waiting, is the fix). The
# outer finally still cleans up the aggregator + context.
return
download_aggregator.start(
req.repo_id,
target=target or "local",
total_bytes=_summary["to_download_bytes"],
files_total=max(0, _summary["n_files"] - _summary["n_cached"]),
)
@@ -718,13 +448,11 @@ async def install_model(req: InstallModelRequest):
"phase": "install_plan",
**_summary,
})
except _InstallCancelled:
raise
except Exception as _pf_err:
# No preflight (older/gated repo, mirror without dry-run, etc.):
# fall back to today's fill-in-as-files-appear behaviour.
logger.info("model install %s: preflight unavailable (%s)", req.repo_id, _pf_err)
download_aggregator.start(req.repo_id, target=target or "local")
download_aggregator.start(req.repo_id)
hf_progress.emit({
"repo_id": req.repo_id,
"filename": req.repo_id,
@@ -738,11 +466,6 @@ async def install_model(req: InstallModelRequest):
_max_attempts = 5
_attempt = 0
# The accelerator is retried across attempts so its manifest-based
# resume actually gets used; it is disabled for the rest of the
# install only when it fails for a reason that is NOT transient
# network trouble (i.e. the accelerator itself is unusable here).
_segmented_off = False
while True:
if req.repo_id in _cancelled:
raise _InstallCancelled()
@@ -750,81 +473,27 @@ async def install_model(req: InstallModelRequest):
try:
# Segmented accelerator (FDL-09, default ON): parallel
# byte-range fetch with real live progress, for the
# legacy-LFS path. A failure that is not transient network
# trouble falls through to snapshot_download, and so does the
# install's last attempt — the accelerator can never
# compromise a correct install (see _segmented_retry_plan).
# legacy-LFS path. Any failure falls through to
# snapshot_download — the accelerator can never compromise a
# correct install.
_snapshot_path = None
if (
not _segmented_off
and not allow_patterns
and _segmented_enabled()
and not _xet_active()
):
if _attempt == 1 and _segmented_enabled() and not _xet_active():
try:
_snapshot_path = _segmented_snapshot(
req.repo_id,
endpoint=_endpoint,
revision=dl_kwargs["revision"],
)
_snapshot_path = _segmented_snapshot(req.repo_id, endpoint=_endpoint)
except _InstallCancelled:
raise
except Exception as _seg_err:
_segmented_off, _seg_reraise = _segmented_retry_plan(
_seg_err, _attempt, _max_attempts
)
logger.info(
"segmented download for %s failed (%s); accelerator %s",
"segmented download for %s failed (%s); falling back to snapshot_download",
req.repo_id, _seg_err,
_segmented_retry_note(_segmented_off, _seg_reraise),
)
if _seg_reraise:
raise
_snapshot_path = None
if _snapshot_path is None:
_snapshot_path = snapshot_download(**dl_kwargs) # nosec B615 -- immutable revision_for pin
_snapshot_path = snapshot_download(**dl_kwargs)
_validate_snapshot_has_weights(req.repo_id, _snapshot_path)
from huggingface_hub.constants import HF_HUB_CACHE
from services.hf_revisions import remember_revision
remember_revision(req.repo_id, dl_kwargs["revision"], HF_HUB_CACHE)
# A pipeline config is not a runnable installation by itself.
# Download its reviewed dependencies only inside this explicit
# install action, retaining the parent cancellation/retry flow.
for dependency in model_spec.get("dependencies") or ():
if req.repo_id in _cancelled:
raise _InstallCancelled()
dependency_id = dependency["repo_id"]
dependency_kwargs = {
**dl_kwargs,
"repo_id": dependency_id,
"revision": revision_for(dependency_id),
}
dependency_kwargs.pop("allow_patterns", None)
if dependency.get("allow_patterns"):
dependency_kwargs["allow_patterns"] = dependency["allow_patterns"]
dependency_path = snapshot_download(**dependency_kwargs) # nosec B615 -- immutable revision_for pin
if not snapshot_is_complete(dependency, dependency_path):
raise OSError(f"{dependency_id}: required model files are missing or incomplete")
remember_revision(dependency_id, dependency_kwargs["revision"], HF_HUB_CACHE)
if req.repo_id in _cancelled:
raise _InstallCancelled()
break
except Exception as net_err:
# #1224: a truncated body ("peer closed connection without
# sending complete message body") arrives as
# httpx.RemoteProtocolError, which inherits from Exception
# — NOT OSError — so it escaped the old
# (HfHubHTTPError, LocalEntryNotFoundError, OSError) tuple
# and aborted a 4.6 GB install at 4.0 GB with no retry.
# Widen to Exception and decide by CLASSIFICATION:
# is_hf_connectivity_error is already the single source of
# truth for "transient download failure" and now knows the
# truncation signatures. Anything unrecognised (a cancel, a
# validation failure, a bug) propagates untouched, exactly
# as before.
if _attempt >= _max_attempts or not _is_retryable_download_error(
net_err
):
except (HfHubHTTPError, LocalEntryNotFoundError, OSError) as net_err:
if _attempt >= _max_attempts:
raise
_backoff = min(30, 2 ** _attempt)
logger.info(
@@ -862,7 +531,7 @@ async def install_model(req: InstallModelRequest):
# Flush the overall bar to 100% with the true byte total (FDL-06):
# under Xet the per-file byte bars don't surface completion, so the
# aggregator can sit below 100% even though every file landed.
download_aggregator.complete(req.repo_id, target=target or "local")
download_aggregator.complete(req.repo_id)
logger.info("model install done: %s", req.repo_id)
hf_progress.emit({
"repo_id": req.repo_id,
@@ -871,28 +540,12 @@ async def install_model(req: InstallModelRequest):
"phase": "install_done",
})
_install_cooldowns.pop(req.repo_id, None) # success clears any cooldown (MM2-06)
_install_failures.pop(req.repo_id, None)
invalidate_cache()
# A saved performance pack owns the desired engine/model policy.
# Reconcile after every successful local install so the new model
# becomes usable without a restart or a second manual selection.
try:
from services.performance_profiles import reconcile_active_profile
activated = reconcile_active_profile()
if activated:
logger.info("model install activated performance profile: %s", activated)
except Exception:
# The model is fully installed even if optional preference
# reconciliation fails; readiness refresh and manual selection
# remain available instead of misreporting the download.
logger.exception("performance profile reconciliation failed after model install")
except _InstallCancelled:
_resolving.set()
logger.info("model install cancelled: %s", req.repo_id)
# A cancel is user intent, not a failure — don't set a cooldown.
_install_cooldowns.pop(req.repo_id, None)
_install_failures.pop(req.repo_id, None)
hf_progress.emit({
"repo_id": req.repo_id,
"filename": req.repo_id,
@@ -903,8 +556,7 @@ async def install_model(req: InstallModelRequest):
_resolving.set()
logger.info("model install failed for %s: %s", req.repo_id, e)
import time as _time_fail
_failed_at = _time_fail.time()
_install_cooldowns[req.repo_id] = _failed_at
_install_cooldowns[req.repo_id] = _time_fail.time()
# #874: when the install failed because the configured HF mirror is
# unreachable, name the mirror + the setting instead of leaking the
# raw connectivity error. #959: likewise for the SOCKS-proxy class
@@ -913,96 +565,23 @@ async def install_model(req: InstallModelRequest):
# class so the wizard can react structurally (HF_MIRROR_UNREACHABLE
# raises the inline mirror picker) without string-matching.
from core.failure import append_hint, classify
_error = append_hint(str(e))
_docs_topic = classify(str(e))
# Gated catalogue entries own their recovery topic. Hugging Face
# uses several exception wordings for the same access verdict, so
# the UI must not depend on parsing an English 401/403 message.
_catalogue_topic = str(model_spec.get("failure_topic") or "")
if _catalogue_topic and _docs_topic in {
"",
"HF_AUTH_FAILED",
"PYANNOTE_LICENSE_REQUIRED",
}:
_docs_topic = _catalogue_topic
# Waiting cannot fix an access/token verdict. Let the user accept
# the terms or update the token and retry immediately.
if _docs_topic in {
"HF_AUTH_FAILED",
"PYANNOTE_LICENSE_REQUIRED",
"POCKETTTS_GATED_WEIGHTS",
}:
_install_cooldowns.pop(req.repo_id, None)
_install_failures[req.repo_id] = {
"failed_at": _failed_at,
"error": _error,
"docs_topic": _docs_topic,
}
hf_progress.emit({
"repo_id": req.repo_id,
"filename": req.repo_id,
"downloaded": 0, "total": 0, "pct": 0.0,
"phase": "install_error",
"error": _error,
"docs_topic": _docs_topic,
"error": append_hint(str(e)),
"docs_topic": classify(str(e)),
})
finally:
_resolving.set()
_cancelled.discard(req.repo_id)
download_aggregator.finish(req.repo_id, target=target or "local")
download_aggregator.finish(req.repo_id)
hf_progress.current_repo_id.reset(token)
hf_progress.current_target.reset(target_token)
with _active_installs_lock:
_active_installs.discard(req.repo_id)
with _active_installs_lock:
if req.repo_id in _active_installs:
return {"status": "already_running", "repo_id": req.repo_id}
_active_installs.add(req.repo_id)
# Admission and task publication are one atomic generation boundary:
# cancellation can never observe an admitted install without its task.
_cancelled.discard(req.repo_id)
_install_failures.pop(req.repo_id, None)
try:
task = loop.create_task(asyncio.to_thread(_do))
_install_tasks.add(task)
_install_tasks_by_repo[req.repo_id] = task
except Exception:
_active_installs.discard(req.repo_id)
raise
def install_finished(completed: asyncio.Task) -> None:
with _active_installs_lock:
_install_tasks.discard(completed)
if _install_tasks_by_repo.get(req.repo_id) is completed:
_install_tasks_by_repo.pop(req.repo_id, None)
task.add_done_callback(install_finished)
loop.create_task(asyncio.to_thread(_do))
return {"status": "install_started", "repo_id": req.repo_id}
async def cancel_install_and_wait(repo_id: str) -> None:
"""Request cancellation and retain authority until its thread exits."""
from worker.async_utils import drain_task # noqa: PLC0415
with _active_installs_lock:
_cancelled.add(repo_id)
_install_cooldowns.pop(repo_id, None)
task = _install_tasks_by_repo.get(repo_id)
if task is None:
return
try:
# asyncio.to_thread cannot stop snapshot_download mid-file. Cancelling
# its wrapper would only detach the thread, so wait until the blocking
# call observes the flag or naturally returns.
await drain_task(task)
finally:
with _active_installs_lock:
current = _install_tasks_by_repo.get(repo_id)
if current is None or current is task:
_cancelled.discard(repo_id)
@router.post("/models/install/cancel")
async def cancel_install(req: InstallModelRequest):
"""Request cancellation of an in-flight install (FDL-11).
@@ -1012,17 +591,8 @@ async def cancel_install(req: InstallModelRequest):
in hf_hub 1.7.2, so an already-streaming file finishes; the cancel takes
effect at the next retry boundary. Clears the cooldown so the user can
immediately restart."""
target = (req.target or "local").strip() or "local"
if target != "local":
from services import gpu_gateway # noqa: PLC0415
try:
return await gpu_gateway.cancel_download(req.repo_id, target=target)
except gpu_gateway.GatewayError as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
_cancelled.add(req.repo_id)
_install_cooldowns.pop(req.repo_id, None)
_install_failures.pop(req.repo_id, None)
return {"cancelling": req.repo_id}
+63 -234
View File
@@ -15,7 +15,7 @@ import sys
import time
from pathlib import Path
from fastapi import APIRouter, HTTPException, Query
from fastapi import APIRouter
logger = logging.getLogger("omnivoice.setup.models")
router = APIRouter()
@@ -90,36 +90,6 @@ def get_model_catalog() -> ModelCatalog:
# ── Platform Detection ─────────────────────────────────────────────────────
def _target_worker():
"""Selected live remote worker, or None when the catalog targets local."""
try:
from worker import routing, service # noqa: PLC0415
decision = routing.decide()
plane = service.control_plane
return plane.pool.get(decision.worker_id) if decision.remote and plane.pool else None
except Exception:
return None
def _target_host() -> dict | None:
"""Selected remote worker host, or None when the catalog targets local."""
live = _target_worker()
return dict(live.record.host or {}) if live is not None else None
def _target_repo_inventory() -> tuple[str, set[str]] | None:
"""Selected worker id and the catalog repositories it reports on disk."""
live = _target_worker()
if live is None:
return None
downloaded: set[str] = set()
for capability in live.record.capabilities or []:
if capability.get("downloaded"):
downloaded.update(str(repo) for repo in capability.get("repo_ids") or [])
return live.worker_id, downloaded
def _current_platform_tags() -> list[str]:
"""Return platform tags that the current host supports.
@@ -130,25 +100,6 @@ def _current_platform_tags() -> list[str]:
``rocm`` (AMD HIP builds), and ``cpu`` (no GPU acceleration at all
Apple Silicon is NOT tagged cpu; it curates via ``darwin-arm64``).
"""
target = _target_host()
if target is not None:
target_os = {"windows": "win32", "darwin": "darwin"}.get(
str(target.get("os") or "").lower(), "linux"
)
arch = str(target.get("arch") or "").lower()
arch = {"amd64": "x86_64", "aarch64": "arm64"}.get(arch, arch)
tags = [target_os, f"{target_os}-{arch}"]
backend = ""
if target.get("gpus"):
backend = str(target["gpus"][0].get("backend") or "").lower()
if backend:
tags.append(backend)
if backend == "rocm":
tags.append("cuda")
if not backend and not (target_os == "darwin" and arch == "arm64"):
tags.append("cpu")
return tags
tags = [sys.platform]
arch = _platform.machine()
tags.append(f"{sys.platform}-{arch}")
@@ -287,7 +238,7 @@ def _hub_cache_roots() -> list[str]:
HF stores repos under ``$HF_HUB_CACHE`` (== ``$HF_HOME/hub`` by default). When
only ``HF_HOME`` (or the ``~/.cache/huggingface`` default) is known, the repos
live under the ``hub`` subdir so we probe both ``<dir>`` (the
``HF_HUB_CACHE``-is-set case, e.g. VoiceStudio's Windows short cache) and
``HF_HUB_CACHE``-is-set case, e.g. OmniVoice's Windows short cache) and
``<dir>/hub`` (the ``HF_HOME``-only case). Without this the WinError-448
fallback would look one level too high and miss the cache (CodeRabbit #137).
"""
@@ -368,44 +319,23 @@ def _snapshot_dirs(repo_id: str) -> list[str]:
return dirs
def snapshot_is_complete(model: dict, snapshot_path: str) -> bool:
"""Apply the same catalogue requirements during installation and listing."""
config_only = bool(model.get("config_only"))
required = tuple(str(name) for name in (
model.get("config_required_files") if config_only else model.get("required_files")
) or ())
if config_only and not required:
return False
try:
present = all(
os.path.isfile(os.path.join(snapshot_path, name))
and os.path.getsize(os.path.join(snapshot_path, name)) >= (
1 if config_only else _WEIGHT_FLOORS.get(os.path.splitext(name)[1].lower(), 1)
)
for name in required
)
return present and (config_only or snapshot_has_weights(snapshot_path))
except OSError:
return False
def cache_is_complete(model: dict) -> bool:
"""True when this model's on-disk cache is usable (not a truncated download).
Config-only repos carry no weight file of their own. Their catalogue entry
declares the small files that make the pipeline usable, so a README left by a
gated 403 is not mistaken for a completed install. A weight-bearing repo is
complete only if at least one snapshot has weights; if no snapshot directory
is found, the size-based caller's cached result is preserved.
Config-only repos (``config_only: true`` in models.yaml e.g. pyannote's
diarisation pipeline, whose real weights live in referenced sub-repos) carry no
weight file of their own, so the weight check would false-positive them as
incomplete (#622 caveat). They're exempt: cache presence alone means complete.
A weight-bearing repo is complete only if at least one of its snapshots has
weights; if no snapshot dir is found on disk we can't prove truncation, so we
don't downgrade (the size-based caller already decided it's cached).
"""
for dependency in model.get("dependencies") or ():
snapshots = _snapshot_dirs(dependency["repo_id"])
if not any(snapshot_is_complete(dependency, path) for path in snapshots):
return False
if model.get("config_only"):
return True
dirs = _snapshot_dirs(model["repo_id"])
if not dirs:
return True
return any(snapshot_is_complete(model, snapshot) for snapshot in dirs)
return any(snapshot_has_weights(d) for d in dirs)
def _is_cached_on_disk(repo_id: str) -> bool:
@@ -470,17 +400,6 @@ def _scan_cache_on_disk() -> dict[str, dict]:
return out
def _cache_dir_missing(exc: Exception) -> bool:
"""Whether Hugging Face is reporting the normal empty-cache state.
``CacheNotFound`` is expected on a clean installation before the first
download. Treating it like a damaged Windows cache makes every model probe
perform a redundant filesystem fallback and fills the first-run log with
warnings. Unexpected scan failures remain visible and recoverable below.
"""
return type(exc).__name__ == "CacheNotFound"
def is_cached(repo_id: str) -> bool:
"""Best-effort check: does HF have this repo in its cache on disk?"""
try:
@@ -491,8 +410,6 @@ def is_cached(repo_id: str) -> bool:
return True
return False
except Exception as e:
if _cache_dir_missing(e):
return False
# scan_cache_dir can raise on Windows (WinError 448 'untrusted mount
# point'); fall back to a direct disk check so a cached model isn't
# mistaken for missing and re-downloaded in a loop (#117/#118). Logged
@@ -531,62 +448,6 @@ def invalidate_cache() -> None:
# ── Endpoints ──────────────────────────────────────────────────────────────
@router.get("/models/access/status")
def model_access_status(repo_id: str = Query(...)):
"""Check gated Hub access without downloading model files.
This route runs only after an explicit UI action. It never returns the
token or a raw Hub exception; callers need only the per-repository verdict.
"""
model = _catalog.get(repo_id)
if model is None:
raise HTTPException(status_code=404, detail="Unknown model")
if not model.get("gated"):
return {
"repo_id": repo_id,
"token_present": False,
"ready": True,
"repositories": [],
}
from services import token_resolver
resolved = token_resolver.resolve()
repositories = [repo_id]
prerequisite = str(model.get("prerequisite_repo_id") or "").strip()
if prerequisite:
repositories.append(prerequisite)
if not resolved:
return {
"repo_id": repo_id,
"token_present": False,
"ready": False,
"repositories": [
{"repo_id": current, "access": "token_missing"}
for current in repositories
],
}
from huggingface_hub import get_hf_file_metadata, hf_hub_url
results = []
for current in repositories:
try:
url = hf_hub_url(current, filename=".gitattributes")
get_hf_file_metadata(url, token=resolved.token)
access = "granted"
except Exception as exc: # Hub exception types vary across releases.
status = getattr(getattr(exc, "response", None), "status_code", None)
access = "required" if status in {401, 403, 404} else "unavailable"
results.append({"repo_id": current, "access": access})
return {
"repo_id": repo_id,
"token_present": True,
"ready": all(item["access"] == "granted" for item in results),
"repositories": results,
}
@router.get("/models")
def list_models():
"""Catalogue every known model + its on-disk install state.
@@ -594,55 +455,35 @@ def list_models():
Uses a 10 s response cache to avoid repeated ``scan_cache_dir()`` disk
walks when the frontend polls.
"""
platform_tags = _current_platform_tags()
remote_inventory = _target_repo_inventory()
target_key = remote_inventory[0] if remote_inventory else "local"
cache_key = "models:" + target_key + ":" + ",".join(sorted(platform_tags))
cached_response = _cached(cache_key)
cached_response = _cached("models")
if cached_response is not None:
return cached_response
cached_by_repo: dict[str, dict] = {}
if remote_inventory is not None:
for model in KNOWN_MODELS:
if model["repo_id"] in remote_inventory[1]:
cached_by_repo[model["repo_id"]] = {
"size_on_disk": int(float(model.get("size_gb") or 0) * _GIB),
"last_accessed": None,
"nb_files": 0,
}
else:
try:
from huggingface_hub import scan_cache_dir
info = scan_cache_dir()
for entry in info.repos:
cached_by_repo[entry.repo_id] = {
"size_on_disk": entry.size_on_disk,
"last_accessed": entry.last_accessed,
"nb_files": entry.nb_files,
}
except Exception as e:
if _cache_dir_missing(e):
cached_by_repo = {}
else:
# WinError-448 fallback (#117/#118): use a direct disk scan so installed
# models still show as installed instead of offering a re-download.
logger.warning("scan_cache_dir failed (%s); using disk fallback", e)
cached_by_repo = _scan_cache_on_disk()
try:
from huggingface_hub import scan_cache_dir
info = scan_cache_dir()
for entry in info.repos:
cached_by_repo[entry.repo_id] = {
"size_on_disk": entry.size_on_disk,
"last_accessed": entry.last_accessed,
"nb_files": entry.nb_files,
}
except Exception as e:
# WinError-448 fallback (#117/#118): use a direct disk scan so installed
# models still show as installed instead of offering a re-download.
logger.warning("scan_cache_dir failed (%s); using disk fallback", e)
cached_by_repo = _scan_cache_on_disk()
out = []
host_tags = set(platform_tags)
host_tags = set(_current_platform_tags())
for m in KNOWN_MODELS:
cached = cached_by_repo.get(m["repo_id"])
on_disk = (
m["repo_id"] in remote_inventory[1]
if remote_inventory is not None
else cached is not None and cached["size_on_disk"] > 0
)
on_disk = cached is not None and cached["size_on_disk"] > 0
# A size-positive cache can still be a truncated download (config landed,
# weight shard didn't). Treat that as not-installed + incomplete so the
# wizard re-offers the download instead of stranding the user (#622).
incomplete = on_disk and remote_inventory is None and not cache_is_complete(m)
incomplete = on_disk and not cache_is_complete(m)
out.append({
**m,
"installed": on_disk and not incomplete,
@@ -655,17 +496,16 @@ def list_models():
"curated": _model_curated(m, host_tags),
})
response = {
"target": target_key,
"models": out,
"total_installed_bytes": sum(m["size_on_disk_bytes"] for m in out),
"hf_cache_dir": "" if remote_inventory is not None else hf_cache_dir(),
"hf_cache_dir": hf_cache_dir(),
# Free space on the cache volume, so the Model Store header can warn
# BEFORE an "Install all" overruns the disk (pairs with the per-install
# disk_space_error guard in setup/download.py).
"disk_free_gb": None if remote_inventory is not None else round(disk_free_bytes() / _GIB, 1),
"platform_tags": platform_tags,
"disk_free_gb": round(disk_free_bytes() / _GIB, 1),
"platform_tags": _current_platform_tags(),
}
_set_cache(cache_key, response)
_set_cache("models", response)
return response
@@ -678,19 +518,18 @@ def recommendations():
TTS model is required; the ASR picks here are the optional "best for your
system" set the wizard and Settings surface for on-demand install.
"""
is_mac_arm = sys.platform == "darwin" and _platform.machine() == "arm64"
is_mac_intel = sys.platform == "darwin" and _platform.machine() == "x86_64"
is_linux = sys.platform.startswith("linux")
is_windows = sys.platform == "win32"
tags = set(_current_platform_tags())
target_os = "darwin" if "darwin" in tags else "win32" if "win32" in tags else "linux"
target_arch = next((tag.split("-", 1)[1] for tag in tags if tag.startswith(target_os + "-")), _platform.machine())
is_mac_arm = target_os == "darwin" and target_arch == "arm64"
is_mac_intel = target_os == "darwin" and target_arch == "x86_64"
is_linux = target_os == "linux"
is_windows = target_os == "win32"
has_cuda = "cuda" in tags and "rocm" not in tags
has_rocm = "rocm" in tags
# Device label — used as the card title.
if is_mac_arm:
device_label = f"Apple Silicon ({target_arch})"
device_label = f"Apple Silicon ({_platform.machine()})"
elif is_mac_intel:
device_label = "macOS Intel (x86_64)"
elif is_windows:
@@ -698,7 +537,7 @@ def recommendations():
elif is_linux:
device_label = "Linux x64" + (" + CUDA" if has_cuda else " + ROCm" if has_rocm else "")
else:
device_label = f"{target_os} / {target_arch}"
device_label = f"{sys.platform} / {_platform.machine()}"
# Curated preset for this host, in catalog order (required entries lead).
curated = [
@@ -708,60 +547,51 @@ def recommendations():
if is_mac_arm:
rationale = (
"Apple Silicon preset: VoiceStudio (required) covers multilingual TTS + "
"Apple Silicon preset: OmniVoice (required) covers multilingual TTS + "
"cloning on its own. The optional picks are Metal-native: MLX Whisper "
"large-v3 for dubbing/transcription, Whisper Turbo (MLX) + Parakeet TDT "
"v3 for live dictation, Kokoro + KittenTTS for instant English TTS."
)
elif has_cuda:
rationale = (
"NVIDIA preset: VoiceStudio (required) runs standalone. Optional ASR picks "
"NVIDIA preset: OmniVoice (required) runs standalone. Optional ASR picks "
"are CUDA-accelerated via CTranslate2 — Whisper large-v3 for dubbing "
"(best word timestamps) and Turbo for 5× faster transcription. Whisper "
"Tiny provides broad-language local dictation. KittenTTS adds CPU-realtime English."
"(best word timestamps), Turbo for 5× faster transcription, Parakeet TDT "
"v3 for live dictation. KittenTTS adds CPU-realtime English."
)
elif has_rocm:
rationale = (
"AMD/ROCm preset: VoiceStudio (required) runs standalone. CTranslate2 has "
"AMD/ROCm preset: OmniVoice (required) runs standalone. CTranslate2 has "
"no ROCm backend, so the PyTorch Whisper large-v3 build is the "
"GPU-accelerated ASR route; faster-whisper works on CPU, and Whisper Tiny "
"provides broad-language local dictation."
"GPU-accelerated ASR route; faster-whisper works on CPU, and Parakeet "
"TDT v3 handles live dictation."
)
else:
rationale = (
"CPU preset: VoiceStudio (required) runs standalone. Optional picks favour "
"CPU preset: OmniVoice (required) runs standalone. Optional picks favour "
"speed on CPU — Whisper large-v3 (int8) for accuracy, Turbo when speed "
"matters, Whisper Tiny (ONNX) for live dictation, KittenTTS for "
"matters, Parakeet TDT v3 (int8 ONNX) for live dictation, KittenTTS for "
"instant English TTS."
)
remote_inventory = _target_repo_inventory()
cached_ids: set[str] = set()
if remote_inventory is not None:
cached_ids = remote_inventory[1]
else:
try:
from huggingface_hub import scan_cache_dir
info = scan_cache_dir()
cached_ids = {
entry.repo_id for entry in info.repos if entry.size_on_disk > 0
}
except Exception as e:
if _cache_dir_missing(e):
cached_ids = set()
else:
# WinError-448 fallback (#117/#118): recommend based on the disk scan.
logger.debug("scan_cache_dir failed (%s); using disk fallback", e)
cached_ids = set(_scan_cache_on_disk().keys())
try:
from huggingface_hub import scan_cache_dir
info = scan_cache_dir()
cached_ids = {
entry.repo_id for entry in info.repos if entry.size_on_disk > 0
}
except Exception as e:
# WinError-448 fallback (#117/#118): recommend based on the disk scan.
logger.debug("scan_cache_dir failed (%s); using disk fallback", e)
cached_ids = set(_scan_cache_on_disk().keys())
entries = []
for meta in curated:
rid = meta["repo_id"]
# Mirror /models: a truncated cache (weights missing) is not installed, so
# the wizard counts it toward the remaining download instead of "all set".
installed = rid in cached_ids and (
remote_inventory is not None or cache_is_complete(meta)
)
installed = rid in cached_ids and cache_is_complete(meta)
entries.append({
"repo_id": rid,
"label": meta.get("label", rid),
@@ -776,10 +606,9 @@ def recommendations():
all_installed = all(e["installed"] for e in entries)
return {
"target": remote_inventory[0] if remote_inventory is not None else "local",
"device": {
"os": target_os,
"arch": target_arch,
"os": sys.platform,
"arch": _platform.machine(),
"is_mac_arm": is_mac_arm,
"is_mac_intel": is_mac_intel,
"is_linux": is_linux,
+12 -45
View File
@@ -19,7 +19,6 @@ import sys
from fastapi import APIRouter
from api.schemas import SetupStatusResponse, PreflightResponse
from core.device_caps import KERNEL_RISK_MARKER
# MIN_FREE_GB + disk_free_bytes are single-sourced in ``.models`` (the lowest
# module in the setup import graph) so the wizard gate, the /models header, and
# the per-install disk guard can't drift apart.
@@ -63,11 +62,6 @@ def setup_status():
_MIN_NVIDIA_DRIVER = 555
_RAM_FAIL_GB = 8
_RAM_WARN_GB = 12
# Installed DIMMs never fully reach the OS: firmware, integrated graphics and
# kernel reservations shave off up to ~7% (an "8 GB" Windows laptop reports
# ~7.8 GB usable). Thresholds are compared with this allowance applied so the
# machines a threshold is meant to admit aren't blocked by that gap (#1618).
_RAM_RESERVED_ALLOWANCE = 0.93
def _run_cmd(args: list[str], timeout: float = 2.0) -> tuple[int, str]:
@@ -175,8 +169,8 @@ def _detect_gpu() -> dict:
return info
def _probe_network(host: str = "huggingface.co", port: int = 443, timeout: float = 8.0) -> bool:
"""Tiny TCP connect test. 8s default — high-latency / China paths often exceed 23s."""
def _probe_network(host: str = "huggingface.co", port: int = 443, timeout: float = 2.0) -> bool:
"""Tiny TCP connect test."""
import socket
try:
with socket.create_connection((host, port), timeout=timeout):
@@ -189,7 +183,7 @@ def _hf_endpoint_host() -> tuple[str, int]:
"""Host/port of the Hugging Face endpoint actually in effect.
Mirror-aware: restricted-network users (e.g. behind the Great Firewall)
point HF_ENDPOINT at a mirror via Settings Network Hugging Face
point HF_ENDPOINT at a mirror via Settings Models Hugging Face
mirror. Probing hardcoded huggingface.co would fail them even when their
configured mirror works fine.
"""
@@ -197,8 +191,7 @@ def _hf_endpoint_host() -> tuple[str, int]:
from core.failure import configured_hf_mirror
mirror = configured_hf_mirror()
except Exception:
logger.warning("Configured Hugging Face endpoint could not be read")
return "", 0
mirror = ""
if mirror:
try:
from urllib.parse import urlsplit
@@ -206,10 +199,7 @@ def _hf_endpoint_host() -> tuple[str, int]:
if u.hostname:
return u.hostname, u.port or (80 if u.scheme == "http" else 443)
except Exception:
logger.warning("Configured Hugging Face endpoint could not be parsed")
return "", 0
logger.warning("Configured Hugging Face endpoint has no host")
return "", 0
pass
return "huggingface.co", 443
@@ -282,14 +272,6 @@ def _network_check() -> dict:
# Manual mode (explicit endpoint) — probe exactly what the user chose.
net_host, net_port = _hf_endpoint_host()
if not net_host:
return {
"id": "network", "label": "Network (configured endpoint)",
"status": "warn",
"detail": "The configured Hugging Face endpoint could not be validated.",
"fix": "Review the endpoint in Settings → Network, then re-check.",
"mirror_reachable": False,
}
net_ok = _probe_network(net_host, net_port)
mirror_reachable = False
if not net_ok and net_host == "huggingface.co":
@@ -358,28 +340,17 @@ def preflight():
# ── RAM
ram = _ram_gb()
# Escape hatch (#1618): a preflight should inform, not brick setup —
# OMNIVOICE_RAM_PREFLIGHT=0 downgrades the hard block to a warning for
# users who accept the OOM risk. Same opt-out shape as
# OMNIVOICE_ASR_VRAM_PREFLIGHT.
ram_gate = os.environ.get(
"OMNIVOICE_RAM_PREFLIGHT", "1"
).strip().lower() not in ("0", "false", "no")
if ram == 0:
ram_status, ram_detail, ram_fix = (
"warn", "Could not detect system RAM.",
"Install psutil in the backend environment or ignore this warning.",
)
elif ram < _RAM_FAIL_GB * _RAM_RESERVED_ALLOWANCE:
elif ram < _RAM_FAIL_GB:
ram_status, ram_detail, ram_fix = (
"fail" if ram_gate else "warn",
f"{ram:.1f} GB total (need ≥ {_RAM_FAIL_GB} GB)",
"The app will OOM on first dub. Close other apps or upgrade RAM."
if ram_gate else
"RAM check disabled via OMNIVOICE_RAM_PREFLIGHT=0 — dubbing may "
"OOM on this machine.",
"fail", f"{ram:.1f} GB total (need ≥ {_RAM_FAIL_GB} GB)",
"The app will OOM on first dub. Close other apps or upgrade RAM.",
)
elif ram < _RAM_WARN_GB * _RAM_RESERVED_ALLOWANCE:
elif ram < _RAM_WARN_GB:
ram_status, ram_detail, ram_fix = (
"warn", f"{ram:.1f} GB total ({_RAM_WARN_GB}+ GB recommended)",
"Long videos may hit swap. Keep other apps closed during dubbing.",
@@ -499,14 +470,10 @@ def preflight():
_why = gpu_routing.get("routing_reason")
if _rs == "accelerated" and not _why:
r_status, r_detail, r_fix = "pass", f"{_eng}{_dev} (accelerated)", None
elif _rs == "accelerated" and KERNEL_RISK_MARKER in (_why or ""):
elif _rs == "accelerated": # driver/arch caveat
r_status, r_detail, r_fix = "warn", f"{_eng}{_dev}: {_why}", (
"GPU selected but may fail at kernel launch — update drivers / "
"reinstall torch for this GPU architecture.")
elif _rs == "accelerated": # low-VRAM caveat — not a driver/arch issue
r_status, r_detail, r_fix = "warn", f"{_eng}{_dev}: {_why}", (
"Unload other models before generating, keep the text short, "
"or pick a lighter engine.")
elif _rs == "cpu_fallback":
r_status, r_detail, r_fix = "warn", (
f"{_eng} runs on CPU here: {_why or 'no GPU path for this host'}"), (
@@ -517,10 +484,10 @@ def preflight():
elif _rs == "unavailable":
r_status, r_detail, r_fix = "fail", (
f"{_eng} can't run on this host: {_why or 'needs a GPU this machine lacks'}"), (
"Select an engine with a CPU path in Model Catalogue.")
"Select an engine with a CPU path in Settings → Engines.")
else: # "none" / unknown
r_status, r_detail, r_fix = "warn", "No active TTS engine resolved for routing.", (
"Pick an engine in Model Catalogue.")
"Pick an engine in Settings → Engines.")
checks.append({
"id": "gpu_routing", "label": "Active engine routing",
"status": r_status, "detail": r_detail, "fix": r_fix,
+8 -22
View File
@@ -5,11 +5,10 @@ SoniTranslate sidecar integration.
"""
import logging
from fastapi import APIRouter, Depends, HTTPException
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel
from typing import Optional
from api.dependencies import require_native_access
from services import sonitranslate as soni
router = APIRouter(prefix="/engines/sonitranslate", tags=["SoniTranslate"])
@@ -64,15 +63,15 @@ async def sonitranslate_stop():
class DubRequest(BaseModel):
video_authorization: str
video_path: str
target_language: str = "Spanish (es)"
source_language: str = "Automatic detection"
tts_voice: str = "es-ES-AlvaroNeural-Male"
max_speakers: int = 1
output_authorization: str | None = None
output_dir: Optional[str] = None
@router.post("/dub", dependencies=[Depends(require_native_access)])
@router.post("/dub")
async def sonitranslate_dub(body: DubRequest):
"""Run full dubbing pipeline via SoniTranslate.
@@ -82,7 +81,7 @@ async def sonitranslate_dub(body: DubRequest):
KNOWN PROVENANCE GAP (#1169, documented — not silently ignored): the
dubbed audio is synthesized and muxed entirely inside the external
SoniTranslate sidecar (its own venv + gradio pipeline, Edge-TTS voices),
which hands back a finished video file. VoiceStudio's tensor-stage
which hands back a finished video file. OmniVoice's tensor-stage
mark_synthetic chokepoint never sees that audio; marking it would require
a demux embed re-mux post-pass on the sidecar's output, which is a
lossy re-encode of a pipeline we don't control. This opt-in engine
@@ -90,28 +89,15 @@ async def sonitranslate_dub(body: DubRequest):
AudioSeal provenance mark that every built-in synthesis path carries.
"""
try:
from core.path_authorization import PathAuthorizationError, consume
try:
video_path = consume(body.video_authorization, "soni_input")
output_dir = (
consume(body.output_authorization, "soni_output_dir")
if body.output_authorization
else None
)
except PathAuthorizationError as exc:
raise HTTPException(status_code=403, detail=str(exc)) from exc
result = await soni.dub_video(
video_path=video_path,
video_path=body.video_path,
target_language=body.target_language,
source_language=body.source_language,
tts_voice=body.tts_voice,
max_speakers=body.max_speakers,
output_dir=output_dir,
output_dir=body.output_dir,
)
return result
except HTTPException:
raise
except Exception as e:
logger.exception("SoniTranslate dub failed")
raise HTTPException(status_code=500, detail=str(e))
-160
View File
@@ -1,160 +0,0 @@
"""Discovery contract for VoiceStudio's local speech platform.
Interfaces should discover this document instead of hard-coding whichever
dictation route the desktop happens to use. Endpoint URLs are relative so the
same response works on loopback, a tailnet GPU host, and a reverse proxy.
"""
from __future__ import annotations
import os
from typing import Literal
from fastapi import APIRouter
from pydantic import BaseModel, Field
from core.version import APP_VERSION
router = APIRouter(tags=["Speech Platform"])
SPEECH_PROTOCOL = "voicestudio.speech.v1"
STREAM_PATH = "/v1/audio/transcriptions/stream"
class EndpointCapability(BaseModel):
path: str
transport: Literal["http", "websocket", "mcp-streamable-http", "mcp-stdio"]
method: str | None = None
protocol: str | None = None
class StreamInputCapability(BaseModel):
framing: Literal["binary"] = "binary"
formats: list[str]
default_format: str
sample_rate_query: str = "sr"
end_control: dict[str, str]
class StreamOutputCapability(BaseModel):
framing: Literal["json"] = "json"
events: list[str]
final_kinds: list[str]
class SpeechFeatureCapabilities(BaseModel):
batch_transcription: bool = True
streaming_transcription: bool = True
partial_transcripts: bool = True
utterance_finals: bool = True
session_summary: bool = True
word_timestamps: bool = True
local_refinement: bool = True
acoustic_echo_cancellation: bool = True
native_dictation_control: bool = False
class SpeechAuthCapabilities(BaseModel):
loopback: Literal["none"] = "none"
remote: Literal["bearer"] = "bearer"
header: str = "Authorization: Bearer <OMNIVOICE_API_KEY>"
browser_session_endpoint: str = "/api/auth/session"
websocket_ticket_endpoint: str = "/api/auth/ws-ticket"
websocket_ticket_query_parameter: Literal["ws_ticket"] = "ws_ticket"
class SpeechCapabilities(BaseModel):
schema_: Literal["voicestudio.speech-capabilities"] = Field(
default="voicestudio.speech-capabilities",
serialization_alias="schema",
)
protocol: Literal["voicestudio.speech.v1"] = SPEECH_PROTOCOL
protocol_version: Literal["1.0"] = "1.0"
service: str = "VoiceStudio"
service_version: str = APP_VERSION
local_first: bool = True
endpoints: dict[str, EndpointCapability]
stream_input: StreamInputCapability
stream_output: StreamOutputCapability
features: SpeechFeatureCapabilities
authentication: SpeechAuthCapabilities
def speech_capabilities() -> SpeechCapabilities:
"""Return the stable, side-effect-free integration contract."""
endpoints = {
"capabilities": EndpointCapability(
path="/.well-known/voicestudio-speech",
transport="http",
method="GET",
),
"batch_transcription": EndpointCapability(
path="/v1/audio/transcriptions",
transport="http",
method="POST",
protocol="openai.audio.transcriptions",
),
"streaming_transcription": EndpointCapability(
path=STREAM_PATH,
transport="websocket",
protocol=SPEECH_PROTOCOL,
),
"mcp": EndpointCapability(
path="/mcp",
transport="mcp-streamable-http",
method="POST",
protocol="mcp",
),
"mcp_stdio": EndpointCapability(
path="python -m backend.mcp_shim",
transport="mcp-stdio",
protocol="mcp",
),
}
native_control = False
try:
control_port = int(os.environ.get("VOICESTUDIO_SPEECH_CONTROL_PORT", ""))
except (TypeError, ValueError):
control_port = 0
if 0 < control_port <= 65535:
native_control = True
endpoints["native_dictation_control"] = EndpointCapability(
path=f"http://127.0.0.1:{control_port}/v1/capabilities",
transport="http",
method="GET",
protocol=SPEECH_PROTOCOL,
)
return SpeechCapabilities(
endpoints=endpoints,
stream_input=StreamInputCapability(
formats=[
"audio/pcm;encoding=s16le;channels=1",
"audio/webm;codecs=opus",
],
default_format="audio/webm;codecs=opus",
end_control={"type": "input_audio.end"},
),
stream_output=StreamOutputCapability(
events=["session.started", "status", "partial", "final", "error"],
final_kinds=["utterance", "summary"],
),
features=SpeechFeatureCapabilities(
native_dictation_control=native_control,
),
authentication=SpeechAuthCapabilities(),
)
@router.get(
"/.well-known/voicestudio-speech",
response_model=SpeechCapabilities,
response_model_by_alias=True,
)
@router.get(
"/v1/audio/capabilities",
response_model=SpeechCapabilities,
response_model_by_alias=True,
)
async def get_speech_capabilities() -> SpeechCapabilities:
"""Advertise batch, streaming, and agent-facing speech transports."""
return speech_capabilities()
+2 -3
View File
@@ -14,7 +14,6 @@ from fastapi import APIRouter, UploadFile, File, Form, HTTPException
from fastapi.responses import Response
from services.ffmpeg_utils import find_ffmpeg, spawn_subprocess
from core.http_headers import content_disposition
router = APIRouter()
@@ -39,7 +38,7 @@ async def stories_encode(
synthesis producer it never calls a TTS engine, so it must not call
mark_synthetic (the upload may be arbitrary user audio, and marking human
speech as synthetic would be wrong). Audio the Stories Editor stitched
from VoiceStudio generations is already marked at its producing route, and
from OmniVoice generations is already marked at its producing route, and
the AudioSeal mark survives the lossy encode here.
"""
fmt = (format or "mp3").lower()
@@ -77,7 +76,7 @@ async def stories_encode(
return Response(
content=encoded,
media_type=mime,
headers={"Content-Disposition": content_disposition(f"story.{ext}")},
headers={"Content-Disposition": f'attachment; filename="story.{ext}"'},
)
finally:
for p in (in_path, out_path):
+109 -519
View File
@@ -11,39 +11,32 @@ from core.prefs import set_ as prefs_set, delete as prefs_delete
from services import network_share
from services import tailscale as _tailscale
from api.schemas import SysinfoResponse, SystemInfoResponse, ModelStatusResponse
from api.dependencies import is_loopback, require_admin, require_admin_action
from api.dependencies import require_loopback
from fastapi.responses import FileResponse, StreamingResponse
import torch
import shutil
import subprocess
import shlex
from core.config import OUTPUTS_DIR, DATA_DIR, CRASH_LOG_PATH, LOG_PATH, IDLE_TIMEOUT_SECONDS
from core.version import APP_VERSION
from core.logging_utils import log_safe
from core.public_errors import public_failure
from services.model_manager import get_model_status, get_best_device, resolve_omnivoice_checkpoint
from services.ffmpeg_utils import find_ffmpeg, run_ffmpeg
# Router-level admin gate. Every route mounted on `router` (GET + POST,
# present and future) is gated by `require_admin`: desktop requests must be
# loopback; server-mode mutations require the long API key. This closes the trust
# Router-level loopback gate. Every route mounted on `router` (GET + POST,
# present and future) is gated by `require_loopback`, which 403s any request
# whose `client.host` is not a loopback address. This closes the same trust
# boundary that PR #81 only patched on `/system/set-env` and that the
# 260518-ivy deferred-items file enumerated for follow-up: /model/unload/*,
# /system/logs/clear, /system/logs/tauri/clear, /system/flush-memory,
# /clean-audio (POSTs) plus the read-side info-disclosure routes
# /system/info, /system/logs, /system/logs/tauri, /system/logs/stream.
# Native Tauri/dev callers remain loopback and need no credential.
router = APIRouter(dependencies=[Depends(require_admin)])
# This router only ever serves the local Tauri shell and the dev frontend
# at http://127.0.0.1:3901 — both are loopback origins.
router = APIRouter(dependencies=[Depends(require_loopback)])
logger = logging.getLogger("omnivoice.api")
# Cache device checks at module load — they don't change at runtime
_is_mac = hasattr(torch.backends, "mps") and torch.backends.mps.is_available()
_is_cuda = torch.cuda.is_available()
try:
_is_xpu = hasattr(torch, "xpu") and torch.xpu.is_available()
except Exception:
_is_xpu = False
# Prime psutil's internal CPU counter so the first non-blocking call returns useful data
psutil.cpu_percent(interval=None)
@@ -57,14 +50,6 @@ def _detect_cpu_model() -> str:
for line in f:
if line.lower().startswith("model name"):
return line.split(":", 1)[1].strip()
if sys.platform == "win32":
import winreg
with winreg.OpenKey(
winreg.HKEY_LOCAL_MACHINE,
r"HARDWARE\DESCRIPTION\System\CentralProcessor\0",
) as key:
return str(winreg.QueryValueEx(key, "ProcessorNameString")[0]).strip()
if sys.platform == "darwin":
import subprocess
return subprocess.check_output(
@@ -75,77 +60,6 @@ def _detect_cpu_model() -> str:
return platform.processor() or ""
def _gpu_name_priority(name: str) -> tuple[int, int]:
lowered = name.lower()
if any(token in lowered for token in ("remote", "virtual", "basic display")):
return (-1, len(name))
if any(token in lowered for token in ("nvidia", "radeon", "amd", "intel arc")):
return (2, len(name))
return (1, len(name))
def _detect_os_gpu_name() -> str:
"""Best-effort display-adapter identity when the active torch build is CPU-only."""
try:
if sys.platform == "win32":
executable = shutil.which("powershell.exe") or shutil.which("powershell")
if not executable:
return ""
result = subprocess.run(
[
executable,
"-NoProfile",
"-NonInteractive",
"-Command",
"Get-CimInstance Win32_VideoController | Select-Object -ExpandProperty Name",
],
capture_output=True,
text=True,
timeout=3,
check=False,
creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0),
)
names = [line.strip() for line in result.stdout.splitlines() if line.strip()]
return max(names, key=_gpu_name_priority, default="")
if sys.platform.startswith("linux"):
executable = shutil.which("lspci")
if not executable:
return ""
result = subprocess.run(
[executable, "-mm"],
capture_output=True,
text=True,
timeout=2,
check=False,
)
names = []
for line in result.stdout.splitlines():
parts = shlex.split(line)
if len(parts) >= 4 and parts[1] in {"VGA compatible controller", "3D controller"}:
names.append(" ".join(parts[2:4]))
return max(names, key=_gpu_name_priority, default="")
if sys.platform == "darwin":
executable = shutil.which("system_profiler")
if not executable:
return ""
result = subprocess.run(
[executable, "SPDisplaysDataType"],
capture_output=True,
text=True,
timeout=3,
check=False,
)
names = [
line.split(":", 1)[1].strip()
for line in result.stdout.splitlines()
if "Chipset Model:" in line
]
return max(names, key=_gpu_name_priority, default="")
except (OSError, ValueError, subprocess.SubprocessError):
return ""
return ""
def _detect_gpu() -> tuple[str, float]:
"""(gpu_name, vram_total_gb) — static for the process lifetime.
@@ -156,15 +70,11 @@ def _detect_gpu() -> tuple[str, float]:
if _is_cuda:
props = torch.cuda.get_device_properties(0)
return torch.cuda.get_device_name(0), round(props.total_memory / (1024 ** 3), 1)
if _is_xpu:
props = torch.xpu.get_device_properties(0)
total_memory = float(getattr(props, "total_memory", 0.0))
return torch.xpu.get_device_name(0), round(total_memory / (1024 ** 3), 1)
if _is_mac:
return "Apple Silicon (MPS)", 0.0
except Exception:
pass
return _detect_os_gpu_name(), 0.0
return "", 0.0
# Static hardware facts, captured once — /system/info is hit on every
@@ -182,37 +92,6 @@ def _disk_free_gb() -> float:
return 0.0
def _nvidia_live_stats() -> tuple[float, float, float] | None:
"""Return GPU%, used VRAM GiB, total VRAM GiB without an optional Python dependency."""
executable = shutil.which("nvidia-smi")
if not executable:
return None
try:
creationflags = subprocess.CREATE_NO_WINDOW if sys.platform == "win32" else 0
result = subprocess.run(
[
executable,
"--query-gpu=utilization.gpu,memory.used,memory.total",
"--format=csv,noheader,nounits",
"--id=0",
],
capture_output=True,
text=True,
timeout=1.5,
check=False,
creationflags=creationflags,
)
if result.returncode != 0:
return None
values = [float(value.strip()) for value in result.stdout.splitlines()[0].split(",")]
if len(values) != 3:
return None
utilization, used_mib, total_mib = values
return utilization, used_mib / 1024, total_mib / 1024
except (OSError, ValueError, IndexError, subprocess.SubprocessError):
return None
def _ui_port() -> int:
"""The Vite UI dev-server port, single-sourced from OMNIVOICE_UI_PORT.
@@ -305,8 +184,8 @@ def loaded_models():
@router.post("/model/unload/{model_id}")
async def unload_model(model_id: str):
"""Unload a specific model by id (MM2-04). Delegates to model_lifecycle;
an unknown id maps to HTTP 400. Supports every id returned by
``GET /model/loaded`` plus the aggregate ``sidecars`` id."""
an unknown id maps to HTTP 400. ``tts`` | ``diarization`` |
``sidecar:<id>`` | ``sidecars``."""
from services import model_lifecycle
try:
return await model_lifecycle.unload(model_id)
@@ -323,40 +202,15 @@ def system_info():
"""
try:
_ffmpeg = find_ffmpeg()
from services import model_manager as _mm
from services import asr_backend as _asr_backend
from core import prefs as _prefs_mod
_asr_engine = _asr_backend.active_backend_id()
_asr_model = (
_asr_backend._offline_asr_repo(_asr_engine)
or os.environ.get("ASR_MODEL")
or _asr_engine
)
_translation_provider = (
os.environ.get("TRANSLATE_PROVIDER")
or _prefs_mod.get("translation_backend", "argos")
)
return {
"app_version": APP_VERSION,
"generate_timeout_s": _mm.GPU_JOB_TIMEOUT_S,
"cpu_generate_timeout_s": _mm.CPU_JOB_TIMEOUT_S,
# #1787 review fix: a saved prefs.json value for either key can be
# silently shadowed by an external env var (os.environ.setdefault
# in core.prefs.restore_env is a no-op when one is already
# present) — the Settings panel must say so rather than promise a
# restart will apply a value that never will.
"generate_timeout_shadowed": _prefs_mod.is_env_shadowed(
"OMNIVOICE_GENERATE_TIMEOUT_S"),
"cpu_generate_timeout_shadowed": _prefs_mod.is_env_shadowed(
"OMNIVOICE_CPU_GENERATE_TIMEOUT_S"),
"code_fingerprint": os.environ.get("OMNIVOICE_BUILD_FINGERPRINT", ""),
"data_dir": DATA_DIR,
"outputs_dir": OUTPUTS_DIR,
"crash_log_path": CRASH_LOG_PATH,
"idle_timeout_seconds": IDLE_TIMEOUT_SECONDS,
"model_checkpoint": resolve_omnivoice_checkpoint(), # #693: show the effective checkpoint, not a leaked raw value
"asr_model": _asr_model,
"translate_provider": _translation_provider,
"asr_model": os.environ.get("ASR_MODEL", "Systran/faster-whisper-large-v3"),
"translate_provider": os.environ.get("TRANSLATE_PROVIDER", "google"),
"has_hf_token": _has_hf_token(),
"fast_download": _fast_download_status(),
"device": get_best_device(),
@@ -385,11 +239,6 @@ def system_info():
logger.exception("system_info failed — returning safe defaults")
return {
"app_version": APP_VERSION,
"generate_timeout_s": 300.0,
"cpu_generate_timeout_s": 600.0,
"generate_timeout_shadowed": False,
"cpu_generate_timeout_shadowed": False,
"code_fingerprint": os.environ.get("OMNIVOICE_BUILD_FINGERPRINT", ""),
"data_dir": DATA_DIR,
"outputs_dir": OUTPUTS_DIR,
"crash_log_path": str(CRASH_LOG_PATH),
@@ -417,7 +266,7 @@ def system_info():
"backend_port": network_share.backend_port(),
"share_port_base": network_share.share_port_base(),
"ui_port": _ui_port(),
"error": "System information is temporarily unavailable; check the backend log for details.",
"error": str(e),
}
@@ -428,142 +277,6 @@ def _tail_file(path: str, tail: int):
return all_lines[-tail:], len(all_lines)
# Must track main.py's _WindowsSafeRotatingFileHandler(backupCount=3). The
# handler rolls omnivoice.log at 2 MB into .1/.2/.3, so up to 6 MB of history
# lives in files this module used to ignore entirely.
_LOG_BACKUP_COUNT = 3
def _rotated_log_paths(base: str) -> list[str]:
"""Existing `<base>.1 … .N`, newest first."""
return [p for p in (f"{base}.{i}" for i in range(1, _LOG_BACKUP_COUNT + 1)) if os.path.exists(p)]
def _tail_rolling(base: str, tail: int):
"""Tail `base`, reaching into its rotated siblings when it runs short.
A rollover leaves omnivoice.log nearly empty, and the Backend tab then
showed a handful of lines or none while the failure the user was asked
to copy sat in omnivoice.log.1. Reading the current file first keeps the
common case at one file read; the backups are only touched when they are
the only place the requested lines can come from.
Returns (lines oldest-first, total lines across the files read, paths read
oldest-first). The total counts only the files it had to open it stops as
soon as `tail` is satisfied, so it is "how much is behind these lines",
not the size of the whole rotation set.
"""
chunks: list[list[str]] = []
paths: list[str] = []
total = 0
remaining = tail
candidates = [p for p in [base, *_rotated_log_paths(base)] if os.path.exists(p)]
for path in candidates:
if remaining <= 0:
break
try:
lines, count = _tail_file(path, remaining)
except FileNotFoundError:
# A rollover can rename a candidate between the existence check
# above and this open, and the handler holds no lock we can take
# from a route. Skip the vanished file rather than 500 the whole
# panel over one member of the set — the previous single-file
# version failed the request outright in the same situation.
#
# A roll landing mid-walk can also shift which chunk a file holds,
# so a tail taken at that instant may repeat or miss a block. The
# panel re-polls every 5s and the next read is clean; buying strict
# consistency here would mean reaching into logging's internals.
continue
except PermissionError as exc:
# Windows only, and only the sharing violation: the handler still
# holds the file it is rolling. Any other permission failure is a
# real misconfiguration and must not be hidden.
if os.name == "nt" and getattr(exc, "winerror", None) == 32:
continue
raise
if count == 0:
continue
chunks.append(lines)
paths.append(path)
total += count
remaining -= len(lines)
# Files were visited newest-first; the reader wants oldest-first.
out: list[str] = []
for chunk in reversed(chunks):
out.extend(chunk)
return out, total, list(reversed(paths))
def _tauri_plugin_log_candidates():
"""The `tauri-plugin-log` files — the shell's own log, and the only thing
the Tauri tab actually displays.
Split out from :func:`_tauri_log_candidates` so Clear can touch these and
leave the backend stdout/stderr redirect alone. See
:func:`clear_tauri_logs`.
"""
home = os.path.expanduser("~")
bid = "com.debpalash.omnivoice-studio"
if sys.platform == "darwin":
return [
os.path.join(home, "Library/Logs", bid, "tauri.log"),
os.path.join(home, "Library/Logs", bid, "VoiceStudio.log"),
]
if sys.platform.startswith("linux"):
data_dir = os.environ.get("XDG_DATA_HOME") or os.path.join(home, ".local/share")
return [
os.path.join(data_dir, bid, "logs", "tauri.log"),
os.path.join(home, ".config", bid, "logs", "tauri.log"),
]
if sys.platform.startswith("win"):
appdata = os.environ.get("APPDATA", home)
localappdata = os.environ.get("LOCALAPPDATA") or os.path.join(home, "AppData", "Local")
return [
os.path.join(localappdata, bid, "logs", "tauri.log"),
os.path.join(appdata, bid, "logs", "tauri.log"),
]
return []
def _backend_redirect_log_candidates():
"""`backend.log` / `backend_err.log` — the spawned backend's stdout and
stderr, written by `src-tauri/src/backend.rs::backend_log_path()`.
Deliberately NOT cleared by the Tauri tab's Clear button.
`open_err_log_for_run()` opens `backend_err.log` **append-only** so "a
respawn must not destroy the previous run's evidence" (#1510), rotates it
to `.1` rather than truncating, and its spawn diagnostics are described
there as "retained in backend_err.log across runs and lands verbatim in bug
reports". A native death (a Windows access violation, a SIGSEGV) writes
nothing to the Python log by construction, so this file is the only record
of it.
`OMNIVOICE_LOG_DIR` is honoured first, in the same precedence
`backend_log_path()` uses. The backend is a child of the shell, so an
ambient override reaches both and a resolver that ignored it would look
in the per-OS default while the writer wrote somewhere else, which is the
divergence class this file already has one of (see #1782).
"""
override = (os.environ.get("OMNIVOICE_LOG_DIR") or "").strip()
if override:
return [
os.path.join(override, "backend.log"),
os.path.join(override, "backend_err.log"),
]
home = os.path.expanduser("~")
if sys.platform == "darwin":
base = os.path.join(home, "Library/Logs/OmniVoice")
elif sys.platform.startswith("linux"):
state_dir = os.environ.get("XDG_STATE_HOME") or os.path.join(home, ".local/state")
base = os.path.join(state_dir, "OmniVoice")
elif sys.platform.startswith("win"):
localappdata = os.environ.get("LOCALAPPDATA") or os.path.join(home, "AppData", "Local")
base = os.path.join(localappdata, "OmniVoice", "Logs")
else:
return []
return [os.path.join(base, "backend.log"), os.path.join(base, "backend_err.log")]
def _tauri_log_candidates():
"""Likely paths for Tauri-side logs, most useful first.
@@ -581,9 +294,34 @@ def _tauri_log_candidates():
startup banners and hard-crash tracebacks land keep all three OS
shapes listed or sidecar crashes become invisible off-macOS.
"""
# Composed from the two halves so the read path keeps seeing every file
# while Clear can be narrowed to the shell's own log.
return _tauri_plugin_log_candidates() + _backend_redirect_log_candidates()
home = os.path.expanduser("~")
bid = "com.debpalash.omnivoice-studio"
if sys.platform == "darwin":
return [
os.path.join(home, "Library/Logs", bid, "tauri.log"),
os.path.join(home, "Library/Logs", bid, "OmniVoice Studio.log"),
os.path.join(home, "Library/Logs/OmniVoice/backend.log"),
os.path.join(home, "Library/Logs/OmniVoice/backend_err.log"),
]
if sys.platform.startswith("linux"):
data_dir = os.environ.get("XDG_DATA_HOME") or os.path.join(home, ".local/share")
state_dir = os.environ.get("XDG_STATE_HOME") or os.path.join(home, ".local/state")
return [
os.path.join(data_dir, bid, "logs", "tauri.log"),
os.path.join(home, ".config", bid, "logs", "tauri.log"),
os.path.join(state_dir, "OmniVoice", "backend.log"),
os.path.join(state_dir, "OmniVoice", "backend_err.log"),
]
if sys.platform.startswith("win"):
appdata = os.environ.get("APPDATA", home)
localappdata = os.environ.get("LOCALAPPDATA") or os.path.join(home, "AppData", "Local")
return [
os.path.join(localappdata, bid, "logs", "tauri.log"),
os.path.join(appdata, bid, "logs", "tauri.log"),
os.path.join(localappdata, "OmniVoice", "Logs", "backend.log"),
os.path.join(localappdata, "OmniVoice", "Logs", "backend_err.log"),
]
return []
@router.get("/system/logs")
@@ -598,24 +336,12 @@ async def system_logs(tail: int = 200):
except Exception:
tail = 200
if os.path.exists(LOG_PATH) or _rotated_log_paths(LOG_PATH):
base = LOG_PATH
else:
base = CRASH_LOG_PATH
if not os.path.exists(base) and not _rotated_log_paths(base):
path = LOG_PATH if os.path.exists(LOG_PATH) else CRASH_LOG_PATH
if not os.path.exists(path):
return {"lines": [], "path": LOG_PATH, "exists": False}
path = base
try:
lines, total, paths = await asyncio.to_thread(_tail_rolling, base, tail)
return {
"lines": lines,
"path": path,
"exists": True,
"total_lines": total,
# Which files the tail actually came from, oldest first. A bug
# report can then say whether it crossed a rollover.
"paths": paths,
}
lines, total = await asyncio.to_thread(_tail_file, path, tail)
return {"lines": lines, "path": path, "exists": True, "total_lines": total}
except Exception as e:
raise HTTPException(
status_code=500,
@@ -637,14 +363,7 @@ async def system_logs_tauri(tail: int = 200):
lines, total = await asyncio.to_thread(_tail_file, p, tail)
return {"lines": lines, "path": p, "exists": True, "total_lines": total}
except Exception as e:
error = public_failure(
logger,
"Could not read Tauri log",
e,
response="Could not read the Tauri log; check the backend log for details.",
traceback=True,
)
return {"lines": [], "path": p, "exists": True, "error": error}
return {"lines": [], "path": p, "exists": True, "error": str(e)}
return {"lines": [], "path": None, "exists": False, "candidates": candidates}
@@ -673,18 +392,13 @@ async def stream_logs(
if not path or not os.path.exists(path):
raise HTTPException(status_code=404, detail=f"Log file not found for source={source}")
try:
initial_position = os.path.getsize(path)
except OSError as exc:
logger.warning("Log stream could not determine its starting position")
raise HTTPException(
status_code=503,
detail="The log stream could not be started. Retry after checking file permissions.",
) from exc
async def _generate():
"""Yield SSE events whenever new lines appear in the log file."""
last_pos = initial_position
last_pos = 0
try:
last_pos = os.path.getsize(path)
except Exception:
pass
while True:
await asyncio.sleep(interval)
try:
@@ -721,23 +435,9 @@ def _read_from_pos(path: str, pos: int) -> list[str]:
@router.post("/system/logs/clear")
async def clear_system_logs():
"""Truncate the rolling runtime log and the crash log (what the Backend tab reads).
Includes the rotated siblings. Truncating only omnivoice.log left up to
6 MB in .1/.2/.3, so Clear freed almost nothing and now that the tail
reaches into those files would have looked like it did nothing at all.
"""
"""Truncate the rolling runtime log and the crash log (what the Backend tab reads)."""
cleared_any = False
# The full fixed name set rather than a snapshot of what exists: enumerating
# first leaves a window where a rollover creates a backup after the scan and
# its history survives a Clear that reported success. Names the handler can
# ever write are known up front, so there is nothing to enumerate.
targets = [
LOG_PATH,
*(f"{LOG_PATH}.{i}" for i in range(1, _LOG_BACKUP_COUNT + 1)),
CRASH_LOG_PATH,
]
for p in targets:
for p in (LOG_PATH, CRASH_LOG_PATH):
if os.path.exists(p):
try:
await asyncio.to_thread(_truncate_file, p)
@@ -753,12 +453,8 @@ async def clear_system_logs():
for key in ("crash_log_acked", "crash_log_acked_size"):
try:
prefs_delete(key)
except Exception as exc:
logger.warning("Cleared logs but could not reset crash acknowledgement state")
raise HTTPException(
status_code=500,
detail="Logs were cleared, but notification state could not be reset. Retry the clear operation.",
) from exc
except Exception:
pass
return {"cleared": cleared_any}
@@ -770,37 +466,20 @@ def _truncate_file(path: str):
@router.post("/system/logs/tauri/clear")
async def clear_tauri_logs():
"""Truncate the shell's own log files. OS-level rotation may recreate them.
The backend stdout/stderr redirect is deliberately excluded. This button
lives on a tab that shows `tauri.log`, and truncating `backend_err.log`
from it destroyed evidence the user was never shown the one record of a
native death, which writes nothing to the Python log. `backend.rs`'s
`open_err_log_for_run()` opens that file append-only precisely so "a
respawn must not destroy the previous run's evidence" (#1510) and rotates
it to `.1` instead of truncating, so it manages its own size and does not
need clearing from here.
"""
"""Truncate whichever Tauri-side log files we know about. OS-level rotation may recreate them."""
cleared = []
failed = 0
for p in _tauri_plugin_log_candidates():
for p in _tauri_log_candidates():
if os.path.exists(p):
try:
await asyncio.to_thread(_truncate_file, p)
cleared.append(p)
except OSError:
failed += 1
if failed:
raise HTTPException(
status_code=500,
detail="One or more desktop log files could not be cleared. Close any app using them and retry.",
)
return {"cleared": cleared, "failed": 0}
except Exception:
pass
return {"cleared": cleared}
@router.get("/sysinfo", response_model=SysinfoResponse)
def get_sys_info():
vram = 0.0
total_vram = 0.0
gpu_active = False
try:
@@ -813,38 +492,18 @@ def get_sys_info():
vram = alloc() / (1024**3)
elif _is_cuda:
vram = torch.cuda.memory_allocated() / (1024**3)
total_vram = torch.cuda.get_device_properties(torch.cuda.current_device()).total_memory / (1024**3)
elif _is_xpu:
vram = torch.xpu.memory_allocated() / (1024**3)
total_vram = float(
getattr(torch.xpu.get_device_properties(0), "total_memory", 0.0)
) / (1024**3)
except Exception:
pass
if vram > 0.01:
gpu_active = True
gpu_utilization = None
nvidia_stats = _nvidia_live_stats() if _is_cuda else None
if nvidia_stats:
gpu_utilization, vram, total_vram = nvidia_stats
gpu_active = gpu_active or gpu_utilization > 0 or vram > 0.01
vm = psutil.virtual_memory()
cpu_frequency = psutil.cpu_freq()
return {
"cpu": psutil.cpu_percent(interval=None),
"cpu_model": _CPU_MODEL,
"cpu_physical_cores": psutil.cpu_count(logical=False) or 0,
"cpu_logical_cores": psutil.cpu_count(logical=True) or 0,
"cpu_frequency_ghz": round((cpu_frequency.current if cpu_frequency else 0.0) / 1000, 2),
"ram": vm.used / (1024**3),
"total_ram": vm.total / (1024**3),
"gpu_name": _GPU_NAME,
"gpu_utilization": gpu_utilization,
"vram": round(vram, 2),
"total_vram": round(total_vram, 2),
"gpu_active": gpu_active
}
@@ -860,18 +519,11 @@ async def flush_memory(unload_model: bool = False):
freed_model = False
if unload_model:
from services import model_lifecycle
# The user-facing action has always promised "Unload all". Route it
# through the lifecycle facade so alternate TTS engines, dictation,
# diarisation, translation and sidecars are released as well as the
# shared OmniVoice model. Individual runtimes still decline while
# leased by active work.
released = await model_lifecycle.unload_all()
freed_model = any(
bool(result.get("success"))
for result in released.get("results", {}).values()
)
import services.model_manager as mm
async with mm._model_lock:
if mm.model is not None:
mm.model = None
freed_model = True
# Multi-pass GC to break reference cycles
gc.collect(generation=2)
@@ -880,25 +532,15 @@ async def flush_memory(unload_model: bool = False):
free_vram()
# Snapshot after flush. Two numbers, because one of them is a lie by
# omission: `memory_allocated` counts live tensors only, so it reads ~0
# after an unload while nvidia-smi still shows gigabytes — which is exactly
# the report we keep getting ("flush says it worked, the GPU says it
# didn't"). `memory_reserved` is what the caching allocator holds from the
# driver, and the gap between reserved and the driver's own figure is the
# CUDA context plus kernel workspaces, which no in-process call can return.
# Snapshot after flush
vram_after = 0.0
vram_reserved = 0.0
try:
if hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
driver = getattr(torch.mps, "driver_allocated_memory", None)
if driver:
vram_after = driver() / (1024**3)
current = getattr(torch.mps, "current_allocated_memory", None)
vram_reserved = (current() / (1024**3)) if current else vram_after
elif torch.cuda.is_available():
vram_after = torch.cuda.memory_allocated() / (1024**3)
vram_reserved = torch.cuda.memory_reserved() / (1024**3)
except Exception:
pass
@@ -909,7 +551,6 @@ async def flush_memory(unload_model: bool = False):
"unloaded_model": freed_model,
"ram_after": round(ram_after, 2),
"vram_after": round(vram_after, 2),
"vram_reserved": round(vram_reserved, 2),
}
@@ -1011,7 +652,7 @@ def system_notifications():
"id": "disk-low",
"level": "warn",
"title": f"Low disk space ({free_gb:.1f} GB free)",
"message": "VoiceStudio needs disk space for models, audio, and temp files.",
"message": "OmniVoice needs disk space for models, audio, and temp files.",
"action": None,
})
except Exception:
@@ -1041,7 +682,7 @@ def system_notifications():
from core import run_sentinel
rec = run_sentinel.newest_record()
if rec is not None and not rec[1] and run_sentinel.warrants_user_notice(rec[0]):
if rec is not None and not rec[1]:
record = rec[0]
last = record.get("last_activity") or {}
doing = f" Last activity: {last.get('kind')}." if last.get("kind") else ""
@@ -1064,7 +705,7 @@ def system_notifications():
},
})
except Exception:
logger.warning("Previous-run crash record could not be checked")
pass
# 5. A previous session logged a crash the user never saw.
# crash_log grew past the last acknowledged size AND predates this
@@ -1087,7 +728,7 @@ def system_notifications():
},
})
except Exception:
logger.warning("Previous-session crash log could not be checked")
pass
return {"notifications": notes, "count": len(notes)}
@@ -1165,6 +806,7 @@ async def ack_crash():
PERSISTENT_KEYS = {
"HTTP_PROXY", "HTTPS_PROXY", "ALL_PROXY",
"http_proxy", "https_proxy", "all_proxy",
"FFMPEG_PATH", "FFPROBE_PATH",
"TRANSLATE_BASE_URL", "TRANSLATE_API_KEY", "TRANSLATE_MODEL",
"DEEPL_API_KEY", "DEEPL_BASE_URL",
"MICROSOFT_API_KEY", "MICROSOFT_BASE_URL",
@@ -1172,14 +814,6 @@ PERSISTENT_KEYS = {
# the Rust sidecar reads OMNIVOICE_PORT at startup and the backend derives
# the LAN-share/UI ports from the others.
"OMNIVOICE_PORT", "OMNIVOICE_SHARE_PORT", "OMNIVOICE_UI_PORT",
# Per-job compute-time budgets (#1787). Both are captured at import time
# by services/model_manager.py (GPU_JOB_TIMEOUT_S / CPU_JOB_TIMEOUT_S), so
# a value saved here takes effect on the NEXT backend restart — same
# contract as OMNIVOICE_PORT above. Restored into os.environ during the
# "env_prefs" startup step (main.py), which runs before model_manager is
# first imported ("ml_imports"), so the restored value is what the module
# captures. The Settings UI must say so (RestartBadge).
"OMNIVOICE_GENERATE_TIMEOUT_S", "OMNIVOICE_CPU_GENERATE_TIMEOUT_S",
}
# Sidecar-engine install dirs (OMNIVOICE_INDEXTTS_DIR, …). The one-click
@@ -1197,16 +831,6 @@ except Exception: # pragma: no cover — defensive: env panel > installer wirin
# being set so a bad value never reaches uvicorn / the share listener.
_PORT_KEYS = {"OMNIVOICE_PORT", "OMNIVOICE_SHARE_PORT", "OMNIVOICE_UI_PORT"}
# Keys whose value is a wall-clock compute-time budget in seconds (#1787).
# Validated the same way as _PORT_KEYS: reject anything that isn't a
# positive number before it reaches services/model_manager.py. Upper bound is
# generous — long enough that a legitimate multi-hour, audiobook-length CPU
# render is never blocked — but still bounded, so a fat-fingered extra digit
# (300 -> 3000000) can't turn a wedged job into one that silently occupies a
# worker for days before the guard ever fires.
_TIMEOUT_KEYS = {"OMNIVOICE_GENERATE_TIMEOUT_S", "OMNIVOICE_CPU_GENERATE_TIMEOUT_S"}
_MAX_GENERATE_TIMEOUT_S = 21600.0 # 6 hours
@router.post("/system/set-env")
async def set_env_var(body: dict):
@@ -1215,11 +839,11 @@ async def set_env_var(body: dict):
Persistent keys (proxy, FFMPEG_PATH, translation provider keys, ) are
saved to ``prefs.json`` so they survive backend restarts (restored at
startup in ``main.py``). HF_TOKEN is persisted via
``huggingface_hub.login()`` (and cleared with the shared token-file helper). Other keys
``huggingface_hub.login()`` (and cleared via ``logout()``). Other keys
are set on ``os.environ`` for the running process.
The loopback-origin gate that previously lived inline here is now applied
at the router level via `dependencies=[Depends(require_admin)]` on
at the router level via `dependencies=[Depends(require_loopback)]` on
`router` see the top of this file. Every route on this router is
gated, including this one. The 403 body and behavior are unchanged.
"""
@@ -1234,6 +858,23 @@ async def set_env_var(body: dict):
)
if value:
# Validate executable paths if the user is setting them manually.
# Reject control characters / null bytes (defense-in-depth against
# path-injection), then require an existing regular file. NOTE: this
# endpoint is loopback-only and MUST remain so — a remote caller able
# to set FFMPEG_PATH/FFPROBE_PATH could point it at an arbitrary
# binary (RCE). Network sharing must never expose /system/set-env.
if key in ("FFMPEG_PATH", "FFPROBE_PATH"):
if any(ord(c) < 0x20 or ord(c) == 0x7F for c in value):
raise HTTPException(
status_code=400,
detail="Invalid path: control characters are not allowed",
)
if not os.path.isfile(value):
raise HTTPException(
status_code=400,
detail=f"File not found: {value}",
)
# Port keys must be a numeric string in the unprivileged range so a
# typo can't drop the backend onto a privileged port (<1024) or an
# out-of-range value uvicorn would reject at bind time.
@@ -1250,24 +891,8 @@ async def set_env_var(body: dict):
status_code=400,
detail=f"Invalid port for {key}: must be between 1024 and 65535.",
)
if key in _TIMEOUT_KEYS:
try:
timeout_n = float(value)
except (TypeError, ValueError):
raise HTTPException(
status_code=400,
detail=f"Invalid timeout for {key}: '{value}' is not a number.",
)
if not (0 < timeout_n <= _MAX_GENERATE_TIMEOUT_S):
raise HTTPException(
status_code=400,
detail=(
f"Invalid timeout for {key}: must be greater than 0 "
f"and at most {_MAX_GENERATE_TIMEOUT_S:.0f} seconds."
),
)
os.environ[key] = value
logger.info("Environment variable set (length=%d)", len(value))
logger.info("Set environment variable: %s (length=%d)", key, len(value))
# Capability 1 / issue #35: HF_TOKEN persists across restarts via
# huggingface_hub.login() — writes the token to $HF_HOME/token so
@@ -1282,22 +907,22 @@ async def set_env_var(body: dict):
# Non-fatal — the runtime env var is still set, so the
# current process will still see the token. We just lose
# persistence across restarts.
logger.warning("Could not persist HF token to disk: %s", log_safe(e))
logger.warning("Could not persist HF token to disk: %s", e)
else:
os.environ.pop(key, None)
logger.info("Environment variable cleared")
logger.info("Cleared environment variable: %s", key)
# Mirror the persistence on clear — wipe the saved token file too.
if key == "HF_TOKEN":
try:
from services.token_resolver import clear_hf_cli_tokens
clear_hf_cli_tokens()
logger.info("Local Hugging Face token files cleared")
except Exception:
raise HTTPException(status_code=500, detail="Could not clear local Hugging Face token files") from None
from huggingface_hub import logout as _hf_logout
_hf_logout()
logger.info("HF token cleared from $HF_HOME/token via logout()")
except Exception as e:
logger.warning("Could not clear HF token file: %s", e)
# HF_TOKEN persistence is handled above via huggingface_hub.login()/
# clear_hf_cli_tokens() — it never touches prefs.json. Everything else in
# logout() — it never touches prefs.json. Everything else in
# PERSISTENT_KEYS (proxy, FFMPEG_PATH, translation provider keys, …) is
# saved to prefs.json so it survives backend restarts (restored at
# startup in main.py). Non-persistent keys stay process-local.
@@ -1308,13 +933,7 @@ async def set_env_var(body: dict):
else:
prefs_delete(prefs_key)
# #1787 review fix: tell the caller up front when the value just saved is
# being shadowed by an external env var — set at THIS process's startup,
# before our own prefs restore ran, so it predicts the next restart too.
# A response that just said {"set": True} let the Settings panel promise
# a restart would apply a value that never will.
from core.prefs import is_env_shadowed
return {"key": key, "set": bool(value), "shadowed": is_env_shadowed(key)}
return {"key": key, "set": bool(value)}
@router.post("/clean-audio")
@@ -1366,26 +985,18 @@ async def _do_clean_audio(audio, tmp_dir, clean_id):
clean_filename = f"mic_{clean_id}.wav"
final_path = os.path.join(OUTPUTS_DIR, clean_filename)
conversion_fallback = False
try:
rc, _, _ = await run_ffmpeg(
await run_ffmpeg(
[ffmpeg, "-y", "-i", clean_path, "-ar", "24000", "-ac", "1", final_path],
timeout=120.0,
)
conversion_fallback = rc != 0
except asyncio.TimeoutError:
conversion_fallback = True
logger.warning("Final clean-audio conversion timed out; returning the cleaned source format")
if conversion_fallback:
shutil.copy2(clean_path, final_path)
elif not os.path.exists(final_path):
pass
if not os.path.exists(final_path):
shutil.copy2(clean_path, final_path)
headers = {"X-Clean-Filename": clean_filename}
if conversion_fallback:
headers["X-Clean-Conversion"] = "fallback"
return FileResponse(final_path, media_type="audio/wav", filename=clean_filename,
headers=headers)
headers={"X-Clean-Filename": clean_filename})
@router.get("/system/asr-backends")
@@ -1405,7 +1016,7 @@ def asr_backends():
def hf_token_state():
"""Return the 3-source HF token cascade state for the Settings UI
(Wave 2 React panel consumes this). Never returns the raw token
only a masked preview and local presence; no outbound validation.
only a masked preview, whoami username, and per-source validity.
"""
from dataclasses import asdict
from services import token_resolver
@@ -1453,10 +1064,7 @@ async def diagnostic_bundle(network: bool = Query(False, description="Include th
# ── Self-check diagnostics ────────────────────────────────────────────────
@router.get(
"/system/diagnose",
dependencies=[Depends(require_admin_action)],
)
@router.get("/system/diagnose")
async def system_diagnose(
network: bool = Query(True, description="Include the HuggingFace hub reachability probe"),
deep: bool = Query(False, description="Also load the active engine and synthesize a short utterance (may cold-load the model — minutes on first run)"),
@@ -1493,21 +1101,12 @@ def quarantine_status():
# ── Network sharing (loopback-only control surface) ──────────────────────────
@router.get("/system/network/state")
async def network_state(request: Request):
async def network_state():
st = network_share.get_state()
# PIN-only server mode permits unauthenticated read-only discovery, but the
# PIN is itself a consumption credential. Reveal it only to the native
# loopback UI or to a remote caller that already passed the configured
# long API-key gate. The boolean lets headless dashboards remain useful.
host = request.client.host if request.client else None
may_reveal_pin = is_loopback(host) or bool(
os.environ.get("OMNIVOICE_API_KEY", "").strip()
)
return {
"enabled": st.enabled,
"share_port": st.share_port,
"pin": st.pin if may_reveal_pin else None,
"pin_required": bool(st.pin),
"pin": st.pin,
"lan_addresses": st.lan_addresses,
}
@@ -1538,16 +1137,7 @@ async def tailscale_status():
@router.post("/system/tailscale/enable")
async def tailscale_enable():
result = _tailscale.serve_enable()
if result.get("ok"):
return result
error = public_failure(
logger,
"Tailscale serve failed",
result.get("error", "unknown error"),
response="Tailscale sharing could not be enabled; check the backend log for details.",
)
return {"ok": False, "error": error}
return _tailscale.serve_enable()
@router.post("/system/tailscale/disable")
+8 -19
View File
@@ -21,16 +21,13 @@ import asyncio
import json
import logging
import os
import re
from typing import Optional
from fastapi import APIRouter, Depends, HTTPException
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel, Field
from services import director, speech_rate, incremental
from services.ffmpeg_utils import find_ffprobe, spawn_subprocess
from api.dependencies import require_native_access
from core.path_security import UnsafePath, resolve_within
logger = logging.getLogger("omnivoice.tools")
router = APIRouter()
@@ -43,7 +40,7 @@ class ProbeReq(BaseModel):
path: str
@router.post("/tools/probe", dependencies=[Depends(require_native_access)])
@router.post("/tools/probe")
async def probe(req: ProbeReq):
target = os.path.realpath(os.path.expanduser(req.path))
if not os.path.exists(target):
@@ -177,26 +174,18 @@ async def analyse_video_context(job_id: str):
from core.config import DUB_DIR
from services.video_context import analyse_video
if not re.fullmatch(r"[A-Za-z0-9_-]{1,64}", job_id or ""):
raise HTTPException(status_code=400, detail="Invalid job id")
try:
job_dir = resolve_within(DUB_DIR, job_id)
except UnsafePath as exc:
raise HTTPException(status_code=400, detail="Invalid job id") from exc
job = _get_job(job_id)
if not job:
from fastapi import HTTPException
raise HTTPException(status_code=404, detail="Job not found")
video_path = resolve_within(DUB_DIR, job_dir / "source.mp4")
if not video_path.is_file():
try:
video_path = resolve_within(DUB_DIR, job.get("video_path", ""))
except UnsafePath:
return {"error": "Source video not found", "segments": {}}
video_path = os.path.join(DUB_DIR, job_id, "source.mp4")
if not os.path.exists(video_path):
video_path = job.get("video_path", "")
if not video_path.is_file():
if not video_path or not os.path.exists(video_path):
return {"error": "Source video not found", "segments": {}}
segments = job.get("segments") or []
ctx = await analyse_video(str(video_path), segments)
ctx = await analyse_video(video_path, segments)
return ctx.to_dict()
+23 -134
View File
@@ -10,8 +10,7 @@ as they're generated. This unlocks:
Protocol:
Client sends JSON: {"text": "...", "voice": "profile_id", ...}
Server sends binary audio chunks (PCM16 @ 24kHz mono) as generated
Server sends JSON: {"type": "done", "duration_s": 4.2,
"gen_time_s": 1.1, "ttfa_ms": 180.0, "rtf": 0.262}
Server sends JSON: {"type": "done", "duration_s": 4.2, "gen_time_s": 1.1}
Server sends JSON: {"type": "error", "detail": "..."}
The chunked delivery targets <100ms time-to-first-audio (TTFA) on warm models.
@@ -34,30 +33,6 @@ logger = logging.getLogger("omnivoice.tts_stream")
# Smaller chunks = lower latency but more WebSocket overhead.
CHUNK_SAMPLES = int(os.environ.get("OMNIVOICE_STREAM_CHUNK", "4800"))
# Module seam for deterministic latency-contract tests. Keep every timing
# sample on the same monotonic clock.
_perf_counter = time.perf_counter
async def _resolve_stream_backend(engine_id: str | None):
"""Resolve the live-stream engine without bypassing host isolation."""
from services.tts_backend import (
OmniVoiceBackend,
active_backend_id,
get_active_tts_backend,
get_backend_class,
)
if engine_id:
return get_backend_class(engine_id)()
cls = get_backend_class(active_backend_id())
if cls is OmniVoiceBackend:
from services.model_manager import get_model
return get_active_tts_backend(model=await get_model())
return get_active_tts_backend()
class StreamTTSRequest(BaseModel):
"""Client request for streaming TTS."""
@@ -87,11 +62,6 @@ async def ws_tts(websocket: WebSocket):
await websocket.accept()
logger.info("TTS streaming WebSocket connected")
# Said once per socket, not once per utterance: a conversational client
# sends many requests down one connection and a repeated notice would be
# noise. See `_announce_local_only`.
announced_local_only = False
try:
while True:
# Wait for a text request from the client
@@ -110,78 +80,34 @@ async def ws_tts(websocket: WebSocket):
})
continue
t0 = _perf_counter()
t0 = time.perf_counter()
text = data["text"]
# Remote GPU: this socket stays on this machine, and says so.
#
# /generate's port trades progressive playback for the remote
# render — the classic path was always a single wait, so spending
# it on a faster GPU is a straight win. This route is the opposite
# shape: it exists to put audio in the user's ear before the
# sentence has finished synthesizing, and sending each utterance to
# a worker would pay queue admission, a round trip and cold-load
# risk per utterance, for the one surface where latency IS the
# feature.
#
# Silence would be worse than the limitation: the header badge
# would read "gpu2" while this machine does 100% of the work, the
# same class of lie the op-aware picker exists to stop. Said once
# per socket — a conversational client sends many requests down one
# connection — and BEFORE engine resolution, so an engine that
# cannot load still tells the user where it would have run.
if not announced_local_only:
announced_local_only = True
try:
from worker import routing as worker_routing
target = worker_routing.decide(op="tts")
except Exception: # noqa: BLE001 — advisory; never break audio
target = None
if target is not None and target.remote:
from core.scrub import scrub_text as _scrub
await websocket.send_json({
"type": "routing",
"status": "local_stream",
"reason": _scrub(
f"{target.label} is your GPU target, but live "
f"streaming runs on this machine"
),
})
try:
# Resolve engine
from services.tts_backend import (
get_active_tts_backend,
get_backend_class,
)
engine_id = data.get("engine")
# #1224: leave a breadcrumb when memory is already tight before
# a heavy load. /generate has done this since the 16 GB-Mac
# reports, but the streaming path — which the desktop UI tries
# FIRST — never did, so the load most likely to tip the machine
# into an OS OOM kill was the one load with no trail. The
# captured stderr tail is what a SIGKILL report has to go on.
# Advisory only: the OS can reclaim cache, and refusing here
# would brick loads that would actually have coped.
try:
from services.memory_budget import log_if_low
log_if_low(f"TTS stream load ({engine_id or 'active engine'})")
except Exception:
pass
backend = await _resolve_stream_backend(engine_id)
if engine_id:
cls = get_backend_class(engine_id)
backend = cls()
else:
from services.model_manager import get_model
model = await get_model()
backend = get_active_tts_backend(model=model)
# ── Routing gate (#21 — no silent CPU fallback). WebSockets have
# no response headers, so this uses frames: an error frame +
# close on `unavailable`, a one-time `routing` frame on
# cpu_fallback / accelerated-with-caveat (before any audio).
from core.device_caps import detect_host_caps
from services.engine_routing import (
routing_notice,
runtime_compute_profile_async,
)
from services.engine_routing import resolve_routing, routing_notice
from core.scrub import scrub_text
_routing = await runtime_compute_profile_async(
backend, detect_host_caps()
)
_routing = resolve_routing(
getattr(backend, "gpu_compat", ("cpu",)), detect_host_caps(),
getattr(backend, "min_vram_gb", 0.0))
if _routing["routing_status"] == "unavailable":
await websocket.send_json({
"type": "error",
@@ -276,11 +202,6 @@ async def ws_tts(websocket: WebSocket):
from services.model_manager import run_on_gpu_pool_guarded
def _generate(sentence_text):
# Timed INSIDE the pool worker: the guarded dispatch below
# can queue behind other jobs, and queue wait is not
# synthesis (review on #1620) — under contention it would
# inflate rtf without the engine slowing at all.
_synth_t0 = _perf_counter()
from services.audio_dsp import apply_mastering, normalize_audio
from services.watermark import mark_synthetic
wav = backend.generate(sentence_text, **kw)
@@ -302,19 +223,12 @@ async def ws_tts(websocket: WebSocket):
# watermark._iter_chunks), which is inherent to marking
# ultra-short clips, not a coverage gap.
wav = mark_synthetic(wav, sr_actual, context="tts_stream.sentence")
return wav, sr_actual, _perf_counter() - _synth_t0
return wav, sr_actual
import torch
total_samples = 0
sr = backend.sample_rate
started = False
first_audio_at: float | None = None
# Synthesis time only. The wall clock below also carries socket
# delivery and the per-chunk event-loop yields, so deriving RTF
# from it reports "how slow was the client" as if it were engine
# throughput — on a slow consumer that inflates RTF without the
# engine having changed at all.
synth_time = 0.0
for sentence in sentences:
# Bounded + pool-reset on hang so a wedged generate can't
@@ -324,12 +238,11 @@ async def ws_tts(websocket: WebSocket):
# Length-scaled budget per sentence (#1190) — the flat 300s
# default is gone from every dispatch.
from services.model_manager import generate_timeout_s
wav_tensor, sr, sentence_synth_s = await run_on_gpu_pool_guarded(
wav_tensor, sr = await run_on_gpu_pool_guarded(
functools.partial(_generate, sentence),
what="TTS generate",
timeout=generate_timeout_s(sentence, engine=backend),
timeout=generate_timeout_s(sentence),
)
synth_time += sentence_synth_s
if not started:
# Send metadata after the first generation so
@@ -356,49 +269,25 @@ async def ws_tts(websocket: WebSocket):
end = min(sent_samples + CHUNK_SAMPLES, n_samples)
chunk = pcm_bytes[sent_samples * 2: end * 2]
await websocket.send_bytes(chunk)
if first_audio_at is None:
# TTFA ends when the first audio bytes have been
# handed to the socket. The previous log used the
# whole-render duration and called it TTFA.
first_audio_at = _perf_counter()
sent_samples = end
# Yield to event loop between chunks for responsiveness
await asyncio.sleep(0)
total_samples += n_samples
finished_at = _perf_counter()
wall_time_raw = max(0.0, finished_at - t0)
synth_time_raw = max(0.0, synth_time)
gen_time = round(wall_time_raw, 3)
gen_time = round(time.perf_counter() - t0, 3)
duration = round(total_samples / sr, 3)
ttfa_ms = (
round(max(0.0, first_audio_at - t0) * 1000.0, 1)
if first_audio_at is not None
else None
)
# RTF is a render metric: synthesis seconds per audio second.
rtf = (
round(synth_time_raw / (total_samples / sr), 3)
if total_samples > 0
else None
)
await websocket.send_json({
"type": "done",
"duration_s": duration,
"gen_time_s": gen_time,
"ttfa_ms": ttfa_ms,
"rtf": rtf,
"samples": total_samples,
"sample_rate": sr,
"engine": backend.id,
})
logger.info(
"TTS stream: %.1fs audio in %.1fs (TTFA=%s, RTF=%s)",
duration,
gen_time,
f"{ttfa_ms:.0f}ms" if ttfa_ms is not None else "n/a",
f"{rtf:.3f}" if rtf is not None else "n/a",
"TTS stream: %.1fs audio in %.1fs (TTFA=%.0fms)",
duration, gen_time, gen_time * 1000,
)
except Exception as e:
-397
View File
@@ -1,397 +0,0 @@
"""Speech-to-speech voice changer — Studio's Convert method (POST /convert).
The user drops (or records) a source clip, picks an existing voice profile,
and gets the same words back in that profile's voice: the active ASR backend
transcribes the clip (no word timestamps the text is all we need), the
active TTS engine re-synthesizes it conditioned on the profile's reference
audio, and by default the take is pitch-preservingly time-stretched
(ffmpeg atempo, clamped to one well-behaved 0.52.0 stage) so it lands near
the source clip's duration.
Deliberately reuses the /generate choke points instead of re-deriving them:
* profile row conditioning via ``generation._resolve_profile_conditioning``
(lock wins, ``kind`` authoritative, #533 language fill),
* engine resolution via ``services.tts_backend.resolve_generation_backend``
(never a silent OmniVoice fallback; ``require_cloning=True`` refuses
clone-less engines with the actionable switch-engine message),
* synthesis via ``generation._run_backend_inference`` on the guarded GPU
pool (#730 bound + reset; busy/timeout → retryable 503),
* provenance + persistence via ``services.watermark.mark_synthetic_async``
and ``generation._finalize_generation`` (watermark WAV in OUTPUTS_DIR
history row retention prune), marked AFTER the stretch so the take users
keep carries exactly one whole-take mark.
Local-first: no network calls; ASR-model-less installs get the same typed
409 download CTA as /transcribe; a backend mid-shutdown surfaces the global
503 ``[shutting_down]`` (ModelLoadInterruptedByShutdown main.py handler).
Reachability matches /generate: loopback bind by default, with the shared
network-share PIN / API-key middleware gating any non-loopback exposure.
"""
from __future__ import annotations
import asyncio
import functools
import logging
import os
import tempfile
import time
from fastapi import APIRouter, File, Form, HTTPException, UploadFile
router = APIRouter()
logger = logging.getLogger("omnivoice.convert")
#: ffmpeg's atempo filter is well-behaved in [0.5, 2.0] per stage. Convert
#: clamps to ONE stage by design: needing more than 2× either way means the
#: synthesized speech differs so much from the source that "matching" it
#: would produce chipmunk/slow-motion artifacts worse than the mismatch.
ATEMPO_MIN = 0.5
ATEMPO_MAX = 2.0
#: Within this relative tolerance the durations already match — stretching
#: would resample the whole take for an inaudible gain.
_MATCH_TOLERANCE = 0.02
#: Convert clips are short conversational inputs, not long-form media. Stream
#: them to disk in bounded chunks so a network-share client cannot make the
#: backend materialize an arbitrarily large multipart upload in memory.
_MAX_SOURCE_AUDIO_BYTES = 64 * 1024 * 1024
_UPLOAD_CHUNK_BYTES = 1024 * 1024
async def _copy_source_upload(audio: UploadFile, destination) -> int:
"""Stream ``audio`` into ``destination`` with the Convert upload cap."""
total = 0
while True:
chunk = await audio.read(_UPLOAD_CHUNK_BYTES)
if not chunk:
return total
total += len(chunk)
if total > _MAX_SOURCE_AUDIO_BYTES:
raise HTTPException(
status_code=413,
detail="Source audio is too large (maximum 64 MB).",
)
destination.write(chunk)
def _clamped_tempo_ratio(tts_duration_s: float, source_duration_s: float) -> "float | None":
"""The atempo ratio that fits the take into the source duration, or None.
ratio > 1 speeds the take up (it came out longer than the source),
ratio < 1 slows it down. Clamped to a single atempo stage's [0.5, 2.0];
None when either duration is unusable or they already match.
"""
if not source_duration_s or source_duration_s <= 0:
return None
if not tts_duration_s or tts_duration_s <= 0:
return None
ratio = tts_duration_s / source_duration_s
if abs(ratio - 1.0) <= _MATCH_TOLERANCE:
return None
return min(ATEMPO_MAX, max(ATEMPO_MIN, ratio))
async def _match_source_duration(audio_tensor, sample_rate: int, source_duration_s: float):
"""Best-effort pitch-preserving stretch of the take toward the source
clip's duration. Returns the input unchanged when no stretch is needed
or ffmpeg fails a duration mismatch is better than a failed convert."""
n_samples = int(audio_tensor.shape[-1])
ratio = _clamped_tempo_ratio(n_samples / sample_rate, source_duration_s)
if ratio is None:
return audio_tensor
target_samples = max(1, int(round(n_samples / ratio)))
from services.ffmpeg_utils import _pitch_preserving_stretch
try:
return await _pitch_preserving_stretch(audio_tensor, target_samples, sample_rate)
except Exception as e: # noqa: BLE001 — stretch is opt-in polish, never fatal
logger.warning("duration match skipped — atempo stretch failed: %s", e)
return audio_tensor
async def _transcribe_source(tmp_path: str, *, source_lease=None) -> dict:
"""Active-ASR transcription of the uploaded clip (no word timestamps).
Mirrors POST /transcribe: typed 409 + download CTA before any backend
is constructed (never a silent multi-GB auto-download), the guarded GPU
pool dispatch (#730), 504 on timeout, and the same 409 when the loader
degrades onto an engine with no weights on disk (#1185).
"""
from services.asr_backend import (
ASRModelMissingError,
ASRTimeoutError,
asr_model_missing_detail,
asr_model_missing_error,
run_transcribe_guarded,
)
missing = await asyncio.to_thread(asr_model_missing_error, purpose="transcribe")
if missing is not None:
raise HTTPException(
status_code=409,
detail={**missing, "message": asr_model_missing_detail(missing)},
)
def _run():
# `load_*`, not `get_*`: the loader runs ensure_loaded() and degrades
# past an engine whose deep import chain is broken (#1185).
from services.asr_backend import load_active_asr_backend
backend = load_active_asr_backend()
return backend.transcribe(tmp_path, word_timestamps=False)
from services.model_manager import _gpu_pool
release = source_lease.acquire() if source_lease is not None else None
abandoned = False
try:
return await run_transcribe_guarded(
_gpu_pool,
_run,
what="Voice convert",
on_abandon=release,
)
except asyncio.CancelledError:
# The guard now owns the lease token until the native worker drains.
abandoned = True
raise
except ASRTimeoutError as e:
abandoned = True
logger.warning("Convert transcription timed out: %s", e)
raise HTTPException(status_code=504, detail=str(e))
except ASRModelMissingError as e:
raise HTTPException(
status_code=409,
detail={**e.payload, "message": asr_model_missing_detail(e.payload)},
)
finally:
if release is not None and not abandoned:
release()
@router.post("/convert")
async def convert_speech(
audio: UploadFile = File(...),
profile_id: str = Form(...),
match_duration: bool = Form(True),
):
"""Convert a spoken clip into an existing voice profile's voice.
Multipart form: ``audio`` (the source clip), ``profile_id`` (an existing
voice profile), optional ``match_duration`` (default on atempo the take
toward the source clip's length, clamped to 0.52.0×).
Returns JSON ``{audio_url, text, duration_s, id}`` the take is saved to
OUTPUTS_DIR and served from the ``/audio`` mount like every other take.
"""
from core.db import db_conn
from api.routers.generation import _resolve_profile_conditioning, _TempReferenceLease
# ── Profile first: strict 404, unlike /generate's silent skip — Convert
# has no meaning without a target voice.
with db_conn() as conn:
row = conn.execute(
"SELECT * FROM voice_profiles WHERE id=?", (profile_id,)
).fetchone()
if not row:
raise HTTPException(
status_code=404,
detail="That voice profile doesn't exist. It may have been deleted from another tab.",
)
cond = _resolve_profile_conditioning(row)
# ── Save the upload before loading an engine. Every ASR backend (and
# ffprobe) needs a file path; the bounded streaming copy rejects oversized
# network-share requests without materializing them in process memory or
# starting heavyweight model work.
ext = os.path.splitext(audio.filename or "audio.wav")[1] or ".wav"
tmp = tempfile.NamedTemporaryFile(delete=False, suffix=ext)
source_lease = None
try:
try:
await _copy_source_upload(audio, tmp)
finally:
tmp.close()
source_lease = _TempReferenceLease(tmp.name)
# ── Engine gate before ASR/TTS work: the shared resolver refuses a
# clone-less engine with the actionable switch-engine message (→ 400),
# and a backend mid-shutdown raises ModelLoadInterruptedByShutdown out
# of the model load → the global 503 [shutting_down] handler.
from services.tts_backend import active_backend_id, resolve_generation_backend
try:
backend = await resolve_generation_backend(
require_cloning=True, cloning_purpose="voice conversion",
)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
result = await _transcribe_source(tmp.name, source_lease=source_lease)
segments = result.get("segments", [])
text = result.get("text", "")
if not text and segments:
text = " ".join(s.get("text", "") for s in segments).strip()
# Same final-text hygiene as /transcribe: strip Whisper hallucination
# loops, then deterministic polish (leading capital + terminal
# punctuation) so the TTS input reads as typed text.
from services.refinement import collapse_repetitive_artifacts
from services.text_polish import polish_text
text = polish_text(collapse_repetitive_artifacts(text))
if not text or not text.strip():
raise HTTPException(
status_code=422,
detail=(
"No speech was recognized in the source clip, so there is "
"nothing to convert. Record or drop a clip with clear, "
"audible speech and try again."
),
)
# #308/#1032 parity with /generate: a clone profile saved without a
# transcript conditions better when its reference clip is transcribed,
# and that transcript is cached onto the row so it happens ONCE, not
# per convert. Best-effort exactly like /generate — a timeout/failure
# degrades to ref_text=None and the engine's own fallback. The ASR
# model is already warm here (the source transcribe above just used it).
if cond["ref_audio_path"] and not cond["ref_text"]:
from api.routers.generation import (
_generate_timeout_s,
_persist_profile_ref_text,
)
from services.asr_backend import transcribe_reference
from services.model_manager import run_on_gpu_pool_guarded
try:
cond["ref_text"] = await run_on_gpu_pool_guarded(
functools.partial(transcribe_reference, cond["ref_audio_path"]),
what="Reference transcribe",
timeout=_generate_timeout_s(""),
)
except TimeoutError as e:
logger.warning(
"reference transcribe hung (%s); using engine ASR fallback", e,
)
cond["ref_text"] = None
if cond["ref_text"] and cond["persist_ref_text"]:
_persist_profile_ref_text(profile_id, cond["ref_text"])
# Source duration for the optional match: the container's own length
# (ffprobe), falling back to the last ASR segment end. Best-effort —
# None just skips the stretch.
source_duration_s = None
if match_duration:
from services.ffmpeg_utils import probe_duration
source_duration_s = await probe_duration(
tmp.name, allowed_root=os.path.dirname(tmp.name),
)
if not source_duration_s and segments:
source_duration_s = max((s.get("end", 0) or 0) for s in segments) or None
# ── Same text choke point as /generate: engine-agnostic normalization
# (numbers→words, junk strip) on the fully resolved language.
from services.text_normalization import normalize_for_tts
language = cond["language"]
text = normalize_for_tts(text, language)
used_seed = cond["seed"]
if used_seed is None:
import random
used_seed = random.randint(0, 2**31 - 1)
from api.routers.generation import (
_finalize_generation,
_generate_timeout_s,
_run_backend_inference,
)
from services.model_manager import (
GpuJobTimeoutError,
GpuPoolBusyError,
run_on_gpu_pool_guarded,
)
from core.device_caps import detect_host_caps
from services.engine_routing import runtime_compute_profile_async
compute_profile = await runtime_compute_profile_async(
backend, detect_host_caps()
)
if compute_profile["routing_status"] == "unavailable":
raise HTTPException(
status_code=400,
detail=compute_profile["routing_reason"],
)
start_time = time.time()
from services.performance_profiles import tts_defaults
_profile_defaults = tts_defaults(active_backend_id())
_render = functools.partial(
_run_backend_inference,
backend, text, language, cond["ref_audio_path"], cond["ref_text"],
cond["instruct"],
None, # duration — the model picks; match_duration owns pacing
_profile_defaults.get("num_step", 16), 2.0,
1.0, # speed
True, _profile_defaults.get("postprocess_output", True),
used_seed,
)
try:
audio_tensor = await run_on_gpu_pool_guarded(
_render,
what="Voice convert",
timeout=_generate_timeout_s(
text,
engine=backend,
execution_device=compute_profile["effective_device"],
min_vram_gb=compute_profile["min_vram_gb"],
hardware_family=compute_profile.get("runtime_hardware_family"),
vram_gb=compute_profile.get("runtime_vram_gb"),
),
min_vram_gb=compute_profile["min_vram_gb"],
)
except GpuPoolBusyError as e:
raise HTTPException(
status_code=503, detail=str(e),
headers={"Retry-After": str(e.retry_after),
"X-OmniVoice-Retryable": "true"},
) from e
except GpuJobTimeoutError as e:
raise HTTPException(
status_code=503, detail=str(e),
headers={"Retry-After": "30", "X-OmniVoice-Retryable": "true"},
) from e
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e)) from e
sample_rate = backend.sample_rate
if match_duration and source_duration_s:
audio_tensor = await _match_source_duration(
audio_tensor, sample_rate, source_duration_s,
)
# Provenance mark AFTER the stretch (one whole-take mark on the audio
# the user actually keeps), then the shared finalize tail — WAV in
# OUTPUTS_DIR, self-healing history row, retention prune, event emit.
from services.watermark import mark_synthetic_async
audio_tensor = await mark_synthetic_async(
audio_tensor, sample_rate, context="convert.finalize",
)
_, meta = await _finalize_generation(
audio_tensor, sample_rate, text=text, history_mode="convert",
ref_audio_path=cond["ref_audio_path"], language=language,
instruct=cond["instruct"], resolved_profile_id=profile_id,
used_seed=used_seed, start_time=start_time,
already_marked=True,
)
return {
"id": meta["id"],
"audio_url": f"/audio/{meta['filename']}",
"text": text,
"duration_s": meta["duration"],
"gen_time_s": meta["gen_time"],
}
finally:
if source_lease is not None:
source_lease.finish_request()
else:
try:
os.unlink(tmp.name)
except OSError:
pass
+4 -11
View File
@@ -1,5 +1,5 @@
"""
Watermark detection API upload audio, check if it was generated by VoiceStudio.
Watermark detection API upload audio, check if it was generated by OmniVoice.
"""
import os
import tempfile
@@ -9,7 +9,6 @@ from fastapi import APIRouter, UploadFile, File, HTTPException
from services.watermark import detect_watermark, is_enabled, _check_available
from core.prefs import get as pref_get, set_ as pref_set
from core.public_errors import public_failure
logger = logging.getLogger("omnivoice.watermark_api")
@@ -19,7 +18,7 @@ router = APIRouter()
@router.post("/watermark/detect")
async def detect_audio_watermark(file: UploadFile = File(...)):
"""
Upload an audio file and check whether it contains a VoiceStudio watermark.
Upload an audio file and check whether it contains an OmniVoice watermark.
Returns confidence score, decoded message, and source attribution.
"""
@@ -50,14 +49,8 @@ async def detect_audio_watermark(file: UploadFile = File(...)):
return result
except Exception as e:
detail = public_failure(
logger,
"Watermark detection failed",
e,
response="Watermark detection failed; check the backend log for details.",
traceback=True,
)
raise HTTPException(status_code=500, detail=detail) from e
logger.exception("Watermark detection failed")
raise HTTPException(status_code=500, detail=str(e))
finally:
try:
os.unlink(tmp_path)
-831
View File
@@ -1,831 +0,0 @@
"""Remote worker management API.
Deliberately small. The council's warning about the original design was that
seven strategies times three execution modes times priorities times weights
times per-model concurrency is a configuration surface nobody can test and
every knob is a compatibility promise forever. So this exposes what a user
actually needs to run their other GPU: see workers, add one, name it, prefer
one, pause one, remove one.
Two things here are not conveniences and must not be softened:
* **Consent is explicit and per worker.** Audio, reference voices, and text
leave the machine for a worker, so each one is approved individually. There
is no global "trust all workers".
* **A token is shown exactly once.** Only its hash is stored, so it cannot be
re-displayed which is the point.
One endpoint here is not part of that surface: `POST /workers/tasks` submits a
single task and waits for it, and exists only because the scheduler otherwise
has no caller at all outside the tests. It is marked dev-only everywhere it
appears and is replaced by the GPU gateway.
"""
from __future__ import annotations
import asyncio
import contextlib
import logging
from fastapi import APIRouter, Depends, HTTPException, Request
from fastapi.responses import JSONResponse
from pydantic import BaseModel, Field
from api.dependencies import require_admin
from worker import registry, routing, service
from worker.async_utils import drain_task, to_thread_and_defer_cancellation
logger = logging.getLogger("omnivoice.worker")
# How often an awaiting request checks whether its caller is still there.
# Starlette does not cancel a handler when the client hangs up, so polling is
# the only way the "cancel what nobody is waiting for" rule can fire before
# the task's own deadline does.
_DISCONNECT_POLL_SECONDS = 1.0
# Management is admin-gated: these endpoints mint join tokens and revoke
# machines, so Docker writes require the API key while desktop stays loopback.
router = APIRouter(prefix="/workers", tags=["workers"], dependencies=[Depends(require_admin)])
class EnableRequest(BaseModel):
enabled: bool
class EnrollRequest(BaseModel):
label: str = Field("", max_length=120)
endpoint: str = Field("", max_length=256)
ttl_seconds: int = Field(900, ge=60, le=24 * 3600)
class JoinRequest(BaseModel):
"""A join code, as pasted (or scanned) from the control plane."""
token: str = Field(..., max_length=4096)
class TargetRequest(BaseModel):
"""`local`, or the id of an enrolled worker."""
target: str = Field(..., max_length=64)
class WorkerUpdate(BaseModel):
name: str | None = Field(None, max_length=120)
enabled: bool | None = None
priority: int | None = Field(None, ge=0, le=100)
class SubmitTaskRequest(BaseModel):
"""One unit of work for a remote worker. **Dev only** — see `submit_task`."""
engine: str = Field(..., max_length=64)
operation: str = Field("tts", max_length=32)
model_id: str = Field("", max_length=128)
params: dict = Field(default_factory=dict)
# Mandatory, and deliberately without a default: the sweeper fails a task
# on its deadline only while it is QUEUED, so one submitted without a
# deadline while no worker is online waits forever with nothing left in
# the system that would ever time it out.
deadline_seconds: float = Field(..., gt=0, le=6 * 3600)
idempotency_key: str | None = Field(None, max_length=128)
class _ClientGone(Exception):
"""The caller hung up while its task was still running."""
class _WaitExpired(Exception):
"""The task did not reach a terminal state inside its deadline."""
@router.get("")
def list_workers() -> dict:
"""Everything the workers panel renders, in one call."""
return service.control_plane.snapshot()
@router.get("/target")
def get_target(op: str = "") -> dict:
"""What the GPU picker shows: the choice, the resolved answer, the options.
`active` is the same answer the generation path uses, so the badge cannot
claim work goes somewhere the router will not send it. Pass `op` for the
surface being rendered omitting it answers for the target as a whole,
which is what the picker's own menu asks.
"""
return routing.status(op=op.strip() or None)
@router.get("/runtime")
async def get_runtime(engine: str = "", op: str = "tts") -> dict:
"""Runtime/model facts for the machine that will execute this operation."""
from services import gpu_gateway # noqa: PLC0415
return await gpu_gateway.status(
engine=engine.strip() or None,
op=op.strip() or "tts",
control_plane=service.control_plane,
)
@router.post("/target")
def set_target(request: TargetRequest) -> dict:
"""Choose where work runs. Exactly one target is active at a time."""
chosen = request.target.strip() or routing.LOCAL
if chosen != routing.LOCAL:
worker = registry.get(chosen)
if worker is None or worker.revoked:
raise HTTPException(status_code=404, detail="No such worker.")
routing.set_target_id(chosen)
return routing.status()
@router.post("/enabled")
async def set_enabled(request: EnableRequest) -> dict:
"""Turn the feature on or off.
Off means off: the control plane stops, the listening socket closes, and
the app is exactly what it was before the toggle existed.
"""
service.set_remote_workers_enabled(request.enabled)
if request.enabled:
try:
await service.control_plane.start()
except Exception as exc:
service.control_plane.startup_error = str(exc)
raise HTTPException(status_code=409, detail=str(exc)) from exc
else:
await service.control_plane.stop()
return service.control_plane.snapshot()
@router.get("/agent")
def agent_status() -> dict:
"""The other side of the same feature: is THIS machine lending its GPU?
Separate from `GET /workers`, which answers for the control plane. A
machine can legitimately be both a desktop that borrows a laptop's GPU
and lends its own to a colleague so neither status can stand in for the
other.
"""
from worker import agent as worker_agent # noqa: PLC0415
return worker_agent.agent.status()
@router.get("/agent/readiness", include_in_schema=False, response_model=None)
def agent_readiness() -> JSONResponse:
"""Container readiness: 200 only after this process registered as a worker."""
from worker import agent as worker_agent # noqa: PLC0415
readiness = worker_agent.agent.readiness()
return JSONResponse(
status_code=200 if readiness["ready"] else 503,
content=readiness,
headers={} if readiness["ready"] else {"Retry-After": "2"},
)
def _refuse_when_env_pinned(worker_agent) -> None:
"""OMNIVOICE_WORKER_MODE wins over the setting everywhere else.
`worker_mode_enabled()` reads the variable first and `status()` reports the
machine as env-pinned, so a route that changed worker mode anyway would
contradict both: it writes a setting nothing consults, and the next restart
undoes whatever the user just saw happen.
"""
if worker_agent.agent.status()["env_pinned"]:
raise HTTPException(
status_code=409,
detail=(
"OMNIVOICE_WORKER_MODE controls this machine's worker mode. Unset it "
"and restart VoiceStudio to manage it from here."
),
)
async def _finish_cleanup(awaitable):
"""Run rollback to completion even if its HTTP task was cancelled."""
task = asyncio.create_task(awaitable)
await drain_task(task)
return task.result()
async def _set_worker_mode(worker_agent, enabled: bool) -> None:
_result, cancelled = await to_thread_and_defer_cancellation(
worker_agent.set_worker_mode_enabled, enabled
)
if cancelled:
raise asyncio.CancelledError
async def _restore_agent_transaction(
worker_agent, previous: dict, *, was_running: bool
) -> None:
"""Restore durable enrollment/settings and the exact prior live state."""
try:
await _finish_cleanup(worker_agent.agent.stop())
await _finish_cleanup(worker_agent.restore_enrollment(previous))
if was_running and not worker_agent.agent.running:
await _finish_cleanup(worker_agent.agent.start())
elif not was_running and worker_agent.agent.running:
await _finish_cleanup(worker_agent.agent.stop())
except worker_agent.EnrollmentRollbackError:
raise
except BaseException as exc:
message = (
"The previous worker state could not be restored safely. "
"Worker mode remains stopped; fix its enrollment/settings storage, then retry."
)
with contextlib.suppress(BaseException):
await _finish_cleanup(worker_agent.agent.stop())
worker_agent.agent.last_error = message
raise worker_agent.EnrollmentRollbackError(message) from exc
def _raise_agent_transaction_failure(
worker_agent, operation: BaseException, rollback: BaseException | None
) -> None:
if isinstance(operation, asyncio.CancelledError):
if rollback is not None:
logger.error(
"Worker rollback failed during request cancellation",
exc_info=(type(rollback), rollback, rollback.__traceback__),
)
raise operation
if rollback is not None:
raise HTTPException(status_code=409, detail=str(rollback)) from rollback
if isinstance(operation, Exception):
worker_agent.agent.last_error = str(operation)
raise HTTPException(status_code=409, detail=str(operation)) from operation
raise operation
@router.post("/agent/join")
async def join_control_plane(request: JoinRequest) -> dict:
"""Redeem a join code and start working for that control plane.
This is the endpoint that makes the feature reachable. Joining used to mean
setting OMNIVOICE_WORKER_MODE and OMNIVOICE_WORKER_TOKEN in the environment
and relaunching the app a step most users will never take, on the machine
that is usually the least convenient to configure by hand.
The code is single-use and short-lived, so a failure here is nearly always
"expired" or "wrong address"; it is returned verbatim rather than as a bare
409, because the user's next action depends on which one it was.
"""
from worker import agent as worker_agent # noqa: PLC0415
token = request.token.strip()
if not token:
raise HTTPException(status_code=422, detail="Paste the join code first.")
# Same rule as the toggle below: joining ENABLES worker mode, so under
# OMNIVOICE_WORKER_MODE it would write a setting the rest of the app
# ignores — and with the variable set to 0, hand the user a machine that
# says it joined and never lends anything (CodeRabbit).
_refuse_when_env_pinned(worker_agent)
async with worker_agent.agent.lifecycle:
try:
previous, cancelled = await to_thread_and_defer_cancellation(
worker_agent.snapshot_enrollment
)
except worker_agent.EnrollmentStateError as exc:
worker_agent.agent.last_error = str(exc)
raise HTTPException(status_code=409, detail=str(exc)) from exc
if cancelled:
raise asyncio.CancelledError
was_running = worker_agent.agent.running
# A rejoin stops a working agent before the replacement is accepted.
# Stop, acceptance and the durable setting are one transaction: every
# failure, including cancellation, restores both trust and live state.
try:
await worker_agent.agent.stop()
await worker_agent.agent.start(token_text=token)
# Success is the control plane ACCEPTING this worker, not the
# connection being scheduled — see wait_until_registered.
await worker_agent.agent.wait_until_registered()
await _set_worker_mode(worker_agent, True)
except BaseException as exc:
rollback_exc = None
try:
await _restore_agent_transaction(
worker_agent, previous, was_running=was_running
)
except BaseException as rollback_error:
rollback_exc = rollback_error
_raise_agent_transaction_failure(worker_agent, exc, rollback_exc)
worker_agent.agent.last_error = ""
return worker_agent.agent.status()
@router.post("/agent/enabled")
async def set_agent_enabled(request: EnableRequest) -> dict:
"""Start or stop lending this machine, without forgetting the enrollment.
Off stops the agent and clears the setting, so nothing dials out; the
pinned certificate stays, which is what lets "on" resume without asking for
another code.
"""
from worker import agent as worker_agent # noqa: PLC0415
_refuse_when_env_pinned(worker_agent)
async with worker_agent.agent.lifecycle:
try:
previous, cancelled = await to_thread_and_defer_cancellation(
worker_agent.snapshot_enrollment
)
except worker_agent.EnrollmentStateError as exc:
worker_agent.agent.last_error = str(exc)
raise HTTPException(status_code=409, detail=str(exc)) from exc
if cancelled:
raise asyncio.CancelledError
was_running = worker_agent.agent.running
try:
if request.enabled:
await worker_agent.agent.start()
await worker_agent.agent.wait_until_registered()
await _set_worker_mode(worker_agent, True)
else:
await worker_agent.agent.stop()
await _set_worker_mode(worker_agent, False)
except BaseException as exc:
rollback_exc = None
try:
await _restore_agent_transaction(
worker_agent, previous, was_running=was_running
)
except BaseException as rollback_error:
rollback_exc = rollback_error
_raise_agent_transaction_failure(worker_agent, exc, rollback_exc)
worker_agent.agent.last_error = ""
return worker_agent.agent.status()
@router.post("/enrollments")
def create_enrollment(request: EnrollRequest) -> dict:
"""Mint a single-use join token.
The plaintext is returned once and never stored the response is the only
time it exists outside the worker that redeems it.
"""
if not service.control_plane.running:
raise HTTPException(
status_code=409,
detail="Remote workers are turned off. Enable them in Settings → System → Remote workers first.",
)
try:
token = service.control_plane.create_enrollment(
endpoint=request.endpoint,
label=request.label,
ttl_seconds=request.ttl_seconds,
)
except service.EndpointCertificateError as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
return {
"token": token.encode(),
"endpoint": token.endpoint,
"fingerprint": token.cert_fingerprint,
"expires_at": token.expires_at,
"shown_once": True,
}
def _persist_worker_update(
worker_id: str, request: WorkerUpdate
):
"""Write policy on a worker thread; live publication stays loop-owned."""
return registry.update_policy(
worker_id,
name=request.name,
enabled=request.enabled,
priority=request.priority,
)
@router.patch("/{worker_id}")
async def update_worker(worker_id: str, request: WorkerUpdate) -> dict:
pool = service.control_plane.pool if service.control_plane.running else None
live = None
was_pending = False
if pool is not None:
# Quiesce dispatch before releasing authority for the SQLite write.
# The publication after the await restores the exact prior state, so a
# concurrent registration handoff remains quiesced for its own reason.
with registry.authority_guard():
live = pool.get(worker_id)
if live is not None:
was_pending = live.registration_pending
live.registration_pending = True
updated = None
cancelled = False
try:
updated, cancelled = await to_thread_and_defer_cancellation(
_persist_worker_update, worker_id, request
)
finally:
if pool is not None:
with registry.authority_guard():
if updated is not None:
# Pool state, including the cached record the scheduler
# reads, belongs to the app's event loop.
pool.refresh_record(updated)
current = pool.get(worker_id)
if current is live:
current.registration_pending = was_pending
if updated is None:
if cancelled:
raise asyncio.CancelledError
raise HTTPException(status_code=404, detail="No such worker.")
if cancelled:
raise asyncio.CancelledError
return updated.to_dict()
@router.post("/{worker_id}/consent")
def grant_consent(worker_id: str) -> dict:
"""Record the user's explicit yes to sending their audio to this machine."""
if registry.get(worker_id) is None:
raise HTTPException(status_code=404, detail="No such worker.")
registry.grant_consent(worker_id)
worker = registry.get(worker_id)
return worker.to_dict() if worker else {}
@router.post("/{worker_id}/resume")
async def clear_breaker(worker_id: str) -> dict:
"""Clear a paused worker's circuit breakers.
The user fixed the machine and knows it a breaker with no manual clear is
the quarantine trap the reputation system had.
"""
if not service.control_plane.running:
raise HTTPException(status_code=409, detail="Remote workers are turned off.")
breakers = service.control_plane.pool.breakers
for breaker in breakers.open_breakers(worker_id):
breaker.force_close()
return {"ok": True}
@router.delete("/{worker_id}")
async def revoke_worker(worker_id: str) -> dict:
"""Remove a worker — which means revoke its key, not hide the row.
Its in-flight work is released so it can be retried elsewhere rather than
waiting out a lease on a machine that will never answer again.
"""
pool = service.control_plane.pool if service.control_plane.running else None
live = None
was_pending = False
if pool is not None:
with registry.authority_guard():
live = pool.get(worker_id)
if live is not None:
was_pending = live.registration_pending
live.registration_pending = True
try:
revoked, cancelled = await to_thread_and_defer_cancellation(
registry.revoke, worker_id
)
except BaseException:
if pool is not None:
with registry.authority_guard():
current = pool.get(worker_id)
if current is live:
current.registration_pending = was_pending
raise
if not revoked:
if pool is not None:
with registry.authority_guard():
current = pool.get(worker_id)
if current is live:
current.registration_pending = was_pending
if cancelled:
raise asyncio.CancelledError
raise HTTPException(status_code=404, detail="No such worker.")
# The tombstone committed before any egress/session mutation. Everything
# below is loop-owned and published under the same scheduler authority read
# used by next_assignment(), so no task can bind in the handoff window.
with registry.authority_guard():
if service.control_plane.running:
if service.control_plane.servicer is not None:
service.control_plane.servicer.revoke_worker_sessions(worker_id)
service.control_plane.scheduler.on_disconnected(worker_id)
service.control_plane.pool.breakers.forget_worker(worker_id)
if cancelled:
raise asyncio.CancelledError
return {"ok": True, "revoked": worker_id}
@router.get("/tasks")
def list_tasks(limit: int = 50) -> dict:
"""Recent remote tasks, for the queue view."""
if not service.control_plane.running:
return {"tasks": [], "queue_depth": 0}
from worker import task_store # noqa: PLC0415
return {
"queue_depth": service.control_plane.scheduler.queue_depth,
"tasks": [t.to_dict() for t in task_store.list_tasks(limit=min(200, max(1, limit)))],
}
@router.post("/tasks")
async def submit_task(request: Request, body: SubmitTaskRequest) -> dict:
"""Run one task on a remote worker and wait for it. **DEV ONLY.**
This is the producer the remote pipeline never had: until it existed the
scheduler had no caller outside the test suite, so picking a remote GPU
changed the badge and nothing else every job still ran locally. It is
the smallest thing that makes remote execution observable end to end, not
the shipping surface: the GPU gateway takes over routing real generation
and this endpoint goes with it.
Loopback-only and behind the same opt-in as the rest of the feature, so a
user who never enabled remote workers cannot reach it at all.
"""
from worker.lifecycle import TaskState # noqa: PLC0415
from worker.scheduler import QueueFull, SchedulerStopped # noqa: PLC0415
if not service.remote_workers_enabled() or not service.control_plane.running:
raise HTTPException(status_code=409, detail="Remote workers are turned off.")
if not routing.supports_operation(body.operation):
raise HTTPException(
status_code=400,
detail=f"'{body.operation}' does not run on a remote worker yet.",
)
scheduler = service.control_plane.scheduler
try:
submit = getattr(scheduler, "submit_async", None)
submit = submit if callable(submit) else scheduler.submit
submitted = submit(
operation=body.operation,
engine=body.engine,
model_id=body.model_id,
params=body.params,
idempotency_key=body.idempotency_key or None,
deadline_seconds=body.deadline_seconds,
pinned_worker_id=routing.decide().worker_id or None,
)
task = await submitted if asyncio.iscoroutine(submitted) else submitted
except QueueFull as exc:
raise HTTPException(status_code=429, detail=str(exc)) from exc
settled = None
reason = "the request was interrupted"
try:
settled = await _await_terminal(
request, scheduler, task.task_id, timeout=body.deadline_seconds
)
except _ClientGone:
reason = "the client disconnected"
raise HTTPException(status_code=499, detail="The client stopped waiting.") from None
except _WaitExpired:
reason = "the task passed its deadline"
raise HTTPException(
status_code=504,
detail=f"The task did not finish within {body.deadline_seconds:g}s.",
) from None
except SchedulerStopped as exc:
# Deliberately no cancel: the worker was never told to stop and may
# still be rendering, so claiming the task is cancelled would be a
# statement about someone else's GPU that we cannot make.
reason = None
raise HTTPException(status_code=503, detail=str(exc)) from None
finally:
# Nothing else will stop it: a worker holds its slot — often its only
# one — until the control plane says otherwise, and the sweeper only
# enforces deadlines on tasks that are still queued. Swallowed because
# a failure here would replace the caller's real error with a 500.
if settled is None and reason is not None:
try:
await service.control_plane.cancel(task.task_id, reason=reason)
except Exception:
logger.exception("Could not cancel abandoned remote task %s", task.task_id)
payload = settled.to_dict()
if settled.state is TaskState.COMPLETED:
return payload
# A failure that answered 200 would be indistinguishable from success to
# anything that does not read `state` — which is the whole point of this
# endpoint existing before the gateway does.
raise HTTPException(
status_code=409 if settled.state is TaskState.CANCELLED else 502, detail=payload
)
async def _await_terminal(request: Request, scheduler, task_id: str, *, timeout: float):
"""Wait for a terminal task, giving up if the caller does first."""
waiter = asyncio.ensure_future(scheduler.wait(task_id, timeout=timeout))
while True:
done, _pending = await asyncio.wait({waiter}, timeout=_DISCONNECT_POLL_SECONDS)
if done:
try:
settled = waiter.result()
except (asyncio.TimeoutError, TimeoutError) as exc:
raise _WaitExpired() from exc
if settled is None or not settled.state.terminal:
raise _WaitExpired()
return settled
if await request.is_disconnected():
waiter.cancel()
raise _ClientGone()
@router.post("/tasks/{task_id}/cancel")
async def cancel_task(task_id: str) -> dict:
if not service.control_plane.running:
raise HTTPException(status_code=409, detail="Remote workers are turned off.")
cancelled = await service.control_plane.cancel(task_id, reason="cancelled by user")
if not cancelled:
raise HTTPException(status_code=404, detail="No such active task.")
return {"ok": True}
# ── Inbound mode ───────────────────────────────────────────────────────────
#
# The other direction: this machine accepts connections from panels, or dials
# out to nodes that do. Outbound enrollment above is unchanged and remains the
# default — see docs/adr/inbound-node-mode.md for why this exists alongside it
# rather than replacing it.
class InboundEnableRequest(BaseModel):
enabled: bool
# Widening the bind is a separate decision from turning the feature on,
# so it is a separate field with a safe default rather than a flag that
# rides along with `enabled`.
bind: str = ""
port: int = 0
class IssueKeyRequest(BaseModel):
label: str = Field(default="", max_length=64)
class ConnectRequest(BaseModel):
connection_string: str = Field(min_length=1, max_length=512)
@router.get("/inbound")
def inbound_status() -> dict:
from worker.inbound import service as inbound_service # noqa: PLC0415
return {
**inbound_service.node.snapshot(),
"connections": inbound_service.outbound.snapshot(),
}
@router.post("/inbound/enabled")
async def set_inbound_enabled(request: InboundEnableRequest) -> dict:
from worker.inbound import service as inbound_service # noqa: PLC0415
if inbound_service.enabled_override() is not None:
raise HTTPException(
status_code=409,
detail=(
"Accept connections is controlled by OMNIVOICE_INBOUND_NODE on this "
"machine. Change that environment setting and restart VoiceStudio."
),
)
requested_bind = (
inbound_service.normalise_bind_host(request.bind)
if request.bind
else inbound_service.bind_host()
)
requested_port = request.port or inbound_service.bind_port()
if (
request.enabled
and inbound_service.node.running
and (
requested_bind != inbound_service.bind_host()
or requested_port != inbound_service.node.port
)
):
# start() is intentionally idempotent while a listener owns its
# socket. Persisting a new endpoint here would make the UI report a
# narrower/different bind while the original socket stayed live.
raise HTTPException(
status_code=409,
detail=(
"Turn off Accept connections before changing its bind address "
"or port."
),
)
if request.bind:
inbound_service.set_bind_host(requested_bind)
if request.port:
inbound_service.set_bind_port(request.port)
inbound_service.set_enabled(request.enabled)
if inbound_service.enabled():
await inbound_service.node.start()
if inbound_service.node.startup_error:
logger.error("Inbound worker listener failed to start; details withheld.")
raise HTTPException(
status_code=409,
detail=(
"The inbound worker listener could not start; "
"check the backend log for details."
),
)
else:
await inbound_service.node.stop()
return inbound_service.node.snapshot()
@router.post("/inbound/keys")
def issue_inbound_key(request: IssueKeyRequest) -> dict:
"""Mint one panel's key and return the string it pastes.
The secret is in this response and nowhere else afterwards only its hash
is stored, so it cannot be shown again, only replaced.
"""
from worker.inbound import service as inbound_service # noqa: PLC0415
from worker.inbound.keys import KeyLimitExceeded # noqa: PLC0415
if not inbound_service.node.running:
raise HTTPException(
status_code=409,
detail=(
"This machine is not accepting connections yet. Turn on "
"Settings → System → Remote workers → Accept connections first."
),
)
try:
issued = inbound_service.node.keys.issue(request.label)
except KeyLimitExceeded as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
return {
"key_id": issued.key.key_id,
"label": issued.key.label,
"connection_string": inbound_service.node.connection_string(issued.secret),
"exposed": inbound_service.is_exposed(),
"shown_once": True,
}
@router.delete("/inbound/keys/{key_id}")
async def revoke_inbound_key(key_id: str) -> dict:
"""Revoke one panel. Everyone else stays connected — the whole reason keys
are per panel rather than one shared node key."""
from worker.inbound import service as inbound_service # noqa: PLC0415
if not await inbound_service.node.revoke_key(key_id):
raise HTTPException(status_code=404, detail="No such key.")
return inbound_service.node.snapshot()
@router.post("/inbound/sessions/{session_id}/disconnect")
def disconnect_inbound_session(session_id: str) -> dict:
from worker.inbound import service as inbound_service # noqa: PLC0415
if not inbound_service.node.log.kick(session_id):
raise HTTPException(status_code=404, detail="That connection has already ended.")
return inbound_service.node.snapshot()
@router.post("/inbound/connections")
async def add_inbound_connection(request: ConnectRequest) -> dict:
"""Paste a connection string from a GPU machine and dial it."""
from worker.inbound import service as inbound_service # noqa: PLC0415
from worker.inbound.connection_string import InvalidConnectionString # noqa: PLC0415
from worker.inbound.connector import InboundConnectionError # noqa: PLC0415
if not service.control_plane.running:
raise HTTPException(
status_code=409,
detail=(
"Remote workers are turned off. Enable them in "
"Settings → System → Remote workers first."
),
)
try:
connection = await inbound_service.outbound.add(
request.connection_string, service.control_plane.servicer
)
except InvalidConnectionString as exc:
# 400 with the parser's own words: every one of these otherwise
# surfaces as "cannot connect", which is what a firewall, a wrong port
# and a dead node all say too.
raise HTTPException(status_code=400, detail=str(exc)) from exc
except InboundConnectionError as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
return {"endpoint": connection.endpoint, "connections": inbound_service.outbound.snapshot()}
@router.delete("/inbound/connections/{endpoint}")
async def remove_inbound_connection(endpoint: str) -> dict:
from worker.inbound import service as inbound_service # noqa: PLC0415
from worker.inbound.connector import InboundConnectionError # noqa: PLC0415
try:
await inbound_service.outbound.remove(endpoint)
except InboundConnectionError as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
return {"connections": inbound_service.outbound.snapshot()}
+1 -23
View File
@@ -15,16 +15,9 @@ class SysinfoResponse(BaseModel):
model_config = ConfigDict(extra="allow")
cpu: float = Field(description="CPU usage percentage (0100)")
cpu_model: str = ""
cpu_physical_cores: int = 0
cpu_logical_cores: int = 0
cpu_frequency_ghz: float = 0.0
ram: float = Field(description="Used RAM in GiB")
total_ram: float = Field(description="Total RAM in GiB")
gpu_name: str = ""
gpu_utilization: float | None = None
vram: float = Field(0.0, description="Used VRAM in GiB")
total_vram: float = Field(0.0, description="Total VRAM in GiB when reported by the runtime")
gpu_active: bool = Field(False, description="Whether a GPU is actively used")
@@ -33,21 +26,6 @@ class SystemInfoResponse(BaseModel):
model_config = ConfigDict(extra="allow")
app_version: str = ""
# Effective compute-time budgets (seconds) for one synthesis job — the
# values services/model_manager.py's GPU_JOB_TIMEOUT_S / CPU_JOB_TIMEOUT_S
# captured at backend import time (#1787). A value just saved via
# /system/set-env is NOT reflected here until the next restart.
generate_timeout_s: float = 300.0
cpu_generate_timeout_s: float = 600.0
# True when an external env var (shell, `.env`, Docker, …) is currently
# shadowing a prefs.json save for this key — see core.prefs.is_env_shadowed.
generate_timeout_shadowed: bool = False
cpu_generate_timeout_shadowed: bool = False
# #1770: the desktop attach handshake's code fingerprint — whatever
# Tauri set OMNIVOICE_BUILD_FINGERPRINT to when it spawned this process,
# echoed back verbatim. Blank when unset (dev mode, a manually started
# backend). See frontend/src-tauri/src/backend.rs::code_fingerprint_is_current.
code_fingerprint: str = ""
data_dir: str
outputs_dir: str
crash_log_path: str
@@ -89,7 +67,7 @@ class ModelStatusResponse(BaseModel):
status: str = Field(description="idle | loading | ready")
checkpoint: str | None = None
loaded_at: str | None = None
sub_stage: str | None = Field(None, description="Current TTS loading sub-stage: importing | loading_weights | compiling | ready | error")
sub_stage: str | None = Field(None, description="Current loading sub-stage: importing | loading_weights | loading_asr | compiling | ready | error")
detail: str | None = Field(None, description="Human-readable detail of current loading phase")
error: str | None = Field(None, description="Error message if loading failed")
Binary file not shown.
@@ -1,3 +0,0 @@
1
00:00:00,000 --> 00:00:13,720
VoiceStudio es una aplicación de escritorio para clonación de voz, doblaje de vídeo y diseño de voz. Funciona completamente en tu máquina. Sin cuentas, sin nube, sin claves de API. Solo abre la aplicación y comienza a crear.
Binary file not shown.
@@ -1,3 +0,0 @@
1
00:00:00,000 --> 00:00:15,000
VoiceStudio est une application de bureau pour le clonage de voix, le doublage vidéo et la conception vocale. Elle fonctionne entièrement sur votre machine. Pas de compte, pas de cloud, pas de clé d'API. Ouvrez l'application et commencez à créer.
Binary file not shown.
@@ -1,3 +0,0 @@
1
00:00:00,000 --> 00:00:16,560
VoiceStudioは、ボイスクローン、ビデオ吹き替え、ボイスデザインのためのデスクトップアプリです。すべてお使いのコンピュータ上で動作します。アカウント、クラウド、APIキーは不要です。アプリを開けば、すぐに制作を始められます。
Binary file not shown.
@@ -1,3 +0,0 @@
1
00:00:00,000 --> 00:00:13,200
VoiceStudio 是一款桌面应用,用于语音克隆、视频配音和声音设计。它完全在你的电脑上运行。无需账户,无需云端,无需 API 密钥。打开应用即可开始创作。
@@ -1,47 +0,0 @@
{
"version": "0.3.0",
"rendered_by": "omnivoice engine + ffmpeg showwaves",
"rendered_at": "2026-08-12T19:47:29Z",
"license": "MIT (synthetic, no third-party IP)",
"source": {
"code": "en",
"label": "English",
"video": "source.mp4",
"srt": "source.srt",
"script": "VoiceStudio is a desktop app for voice cloning, video dubbing, and voice design. It runs entirely on your machine. No accounts, no cloud, no API keys. Just open the app and start creating."
},
"dubbed": [
{
"code": "es",
"label": "Español",
"video": "dubbed_es.mp4",
"srt": "dubbed_es.srt",
"dir": "ltr",
"script": "VoiceStudio es una aplicación de escritorio para clonación de voz, doblaje de vídeo y diseño de voz. Funciona completamente en tu máquina. Sin cuentas, sin nube, sin claves de API. Solo abre la aplicación y comienza a crear."
},
{
"code": "fr",
"label": "Français",
"video": "dubbed_fr.mp4",
"srt": "dubbed_fr.srt",
"dir": "ltr",
"script": "VoiceStudio est une application de bureau pour le clonage de voix, le doublage vidéo et la conception vocale. Elle fonctionne entièrement sur votre machine. Pas de compte, pas de cloud, pas de clé d'API. Ouvrez l'application et commencez à créer."
},
{
"code": "zh",
"label": "中文",
"video": "dubbed_zh.mp4",
"srt": "dubbed_zh.srt",
"dir": "ltr",
"script": "VoiceStudio 是一款桌面应用,用于语音克隆、视频配音和声音设计。它完全在你的电脑上运行。无需账户,无需云端,无需 API 密钥。打开应用即可开始创作。"
},
{
"code": "ja",
"label": "日本語",
"video": "dubbed_ja.mp4",
"srt": "dubbed_ja.srt",
"dir": "ltr",
"script": "VoiceStudioは、ボイスクローン、ビデオ吹き替え、ボイスデザインのためのデスクトップアプリです。すべてお使いのコンピュータ上で動作します。アカウント、クラウド、APIキーは不要です。アプリを開けば、すぐに制作を始められます。"
}
]
}
Binary file not shown.
Binary file not shown.
@@ -1,3 +0,0 @@
1
00:00:00,000 --> 00:00:11,400
VoiceStudio is a desktop app for voice cloning, video dubbing, and voice design. It runs entirely on your machine. No accounts, no cloud, no API keys. Just open the app and start creating.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More