diff --git a/.claude/skills/omnivoice/SKILL.md b/.claude/skills/omnivoice/SKILL.md new file mode 100644 index 00000000..db5609fa --- /dev/null +++ b/.claude/skills/omnivoice/SKILL.md @@ -0,0 +1,169 @@ +--- +name: omnivoice +description: "Local TTS, voice cloning, voice design, and video dubbing via the OmniVoice Studio MCP server (open-source ElevenLabs alternative; nothing leaves the machine, runs on MPS/CUDA/CPU). Use when: (1) generating speech from text in any of 646 languages, (2) cloning a voice from a 3-second reference clip, (3) designing a voice by gender/age/accent/pitch/style, (4) dubbing a video into another language, (5) listing voice profiles or personality presets, (6) producing narration where privacy, cost, or absent API keys matter, (7) non-English narration where Edge TTS/kokoro fall short, (8) batch audio for blog posts or content pipelines. Triggers: 'omnivoice', 'voice clone', 'clone this voice', 'tts', 'narrate', 'generate speech', 'voice synthesis', 'dub video', 'voice design', 'local tts', 'multilingual voice', 'narrate this post', 'elevenlabs alternative'." +--- + +# OmniVoice + +## Overview + +Generate audio locally via the OmniVoice Studio MCP server. Tools: `generate_speech`, `list_voices`, `list_personalities`, `list_languages`, `check_health`. Resources: `voice://{id}`, `history://recent`. + +## Prerequisites — Backend Must Be Running + +The MCP tools all hit `$OMNIVOICE_API_URL` (default `http://localhost:3900`). If the backend is down, every tool returns a connection error. Install + boot: + +```bash +git clone https://github.com/debpalash/OmniVoice-Studio.git "$OMNIVOICE_HOME" +cd "$OMNIVOICE_HOME" +uv sync +VIRTUAL_ENV="$(pwd)/.venv" uv pip install 'mcp[cli]' +``` + +Then: + +```bash +scripts/check-health.sh # exit 0 if up +scripts/start-backend.sh # boot in background (MPS/CUDA auto-detected) +``` + +First synthesis call lazy-downloads the `k2-fsa/OmniVoice` model (~2.4 GB) from HuggingFace — cached on subsequent boots. + +## Task Index — Pick the Right Tool + +| Task | Tool | Notes | +|---|---|---| +| Verify backend is up | `check_health` | Returns `{"status":"ok","device":"mps|cuda|cpu"}` | +| Text → audio with a saved voice | `generate_speech(text, profile_id)` | Returns base64 WAV. `profile_id="demo0001"` is the bundled demo voice | +| Text → audio without a clone (voice design) | `generate_speech(text, instruct="…")` | Omit `profile_id`; pass an `instruct` like `"warm middle-aged female narrator, calm pace"` | +| Multilingual narration | `generate_speech(text, language="es")` | Any ISO 639 code or `"Auto"` | +| List existing voices | `list_voices` | Returns id, name, type, personality | +| List personality presets | `list_personalities` | Returns narrator / casual / news-anchor / etc. with their `instruct` strings | +| List supported languages | `list_languages` | 646 total; returns 20 popular + the full count | + +For non-trivial decisions (which engine to use, when to pick OmniVoice over kokoro / Edge TTS / ElevenLabs), see [references/engines-comparison.md](references/engines-comparison.md). + +For MCP wiring details, backend lifecycle, troubleshooting, and a clean teardown, see [references/mcp-setup.md](references/mcp-setup.md). + +## Common Workflows + +### 1. One-shot narration with the demo voice + +```python +# As called through the MCP client (your agent will do this for you): +result = generate_speech( + text="Hello — this is OmniVoice generating speech locally.", + profile_id="demo0001", + language="English", + steps=16, # 8 = fast/draft · 16 = balanced · 32 = quality +) +# result is JSON with audio_id, generation_time_s, audio_duration_s, format, wav_base64 +``` + +Benchmark: 4.2 s of audio in ~24 s server-side on Apple Silicon MPS at 16 diffusion steps. + +### 2. Save the WAV to disk and play + +Tool returns base64 PCM WAV (16-bit, mono, 24 kHz). Decode + write: + +```python +import base64, json +payload = json.loads(result_text) # parse JSON the tool returns +open("out.wav","wb").write(base64.b64decode(payload["wav_base64"])) +``` + +On macOS: `afplay out.wav`. Convert to MP3 with `ffmpeg -i out.wav -codec:a libmp3lame -b:a 128k out.mp3`. + +### 3. Voice clone — end-to-end recipe + +Cloning needs a 3-10 second reference clip the model will use as a speaker embedding. The MCP server does NOT expose profile creation — it only reads existing profiles. Two paths to create one: + +**Path A — bundled helper (macOS, recommended for fresh clones):** + +```bash +scripts/record-reference.sh ~/Downloads/my-ref.wav 12 1 +# args: output_path raw_duration_sec mic_index +# Default mic_index=1 (MacBook built-in); list devices via: +# ffmpeg -f avfoundation -list_devices true -i "" +``` + +The script gives **audible** countdown + start/stop cues via macOS `say` + `/System/Library/Sounds/Ping.aiff` so the user knows when to speak (terminal stdout is buffered — text "speak now" prompts arrive too late). It records a longer raw window, then trims to ~10 seconds of speech via `silenceremove + atrim`, plays back for verification, and prints the next-step `curl` command. + +**Path B — manual:** + +```bash +# 1. Record (mono, 24 kHz native — matches model's internal rate) +ffmpeg -f avfoundation -i ":1" -t 12 -ac 1 -ar 24000 raw.wav + +# 2. Trim leading silence + take first 10 sec of speech +ffmpeg -i raw.wav \ + -af "silenceremove=start_periods=1:start_silence=0.05:start_threshold=-40dB,atrim=end=10" \ + -ac 1 -ar 24000 ref.wav + +# 3. Verify +ffmpeg -i ref.wav -af volumedetect -f null - 2>&1 | grep volume # max should be > -20 dB +afplay ref.wav +``` + +**POST to /profiles** (multipart/form-data — required fields: `name`, `ref_audio`): + +```bash +curl -X POST http://127.0.0.1:3900/profiles \ + -F "name=carlos-clone" \ + -F "ref_audio=@ref.wav" \ + -F "ref_text=The exact text spoken in the clip" \ + -F "language=English" \ + | python3 -m json.tool +# returns { "id": "abc12345", "name": "carlos-clone" } +``` + +Once created, pass `profile_id` to `generate_speech` (via MCP) or directly via `POST /generate`. Profiles persist in SQLite + reference-audio files at `~/Library/Application Support/OmniVoice/voices/.` (the backend preserves the uploaded extension — `.wav` if you uploaded a WAV, `.mp3` if MP3, etc.). State persists across backend restarts. + +**Reference clip tips that materially affect quality:** + +| Factor | Why it matters | +|---|---| +| Single speaker | Mixed speakers blur the embedding | +| Clean speech, no music/noise | Model embeds the noise too | +| Natural prosody (avoid pangrams) | Diffusion samples replicate prosody, not just timbre | +| 3-10 sec is the sweet spot | < 3 s lacks information; > 10 s adds compute without quality gain | +| Match `ref_text` to what's spoken | Improves alignment, especially on noisy refs | +| `language` correct | Wrong language → cross-lingual transfer artifacts | +| Loudness peak ≥ -15 dB | Quiet refs work but normalize poorly | + +### 4. Voice design (no reference clip) + +Skip `profile_id`; provide an `instruct` string describing the desired voice: + +```python +generate_speech( + text="Welcome to the future of agentic systems.", + instruct="warm middle-aged female narrator, calm authoritative pace, documentary style", +) +``` + +Get pre-made instructs via `list_personalities` and copy the one matching the brief (narrator, casual, news-anchor, etc.). + +### 5. Video dubbing (web UI only) + +The MCP server does not expose the dubbing endpoint. The full transcribe → translate → re-voice → mux pipeline lives behind the desktop UI (`bun run desktop` in `$OMNIVOICE_HOME`) and the `/dub/*` REST routes. When the user asks to dub a video, point them to the UI; surface this skill only for the synthesis primitives above. + +## When NOT to use OmniVoice + +- **Fast English-only narration on weak hardware** → `kokoro-tts` is ~10× smaller and 2× realtime on CPU (see [references/engines-comparison.md](references/engines-comparison.md)) +- **Lowest-friction one-off TTS** → Edge TTS needs no install or backend +- **Highest possible quality regardless of cost** → ElevenLabs still wins on English narration polish; OmniVoice ties or wins on multilingual + cloning +- **Real-time streaming dictation** → use the OmniVoice desktop widget (`⌘+⇧+Space`), not the MCP server + +## Resources + +- [references/engines-comparison.md](references/engines-comparison.md) — Decision tree across OmniVoice / kokoro / Voicebox / Edge TTS / ElevenLabs / cloud APIs +- [references/mcp-setup.md](references/mcp-setup.md) — MCP wiring, backend lifecycle, env vars, troubleshooting +- [scripts/check-health.sh](scripts/check-health.sh) — `curl /health`, exit 0/1 +- [scripts/start-backend.sh](scripts/start-backend.sh) — Start uvicorn on 127.0.0.1:3900 with health probe +- [scripts/stop-backend.sh](scripts/stop-backend.sh) — Clean shutdown via `kill -TERM` on the bound PID +- [scripts/record-reference.sh](scripts/record-reference.sh) — macOS-only: record + trim + verify a reference clip for cloning, with audible cues (`say` + system beeps) that bypass terminal output buffering + +Backend Swagger / OpenAPI: `http://127.0.0.1:3900/docs` (when backend is up). + +Upstream: github.com/debpalash/OmniVoice-Studio — FSL-1.1-ALv2 (free for personal/internal/non-commercial; auto-converts to Apache-2.0 two years after each release). diff --git a/.claude/skills/omnivoice/references/engines-comparison.md b/.claude/skills/omnivoice/references/engines-comparison.md new file mode 100644 index 00000000..79b19d6b --- /dev/null +++ b/.claude/skills/omnivoice/references/engines-comparison.md @@ -0,0 +1,71 @@ +# TTS Engine Selection — Decision Tree + +When to pick OmniVoice vs other engines available in this workspace. Match the user's constraint to the right column. + +## Decision tree + +``` +Is voice cloning required? +├─ yes → OmniVoice (3-sec ref clip, zero-shot, 646 langs) +└─ no → + Is the language non-English? + ├─ yes → OmniVoice (646 langs) or Edge TTS (subset, cloud) + └─ no (English) → + Is privacy required (no cloud)? + ├─ yes → + │ Is GPU available? + │ ├─ yes (CUDA/MPS) → OmniVoice (best quality) or Voicebox + │ └─ no (CPU only) → kokoro-tts (2× realtime CPU) or OmniVoice on CPU (slow) + └─ no (cloud OK) → + Is cost-no-object? + ├─ yes → ElevenLabs (best polish), then OpenAI TTS + └─ no → Edge TTS (free, unofficial, MS Azure neural) +``` + +## Full comparison + +| Engine | Quality | Clone | Multilingual | Cost | Privacy | Setup | Best for | +|---|---|---|---|---|---|---|---| +| **OmniVoice** | 8-9/10 | ✅ 3-sec ref | 646 langs | Free | Local | Bun + uv install | Multilingual, cloning, privacy-critical | +| ElevenLabs | 9-10/10 | ✅ 3-sec ref | 32 langs | $5-330/mo | Cloud | API key | Best English polish, fastest cloud TTS | +| Voicebox (Qwen3-TTS) | 8-9/10 | ✅ | Multi | Free | Local | Docker | Self-hosted alternative to OmniVoice | +| Voicebox (LuxTTS) | 7/10 | ❌ | Multi | Free | Local | Docker | CPU at 150× realtime | +| kokoro-tts | 7-8/10 | ❌ | Multi (limited) | Free | Local | pip | Fast English narration on CPU | +| mlx-audio | 7-8/10 | varies | Multi | Free | Local | pip | Apple Silicon native, 14+ sub-engines | +| Edge TTS | 7-8/10 | ❌ | 50+ | Free* | Cloud | pip | Zero-friction one-off | +| OpenAI TTS | 8/10 | ❌ | Multi | $0.015/1k chars | Cloud | API key | Convenient, cheap-ish, good quality | +| Google Cloud TTS | 8/10 | ❌ | Multi | $4/1M chars (WaveNet) | Cloud | GCP project | Large free tier (1M chars/mo) | + +*Edge TTS is unofficial. Microsoft could block it at any time. + +## When OmniVoice wins decisively + +1. **Voice cloning** — 3-sec reference clip, zero-shot, no fine-tuning. ElevenLabs is the only competitor; OmniVoice is free and local. +2. **Long-tail languages** — 646 supported. ElevenLabs covers 32; everything else fewer. +3. **Privacy / regulatory** — Nothing leaves the machine. ElevenLabs and OpenAI ship audio to their servers. +4. **No-API-key constraint** — Local-first. No accounts. +5. **Bulk generation without metered cost** — ElevenLabs bills per character. OmniVoice is free at any volume. + +## When OmniVoice loses + +1. **Lowest-friction one-off TTS** — Backend install + ~3 GB model + uvicorn boot. Edge TTS or OpenAI TTS is one command. +2. **Fast English narration on weak hardware** — kokoro-tts is ~30 MB vs OmniVoice's 2.4 GB and runs 2× realtime on CPU. Use kokoro for blog-narration batch jobs unless you need cloning. +3. **Streaming real-time TTS** — OmniVoice is diffusion-based and not streaming. Use Edge TTS or cloud APIs for true streaming. +4. **Apple Silicon-only specialized voices** — `mlx-audio` ships 14 engines (Kokoro, CSM, Dia, Qwen3-TTS, etc.) that may match a specific voice better. + +## Composition with content pipelines + +OmniVoice fits between visual asset generation and video assembly: + +``` +research → narrative → visual assets → AUDIO (OmniVoice) → video assembly → distribution +``` + +Default for blog-post audio narration: + +- **English, no cloning needed, fast** → kokoro-tts (cheap CPU) +- **English, want a specific cloned voice** → OmniVoice with a saved profile +- **Non-English** → OmniVoice +- **One-time, no install** → Edge TTS + +For Remotion-based video pipelines that previously required ElevenLabs, OmniVoice closes the last cloud dependency — pair it with any local image/video generator for a fully self-hosted multimedia stack. diff --git a/.claude/skills/omnivoice/references/mcp-setup.md b/.claude/skills/omnivoice/references/mcp-setup.md new file mode 100644 index 00000000..6a649838 --- /dev/null +++ b/.claude/skills/omnivoice/references/mcp-setup.md @@ -0,0 +1,102 @@ +# OmniVoice MCP Setup, Lifecycle, Troubleshooting + +## Install + +```bash +# Pick any location. The scripts in this skill default to ~/OmniVoice-Studio if +# $OMNIVOICE_HOME is unset. +export OMNIVOICE_HOME="${HOME}/OmniVoice-Studio" + +git clone https://github.com/debpalash/OmniVoice-Studio.git "$OMNIVOICE_HOME" +cd "$OMNIVOICE_HOME" +uv sync # ~1.6 GB venv on darwin arm64 +VIRTUAL_ENV="$(pwd)/.venv" uv pip install 'mcp[cli]' # SDK not in their lockfile yet +``` + +Any non-default install location works as long as `$OMNIVOICE_HOME` is set in the env that launches the MCP server. + +## MCP Wiring + +Drop into your MCP client config (Claude Desktop, Claude Code at `~/.claude.json`, Cursor, OpenClaw, etc.). Replace `` with the absolute path: + +```json +{ + "mcpServers": { + "omnivoice": { + "type": "stdio", + "command": "uv", + "args": [ + "--directory", "", + "run", "python", "-m", "backend.mcp_server" + ], + "env": { "OMNIVOICE_API_URL": "http://localhost:3900" } + } + } +} +``` + +Restart the MCP client. The server only starts at client launch — in-session edits do not hot-reload. + +> **Note (mcp SDK ≥ 1.10):** If you see `TypeError: FastMCP.__init__() got an unexpected keyword argument 'version'`, your `OmniVoice-Studio` checkout is older than [debpalash/OmniVoice-Studio#112](https://github.com/debpalash/OmniVoice-Studio/pull/112). Either `git pull` once that PR lands, or apply the 3-line patch manually: replace `version="…", description=(…)` with `instructions=(…)` in `backend/mcp_server.py`. + +## Backend Lifecycle + +The MCP server needs the FastAPI backend running: + +```bash +# Foreground (logs in terminal) +cd "$OMNIVOICE_HOME" +uv run uvicorn main:app --app-dir backend --host 127.0.0.1 --port 3900 + +# Detached, via the helper script in this skill +scripts/start-backend.sh # idempotent +scripts/check-health.sh # exit 0/1 +scripts/stop-backend.sh # graceful SIGTERM +``` + +`127.0.0.1` keeps the API local-only. The project's `package.json` defaults to `0.0.0.0` which exposes the API on all interfaces — wider than needed for personal use. + +First boot runs alembic migrations on the SQLite settings DB at `/omnivoice.db`. Idempotent — safe to re-run. + +First synthesis call lazy-downloads the `k2-fsa/OmniVoice` model (~2.4 GB) into the HuggingFace cache. Path varies by OS: + +- **macOS / Linux**: `~/.cache/huggingface/hub/` +- **Windows**: `%LOCALAPPDATA%\OmniVoice\hf_cache` (OmniVoice redirects via `backend/core/config.py` to keep the cache off the system drive root) + +Cached on subsequent boots. + +## Idle Behavior + +`GET /system/info` exposes `idle_timeout_seconds: 900`. After 15 min of no synthesis, the diffusion model is evicted from GPU memory but the FastAPI server stays up. Next call pays ~5-10 s warm-up. + +## Environment Variables + +| Var | Default | Purpose | +|---|---|---| +| `OMNIVOICE_HOME` | `~/OmniVoice-Studio` | Where the OmniVoice Studio repo is cloned (used by scripts in this skill) | +| `OMNIVOICE_API_URL` | `http://localhost:3900` | MCP server's target backend URL | +| `OMNIVOICE_TTS_BACKEND` | `omnivoice` | Switch engine: `cosyvoice`, `mlx-audio`, `voxcpm2`, `moss-tts-nano`, `kittentts` | +| `HF_TOKEN` | (none) | Only needed for gated pyannote diarization models — basic TTS does not require one | + +## Troubleshooting + +| Symptom | Cause | Fix | +|---|---|---| +| MCP tool returns connection error | Backend not running | `scripts/start-backend.sh` | +| `address already in use` | Stale uvicorn on 3900 | `lsof -nP -iTCP:3900 -sTCP:LISTEN` → `kill -TERM ` | +| `FastMCP.__init__() got unexpected keyword argument 'version'` | mcp SDK ≥ 1.10 dropped `version`/`description`, checkout pre-dates [#112](https://github.com/debpalash/OmniVoice-Studio/pull/112) | Update the checkout or apply the 3-line patch manually | +| First call hangs 5-10 min | Model download from HuggingFace | Watch `~/.cache/huggingface/hub/models--k2-fsa--OmniVoice/` grow | +| `/health` returns 500 | Alembic migration failed | Inspect `/crash_log.txt` | +| Voice profile not found | `profile_id` invalid or profile not yet created | `list_voices` first to get valid IDs | +| `pyannote.audio` errors at startup | Missing `HF_TOKEN` for diarization | Only matters for dub pipeline; basic TTS unaffected | +| Generation slow on Apple Silicon | Diffusion fell back to CPU | `/health` should return `"device":"mps"`. Lower `steps` from 16 → 8 for drafts | + +## Clean teardown + +```bash +scripts/stop-backend.sh # graceful shutdown +# Uninstall: rm -rf "$OMNIVOICE_HOME" ~/.cache/huggingface/hub/models--k2-fsa--OmniVoice +# Remove the `omnivoice` entry from your MCP client config +``` + +User profiles + history live in the platform data dir (`~/Library/Application Support/OmniVoice/` on macOS; `~/.local/share/OmniVoice/` on Linux). Preserve across reinstalls if you want to keep your saved voice profiles. diff --git a/.claude/skills/omnivoice/scripts/check-health.sh b/.claude/skills/omnivoice/scripts/check-health.sh new file mode 100755 index 00000000..2c57289b --- /dev/null +++ b/.claude/skills/omnivoice/scripts/check-health.sh @@ -0,0 +1,9 @@ +#!/usr/bin/env bash +set -euo pipefail +URL="${OMNIVOICE_API_URL:-http://127.0.0.1:3900}/health" +if out=$(curl -sf --max-time 3 "$URL" 2>/dev/null); then + echo "$out" + exit 0 +fi +echo "omnivoice backend not reachable at $URL" >&2 +exit 1 diff --git a/.claude/skills/omnivoice/scripts/record-reference.sh b/.claude/skills/omnivoice/scripts/record-reference.sh new file mode 100755 index 00000000..9adc8dd8 --- /dev/null +++ b/.claude/skills/omnivoice/scripts/record-reference.sh @@ -0,0 +1,147 @@ +#!/usr/bin/env bash +# Record a clean reference clip for OmniVoice voice cloning. +# +# Usage: +# scripts/record-reference.sh [output.wav] [duration_seconds] [mic_index] +# +# Defaults: +# output = ~/Downloads/omnivoice-ref.wav +# duration = 12 seconds raw capture (trimmed to 10 sec of speech) +# mic_index = 1 (typically MacBook built-in; run `ffmpeg -f avfoundation -list_devices true -i ""` to enumerate) +# +# Why this script exists: +# - macOS Terminal buffers stdout — "speak now" prints arrive AFTER the recording finishes. +# This script uses `say` (macOS TTS) + system sound beeps to give the user *audible* cues +# that bypass terminal buffering. +# - 24 kHz mono is what OmniVoice's diffusion model expects internally; recording natively at +# that rate avoids a resample step. +# - silenceremove + atrim crops the user's actual speech window out of a longer raw capture, +# so the user doesn't have to time their start perfectly. +# +# Cross-platform note: macOS-only (relies on avfoundation, `say`, /System/Library/Sounds). +# Linux equivalent would use `arecord` + `espeak` + `aplay`; not implemented here. +# +# Exit codes: +# 0 success +# 2 environment problem (wrong OS, missing tool) +# 3 user-action problem (mic permission denied — captured silence) +# 4 verification failed (trimmed reference too short) + +set -euo pipefail + +# Platform guard FIRST — before mktemp / trap / anything else macOS-specific +if [ "$(uname)" != "Darwin" ]; then + echo "✗ this helper is macOS-only (uses avfoundation, say, /System/Library/Sounds)." >&2 + echo " Linux equivalent: arecord + espeak + aplay; not implemented." >&2 + exit 2 +fi + +if ! command -v ffmpeg >/dev/null 2>&1; then + echo "✗ ffmpeg not found — install via brew install ffmpeg" >&2 + exit 2 +fi + +if ! command -v ffprobe >/dev/null 2>&1; then + echo "✗ ffprobe not found — install via brew install ffmpeg" >&2 + exit 2 +fi + +OUT="${1:-$HOME/Downloads/omnivoice-ref.wav}" +DUR="${2:-12}" +MIC="${3:-1}" + +RAW="$(mktemp -t omnivoice-raw-XXXXX).wav" +# Cover all common termination paths so the temp file never leaks +trap 'rm -f "$RAW"' EXIT INT TERM HUP + +# Beep helper — uses system sound if available, otherwise terminal bell. +beep() { + local snd="$1" + if [ -f "$snd" ]; then + afplay "$snd" & + else + printf '\a' >&2 + fi +} + +PING="/System/Library/Sounds/Ping.aiff" +POP="/System/Library/Sounds/Pop.aiff" + +echo "▶︎ Recording reference for OmniVoice voice cloning" +echo " mic index: $MIC (run \`ffmpeg -f avfoundation -list_devices true -i \"\"\` to list)" +echo " raw window: ${DUR}s · output: $OUT" +echo + +# Spoken instructions + countdown (heard in real time, bypasses terminal buffering) +say -r 180 "Recording in three seconds. After the high beep, speak your reference phrase. Recording continues for ${DUR} seconds." +sleep 0.3 +say -r 220 "three"; say -r 220 "two"; say -r 220 "one" + +# Start cue +beep "$PING" + +# Capture +ffmpeg -hide_banner -loglevel error -y -f avfoundation -i ":$MIC" -t "$DUR" -ac 1 -ar 24000 "$RAW" + +# End cue +beep "$POP" + +echo "✓ raw captured" + +# Sanity-check levels — bail loudly if the recording is silent (mic permission denied is the +# most common cause: macOS gives the calling process a silent stream and ffmpeg returns 0). +LEVELS=$(ffmpeg -hide_banner -i "$RAW" -af "volumedetect" -f null - 2>&1) +echo "raw levels:" +echo "$LEVELS" | grep -E "mean_volume|max_volume" | sed 's/^/ /' + +MEAN=$(echo "$LEVELS" | grep -oE "mean_volume:[[:space:]]*-?[0-9.]+" | grep -oE "\-?[0-9.]+$" | head -1) +if [ -z "$MEAN" ]; then + echo "✗ could not parse audio levels — ffmpeg may have failed to record" >&2 + exit 3 +fi +# Compare via awk (bash can't do floating-point natively) +if awk -v m="$MEAN" 'BEGIN { exit !(m < -50) }'; then + echo "✗ recording is silent (mean ${MEAN} dB < -50 dB)." >&2 + echo " Most likely: macOS denied microphone access to the calling process." >&2 + echo " Fix: System Settings → Privacy & Security → Microphone → enable for your terminal" >&2 + echo " (or for the agent harness). Then re-run this script." >&2 + exit 3 +fi + +# Trim leading silence + take first ~10 seconds of speech (or all of it if shorter) +TARGET=10 +ffmpeg -hide_banner -loglevel error -y -i "$RAW" \ + -af "silenceremove=start_periods=1:start_silence=0.05:start_threshold=-40dB,atrim=end=${TARGET}" \ + -ac 1 -ar 24000 "$OUT" + +# Verify the trim produced a usable reference (≥ 2.0s of speech) +REF_DUR=$(ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 "$OUT" 2>/dev/null || echo "0") +if awk -v d="$REF_DUR" 'BEGIN { exit !(d < 2.0) }'; then + echo "✗ trimmed reference is only ${REF_DUR}s (< 2s minimum)." >&2 + echo " silenceremove found very little speech above -40 dB threshold." >&2 + echo " Likely cause: you spoke too softly, or the mic captured mostly background noise." >&2 + echo " Try again, speak closer to the mic." >&2 + exit 4 +fi + +echo "✓ trimmed reference: $OUT" +ffmpeg -hide_banner -i "$OUT" -af "volumedetect" -f null - 2>&1 \ + | grep -E "Duration|mean_volume|max_volume" \ + | sed 's/^/ /' + +# Auto-play back so the user can verify before POSTing. +# Don't swallow stderr — if playback fails the user should know. +echo +echo "▶︎ playing reference for verification..." +if ! afplay -v 3 "$OUT"; then + echo "⚠ verification playback failed — afplay exited non-zero. The file may still be valid;" >&2 + echo " try \`afplay $OUT\` manually or open it in a player to confirm." >&2 +fi +echo "✓ done" +echo +echo "Next: POST to /profiles to create a voice profile:" +echo " curl -X POST http://127.0.0.1:3900/profiles \\" +echo " -F \"name=my-voice\" \\" +echo " -F \"ref_audio=@$OUT\" \\" +echo " -F \"ref_text=\" \\" +echo " -F \"language=English\"" diff --git a/.claude/skills/omnivoice/scripts/start-backend.sh b/.claude/skills/omnivoice/scripts/start-backend.sh new file mode 100755 index 00000000..196db02b --- /dev/null +++ b/.claude/skills/omnivoice/scripts/start-backend.sh @@ -0,0 +1,72 @@ +#!/usr/bin/env bash +# Start the OmniVoice FastAPI backend on 127.0.0.1:3900, detached, idempotent. +# Honors $OMNIVOICE_HOME (default ~/OmniVoice-Studio). +# +# Exit codes: +# 0 success (already running, or freshly started + healthy within 60s) +# 2 $OMNIVOICE_HOME doesn't exist +# 3 port 3900 held but /health unresponsive (won't auto-kill — caller decides) +# 4 backend started but /health didn't respond within 60s +# 5 uvicorn process died during the wait + +set -euo pipefail + +HOME_DIR="${OMNIVOICE_HOME:-$HOME/OmniVoice-Studio}" +URL="${OMNIVOICE_API_URL:-http://127.0.0.1:3900}" +LOG="$HOME_DIR/backend.log" + +if [ ! -d "$HOME_DIR" ]; then + echo "OMNIVOICE_HOME not found: $HOME_DIR" >&2 + echo "See references/mcp-setup.md for install steps." >&2 + exit 2 +fi + +# Already up? +if curl -sf --max-time 2 "$URL/health" >/dev/null 2>&1; then + echo "already running: $URL" + curl -sf "$URL/health"; echo + exit 0 +fi + +# Port held by something else? Identify it before refusing. +BIND_PID="$(lsof -nP -iTCP:3900 -sTCP:LISTEN -t 2>/dev/null | head -1)" +if [ -n "$BIND_PID" ]; then + BIND_CMD="$(ps -o command= -p "$BIND_PID" 2>/dev/null || echo "?")" + echo "port 3900 held by PID $BIND_PID but /health not responding — investigate before starting" >&2 + echo " bound process: $BIND_CMD" >&2 + case "$BIND_CMD" in + *uvicorn*main:app*) + echo " → looks like a stale uvicorn from a previous run; consider scripts/stop-backend.sh" >&2 + ;; + *) + echo " → unknown process holds the port; resolve before re-running this script" >&2 + ;; + esac + exit 3 +fi + +cd "$HOME_DIR" +nohup uv run uvicorn main:app --app-dir backend --host 127.0.0.1 --port 3900 \ + > "$LOG" 2>&1 & +PID=$! +echo "starting backend (PID $PID, log: $LOG)..." + +# Wait up to 60s for /health, AND verify the child stays alive. +# A dead child means immediate bind failure (port grabbed in the TOCTOU window above) +# or an early crash — surface it instead of waiting the full timeout. +for i in $(seq 1 30); do + sleep 2 + if ! kill -0 "$PID" 2>/dev/null; then + echo "✗ uvicorn (PID $PID) exited during startup — see $LOG" >&2 + tail -20 "$LOG" >&2 || true + exit 5 + fi + if curl -sf --max-time 2 "$URL/health" >/dev/null 2>&1; then + curl -sf "$URL/health"; echo + echo "ready after $((i*2))s" + exit 0 + fi +done + +echo "backend did not respond on $URL/health within 60s — see $LOG" >&2 +exit 4 diff --git a/.claude/skills/omnivoice/scripts/stop-backend.sh b/.claude/skills/omnivoice/scripts/stop-backend.sh new file mode 100755 index 00000000..a38388d8 --- /dev/null +++ b/.claude/skills/omnivoice/scripts/stop-backend.sh @@ -0,0 +1,70 @@ +#!/usr/bin/env bash +# Gracefully stop the OmniVoice backend bound to 127.0.0.1:3900. +# +# Exit codes: +# 0 stopped (or never running) +# 1 process(es) still bound to 3900 after SIGTERM + SIGKILL escalation +# 2 permission denied killing a bound process (EPERM — try sudo or a different account) + +set -euo pipefail + +# Helper: discover current PIDs bound to 3900 (recomputed each time to avoid stale data) +current_pids() { + lsof -nP -iTCP:3900 -sTCP:LISTEN -t 2>/dev/null || true +} + +PIDS="$(current_pids)" +if [ -z "$PIDS" ]; then + echo "no listener on 3900" + exit 0 +fi + +# SIGTERM phase — let processes shut down cleanly. Surface EPERM loudly so the user +# knows when they can't actually stop the backend (wrong user, sandboxed process, etc.). +EPERM_HIT=0 +for pid in $PIDS; do + echo "kill -TERM $pid" + if ! kill -TERM "$pid" 2>/tmp/.omnivoice-kill-err; then + if grep -qi 'permitted\|denied' /tmp/.omnivoice-kill-err 2>/dev/null; then + echo " ✗ EPERM — cannot signal PID $pid (different user / sandboxed)" >&2 + EPERM_HIT=1 + elif grep -qi 'no such process' /tmp/.omnivoice-kill-err 2>/dev/null; then + : # benign — process already gone + else + cat /tmp/.omnivoice-kill-err >&2 || true + fi + fi +done +rm -f /tmp/.omnivoice-kill-err + +if [ "$EPERM_HIT" -eq 1 ]; then + echo "✗ at least one bound process refused SIGTERM (permission denied)." >&2 + echo " Try \`sudo $(realpath "$0")\` or stop the owning process manually." >&2 + exit 2 +fi + +# Wait up to 10s for graceful exit (poll lsof, not the captured PID list — PIDs may have been +# reaped or recycled by the kernel during this window). +for i in $(seq 1 5); do + sleep 2 + if [ -z "$(current_pids)" ]; then + echo "stopped" + exit 0 + fi +done + +# Escalate to SIGKILL on whatever is currently bound (re-query — don't trust stale PIDs). +echo "still running after 10s — escalating to SIGKILL" >&2 +PIDS="$(current_pids)" +for pid in $PIDS; do kill -KILL "$pid" 2>/dev/null || true; done +sleep 1 + +# Verify the port is actually free now. If it's still held, the script failed its job. +if [ -n "$(current_pids)" ]; then + echo "✗ port 3900 STILL bound after SIGKILL:" >&2 + lsof -nP -iTCP:3900 -sTCP:LISTEN 2>&1 | head -3 >&2 + exit 1 +fi + +echo "stopped (after SIGKILL)" +exit 0 diff --git a/.gitignore b/.gitignore index 997e0065..3773ef0f 100644 --- a/.gitignore +++ b/.gitignore @@ -38,7 +38,11 @@ Thumbs.db # ───────────────────────────────────────────────────────────────────────── # Editor / tool caches # ───────────────────────────────────────────────────────────────────────── -.claude/ +# Ignore ad-hoc Claude Code state, but allow project-bundled skills +# (CLAUDE.md invites `.claude/skills//SKILL.md`). +.claude/* +!.claude/skills/ +!.claude/skills/** /.cache* /.tmp/ diff --git a/backend/mcp_server.py b/backend/mcp_server.py index 03762924..f23eb0ec 100644 --- a/backend/mcp_server.py +++ b/backend/mcp_server.py @@ -48,8 +48,7 @@ def create_mcp_server(): FastMCP = _ensure_mcp() mcp = FastMCP( "OmniVoice Studio", - version="0.3.0", - description=( + instructions=( "AI-agent interface for OmniVoice Studio — voice cloning, " "voice design, and video dubbing in 646 languages." ),