Files

T

Carlos D. Escobar-Valbuena fa9c7d43ca feat: bundle Claude Code agent skill at .claude/skills/omnivoice/ (#113 )

* fix(mcp): drop unsupported FastMCP kwargs (mcp SDK >= 1.10)

The MCP server passes `version=` and `description=` to FastMCP(), but
neither kwarg exists on mcp >= 1.10 — the protocol version is now
managed internally and `description` was renamed to `instructions`.

Symptom on a fresh install (uv sync && pip install 'mcp[cli]'):

    TypeError: FastMCP.__init__() got an unexpected keyword argument 'version'

Tested locally end-to-end:
- create_mcp_server() now constructs cleanly
- All 5 tools register and are listable via FastMCP.list_tools()
- generate_speech round-trip returns base64 WAV; ~24s server-side
  for 4.2s of audio at steps=16 on Apple Silicon MPS
- pytest backend/ -x -q: 45 passed

* feat: bundle Claude Code agent skill at .claude/skills/omnivoice/

CLAUDE.md already invites contributions at .claude/skills/:

  "No project skills found. Add skills to any of: .claude/skills/,
   .agents/skills/, .cursor/skills/, .github/skills/, or .codex/skills/
   with a SKILL.md index file."

But the existing .gitignore blanket-ignored .claude/ (line 41), making
the invited path un-trackable. This commit narrows the ignore so ad-hoc
Claude state stays out while deliberate skill bundles are tracked:

    -.claude/
    +.claude/*
    +!.claude/skills/
    +!.claude/skills/**

Once merged, any compatible agent client running
`npx skills add debpalash/OmniVoice-Studio` gets immediate context on:

- What the MCP server exposes (5 tools + 2 resources)
- When to pick OmniVoice vs other engines
- How to wire the stdio MCP server into a client config
- Backend lifecycle: start / health / stop scripts
- Common failure modes + fixes (port collision, model download stall,
  missing HF_TOKEN, MPS fallback, voice-profile-not-found, etc.)

Conforms to Anthropic skill-creator conventions: frontmatter
description under 1024-char limit, body under 500 lines, references/
for detail, scripts/ for deterministic ops, no README/CHANGELOG
inside the skill, validates clean against quick_validate.py.

Verified locally that `npx skills list` discovers the bundled skill
automatically once cloned. End-to-end tested through MCP:
- generate_speech (English, demo voice, steps=16) -> 4.2 s WAV
- generate_speech (voice design via instruct only, steps=8) -> 6.3 s WAV
- generate_speech (Spanish, demo voice, steps=16) -> 2.8 s WAV

Depends on #112 (FastMCP API fix). Without it, every MCP tool call
fails with TypeError at server construction.

* feat(skill): add voice-clone end-to-end recipe + record-reference.sh helper

Two additions to the bundled skill, closing the gap where agents had no
procedural knowledge for creating a voice profile (the previous SKILL.md
said "use the UI or POST /profiles" but didn't include the recording +
trimming + verification workflow).

1. scripts/record-reference.sh — macOS-only helper that records a clean
   reference clip with **audible** countdown + start/stop cues via
   `say` + /System/Library/Sounds/Ping.aiff. Solves the buffering bug
   where text-mode "speak now" prompts arrive after recording starts.
   Captures a longer raw window then trims to ~10 sec of speech via
   silenceremove + atrim. Plays back for verification. Prints the
   next-step `curl` command for POST /profiles.

2. SKILL.md "Voice clone — end-to-end recipe" section (replaces the
   stub one-liner). Covers:
   - Path A: the bundled helper (one command, audible cues)
   - Path B: manual ffmpeg flow if the helper doesn't fit
   - POST /profiles multipart/form-data fields (required: name +
     ref_audio; optional: ref_text, language, instruct, seed, personality)
   - Reference clip quality factors that materially affect output
     (single speaker, natural prosody, 3-10 sec sweet spot, ref_text
     alignment, language correctness, loudness ≥ -15 dB peak)

Tested locally: recorded a 10-sec Spanish reference + 3-sec English
reference, created two profiles via the helper + curl flow, generated
14.1 sec of Spanish + 10.2 sec of English audio in the user's cloned
voice. Round-trip works end-to-end at steps=16 on Apple Silicon MPS.

Frontmatter description unchanged (860 chars, under the 1024 limit).
Body grew from ~120 to 169 lines (still well under the 500-line skill
ceiling).

* fix(skill): address P20 cross-review findings on PR #113

Adversarial multi-agent review (code + comment + silent-failure analyzers
on parallel reviewers) surfaced one blocker, one critical silent-failure
class, two medium-severity bugs, and two minor doc inaccuracies. All
addressed in this commit.

Blocker (cited 3x by both code-reviewer and comment-analyzer):
- SKILL.md linked references/engines-comparison.md three times (lines 44,
  153, 160) but the file was never copied into the upstream skill tree.
  + Added the file (engine decision tree across OmniVoice / kokoro /
    Voicebox / Edge TTS / ElevenLabs / cloud APIs).

Critical — record-reference.sh (was 4/10):
- Mic-permission silent failure: macOS denies the mic by sending a silent
  stream; ffmpeg exits 0 with a valid silent WAV. The script printed
  "✓ raw captured" and produced a degenerate reference clip that would
  train a broken voice profile.
  + Parse mean_volume from volumedetect; exit 3 with a diagnostic
    pointing the user to System Settings → Privacy → Microphone if
    the recording is below -50 dB.
- afplay backgrounded with no exit check; if /System/Library/Sounds/*.aiff
  is missing the user gets no audible cue.
  + beep() helper falls back to printf '\a' (terminal bell) when the
    system sound file is missing.
- silenceremove silent corruption: silent input → near-empty output WAV,
  exit 0.
  + ffprobe duration check after trim; exit 4 if < 2.0 sec.
- trap only covered EXIT; Ctrl-C / SIGTERM mid-recording leaked tmp file.
  + trap '...' EXIT INT TERM HUP.
- macOS guard ran after mktemp + trap.
  + Moved guard to first executable line.
- afplay verification swallowed stderr.
  + Drop 2>/dev/null; surface failure as a warning.
- Documented exit codes in header (0/2/3/4).

Medium — start-backend.sh (was 6/10):
- TOCTOU race: lsof check → uvicorn start could lose the port to another
  process; only signal was a 60s health timeout.
  + Added `kill -0 $PID` check inside the probe loop; immediate exit 5
    with log tail if uvicorn died.
- lsof check couldn't tell "stale us" from "third party" — same exit 3
  for both.
  + ps -o command attribution; the message now tells the user whether
    it's a stale uvicorn (suggest stop-backend.sh) or unknown process.
- Documented exit codes (0/2/3/4/5).

Medium — stop-backend.sh (was 7/10):
- No post-SIGKILL verification — script exited 0 even if process still
  bound.
  + Added current_pids() helper; re-query after SIGKILL; exit 1 if still
    bound, with lsof dump for diagnostics.
- 2>/dev/null || true on kill swallowed EPERM silently.
  + Capture stderr; classify EPERM vs ESRCH; exit 2 on EPERM with
    actionable hint (try sudo).
- Documented exit codes (0/1/2).

Minor docs (comment-analyzer):
- SKILL.md line 120 claimed profiles persist as `<id>.wav`. Actual
  backend (profiles.py:48-50) preserves uploaded extension.
  + Reworded to `<id>.<ext>` with explanation.
- mcp-setup.md line 68 cited HF cache path as Linux/macOS only.
  Windows redirects via backend/core/config.py:38 to
  %LOCALAPPDATA%\OmniVoice\hf_cache.
  + Added Windows row + reference to config.py.

Re-validated: all 6 files compile under set -euo pipefail; SKILL.md
frontmatter description stays at 860 chars (under 1024 cap); skill body
under 500 lines.

Diff: 6 files changed, ~+269/-47.

2026-05-29 09:38:41 +05:30

3.8 KiB

Raw Blame History

TTS Engine Selection — Decision Tree

When to pick OmniVoice vs other engines available in this workspace. Match the user's constraint to the right column.

Decision tree

Is voice cloning required?
├─ yes → OmniVoice (3-sec ref clip, zero-shot, 646 langs)
└─ no →
   Is the language non-English?
   ├─ yes → OmniVoice (646 langs) or Edge TTS (subset, cloud)
   └─ no (English) →
      Is privacy required (no cloud)?
      ├─ yes →
      │  Is GPU available?
      │  ├─ yes (CUDA/MPS) → OmniVoice (best quality) or Voicebox
      │  └─ no (CPU only) → kokoro-tts (2× realtime CPU) or OmniVoice on CPU (slow)
      └─ no (cloud OK) →
         Is cost-no-object?
         ├─ yes → ElevenLabs (best polish), then OpenAI TTS
         └─ no → Edge TTS (free, unofficial, MS Azure neural)

Full comparison

Engine	Quality	Clone	Multilingual	Cost	Privacy	Setup	Best for
OmniVoice	8-9/10	✅ 3-sec ref	646 langs	Free	Local	Bun + uv install	Multilingual, cloning, privacy-critical
ElevenLabs	9-10/10	✅ 3-sec ref	32 langs	$5-330/mo	Cloud	API key	Best English polish, fastest cloud TTS
Voicebox (Qwen3-TTS)	8-9/10	✅	Multi	Free	Local	Docker	Self-hosted alternative to OmniVoice
Voicebox (LuxTTS)	7/10	❌	Multi	Free	Local	Docker	CPU at 150× realtime
kokoro-tts	7-8/10	❌	Multi (limited)	Free	Local	pip	Fast English narration on CPU
mlx-audio	7-8/10	varies	Multi	Free	Local	pip	Apple Silicon native, 14+ sub-engines
Edge TTS	7-8/10	❌	50+	Free*	Cloud	pip	Zero-friction one-off
OpenAI TTS	8/10	❌	Multi	$0.015/1k chars	Cloud	API key	Convenient, cheap-ish, good quality
Google Cloud TTS	8/10	❌	Multi	$4/1M chars (WaveNet)	Cloud	GCP project	Large free tier (1M chars/mo)

*Edge TTS is unofficial. Microsoft could block it at any time.

When OmniVoice wins decisively

Voice cloning — 3-sec reference clip, zero-shot, no fine-tuning. ElevenLabs is the only competitor; OmniVoice is free and local.
Long-tail languages — 646 supported. ElevenLabs covers 32; everything else fewer.
Privacy / regulatory — Nothing leaves the machine. ElevenLabs and OpenAI ship audio to their servers.
No-API-key constraint — Local-first. No accounts.
Bulk generation without metered cost — ElevenLabs bills per character. OmniVoice is free at any volume.

When OmniVoice loses

Lowest-friction one-off TTS — Backend install + ~3 GB model + uvicorn boot. Edge TTS or OpenAI TTS is one command.
Fast English narration on weak hardware — kokoro-tts is ~30 MB vs OmniVoice's 2.4 GB and runs 2× realtime on CPU. Use kokoro for blog-narration batch jobs unless you need cloning.
Streaming real-time TTS — OmniVoice is diffusion-based and not streaming. Use Edge TTS or cloud APIs for true streaming.
Apple Silicon-only specialized voices — mlx-audio ships 14 engines (Kokoro, CSM, Dia, Qwen3-TTS, etc.) that may match a specific voice better.

Composition with content pipelines

OmniVoice fits between visual asset generation and video assembly:

research → narrative → visual assets → AUDIO (OmniVoice) → video assembly → distribution

Default for blog-post audio narration:

English, no cloning needed, fast → kokoro-tts (cheap CPU)
English, want a specific cloned voice → OmniVoice with a saved profile
Non-English → OmniVoice
One-time, no install → Edge TTS

For Remotion-based video pipelines that previously required ElevenLabs, OmniVoice closes the last cloud dependency — pair it with any local image/video generator for a fully self-hosted multimedia stack.

3.8 KiB Raw Blame History Unescape Escape