diff --git a/.claude/skills/omnivoice/SKILL.md b/.claude/skills/omnivoice/SKILL.md index 1c7fa9f5..c1a0dd64 100644 --- a/.claude/skills/omnivoice/SKILL.md +++ b/.claude/skills/omnivoice/SKILL.md @@ -1,172 +1,29 @@ --- name: omnivoice -description: "Local TTS, voice cloning, voice design, and video dubbing via the VoiceStudio MCP server (open-source ElevenLabs alternative; nothing leaves the machine, runs on MPS/CUDA/CPU). Use when: (1) generating speech from text in any of 646 languages, (2) cloning a voice from a 3-second reference clip, (3) designing a voice by gender/age/accent/pitch/style, (4) dubbing a video into another language, (5) listing voice profiles or personality presets, (6) producing narration where privacy, cost, or absent API keys matter, (7) non-English narration where Edge TTS/kokoro fall short, (8) batch audio for blog posts or content pipelines. Triggers: 'omnivoice', 'voice clone', 'clone this voice', 'tts', 'narrate', 'generate speech', 'voice synthesis', 'dub video', 'voice design', 'local tts', 'multilingual voice', 'narrate this post', 'elevenlabs alternative'." +description: Legacy VoiceStudio skill alias for existing Claude installations. Generate local speech, discover saved voices, and transcribe audio through the running VoiceStudio backend. --- -# VoiceStudio +# VoiceStudio compatibility entry -The canonical cross-agent package lives at `skills/omnivoice/SKILL.md`. This -Claude-specific package retains the MCP lifecycle helpers and references. +The current cross-agent package is [voicestudio](../../../skills/voicestudio/SKILL.md). +For new installations use `npx skills add debpalash/VoiceStudio --skill voicestudio`. -## Overview +Use the running backend at the user's configured address (default +`http://localhost:3900`). Check `/health`, discover `/openapi.json` and +`/v1/audio/voices`, then use the installed schema for speech, transcription, +profiles, and jobs. The HTTP MCP endpoint is `/mcp`; discover tools from the +connected server instead of assuming this older package's tool inventory. -Generate audio locally via the VoiceStudio MCP server. Tools: `generate_speech`, `list_voices`, `list_personalities`, `list_languages`, `check_health`. Resources: `voice://{id}`, `history://recent`. +Launch the installed Electron app if the backend is unavailable. For source +development follow the checkout's Electron README. Existing helpers in +`scripts/` support legacy source installations; inspect their environment +and dependency assumptions before running them. -## Prerequisites — Backend Must Be Running +Model downloads and remote services require the user's choice. Never silently +install models, promise fixed latency, or treat compatibility voice names as +real provider voices. Validate saved audio and asynchronous job completion +before reporting success. Protected backends require configured credentials; +never disable authentication to make an example work. -The MCP tools all hit `$OMNIVOICE_API_URL` (default `http://localhost:3900`). If the backend is down, every tool returns a connection error. Install + boot: - -```bash -git clone https://github.com/debpalash/VoiceStudio.git "$OMNIVOICE_HOME" -cd "$OMNIVOICE_HOME" -uv sync -VIRTUAL_ENV="$(pwd)/.venv" uv pip install 'mcp[cli]' -``` - -Then: - -```bash -scripts/check-health.sh # exit 0 if up -scripts/start-backend.sh # boot in background (MPS/CUDA auto-detected) -``` - -First synthesis call lazy-downloads the `k2-fsa/OmniVoice` model (~2.4 GB) from HuggingFace — cached on subsequent boots. - -## Task Index — Pick the Right Tool - -| Task | Tool | Notes | -|---|---|---| -| Verify backend is up | `check_health` | Returns `{"status":"ok","device":"mps|cuda|cpu"}` | -| Text → audio with a saved voice | `generate_speech(text, profile_id)` | Returns base64 WAV. `profile_id="demo0001"` is the bundled demo voice | -| Text → audio without a clone (voice design) | `generate_speech(text, instruct="…")` | Omit `profile_id`; pass an `instruct` like `"warm middle-aged female narrator, calm pace"` | -| Multilingual narration | `generate_speech(text, language="es")` | Any ISO 639 code or `"Auto"` | -| List existing voices | `list_voices` | Returns id, name, type, personality | -| List personality presets | `list_personalities` | Returns narrator / casual / news-anchor / etc. with their `instruct` strings | -| List supported languages | `list_languages` | 646 total; returns 20 popular + the full count | - -For non-trivial decisions (which engine to use, when to pick VoiceStudio over kokoro / Edge TTS / ElevenLabs), see [references/engines-comparison.md](references/engines-comparison.md). - -For MCP wiring details, backend lifecycle, troubleshooting, and a clean teardown, see [references/mcp-setup.md](references/mcp-setup.md). - -## Common Workflows - -### 1. One-shot narration with the demo voice - -```python -# As called through the MCP client (your agent will do this for you): -result = generate_speech( - text="Hello — this is VoiceStudio generating speech locally.", - profile_id="demo0001", - language="English", - steps=16, # 8 = fast/draft · 16 = balanced · 32 = quality -) -# result is JSON with audio_id, generation_time_s, audio_duration_s, format, wav_base64 -``` - -Benchmark: 4.2 s of audio in ~24 s server-side on Apple Silicon MPS at 16 diffusion steps. - -### 2. Save the WAV to disk and play - -Tool returns base64 PCM WAV (16-bit, mono, 24 kHz). Decode + write: - -```python -import base64, json -payload = json.loads(result_text) # parse JSON the tool returns -open("out.wav","wb").write(base64.b64decode(payload["wav_base64"])) -``` - -On macOS: `afplay out.wav`. Convert to MP3 with `ffmpeg -i out.wav -codec:a libmp3lame -b:a 128k out.mp3`. - -### 3. Voice clone — end-to-end recipe - -Cloning needs a 3-10 second reference clip the model will use as a speaker embedding. The MCP server does NOT expose profile creation — it only reads existing profiles. Two paths to create one: - -**Path A — bundled helper (macOS, recommended for fresh clones):** - -```bash -scripts/record-reference.sh ~/Downloads/my-ref.wav 12 1 -# args: output_path raw_duration_sec mic_index -# Default mic_index=1 (MacBook built-in); list devices via: -# ffmpeg -f avfoundation -list_devices true -i "" -``` - -The script gives **audible** countdown + start/stop cues via macOS `say` + `/System/Library/Sounds/Ping.aiff` so the user knows when to speak (terminal stdout is buffered — text "speak now" prompts arrive too late). It records a longer raw window, then trims to ~10 seconds of speech via `silenceremove + atrim`, plays back for verification, and prints the next-step `curl` command. - -**Path B — manual:** - -```bash -# 1. Record (mono, 24 kHz native — matches model's internal rate) -ffmpeg -f avfoundation -i ":1" -t 12 -ac 1 -ar 24000 raw.wav - -# 2. Trim leading silence + take first 10 sec of speech -ffmpeg -i raw.wav \ - -af "silenceremove=start_periods=1:start_silence=0.05:start_threshold=-40dB,atrim=end=10" \ - -ac 1 -ar 24000 ref.wav - -# 3. Verify -ffmpeg -i ref.wav -af volumedetect -f null - 2>&1 | grep volume # max should be > -20 dB -afplay ref.wav -``` - -**POST to /profiles** (multipart/form-data — required fields: `name`, `ref_audio`): - -```bash -curl -X POST http://127.0.0.1:3900/profiles \ - -F "name=carlos-clone" \ - -F "ref_audio=@ref.wav" \ - -F "ref_text=The exact text spoken in the clip" \ - -F "language=English" \ - | python3 -m json.tool -# returns { "id": "abc12345", "name": "carlos-clone" } -``` - -Once created, pass `profile_id` to `generate_speech` (via MCP) or directly via `POST /generate`. Profiles persist in SQLite + reference-audio files at `~/Library/Application Support/OmniVoice/voices/.` (the backend preserves the uploaded extension — `.wav` if you uploaded a WAV, `.mp3` if MP3, etc.). State persists across backend restarts. - -**Reference clip tips that materially affect quality:** - -| Factor | Why it matters | -|---|---| -| Single speaker | Mixed speakers blur the embedding | -| Clean speech, no music/noise | Model embeds the noise too | -| Natural prosody (avoid pangrams) | Diffusion samples replicate prosody, not just timbre | -| 3-10 sec is the sweet spot | < 3 s lacks information; > 10 s adds compute without quality gain | -| Match `ref_text` to what's spoken | Improves alignment, especially on noisy refs | -| `language` correct | Wrong language → cross-lingual transfer artifacts | -| Loudness peak ≥ -15 dB | Quiet refs work but normalize poorly | - -### 4. Voice design (no reference clip) - -Skip `profile_id`; provide an `instruct` string describing the desired voice: - -```python -generate_speech( - text="Welcome to the future of agentic systems.", - instruct="warm middle-aged female narrator, calm authoritative pace, documentary style", -) -``` - -Get pre-made instructs via `list_personalities` and copy the one matching the brief (narrator, casual, news-anchor, etc.). - -### 5. Video dubbing (web UI only) - -The MCP server does not expose the dubbing endpoint. The full transcribe → translate → re-voice → mux pipeline lives behind the desktop UI (`bun run desktop` in `$OMNIVOICE_HOME`) and the `/dub/*` REST routes. When the user asks to dub a video, point them to the UI; surface this skill only for the synthesis primitives above. - -## When NOT to use VoiceStudio - -- **Fast English-only narration on weak hardware** → `kokoro-tts` is ~10× smaller and 2× realtime on CPU (see [references/engines-comparison.md](references/engines-comparison.md)) -- **Lowest-friction one-off TTS** → Edge TTS needs no install or backend -- **Highest possible quality regardless of cost** → ElevenLabs still wins on English narration polish; VoiceStudio ties or wins on multilingual + cloning -- **Real-time streaming dictation** → use the VoiceStudio desktop widget (`⌘+⇧+Space`), not the MCP server - -## Resources - -- [references/engines-comparison.md](references/engines-comparison.md) — Decision tree across VoiceStudio / kokoro / Voicebox / Edge TTS / ElevenLabs / cloud APIs -- [references/mcp-setup.md](references/mcp-setup.md) — MCP wiring, backend lifecycle, env vars, troubleshooting -- [scripts/check-health.sh](scripts/check-health.sh) — `curl /health`, exit 0/1 -- [scripts/start-backend.sh](scripts/start-backend.sh) — Start uvicorn on 127.0.0.1:3900 with health probe -- [scripts/stop-backend.sh](scripts/stop-backend.sh) — Clean shutdown via `kill -TERM` on the bound PID -- [scripts/record-reference.sh](scripts/record-reference.sh) — macOS-only: record + trim + verify a reference clip for cloning, with audible cues (`say` + system beeps) that bypass terminal output buffering - -Backend Swagger / OpenAPI: `http://127.0.0.1:3900/docs` (when backend is up). - -Upstream: github.com/debpalash/VoiceStudio. The app uses AGPL-3.0-only; optional engines and downloaded models retain their own licenses. See `LICENSE-NOTICE.md` in the repository. +Source and current setup documentation: +https://github.com/debpalash/VoiceStudio diff --git a/README.md b/README.md index a40b1ada..5a2182bb 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,6 @@ ![A tour of the Electron app: voice cloning, voice design, dubbing, and model management](docs/media/electron/voicestudio.gif) -

Captured from the running Electron preview with the bundled demo voice. Published installers may look different.

## Create with VoiceStudio @@ -79,7 +78,7 @@ See [Electron setup](electron/README.md) for prerequisites and backend configura | Integrations | [Local API](docs/speech-platform.md) · [MCP](docs/mcp.md) · [Examples](examples/README.md) | | Development | [Contributing](.github/CONTRIBUTING.md) · [Electron](electron/README.md) · [Changelog](CHANGELOG.md) | -Agent skills: `npx skills add debpalash/VoiceStudio` +Agent skills: `npx skills add debpalash/VoiceStudio` — choose **voicestudio** for audio workflows or **oss-maintainer** for repository maintenance. ## Support VoiceStudio diff --git a/docs/media/electron/dubbing.png b/docs/media/electron/dubbing.png index 423a4849..57d937c3 100644 Binary files a/docs/media/electron/dubbing.png and b/docs/media/electron/dubbing.png differ diff --git a/docs/media/electron/models.png b/docs/media/electron/models.png index 5eb259fa..c059e165 100644 Binary files a/docs/media/electron/models.png and b/docs/media/electron/models.png differ diff --git a/docs/media/electron/voice-cloning.png b/docs/media/electron/voice-cloning.png index 8ce05052..10330c6e 100644 Binary files a/docs/media/electron/voice-cloning.png and b/docs/media/electron/voice-cloning.png differ diff --git a/docs/media/electron/voice-design.png b/docs/media/electron/voice-design.png index e003718a..fdd5abbe 100644 Binary files a/docs/media/electron/voice-design.png and b/docs/media/electron/voice-design.png differ diff --git a/docs/media/electron/voicestudio.gif b/docs/media/electron/voicestudio.gif index 6dd21960..1d2135eb 100644 Binary files a/docs/media/electron/voicestudio.gif and b/docs/media/electron/voicestudio.gif differ diff --git a/electron/README.md b/electron/README.md index 05dc6d45..3fcc1498 100644 --- a/electron/README.md +++ b/electron/README.md @@ -123,7 +123,7 @@ Appearance and General have direct routes and share a breadcrumb header, searcha sidebar, max-w-4xl scroll frame, grouped sections, and consistent setting rows. Sidebar active/hover surfaces use the shared T3 theme tokens. -The local palette library includes Signal, Canopy, Current, Hearth, and Orchid, with +The local palette library includes VoiceStudio Original, Canopy, Current, Hearth, and Orchid, with upstream light/dark color definitions with VoiceStudio display names from T3 Code (MIT). Each appearance keeps its own selected palette. System mode follows live OS appearance changes; the sidebar toggle explicitly switches to light or dark. Choices persist under @@ -206,3 +206,9 @@ open a form in the browser; they do not publish a voice automatically. Saved voice editor > Export persona downloads a portable `.ovsvoice` bundle. Include voice clip controls whether the original reference accompanies the watermarked preview. Gallery > My Imports accepts the exported bundle again. + +Workspace navigation groups Clone, Design, Profiles, and Gallery under Voice; +Stories and Audiobook under Stories; and single/batch dubbing under Dubbing. +The current workflow opens automatically. Group buttons can expand or collapse +without navigating; the compact rail opens the same destinations in a flyout. +Transcribe, Projects, Tools, and Integrations remain directly accessible. diff --git a/electron/src/renderer/src/components/app-shell/sponsor-footer.css b/electron/src/renderer/src/components/app-shell/sponsor-footer.css index b2da683a..37c0f749 100644 --- a/electron/src/renderer/src/components/app-shell/sponsor-footer.css +++ b/electron/src/renderer/src/components/app-shell/sponsor-footer.css @@ -128,7 +128,7 @@ height: 34px; flex: 0 0 34px; border: 0; - border-radius: 0; + border-radius: 50%; background: transparent; color: var(--primary); filter: drop-shadow(0 3px 6px color-mix(in srgb, var(--primary) 25%, transparent)); @@ -139,7 +139,7 @@ z-index: 0; inset: 5px 4px 3px; content: ''; - clip-path: polygon(50% 0, 100% 100%, 0 100%); + border-radius: 50%; background: radial-gradient(circle at 50% 42%, color-mix(in srgb, var(--primary) 58%, transparent), transparent 65%); filter: blur(2px); opacity: 0.9; diff --git a/electron/src/renderer/src/components/app-shell/sponsor-footer.tsx b/electron/src/renderer/src/components/app-shell/sponsor-footer.tsx index b0e0c2ef..c38e0a00 100644 --- a/electron/src/renderer/src/components/app-shell/sponsor-footer.tsx +++ b/electron/src/renderer/src/components/app-shell/sponsor-footer.tsx @@ -6,6 +6,7 @@ import { SponsorInquiry } from './sponsor-inquiry'; import { ArrowUpRightIcon, BlocksIcon, + CircleIcon, SearchIcon, GemIcon, PlusIcon, @@ -265,7 +266,7 @@ export function SponsorFooter() { } > diff --git a/electron/src/renderer/src/components/app-shell/sponsor-inquiry.css b/electron/src/renderer/src/components/app-shell/sponsor-inquiry.css index 6ee787dc..d6021554 100644 --- a/electron/src/renderer/src/components/app-shell/sponsor-inquiry.css +++ b/electron/src/renderer/src/components/app-shell/sponsor-inquiry.css @@ -82,3 +82,74 @@ @media (max-width: 560px) { .sponsor-inquiry-perks { grid-template-columns: repeat(2, minmax(0, 1fr)); } } + +/* Editorial header: one focal point, three concrete placements, quiet navigation. */ +.sponsor-inquiry-dialog { + gap: 20px; + background: var(--sidebar); +} +.sponsor-inquiry-heading { + display: flex; + align-items: center; + gap: 14px; + padding-right: 24px; +} +.sponsor-inquiry-heading h2 { + font-size: 21px; + line-height: 1.25; + letter-spacing: -0.035em; + font-weight: 600; +} +.sponsor-inquiry-heading p { + margin-top: 4px; + font-size: 13px; + line-height: 1.5; +} +.sponsor-inquiry-hero-icon { + width: 44px; + height: 44px; + flex-basis: 44px; + color: color-mix(in srgb, var(--primary) 45%, var(--sidebar-foreground)); + background: color-mix(in srgb, var(--primary) 8%, var(--sidebar)); + border: 0; + box-shadow: none; +} +.sponsor-inquiry-perks { + display: flex; + flex-wrap: wrap; + gap: 10px 22px; + padding: 0 0 4px; +} +.sponsor-inquiry-perks span { gap: 7px; } +.sponsor-inquiry-perks svg { + width: 15px; + height: 15px; + color: color-mix(in srgb, var(--primary) 40%, var(--sidebar-foreground)); +} +.sponsor-inquiry-perks small { font-size: 12px; line-height: 1.4; } +.sponsor-inquiry-methods { + display: flex; + align-items: center; + justify-content: space-between; + gap: 8px; + border-bottom: 1px solid var(--sidebar-border); + padding-bottom: 10px; +} +.sponsor-inquiry-tabs { + display: flex; + gap: 4px; + border: 0; + background: transparent; +} +.sponsor-inquiry-tab { padding-inline: 14px; } +.sponsor-inquiry-tab[aria-selected='true'] { + border-color: transparent; + background: color-mix(in srgb, var(--sidebar-foreground) 10%, var(--sidebar)); + box-shadow: none; +} +.sponsor-inquiry-form-link { width: auto; padding-inline: 8px; } +@media (max-width: 560px) { + .sponsor-inquiry-heading h2 { font-size: 18px; } + .sponsor-inquiry-dialog { gap: 16px; } + .sponsor-inquiry-perks { gap: 8px 16px; } +} diff --git a/electron/src/renderer/src/components/app-shell/sponsor-inquiry.tsx b/electron/src/renderer/src/components/app-shell/sponsor-inquiry.tsx index e04ba529..216ffcf5 100644 --- a/electron/src/renderer/src/components/app-shell/sponsor-inquiry.tsx +++ b/electron/src/renderer/src/components/app-shell/sponsor-inquiry.tsx @@ -2,13 +2,10 @@ import { useState } from 'react'; import { BlocksIcon, BookOpenIcon, - BadgeCheckIcon, CopyIcon, - DownloadIcon, EyeIcon, ExternalLinkIcon, MailIcon, - ShieldCheckIcon, PinIcon, XIcon, } from 'lucide-react'; @@ -69,57 +66,31 @@ export function SponsorInquiry({ >