docs(readme): add donate CTA, update features/engines/roadmap for v0.3.6

- Add Sponsor/Donate section with Ko-fi, PayPal, GitHub Sponsors links and progress bar ($10/$200 agent bill fund) - Add Ko-fi + GitHub Sponsors badges to header - Expand features table: Audiobook, Stories, Diagnostics, Engine Routing, Portable Personas, Unlimited TTS, Remote Backend, Dictation+LLM (3x4 → 5x4) - Update TTS engines: 6 → 11 (add GPT-SoVITS, Sherpa-ONNX, IndexTTS 2, OmniVoice GGUF, Supertonic 3) - Update ASR engines: 7 → 8 (add isolated Faster-Whisper) - Expand roadmap Shipped with v0.3.6 additions (Longform, engine routing, diagnostics, MCP server, remote backend, reliability, etc.) - Update architecture diagram (100+ endpoints, engine routing) - Add comparison table rows: Audiobook/Stories, TTS count, ASR count, MCP Server, Self-check - Update FAQ engine count to 11 - Fix CTranslate2 link typo - Remove already-shipped Audiobook Editor from Up Next
2026-06-17 12:16:27 +05:30
parent 05369e748a
commit 69b8c7c0bc
1 changed files with 115 additions and 32 deletions
@@ -10,6 +10,8 @@
    <a href="LICENSE"><img src="https://img.shields.io/badge/license-AGPL--3.0-blue?style=flat-square" alt="License" /></a>
    <a href="https://github.com/debpalash/OmniVoice-Studio/issues"><img src="https://img.shields.io/github/issues/debpalash/OmniVoice-Studio?style=flat-square&color=ef4444" alt="Issues" /></a>
    <a href="https://discord.gg/bzQavDfVV9"><img src="https://img.shields.io/badge/Discord-Join_Community-5865F2?style=flat-square&logo=discord&logoColor=white" alt="Discord" /></a>
+    <a href="https://ko-fi.com/debpalash"><img src="https://img.shields.io/badge/Ko--fi-Support_Us-FF5E5B?style=flat-square&logo=ko-fi&logoColor=white" alt="Ko-fi" /></a>
+    <a href="https://github.com/sponsors/debpalash"><img src="https://img.shields.io/badge/GitHub-Sponsor-ff69b4?style=flat-square&logo=github&logoColor=white" alt="GitHub Sponsors" /></a>
  </p>

  <p>
@@ -18,6 +20,7 @@
    <a href="#why-omnivoice-studio">Why OmniVoice Studio?</a> ·
    <a href="#tts-engines">TTS Engines</a> ·
    <a href="#asr-engines">ASR Engines</a> ·
+    <a href="#sponsor--donate">Donate</a> ·
    <a href="#contributing">Contributing</a> ·
    <a href="https://discord.gg/bzQavDfVV9">Discord</a> ·
    <a href="README_CN.md"><strong>简体中文</strong></a>
@@ -58,20 +61,28 @@

 <table>
 <tr>
-  <td align="center" width="33%">
+  <td align="center" width="25%">
    <h3>🎙️ Voice Cloning</h3>
    <p>3-second clip → mirror any voice.<br/><b>646 languages</b>, zero-shot.</p>
  </td>
-  <td align="center" width="33%">
+  <td align="center" width="25%">
    <h3>🎨 Voice Design</h3>
    <p>Gender, age, accent, pitch, speed,<br/>emotion, dialect — <b>dial it in</b>.</p>
  </td>
-  <td align="center" width="33%">
+  <td align="center" width="25%">
    <h3>🎬 Video Dubbing</h3>
    <p>YouTube URL or file → transcribe →<br/>translate → re-voice → <b>MP4</b>.</p>
  </td>
+  <td align="center" width="25%">
+    <h3>📖 Audiobook Editor</h3>
+    <p>Import text, EPUB, or PDF. Auto-chapter,<br/>loudnorm, metadata. Export <b>.m4b</b>.</p>
+  </td>
 </tr>
 <tr>
+  <td align="center" valign="top">
+    <h3>🎭 Stories</h3>
+    <p>Multi-voice editor. Assign voices<br/>per-line, preview, <b>export full cast</b>.</p>
+  </td>
  <td align="center" valign="top">
    <h3>⌨️ Dictation Widget</h3>
    <p><code>⌘+⇧+Space</code> from <b>any app</b>.<br/>Transcribes, auto-pastes, disappears.</p>
@@ -98,6 +109,10 @@
    <h3>🛡️ AI Watermark</h3>
    <p>AudioSeal (Meta). <b>Invisible</b>,<br/>survives compression.</p>
  </td>
+  <td align="center" valign="top">
+    <h3>🔬 Diagnostics</h3>
+    <p>Self-check, error journal,<br/>scrubbed <b>diagnostic bundle</b>.</p>
+  </td>
 </tr>
 <tr>
  <td align="center" valign="top">
@@ -110,7 +125,29 @@
  </td>
  <td align="center" valign="top">
    <h3>🧩 Extensible</h3>
-    <p>Subclass <code>TTSBackend</code>,<br/>add any engine in <b>~50 lines</b>.</p>
+    <p>Subclass <code>TTSbackend</code>,<br/>add any engine in <b>~50 lines</b>.</p>
+  </td>
+  <td align="center" valign="top">
+    <h3>🧭 Engine Routing</h3>
+    <p>Preflight GPU check per engine.<br/><b>No silent CPU fallback</b>.</p>
+  </td>
+</tr>
+<tr>
+  <td align="center" valign="top">
+    <h3>🎒 Portable Personas</h3>
+    <p>Export voices as <code>.ovsvoice</code><br/>bundles — identity + <b>watermark</b>.</p>
+  </td>
+  <td align="center" valign="top">
+    <h3>♾️ Unlimited TTS</h3>
+    <p>Sentence-chunked generation.<br/><b>No length cap</b>. Streaming via WS.</p>
+  </td>
+  <td align="center" valign="top">
+    <h3>🌐 Remote Backend</h3>
+    <p>Point UI at a remote server.<br/>Tailscale-friendly. <b>Bearer auth</b>.</p>
+  </td>
+  <td align="center" valign="top">
+    <h3>🧠 Dictation + LLM</h3>
+    <p>Local LLM cleanup of transcripts.<br/>Optional echo <b>cancellation</b>.</p>
  </td>
 </tr>
 </table>
@@ -200,12 +237,17 @@ ElevenLabs charges **$5–$330/mo** and processes your audio on their servers. O
 | **Pricing** | $5–$330/mo, per-character billing | Free & open-source (AGPL-3.0) · [Commercial license](#license) for proprietary use |
 | **Voice Cloning** | ✅ 3s clip | ✅ 3s clip, zero-shot |
 | **Voice Design** | ✅ Gender, age | ✅ Gender, age, accent, pitch, style, dialect |
+| **Audiobook / Stories** | ❌ | ✅ Full audiobook editor + multi-voice stories (EPUB/PDF import, .m4b export) |
 | **Languages** | 32 | **646** |
 | **Video Dubbing** | ✅ Cloud-only | ✅ Fully local |
 | **Data Privacy** | Audio sent to cloud | **Nothing leaves your machine** |
 | **API Keys** | Required | Not needed |
 | **GPU Support** | N/A (cloud) | CUDA · Apple Silicon · ROCm · CPU |
 | **Desktop App** | ❌ | ✅ macOS · Windows · Linux |
+| **TTS Engines** | 1 | **11** (OmniVoice, CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS-Nano, KittenTTS, MLX-Audio, Sherpa-ONNX, IndexTTS 2, OmniVoice GGUF, Supertonic 3) |
+| **ASR Engines** | 1 | **8** (WhisperX, Faster-Whisper, MLX Whisper, PyTorch Whisper, Parakeet, Moonshine, FunASR, isolated Faster-Whisper) |
+| **MCP Server** | ❌ | ✅ Use from Claude, Cursor, any MCP client |
+| **Self-check** | ❌ | ✅ Diagnostics suite, error journal, scrubbed debug bundles |
 | **Customizable** | ❌ Closed | ✅ Fork it, extend it, ship it |

 OmniVoice Studio gives you professional-grade AI tools without the subscription or the cloud.
@@ -241,12 +283,17 @@ OmniVoice ships a multi-engine TTS backend. The default engine (OmniVoice) is al
 |--------|:---------:|:-----:|:--------:|:-----:|:---------:|:-------:|:-------:|
 | **OmniVoice** (default) | 600+ | ✅ | ✅ | ✅ CUDA/CPU | ✅ MPS | ✅ CUDA/CPU | Built-in |
 | **CosyVoice 3** | 9 + 18 dialects | ✅ | ✅ | ✅ CUDA/CPU | ✅ MPS | ✅ CUDA/CPU | Apache-2.0 |
-| **MLX-Audio** (Kokoro, Qwen3-TTS, CSM, Dia, …) | Multi | Varies | Varies | ❌ | ✅ Native | ❌ | Varies |
+| **GPT-SoVITS** | 5 | ✅ | — | ✅ CUDA/CPU | — | ✅ CUDA/CPU | MIT |
 | **VoxCPM2** | 30 | ✅ | ✅ | ✅ CUDA/CPU | ✅ MPS | ✅ CUDA/CPU | Apache-2.0 |
-| **MOSS-TTS-Nano** | 20 | ✅ | ❌ | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
-| **KittenTTS** | English | ❌ | ❌ | ✅ CPU | ✅ CPU | ✅ CPU | MIT |
+| **MOSS-TTS-Nano** | 20 | ✅ | — | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
+| **KittenTTS** | English | — | — | ✅ CPU | ✅ CPU | ✅ CPU | MIT |
+| **MLX-Audio** (Kokoro, Qwen3-TTS, CSM, Dia, …) | Multi | Varies | Varies | ❌ | ✅ Native | ❌ | Varies |
+| **Sherpa-ONNX** | 20+ | — | — | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
+| **IndexTTS 2** ⚡ | Multi | ✅ | — | ✅ CUDA | — | ✅ CUDA | Apache-2.0 |
+| **OmniVoice GGUF** ⚡ | 600+ | ✅ | ✅ | ✅ CPU | ✅ CPU | ✅ CPU | Built-in |
+| **Supertonic 3** ⚡ | 31 | — | — | ✅ CPU | ✅ CPU | ✅ CPU | OpenRAIL-M |

-> **CUDA** = GPU-accelerated · **MPS** = Apple Silicon Metal · **CPU** = runs everywhere, slower for large models · KittenTTS and MOSS-TTS-Nano run realtime on CPU · MLX-Audio is Apple Silicon only.
+> **CUDA** = GPU-accelerated · **MPS** = Apple Silicon Metal · **CPU** = runs everywhere, slower for large models · KittenTTS and MOSS-TTS-Nano run realtime on CPU · MLX-Audio is Apple Silicon only · ⚡ = lazy-registered (installed on first use)

 ### ASR Engines

@@ -256,8 +303,9 @@ OmniVoice ships a multi-engine ASR (speech-to-text) backend that powers dictatio
 |--------|-------------------------|:---------:|----------|
 | **WhisperX** (default) | `whisperx` | ~100 | Dubbing & subtitles — word-level timing via wav2vec2 forced alignment |
 | **Faster-Whisper** | `faster-whisper` | ~100 | Fast transcription on Linux / macOS / Windows (CTranslate2) |
+| **Faster-Whisper (isolated)** | `faster-whisper-isolated` | ~100 | Same as Faster-Whisper but crash-isolated in a subprocess — an ASR crash won't take down the app |
 | **MLX Whisper** | `mlx-whisper` | ~100 | Native Apple Silicon speed (Apple MLX / Metal) |
-| **PyTorch Whisper** | `pytorch-whisper` | ~100 | CUDA / CPU fallback via 🤗 Transformers |
+| **PyTorch Whisper** | `pytorch-whisper` | ~100 | CUDA / CPU fallback via 🤗 Transformers (no cuDNN 8 needed) |
 | **Parakeet TDT** | `nemo-parakeet` | English + 25 EU | SOTA English accuracy, auto language detection (NVIDIA NeMo, GPU only) |
 | **Moonshine** | `moonshine` | English | Edge / low-latency, ONNX |
 | **FunASR** | `funasr` | 50+ | All-in-one multilingual — built-in VAD + inline speaker diarization (SenseVoice) |
@@ -269,18 +317,19 @@ OmniVoice ships a multi-engine ASR (speech-to-text) backend that powers dictatio
 ## Architecture

 ```
-┌─────────────────────────────────────────────────┐
-│                  Frontend (React)                │
-│  DubTab · VoicePreview · BatchQueue · Gallery    │
-├─────────────────────────────────────────────────┤
-│                Backend (FastAPI)                  │
-│  97 API endpoints · SSE streaming · SQLite       │
-├──────────┬──────────┬──────────┬────────────────┤
-│ WhisperX │  Demucs  │OmniVoice │   Pyannote     │
-│   ASR    │  Source  │   TTS    │  Diarization   │
-│          │  Sep.    │          │                │
-└──────────┴──────────┴──────────┴────────────────┘
-        CUDA / MPS / ROCm / CPU (auto-detected)
+┌─────────────────────────────────────────────────────────────┐
+│                    Frontend (React)                          │
+│  DubTab · VoiceConsole · Stories · Audiobook · Gallery     │
+│  Dictation · BatchQueue · Diagnostics · MCP Client          │
+├─────────────────────────────────────────────────────────────┤
+│                  Backend (FastAPI)                           │
+│  100+ API endpoints · SSE+WSS streaming · SQLite            │
+├──────────┬──────────┬──────────┬──────────┬────────────────┤
+│ WhisperX │  Demucs  │OmniVoice │ Pyannote │ Engine Routing  │
+│  (+7 ASR │  Source  │  (+10    │ Diariz-  │ ↳ GPU preflight │
+│ engines) │  Sep.    │  TTS)    │ ation    │ ↳ No silent CPU │
+└──────────┴──────────┴──────────┴──────────┴────────────────┘
+         CUDA / MPS / ROCm / CPU (auto-detected + routed)
 ```

 ---
@@ -291,27 +340,57 @@ OmniVoice ships a multi-engine ASR (speech-to-text) backend that powers dictatio

 | Category | Features |
 |----------|----------|
-| **Dubbing** | Full pipeline (transcribe→translate→synthesize→mux), scene-aware splitting, lip-sync scoring, streaming TTS |
-| **Voice** | Zero-shot cloning, voice design, A/B comparison, voice preview widget, gallery with favorites/tags |
-| **Audio** | Demucs vocal isolation, per-segment gain, selective track export, stem/SRT/VTT/MP3 export |
+| **Longform** | Audiobook editor (text/EPUB/PDF → chaptered .m4b), Stories multi-voice editor, two-pass loudnorm mastering, crash-resume for interrupted renders, pronunciation control + SSML-lite prosody |
+| **Dubbing** | Full pipeline (transcribe→translate→synthesize→mux), scene-aware splitting, lip-sync scoring, streaming TTS, per-speaker voice assignment, Smart Fit timing + second-pass QC, dedicated Dub home |
+| **Voice** | Zero-shot cloning, voice design, A/B comparison, voice preview widget, gallery with favorites/tags, portable persona bundles (`.ovsvoice`), voice console workspace |
+| **Audio** | Demucs vocal isolation, per-segment gain, selective track export, stem/SRT/VTT/MP3 export, unlimited-length TTS via sentence-chunked generation |
 | **Multi-Lang** | Multi-language batch picker, batch dubbing queue with sequential GPU execution |
 | **Diarization** | Pyannote ML diarization, auto speaker clone extraction, per-speaker voice assignment |
-| **Infra** | Docker deployment, CUDA/MPS/ROCm auto-detect, cuDNN 8 compat, VRAM-aware model offloading |
+| **ASR** | 8 engines (WhisperX, Faster-Whisper, isolated Faster-Whisper, MLX Whisper, PyTorch Whisper, Parakeet TDT, Moonshine, FunASR/SenseVoice), crash-isolated subprocess backend |
+| **TTS** | 11 engines (OmniVoice, CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS-Nano, KittenTTS, MLX-Audio, Sherpa-ONNX, + lazy: IndexTTS 2, OmniVoice GGUF, Supertonic 3), engine routing with GPU preflight |
+| **Infra** | Docker deployment, CUDA/MPS/ROCm auto-detect, cuDNN 8 compat, VRAM-aware model offloading, engine routing (no silent CPU fallback), diagnostics suite & error journal, restricted-network mirror support |
 | **AI Provenance** | AudioSeal invisible watermarking (SynthID-like), video logo overlay, watermark detection API |
-| **UX** | Undo/redo, keyboard shortcuts, drag-and-drop, session persistence, glassmorphism design system |
+| **UX** | Undo/redo, keyboard shortcuts, drag-and-drop, session persistence, glassmorphism design system, UI scale fix for Linux/WebKitGTK |
 | **Real-time Events** | WebSocket event bus — instant sidebar refresh on data mutations, exponential backoff reconnect |
 | **State Management** | Zustand store migration — `uiSlice`, `pillSlice`, `dubSlice`, `generateSlice`, `prefsSlice`, `glossarySlice` |
-| **Desktop** | Cross-platform Tauri installers (macOS DMG, Windows MSI, Linux deb/AppImage), auto-update infrastructure |
-| **Windows Hardening** | Cross-platform log paths, Triton workaround, HF symlink bypass, 300s health check timeout |
-| **Dictation** | Global system-wide hotkey (`⌘+⇧+Space`), frameless floating widget, streaming ASR via WebSocket, auto-paste |
+| **Desktop** | Cross-platform Tauri installers (macOS DMG/Intel, Windows MSI, Linux deb/AppImage), auto-update infrastructure, single-instance enforcement, close-to-tray, macOS Gatekeeper fix |
+| **Dictation** | Global system-wide hotkey (`⌘+⇧+Space`), frameless floating widget, streaming ASR via WebSocket, auto-paste, customizable hotkey, local-LLM transcript refinement |
 | **Batch Pipeline** | Full batch TTS: extract → transcribe → translate → generate → mix → export, with live progress tracking |
+| **MCP Server** | OmniVoice as a local TTS/STT provider for Claude, Cursor, and any MCP client |
+| **Remote Backend** | Point the desktop UI at a remote backend URL with bearer auth (Tailscale-documented) |
+| **Reliability** | Stall watchdog on bootstrap splash, per-engine GPU compatibility matrix, actionable errors for non-executable engine binaries, setuptools auto-repair |

 ### 🔜 Up Next

 - 🎬 **Lip-sync v2** — visual speech timing with wav2lip
- 📖 **Audiobook Editor** — chapter-aware long-form narration
 - 🌐 **Hosted Demo** — try OmniVoice without installing anything
 - 🔌 **Plugin Marketplace** — community-contributed TTS engines and effects
+- 🎵 **Real-time Voice Changer** — live microphone transformation during calls
+
+---
+
+## Sponsor / Donate
+
+OmniVoice Studio is built by one developer using Claude Code and AI agents — and the agent bills are real. Over the last three months I've spent thousands of dollars on Claude subscriptions to keep the features shipping, the bugs fixed, and your issues answered. If OmniVoice has created value for you, helping cover those bills means I can keep developing full-time.
+
+<div align="center">
+
+**This month's agent bill fund**
+
+<img src="https://img.shields.io/badge/raised_%2410_of_%24200-5%25-FF5E5B?style=for-the-badge" alt="$10 / $200 raised" />
+
+<br/><br/>
+
+<a href="https://ko-fi.com/debpalash"><img src="https://img.shields.io/badge/Ko--fi-Support_❤️-FF5E5B?style=for-the-badge&logo=ko-fi&logoColor=white" alt="Ko-fi" /></a>
+&nbsp;&nbsp;
+<a href="https://paypal.me/palashCoder"><img src="https://img.shields.io/badge/PayPal-Donate-00457C?style=for-the-badge&logo=paypal&logoColor=white" alt="PayPal" /></a>
+&nbsp;&nbsp;
+<a href="https://github.com/sponsors/debpalash"><img src="https://img.shields.io/badge/GitHub-Sponsor-ff69b4?style=for-the-badge&logo=github&logoColor=white" alt="GitHub Sponsors" /></a>
+
+<br/>
+<sub>Every dollar goes directly to agent bills — keeping OmniVoice development continuous.</sub>
+
+</div>

 ---

@@ -380,7 +459,7 @@ Yes. MPS acceleration is auto-detected. MLX-optimized Whisper models are availab
 <details>
 <summary><b>Can I add my own TTS engine?</b></summary>
 <br/>
-Yes. OmniVoice uses a <b>built-in backend registry</b>. To add an engine in ~50 lines, subclass <code>TTSBackend</code> in <code>backend/services/tts_backend.py</code> and add it to the <code>_REGISTRY</code> dictionary at the bottom. Six engines are built in: OmniVoice, CosyVoice, MLX-Audio (14+ sub-engines), VoxCPM2, MOSS-TTS-Nano, and KittenTTS. See the <a href="#tts-engines">TTS Engines</a> section for details.
+Yes. OmniVoice uses a <b>built-in backend registry</b>. To add an engine in ~50 lines, subclass <code>TTSBackend</code> in <code>backend/services/tts_backend.py</code> and add it to the <code>_REGISTRY</code> dictionary. Eleven engines are built in: OmniVoice, CosyVoice 3, GPT-SoVITS, MLX-Audio (14+ sub-engines), VoxCPM2, MOSS-TTS-Nano, KittenTTS, Sherpa-ONNX, plus lazy-registered IndexTTS 2, OmniVoice GGUF, and Supertonic 3. See the <a href="#tts-engines">TTS Engines</a> section for details.
 </details>

 ---
@@ -410,6 +489,9 @@ OmniVoice Studio is built on the shoulders of exceptional open-source work:
 | [**CTranslate2**](https://github.com/OpenNMT/CTranslate2) | Optimized Transformer inference on CPU and GPU |
 | [**AudioSeal (Meta)**](https://github.com/facebookresearch/audioseal) | Invisible neural audio watermarking for AI provenance |
 | [**Tauri**](https://tauri.app) | Native desktop app framework |
+| [**Supertone / Supertonic 3**](https://huggingface.co/Supertone/supertonic-3) | ONNX TTS engine — 31 languages, CPU-efficient |
+| [**Sherpa-ONNX**](https://github.com/k2-fsa/sherpa-onnx) | WASM-ready universal TTS/ASR runtime |
+| [**GPT-SoVITS**](https://github.com/RVC-Boss/GPT-SoVITS) | Zero-shot TTS engine — 5 languages, RTF 0.014 |

 ---

@@ -419,7 +501,8 @@ OmniVoice Studio is built on the shoulders of exceptional open-source work:

 If you read this far, you're our kind of person.<br/>
 **[⭐ Star this repo](https://github.com/debpalash/OmniVoice-Studio)** so others can find it too.<br/>
-**[💬 Join the Discord](https://discord.gg/bzQavDfVV9)** to share what you build.
+**[💬 Join the Discord](https://discord.gg/bzQavDfVV9)** to share what you build.<br/>
+**[❤️ Support development](https://ko-fi.com/debpalash)** — fund the AI agent bills that keep OmniVoice shipping.

 <br/>