VoiceStudio/.planning/research/2026-07-10-adjacent-projects.md

Adjacent open-source projects — research notes (2026-07-10)

Owner-requested research on five neighboring projects, read against OmniVoice Studio's current feature-maturity map. Each section ends with what we should take from it. Priorities are consolidated at the bottom.

Project	Stars	License	Status	Why it matters to us
Real-Time-Voice-Cloning	~60k	MIT	Retired (models frozen 2019, maintainer quit 2020)	Positioning/SEO opportunity, cautionary tales
VoxCPM	~33k	Apache-2.0	Very active (VoxCPM2, Apr 2026)	Upstream of our voxcpm2 engine — sync items below
ebook2audiobook	~19.5k	Apache-2.0 (default XTTS weights are CPML non-commercial)	Very active, weekly releases	The playbook for our weakest shipped surface (audiobook)
VideoLingo	~17.7k	Apache-2.0	Active, bursty	Dub-pipeline techniques (translation loop, timeline fit)
voicebox	~40.2k	MIT	Very active, post-viral triage debt	Direct competitor — same stack, same pitch, 10x the audience

1. Real-Time-Voice-Cloning — the retired ancestor

The 2019 SV2TTS implementation ("clone a voice in 5 seconds") that created the DIY voice-cloning category. Explicitly retired: the maintainer said in 2020 he won't develop it again; the README now calls itself old and redirects users to Chatterbox. Models are frozen 2019 checkpoints — 16 kHz, English-only, weak similarity, Tacotron+WaveRNN. Community PRs keep the install alive (uv one-command install landed Sept 2025), but ~163 open issues are mostly "how do I make it sound good" — the answer is: you can't.

Integrating it as an engine: no. Strictly worse than everything we ship, plus PyQt/legacy baggage.

Take:

• 60k stars of traffic reads a README that says "go elsewhere," and the redirect target is a model repo, not a product. An honest "Real-Time-Voice-Cloning alternative" comparison page is cheap, truthful, and lands exactly our pitch (local, free, modern quality, 646 languages, actual installer).
• Its headline copy discipline ("Clone a voice in 5 seconds, generate arbitrary speech in real-time") is better than ours; our 3-second-reference claim deserves the same outcome-first, time-boxed phrasing.
• Its failure modes validate our Core Value: out-of-band model links rotted for years; a research toolbox without packaging drowned in install issues.

2. VoxCPM — upstream of our voxcpm2 engine

Tokenizer-free TTS on a MiniCPM-4 backbone. Current model is VoxCPM2 (Apr 2026): 2B params, 30 languages + 9 Chinese dialects, 48 kHz, ~8 GB VRAM, RTF ~0.30 (0.13 with Nano-vLLM). Latest tag v2.0.3 (May 2026); main has unreleased seed support and timestamp alignment. Apache-2.0, healthy cadence, ~868k monthly HF downloads.

Sync items for our integration (we install voxcpm unpinned):

1. Floor the install at voxcpm>=2.0.3 — it carries the MPS audio-quality fix (low-precision dtypes promoted to float32 on Apple Silicon). Directly relevant to our default-platform-parity rule.
2. v2.0.1 removed reference-audio auto-trim — if we hand raw user clips to cloning, we now own trim/normalize. Verify our clone path; cloning quality may have silently regressed when upstream released 2.0.1.
3. Trailing-audio guard: end-of-audio gibberish/hallucination is a known open upstream bug (#352). A trailing-silence/garbage trim on our side is cheap insurance.
4. Later, when tagged: seed support (reproducible generation — currently buggy upstream, #351) and timestamp alignment (useful for dub sync); generate_streaming() is a candidate for tts_stream.py.
5. Risk: unpinned dependency + active upstream = next release lands silently in fresh installs. Consider pinning a tested range.

3. ebook2audiobook — the audiobook playbook

Any-format ebook (epub/pdf/docx/even scanned images via OCR) → Calibre normalize to EPUB → TOC/spine chapters ("blocks") → per-language sentence split → per-sentence TTS → chapterized m4b with metadata/cover. Gradio UI + headless CLI + Docker for every accelerator. Engine roster is 2023-era Coqui (XTTSv2 default, Bark, Piper, MMS…), with voice-conversion post-processing to fake cloning on non-cloning engines. 19.5k stars, near-weekly releases, only 4 open issues.

This is the mature version of exactly the surface where we're weakest: our audiobook/stories feature is a thin UI over per-chapter render caching, with no server-side ebook parsing and no per-segment regeneration.

Take (prioritized):

1. Per-sentence render cache + content-hashed blocks + missing-file resume. Every sentence is its own file; restart re-renders only what's missing; editing a block invalidates only that block. This closes our biggest audiobook gap (per-chapter cache, no crash resume) and is the same span-level model spec 03 already calls for — dub's incremental.py pattern, extended to longform.
2. Normalize-to-EPUB ingestion (Calibre ebook-convert) instead of building N format parsers; blocks carry keep/drop flags for front matter.
3. Engine-agnostic text-normalization pre-pass: per-language abbreviation maps, num2words, roman numerals, and a non-text character filter that kills TTS hallucination triggers. Benefits every engine we ship, not just audiobooks.
4. Chapterized m4b output (ffmpeg FFMETADATA chapters, cover art, VTT sidecar) — small work, high perceived value.
5. Inline voice/pause tags for multi-voice narration — our cloning quality makes this worth more to us than it is to them.

Where we already win: native desktop UX, modern engine quality (CosyVoice3/IndexTTS2/VoxCPM2 vs 2023 Coqui), real zero-shot cloning without VC hacks, no Calibre-wall install, and a commercially-clean default engine (their default XTTS weights are CPML non-commercial).

4. VideoLingo — dub-pipeline techniques

"Netflix-quality subtitles + dubbing" as a 14-stage Streamlit pipeline: yt-dlp → WhisperX word-level ASR → spaCy + LLM two-candidate semantic split → summarize-first terminology glossary → 3-step Translate–Reflect–Adapt → length-constrained subtitles → duration-aware dub-chunk planning → per-chunk reference audio → TTS → merge. Its recommended path is cloud-heavy (API LLM/TTS, optionally API ASR); fully-local is possible but fragile. Single-speaker only — it explicitly gave up on diarized multi-voice dubbing. Apache-2.0, ~17.7k stars, bursty maintenance, install pain on Windows/CUDA.

Take (prioritized):

1. Translate–Reflect–Adapt — add a reflection/critique pass to our per-segment translation prompt. Prompt-level change, meaningful quality win on idiomatic output.
2. Summarize-first glossary — extract theme + terminology once per video, inject into every segment's translation. Fixes term drift on long videos.
3. Duration-aware chunk planning — estimate TTS duration before generating; classify each line ok / needs-speedup / impossible; borrow inter-subtitle gap time and merge adjacent segments before resorting to atempo; for impossible lines, LLM-trim filler from the dub text instead of chipmunking. Our smart-fit handles the tail of this; their pre-planning avoids generating doomed audio at all.
4. Two-candidate split prompt — generate two [br] segmentations, have the LLM pick, instead of accepting the first.

Where we already win: fully local by design, per-segment regeneration + directorial AI (they have coarse folder-state resume, no per-segment redo), cross-platform installers, cloning stable across languages. Their single-speaker ceiling is our opening if diarized multi-voice dubbing ever ships.

5. voicebox — the direct competitor

Jamie Pine's (Spacedrive founder) "open-source AI voice studio. Clone, dictate, create." — architecturally a near-twin: Tauri + React/TS + FastAPI/Python + SQLite, MIT, local-first, explicitly pitched as ElevenLabs-out + WisprFlow-in replacement. Launched Jan 29, 2026; the launch post did ~17M views on X, and it sits at ~40.2k stars with ~10 community contributors and heavy AI co-authorship. Latest tagged release v0.5.0 (Apr 2026); main is active but untagged for ~10 weeks, with 434 open issues / 105 open PRs — a polished happy path with thin edges.

Engines: Qwen3-TTS 0.6B/1.7B (flagship cloner), Qwen CustomVoice, LuxTTS, Chatterbox Multilingual (23 langs) + Turbo, HumeAI TADA, Kokoro. Features where they lead: global-hotkey dictation overlay with LLM transcript cleanup (macOS-verified), Pedalboard post-FX chain, generation versioning/starring, multi-track Stories editor, MCP per-client voice bindings ("Claude Code speaks in your cloned voice") used as a viral wedge, DirectML/Intel-Arc coverage, and an agent-facing CONTRIBUTING pattern that farms drive-by contributions.

Two strategic facts:

• They are adding accounts. "Log in with browser" auth for a voicebox.sh cloud tier merged July 5 (their PR #812). Open-core with a paid cloud is visibly forming — which cuts against the pitch that won them their audience.
• Press already flagged their missing consent/misuse policy — we ship watermarking by default and consent attestation in .ovsvoice.

Where we're ahead: 646 languages vs 23, video dubbing (they have none), voice design from text descriptions (roadmap item for them, shipped for us), engine breadth (CosyVoice3/VoxCPM2/IndexTTS2/GPT-SoVITS/sherpa-onnx), and backward-compat/release discipline.

Take:

1. Positioning: own "no accounts, ever." Their cloud login is our opening — state the local-first guarantee in the README as a permanent commitment, next to the 646-language and dubbing advantages they can't match today.
2. Tell the MCP agent-voice story loudly. We already ship an MCP server and Agent Skills; per-client voice bindings + a speak-in-your-voice demo was their single best growth hook and costs us mostly marketing effort.
3. Generation versioning/starring and post-FX presets — cheap, high-perceived-value Studio features worth absorbing.
4. Watch their triage debt (434 open issues): our absorb-or-decline queue discipline is a real contributor-trust differentiator — keep it.

Consolidated priorities

Ordered by (user impact on already-shipped surfaces) × (effort):

1. voxcpm2 upstream sync (§2 items 1–3): version floor, ref-clip trim audit, trailing-audio guard. Small, protects an engine users already run.
2. Dub translation quality loop (§4 items 1–2): reflect pass + glossary. Prompt-level, no new deps, lifts the flagship dubbing feature.
3. Audiobook maturity via per-sentence cache + resume (§3 item 1): the established pattern for the feature the maturity survey ranked weakest — and it's the same architecture spec 03 already prescribes.
4. Text-normalization pre-pass (§3 item 3): engine-agnostic hallucination reduction; pairs with the pronunciation dictionary we already shipped.
5. Duration-aware dub planning (§4 item 3) and chapterized m4b export (§3 item 4): next tier, both self-contained.
6. Competitive positioning vs voicebox (§5 items 1–2): own "no accounts, ever" while they onboard a cloud tier, and tell the MCP agent-voice story we already technically ship.
7. RTVC comparison/migration page (§1): marketing, not engineering; cheap and honest.

Method note: compiled from five parallel research passes over the repos' READMEs, releases, issues, and (for ebook2audiobook) source; figures as of 2026-07-10.