Files

T

mergetestandClaude Opus 4.8 3777d3a62c feat(tts): add MOSS-TTS-v1.5 (8B) and dots.tts (2B) as opt-in engines (#498 )

Adds two zero-shot voice-cloning TTS engines requested in #498, both
opt-in and subprocess-isolated with their own dedicated venv — the same
pattern as IndexTTS-2. The dedicated venv is forced, not just chosen:
each upstream pins a transformers version that conflicts with the
parent's >=5.3 (MOSS-TTS-v1.5 ==5.0.0, dots.tts ==4.57.0), so they cannot
share the parent interpreter.

Because they use the clone+venv bootstrap (env var -> clone -> uv venv),
this touches no pyproject.toml / uv.lock / bun.lock — `uv sync
--all-extras` and Docker's `bun install --frozen-lockfile` are unchanged,
so main's CI/Docker matrix stays green.

Engines:
- moss-tts-v15: 8B, 31 langs, ~16 GB weights, 24 kHz. AutoModel/
  AutoProcessor via trust_remote_code. gpu_compat=(cuda,cpu) — MPS is
  undocumented/untested upstream so it is never claimed; on a Mac it runs
  on CPU. Apache-2.0, no license gate.
- dots-tts: 2B, 24 langs, ~9 GB weights, 48 kHz. DotsTtsRuntime;
  continuation cloning (prompt_audio_path+prompt_text). Upstream is
  Linux/macOS-only, so is_available() gates it off cleanly on Windows
  (cross-platform parity rule — it is opt-in, never a broken default).

Wiring: registered in _LAZY_REGISTRY + _INSTALL_HINTS. list_backends()
surfaces both as subprocess/[cuda,cpu]/available-until-installed; the
data-driven Settings engine picker needs no frontend change.

Tests (19, fail-before/pass-after): registry resolution, subprocess
marker, no-MPS gpu_compat, the Windows gate, not-installed honesty, and
the parent-side generate() kwarg arbitration. Existing engine suite still
55 passed / 5 skipped. Sidecar inference follows the upstream-documented
APIs but, like IndexTTS/Supertonic, can't be executed in CI without the
multi-GB model clones.

Docs (same-PR per docs-sync rule): README + README_CN engine tables, new
docs/engines/moss-tts-v15.md + dots-tts.md, disk-usage.md (torch-dedup
note), CHANGELOG.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-06-20 00:02:27 +05:30

4.5 KiB

Raw Blame History

OmniVoice Studio — dots.tts Engine

dots.tts (rednote-hilab) is a 2B fully-continuous autoregressive TTS, widely cited as one of the strongest open zero-shot voice-cloning models. It covers 24 languages, emits 48 kHz audio, and is released under Apache-2.0 (code + checkpoints).

It runs in its own subprocess and its own Python venv with transformers==4.57.0, isolated from the OmniVoice parent process which pins transformers>=5.3 — the same isolation primitive used by IndexTTS-2 and MOSS-TTS-v1.5.

Opt-in, and never a default. dots.tts is selected explicitly in Settings → Engines (or OMNIVOICE_TTS_BACKEND=dots-tts). It is not part of the default install.

Platform support

Linux / macOS only. dots.tts's upstream package declares Linux and macOS classifiers and has no Windows install path. On Windows the engine reports itself unavailable in Settings → Engines with a clear reason — run OmniVoice under WSL2 or use a Linux/macOS host.
No MPS. Upstream device selection is CUDA-or-CPU with no Metal branch, so on Apple Silicon the official package runs on CPU (slow but correct). A faster Apple-Silicon path exists only via community MLX ports, which OmniVoice does not auto-wire.
VRAM: ~9 GB checkpoint; a 12–16 GB CUDA GPU is the realistic target.

Install

dots.tts is not bundled (large checkpoint + conflicting transformers).

Clone the dots.tts repo on disk:

git clone https://github.com/rednote-hilab/dots.tts.git

Install the editable package into a fresh venv with the upstream constraints. Use uv pip install -e . -c constraints/recommended.txt — never uv sync --all-extras, which would overwrite OmniVoice's lock file with transformers==4.57 and break the parent process:
```
cd dots.tts
uv venv .venv
uv pip install -e . -c constraints/recommended.txt
```
The ~9 GB checkpoint downloads from HuggingFace on first synthesize. The parent forwards HF_HOME / HF_HUB_CACHE to the sidecar so the cache is shared with the rest of OmniVoice's downloads.

Set OMNIVOICE_DOTS_TTS_DIR to the repo root (the directory that contains pyproject.toml and constraints/):

# macOS / Linux
echo 'export OMNIVOICE_DOTS_TTS_DIR=$HOME/code/dots.tts' >> ~/.zshrc
source ~/.zshrc

Restart OmniVoice. dots.tts appears in Settings → Engines with available: true and isolation_mode: subprocess.

Venv resolution order

OmniVoice probes for a usable dots.tts Python interpreter in this priority order (see backend/engines/dots_tts/bootstrap.py):

${OMNIVOICE_DOTS_TTS_DIR}/.venv/ — your existing clone's venv.
backend/engines/dots_tts/.venv/ — OmniVoice's own venv, created on demand by step 3.
Lazy bootstrap — uv venv then uv pip install -e <clone> -c <clone>/constraints/recommended.txt. Requires OMNIVOICE_DOTS_TTS_DIR.

Voice cloning

For best fidelity ("continuation cloning"), pass both a reference clip (ref_audio) and its exact transcript (ref_text). A reference clip alone does x-vector-only cloning. Keep the reference ~10 s. Upstream requires the reference audio whenever a transcript is given, so OmniVoice drops a stray ref_text that arrives without ref_audio.

Optional env knobs

Variable	Default	Purpose
`OMNIVOICE_DOTS_TTS_DIR`	—	Path to the dots.tts clone (required).
`OMNIVOICE_DOTS_TTS_MODEL`	`rednote-hilab/dots.tts-soar`	Checkpoint override (`-base`, `-soar`, `-mf`).
`OMNIVOICE_DOTS_TTS_PRECISION`	`bfloat16` (CUDA) / `float32` (CPU)	Inference precision.
`OMNIVOICE_DOTS_TTS_OPTIMIZE`	`0`	`1` enables `torch.compile` (slower first call, faster after).

Using the dots.tts-mf (MeanFlow-distilled) checkpoint? It's tuned for 4 flow-matching steps — pass num_step=4.

Common errors

`dots.tts is not supported on Windows ...`

Upstream is Linux/macOS only. Use WSL2 or a Linux/macOS host.

`dots.tts venv not found. Set OMNIVOICE_DOTS_TTS_DIR ...`

You haven't pointed OmniVoice at a dots.tts clone yet. Follow Install.

License

Apache-2.0 (code and checkpoints). See the upstream README.

dots.tts runs in a dedicated sidecar venv (it pins transformers==4.57, which conflicts with the parent's transformers>=5.3). For why that adds disk and how uv keeps the cost down, see Engine venvs & disk usage.

4.5 KiB Raw Blame History Unescape Escape