Files

T

mergetestandClaude Opus 4.8 3777d3a62c feat(tts): add MOSS-TTS-v1.5 (8B) and dots.tts (2B) as opt-in engines (#498 )

Adds two zero-shot voice-cloning TTS engines requested in #498, both
opt-in and subprocess-isolated with their own dedicated venv — the same
pattern as IndexTTS-2. The dedicated venv is forced, not just chosen:
each upstream pins a transformers version that conflicts with the
parent's >=5.3 (MOSS-TTS-v1.5 ==5.0.0, dots.tts ==4.57.0), so they cannot
share the parent interpreter.

Because they use the clone+venv bootstrap (env var -> clone -> uv venv),
this touches no pyproject.toml / uv.lock / bun.lock — `uv sync
--all-extras` and Docker's `bun install --frozen-lockfile` are unchanged,
so main's CI/Docker matrix stays green.

Engines:
- moss-tts-v15: 8B, 31 langs, ~16 GB weights, 24 kHz. AutoModel/
  AutoProcessor via trust_remote_code. gpu_compat=(cuda,cpu) — MPS is
  undocumented/untested upstream so it is never claimed; on a Mac it runs
  on CPU. Apache-2.0, no license gate.
- dots-tts: 2B, 24 langs, ~9 GB weights, 48 kHz. DotsTtsRuntime;
  continuation cloning (prompt_audio_path+prompt_text). Upstream is
  Linux/macOS-only, so is_available() gates it off cleanly on Windows
  (cross-platform parity rule — it is opt-in, never a broken default).

Wiring: registered in _LAZY_REGISTRY + _INSTALL_HINTS. list_backends()
surfaces both as subprocess/[cuda,cpu]/available-until-installed; the
data-driven Settings engine picker needs no frontend change.

Tests (19, fail-before/pass-after): registry resolution, subprocess
marker, no-MPS gpu_compat, the Windows gate, not-installed honesty, and
the parent-side generate() kwarg arbitration. Existing engine suite still
55 passed / 5 skipped. Sidecar inference follows the upstream-documented
APIs but, like IndexTTS/Supertonic, can't be executed in CI without the
multi-GB model clones.

Docs (same-PR per docs-sync rule): README + README_CN engine tables, new
docs/engines/moss-tts-v15.md + dots-tts.md, disk-usage.md (torch-dedup
note), CHANGELOG.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-06-20 00:02:27 +05:30

5.2 KiB

Raw Blame History

OmniVoice Studio — MOSS-TTS-v1.5 Engine

MOSS-TTS-v1.5 (OpenMOSS) is an 8B flagship zero-shot TTS — a Qwen3-8B language backbone plus a 1.6B audio codec. It covers 31 languages, does zero-shot voice cloning, token-level duration control and inline [pause Ns] markers. Released under Apache-2.0 (code + weights).

It runs in its own subprocess and its own Python venv with transformers==5.0.0, isolated from the OmniVoice parent process which pins transformers>=5.3. This is the same isolation primitive used by IndexTTS-2: the two transformers pins cannot share one interpreter, so MOSS runs behind backend/services/subprocess_backend.py::SubprocessBackend.

Opt-in, and never a default. MOSS-TTS-v1.5 is selected explicitly in Settings → Engines (or OMNIVOICE_TTS_BACKEND=moss-tts-v15). It is not part of the default install and does not change OmniVoice's out-of-the-box behaviour on any platform.

Hardware

VRAM/RAM: an 8B model. The upstream llama.cpp pipeline fits the 8B on 8 GB GPUs when quantized; the bf16 Transformers path used here is ~16 GB of weights, so a 16 GB+ GPU is the realistic CUDA target. It also runs on CPU (fp32) — correct but slow.
Device: CUDA when present, else CPU. There is no MPS path — upstream documents only CUDA/CPU and the custom modelling code is untested on Apple Silicon, so OmniVoice never routes MOSS to MPS. On a Mac it runs on CPU.

Install

MOSS-TTS-v1.5 is not bundled (the model is large and the package pins a conflicting transformers). OmniVoice ships a sidecar runner that loads it into an isolated venv on demand.

Clone the MOSS-TTS repo on disk:

git clone https://github.com/OpenMOSS/MOSS-TTS.git

Install the editable package into a fresh venv. Use uv pip install -e ".[torch-runtime]" — never uv sync --all-extras, which would overwrite OmniVoice's lock file with transformers==5.0 and break the parent process. The torch-runtime extra is CUDA (+cu128):
```
cd MOSS-TTS
uv venv .venv
uv pip install -e ".[torch-runtime]"
```
On a non-CUDA / CPU host (e.g. Apple Silicon), install plain torch/torchaudio/transformers==5.0.0 into the venv instead of the +cu128 extra (the auto-bootstrap below only targets CUDA hosts).
The ~16 GB weights download from HuggingFace on first synthesize. The parent forwards HF_HOME / HF_HUB_CACHE to the sidecar so the cache is shared with the rest of OmniVoice's downloads.

Set OMNIVOICE_MOSS_TTS_V15_DIR to the repo root (the directory that contains pyproject.toml):

# macOS / Linux
echo 'export OMNIVOICE_MOSS_TTS_V15_DIR=$HOME/code/MOSS-TTS' >> ~/.zshrc
source ~/.zshrc

# Windows PowerShell
[Environment]::SetEnvironmentVariable("OMNIVOICE_MOSS_TTS_V15_DIR","$env:USERPROFILE\code\MOSS-TTS","User")

Restart OmniVoice. MOSS-TTS-v1.5 appears in Settings → Engines with available: true and isolation_mode: subprocess.

Venv resolution order

OmniVoice probes for a usable MOSS Python interpreter in this priority order (see backend/engines/moss_tts_v15/bootstrap.py):

${OMNIVOICE_MOSS_TTS_V15_DIR}/.venv/ — your existing clone's venv. Highest priority, so a power user who already set MOSS up gets zero re-install.
backend/engines/moss_tts_v15/.venv/ — OmniVoice's own venv, created on demand by step 3.
Lazy bootstrap — if neither venv exists, OmniVoice runs uv venv then uv pip install --python <python> -e "${DIR}[torch-runtime]". Requires OMNIVOICE_MOSS_TTS_V15_DIR; raises a clear error otherwise. On a non-CUDA host the +cu128 extra cannot resolve — set the venv up manually per step 2.

Voice cloning

Pass a reference clip as ref_audio. MOSS's zero-shot clone mode needs only the audio (no transcript). Without a reference, MOSS synthesizes in its own default voice. duration (seconds) maps to MOSS's tokens argument at ~12.5 tokens/second.

Optional env knobs

Variable	Default	Purpose
`OMNIVOICE_MOSS_TTS_V15_DIR`	—	Path to the MOSS-TTS clone (required).
`OMNIVOICE_MOSS_TTS_V15_MODEL`	`OpenMOSS-Team/MOSS-TTS-v1.5`	HF repo id override (mirror / air-gapped).
`OMNIVOICE_MOSS_TTS_V15_ATTN`	`sdpa`	Attention impl; set `flash_attention_2` on Ampere+ CUDA with `flash-attn` installed.

Common errors

`MOSS-TTS-v1.5 venv not found. Set OMNIVOICE_MOSS_TTS_V15_DIR ...`

You haven't pointed OmniVoice at a MOSS-TTS clone yet. Follow Install.

`uv pip install -e failed ... '[torch-runtime]' extra (cu128) cannot resolve`

You're on a non-CUDA host. The upstream torch-runtime extra is CUDA-only; set up the venv manually with plain torch/transformers==5.0.0 (step 2).

License

Apache-2.0 (code and weights) — no acceptance gate. See the upstream README.

MOSS-TTS-v1.5 runs in a dedicated sidecar venv (it pins transformers==5.0, which conflicts with the parent's transformers>=5.3). For why that adds disk and how uv keeps the cost down, see Engine venvs & disk usage.

5.2 KiB Raw Blame History