Files

T

VishvakR f172d0c3be fix(generate): apply the VRAM-floor budget to remote workers and /convert

Review findings on the PR, fixed here rather than left for a fourth report.

Greptile (P1): the control plane sets a remote attempt's deadline, so the same
inversion reached remote workers. Its suggested fix -- thread the engine floor
into generate_timeout_s() -- would read the wrong machine: that function probes
THIS host, so a Mac control plane dispatching to a 4 GB Windows worker learns
nothing (MPS is excluded by design), and a 4 GB box dispatching to a 24 GB
worker would wrongly get the longer budget. The worker already advertises both
figures it takes -- free_memory_bytes and min_memory_bytes, both set in
worker/capabilities.py -- so ConnectedWorker.under_provisioned() decides from
those, and deadlines.for_task() floors the execution budget at what the same job
would get on a CPU. The task-level ceiling in gpu_gateway._default_deadline is
computed before a worker is bound and already asks for the CPU budget, so it
still covers the raised lease; a test pins that.

CodeRabbit (major): /convert had the identical split -- min_vram_gb to the
guard so a timeout could name the card, and a budget computed without it.

CodeRabbit (minor): the docs promised the CPU-class floor for any GPU, while
the code scopes it to dedicated-VRAM families. Reworded to say CUDA/ROCm and to
say why MPS is excluded.

CodeRabbit (minor): the call-site assertion compared global occurrence counts,
so one dispatch could drop both arguments while another gained an extra and the
total still matched. It now walks the AST and checks each dispatch on its own,
and the pairing is additionally enforced repo-wide across backend/api/routers:
a dispatch that knows the engine's floor well enough to explain a timeout must
know it well enough to set the budget.

Three inline capability-selection loops in ConnectedWorker collapse into one
_capability_for(), so the new predicate cannot select a different capability
than execution_device() does.

2026-09-04 13:33:00 +05:30

5.8 KiB

Raw Blame History

VoiceStudio — OmniVoice Engine (default)

OmniVoice (k2-fsa/OmniVoice) is VoiceStudio's default TTS engine — the one a fresh install uses without any configuration. It does zero-shot voice cloning across 600+ languages and outputs 24 kHz mono audio. Voice cloning, dubbing, and dictation all run on it out of the box.

When to pick it

You want cloning plus the broadest language coverage (see languages.md).
You have a GPU (CUDA, AMD ROCm on Linux, or Apple Silicon MPS) with ~6 GB VRAM or more.
You just installed VoiceStudio — it's already selected.

For low-VRAM or CPU-only machines, the OmniVoice GGUF variant runs the same model through a quantized native binary with a much smaller memory footprint.

Requirements

Runs on CUDA, AMD ROCm (Linux), MPS (Apple Silicon), or CPU — auto-detected.
Recommended VRAM floor: 6 GB on a dedicated GPU. This is the only engine with a measured floor: on 4 GB cards (GTX 1650 Ti, Quadro P2000 — issues #1226 / #1222) the driver pages to system RAM and a render that should take seconds runs for minutes until the compute budget kills it. The UI warns before you wait; nothing hard-blocks, since short inputs can still fit. A CUDA or ROCm card below the floor is also budgeted as the CPU-class hardware it performs like — the longer OMNIVOICE_CPU_GENERATE_TIMEOUT_S (600 s), not the accelerated 300 s (#1804). Apple Silicon is excluded: unified memory has no dedicated pool to compare against.
No extra install — the model ships with the app and downloads its weights on first use (see downloading-models.md).

Selecting the engine

OmniVoice is the default, so normally there is nothing to do. If you switched away and want it back:

Model Catalogue → Engines, or
set OMNIVOICE_TTS_BACKEND=omnivoice.

The env var overrides the persisted UI choice.

Behaviour notes

Weights load lazily on first use and are shared with the rest of the app (dubbing, dictation) — the model is never double-loaded.
On CUDA and ROCm the model runs fp16 with torch.compile; PyTorch exposes ROCm/HIP devices through its cuda API, while VoiceStudio's engine matrix reports the hardware as ROCm. A speech recognizer is co-loaded for the cloning path.
Output is 24 kHz mono; the shared mastering chain (highpass + compressor) is tuned for this rate and applied automatically.
Cloning takes a short reference clip (ref_audio); 3–10 seconds is the sweet spot. A transcript of the clip improves conditioning — if the profile has none, VoiceStudio transcribes the clip automatically on first use and saves the result to the profile. A clip with a supplied transcript is limited to 20 seconds so the two stay aligned; trim both to the same passage. Without a transcript, VoiceStudio can search up to 75 seconds in five contiguous, bounded transcription passes and selects the passage with detected speech. Longer clips must be trimmed first. If no spoken words are detected, trim to a clear 3–10 second passage or provide its matching transcript.
Encoded voice references persist on disk (prompt_cache/ in the app data dir), so the first generation with a known voice after a restart skips the re-encode and any transcription pass. Set OMNIVOICE_PROMPT_DISK_CACHE=0 to keep the cache in memory only.
Style attributes (instruct) and a reference clip can be combined: when they agree, the instruct stabilizes cloning for the attributes it names (upstream documents dialect cloning as the canonical case — dialect reference + matching dialect instruct). When they conflict, the reference audio wins.
Inline pronunciation control: Chinese via pinyin with tone numbers (打ZHE2出售), English via bracketed CMU phonemes ([B EY1 S]). Non-verbal tags like [laughter] are covered in expressive-speech.md.
Voice design works from attributes (gender, age, pitch, whisper, English accents, Chinese dialects) via the Design tab — no reference audio needed.
Optional FlashInfer acceleration on CUDA: set OMNIVOICE_FLASHINFER=1 (or =graph for CUDA-graph capture, best for one render at a time) after installing the flashinfer-python package — see performance.md. Off by default; if the package is missing or a kernel fails, the app logs why and continues on the standard path.

Known limits

Voice design understands only the fixed attribute vocabulary — free-form design prose is mapped onto those attributes, and wording outside them is ignored. Design is trained on English and Chinese and can be unstable in low-resource languages; for description-driven design in other cases try VoxCPM2.
Below the 6 GB VRAM floor on a CUDA or ROCm card, expect very slow renders; they get the longer CPU compute-time budget rather than the accelerated one, but can still time out. Prefer OmniVoice GGUF or a CPU engine such as PocketTTS.

Troubleshooting

"Too heavy for the available compute" on a small GPU: see the VRAM floor above — switch to OmniVoice GGUF or close other GPU apps.
First generation is slow: the first call downloads multi-GB weights. To keep the first render quick, install the model ahead of time from Model Catalogue → Models — a long first generate is almost always the download, not a hang.
General install issues: install/troubleshooting.md.

5.8 KiB Raw Blame History Unescape Escape