# VoiceStudio — audio.cpp Engine (Breeze-TTS-2)

[audio.cpp](https://github.com/0xShug0/audio.cpp) is a pure-C++ ggml audio
inference framework with prebuilt binaries for Windows, macOS, and Linux and
no Python dependency. VoiceStudio discovers the compute providers compiled
into the installed binary and selects the best available device. VoiceStudio drives its
`audiocpp_server` over loopback HTTP — v1 serves the **`breeze_tts`**
family: **Breeze-TTS-2** (BreezeBlue, 3B params, English + Chinese, voice
clone + voice design + voice direction, 24 kHz).

> **Opt-in, and never a default.** Select `audiocpp` explicitly in **Model
> Catalogue → Engines** (or `OMNIVOICE_TTS_BACKEND=audiocpp`).

## License — read before enabling

- **audio.cpp code:** Apache-2.0.
- **Breeze-TTS-2 weights** (upstream `BreezeBlue/Breeze-TTS-2` and the
  `audio-cpp/audio.cpp-gguf` GGUF repack): **research and non-commercial
  use only** under the [BreezeBlue Research and Non-Commercial License](
  https://huggingface.co/BreezeBlue/Breeze-TTS-2/blob/main/LICENSE).
  Self-hosted outputs inherit the restriction; a BreezeBlue paid
  subscription covers hosted-platform outputs only, not this engine.

## Platform support

| Host | Binary | Compute |
|---|---|---|
| Windows x64 | CPU, Vulkan, CUDA 12.4/13.3 prebuilts | CPU, Vulkan, CUDA |
| Linux x64 | CPU and Vulkan prebuilts | CPU, Vulkan; CUDA/ROCm from a self-build |
| macOS arm64 | upstream Metal prebuilt | Metal + CPU |
| macOS x64 | upstream `metal` archive (Metal disabled by upstream) | CPU |
| Linux aarch64 | none upstream | unavailable in v1 |

VoiceStudio runs `audiocpp_server --list-devices` once per installed binary,
prefers CUDA, ROCm, Metal, then a discrete Vulkan GPU, and keeps CPU as a safe
fallback. Device numbers are local to each backend registry. The Q8_0 GGUF is
approximately 4.73 GiB, plus runtime memory.

Allow about 6 GB of dedicated VRAM for CUDA, HIP, or a discrete GPU exposed
through Vulkan. Metal and Vulkan integrated GPUs use unified-memory handling
instead of that dedicated-VRAM floor.

## Install

On a supported desktop, install the model from **Settings → Models**, then use
**Install runtime** on the audio.cpp/Sortformer engine row. VoiceStudio fetches
the pinned archive, verifies its published size and SHA-256, rejects unsafe
archive paths, probes its device list, and installs it under the update-surviving
app-data engine directory. The action is explicit; generation never downloads
or updates executable code.

For a user-managed runtime:

1. Download the v0.7.4 prebuilt for your platform from
   [audio.cpp releases](https://github.com/0xShug0/audio.cpp/releases/tag/v0.7.4)
   and extract it. Use the Vulkan archive on Windows or Linux for broad GPU
   support, the CPU archive when Vulkan is unavailable, or the matching CUDA
   archive on Windows for NVIDIA. A Windows CUDA install needs both the
   `bin-…-cuda…` and matching `cudart-…-cuda…` archives extracted into the
   same directory, as required by upstream. Linux archives do not preserve the
   executable bit, so run `chmod +x audiocpp_server` after extracting one.

   Verify the archive before extracting it. The pinned SHA-256 checksums are:

   | Archive | SHA-256 |
   |---|---|
   | `audio-v0.7.4-bin-windows-x64-cpu-portable.zip` | `d241c56ba78fd3c1b28bf289792fb8ec258d36586b4e0c8d667080ec248c0d2f` |
   | `audio-v0.7.4-bin-windows-x64-vulkan.zip` | `057332f9e3fb37706a8ecb5075ac1797efcd85fdccd739f7b65761a5920f2828` |
   | `audio-v0.7.4-bin-windows-x64-cuda12.4.zip` | `83fdd5b6e7bd4362604c10cc88d7d3564ef82030dc1d21c693a62cdcbe2e5e38` |
   | `audio-v0.7.4-cudart-windows-x64-cuda12.4.zip` | `88d8943a2a8011f02c2a4efa7dbbe258608362615cce51e7f0e0e3a0c62f5a43` |
   | `audio-v0.7.4-bin-windows-x64-cuda13.3.zip` | `af56012969bcb68f54e6ea14a123e5c6ecd62830c1b2816080eb7f99d985a779` |
   | `audio-v0.7.4-cudart-windows-x64-cuda13.3.zip` | `c20793d8cc9b7c66ab28ab335aa908c726f2df15832330ddcfdbc837c7670145` |
   | `audio-v0.7.4-bin-ubuntu-x64-cpu.tar.gz` | `638e6114550c5ea02b96907de400379c8f31bd13325525083f98d158027acc40` |
   | `audio-v0.7.4-bin-ubuntu-x64-vulkan.tar.gz` | `e0ef3123a9f94e130ad463db0db5a69b65485ef8db1b46edead00c03a86fa787` |
   | `audio-v0.7.4-bin-macos-arm64-metal.tar.gz` | `639926715b1cb537f82aa31656aabbae5d9a85ac36568c402026968f3072e2b3` |
   | `audio-v0.7.4-bin-macos-x64-metal.tar.gz` | `bdb797d54dcf8416bd5ac0fac282ce5500dd08843f8f22e20e9fc378ebc24c1f` |

   Run `sha256sum <archive>` on Linux, `shasum -a 256 <archive>` on macOS,
   or `Get-FileHash <archive> -Algorithm SHA256` in PowerShell and compare the
   complete result with the table.
2. Set `OMNIVOICE_AUDIOCPP_BIN` to the `audiocpp_server` binary
   (`audiocpp_server.exe` on Windows):

   ```bash
   # macOS / Linux
   echo 'export OMNIVOICE_AUDIOCPP_BIN=$HOME/apps/audio.cpp/audiocpp_server' >> ~/.zshrc
   source ~/.zshrc
   ```

   Alternatively set `OMNIVOICE_AUDIOCPP_DIR` to the directory containing it.
3. Restart VoiceStudio, open **Model Catalogue (TTS tab → the engine's Weights)**, find
   **Breeze-TTS-2 Q8_0 for audio.cpp**, review its research/non-commercial
   license note, and click **Install**. Generation never starts this ~4.73 GiB
   download automatically.
4. Pick `audiocpp` in **Model Catalogue** (TTS tab → **Use**). The server starts
   lazily on first generate (`server.json` + `server.log` live under the app
   data `audiocpp/` directory).

## Voice modes

All three go through the one speech endpoint — reference presence selects:

- **Clone:** `ref_audio` + `ref_text` (exact transcript, as upstream).
- **Direction:** `ref_audio` + `ref_text` + `instruct`
  (e.g. "Speak slowly with a restrained, serious tone").
- **Design:** `description` (or `instruct`) with no `ref_audio`
  (e.g. "A warm, thoughtful young woman…"). Upstream strengthens
  instruction-following with `guidance_scale` ≈ 4.

## Optional env knobs

| Variable | Default | Purpose |
|----------|---------|---------|
| `OMNIVOICE_AUDIOCPP_BIN` | — | Absolute path to `audiocpp_server`. |
| `OMNIVOICE_AUDIOCPP_DIR` | — | Directory containing `audiocpp_server`. |
| `OMNIVOICE_AUDIOCPP_MODEL` | Model Catalogue cache | GGUF file or directory override. |
| `OMNIVOICE_AUDIOCPP_PACKAGE` | `breeze-tts-2-q8_0.gguf` | Package filename (`…-bf16.gguf` for full precision). |
| `OMNIVOICE_AUDIOCPP_PORT` | `17860` | Loopback port. |
| `OMNIVOICE_AUDIOCPP_BACKEND` | Settings, then auto | Exact runtime: `cuda`, `hip`/`rocm`, `vulkan`, `metal`, or `cpu`. |
| `OMNIVOICE_AUDIOCPP_DEVICE` | best device | Backend-local non-negative device index; requires `OMNIVOICE_AUDIOCPP_BACKEND`. |

The audio.cpp overrides take precedence over the global Settings compute
choice. A global CUDA/ROCm choice can match an NVIDIA/AMD GPU exposed through
Vulkan. An unavailable explicit audio.cpp override is an error; an unavailable
global preference falls back to CPU and is shown as a routing fallback.

## Common errors

### `audiocpp_server not found ...`

The binary isn't installed. Follow **Install** — the message carries the
exact release URL and SHA for your platform.

### `audiocpp_server exited during startup ...`

The managed loopback port may be taken. Check `server.log` next to
`server.json` in the app data `audiocpp/` directory, or set a different
`OMNIVOICE_AUDIOCPP_PORT` and restart VoiceStudio.

### `Breeze-TTS-2 ... not installed` or `package ... not completely installed`

Install the model from **Model Catalogue (TTS tab → the engine's Weights)**. If an interrupted install
left it incomplete, use **Reinstall** there. If the error persists after a
complete reinstall, file an issue with the package listing.

---

audio.cpp runs as a managed native server (no Python venv, no
`transformers` conflict). Only the downloaded GGUF counts toward
[sidecar disk usage](disk-usage.md).

The catalogue checks the exact Breeze Q8_0 package inside the shared GGUF repository. Other audio.cpp packages do not make Breeze appear installed. Required weight files must also satisfy the normal weight-size floor; empty or truncated placeholders remain incomplete. Broader native model discovery and execution are still pending integration.