Files
VoiceStudio/docs/engines/dots-tts.md
T
Palash DebnathandClaude Opus 5 41722afe3b refactor(launchpad): quieter, borderless design refresh (#1515)
* refactor(launchpad): quieter, borderless design refresh

The launchpad carried decoration from an earlier direction: icon chips,
corner-hung count badges, a permanently visible filled arrow, uppercase
mono card titles, and a dotted stipple divider — plus a frame that had
been invisible since the app-wide border tokens were zeroed.

Rework it around what the borderless direction actually implies:

- Feature tiles get a whisper-faint surface instead of a dead frame, and
  read as three bands (bare glyph + count / title + arrow / description).
  `--card-hue` is spent sparingly — the glyph at rest, the surface, count
  and arrow only once raised. Titles move to sans sentence case; counts
  are plain tabular numerals. Lift softened 4px -> 2px, coloured glow ->
  neutral shadow, plus an explicit focus ring and a staggered entrance.
- Hero drops the boxed "646" pill and the filled A/B-Compare button for
  quiet type, with a hairline standing in for the separation.
- Section labels trade the dotted stipple for a single fading hairline;
  rows are transparent until hover and reveal "Open" on hover/focus (it
  stays in the DOM, so AT and keyboard always reach it).
- Hero, tiles, recent files, callout and project lists now share one
  1180px column — previously only the top half was capped, so lists ran
  edge-to-edge on a wide display while the deck stayed centred.

Two bugs found and fixed while doing it:

- Buttons that had `border border-solid border-transparent` removed fell
  back to the UA default border and rendered a visible 1px outline. They
  now carry `border-0` explicitly.
- `.lp-animate` used `animation-fill-mode: both`, so after the entrance
  it kept owning `transform` — and animation-origin declarations outrank
  normal ones, which silently killed the card hover lift. Now `backwards`,
  which still holds the from-state through the stagger delay.

Also drops CSS the page has not rendered since #904: the cursor-spotlight
layer, the breath ring, and the per-card waveform strip.

Verified with headless renders at 1600/1280/940 and the empty state.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(dictation): decode Wayland portal signals and show the capture pill

The GlobalShortcuts portal declares Activated/Deactivated as
(o session, s shortcut_id, t timestamp, a{sv} options). We decoded the
timestamp as u32, so zbus rejected every signal with

  Signature mismatch: got `(osta{sv})`, expected `(osua{sv})`

and the press was dropped as an invalid signal. Registration succeeded
and the desktop even reported the bound chord back, so the hotkey looked
wired up while doing nothing at all — on every Wayland compositor, for
the whole life of the feature (#1490). Decode the 64-bit timestamp, and
keep the 32-bit spelling as a fallback so a non-conforming portal
degrades to working rather than to silence.

With presses arriving, the second half of the failure showed: nothing
had shown the widget window since it became a hidden recorder host, so a
capture ran with no pill on screen — and a mic or Accessibility failure
rendered into a window nobody could see. Add show_dictation_pill, which
bottom-centres the capsule on the monitor under the pointer and shows it
without taking focus (Windows keeps SW_SHOWNOACTIVATE so paste still
lands in the user's document), and call it from the widget for every
state but idle. Wayland denies clients their own placement, so the
compositor picks the spot there; the pill still appears.

dispatch_dictation_capture now logs whether a press was emitted or
queued — a press that reaches Rust and produces nothing was otherwise
indistinguishable from one the compositor never delivered.

Tests: portal signals decode at both timestamp widths (the 64-bit case
fails before this change with the exact production error); pill
placement centres, respects a second monitor's origin, and clamps rather
than going off-screen; the widget shows for a state needing the user,
stays hidden while idle, and never shows for a press that arrives while
dictation is disabled.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: sync in-progress workspace changes

Uncommitted work already in the tree, checkpointed so the branch matches
the local machine:

- Remote GPU workers: join-from-the-app flow, one-time secrets, QR join
  codes, a Compute control in the status bar, and the device-list
  Workers panel (#1516)
- Model Catalogue workspace, with Settings pointing at it
- Settings sidebar search and keyboard navigation
- Demo assets for dubbing, dictation and voice design, plus the scripts
  that render them
- Backend: validation-error handling, ASR request-path degradation, and
  the accompanying tests
- CHANGELOG entries for the above and for the Wayland dictation fix

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(tests): follow Engines to the Model Catalogue, and green the sweep

- test_supertonic3 asserted the license gate points at "Settings" while
  the engine now names Model Catalogue → Engines, which is where the
  accept button actually lives. The assertion follows the move; what it
  pins is unchanged — the hint must name a place the user can reach it.
- Carries the CJK allowlist entries for the rendered dub bundle (#1517)
  and the regenerated route snapshot for /workers/agent (#1516), both of
  which this branch inherits from the workspace sync.
- docs/install/linux.md: the dictation capsule is bottom-anchored
  everywhere except Wayland, where the protocol gives applications no
  say in their placement. Documented rather than left as a surprise
  (CodeRabbit).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* ci: stop a flaky dependency fetch from failing green runs

en-core-web-sm resolves to a direct GitHub release URL, and github.com
intermittently answers `http2 error: refused stream before processing
any application logic`. uv's own three retries all land within the same
few seconds and fail together, so the whole job dies on a dependency
that has nothing to do with the change under test — it cost #1518 and
#1517 an otherwise-green run tonight.

Two changes: back off between whole `uv sync` attempts, which is what
actually clears it, and pass --no-sync to the pytest steps. `uv run`
re-resolves the environment before running, so every test step was a
fresh chance to hit the same fetch even though the install step had
already synced — that is exactly how #1518 failed, in the isolated
backend/tests step, with all 5467 tests already passed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* ci: one retry seam for every uv sync, not just the job that failed last

en-core-web-sm resolves to a direct GitHub *release* URL rather than a
package index, and github.com intermittently answers `http2 error:
refused stream before processing any application logic`. uv's own
retries all land inside the same ~10 seconds and fail together, so a job
dies on a dependency unrelated to the change under test. Tonight that
cost four otherwise-green runs across #1515, #1517 and #1518 — and the
first fix only covered the Tests job, so the next failure simply moved
to Smoke (Linux), which syncs separately.

The fetch is per-job, so the fix has to be per-job: scripts/uv-sync-retry.sh
backs off between whole attempts (15s, 45s, 90s) and every workflow that
syncs now goes through it — ci.yml (tests + the platform matrix),
release.yml, security.yml, evals.yml. It still fails loudly after four
attempts, so a genuinely broken lockfile is not disguised as a flake.

The Tests job also lacked the UV_HTTP_TIMEOUT / UV_HTTP_RETRIES the smoke
matrix has always set, which is part of why it was the one that kept
dying; it has them now.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test(ci): pin the Intel-Mac contract by intent, not by command spelling

test_ci_verifies_intel_mac_as_the_documented_remote_only_host asserted
the literal line `run: uv sync --extra pockettts`, so routing every sync
through scripts/uv-sync-retry.sh read as a broken Intel-Mac contract. The
contract it exists to protect is that the pockettts extra installs ONLY
on backend_supported legs — which the regex now pins, while leaving how
the sync is invoked free to change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* ci: keep every uv run out of the resolver, and bound the retry budget

CodeRabbit, #1517:

- `uv run` re-resolves before running, so the smoke suite, the
  worker-artifact tests, the release test run and the eval run were each
  a fresh chance to hit the flaky direct-URL fetch outside the retry
  loop. All of them pass --no-sync now; the environment is already
  synced by the step that owns the retries. security.yml's
  `uv run --with pip-audit` is deliberately left alone — it layers an
  ephemeral package rather than running the project's own tests.
- The retry count multiplied uv's own budget (UV_HTTP_RETRIES=5 with a
  120 s timeout on the smoke matrix). Three attempts and 60 s of total
  backoff outlast the refusals actually observed while staying well
  inside the jobs' timeout-minutes.
- The Intel-Mac contract test pinned the smoke command literally too, so
  --no-sync tripped it exactly like the sync line did. Same fix: assert
  the contract (smoke runs only on backend_supported legs), not its
  spelling.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 23:46:07 +00:00

4.6 KiB
Raw Blame History

VoiceStudio — dots.tts Engine

dots.tts (rednote-hilab) is a 2B fully-continuous autoregressive TTS, widely cited as one of the strongest open zero-shot voice-cloning models. It covers 24 languages, emits 48 kHz audio, and is released under Apache-2.0 (code + checkpoints).

It runs in its own subprocess and its own Python venv with transformers==4.57.0, isolated from the VoiceStudio parent process which pins transformers>=5.3 — the same isolation primitive used by IndexTTS-2 and MOSS-TTS-v1.5.

Opt-in, and never a default. dots.tts is selected explicitly in Model Catalogue → Engines (or OMNIVOICE_TTS_BACKEND=dots-tts). It is not part of the default install.

Platform support

  • Linux / macOS only. dots.tts's upstream package declares Linux and macOS classifiers and has no Windows install path. On Windows the engine reports itself unavailable in Model Catalogue → Engines with a clear reason — run VoiceStudio under WSL2 or use a Linux/macOS host.
  • No MPS. Upstream device selection is CUDA-or-CPU with no Metal branch, so on Apple Silicon the official package runs on CPU (slow but correct). A faster Apple-Silicon path exists only via community MLX ports, which VoiceStudio does not auto-wire.
  • VRAM: ~9 GB checkpoint; a 1216 GB CUDA GPU is the realistic target.

Install

dots.tts is not bundled (large checkpoint + conflicting transformers).

  1. Clone the dots.tts repo on disk:

    git clone https://github.com/rednote-hilab/dots.tts.git
    
  2. Install the editable package into a fresh venv with the upstream constraints. Use uv pip install -e . -c constraints/recommended.txtnever uv sync --all-extras, which would overwrite VoiceStudio's lock file with transformers==4.57 and break the parent process:

    cd dots.tts
    uv venv .venv
    uv pip install -e . -c constraints/recommended.txt
    
  3. The ~9 GB checkpoint downloads from HuggingFace on first synthesize. The parent forwards HF_HOME / HF_HUB_CACHE to the sidecar so the cache is shared with the rest of VoiceStudio's downloads.

  4. Set OMNIVOICE_DOTS_TTS_DIR to the repo root (the directory that contains pyproject.toml and constraints/):

    # macOS / Linux
    echo 'export OMNIVOICE_DOTS_TTS_DIR=$HOME/code/dots.tts' >> ~/.zshrc
    source ~/.zshrc
    
  5. Restart VoiceStudio. dots.tts appears in Model Catalogue → Engines with available: true and isolation_mode: subprocess.

Venv resolution order

VoiceStudio probes for a usable dots.tts Python interpreter in this priority order (see backend/engines/dots_tts/bootstrap.py):

  1. ${OMNIVOICE_DOTS_TTS_DIR}/.venv/ — your existing clone's venv.
  2. backend/engines/dots_tts/.venv/ — VoiceStudio's own venv, created on demand by step 3.
  3. Lazy bootstrapuv venv then uv pip install -e <clone> -c <clone>/constraints/recommended.txt. Requires OMNIVOICE_DOTS_TTS_DIR.

Voice cloning

For best fidelity ("continuation cloning"), pass both a reference clip (ref_audio) and its exact transcript (ref_text). A reference clip alone does x-vector-only cloning. Keep the reference ~10 s. Upstream requires the reference audio whenever a transcript is given, so VoiceStudio drops a stray ref_text that arrives without ref_audio.

Optional env knobs

Variable Default Purpose
OMNIVOICE_DOTS_TTS_DIR Path to the dots.tts clone (required).
OMNIVOICE_DOTS_TTS_MODEL rednote-hilab/dots.tts-soar Checkpoint override (-base, -soar, -mf).
OMNIVOICE_DOTS_TTS_PRECISION bfloat16 (CUDA) / float32 (CPU) Inference precision.
OMNIVOICE_DOTS_TTS_OPTIMIZE 0 1 enables torch.compile (slower first call, faster after).

Using the dots.tts-mf (MeanFlow-distilled) checkpoint? It's tuned for 4 flow-matching steps — pass num_step=4.

Common errors

dots.tts is not supported on Windows ...

Upstream is Linux/macOS only. Use WSL2 or a Linux/macOS host.

dots.tts venv not found. Set OMNIVOICE_DOTS_TTS_DIR ...

You haven't pointed VoiceStudio at a dots.tts clone yet. Follow Install.

License

Apache-2.0 (code and checkpoints). See the upstream README.


dots.tts runs in a dedicated sidecar venv (it pins transformers==4.57, which conflicts with the parent's transformers>=5.3). For why that adds disk and how uv keeps the cost down, see Engine venvs & disk usage.