Files
VoiceStudio/docs/features/diarization.md
T
Palash DebnathandClaude Opus 4.7 715766cb04 Phase 1 Wave 2: per-OS install docs + Settings UI + error→docs deeplinks (#94)
* docs(install): per-OS install pages + drift validator + CI gate

Splits the 600-line README install section into self-contained per-OS docs
under docs/install/{macos,windows,linux,docker}.md plus a Top-10
troubleshooting index. Each OS doc is end-to-end: a user opens it and
reaches a working app following only commands inside that file.

Adds:
- docs/install/{macos,windows,linux,docker}.md  (OS-specific install paths)
- docs/install/troubleshooting.md               (top 10 install errors)
- docs/engines/cosyvoice.md                     (closes #55 docs half)
- docs/features/diarization.md                  (pyannote license flow)
- docs/setup/huggingface-token.md               (3-source cascade guide)
- scripts/validate-install-docs.py              (INST-06 docs-drift gate)
- tests/scripts/test_validate_install_docs.py   (B-5: validator self-tests)
- .github/workflows/ci.yml step running the validator on every PR

Implements INST-02 (README routing), INST-03 (macOS Gatekeeper anchor),
INST-12 docs half (Windows torch-compile-oom anchor), DOCS-01..05.

The validator is a one-way diff: every `<!-- validate -->`-tagged line
in docs must appear in scripts/desktop-prod.sh after normalisation
(prompt-prefix strip, CRLF, trailing whitespace, blank-and-comment skip).
A `<!-- validate: skip -->` marker opts out for human-readability blocks.
Its own 10 unit tests catch regressions in the gate itself.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(deeplinks): links.py + error_docs_map (Python + TS mirror)

Adds the single source of truth for the project repo URL and the 4-class
error → docs taxonomy that both the in-app ErrorBoundary deeplink button
(Wave 2 Task 3) and the Phase 5 bug reporter will consume.

New:
- backend/core/links.py            — PROJECT_REPO_URL + BLOB_MAIN resolver
                                      (Tauri config first, pyproject fallback)
- backend/core/error_docs_map.py   — lookup(error_class) → docs URL
- frontend/src/utils/errorDocsMap.ts (TS mirror with classifyError helper)
- tests/backend/core/test_links.py + test_error_docs_map.py
- frontend/src/utils/errorDocsMap.test.ts

Resolves checker B-6 (links.py ownership) and Open Question #3 (which fork
the deeplinks resolve to — the Tauri updater endpoint wins, which points
at the desktop app fork debpalash/OmniVoice-Studio).

The TS BASE constant is documented as the second hardcoded URL drift site;
the keys-sync test (`test_keys_match_python_map` equivalent) guards the
4-class taxonomy contract between Python + TS halves.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(ui): Settings → API Keys panel + ErrorBoundary docs deeplink

Wave 2 AUTH-03 UI half + ErrorBoundary deeplink wiring.

ErrorBoundary fallback now renders an "Open docs for this error" button
that classifies the thrown Error message (heuristic: pkg_resources → 401 /
HfHubHTTP → WebKit / white screen → quarantine / Gatekeeper) and opens the
matching docs anchor via Tauri shell.open (with a window.open fallback
in browser dev mode).

ApiKeysPanel consumes the Wave 1 resolver state endpoint:
  - 3 source rows (App / Env var / HF CLI) with set/unset indicator,
    masked token preview, whoami username + green check
  - "Active" badge on whichever source is currently serving the cascade
  - App-row only: Save (POST /api/settings/hf-token) +
    Clear (DELETE with optional "also clear HF CLI" confirm dialog)
  - "Test now" button refetches state (invalidates the resolver's
    validation cache via the same endpoint hit)

Panel mounted in the existing Settings → Credentials tab; the legacy
HF_TOKEN row from CREDENTIAL_FIELDS is filtered out so the two paths
don't fight over the same key.

Threat T-02-02: the panel never displays the full token. The masked
value comes from the resolver state endpoint; the full token only
crosses the IPC boundary on Save (POST) and is cleared from local
state on success.

Closes AUTH-03 fully (Wave 1 backend + this Wave 2 UI).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(perf): INST-12 Disable torch.compile (Windows) toggle (backend + UI)

Wave 2 Task 4 — full INST-12 delivery per checker B-2/B-7 v0.3.0 fat-release
decision. Both the docs half (windows.md anchor, shipped in earlier commit)
and the runtime toggle are now in Phase 1.

Backend:
- backend/services/settings_store.py: adds get_text/set_text helpers for
  non-secret config (refuses to write to the encrypted hf_token key).
- backend/api/routers/settings.py: GET + PUT
  /api/settings/perf/torch-compile-disabled, both under the existing
  loopback guard (threat T-02-04).
- backend/services/engine_env.py: new `build_engine_env()` helper that
  centralises HF_TOKEN/YOUR_HF_TOKEN injection from the 3-source resolver
  AND injects TORCH_COMPILE_DISABLE=1 when the flag is set on win32.
  Phase 2 SubprocessBackend launchers should adopt the same helper.
- backend/services/sonitranslate.py: migrated to engine_env.build_engine_env()
  while preserving the source-level `env["HF_TOKEN"]` sentinel that
  test_sonitranslate_module_uses_resolver checks.

Frontend:
- frontend/src/components/settings/PerformancePanel.{jsx,css,test.jsx}:
  toggle UI with the explainer for #65; renders disabled with a "not
  applicable" badge on macOS/Linux.
- frontend/src/pages/Settings.jsx: mounts the panel into the Credentials
  tab alongside the API Keys panel.

Tests:
- tests/backend/test_perf_settings.py: 7 backend tests (default state,
  PUT persistence, T-02-04 non-loopback rejection, settings_store round-
  trip, env injection on win32, NO injection on macOS/Linux, NO injection
  when disabled).
- frontend PerformancePanel.test.jsx: 5 tests (renders from GET state,
  PUT on toggle, disabled on non-Windows platforms, pre-enabled state).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(planning): Wave 2 SUMMARY + REQUIREMENTS status updates

- .planning/phases/01.../01-02-SUMMARY.md: full implementation report
  per template (truths, commits, tests, deviations, drift-site
  acknowledgments per W-3, launcher seam name for Phase 2,
  taxonomy keys for Phase 5).
- .planning/REQUIREMENTS.md: flips Wave 2 closures to Done:
    AUTH-03, INST-02, INST-03 (docs half), INST-06, INST-12,
    DOCS-01..05.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 06:22:10 +05:30

2.8 KiB

OmniVoice Studio — Speaker Diarization

Diarization splits a single audio stream into per-speaker tracks: who said what, and when. OmniVoice uses pyannote + WhisperX under the hood — the same stack the original WhisperX paper used.

What diarization buys you

  • Multi-speaker dubbing: each detected speaker gets its own voice clone in the target language.
  • Subtitle styling: speaker labels (SPEAKER_00:, SPEAKER_01:, …) on the exported SRT/VTT files.
  • Audio editing: per-speaker tracks in the timeline view.

License acceptance flow

The diarization model — pyannote/speaker-diarization-3.1 — is gated on HuggingFace. A valid HF token alone is not enough: you also need to accept the model's license once.

  1. Get a HF token if you don't have one — see docs/setup/huggingface-token.md.
  2. Set the token via Settings → API Keys (or any of the other supported paths).
  3. While signed in to HuggingFace with the same account, visit:
  4. Restart the dub job. The first run downloads ~600 MB of model weights.

If you skip the license acceptance, the HF API returns 401 Unauthorized for the download — the same error class the in-app "Open docs for this error" button deeplinks to.

Fallback behaviour

When diarization is unavailable (no HF token, license not accepted, model download failed mid-run), OmniVoice's dub pipeline falls back to a silence-gap heuristic that splits speakers on long quiet stretches. You'll see a warning toast and the dub_core.py reason string surfaces in the job log:

  • "diarization_skipped:no_token" — no token resolved from the cascade.
  • "diarization_skipped:401" — token present but unauthorised on the gated model (license not accepted).
  • "diarization_skipped:network" — model download interrupted.

The heuristic is not as accurate as pyannote — speakers with similar pitch or rapid turn-taking conversation get merged — but it lets the dub finish end-to-end instead of erroring.

HF token requirement

Diarization is the one OmniVoice feature where a HF token is required, not just recommended. See docs/setup/huggingface-token.md for the three-source cascade and how the in-app Settings → API Keys panel works.

Troubleshooting

  • HF 401 → see troubleshooting.md#2-hf-401--pyannote-license-not-accepted.
  • Model download stuck → check ~/.cache/huggingface/hub/models--pyannote--* size grows during the dub; if it stalls at 0 bytes, your token isn't being read — confirm in Settings → API Keys that the active source has a green checkmark.