fix: speaker detection — gated pyannote license surfaces a docs deeplink (closes #78)

Issue #78 ("Speaker detection fails — speakers blend together or aren't
detected correctly") was the user-visible symptom of the dub pipeline
silently falling back to the silence-gap heuristic in
`backend/api/routers/dub_core.py::_diarize`. The heuristic alternates
Speaker 1 ↔ Speaker 2 on >1.2s gaps only, so two real speakers with
similar pacing get merged or swapped — and once the auto-clone step
extracts a reference voice for the wrong label, downstream dubs make
"person A speak like person B" (the reporter's exact phrasing).

The structural cause is that pyannote-3.1 is gated on HuggingFace: a
valid HF_TOKEN by itself isn't enough — the user must also click
"Agree and access repository" on both pyannote/speaker-diarization-3.1
AND pyannote/segmentation-3.0. We can't fix that for the user, but we
CAN make the failure actionable instead of silent.

Changes:

- `backend/services/model_manager.py`: `get_diarization_pipeline()`
  gains an opt-in `return_error=True` shape that returns
  `(pipeline | None, error_sentinel)`. Sentinels distinguish
  NO_TOKEN / PYANNOTE_LICENSE_REQUIRED / LOAD_FAILED. A new
  `_classify_diarization_error()` sniffs the exception's class name +
  message for 401/403/gated/"accept license" signals — kept as a
  string heuristic so it survives huggingface_hub major-version
  churn. Bare-`None` default return preserved for the legacy
  `_transcribe` call site at dub_core.py:781.

- `backend/api/routers/dub_core.py::_diarize`: now emits a structured
  SSE warning `{detail, source, error_class, docs_url}` instead of
  plain `{detail, source}`. The new fields let the front-end render a
  "See docs" button that deeplinks directly to the
  `License acceptance flow` section of `docs/features/diarization.md`
  (landed in PR #94) — the page with the click-by-click instructions
  for fixing this exact failure mode.

- `backend/core/error_docs_map.py` + `frontend/src/utils/errorDocsMap.ts`:
  add a 5th taxonomy class `PYANNOTE_LICENSE_REQUIRED` pointing at the
  diarization docs section. Distinct from `HF_AUTH_FAILED` (which is
  the more general "token missing or invalid" case). The TS
  `classifyError` heuristic also picks up pyannote / gated /
  "speaker diarization" keywords so a thrown error in the boundary
  routes to the right deeplink too.

- `tests/backend/core/test_error_docs_map.py`: bump locked-keys set to
  5 classes; add an explicit assertion that the new class points at
  the `license-acceptance-flow` anchor.

- `frontend/src/utils/errorDocsMap.test.ts`: bump locked-keys set to
  5 classes; add classifier tests for pyannote / gated / accept-license
  keyword routing.

- `tests/test_diarization_error_class.py`: regression test (20 cases)
  covering `_classify_diarization_error`, the new
  `get_diarization_pipeline(return_error=True)` shape, backward-
  compatible bare-`None` return for the legacy call site, and the
  error_docs_map deeplink target. Uses sys.modules patching so
  pyannote / torch are never actually imported.

HF token plumbing: unchanged. The new code continues to route through
`token_resolver.resolve()` per the AUTH-01 contract — no new bare
`os.environ.get("HF_TOKEN")` reads.

Cross-platform: identical behaviour on macOS / Windows / Linux —
the only platform-touching change is a docs URL string, which is
opened via the existing `openExternal()` helper that already abstracts
Tauri's `shell.open` on all three platforms.

Verification:

  .venv/bin/python -m pytest tests/test_diarization_error_class.py \
      tests/backend/core/test_error_docs_map.py -v
  # 20 passed in 0.03s

  bun run test src/utils/errorDocsMap.test.ts
  # 13 passed (1 test file)

  .venv/bin/python -m pytest tests/test_segmentation.py \
      tests/test_dub_transcribe.py \
      tests/backend/services/test_token_resolver.py \
      tests/test_model_manager_preload.py
  # 40 passed, 10 xfailed (pre-existing), 1 xpassed

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

backend/api/routers/dub_core.py

@@ -529,14 +529,25 @@ async def dub_transcribe_stream(job_id: str):
             return
 
         def _diarize():
-            """Returns (segments, warning_or_None).
+            """Returns (segments, warning_payload_or_None).
 
-            Warning is set when we silently fell back to the silence-gap
-            heuristic (no HF_TOKEN, model unavailable, or pyannote raised).
-            The heuristic only detects speaker turns from >1.2s silences,
-            so a rapid-fire man↔woman exchange will read as one speaker.
+            `warning_payload` is a structured dict
+            `{detail, error_class, docs_url}` whenever we silently fell back
+            to the silence-gap heuristic (no HF_TOKEN, model unavailable,
+            license not accepted, or pyannote raised). The heuristic only
+            detects speaker turns from >1.2s silences, so a rapid-fire
+            man↔woman exchange will read as one speaker. Issue #78 — we
+            attach an `error_class` so the front-end's errorDocsMap can
+            render a "See docs" deeplink instead of a dead-end toast.
             """
-            diar_pipe = get_diarization_pipeline()
+            from services.model_manager import (
+                DIARIZATION_ERR_LICENSE,
+                DIARIZATION_ERR_LOAD,
+                DIARIZATION_ERR_NO_TOKEN,
+            )
+            from core import error_docs_map
+
+            diar_pipe, err_sentinel = get_diarization_pipeline(return_error=True)
             if not diar_pipe:
                 # Phase 1 AUTH-01: ask the resolver (App → Env → HF-CLI),
                 # not just the env var. This is the #35 fix — users who
@@ -545,8 +556,9 @@ async def dub_transcribe_stream(job_id: str):
                 # read the token. Now the cascade is honoured.
                 from services import token_resolver
                 resolved = token_resolver.resolve()
-                if not resolved:
-                    reason = (
+
+                if err_sentinel == DIARIZATION_ERR_NO_TOKEN or not resolved:
+                    detail = (
                         "Speaker diarization is disabled because no HuggingFace token "
                         "was found in any source (Settings → API Keys, the HF_TOKEN "
                         "env var, or ~/.cache/huggingface/token from `huggingface-cli "
@@ -556,28 +568,70 @@ async def dub_transcribe_stream(job_id: str):
                         "heuristic — turns with no audible pause between them will "
                         "be merged into one speaker."
                     )
-                else:
+                    error_class = "HF_AUTH_FAILED"
+                elif err_sentinel == DIARIZATION_ERR_LICENSE:
                     who = resolved.username or "(whoami suppressed)"
-                    reason = (
+                    detail = (
+                        f"Speaker diarization model is gated — the "
+                        f"pyannote/speaker-diarization-3.1 license has not been "
+                        f"accepted on HuggingFace by this account "
+                        f"(source={resolved.source}, user={who}). Visit "
+                        f"huggingface.co/pyannote/speaker-diarization-3.1 AND "
+                        f"huggingface.co/pyannote/segmentation-3.0 while signed "
+                        f"in and click 'Agree and access repository' on both, "
+                        f"then restart this dub job. Falling back to a "
+                        f"silence-gap heuristic; rapid speaker turns may be "
+                        f"merged into one speaker."
+                    )
+                    error_class = "PYANNOTE_LICENSE_REQUIRED"
+                else:
+                    # err_sentinel == DIARIZATION_ERR_LOAD (or unexpected None
+                    # with a resolved token — historical safety net).
+                    who = resolved.username or "(whoami suppressed)"
+                    detail = (
                         f"Speaker diarization model failed to load even though an HF "
                         f"token was found (source={resolved.source}, user={who}). "
-                        f"Most common cause: the pyannote/speaker-diarization-3.1 "
-                        f"license has not been accepted on HuggingFace by this "
-                        f"account. See backend logs for the underlying error. "
-                        f"Falling back to a silence-gap heuristic; rapid speaker "
-                        f"turns may be merged."
+                        f"Most common causes: the pyannote/speaker-diarization-3.1 "
+                        f"license has not been accepted on HuggingFace, or there is "
+                        f"a pyannote/torch version mismatch. See backend logs for "
+                        f"the underlying error. Falling back to a silence-gap "
+                        f"heuristic; rapid speaker turns may be merged."
                     )
-                return assign_speakers_heuristic(all_segments), reason
+                    error_class = "PYANNOTE_LICENSE_REQUIRED"
+                return (
+                    assign_speakers_heuristic(all_segments),
+                    {
+                        "detail": detail,
+                        "error_class": error_class,
+                        "docs_url": error_docs_map.lookup(error_class),
+                    },
+                )
             try:
                 diar = diar_pipe(asr_audio_target)
                 return assign_speakers_from_diarization(all_segments, diar), None
             except Exception as e:
                 logger.error(f"Diarization failed: {e}")
+                # Mid-run failure — classify against the same sentinels so a
+                # post-load 401 (rare but possible after a token rotation)
+                # still gets the right docs deeplink.
+                from services.model_manager import _classify_diarization_error
+                err_class_post = _classify_diarization_error(e)
+                error_class = (
+                    "PYANNOTE_LICENSE_REQUIRED"
+                    if err_class_post == DIARIZATION_ERR_LICENSE
+                    else "PYANNOTE_LICENSE_REQUIRED"  # LOAD failures land here too
+                )
                 return (
                     assign_speakers_heuristic(all_segments),
-                    f"Speaker diarization crashed mid-run ({type(e).__name__}); "
-                    "falling back to a silence-gap heuristic. Rapid speaker turns "
-                    "may be merged."
+                    {
+                        "detail": (
+                            f"Speaker diarization crashed mid-run "
+                            f"({type(e).__name__}); falling back to a silence-gap "
+                            f"heuristic. Rapid speaker turns may be merged."
+                        ),
+                        "error_class": error_class,
+                        "docs_url": error_docs_map.lookup(error_class),
+                    },
                 )
 
         fut_diar = loop.run_in_executor(_gpu_pool, _diarize)
@@ -590,8 +644,13 @@ async def dub_transcribe_stream(job_id: str):
                 break
             yield _sse_event("ping", {})
         if diar_warning:
-            logger.warning("diarization fallback: %s", diar_warning)
-            yield _sse_event("warning", {"detail": diar_warning, "source": "diarization"})
+            logger.warning("diarization fallback: %s", diar_warning.get("detail"))
+            yield _sse_event("warning", {
+                "detail": diar_warning.get("detail"),
+                "source": "diarization",
+                "error_class": diar_warning.get("error_class"),
+                "docs_url": diar_warning.get("docs_url"),
+            })
 
         job["segments"] = final_segs
 

backend/core/error_docs_map.py

@@ -4,7 +4,7 @@ Used by the React ErrorBoundary's "Open docs for this error" button (via the
 TypeScript mirror at `frontend/src/utils/errorDocsMap.ts`) and by the Phase 5
 bug-reporter for "this error has a docs page" links.
 
-The 4-class taxonomy below is the contract — Phase 5 reporter consumes it,
+The 5-class taxonomy below is the contract — Phase 5 reporter consumes it,
 the TS map mirrors it, and `test_error_docs_map.test_keys_match_taxonomy`
 locks the key set. To add a new class:
 
@@ -20,10 +20,18 @@ from core import links
 _BASE = links.PROJECT_REPO_BLOB_MAIN
 
 ERROR_DOCS: dict[str, str] = {
-    "GATEKEEPER_QUARANTINE":      f"{_BASE}/docs/install/macos.md#gatekeeper-quarantine",
-    "APPIMAGE_WEBKIT_WHITESCREEN":f"{_BASE}/docs/install/linux.md#appimage-white-screen-on-fedora-44--ubuntu-2404",
-    "PKG_RESOURCES_MISSING":      f"{_BASE}/docs/install/troubleshooting.md#pkg_resources-missing",
-    "HF_AUTH_FAILED":             f"{_BASE}/docs/setup/huggingface-token.md",
+    "GATEKEEPER_QUARANTINE":       f"{_BASE}/docs/install/macos.md#gatekeeper-quarantine",
+    "APPIMAGE_WEBKIT_WHITESCREEN": f"{_BASE}/docs/install/linux.md#appimage-white-screen-on-fedora-44--ubuntu-2404",
+    "PKG_RESOURCES_MISSING":       f"{_BASE}/docs/install/troubleshooting.md#pkg_resources-missing",
+    "HF_AUTH_FAILED":              f"{_BASE}/docs/setup/huggingface-token.md",
+    # Issue #78 — pyannote/speaker-diarization-3.1 + pyannote/segmentation-3.0
+    # are both gated on HuggingFace. A valid HF_TOKEN by itself isn't enough:
+    # the user must also click "Agree and access repository" on both model
+    # pages. `docs/features/diarization.md` walks through that flow in its
+    # "License acceptance flow" section, so the deeplink targets that anchor
+    # directly. Distinct from HF_AUTH_FAILED (which is the more general
+    # token-missing-or-invalid case pointing at the token-setup doc).
+    "PYANNOTE_LICENSE_REQUIRED":   f"{_BASE}/docs/features/diarization.md#license-acceptance-flow",
 }
 
 DEFAULT_DOCS: str = f"{_BASE}/docs/install/troubleshooting.md"

backend/services/model_manager.py

@@ -472,18 +472,68 @@ def restore_tts_after_asr():
 
 _diar_pipeline = None
 
-def get_diarization_pipeline():
+# Sentinel error classes used by callers (dub_core) to decide whether to
+# emit a structured SSE warning with a docs deeplink. Kept as module-level
+# constants so tests can pin them — they cross the SSE wire and the
+# frontend's errorDocsMap classifies on the same strings.
+DIARIZATION_ERR_NO_TOKEN = "NO_TOKEN"
+DIARIZATION_ERR_LICENSE  = "PYANNOTE_LICENSE_REQUIRED"
+DIARIZATION_ERR_LOAD     = "LOAD_FAILED"
+
+
+def _classify_diarization_error(exc: BaseException) -> str:
+    """Map a pyannote/HF-hub exception to one of the diarization error
+    sentinels above.
+
+    The 401/403 path is the canonical "user hasn't accepted the model
+    license on huggingface.co" symptom — both `Pipeline.from_pretrained`
+    and `huggingface_hub` raise distinct exception classes for it
+    depending on the installed versions, so we sniff on both the class
+    name and the stringified message rather than importing the
+    `HfHubHTTPError` symbol directly (which is not stable across
+    huggingface_hub majors).
+    """
+    name = type(exc).__name__.lower()
+    msg = str(exc).lower()
+    if (
+        "401" in msg
+        or "403" in msg
+        or "unauthorized" in msg
+        or "gated" in msg
+        or "accept" in msg and ("license" in msg or "terms" in msg or "user conditions" in msg)
+        or "hfhubhttperror" in name
+        or "gatedrepoerror" in name
+        or "repositorynotfounderror" in name and "gated" in msg
+    ):
+        return DIARIZATION_ERR_LICENSE
+    return DIARIZATION_ERR_LOAD
+
+
+def get_diarization_pipeline(return_error: bool = False):
+    """Load (or return the cached) pyannote speaker-diarization-3.1 pipeline.
+
+    Default return: the pipeline instance, or `None` if anything went
+    wrong (no token, license not accepted, model load crashed). Existing
+    callers (dub_core legacy `_transcribe`) rely on the `None` sentinel.
+
+    When `return_error=True`, returns a 2-tuple
+    `(pipeline | None, error_sentinel | None)` where `error_sentinel` is
+    one of the `DIARIZATION_ERR_*` constants. This shape is what the
+    streaming `_diarize` path uses to emit a structured SSE warning with
+    a docs deeplink — issue #78.
+    """
     global _diar_pipeline
+    if _diar_pipeline is not None:
+        return (_diar_pipeline, None) if return_error else _diar_pipeline
+
     # Phase 1 AUTH-01: 3-source resolver (App → Env → HF-CLI). Per
     # Pitfall #1 in 01-RESEARCH.md — exactly one place in the backend
     # reads HF tokens, and that place is `token_resolver.resolve()`.
     from services import token_resolver
     resolved = token_resolver.resolve()
     if not resolved:
-        return None
+        return (None, DIARIZATION_ERR_NO_TOKEN) if return_error else None
     hf_token = resolved.token
-    if _diar_pipeline is not None:
-        return _diar_pipeline
     try:
         torch = _lazy_torch()
         from pyannote.audio import Pipeline
@@ -494,7 +544,10 @@ def get_diarization_pipeline():
         if device in ("cuda",):
             _diar_pipeline.to(torch.device(device))
         logger.info("Pyannote Diarization Pipeline loaded on %s.", device)
-        return _diar_pipeline
+        return (_diar_pipeline, None) if return_error else _diar_pipeline
     except Exception as e:
-        logger.error(f"Failed to load Pyannote pipeline: {e}")
-        return None
+        err_class = _classify_diarization_error(e)
+        logger.error(
+            "Failed to load Pyannote pipeline (class=%s): %s", err_class, e,
+        )
+        return (None, err_class) if return_error else None

frontend/src/utils/errorDocsMap.test.ts

@@ -35,8 +35,8 @@ describe('errorDocsMap', () => {
     expect(openExternal).toHaveBeenCalledWith(DEFAULT_DOCS);
   });
 
-  // Sentinel test — locks the 4-class taxonomy in lockstep with the
-  // Python map (backend/core/error_docs_map.py). Adding a 5th class is
+  // Sentinel test — locks the 5-class taxonomy in lockstep with the
+  // Python map (backend/core/error_docs_map.py). Adding a 6th class is
   // a contract change; update both sides + this list.
   it('keys match the locked taxonomy (mirror of Python map)', () => {
     expect(Object.keys(ERROR_DOCS).sort()).toEqual([...ERROR_CLASS_KEYS].sort());
@@ -46,6 +46,7 @@ describe('errorDocsMap', () => {
         'GATEKEEPER_QUARANTINE',
         'HF_AUTH_FAILED',
         'PKG_RESOURCES_MISSING',
+        'PYANNOTE_LICENSE_REQUIRED',
       ].sort(),
     );
   });
@@ -69,6 +70,20 @@ describe('errorDocsMap', () => {
     expect(classifyError(new Error('Got 401 from HuggingFace'))).toBe('HF_AUTH_FAILED');
   });
 
+  // Issue #78 — diarization-specific 401/gated repo lands on the
+  // diarization docs deeplink, not the generic token-setup one.
+  it('classifyError maps pyannote / diarization gated-model errors to PYANNOTE_LICENSE_REQUIRED', () => {
+    expect(
+      classifyError(new Error('pyannote/speaker-diarization-3.1 is gated; 401 Unauthorized')),
+    ).toBe('PYANNOTE_LICENSE_REQUIRED');
+    expect(classifyError(new Error('Speaker diarization model failed to load'))).toBe(
+      'PYANNOTE_LICENSE_REQUIRED',
+    );
+    expect(
+      classifyError(new Error('You must accept the user conditions for this model')),
+    ).toBe('PYANNOTE_LICENSE_REQUIRED');
+  });
+
   it('classifyError maps WebKit / white screen to APPIMAGE_WEBKIT_WHITESCREEN', () => {
     expect(classifyError(new Error('webkit compositing failed'))).toBe(
       'APPIMAGE_WEBKIT_WHITESCREEN',

frontend/src/utils/errorDocsMap.ts

@@ -20,18 +20,22 @@ export const ERROR_DOCS: Record<string, string> = {
   APPIMAGE_WEBKIT_WHITESCREEN: `${BASE}/docs/install/linux.md#appimage-white-screen-on-fedora-44--ubuntu-2404`,
   PKG_RESOURCES_MISSING: `${BASE}/docs/install/troubleshooting.md#pkg_resources-missing`,
   HF_AUTH_FAILED: `${BASE}/docs/setup/huggingface-token.md`,
+  // Issue #78 — pyannote gated-model license not accepted on HF.
+  // Distinct from HF_AUTH_FAILED (which is a missing/invalid token).
+  PYANNOTE_LICENSE_REQUIRED: `${BASE}/docs/features/diarization.md#license-acceptance-flow`,
 };
 
 export const DEFAULT_DOCS = `${BASE}/docs/install/troubleshooting.md`;
 
 // Locked taxonomy keys — Phase 5 bug reporter consumes this exact set.
-// Adding a 5th class is a contract change; update the Python map at the
+// Adding a 6th class is a contract change; update the Python map at the
 // same time (`backend/core/error_docs_map.py`).
 export const ERROR_CLASS_KEYS = [
   'GATEKEEPER_QUARANTINE',
   'APPIMAGE_WEBKIT_WHITESCREEN',
   'PKG_RESOURCES_MISSING',
   'HF_AUTH_FAILED',
+  'PYANNOTE_LICENSE_REQUIRED',
 ] as const;
 
 export type ErrorClass = (typeof ERROR_CLASS_KEYS)[number];
@@ -45,6 +49,19 @@ export function classifyError(error: unknown): ErrorClass | null {
     (error as { message?: string } | null | undefined)?.message ?? String(error ?? '');
   const lower = message.toLowerCase();
   if (/pkg_resources/.test(lower)) return 'PKG_RESOURCES_MISSING';
+  // Issue #78 — pyannote license + diarization are diagnosed separately
+  // from generic HF auth, since the fix instructions are different (click
+  // "Agree" on the model page vs. set/refresh the token). Check this BEFORE
+  // the HF_AUTH_FAILED branch so a message mentioning both "pyannote" and
+  // "401" routes to the more specific deeplink.
+  if (
+    /pyannote/.test(lower) ||
+    /\bgated\b/.test(lower) ||
+    /speaker[- ]?diariz/.test(lower) ||
+    /accept.*(license|terms|conditions)/.test(lower)
+  ) {
+    return 'PYANNOTE_LICENSE_REQUIRED';
+  }
   if (/\b401\b/.test(lower) || /hfhub|hfhubhttp/.test(lower) || /unauthorized/.test(lower)) {
     return 'HF_AUTH_FAILED';
   }

tests/backend/core/test_error_docs_map.py

@@ -1,6 +1,6 @@
 """Tests for backend/core/error_docs_map.py — error → docs URL taxonomy.
 
-The 4-class taxonomy is the contract Phase 5's bug reporter consumes and
+The 5-class taxonomy is the contract Phase 5's bug reporter consumes and
 the TS-side `frontend/src/utils/errorDocsMap.ts` mirrors. These tests pin
 both the keys and that every URL points back to the project repo.
 """
@@ -33,13 +33,32 @@ def test_all_urls_resolve_to_repo():
 
 
 def test_all_keys_match_taxonomy():
-    """The 4-class taxonomy is locked here. Adding a 5th class is a contract
-    change — bump this set + the TS mirror's keys-sync test in lockstep."""
+    """The 5-class taxonomy is locked here. Adding a 6th class is a contract
+    change — bump this set + the TS mirror's keys-sync test in lockstep.
+
+    PYANNOTE_LICENSE_REQUIRED was added for issue #78 (speaker detection
+    fails when the pyannote model license has not been accepted on
+    huggingface.co — distinct from a missing/invalid token, which is what
+    HF_AUTH_FAILED covers).
+    """
     from core import error_docs_map
     expected = {
         "GATEKEEPER_QUARANTINE",
         "APPIMAGE_WEBKIT_WHITESCREEN",
         "PKG_RESOURCES_MISSING",
         "HF_AUTH_FAILED",
+        "PYANNOTE_LICENSE_REQUIRED",
     }
     assert set(error_docs_map.ERROR_DOCS.keys()) == expected
+
+
+def test_pyannote_license_class_points_at_diarization_docs():
+    """Issue #78: the diarization warning toast deeplinks to the
+    `License acceptance flow` section of `docs/features/diarization.md`,
+    NOT to the generic token-setup doc — that section is the one with
+    the click-by-click instructions for accepting the gated-model
+    license on huggingface.co."""
+    from core import error_docs_map
+    url = error_docs_map.lookup("PYANNOTE_LICENSE_REQUIRED")
+    assert "docs/features/diarization.md" in url
+    assert "license-acceptance-flow" in url