Adds opt-in word-timed ASS karaoke captions while preserving the existing line-caption default.\n\nCo-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
* fix(dub): the speaker-count hint is honored on every diarization path + clone-purity guard
The dub "Speakers" count reached _diarize() and then died on 3 of its 4
branches, so setting it changed nothing, speakers blended, and auto-clones
were cut from mixed-speaker audio ("made up" voices):
- FunASR inline-turns shortcut returned before the hint was ever consulted
→ now an explicit num_speakers routes the job through pyannote (the one
engine that honors an exact count); turns stay the fast path only when no
hint is set, and remain the fallback (with an honest "hint ignored"
warning) when pyannote can't load or crashes mid-run.
- pyannote-unavailable fallback used a hardcoded 2-speaker silence-gap
heuristic → assign_speakers_heuristic now takes num_speakers and cycles N
labels on gap boundaries (1 → single speaker; None → legacy alternation),
and the existing diarization warning says the hint is only approximately
honored.
- pyannote-crash fallback dropped the hint the same way → same treatment.
No branch drops the hint silently anymore: every degraded path extends the
existing `warning` SSE payload (detail + a machine-readable speaker_hint
field) that the frontend already renders.
Parity + purity:
- POST /dub/transcribe/{job_id} (the CLI's endpoint) gains the same clamped
num_speakers query param, forwarded to pyannote and the heuristic; the
omnivoice-dub CLI gains --speakers N.
- Clone-purity guard: _pick_reference_slices rejects sub-1.5s slices, prefers
slices not temporally adjacent (<0.3s) to another speaker's turn (scoring
preference, not a hard filter), and extract_speaker_clones skips extraction
entirely when labels came from the heuristic (labels_source kwarg threaded
from _diarize; missing kwarg keeps the old behavior) — with a user-facing
warning pointing at Settings → Models → pyannote.
Tests: fail-before/pass-after coverage in tests/test_speaker_hint.py (all
four _diarize branches driven through the real SSE stream), clone-purity
guards in tests/test_speaker_clone_purity.py, heuristic hint semantics in
tests/test_segmentation.py.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(changelog): add the speaker-hint + clone-purity fix under [Unreleased] (#952)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Segmentation groups words into sentences BEFORE diarization, so a two-speaker
exchange can land in one segment; assign_speakers_* then only relabels it with
the majority speaker, losing the turn boundary (the second half of #486 — the
per-speaker voice auto-assign was fixed in #490).
Add a post-diarization pass that re-splits any segment whose words span >1
speaker at the word-level boundary, assigning each piece its speaker:
- backend/services/segmentation.py: resplit_segments_by_diarization /
resplit_segments_by_turns + a pure _resplit_core. Single-speaker segments are
returned BYTE-FOR-BYTE UNCHANGED (same dict/id/text/start/end) — the
no-single-speaker-regression guarantee. Pieces keep the segment's outer
start/end (preserving onset-snap) and use word times for interior splits, so
they exactly cover the original span. A lone mis-attributed word is smoothed,
not split (diarization noise).
- backend/api/routers/dub_core.py: accumulate global-timeline words alongside
segments; apply the re-split after both the pyannote and FunASR-turns assign.
Heuristic fallback (no word-speaker data) is untouched.
8 regression tests pin the invariant + the split/3-way/noise-smoothing/label
behaviour. Full suite: 1836 passed.
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Autofixes the genuine lint behind the CodeQL py/unused-import and
py/unused-local-variable note-level alerts — actually removing the dead
code rather than dismissing it. 68 safe fixes via 'ruff check --select
F401,F841 --fix' across 29 backend files (dead stdlib/symbol imports like
io/sys/json/torch/typing.Optional and unused locals). Only ruff's safe
fixes applied — the 9 'unsafe' fixes and the audio_dsp numpy availability
import were left untouched.
Not touched: empty-except (needs per-site judgement, not autofixable);
frontend js/unused-local-variable (eslint no-unused-vars has no autofix);
the loopback-low-risk path/log/stack-trace alerts (real, left visible).
Verified: full tests/ suite unchanged at 601 passed (the 2 test_supertonic3
failures are pre-existing on main, local .venv state, green in CI).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When FunASR is the active ASR backend, use its cam++ per-segment speaker IDs
directly and skip pyannote — the 'all-in-one' diarization the issue asked for.
- asr_backend: FunASRBackend loads spk_model='cam++' (ASR_FUNASR_SPK, set '' to
disable); transcribe()/_normalize_funasr already surface per-segment speakers.
- segmentation.assign_speakers_from_turns(segments, turns): generalised
overlap-weighted speaker assignment from {start,end,speaker} turns (mirrors
assign_speakers_from_diarization without a pyannote object; falls back to the
silence-gap heuristic when no turns). Pure + tested.
- dub_core: _transcribe_chunk collects offset-shifted speaker turns; they
accumulate across chunks; _diarize() uses them and skips pyannote when present.
Default (WhisperX) flow unchanged — no turns → existing pyannote/heuristic path.
4 new tests (overlap winner, containing turn, malformed-turn filtering, empty→
heuristic). Router smoke confirms boot.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Matt Van Horn