Commit Graph
4 Commits
Author SHA1 Message Date
a63c8e851b fix(dub): speaker-aware re-split so merged speaker turns separate (#486) (#616)
Segmentation groups words into sentences BEFORE diarization, so a two-speaker
exchange can land in one segment; assign_speakers_* then only relabels it with
the majority speaker, losing the turn boundary (the second half of #486 — the
per-speaker voice auto-assign was fixed in #490).

Add a post-diarization pass that re-splits any segment whose words span >1
speaker at the word-level boundary, assigning each piece its speaker:
- backend/services/segmentation.py: resplit_segments_by_diarization /
  resplit_segments_by_turns + a pure _resplit_core. Single-speaker segments are
  returned BYTE-FOR-BYTE UNCHANGED (same dict/id/text/start/end) — the
  no-single-speaker-regression guarantee. Pieces keep the segment's outer
  start/end (preserving onset-snap) and use word times for interior splits, so
  they exactly cover the original span. A lone mis-attributed word is smoothed,
  not split (diarization noise).
- backend/api/routers/dub_core.py: accumulate global-timeline words alongside
  segments; apply the re-split after both the pyannote and FunASR-turns assign.
  Heuristic fallback (no word-speaker data) is untouched.

8 regression tests pin the invariant + the split/3-way/noise-smoothing/label
behaviour. Full suite: 1836 passed.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 22:39:07 +05:30
Palash DebnathandClaude Opus 4.8 47729057bd chore(lint): remove unused imports + variables (ruff F401/F841) (#210)
Autofixes the genuine lint behind the CodeQL py/unused-import and
py/unused-local-variable note-level alerts — actually removing the dead
code rather than dismissing it. 68 safe fixes via 'ruff check --select
F401,F841 --fix' across 29 backend files (dead stdlib/symbol imports like
io/sys/json/torch/typing.Optional and unused locals). Only ruff's safe
fixes applied — the 9 'unsafe' fixes and the audio_dsp numpy availability
import were left untouched.

Not touched: empty-except (needs per-site judgement, not autofixable);
frontend js/unused-local-variable (eslint no-unused-vars has no autofix);
the loopback-low-risk path/log/stack-trace alerts (real, left visible).

Verified: full tests/ suite unchanged at 601 passed (the 2 test_supertonic3
failures are pre-existing on main, local .venv state, green in CI).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 07:53:06 +05:30
Palash DebnathandClaude Opus 4.8 cd155f4384 feat(dub): FunASR cam++ inline diarization → dub speakers (#182 Phase 2) (#197)
When FunASR is the active ASR backend, use its cam++ per-segment speaker IDs
directly and skip pyannote — the 'all-in-one' diarization the issue asked for.

- asr_backend: FunASRBackend loads spk_model='cam++' (ASR_FUNASR_SPK, set '' to
  disable); transcribe()/_normalize_funasr already surface per-segment speakers.
- segmentation.assign_speakers_from_turns(segments, turns): generalised
  overlap-weighted speaker assignment from {start,end,speaker} turns (mirrors
  assign_speakers_from_diarization without a pyannote object; falls back to the
  silence-gap heuristic when no turns). Pure + tested.
- dub_core: _transcribe_chunk collects offset-shifted speaker turns; they
  accumulate across chunks; _diarize() uses them and skips pyannote when present.
  Default (WhisperX) flow unchanged — no turns → existing pyannote/heuristic path.

4 new tests (overlap winner, containing turn, malformed-turn filtering, empty→
heuristic). Router smoke confirms boot.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-31 05:43:33 +05:30
debpalashandClaude Opus 4.7 67328d04fe refactor: split backend into api/core/services/schemas, harden security + fd pressure, add searchable language picker, fix segment fragmentation
Backend:
- Split monolithic main.py into backend/{api/routers,core,schemas,services}
- core/db.py: allowlist-gated migrations, db_conn context manager (kills SQL injection on ALTER)
- core/tasks.py: lock-guarded listener add/remove/push, snapshot-before-iterate
- services/ffmpeg_utils.py: run_ffmpeg helper with concurrency semaphore, EAGAIN retry, guaranteed reap
- services/segmentation.py: Bengali/CJK/Arabic punctuation, ultra-short tier, stitch_adjacent_shorts,
  bounded-loop merge; public clean_up_segments API
- services/model_manager.py: robust lock.locked() handling
- api/routers/dub_core.py: job_id traversal guard, thread-safe _active_procs, timeouts on ffmpeg/demucs,
  POST /dub/cleanup-segments endpoint
- api/routers/dub_export.py: guarded SSE listener remove, ffmpeg timeouts via run_ffmpeg
- api/routers/exports.py: destination_path validation, safe source resolver, subprocess list-form
- api/routers/generation.py: contextlib.suppress on tempfile cleanup, db_conn usage, safe output-path helper
- api/routers/system.py: try/finally tmp cleanup, subprocess timeouts
- schemas/requests.py: TranslateSegment.id int->str to match hex segment IDs
- main.py: threading.Lock around crash log writes

Frontend:
- components/SearchableSelect.jsx: popover combobox with search, keyboard nav, popular+recent pins, 200-item cap
- App.jsx: wire SearchableSelect for dub language / ISO code / voice-gen language; Clean Up segments button;
  fix blob URL leak (object-shaped prev in setter, unmount cleanup via ref)
- components/WaveformTimeline.jsx: explicit <video> detach instead of innerHTML='' to release decoder
- index.css: ss-* combobox styles matching Gruvbox theme

Tests:
- tests/test_segmentation.py (26 cases), test_dub_transcribe.py, test_dub_export_unique.py, conftest.py

Chore:
- .gitignore: exclude omnivoice.zip, /research/ reference clones
- Remove tracked stray root test scripts + crash_log.txt

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 17:20:19 +05:30