upstream-archive/VoiceStudio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
154 Commits 24 Branches 39 Tags
fb521400a2e61eb4313e4151b5efe998f073eb5c
Commit Graph
43 Commits
Author SHA1 Message Date
debpalashandClaude Opus 4.7 fb521400a2 P0(dub): atomic WAV writes — closes #48 
Direct torchaudio.save(path, ...) writes bytes as the encoder produces
them. SIGKILL, OOM kill, or Tauri sidecar reap mid-write leaves the file
at `path` truncated. Downstream tools (ffmpeg in the dub mux, NLEs the
user imports the WAV into) happily read truncated RIFF — the header
appears first, then the data chunk gets cut short — and surface as
silently corrupt audio later in the pipeline. That's the shape of #48.

New helper services/audio_io.py:atomic_save_wav() writes to a sibling
temp file in the same directory then os.replace() into place. POSIX
rename(2) is atomic; os.replace() ports the same guarantee to Windows.
Either the target ends up with a complete WAV or it keeps its previous
contents (or never exists) — no third state.

Migrated three call sites in api/routers/dub_generate.py:
- L289: RVC per-segment write
- L328: deferred batch write of all segments
- L390: final mixed-track export

Left L508 alone — it writes to BytesIO (in-memory response body), no
atomicity needed.

Implementation note (recorded as a docstring in audio_io.py): the temp
file must end in `.wav`, not `.tmp`. torchaudio.save infers the output
format from the path suffix and ignores the `format=` kwarg with the
soundfile backend. A `.tmp` suffix raises "Unsupported format: tmp". The
leading dot + target-name prefix still marks the file as transient.

Tests in backend/tests/test_atomic_wav.py:
- success path: writes valid WAV, no temp leaks, overwrites cleanly
- atomicity: target unchanged when save raises (pre-existing target)
- atomicity: target absent when save raises (new target path)
- no temp leaks on failure
- temp file lives in target_dir (cross-fs renames are not atomic)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-19 09:24:50 +05:30
debpalashandClaude Opus 4.7 92f716e0d4 P0(security): loopback guard on /ws/transcribe before accept() 
The streaming-ASR WebSocket accepted any local connection without an
origin check. Any process running as the same user (rogue extension,
malware, sibling Electron app) could open ws://127.0.0.1:3900/ws/transcribe
and exfiltrate the user's live microphone audio in real time.

HTTP routers gate sensitive endpoints with Depends(require_loopback) at
the router level (see backend/api/dependencies.py). FastAPI's WebSocket
dependency injection differs across versions, so the guard is inlined in
ws_transcribe before websocket.accept() — non-loopback origins receive a
1008 (Policy Violation) close and never see the open socket.

Three source-level tests in backend/tests/test_capture_ws.py guard
against regression — same shape as tests/test_bind_host.py:
- references _LOOPBACK_HOSTS in the handler
- closes non-loopback with code 1008
- close() appears before accept() in the source

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-19 09:24:50 +05:30
Palash Debnath e4dbf4c8c0 P0: release.yml typecheck + bind audit + loopback middleware (#84) 
Three P0 fixes bundled — foundation cleanup before v0.3.0 phase work. Closes release.yml drift (PR #51's tabs broke v0.3.0 tag releases), production bind exposure (Critic F1), and 9-endpoint LAN gap on /system/* (Critic F2+F3). 5 new tests; 243 full pass.
 2026-05-18 22:00:59 +05:30
Palash Debnath 1941e3fcc0 fix(docker): GPU detection in containers + compose profiles + sonitranslate cuDNN sub-repo (#74) 
Docker GPU support hardening + documentation.

- Restores docker compose --profile gpu up path; documents NVIDIA Container Toolkit setup in README
- Splits CPU vs GPU compose services cleanly (deploy/docker-compose.yml)
- backend/api/routers/setup/wizard.py: GPU detection in containerized environments uses torch.cuda fallback
- New scripts/setup.py replaces deleted scripts/setup_cudnn.py
- New test: tests/test_setup_preflight.py
- CHANGELOG.md + README.md updated

Complementary to PR #77 (community PYTHONPATH fix) — different sections of docker-compose.yml.
 2026-05-18 16:31:01 +05:30
Palash Debnath 141546b8a7 fix: stabilize dub/diarization UI + production deployment + sonitranslate plumbing (#75) 
Production deployment hardening, dub OOM recovery, new SoniTranslate sidecar engine, ASR backend expansion.

- Dub generation OOM recovery: backend/api/routers/dub_generate.py:163-209 adds OOM detection + one retry with reduced nstep
- New SoniTranslate sidecar engine: backend/api/routers/sonitranslate.py + backend/services/sonitranslate.py (subprocess-based dubbing pipeline, opt-in)
- ASR backends expansion: backend/services/asr_backend.py adds NeMo Parakeet TDT, Moonshine, additional Whisper variants; new GET /system/asr-backends endpoint
- Dub UI polish: tighter spacing in DubSegmentRow.css, DubTab.css

Issue #78 (speaker diarization mis-assignment) NOT addressed by this PR — the bundled diarization changes are in the new SoniTranslate sidecar, not the existing pyannote pipeline. Keeping #78 open.

No DB schema changes, no migration. Backward-compatible for existing user data.
 2026-05-18 16:30:46 +05:30
Palash Debnath 6825b8b0a9 Cross-platform bug bash + Stories tab + VRAM-aware GPU pool (#51) 
First v0.3.x release on the Phase 0 cross-platform CI baseline.

## Cross-platform bug fixes (375ea4e)

User-reported bugs from a Pinokio/Windows session:
- Docker `compose --profile gpu up` no longer port-conflicts on 3900 — restored `profiles: ["cpu"]` that #49 wrongly reverted on CodeRabbit's advice.
- Argos / pip install from the UI now works inside Docker — added `_in_virtualenv()` runtime check; `run_pip` injects `--system` automatically when on system Python.
- Speaker diarization warning toast — when pyannote silently falls back to the silence-gap heuristic (missing HF_TOKEN, license not accepted, network blocked), `_diarize()` now returns `(segments, warning)`; `useDubWorkflow` renders an 8-second toast.

## Dub editor UX (d5df454)

Six fixes per annotated screenshots:
- Editable segment start times (`m:ss.s` or raw seconds; Esc reverts, Enter commits; rejects overlap with end).
- Click a transcript row → seek the waveform/video (`WaveformTimeline` now forwardRef's `seekTo(time)`).
- Speaker is datalist-backed (pulls from detected speaker clones; free text still allowed).
- Scissors menu splits at cursor — uses live caret, then last caret, then sentence-boundary fallback.
- Mouse-wheel scrolls the waveform; Cmd/Ctrl left alone for browser pinch-zoom.
- Menu popover collision: added `avoidCollisions` + `collisionPadding=8` to Radix Content; removed `position: fixed` from `.ui-menu`.

## VRAM-aware GPU pool (73dbe18)

`_gpu_pool` was hardcoded `ThreadPoolExecutor(max_workers=1)` since introduction — every TTS forward serialized through one thread.
- CUDA / ROCm: `workers = clamp(1, free_GB // 2.5, 4)`. 16 GB card with ~14 GB free → 4 workers → ~4× throughput on multi-segment dubs.
- MPS / CPU / unknown: 1 worker.
- `OMNIVOICE_GPU_WORKERS` env var override (clamped 1..16).
- Module `__getattr__` preserves the public `_gpu_pool` symbol for existing callers.

## Stories tab — wire-up + UX (f6bbc7a)

The 264-line `StoriesEditor` component existed but was mounted nowhere. Now wired into NavRail + lazy-loaded on `mode === 'stories'`. Added Paste & Split panel (sentence-boundary chunking) and per-track `[pause 0.5s]` insertion.

## Stories — pauses + inline voice (edd3a1d)

`frontend/src/utils/storyTokens.js` — tokenizer for `[pause X.Ys]` and `[voice:X]…[voice:default]` markers. Voice switches are stateful (carry forward). 13 new vitest cases (vitest now 24/24).

## Verified

- 214 backend tests pass (3 skipped, 10 xfailed, 3 xpassed)
- 23 router-smoke tests pass
- 24/24 vitest cases pass (13 new)
- All 7 Phase 0 CI checks green (Tauri shell + Smoke on macOS/Windows/Linux + Tests)

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-05-18 15:57:42 +05:30
Palash Debnath 21c338c821 security: add loopback origin check to /system/set-env (#81) 
Adds `request.client.host` allow-list check (`127.0.0.1` / `::1` / `localhost`) to `POST /system/set-env`. Non-loopback callers receive `403` instead of being able to mutate `os.environ` for HF_TOKEN / TRANSLATE_API_KEY.

Surfaced during security review of PR #66, which widens the pre-existing window by persisting these keys to disk via prefs.json. This fix closes the underlying vulnerability so PR #66's revision lands onto a clean base.

Defensive `request.client is None` branch handles ASGI middleware that strips client info. Three new tests cover non-loopback reject, loopback allow, and allow-list still validated on loopback.

Follow-up: `260518-ivy-deferred-items.md` enumerates 5 sibling POST routes in `system.py` that share the same gap — separate PR.
 2026-05-18 14:09:00 +05:30
Palash DebnathandClaude Opus 4.7 766e2f7284 Phase 0 — Gates: cross-platform CI matrix + regression fixture + release smoke (#71) 
* docs: initialize OmniVoice stabilization milestone project

* chore: add project config (yolo + balanced)

* docs: domain research for stabilization milestone

* docs: define v1 requirements for stabilization milestone

* docs: add GGUF + singing engine spike requirements (Phase 4 new)

* docs: roadmap revision + CLAUDE.md (7 phases, 62 reqs, +GGUF/SING spikes)

* docs(phase-0): add Gates phase RESEARCH.md

Phase 0 research synthesizes the cross-platform CI matrix, frozen
omnivoice_data fixture, installer post-build smoke, SHA-256 checksum
publishing, and PR-template extension into copy-paste-ready YAML and
Python snippets composed entirely from existing in-repo patterns.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(phase-0): add Gates phase CONTEXT, PATTERNS, and PLAN

Phase 0 — Gates is the hard pre-condition for v0.3.x stabilization.
Lays cross-platform CI matrix (macos-14/windows-2022/ubuntu-22.04),
regression fixture (≤200 KB), installer smoke on tag push, SHA-256
checksums in release body + per-OS SHA256SUMS-*.txt assets, PR
template with RC cadence + fixture line, and the open-PR landing
for #51.

Plan covers GATE-01..06; structured into 7 slices (A–G) with explicit
Slice C → Slice G dependency reordering so the new smoke-matrix lands
on main before PR #51 (CONTEXT.md L86 interleave decision).

Plan-checker iteration 2: APPROVED — all 3 BLOCKERs + 3 MAJORs from
iteration 1 resolved (file truncation/Slice-G missing, GATE-06 sibling
PR verification, Slice C ordering, Truth #5 wording, macOS Tauri
WebView avoidance per Pitfall #5, Windows taskkill per Pitfall #2).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(00-gates): seed regression fixture (GATE-01)

- scripts/seed-test-fixture.py — deterministic builder for tests/fixtures/omnivoice_data/
  - wipes + rebuilds; fixed created_at=1700000000.0; all-zero PCM for byte-deterministic diffs
  - calls backend.core.db.init_db() directly (alembic versions/ is empty — see CONTEXT.md)
  - checkpoints WAL → DELETE on close so no -shm/-wal sidecars pollute git status
  - exits non-zero if fixture > 200 KB
- tests/fixtures/omnivoice_data/{omnivoice.db, README.md} — 8-table empty DB + 1 voice_profiles row
- tests/fixtures/omnivoice_data/voices/test-voice/{profile.json, sample.wav} — 1-sec 24 kHz mono silence
- .gitignore — explicit allow-list (!tests/fixtures/omnivoice_data/**) so the existing
  omnivoice_data/, *.db, *.wav patterns don't hide the fixture from git

Verifies: du = 144 KB on disk; sqlite_master lists 8 init_db tables + sqlite_sequence;
voice_profiles has exactly 1 row id='test-voice'; 0 rows in generation_history.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(00-gates): add tests/smoke/test_boot_smoke.py (GATE-01)

- tests/smoke/__init__.py — package marker so pytest treats tests/smoke/ as a module
- tests/smoke/test_boot_smoke.py — 4 in-process FastAPI TestClient smoke tests:
    * test_health_returns_ok — /health returns 200 + {status:ok, device:...}
    * test_profiles_endpoint_lists_fixture_voice — /profiles surfaces the seeded
      test-voice row (validates OMNIVOICE_DATA_DIR wiring → DB_PATH → init_db schema)
    * test_system_info_includes_data_dir — /system/info resolves data_dir
    * test_history_endpoint_empty — /history reaches DB and returns []
  Test isolation env vars (OMNIVOICE_MODEL=test, OMNIVOICE_DISABLE_FILE_LOG=1)
  set at module top BEFORE any backend import — pattern from tests/test_router_smoke.py.
  Fixture is copied to a per-session temp dir so the test never mutates the
  checked-in artifact (SQLite file-change counter + runtime subdirs like dub_jobs/
  would otherwise dirty `git status` after every run).
  Failure mode: if tests/fixtures/omnivoice_data/ is missing, pytest.fail at
  import time with the regenerate command.
- .gitignore — tighten the GATE-01 allow-list to ONLY the seed-produced files
  (README.md, omnivoice.db, voices/test-voice/profile.json, sample.wav).
  Prevents future runtime subdirs the backend may create under the fixture
  from being accidentally committed.

Verifies: `uv run pytest tests/smoke/ -q --tb=short` → 4 passed in 1.31 s
(target was < 30 s). `git status` clean after a test run.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(triage): record post-planning GitHub state — PR #62, new issues, OOS deferrals

- GATE-06: mark #53 + #61 merged (2026-05-16); add #62 (Wave 1 quick wins) to gate set
- INST-01: note PR #62 implements setuptools pin (closes #58)
- INST-04: note PR #62 lands README docs for #56 workaround
- INST-12: new requirement for #65 Windows Triton/torch.compile OOM (filed post-planning)
- Out of Scope: defer #67/PR #68 (audio effects), #64 (custom model dir),
  PR #66 zh-CN (i18n milestone), #63 (empty-template bug)

PR #62 is the user's own Wave 1 work landed as a separate PR while
GSD planning ran in parallel. Merging it eliminates duplicate work
in Phase 1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci(00-gates): add cross-platform smoke matrix (GATE-02)

- New smoke-matrix job on macos-14, windows-2022, ubuntu-22.04
- needs: test, fail-fast: false, timeout-minutes: 10
- Pinned actions: checkout@v4, setup-python@v5, setup-uv@v3 (cache enabled)
- Per-OS ffmpeg + libsndfile install (brew/choco/apt via awalsh128 cache)
- UV_HTTP_TIMEOUT=120, UV_HTTP_RETRIES=5 for restricted-network resilience
- Narrow scope: uv run pytest tests/smoke/ -q --tb=short
- Existing `test` and `tauri-cross-platform` jobs untouched

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: add workflow_dispatch to ci.yml so smoke-matrix can run on feature branches

* feat(00-gates): add --health-check CLI flag to backend entrypoint (GATE-03)

- argparse on __main__ block; --health-check boots uvicorn in a daemon
  thread and polls http://127.0.0.1:3900/health every 5s for up to 60s.
- Prints 'OK — /health responded 200 after Ns' and exits 0 on first 200.
- Prints 'FAIL — /health did not respond 200 within 60s' to stderr and
  exits 1 on timeout. Default invocation behavior unchanged.
- No new deps (stdlib argparse/threading/time/urllib.request/sys + uvicorn).
- Consumed by per-OS installer-smoke step in .github/workflows/release.yml.

Verified locally: exits 0 in 5s against tests/fixtures/omnivoice_data/.

* ci(00-gates): add per-OS installer smoke to release.yml (GATE-03)

Adds three matrix-leg-specific steps after 'Build + release (Tauri)',
each gated by runner.os with timeout-minutes: 5:

- macOS (macos-14): hdiutil attach DMG → locate bundled Python backend
  inside *.app/Contents (NOT the Tauri WebView shell — RESEARCH Pitfall
  #5: WebView hangs on headless runners) → invoke --health-check →
  hdiutil detach. Falls back to *.app/Contents/Resources and hard-fails
  with a directory listing if no backend binary found.

- Windows (windows-2022): msiexec /quiet install → find backend.exe
  under 'C:/Program Files/OmniVoice Studio' → invoke --health-check in
  background, wait, then taskkill //F //T //PID to cleanup orphaned
  PyInstaller child processes on port 3900 (RESEARCH Pitfall #2).

- Linux (ubuntu-22.04): --appimage-extract (no FUSE on GH runners),
  locate binary or AppRun, run under xvfb-run -a.

Bundle-only regressions (PyInstaller missing-module, Tauri sidecar
path mismatch) are invisible to ci.yml's in-process smoke matrix —
this step closes that gap before any release is published.

Verified: YAML parses; all three steps present; gating + timeout
correct; Pitfall #2/#5 mitigations preserved.

* ci(00-gates): publish SHA-256 checksums in release body + as asset (GATE-05)

- Add 'Compute SHA-256 checksums' step writing SHA256SUMS-<label>.txt
  per matrix leg using native shasum/sha256sum (Git Bash on Windows).
- Add 'Append checksums to release + attach SHA256SUMS file' step using
  softprops/action-gh-release@v2 with append_body: true so the hashes
  land in the release body alongside tauri-action's content (not
  replacing it) and the file is uploaded as a release asset for
  'shasum -c SHA256SUMS-<label>.txt' verification.
- Both steps gated by 'github.event_name == push && refs/tags/v*' so
  workflow_dispatch dry-runs do not attempt to attach to a non-existent
  release (per CONTEXT.md L70 + RESEARCH Pitfall #7 deferral of any
  aggregate cross-leg SHA256SUMS job).
- fail_on_unmatched_files: true to surface path-resolution errors loudly.

* docs(00-gates): document RC cadence + regression-fixture check in PR template (GATE-04)

* docs(setup): add HF token persistence guide for macOS/Windows/Linux (DOCS-05)

Covers two persistent paths:
- Method A — canonical ~/.cache/huggingface/token via huggingface-cli login
- Method B — shell env var (~/.zshrc / ~/.bashrc / Windows User scope)

Documents the v0.2.7 "session only" in-app behavior + notes that
Phase 1 AUTH-03 will make in-app pastes write to the canonical file.

Bundled with Phase 0 PR per user request. Strictly DOCS-05 scope —
zero code changes, no engine touches.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* spec(auth): redesign HF token resolution as 3-source cascade with fallback (AUTH-01..06)

Replaces the env_store.py file-based design with a SQLite-backed app
store + cascade resolver that checks app → env var → ~/.cache/huggingface/token
in priority order, with automatic fallback to next source on HTTP 401.

User-explicit design decision:
- App-stored token (SQLite settings table, AES-GCM encrypted) wins
- Env var ($HF_TOKEN) second
- Global huggingface-cli login file third
- All three sources visible in Settings → API Keys with "Active" badge
- Save action populates BOTH app store AND canonical HF file (defense in depth)

New requirement:
- AUTH-06 — on 401, auto-retry next source in cascade before erroring

Also: traceability count corrected (62 → 74 — undercount at planning +
INST-12 + AUTH-06 added post-planning). All 74 v1 reqs mapped.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(auth): backend recognizes HF token from canonical file, not just env var

Two call sites were only checking $HF_TOKEN env var, missing the canonical
~/.cache/huggingface/token file written by `huggingface-cli login` (or the
app's future Save action):

- system.py `/system/info` `has_hf_token` flag — UI showed "No HF token"
  even when `huggingface-cli login` had populated the file.
- model_manager.get_diarization_pipeline — pyannote diarization silently
  returned None when only the canonical file was set. This is the bug
  behind issue #35 (speaker diarization setup failure).

Both fixes use the same pattern: env var > huggingface_hub.get_token()
(which reads the canonical file). Adds a local _has_hf_token() helper
to system.py with a comment marking it as prelude to the AUTH-01..06
cascade (Phase 1 token_resolver.py will layer SQLite app-store on top).

Closes #35 sub-issue (canonical token invisible to diarization).
Cross-cuts AUTH-02 + AUTH-06 design for Phase 1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(dictation): make pill-widget mode reachable from GUI + scripts (INST-13)

The dictation widget infrastructure shipped in PR #40 but was only reachable
via the undocumented --pill CLI flag. Adds three discovery paths:

1. Tray menu: "Switch to Dictation Widget" (studio mode) — saves
   launch_as_widget=true to config, relaunches with --pill, exits current.
   Mirrors the existing "Open Studio" path in pill-mode tray.

2. Persistent config: AppConfig.launch_as_widget (bool, default false). Read
   at startup via load_config_pre_app() (uses dirs-next, no AppHandle
   required). CLI --pill still takes precedence when explicitly passed.

3. Tauri commands: get_launch_as_widget / set_launch_as_widget for the
   Phase 2 Settings UI to bind a checkbox to.

4. Scripts: bun desktop-prod:pill / desktop-prod:run:pill — forward --pill
   to the bundled app launch. macOS uses `open -n --args` to spawn fresh
   instance with the flag.

Closes the GUI half of INST-13. Phase 2 closes the Settings UI half.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(dictation): show widget unconditionally on pill-mode launch + visible Suspense fallback

Before: pill mode set up correctly but the widget window stayed hidden
until ⌘⇧Space was pressed. New users saw absolutely nothing on launch
(no main window, no dock icon, hidden widget) and assumed the app
failed. If global-shortcut Accessibility permission wasn't granted,
they had no path to discover the widget at all.

Two changes:

1. lib.rs: in pill_mode_setup, explicitly show + position + focus the
   widget window after hiding main. With per-call error logging so we
   can diagnose failures (and a clear error log if widget window
   wasn't created at all — points at tauri.conf.json regression).

2. main-app.jsx: Suspense fallback was `null`, which combined with
   widget's transparent+decorations:false config made any lazy-import
   delay or failure invisible. Now renders a dark pill saying
   "Loading dictation…" so even if CaptureWidget lazy-import stalls,
   the user sees the window exists.

Studio mode behavior unchanged — widget stays hidden until hotkey
or tray click triggers it (existing show() call in the shortcut/
menu handlers is preserved).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(dictation): create widget window programmatically; Tauri 2 silently dropped config-array creation

Root cause: declaring the widget window in tauri.conf.json's app.windows[]
silently failed in Tauri 2 — get_webview_window("widget") returned None
even though the config was syntactically valid. Probable culprit was the
transparent + decorations:false + visible:false combo, but Tauri offered
no error message either at startup or via webview_windows() enumeration.

Diagnosed by adding webview_windows() enumeration logging at setup start
(only ["main"] ever appeared) and a programmatic WebviewWindowBuilder
fallback that surfaces real Result errors.

Fix:
- tauri.conf.json: widget entry now has `create: false` to make the
  config-vs-programmatic handoff explicit.
- lib.rs setup(): call WebviewWindowBuilder::new(app, "widget", ...).build()
  with the exact same surface attributes the config used to declare.
- capabilities/default.json: include "widget" in windows array so the new
  window inherits the same Tauri permissions as main.
- tauri.conf.json: remove the invalid `"url": "/?window=widget"` field —
  WebviewUrl::App takes a path only, query strings aren't supported.
  Both windows now load index.html.
- main-app.jsx: replace URL-query-based widget detection with
  getCurrentWindow().label === 'widget' via @tauri-apps/api/window. This
  is the Tauri 2-recommended pattern for multi-window apps and works
  regardless of URL routing.

Closes the immediate UX bug behind the dictation widget being invisible.
Builds cleanly + manually verified: pill widget visible on screen at
top-center after `bun desktop-prod:pill`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-17 12:34:32 +05:30
Adam Łukawski ba4cf6a35f fix: lazy-load ASR during desktop startup 
* fix: lazy-load ASR during desktop startup

* fix: handle ASR preload edge cases

* fix: unload ASR backend on failure
 2026-05-16 12:39:11 +05:30
Palash DebnathandClaude Opus 4.7 e46b4e3d47 feat: import .srt subtitles to bypass Whisper (closes #52) 
Closes #52. Users who already have correct, pre-synced subtitles can now
skip ASR entirely — they upload a video as normal and then hit "Import
.srt" instead of "Upload & Transcribe". The .srt cues populate the dub
segment list directly, so the rest of the pipeline (translate, dub,
export) just works.

Backend
- services/srt_parser.py: lenient SubRip parser. Tolerates BOM, CRLF,
  missing index numbers, dot-vs-comma ms separator, and overlap (shifts
  the later cue's start to the earlier's end rather than dropping). Skips
  cues with non-positive duration or empty bodies; reports counts so the
  UI can warn.
- dub_core.py: new POST /dub/import-srt/{job_id} accepts the .srt file,
  parses it, clamps cues that run past the source media's duration, and
  replaces job["segments"]. Tries UTF-8 with BOM first, falls back to
  latin-1 for legacy Windows subs.

Frontend
- api/dub.ts: dubImportSrt helper with a typed response.
- hooks/useDubWorkflow.js: handleDubImportSrt — sets segments, flips
  dubStep to 'editing', shows a toast with per-bucket counts (imported /
  skipped / overlap-shifted / clamped) so the user sees what happened.
- pages/DubTab.jsx: "Import .srt" button next to "Upload & Transcribe"
  once a job exists, plus a smaller "Import .srt instead" affordance in
  the transcription-failure banner — the exact recovery path the
  reporter asked for.

Tests
- tests/test_srt_parser.py: 12 cases covering well-formed input,
  multi-line cues, dot-as-separator, BOM, CRLF, malformed cues, empty
  bodies, overlap shift, overlap-becomes-zero-drop, missing indices,
  empty input, and segment shape (sequential ids, speaker filler).
  pytest is now 226 passed (was 214); vitest unchanged at 11.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-16 12:39:01 +05:30
Palash DebnathandClaude Opus 4.7 9082705676 Post-refactor cleanup: wire fingerprints, drop dead code, scope pytest (#50) 
* chore: post-refactor cleanup — wire fingerprints, drop dead code, scope pytest

Follow-up to PR #49. Fixes residual issues from the App.jsx hooks split
and tightens repo hygiene so a bare `pytest` doesn't foot-gun.

Real bug
- frontend/src/hooks/useDubWorkflow.js: setLastGenFingerprints lives in
  useSegmentEditing, not on the store. The previous code called
  useAppStore.getState().setLastGenFingerprints?.(...) — the optional
  chain swallowed the missing method, so the "N segments changed" badge
  never updated after a fresh generate until a project save+reopen.
  Thread setLastGenFingerprints in from App.jsx; useSegmentEditing()
  now runs before useDubWorkflow() to make the setter available.

Dead code from the refactor
- frontend/src/App.jsx: drop unused `showAllProjects` useState and
  `pushUndo` from the useSegmentEditing destructure.
- frontend/src/hooks/useDubWorkflow.js: drop 5 unused selectors
  (preserveBg, defaultTrack, exportTracks, dualSubs, burnSubs) — the
  dub-download logic that needs these lives in App.jsx, not the hook.

Repo hygiene
- backend/api/routers/setup.py.bak: delete 38 KB tracked-in-git backup.
  The setup/ subpackage replacement has been in place for a while.
- pyproject.toml: add [tool.pytest.ini_options] with testpaths +
  norecursedirs. Previously a bare `pytest` would INTERNALERROR walking
  into research/ (1.2 GB of vendored upstream projects with their own
  test_*.py files that call sys.exit at module level).
- .github/workflows/ci.yml: run backend/tests/ as a second pytest
  invocation. The 23 tests there stub core.config in sys.modules to
  avoid the heavy main app import chain — that pollutes import state
  for other tests, so they need their own session. Previously these
  tests existed in the repo but never ran on CI.

Net effect on lint: 60 → 52 problems (-8) from dead-code removal.
Test counts unchanged: pytest 214 + 23, vitest 11.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: log silent catch failures that mask real bugs

CodeRabbit nitpick on #50: empty catch on the incremental-plan fallback
swallows errors. Extending the fix to the catches in this area that have
the same problem (a real failure would be invisible) while leaving the
genuinely non-actionable cleanup catches alone (EventSource.close(),
localStorage.setItem, fire-and-forget UI promises).

Logged:
- useDubWorkflow.js:97  — transcribe SSE message handler
- useDubWorkflow.js:347 — incremental-plan fallback (the CR finding)
- useDubWorkflow.js:352 — dub generate SSE event dispatch
- App.jsx:552         — exportRecord on Tauri save path
- App.jsx:580         — exportRecord on browser download path

Left silent (cleanup / non-actionable):
- useDubWorkflow.js:68, 112 — evt.close() in SSE teardown
- useDubWorkflow.js:102    — SSE error-event payload parse fallback
- App.jsx:124              — localStorage.setItem (quota / privacy mode)
- App.jsx:789, 901         — fire-and-forget UI promise tails

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-12 22:38:40 +05:30
Palash DebnathandClaude Opus 4.7 a1ef66c321 Stability pass: DB leaks, App.jsx hooks refactor, desktop bootstrap (#49) 
* fix: eliminate DB connection leaks, race conditions, and deprecated asyncio API

## DB Connection Leaks (P0)
- Convert 38 raw get_db() calls to db_conn() context manager across 14 router files
- Connections are now guaranteed to close even when exceptions are raised
- profiles.py create_profile: clean up orphaned audio file if DB insert fails
- profiles.py lock_profile: consolidate 3 separate conn.close() error paths

## Race Condition (P1)
- Add _dub_jobs_lock (threading.Lock) to protect _dub_jobs dict in dub_pipeline.py
- get_job/put_job now thread-safe for concurrent dub sessions

## asyncio Deprecation (P2)
- Replace 23 asyncio.get_event_loop() calls with asyncio.get_running_loop()
- Prevents DeprecationWarning on Python 3.12+ and future breakage on 3.14

## Quick Fixes
- gallery.py preview_voice: remove filesystem path from error response (P2)
- dub_pipeline.py parse_vtt_segments: remove redundant `import re` inside loop (P3)
- gallery.py _init_gallery_db: use db_conn() context manager (P2)

* refactor: extract hooks, centralize isTauri, add pytest-cov

## Frontend
- Extract useTTS hook (150 LOC) — TTS generation, streaming, audio ingestion
- Extract useProfiles hook (219 LOC) — voice profile CRUD, lock/unlock, preview
- Centralize isTauri detection: dialog.js, VoiceGallery.jsx, Settings.jsx
  now import from utils/media.js instead of 4 different detection patterns

## Backend
- Add pytest-cov to dev dependencies
- Baseline coverage: 39% across backend/ (214 tests pass)
- Add .coverage to .gitignore

* feat: add Vitest + checkJs, extract useDubWorkflow + useAppData hooks

## Frontend Testing (new)
- Set up Vitest with jsdom environment + @testing-library/react
- 11 tests: utils (isTauri, formatTime, constants) + Zustand store (mode, text, dubStep, pill)
- Scripts: 'test' (vitest run), 'test:watch' (vitest), 'test:legacy' (node runner)

## App.jsx Decomposition (continued)
- Extract useDubWorkflow hook (387 LOC) — upload, ingest, transcribe SSE,
  translate, generate SSE, abort, stop, cleanup
- Extract useAppData hook (181 LOC) — data loading, localStorage persistence,
  WebSocket real-time updates, model-status pill management

## TypeScript checkJs
- Enable checkJs: true in tsconfig.json for IDE-level type checking
- 947 existing errors (informational, not blocking builds)
- noImplicitAny remains false to avoid blocking

* ci: add Vitest step, fix useProfiles duplicate state

## CI
- Add 'Run Vitest (frontend)' step — runs 11 unit tests
- Override --checkJs false in CI typecheck to avoid 947 pre-existing errors
- Rename legacy test step for clarity

## Hooks
- Fix useProfiles to accept loadProfiles from parent (useAppData)
  instead of managing its own duplicate profiles array

* refactor: wire hooks into App.jsx — 2067 → 1129 LOC (-45%)

App.jsx now delegates to extracted hooks instead of inline logic:
- useAppData: data loading, localStorage, WebSocket, model pill
- useProfiles: voice profile CRUD, lock/unlock, preview
- useTTS: generation, streaming, audio ingestion
- useDubWorkflow: upload, transcribe SSE, translate, generate SSE

988 lines removed. All handler logic lives in focused,
independently testable hooks. Store selectors and render
JSX stay in App.jsx as the shell.

Verified: vite build clean, 11 frontend + 214 backend tests pass.

* feat: show real-time percentage on model loading pill

Backend: register hf_progress listener during _load_model_sync()
so download/weight-loading tqdm events update _loading_detail with
a progress percentage (0-99%). get_model_status() now includes a
'progress' field that the frontend polls.

Frontend: useAppData reads msQuery.data.progress and calls
setPillProgress() — the FloatingPill already renders the percentage
text and progress bar width from this value.

* fix: prevent FileNotFoundError in desktop bundle during model init

transformers >=4.52 calls _can_set_experts_implementation() and
_can_set_attn_implementation() during PreTrainedModel.__init__,
which open the class source file via open(class_file). In a Tauri
desktop bundle, module.__file__ points to a path that doesn't
exist on disk, causing:

  FileNotFoundError: .../omnivoice/models/omnivoice.py

Override both classmethods on OmniVoice to return static values
without filesystem access. OmniVoice doesn't use MoE experts
(return False), but does support flex/flash attn (return True).

* fix: sync source dirs on every bootstrap, not just first run

The Tauri bootstrap previously only copied omnivoice/ and backend/
to Application Support on the first run. Subsequent app updates
kept using stale source files, preventing bug fixes from landing.

Now ensure_venv_ready() always syncs both directories from the
bundle resources before returning, even when the venv is healthy.
This fixes the FileNotFoundError crash where the old omnivoice.py
lacked the _can_set_experts_implementation override.

* ui: premium setup wizard polish

- Primary button: solid gradient fill with hover glow + lift + press
- Stepper nav: connected pills with glow ring on active step
- Welcome cards: glassmorphism with stagger-in animations, lucide icons,
  left-border accent strip, hover translate
- Preflight panel: colored icon pill backgrounds, stagger-slide entrance
- Step transitions: fade+slide animation via keyed wrapper
- Footnote: shortened paths (~/ notation), Reveal in Finder button
- Recommendation banner: gradient background with accent glow
- Compact spacing throughout for denser, professional layout

* fix: kill zombie backend on clean+retry bootstrap

When clean_and_retry_bootstrap removes the project dir, any old
uvicorn process still running from the deleted paths remains alive
on port 3900. The subsequent retry_bootstrap sees the port is
healthy and attaches to the zombie instead of re-bootstrapping.

Now explicitly kill any process on the backend port after cleaning,
before calling retry_bootstrap.

* feat: integrate speaker clones into dubbing interface, sanitize system environment variables for subprocesses, and improve FFMPEG binary path resolution.

* fix: restore docker compose default + drop dead setSeed call

- deploy/docker-compose.yml: remove profiles: ["cpu"] from the default
  service so `docker compose up` matches the comment on line 5. With the
  profile present, no service auto-started.

- frontend/src/App.jsx: drop the setSeed call in restoreHistory. The
  selector was never reintroduced after the App.jsx hooks split, and
  there is no seed state in the store — seeds are generated fresh per
  call in useTTS and only read from history items for display.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: address CodeRabbit review — async detection, dub stream, bootstrap fail-fast

- backend/services/tts_backend.py: invert async-context detection in
  _ensure_loaded. The previous code unconditionally caught its own
  diagnostic RuntimeError and then called asyncio.run() inside a
  running loop, masking the intended error message.

- frontend/src/hooks/useDubWorkflow.js: require a terminal `done` event
  before reporting dub success. Without this, a dropped stream after
  partial progress would flip the UI to `done`, refresh history, and
  play the completion ping as if generation finished.

- frontend/src/hooks/useDubWorkflow.js: restore the previous step when
  tasksCancel() fails. The UI was getting stuck in `stopping` forever
  on cancel errors.

- frontend/src-tauri/src/bootstrap.rs: fail-fast when source sync fails
  after the existing directory has already been removed. The previous
  warn-and-continue path could leave the install with no backend/ or
  omnivoice/ sources and defer the failure to backend startup with a
  cryptic error.

- backend/api/routers/generation.py: add `from e` to the ValueError →
  HTTPException re-raise (Ruff B904).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: preserve % suffix in TTS generation timer

The 100ms timer in useTTS was rewriting generationTime to a plain
elapsed-seconds string, which immediately wiped the "(xx%)" download
suffix written on the next iteration of the response-body loop. The
real-time percentage was flickering on/off as a result.

Read the previous value inside the setter and reattach any existing
percent suffix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-12 21:49:04 +05:30
Palash Debnath 545b39c912 feat: Scalar API docs, community health files, Quickstart cards (#41) 
* feat: Scalar API docs, community health files, Quickstart cards, GHCR Docker

Backend:
- Replace Swagger UI with Scalar at /docs (scalar-fastapi)
- Add OpenAI-compatible /v1/audio endpoints (openai_compat router)
- Add TTS streaming endpoint (tts_stream router)
- Add voice marketplace router (marketplace)
- Update TTS backend registry

Frontend:
- Refine CaptureWidget, WaveformTimeline, App layout
- CSS polish and index.css updates

Community health:
- SECURITY.md — vulnerability reporting policy
- CODE_OF_CONDUCT.md — Contributor Covenant v2.1
- .github/FUNDING.yml — GitHub Sponsors
- .github/ISSUE_TEMPLATE/ — bug report + feature request
- .github/pull_request_template.md — PR checklist

README:
- Quickstart redesigned as 3-column progressive cards
- Docker section updated with GHCR pull instructions
- API Docs row added to service table

Infra:
- scalar-fastapi added to pyproject.toml + uv.lock
- research/ added to .gitignore

* refactor: clean up documentation and logging while enhancing desktop packaging dependencies and capture UI performance.

* fix: address CodeRabbit review — streaming, escaping, thresholds

Backend:
- marketplace: stream zip entries via ZipFile.open()/copyfileobj, add 100MB
  upload cap, fix raise-from exception chaining (OOM prevention)
- openai_compat: _encode_audio returns actual file ext so Content-Disposition
  matches real format; forward non-profile voices when DB row not found
- tts_stream: send 'start' frame after generation so sample_rate is real;
  forward non-profile voices on DB miss
- capture_ws: split MIN_BUFFER_BYTES into separate partial/final thresholds
  so short utterances (<2s) still get transcribed

Frontend (Tauri):
- lib.rs: tray 'dictate' now toggles start/stop based on widget visibility
- commands.rs: XML-escape exe path in LaunchAgent plist, shell-quote in
  .desktop Exec line to prevent injection from special-char paths
- CaptureWidget.css: fix Stylelint violations (empty lines, font-family quotes)
 2026-05-04 10:57:37 +05:30
Palash Debnath 0c1a3829d5 Fix transcription stream drops, IndexError, BrokenPipeError, FK constraint, and Tauri CSP 
Fix transcription stream drops and Tauri CSP
 2026-04-30 19:20:10 +05:30
riyaaa-04 81c4b7d1ed Fix transcription stream drops, IndexError, and Tauri CSP 2026-04-30 18:49:42 +05:30
debpalash 41c23f6b3a feat: enhance ASR performance and reliability with binary bundling, model warmup, sub-stage progress tracking, and optimized polling. 2026-04-30 07:46:35 +05:30
debpalashandClaude Opus 4.7 79d4f3b53d feat(0.2.6): tray-aware shell, hotkey customization, WS dictation dedupe 
Tray + lifecycle:
- tauri-plugin-single-instance — second launch focuses existing window
  instead of racing for port 3900.
- Window close hides instead of destroying; backend shutdown moved to
  RunEvent::ExitRequested so only the tray "Quit" item (or Cmd+Q on macOS)
  actually exits.
- Tray icon flips to red-dot variant during dictation recording.

Hotkey customization:
- Settings → Capture tab. Records any modifier+key combo, persists to
  app config, re-registers on launch.
- set_dictation_shortcut rolls back to the previous binding on register
  failure so a bad combo never leaves the user with no shortcut.

Dictation latency / correctness:
- WS-final treated as source of truth; HTTP POST /transcribe runs only as
  fallback (WS error / timeout / no-WS path). Audio transcribed once
  instead of twice. Server accepts an "EOF" text frame (or empty binary
  frame) so the socket stays open for `final` to be delivered before the
  client closes.
- MediaRecorder chunks queued during the WS handshake are drained in
  ws.onopen — the server's final transcript no longer drops the first
  ~250 ms of audio.
- Fallback timeout scales with recording length (max(15s, recordedMs+10s))
  so long-form dictations don't trip duplicate transcription.

Donate page:
- Drop Patreon, Bitcoin / Ethereum / Solana cards. Drop qrcode.react.
- Move "Commercial License" CTA from page bottom to top-right header bar.

Docker hygiene:
- docker-compose binds 127.0.0.1 by default. README documents the LAN
  exposure trade-off + recommends a reverse proxy with auth.

CI:
- New cross-platform `tauri-cross-platform` job runs `cargo check` against
  the Tauri shell on macOS / Windows / Linux per PR. Catches platform
  cfg-gate regressions without paying the full ~15min/platform bundle
  cost (full bundling stays in release.yml on tag push).

Tests:
- tests/test_capture_ws.py (3 cases) covers EOF text-frame, empty-binary
  EOF, and legacy disconnect-finalize paths.

Includes the user's previously-staged 0.2.5 polish: cross-platform
desktop-prod.sh, Dockerfile base-image fix, bun.lock churn.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-29 10:33:22 +05:30
debpalash 3a8c1dff76 fix: resolving heartbeat, fmtBytes(0), smarter build error handling 
- Backend emits 'resolving' heartbeat every 2s during HF metadata
  resolution so UI shows 'Resolving repo metadata...' instead of
  being stuck on 'Connecting to HuggingFace…' indefinitely
- fmtBytes(0) now returns '0 B' instead of '—'
- desktop-prod.sh only tolerates signing errors, surfaces real
  build failures with exit code
- Handle install_retry phase in frontend with attempt number
 2026-04-29 01:28:48 +05:30
debpalash 835280dc3e fix: disk space check walks up to existing parent when cache dir wiped 
shutil.disk_usage() throws on non-existent paths, causing the
preflight to report 0.0 GB free after a fresh wipe. Now resolves
up to the nearest existing ancestor directory so it probes the
actual volume free space correctly.
 2026-04-29 00:06:14 +05:30
Palash Debnath 8d11e19494 feat: dictation maturity + batch TTS pipeline + tests (#32) 
Global Hotkey:
- Register ⌘+⇧+Space system-wide via tauri-plugin-global-shortcut
- Shows/focuses window and emits tray-dictate event from any app

Auto-Paste:
- enigo crate simulates ⌘V/Ctrl+V after transcription
- Text auto-pastes into whatever app was active before dictation

Streaming ASR:
- WebSocket endpoint /ws/transcribe for live partial transcription
- 2s buffer interval, configurable via OMNIVOICE_STREAM_INTERVAL
- CaptureButton streams audio chunks, shows italic partial text
- Falls back to HTTP POST if WebSocket unavailable

Batch TTS Pipeline:
- Replace stub worker with full pipeline:
  extract → transcribe → translate → generate → mix → export
- Per-job progress tracking (stage, percent, current_lang, segment)
- GoogleTranslator integration via deep_translator
- Download endpoint GET /batch/download/{id}/{lang}
- BatchQueue UI rewritten: progress bars, cancel/delete, downloads
- Type-safe API client (api/batch.ts)

Tests:
- 23 tests for batch endpoints + streaming ASR helpers
- Lightweight fixtures that stub GPU deps

UX (earlier sessions):
- Dual-mode ASR (Turbo MLX + WhisperX Accurate)
- Enhanced download progress (speed, ETA, bytes)
- Status bar black flash fix
- Cold-start model preloading
- Full accessibility audit (ARIA, focus-visible)
- Compact UI layout improvements
- README updated with new features
 2026-04-28 18:54:26 +05:30
debpalash a7b7e1f897 feat: flush dropdown, credentials tab, whisper model selector, reactive transcriptions 
Flush Dropdown:
- Flush button now opens a dropdown showing all loaded models
- Each model shows device, VRAM usage, and individual Unload button
- Backend endpoints: GET /model/loaded, POST /model/unload/{id}
- Bottom actions: Flush caches, Unload all + flush

Credentials Tab:
- New Settings > Credentials tab with HF_TOKEN and TRANSLATE_API_KEY
- Session-scoped via POST /system/set-env (no ElevenLabs — we ARE the alternative)
- Shows 'Set' / 'Not set' badge for HF token

Notification Panel:
- Moved from header dropdown to footer status bar (4th tab: Notifications)
- Bell icon in header dispatches event to open footer tab
- Click notification → navigates to relevant page (e.g., Settings for HF token)
- No inline inputs — notifications are purely informational + navigational

Whisper Model Selector:
- Capture widget now has quality preset picker: tiny → large-v3
- Persisted in localStorage; sent to backend as 'model' form field
- Backend passes chosen model to ASR backend

Reactive Transcriptions:
- Custom window event (omni:transcription-added) bridges CaptureButton → TranscriptionsPage
- Page updates in realtime when new dictation completes
 2026-04-28 12:33:10 +05:30
debpalash 22de8c43fe feat: notification panel, HF token setter, transcriptions page 
Notification Panel:
- Bell icon in header with badge count (red/amber by severity)
- Polls GET /system/notifications every 30s
- Surfaces: missing HF_TOKEN, missing ffmpeg, low disk, CPU-only mode
- Inline HF_TOKEN input — set token without leaving the app
- Dismiss individual or all notifications (persisted in localStorage)
- Click-outside to close, slide-in animation

Backend:
- GET /system/notifications — returns actionable notifications
- POST /system/set-env — safely set HF_TOKEN, TRANSLATE_API_KEY,
  ELEVENLABS_API_KEY at runtime (allowlisted keys only)

Transcriptions Page:
- New nav rail item (Transcripts) with FileText icon
- Searchable list + detail split-pane layout
- Stores all dictation results in localStorage (max 200)
- Copy, delete, export all as .txt
- Shows timestamps, language, duration, and segment breakdown
- CaptureButton auto-saves to Transcriptions on success

Header:
- Added gallery + transcriptions to VIEW_META breadcrumbs
 2026-04-28 12:18:44 +05:30
debpalash 5e5ac69f22 fix: capture transcribe() — remove unsupported language kwarg 
WhisperXBackend.transcribe() signature is (audio_path, *, word_timestamps)
with no language parameter. Language is auto-detected by Whisper.
 2026-04-28 12:08:52 +05:30
debpalash 2cd1ab4fb9 feat: batched TTS, cold start, audiobook editor, context-aware pipeline 
Batched TTS:
- Profile-grouped segment processing for cache locality
- CPU/GPU pipelining (ref audio load overlaps TTS inference)
- ~25-40% throughput improvement over sequential loop
- SegmentSpec container + generate_segments_batched() async API

Cold Start Optimization:
- Deferred torch + OmniVoice imports in model_manager.py
- Server starts in ~0.03s (was ~4s) — health/status respond immediately
- _lazy_torch() / _lazy_omnivoice() wrappers with singleton caching
- All downstream refs updated (idle_worker, free_vram, offload, restore)

Stories / Audiobook Editor:
- StoriesEditor component — multi-track with per-character voice assignment
- 7 character slots (Narrator + 6 characters) with color-coded dots
- Inline TTS preview per line via /dub/preview-segment endpoint
- Add/remove/reorder tracks, Generate All workflow
- Character stats footer (lines, characters, est. duration)

Context-Aware Pipeline:
- Video frame extraction via ffmpeg at segment midpoints
- Frame analysis: brightness, mood, complexity via PIL image stats
- Per-segment and global context (VideoContext container)
- get_segment_context() → natural-language TTS instruct hints
  e.g. 'Speak with vibrant energy, dark atmosphere, fast-paced scene'
- POST /tools/video-context/{job_id} API endpoint

Roadmap: ALL items completed ✅
 2026-04-28 12:00:02 +05:30
debpalash b054249be2 feat: plugin SDK, GPU sandbox, waveform v2, accessibility 
Plugin SDK:
- Abstract TTSPlugin base class with register/discover pattern
- Built-in plugins: ElevenLabs (cloud) + Bark (local)
- Auto-discovery from backend/plugins/ directory
- GET /tools/plugins API for frontend engine picker

GPU Crash Sandbox:
- Subprocess isolation for GPU-intensive operations
- CUDA OOM / driver crash kills worker, not the server
- Async wrapper with configurable timeout
- Platform availability check

Waveform Timeline v2:
- Added MinimapPlugin (20px overview bar)
- Added TimelinePlugin (time labels)
- Keyboard shortcuts: J/K/L (rewind/play/forward), Space
- Full ARIA labels on all controls
- role=region, role=toolbar for assistive tech

Accessibility:
- ARIA labels on waveform controls, theme picker, capture button
- role=radiogroup on theme dots
- aria-checked state on theme selection
- Keyboard hint icon (J/K/L) in waveform toolbar

LLM Translation: already implemented (OpenAI provider in dub_translate)
Roadmap: cleaned up, only batched TTS + vision items remain
 2026-04-28 11:52:53 +05:30
debpalash 809943b881 feat: dictation capture, casting view, real-time dub preview 
Voice Capture (Dictation):
- CaptureButton FAB with ⌘+⇧+Space global shortcut
- Records mic → POST /transcribe → displays text → copy to clipboard
- Backend capture.py: standalone ASR endpoint (no dub job needed)
- Animated waveform bars, glassmorphic panel, pulse recording indicator

Speaker Casting:
- CastingView component — visual speaker-to-voice assignment grid
- Auto-cast from video speaker clones or manually pick saved profiles
- Dropdown picker with personality tags, preview button
- Registered in CastingView.css with premium glassmorphism

Real-time Dub Preview:
- POST /dub/preview-segment/{job_id} — 8-step fast TTS for single segment
- No disk write, no watermark, no mix — just instant audio feedback
- Returns WAV bytes directly for immediate playback
 2026-04-28 11:39:37 +05:30
debpalash e2f576f59e feat: MCP server + audio effects chain 
MCP Server:
- Full Model Context Protocol server (backend/mcp_server.py)
- 5 tools: generate_speech, list_voices, list_personalities,
  list_languages, check_health
- 2 resources: voice://{id}, history://recent
- stdio + SSE transports for Claude Desktop / Cursor / remote agents
- Example config: mcp.json

Audio Effects Chain:
- 6 presets: Broadcast, Cinematic, Podcast, Warm, Bright, Raw
- Configurable pipeline via apply_effects_chain() with pedalboard
- Effects: highpass, lowpass, compressor, reverb, noise_gate, eq, limiter
- GET /tools/effects API for frontend preset picker
- Graceful fallback when pedalboard isn't installed
 2026-04-28 11:28:26 +05:30
debpalash c77bf18ac4 feat: onboarding demo profile, voice personalities, i18n framework 
- Onboarding: seed 'OmniVoice Demo' profile on first run (empty DB)
  with bundled reference audio so Launchpad isn't empty
- Voice Personalities: 6 built-in presets (Narrator, Casual, News
  Anchor, Storyteller, Corporate, Energetic) with instruct text
  auto-fill in Voice Design mode
- i18n: react-i18next with English locale, browser language detection,
  Launchpad & CloneDesignTab strings extracted to en.json
- DB migration v4: personality TEXT column on voice_profiles
- New API: GET /personalities returns preset list
- CSS: demo callout banner + personality picker strip
 2026-04-28 11:11:19 +05:30
Palash Debnath 34610ca091 feat: real-time WebSocket event bus + sidebar reactivity fixes (#27) 
## Core Infrastructure
- Add backend event bus (core/event_bus.py) — in-memory pub/sub with
  emit(), subscribe(), unsubscribe()
- Add WebSocket endpoint /ws/events (api/routers/events.py) with 25s
  keepalive pings and auto-cleanup on disconnect
- Add frontend hook useRealtimeEvents.js — single WS connection with
  exponential backoff reconnect (2s→60s)

## Backend Event Integration
- projects.py: emit on create/update/delete
- profiles.py: emit on create/update/lock/unlock/delete
- dub_core.py: emit on clear/delete history
- dub_pipeline.py: emit on save_job (every pipeline write)
- exports.py: emit on export/record
- generation.py: emit on generate/clear/delete
- gallery.py: emit on save-as-profile/to-profile

## Frontend Improvements
- Replace 45s polling interval with instant WS-based invalidation
- Fix critical bug: apiModelStatus was undefined, causing loadAll()
  to loop forever — sidebar data never loaded on startup
- Add websockets to main deps (was optional, got removed by uv sync)
- Reduce model/status polling from 5s to 10s, disable background
  polling for logs
- Add ReadinessChecklist and FloatingPill components
- Default UI scale changed from S (1.0) to M (1.3)

## Dependencies
- Add websockets>=16.0 to main dependencies for uvicorn WS support

Closes #3 (native desktop app exists via Tauri)
Closes #5 (Dockerfile already uses root bun.lock)
Resolves #26 (Triton workaround documented)
 2026-04-27 22:16:28 +05:30
debpalash f8b4673e1f fix(ui): Fix segment row layout collapse, memory bugs, enterprise page, and UI enhancements 2026-04-27 00:21:47 +05:30
debpalash fc76e79ff8 feat: setup wizard, donate page, CI fixes, performance optimizations, and style extraction 
- Implement donate page and migrate API fetching to react-query hooks
- Add setup wizard for batch job management and voice clip editing
- Refactor setup router into package (wizard, models, download sub-modules)
- Fix 9 CI test failures from setup router refactor
- Fix cross-device link error in prefs.py atomic writes
- Fix event loop mismatch in export test fixtures
- Modernize README with architecture diagram and 13 app screenshots
- Defer per-segment disk writes in dub_generate for ~6s faster dubs
- Extract 45 inline styles from Launchpad, KeyboardCheatsheet, DubSegmentRow
- Add playwright dev dep and screenshot capture script
 2026-04-26 16:47:00 +05:30
debpalash 811c842a75 feat: implement Voice Gallery feature with backend routing, API client, and frontend navigation integration 2026-04-25 19:29:29 +05:30
debpalash 901eb040a8 feat: add live bootstrap progress bars and log inspection to splash screen 2026-04-25 17:22:51 +05:30
debpalash 4a8b06c25e feat: implement structured progress tracking for model downloads and add local environment variable loading support. 2026-04-24 18:51:53 +05:30
debpalashandClaude Opus 4.7 93d8cd70d5 feat: pre-flight system check + actionable error surfacing 
## /setup/preflight endpoint (backend/api/routers/setup.py)
New one-shot health check the setup wizard calls before model install.
Probes every runtime requirement so GPU driver mismatches, missing
ffprobe, low RAM, stale AMD ROCm setups, and unreachable HF no longer
manifest as silent CPU fallbacks or opaque runtime errors.

Checks returned as {id, label, status, detail, fix?}:
  - Operating system + arch
  - Python runtime
  - System RAM (fail <8 GB, warn <12 GB)
  - Disk free on HF cache partition (fail <10 GB)
  - HuggingFace cache writable
  - FFmpeg (required)
  - FFprobe (warn — some endpoints degrade without it)
  - GPU acceleration — vendor-aware detection:
      * Apple Silicon  → MPS available?
      * NVIDIA         → nvidia-smi parse; fail if driver < R555
                         (cu128 wheels we ship need ≥ 555)
      * AMD            → rocm-smi detect; warn if torch not built w/ ROCm
      * none/unknown   → warn, CPU-only note
  - Network reachability to huggingface.co:443

Aggregate: {ok, has_warnings, checks, device}. Wizard blocks forward-
nav on any fail; passes warnings through with a labelled Continue.

## SetupWizard 4-step flow (frontend/src/pages/SetupWizard.jsx)
Insert "System check" as step 1 between Welcome and Install models.
Renders preflight report with pass/warn/fail icons, inline fix
instructions, and a Re-check button for users who resolve a blocker
without restarting the app. Continue button labels shift based on
status ("All good — continue" / "Continue (with warnings)" /
"Resolve blockers to continue").

## Transcribe-stream error clarity
dub_core.py: move ASR / missing-audio preflight out of HTTP-status
error paths into in-stream `error` events, since EventSource on the
client can't read non-2xx bodies and previously surfaced 503s as
opaque "network error" strings. Users now see the actionable message
(e.g. "ASR isn't loaded yet — check Settings → Models") inline.

App.jsx: on transcribe-stream drop before any final segment, force-
close + reject with a pointed message instead of waiting for
EventSource auto-reconnect to thrash against a broken endpoint.

## API client (frontend/src/api/setup.ts)
Add PreflightReport / PreflightCheck / PreflightDevice types + preflight()
call matching the new endpoint.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-22 18:19:16 +05:30
debpalashandClaude Opus 4.7 994c6cf065 feat(backend): setup wizard router, translation engines, export options, client-disconnect handling 
- Add setup router (backend/api/routers/setup.py) for first-run wizard:
  system checks, engine probes, model downloads with progress
- Add translation engines service with pluggable backends
- Add utils/hf_progress for HuggingFace download progress streaming
- Add PyInstaller runtime hooks (numpy compat, torch compiler disable)
- Global exception handler short-circuits h11 LocalProtocolError and
  Starlette ClientDisconnect with HTTP 499 to silence noisy stack traces
  when users scrub or cancel video mid-stream
- /dub/download-mp3 accepts bitrate query param (clamped 64–320kbps)
- Refactor ASR/TTS backends, dub pipeline, engine management
- Update backend.spec for PyInstaller packaging
- Bump pyproject version to 0.2.0; refresh uv.lock

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-22 17:49:04 +05:30
debpalash 52d68d05dc refactor: update backend architecture, expand frontend state management, and synchronize voice-pro research modules. 2026-04-21 18:32:25 +05:30
debpalash 6f124fb175 feat: implement frontend UI components and expand research documentation for voice processing and translation workflows. 2026-04-21 04:31:08 +05:30
debpalash 2c3e12d8de feat: implement responsive layout adjustments for small screens and add cached status visualization to dubbing workflow 2026-04-20 11:33:30 +05:30
debpalash 6e89db0f77 feat: add YouTube/URL ingestion support using yt-dlp and update UI with granular preparation progress tracking 2026-04-19 17:54:07 +05:30
debpalashandClaude Opus 4.7 15ef65a686 refactor: extract App.jsx into pages/components/api, add studio design pass + NavRail 
Frontend:
- Split App.jsx (2850 -> ~1300 lines) into pages/{Launchpad,CloneDesignTab,DubTab,Settings}.jsx
  and components/{Header,Sidebar,NavRail,CompareModal,DubSegmentTable,DubSegmentRow}.jsx.
- Centralize every fetch through api/{client,dub,generate,profiles,projects,system,exports}.js
  with consistent ApiError + JSON error detail extraction.
- Extract utils: constants (TAGS/CATEGORIES/PRESETS/POPULAR_*), languages (LANG_CODES),
  format (formatTime/probeAudioDuration), consoleBuffer (ring for Settings > Logs > Frontend).
- Lazy-load AudioTrimmer, DubSegmentTable, Launchpad, CloneDesignTab, DubTab, Sidebar, CompareModal, Settings.
  Initial bundle 438KB -> 220KB (-50%).
- Virtualize segment table (react-window) + React.memo row; dynamic row height when
  original text row shown. Fix {proj.is_locked && ...} rendering literal "0" on falsy.
- Segment UX: multi-select + bulk voice/lang/delete, Ctrl+D split-at-cursor,
  Ctrl+M merge-with-next, search/filter/speaker filter, char-budget warn
  when translated text >1.3x source, preserve text_original across translations,
  always stream segments via EventSource /dub/transcribe-stream.
- AudioTrimmer: pro-grade zoom/pan/scrub, rAF-throttled drag, peak precompute
  + async refine (keeps UI responsive on 1700s mp3), keyboard shortcuts,
  click-drag = fresh selection, loop preview, Enter/Esc/Space/Home/End bindings.
  Tests: 22 cases in tests/frontend/audioTrim.test.mjs cover encodeWav header,
  peak min/max invariants, drag modes (start/end/region/pan/new), zoom math,
  slice-to-mono, tick interval picker.
- NavRail: left/right vertical icon rail (VS Code style), side persisted to
  localStorage. Removes tab group from Header.
- View-specific sidebar: hidden on Launchpad/Settings; dub gets 3 tabs; clone/design 2.
  app-container grid updated with sidebar-hidden / rail-right variants.

Design pass (hand-drawn "cute" identity):
- Fraunces italic serif for headlines, Nunito rounded sans for body.
- Wobbly non-uniform border-radius on cards/buttons/inputs.
- Warm peach/rose/lime palette across Launchpad, Settings, Header, panels,
  sidebar items, segment table.
- Header HQ: view breadcrumb (pulsing dot + kicker + accent-colored view label
  + active project), live mini-waveform reacting to model status.
- Settings: tabbed Models / Logs / About / Privacy with accent-colored pills;
  Logs sub-tabbed Backend / Frontend / Tauri.
- Clone/Design: two columns, each split into two studio-panels (prompt vs lang/steps;
  voice source vs overrides+synth). Rectangular corners + launchpad-warm gradient.
- Dub tab: migrate panels to studio-panel look.
- Sidebar item overhaul: kind pill + ago timestamp + hover-reveal pill actions,
  accent left-edge bar on hover, click-whole-card = primary action.

Backend:
- Per-segment translate retry + auto-src fallback; surfaces errors per-segment
  with logged type. Tests: 9 cases in tests/test_dub_translate.py (code coverage,
  source-lang resolution, retry/auto fallback, empty text handling).
- Thumbnail extraction during dub upload (ffmpeg @ ~10% offset, scale 320px wide)
  served via GET /dub/thumb/{job_id}.
- Preserve text_original during transcription so cross-language retranslations
  re-run from pristine source, not compounding on prior translation.
- Settings endpoints: GET /system/info, GET /system/logs?tail=N,
  GET /system/logs/tauri, POST /system/logs/clear. FK-safe profile_id NULL on
  history insert to avoid "FOREIGN KEY constraint failed" on stale/preset ids.
- /dub/transcribe-stream SSE: chunked mlx_whisper / pytorch pipeline per 30s
  window, diarization final pass, honors job.aborted.

Tests: 22 frontend trim + 9 backend translate all pass. Production build 220KB
gzipped (from 127KB main-only pre-split, which hid unshipped modules in main).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-19 04:52:54 +05:30
debpalash 508d4ba116 feat: add audio trimming for reference clips, implement streaming transcription, and refactor ffmpeg utility handling 2026-04-18 22:37:20 +05:30
debpalashandClaude Opus 4.7 67328d04fe refactor: split backend into api/core/services/schemas, harden security + fd pressure, add searchable language picker, fix segment fragmentation 
Backend:
- Split monolithic main.py into backend/{api/routers,core,schemas,services}
- core/db.py: allowlist-gated migrations, db_conn context manager (kills SQL injection on ALTER)
- core/tasks.py: lock-guarded listener add/remove/push, snapshot-before-iterate
- services/ffmpeg_utils.py: run_ffmpeg helper with concurrency semaphore, EAGAIN retry, guaranteed reap
- services/segmentation.py: Bengali/CJK/Arabic punctuation, ultra-short tier, stitch_adjacent_shorts,
  bounded-loop merge; public clean_up_segments API
- services/model_manager.py: robust lock.locked() handling
- api/routers/dub_core.py: job_id traversal guard, thread-safe _active_procs, timeouts on ffmpeg/demucs,
  POST /dub/cleanup-segments endpoint
- api/routers/dub_export.py: guarded SSE listener remove, ffmpeg timeouts via run_ffmpeg
- api/routers/exports.py: destination_path validation, safe source resolver, subprocess list-form
- api/routers/generation.py: contextlib.suppress on tempfile cleanup, db_conn usage, safe output-path helper
- api/routers/system.py: try/finally tmp cleanup, subprocess timeouts
- schemas/requests.py: TranslateSegment.id int->str to match hex segment IDs
- main.py: threading.Lock around crash log writes

Frontend:
- components/SearchableSelect.jsx: popover combobox with search, keyboard nav, popular+recent pins, 200-item cap
- App.jsx: wire SearchableSelect for dub language / ISO code / voice-gen language; Clean Up segments button;
  fix blob URL leak (object-shaped prev in setter, unmount cleanup via ref)
- components/WaveformTimeline.jsx: explicit <video> detach instead of innerHTML='' to release decoder
- index.css: ss-* combobox styles matching Gruvbox theme

Tests:
- tests/test_segmentation.py (26 cases), test_dub_transcribe.py, test_dub_export_unique.py, conftest.py

Chore:
- .gitignore: exclude omnivoice.zip, /research/ reference clones
- Remove tracked stray root test scripts + crash_log.txt

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-18 17:20:19 +05:30