034d1c4811d77458dd147cc9fbbd9bb225464647
45
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Palash DebnathandClaude Opus 4.8
|
87eb5ad078 |
feat(dub): audio-only dubbing mode (#119) (#150)
* feat(dub): audio-only dubbing mode (#119) Add an audio→audio dubbing path: upload an audio file, get dubbed audio out, with no video processing. The transcribe → translate → TTS core is unchanged; only the video-coupled stages are skipped. Backend: - dub_core /dub/upload: new `input_type` form field ("video"|"audio"). Audio mode validates the upload is a known audio container (else 400) and threads input_type into the ingest source dict. - dub_pipeline ingest: for audio input, skip scene detection + thumbnail ffmpeg passes (still emits scene_done count=0 so the prep SSE contract the frontend waits on is unchanged); stores input_type on the job. - dub_export /dub/download: for audio jobs, branch to an audio-only export (_build_audio_export_cmd) — no video input/map/codec/subtitle pass. Outputs dubbed_audio_{lang}_{stamp}.{wav|m4a|mp3|flac} via `out_format` (default m4a), optionally mixed with the separated background. Unknown formats fall back to AAC. Frontend: - dubSlice: dubInputType state + setter (default 'video'). - DubTab: auto-select audio-only mode when an audio file is dropped/picked. - dub.ts/useDubWorkflow: pass input_type on upload. Tests (11): _build_audio_export_cmd format/mix matrix; end-to-end audio-only export produces an audio file (no video mux); unknown-format fallback; upload rejects a video extension in audio mode. Closes #119. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * harden(#119): allowlist-sanitize lang_code in audio export path The track id is already constrained to an existing track key, but allowlist-sanitize it before it reaches the output path (same pattern as the existing safe_name) so a path component can never carry separators — clears the CodeQL path-injection flag on the new audio-export branch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * polish(#119): address Greptile P2s on audio-only dubbing - dub_pipeline: emit scene_start before scene_done(count=0) for audio so the prep SSE stage sequence is symmetric with the video path. - useDubWorkflow: 'Preparing audio…' pill for audio jobs (was always 'Preparing video…'). - DubTab: widen the drop-accept regex + file-input accept to the full supported audio set (aac/opus/wma) so it matches the input-type detection and the backend allowlist. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(#119): drop unused dubInputType read in DubTab (CodeQL) Only setDubInputType is used; the value read was dead. Clears the CodeQL unused-variable alert. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Palash DebnathandClaude Opus 4.8
|
8b00dc1f4f |
feat: onboarding demos, opt-in bug reporting, error-docs deeplinks + issue triage (#133)
* feat: onboarding demos, opt-in bug reporting, error-docs deeplinks + issue triage Working-tree snapshot bundling several in-flight workstreams (v0.3.0): - Onboarding/demo system: DemoPresetGrid, DictationDemo, DubbingDemo components + tests, render scripts (render_demos_omnivoice.py, build_demos.sh, build_dub_demo.sh), personalities preview URLs, alembic 0002 voice-profile demo fields. - Opt-in bug reporting: ReportBugButton (prefilled GitHub-issue URL path). - Error transparency UX: errorDocsMap deeplinks + BootstrapSplash/error wiring. - Dub workspace: DubSegmentRow/Table, WaveformTimeline, dubSlice tweaks. - Issue triage: .planning/issue-clusters/ (plan-01..05 root-cause masters, GH #128-#132). - CLAUDE.md: hard rule — everything ships on v0.3.0, no version bumps. KNOWN GAP (why this is a draft): the generated demo audio assets are NOT in this tree, and backend/assets/samples/demo_voice.wav is deleted. onboarding.py guards the missing file (skips seeding the demo profile with a warning), so no crash — but first-run Launchpad will be empty and /demo_audio/ preview URLs 404 until assets are regenerated via scripts/build_demos.sh. Do not merge before regenerating + committing the demo assets. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(dub): timing strategies — kill audio compression, add Concise + Stretch Video Replaces the current audio time-compression default (atempo squeeze to fit slot) that produced chipmunk/alien output on high-density target languages like Bengali. Two new user-selectable modes; legacy behaviour kept behind an explicit "Strict slot" choice. New `DubRequest.timing_strategy` enum (default "concise"): - "concise" Translator trims text to fit at natural rate; if it still overflows, hard-trim at slot with a fade so we never overlap the next speaker. Surface overflow_s per segment so the user can shorten the text. - "stretch_video" Audio plays at natural 1.0× rate. Backend computes a per-segment new timeline; persists a video_stretch_plan on the job. Mux step (dub_export) builds an ffmpeg trim+setpts+concat filter graph that stretches each segment's video portion to match the natural-rate dub audio. Gaps/pre-roll/tail pass through at 1.0×. Sub burn under stretch_video is skipped in one pass (cues would drift). - "strict_slot" Legacy atempo squeeze. Retained for back-compat. Director rate-bias side-effect (seg_speed *= bias) now gated on strict_slot only, so "urgent"/"slow" direction tokens keep their instruct effect in the new modes without chipmunking. Per-segment fit_status emitted in the SSE done event: {status: "fits" | "overflows" | "video_stretched", overflow_s?, stretch_ratio?} DubSegmentRow's "Sync: 100%" badge (which was lying — sync_ratio was always ~1.0 because the TTS loop pre-trimmed to slot) is replaced with a truthful "Fits / Overflows +Ns / Video 1.18×" label. Frontend: - prefsSlice.timingStrategy (persisted, store v3→v4 with safe migrate). - DubTab footer Segmented control: "Concise · Stretch Video · Strict slot". - useDubWorkflow passes timing_strategy on /dub/generate; consumes fit_status. Tests: tests/test_dub_timing_strategy.py — 13 cases covering schema defaults/validation, _build_video_stretch_filter_graph (pre-roll, gap, tail, empty-plan early return, post-subtitle chain-in), and _video_stretch_plan_for guards. 30/30 existing dub tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(waveform): surface missing source as "Source media missing" instead of code-4 black box When a project's underlying media file is gone (moved or deleted between save and reload) the <video> element fires MediaError code 4 and the companion audio fetch returns HTTP 404 — both were silently warned to the console while the user stared at an unresponsive black panel and an empty waveform. - WaveformTimeline now flips loadError when the video element rejects code 3 (decode) or 4 (src not supported), and tracks `sourceMissing` separately so the error UI can name the actual problem. - The audio decode fallback chain catches HTTP 404 specifically and treats it as source-missing instead of loading silent empty peaks — an empty waveform on a deleted source is more confusing than a clear "Re-upload the video to continue" message. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(tray): "Show OmniVoice" reloads when the webview is blank When the dev Vite server restarts (or the main window is created before the backend is ready), the webview load fails and the window is left with `<body></body>` plus a "Could not connect to the server" console error. Clicking "Show OmniVoice" from the tray menu just re-showed the broken window — there was no recovery path short of quit+relaunch. Now the show handler runs a tiny eval after `show()`/`set_focus()` that calls `location.reload()` only when `document.body.childElementCount === 0`. A healthy window doesn't blink (body is non-empty); a blank one self-recovers as soon as the user clicks Show. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(#133): bug-report diagnostics field mapping + drop unused imports Address PR #133 review: - ReportBugButton: /system/info exposes `platform` + `device`, not `os`/`torch_device`/`gpu` — those reads silently dropped OS/GPU from every bug report. Map to the real fields (CodeRabbit). Also remove the dead `home` local in stripHome (CodeQL unused-variable). - DictationDemo: drop unused `Loader` import (CodeQL unused-import). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Palash DebnathandClaude Opus 4.8
|
adf486ee03 |
chore: set version to 0.3.0 across all sources (+ drop v0.4 references) (#145)
* chore: drop stray v0.4 references — everything ships on the v0.3.0 line Per the project's versioning rule (no v0.4, no unprompted version chatter): - backend/main.py + marketplace.py: the app reported version "0.4.0" (ahead of even pyproject's 0.2.7 and referencing a forbidden version). Aligned to "0.2.7" to match pyproject.toml / tauri.conf.json — a consistency fix, not a bump. - errorDocsMap.ts / indextts/bootstrap.py / _secret_key.py: reworded "v0.4" deferral comments to version-agnostic "deferred / later hardening pass". - docs/install/troubleshooting.md: the "tracked for v0.4" notarization line now matches macos.md (signing is wired; activates on the Apple cert secrets). Note: historical planning records under .planning/ still contain "defer to v0.4" notes; left as-is (a record of superseded decisions) — CLAUDE.md + the constitution are the live source of truth. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: set version to 0.3.0 across all sources (current dev line) The current/upcoming version is v0.3.0 (0.2.7 is the prior stable). Bump every version source so the codebase consistently reports 0.3.0 — the in-code dev version; the git *tag* still happens later per the release cadence. - pyproject.toml, frontend/src-tauri/Cargo.toml, tauri.conf.json, frontend/package.json: 0.2.7 → 0.3.0 - backend/main.py (FastAPI) + marketplace.py export metadata → 0.3.0 (these had drifted to a phantom "0.4.0") - CHANGELOG.md: "[0.2.7] — Unreleased" → "[0.3.0] — Unreleased" - uv.lock + Cargo.lock reconciled (1-line each) so `--frozen` installs hold. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(version): read app version from package metadata (no more drift) Greptile (#145): the FastAPI version + marketplace bundle metadata were bare string literals — they'd go stale-wrong again at the next bump (the exact class of bug this PR fixes; that's how "0.4.0" happened). Read once from importlib.metadata.version("omnivoice") via core.version.APP_VERSION, with a "0.3.0" fallback only for a non-installed source checkout. pyproject.toml is now the single source of truth for the runtime version. Tests: tests/test_app_version.py (semver + equals installed metadata). 2 pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Palash DebnathandClaude Opus 4.8
|
898f41a57d |
fix(windows): gate torch.compile on Triton + ASR critical-path smoke (plan-02, closes #65) (#138)
* fix(windows): gate torch.compile on Triton availability (#129, closes #65) plan-02. torch.compile(mode="reduce-overhead") needs Triton at runtime; Triton has no Windows wheel, so the old `device=="cuda"`-only guard in model_manager.py failed on Windows+CUDA and surfaced as a confusing "OOM" (#65). Inference-time, hard to diagnose. - engine_env.should_torch_compile(device): requires CUDA + find_spec("triton") + the existing perf.torch_compile_disabled setting being off; logs the skip reason at INFO and falls back to eager. - model_manager.py call site uses it instead of the bare cuda check. - smoke-test.sh INST-02: import torch + ctranslate2 + whisperx (full ASR path) so a missing transitive dep fails the build instead of crashing mid- transcription (#116). Runs in the CI smoke-matrix on Win/macOS/Linux. setuptools>=75.0 (fix-sequence step 1) already pinned (#58). Linux/CUDA+Triton behaviour unchanged. Tests (TDD): tests/test_torch_compile_gate.py (4). Closes #65; addresses #129/#116. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(windows): also gate subprocess torch.compile on Triton (Greptile #138) Greptile flagged that the in-process gate left a parallel gap: engine subprocesses honour TORCH_COMPILE_DISABLE, but build_engine_env() only set it on the user's Performance toggle — so a Triton-absent host (Windows, or macOS) still exposed subprocess engines to the same crash this PR fixes in-process. - build_engine_env(): set TORCH_COMPILE_DISABLE=1 when the user disabled compile OR Triton is unavailable (find_spec), cross-platform — mirrors should_torch_compile(). Drops the Windows-only scoping (and the now-unused `import sys`). - Refreshed the stale module docstring. - 3 new tests cover the subprocess gate (triton-missing, triton-present, user-opt-out). 7/7 pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * revert(engine_env): keep subprocess TORCH_COMPILE_DISABLE user-driven Reverts the build_engine_env() broadening from the previous commit. Auto- disabling subprocess torch.compile on Triton-absence conflicts with a deliberate, tested contract (test_perf_settings: Windows + flag-off ⇒ no injection; non-Windows ⇒ never inject) — the subprocess var is intentionally under the user's explicit control. The #65 fix is the in-process should_torch_compile() gate (unchanged here), which IS automatic and fully tested. Pushing back on the subprocess auto-gate as a separate, deliberate contract change rather than forcing it through by rewriting established tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Palash DebnathandClaude Opus 4.8
|
b64f53b0af |
feat: pipeline error transparency — no more silent "unknown error" (plan-04, closes #131) (#136)
* docs(plan-04): spec + plan for pipeline error transparency (#131) speckit spec/plan/research/data-model/contract/quickstart for plan-04. Grounds the fix in the real code map: shared failure-event builder (backend/core/failure.py) feeding tasks.py + dub_pipeline.py + dub_core.py, non-empty reason guarantee, sanitized diagnostic block, frontend renderer with docs deeplink. Closes-target: #131 (children #122, #63). Design only — no code changes yet. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): structured, non-empty failure events + logged tracebacks (#131) plan-04 backend: no more silent "unknown error". A shared failure helper guarantees a non-empty reason at every emit site and a sanitized, copyable diagnostic block. - backend/core/failure.py: build_failure()/build_failure_event() (reason falls back to the exception class name), sanitize() (reuses the logging_filter HF-token regex + redacts *TOKEN*/*KEY*/*SECRET* env values + home→~), diagnostic() (reuses the env capture), classify() reusing the error_docs_map 5-class taxonomy for the docs deeplink + hint. - core/tasks.py worker: structured event instead of bare str(e); keeps the logged traceback. - services/dub_pipeline.py: enrich download/extract error yields; ADD the missing outer `except Exception` (the #122 path — unhandled ingest errors were never surfaced with stage context); surface the previously-silent demucs/scene/thumbnail degradations as non-fatal `warning` events. - api/routers/batch.py: guaranteed non-empty batch failure reason. SSE payload is additive (legacy `error`/`stage`/`detail` keys preserved), so existing frontends keep working and already show the specific reason. Tests (TDD, fail-before/pass-after): 14 cases — non-empty-reason guarantee, redaction, diagnostic sanitization, and the 3 Test-matrix triggers (worker / extract / url). 483 passed, 0 regressions. Closes #131. Refs #122, #63. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dub-ui): show specific cause + docs deeplink + copyable diagnostic (#131) plan-04 frontend. The backend now sends a structured, non-empty failure; surface it to the user instead of "extract: unknown error". - dubSlice: DubFailure type + dubFailure state/setter. - useDubWorkflow: capture the structured failure on the SSE error event (reason/error_class/stage/hint/docs_topic/diagnostic); clear on new runs. - DubTab: DubFailureNotice renders the actionable hint, an "Open docs" deeplink (via the existing errorDocsMap classifier), and a "Copy diagnostic" button — shown beneath the error badge in both failure banners. typecheck + build clean; 66 frontend tests pass. Refs #131, #122, #63. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(failure): annotate intentional best-effort excepts (CodeQL) The new security workflow's CodeQL flagged 5 bare `except: pass` blocks. All are deliberate best-effort guards (sanitize/diagnostic must never throw on the failure path; the test cancels the worker to tear it down). Added explanatory comments per CodeQL's py/empty-except rule. No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Palash DebnathandClaude Opus 4.7
|
b34dcd9e11 |
Phase 4 Plan 04-01: SPIKE-01 GGUF — GO + integration (#100)
* Phase 4 Plan 04-01: SPIKE-01 GGUF — GO + Wave 1 integration Integrates Serveurperso/OmniVoice-GGUF as a hardware-adaptive default voice-cloning engine, with overridable fallback to the in-process OmniVoiceBackend. Spike confirmed GO: the model is a clean quantization of k2-fsa/OmniVoice (Apache-2.0 + MIT runtime, `omnivoice-lm` custom architecture so it does NOT load in vanilla llama.cpp). Pinned SHAs: * Serveurperso/OmniVoice-GGUF revision: 361609388ae572a820d085185bbbe2a2aac4b30e * ServeurpersoCom/omnivoice.cpp master: 886fc079838ca7400cb2b42b36e2a65aa1daabe8 Implements GGUF-01 (hardware probe) through GGUF-05 (default-engine resolver with graceful fallback). The four `bin/omnivoice-tts-*` artifacts are committed as zero-byte placeholders; the new CI matrix job builds the real binaries per platform from the pinned commit SHA and appends a SHA-256 manifest used by `is_available()` for tampering detection (T-04-01). The macos-14 (Apple Silicon) slot is marked `continue-on-error: true` because omnivoice.cpp publishes no `buildmetal.sh` (Pitfall 1 / Assumption A1) — failure feeds into Task 3's GO/NO-GO call. Quant override is allow-listed against quant_map.json entries only (T-04-05). Argv is composed from typed Path objects rooted in HF_HUB_CACHE; never uses `shell=True`. HF token redaction applies to captured stderr before logging (AUTH-05 / T-04-04). Tests: 36 new (8 hardware-probe + 13 GGUF engine + 6 settings_store quant override + grep gate); 428 passed in full suite vs 402+ baseline. ADR Status stays "Proposed (research-supported)" — Task 3 (human checkpoint) flips to Accepted after CI produces real binaries and a reviewer signs off on the GGUF-06 cross-hardware smoke. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: install libopenblas-dev on linux-x86_64 omnivoice-tts build The pinned omnivoice.cpp commit (886fc079...) ships a `buildcpu.sh` that passes `-DGGML_BLAS=ON`. ubuntu-latest has no BLAS implementation preinstalled, so the cmake configure step fails with `Could NOT find BLAS (missing: BLAS_LIBRARIES)` and the job exits in 13 s before producing the linux-x86_64 binary. macOS (Accelerate, built in) and Windows (BLAS off by default in the ggml CMakeLists for non-APPLE platforms — the build script doesn't invoke buildcpu.sh on those slots) are unaffected and stay green. Adds a Linux-gated apt step to install libopenblas-dev + pkg-config before the build, restoring cross-platform parity per the CLAUDE.md "default features must work on every platform" rule. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(gguf): constrain ref_audio to project roots — block /etc/shadow on Linux The GGUF engine's `_build_argv` previously validated ref_audio only via `ref_path.is_file()` — i.e. "does this path exist?" That check is platform-dependent: `/etc/shadow` doesn't exist on macOS (rejected naturally), but it IS a real system file on Linux, so the validation silently accepted it. CI's ubuntu-22.04 runner exposed the gap via `test_generate_blocks_freeform_ref_audio`, which exists precisely to guard the "freeform ref_audio path" attack surface. Fix: confine ref_audio to one of three allowed roots before existence checks: - VOICES_DIR (user-saved voice profiles) - DUB_DIR (per-job auto-clones extracted from source video) - tempfile.gettempdir() (browser-upload temp files; existing `cleanup_ref` flow in generation.py) Anything outside those roots → FileNotFoundError, matching the existing failure-mode contract callers handle. Existence check still runs after, so the test's mocked subprocess.run is never reached and the test passes deterministically on all three platforms. Cross-platform parity (per CLAUDE.md 2026-05-20 rule): identical behaviour on macOS / Windows / Linux — the allow-list is computed from core.config which uses platform-specific path resolution but yields the same logical "project tree" on every OS. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci(gguf): mark darwin-x86_64 binary build as experimental GitHub's macos-13 (Intel) runner pool is heavily contended — PR #100 queued for 30+ minutes waiting on darwin-x86_64 while every other platform finished in ~1m. Intel Macs are also fading hardware (Apple's platform momentum is entirely on Apple Silicon), and the GGUF engine's runtime already handles a missing binary gracefully (`is_available()` returns False on Intel Mac with a "binary not bundled for this platform" message, same path used for first-launch before any binaries build). `experimental: true` mirrors what darwin-arm64 (Metal) already has — slot still runs and uploads its binary when successful, but a failure or runner backlog no longer blocks merges. Keeps the GGUF engine shippable across the dominant arm64 / Linux / Windows surface without holding the inbox on a slow-runner queue. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Palash DebnathandClaude Opus 4.7
|
f7dedfcfae |
fix: speaker detection — gated pyannote license surfaces a docs deeplink (closes #78) (#110)
* fix: speaker detection — gated pyannote license surfaces a docs deeplink (closes #78)
Issue #78 ("Speaker detection fails — speakers blend together or aren't
detected correctly") was the user-visible symptom of the dub pipeline
silently falling back to the silence-gap heuristic in
`backend/api/routers/dub_core.py::_diarize`. The heuristic alternates
Speaker 1 ↔ Speaker 2 on >1.2s gaps only, so two real speakers with
similar pacing get merged or swapped — and once the auto-clone step
extracts a reference voice for the wrong label, downstream dubs make
"person A speak like person B" (the reporter's exact phrasing).
The structural cause is that pyannote-3.1 is gated on HuggingFace: a
valid HF_TOKEN by itself isn't enough — the user must also click
"Agree and access repository" on both pyannote/speaker-diarization-3.1
AND pyannote/segmentation-3.0. We can't fix that for the user, but we
CAN make the failure actionable instead of silent.
Changes:
- `backend/services/model_manager.py`: `get_diarization_pipeline()`
gains an opt-in `return_error=True` shape that returns
`(pipeline | None, error_sentinel)`. Sentinels distinguish
NO_TOKEN / PYANNOTE_LICENSE_REQUIRED / LOAD_FAILED. A new
`_classify_diarization_error()` sniffs the exception's class name +
message for 401/403/gated/"accept license" signals — kept as a
string heuristic so it survives huggingface_hub major-version
churn. Bare-`None` default return preserved for the legacy
`_transcribe` call site at dub_core.py:781.
- `backend/api/routers/dub_core.py::_diarize`: now emits a structured
SSE warning `{detail, source, error_class, docs_url}` instead of
plain `{detail, source}`. The new fields let the front-end render a
"See docs" button that deeplinks directly to the
`License acceptance flow` section of `docs/features/diarization.md`
(landed in PR #94) — the page with the click-by-click instructions
for fixing this exact failure mode.
- `backend/core/error_docs_map.py` + `frontend/src/utils/errorDocsMap.ts`:
add a 5th taxonomy class `PYANNOTE_LICENSE_REQUIRED` pointing at the
diarization docs section. Distinct from `HF_AUTH_FAILED` (which is
the more general "token missing or invalid" case). The TS
`classifyError` heuristic also picks up pyannote / gated /
"speaker diarization" keywords so a thrown error in the boundary
routes to the right deeplink too.
- `tests/backend/core/test_error_docs_map.py`: bump locked-keys set to
5 classes; add an explicit assertion that the new class points at
the `license-acceptance-flow` anchor.
- `frontend/src/utils/errorDocsMap.test.ts`: bump locked-keys set to
5 classes; add classifier tests for pyannote / gated / accept-license
keyword routing.
- `tests/test_diarization_error_class.py`: regression test (20 cases)
covering `_classify_diarization_error`, the new
`get_diarization_pipeline(return_error=True)` shape, backward-
compatible bare-`None` return for the legacy call site, and the
error_docs_map deeplink target. Uses sys.modules patching so
pyannote / torch are never actually imported.
HF token plumbing: unchanged. The new code continues to route through
`token_resolver.resolve()` per the AUTH-01 contract — no new bare
`os.environ.get("HF_TOKEN")` reads.
Cross-platform: identical behaviour on macOS / Windows / Linux —
the only platform-touching change is a docs URL string, which is
opened via the existing `openExternal()` helper that already abstracts
Tauri's `shell.open` on all three platforms.
Verification:
.venv/bin/python -m pytest tests/test_diarization_error_class.py \
tests/backend/core/test_error_docs_map.py -v
# 20 passed in 0.03s
bun run test src/utils/errorDocsMap.test.ts
# 13 passed (1 test file)
.venv/bin/python -m pytest tests/test_segmentation.py \
tests/test_dub_transcribe.py \
tests/backend/services/test_token_resolver.py \
tests/test_model_manager_preload.py
# 40 passed, 10 xfailed (pre-existing), 1 xpassed
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test: add regression test for diarization error classification (issue #78)
Companion to the fix in
|
||
|
Palash Debnathand4shil
|
1edd35cfd0 |
Per-segment audio effects DSP preset selector (closes #67, rebased from #68) (#109)
* Add per-segment audio effects DSP preset selector to dub pipeline * Add shape assertions to podcast, warm, and bright preset tests * Fix raw preset semantics, add preset validation, update docs, remove duplicate sys.path * Narrow OOM catch to model.generate only in dub_generate * Preserve original OOM exception context in dub_generate * Bind effect_preset to _gen via explicit parameter to avoid loop capture * Catch RuntimeError instead of torch.mps.MPSError for MPS OOM --------- Co-authored-by: 4shil <166588383+4shil@users.noreply.github.com> |
||
|
|
f4e4082ee9 |
Fix NameError: '_gpu_pool' is not defined in get_model() (#90)
The `_gpu_pool` variable is a lazy module attribute that's only accessible via `__getattr__` or the `_get_gpu_pool()` accessor. Line 326 was using `_gpu_pool` directly, causing a NameError when `get_model()` was called. Line 357 already correctly uses `_get_gpu_pool()`. This fix aligns line 326 with the rest of the codebase by using the proper accessor function. Co-authored-by: Nexlabz <contact@nexlabz.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Palash DebnathandClaude Opus 4.7
|
93aa66ab0a |
Phase 3 Plan 03-01: Supertonic-3 engine on SubprocessBackend (#101)
* Phase 3 Plan 03-01: Supertonic-3 engine on SubprocessBackend
Adds Supertonic-3 as a 7th opt-in TTS engine on the Phase 2
SubprocessBackend primitive. Closes TTS-01..06 (REQUIREMENTS.md):
* TTS-01 — _REGISTRY["supertonic3"] resolves to Supertonic3Backend,
a SubprocessBackend subclass.
* TTS-02 — `supertonic==1.3.1` lives under [project.optional-dependencies];
default `uv sync --no-dev` does NOT install it. Exactly one
`onnxruntime` row in `uv pip list` after `--extra supertonic`.
* TTS-03 — Model revision pinned by 40-char commit SHA
(724fb5abbf5502583fb520898d45929e62f02c0b — the "Initial
Supertonic 3 release" SHA, same as the SDK's own pin).
Resolver script for intentional bumps:
scripts/resolve_supertonic3_sha.py.
* TTS-04 — Honest CPU-only reporting. `is_available()` message says
"ready (CPU-only via onnxruntime)" and never mentions
"cuda" or "mps". `gpu_compat = ("cpu",)`.
* TTS-05 — License gate via settings_store helpers
(get/set_license_accepted) + Loopback-only
/api/settings/license endpoint + SupertonicLicenseDialog
frontend modal showing MIT (code) and OpenRAIL-M (model).
Wired into EngineCompatibilityMatrix as an "Accept license"
button on rows whose `reason` mentions "license not
accepted".
* TTS-06 — 3 langs (en/ja/ru) × 3 sec smoke test in
tests/test_supertonic3.py::test_smoke_3langs_3sec
(OMNIVOICE_SMOKE-gated; asserts no onnxruntime-gpu row
post-synthesize).
Package legitimacy gate (Task 1 in plan): supertonic on PyPI verified
to be published by Supertone Inc. (ato@supertone.ai), repo
github.com/supertone-inc/supertonic, wheel is pure-Python with no
postinstall scripts. Same publisher ships supertonic-js on npm under
the same maintainer email.
Test results:
* tests/test_supertonic3.py — 10 passed, 3 skipped (network-gated).
* tests/smoke/ — 4 passed.
* tests/ (full, --ignore=tests/manual) — 412 passed, 0 failed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* ci(tests): uv sync --all-extras so optional-engine tests can import their package
Phase 3 added `supertonic` as an optional dependency. The CI Tests job
runs `uv sync` (no extras), so `test_cpu_only_honest` and `test_license_gate`
in tests/test_supertonic3.py hit the "supertonic package not installed"
fallback instead of the real import path, and fail.
Bare `uv sync` is the right default for users (engines are opt-in), but
the test environment should exercise the full surface. `--all-extras`
keeps the smoke job lean (still bare `uv sync`) while letting Tests
verify the integrated behavior of every optional engine.
Future-proofs against the same failure mode in Phase 4 (GGUF) and any
later optional engines.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Palash DebnathandClaude Opus 4.7
|
84fffa5409 |
Phase 2 Plan 02-04: Engine Compatibility Matrix API + UI (#99)
* Phase 2 Plan 02-04: GET /engines/{id}/health + gpu_compat + HF mask
ENGINE-06 backend half. Adds the data + spawn-on-demand endpoint the new
Engine Compatibility Matrix UI will consume:
* `gpu_compat: tuple[str, ...]` class attribute on `TTSBackend`, overridden
per backend with reasonable defaults (cuda+mps+cpu for OmniVoice/VoxCPM2;
cpu-only for KittenTTS; mps+cpu for MLX-Audio; etc.). `list_backends()`
serializes it as a list.
* `_HF_TOKEN_MASK_RE` (`hf_[A-Za-z0-9]{30,}`) scrubs the `reason` and
`last_error` fields before they leave the registry — Phase 1's
HFTokenRedactor logging filter does not run on FastAPI response bodies,
so this closes T-02-12.
* `GET /engines/{engine_id}/health` — loopback-gated route that resolves
the backend across tts/asr/llm registries, then either calls
`SubprocessBackend.health_check()` (spawn-and-ping) for subprocess
engines or falls back to `is_available()` for in-process engines.
Returns `{ id, ok, message, latency_ms }`. Engine instances are cached
per-class so repeated checks don't leak atexit hooks or spawn extra
sidecars. The masked-redactor is reapplied on the way out.
Test coverage (tests/backend/api/test_engines_route_shape.py, 11 tests):
* Response shape includes the new fields for every TTS entry
* IndexTTS2 isolation_mode == "subprocess", OmniVoice == "in-process"
* Health route round-trips with mocked SubprocessBackend success
* Health route falls back to is_available for in-process backends
* Unknown engine id → 404
* Non-loopback origin → 403
* Engine instance cache reuses the singleton across calls
* HF tokens leaked into is_available() / health_check() are masked
in both the /engines and /engines/{id}/health response bodies
Existing tts_backend_registry shape test updated to include `gpu_compat`.
Full suite: 402 passed, 0 failures (up from 391+ baseline).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Phase 2 Plan 02-04: EngineCompatibilityMatrix UI + Settings wiring
ENGINE-06 frontend half. Mounts a new component on Settings → Engines
that surfaces, end-to-end, the data shape Plan 02-01 + Plan 02-03 added
to the backend registry:
* `frontend/src/components/EngineCompatibilityMatrix.jsx` (270 lines) —
semantic <table> with role=row/cell so RTL queries work; one row per
registered backend. Columns:
- Engine name + install hint + Last error line
- Install state badge (Available / Unavailable + inline reason)
- GPU compat chips (CUDA / MPS / ROCm / CPU with colored variants)
- Isolation mode badge (subprocess for IndexTTS, in-process for the
rest — makes the Phase 2 architectural shift legible to users)
- "Test engine" button → `/engines/{id}/health` round-trip; renders
latency in ms inline next to the button; disabled while inflight;
5 s cooldown to prevent click-storms.
Mount does NOT auto-test any engine — per the plan's Open Question #2,
spawning sidecars is gated on user action.
* `frontend/src/components/EngineCompatibilityMatrix.css` — minimal
styling that reuses chrome tokens; chip colors per GPU target.
* `frontend/src/api/engines.ts` — `getEngineHealth(id)` client function
wraps the new backend route through the shared apiJson helper.
* `frontend/src/api/types.ts` — extends EngineBackend with optional
`isolation_mode`, `last_error`, `install_hint`, `gpu_compat` so the
TypeScript surface tracks the backend wire shape, and adds
EngineHealthResponse.
* `frontend/src/pages/Settings.jsx` — replaces the hand-rolled Engines
table inside EnginesTab with `<EngineCompatibilityMatrix family="tts"
onSelect={...} />`. selectEngine still wires up the picker; the
matrix's onSelect prop renders the Use button per row when provided.
Removes the now-unused FAMILY_META local map.
Test coverage (`frontend/src/test/EngineCompatibilityMatrix.test.jsx`,
8 tests via vitest):
* Renders one row per backend with documented columns
* isolation_mode badge: subprocess for IndexTTS2, in-process for
OmniVoice / KittenTTS
* GPU compat chips: omnivoice → cuda/mps/cpu; kittentts → cpu only
* Unavailable rows render the failure reason inline
* last_error line renders below status when populated; masked HF
token sentinel survives verbatim
* Test engine click fires getEngineHealth(id) and renders latency_ms
* Test button disabled while inflight; second click is a no-op
* Failure path (ok=false) renders a failure marker
Frontend suite: 65 passed (8 new). Lint: 0 new errors. typecheck:ci: clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Phase 2 Plan 02-04: SUMMARY
Recap of Engine Compatibility Matrix delivery — backend route +
gpu_compat metadata + HF-token redaction, frontend EngineCompatibility-
Matrix component, full test counts, deviations, gpu_compat confidence
matrix, frontend test-runner command notes for Phase 6 CI.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Palash DebnathandClaude Opus 4.7
|
c3695e1668 |
Phase 2 Plan 02-03: IndexTTS on SubprocessBackend (closes #42) (#98)
Migrates IndexTTS-2 off the in-process import path and onto the SubprocessBackend primitive shipped in Plan 02-01. Closes issue #42 with a structural fix — the parent's transformers>=5.3 and IndexTTS's transformers<5 now live in separate OS processes and can never collide. * New: backend/engines/indextts/ — sidecar package (__init__.py hosts IndexTTS2Backend, main.py is the sidecar entrypoint, bootstrap.py owns the 3-step venv probe + lazy uv-based bootstrap). * services.tts_backend: IndexTTS2Backend's in-process body removed; registry resolves the class lazily via a _LazyRegistry indirection + PEP 562 __getattr__ re-export. This breaks the import cycle that arose when both subprocess_backend and tts_backend tried to import each other at module load. * docs/engines/indextts.md: install walkthrough + venv resolution order + common errors (linked from is_available()'s unavailable message). * tests: - test_indextts_backward_compat.py (8) — probe priority, no-spawn discipline, HF cache marker preservation (ENGINE-07). - test_indextts_sidecar.py (17) — subclass shape, isolation_mode, parent-side emotion arbitration (vector/audio/text/description), coexist-with-OmniVoice (headline #42 closure), env forwarding. - tests/fixtures/mock_indextts_sidecar.py — stdlib-only sidecar mimicking the production wire protocol; emits 1 s sine wave. - test_issue_fixes.py: two obsolete in-process-conflict tests rewritten to assert the new subprocess contract (no indextts.* import in the parent). Hard constraints honored: backend/services/sonitranslate.py and gpu_sandbox.py are untouched (D1 / D4). Existing v0.2.7 users with OMNIVOICE_INDEXTTS_DIR and a populated HF cache reach a working generation with zero re-download and zero re-install. 44 tests pass across the four exercised files. Full suite: 391 passed, 10 skipped, 13 xfailed, 1 xpassed in 57 s. Smoke: 4 passed. Closes #42. Requirements: ENGINE-02, ENGINE-03, ENGINE-04, ENGINE-07. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Palash DebnathandClaude Opus 4.7
|
0fc5ea6cf3 |
Phase 2 Plan 02-01: SubprocessBackend primitive (Wave 1 of Phase 2) (#97)
* Phase 2 Plan 02-01: SubprocessBackend primitive + echo sidecar + ENGINE-05 wrap
Lands the durable SubprocessBackend primitive — the architectural keystone
that Plans 02-03 (IndexTTS migration), Phase 3 (Supertonic-3), and
Phase 4 (GGUF / Singing) plug into.
Files added:
- backend/services/subprocess_backend.py — base class owning spawn,
shutdown, _send/_recv (length-prefixed JSON), GPU-slot acquire-release,
atexit teardown, stderr drain, op allowlist (T-02-04), and 64 MB
frame cap (T-02-01). No multiprocessing — subprocess.Popen
exclusively so subclasses can target a *different* venv's interpreter
(Locked Decision D4 / Pitfall 1).
- backend/engines/_echo/main.py — permanent CI regression sidecar.
Stdlib-only, runs under the parent's sys.executable. Implements
ready/ping-pong/synthesize/shutdown plus test-only probe_env and
emit_unknown ops for env-forwarding and op-allowlist tests. DO NOT
DELETE — the round-trip test depends on this file.
- tests/backend/services/test_subprocess_backend.py — 13 tests:
round-trip, health_check, no-zombie, shutdown idempotency, env
forwarding (HF_TOKEN/HF_HOME/HF_ENDPOINT/HF_HUB_CACHE), oversize
frame, short read, op-allowlist drop, op-allowlist constant shape,
sidecar-crash recovery, no-multiprocessing grep gate, MAX_FRAME_BYTES.
- tests/backend/services/test_tts_backend_registry.py — 6 tests for
list_backends() resilience + shape + isolation_mode + last_error
caching + existing-engines preservation + install_hint passthrough.
Files modified:
- backend/services/tts_backend.py:
* Adds module-level _LAST_ERRORS dict for ENGINE-06.
* Rewrites list_backends() to wrap each is_available() in try/except
so one broken engine cannot blank the picker (ENGINE-05).
* Adds last_error + isolation_mode keys to each response entry
(ENGINE-06 UI in Plan 02-04 consumes via the same /engines route).
* Uses a duck-typed _is_subprocess_isolated marker rather than
issubclass(cls, SubprocessBackend) because test fixtures (token
resolver suite) purge sys.modules["services"] between tests and the
re-imported SubprocessBackend would be a different class object.
Threat-model mitigations (Plan 02-01 frontmatter):
T-02-01 DoS via length-prefix → MAX_FRAME_BYTES = 64 * 1024 * 1024
T-02-02 GPU slot leak on sidecar death → try/finally in generate
T-02-03 token bytes in stderr → drained via parent logger
(HFTokenRedactor from Phase 1 already on root)
T-02-04 unknown ops from compromised sidecar → PARENT_INBOUND_OPS
allowlist, unknown frames logged and dropped
T-02-05 Tauri group-kill scope → start_new_session=True on Unix /
CREATE_NEW_PROCESS_GROUP on Windows
Verification:
- 337 passed, 6 skipped, 12 xfailed, 1 xpassed (full suite,
`uv run pytest tests/ --ignore=tests/manual`)
- All 19 new tests pass on macOS Apple Silicon
- Smoke tests still pass: `uv run pytest tests/smoke/ -q` → 4 passed
- SoniTranslate untouched (D1 locked decision)
- Zero new Python dependencies
Closes part of ENGINE-01 + ENGINE-05.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(02-01): plan summary — public API, invariants, deviations
Documents the SubprocessBackend public API so Plan 02-03 (IndexTTS) and
Phase 3 (Supertonic-3) authors don't need to re-read the source.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Palash DebnathandClaude Opus 4.7
|
c6e9bbc191 |
Phase 2 Plan 02-02: audio I/O hardening + WAV-export correctness (#96)
* Phase 2 02-02: add _safe_torchaudio_save + _safe_soundfile_write helpers
Centralizes WAV/audio writes through a single audited path that defends
against the four documented torchaudio.save failure modes (CUDA/MPS
tensor, non-contiguous, out-of-range, wrong dtype) AND the torchaudio
2.9+ TorchCodec-delegation behavior drift.
* services/audio_io.py:_safe_torchaudio_save now performs:
- .cpu() move (torchaudio cannot serialize CUDA/MPS)
- dtype coercion to torch.float32
- .clamp(-1.0, 1.0) (out-of-range = silent clipping on some backends)
- .unsqueeze(0) for 1D (mono) inputs
- .contiguous() (torch.cat of slices = non-contig = silent corruption)
- explicit encoding="PCM_S/PCM_F" + bits_per_sample so future
torchaudio backend selection cannot drift the on-disk format
- format passthrough for wav/flac/mp3/ogg with encoding-kwarg fallback
for older codec builds
* services/audio_io.py:_safe_soundfile_write — sibling helper for the
one sf.write call site (dub_core.py). Applies the same dtype/contig/
range checks before delegating to soundfile.write.
* services/audio_io.py:atomic_save_wav (existing P0 helper) now
delegates the actual encode to _safe_torchaudio_save so atomicity
and correctness compose: every byte that lands at the target path
was produced by the audited helper.
* tests/backend/services/test_audio_io.py — 29 tests (25 pass + 4
skipped for MPS dtype incompatibility): parametric round-trip across
dtype x device x contiguity, plus out-of-range clamp, format
passthrough, in-memory buffer, empty-tensor rejection, 1D auto-
unsqueeze, and a smoke check that atomic_save_wav inherits the
safety guarantees.
No new Python dependencies. SoniTranslate untouched (D1 locked).
Refs BUG-01 / #48.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Phase 2 02-02: migrate router audio writes through audited helpers
Migrates all 12 grep-audit bare audio-write call sites in
backend/api/routers/ to route through services.audio_io. Closes the
last surface area of BUG-01 / #48 that the P0 atomic-write commit
(
|
||
|
Palash DebnathandClaude Opus 4.7
|
715766cb04 |
Phase 1 Wave 2: per-OS install docs + Settings UI + error→docs deeplinks (#94)
* docs(install): per-OS install pages + drift validator + CI gate
Splits the 600-line README install section into self-contained per-OS docs
under docs/install/{macos,windows,linux,docker}.md plus a Top-10
troubleshooting index. Each OS doc is end-to-end: a user opens it and
reaches a working app following only commands inside that file.
Adds:
- docs/install/{macos,windows,linux,docker}.md (OS-specific install paths)
- docs/install/troubleshooting.md (top 10 install errors)
- docs/engines/cosyvoice.md (closes #55 docs half)
- docs/features/diarization.md (pyannote license flow)
- docs/setup/huggingface-token.md (3-source cascade guide)
- scripts/validate-install-docs.py (INST-06 docs-drift gate)
- tests/scripts/test_validate_install_docs.py (B-5: validator self-tests)
- .github/workflows/ci.yml step running the validator on every PR
Implements INST-02 (README routing), INST-03 (macOS Gatekeeper anchor),
INST-12 docs half (Windows torch-compile-oom anchor), DOCS-01..05.
The validator is a one-way diff: every `<!-- validate -->`-tagged line
in docs must appear in scripts/desktop-prod.sh after normalisation
(prompt-prefix strip, CRLF, trailing whitespace, blank-and-comment skip).
A `<!-- validate: skip -->` marker opts out for human-readability blocks.
Its own 10 unit tests catch regressions in the gate itself.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(deeplinks): links.py + error_docs_map (Python + TS mirror)
Adds the single source of truth for the project repo URL and the 4-class
error → docs taxonomy that both the in-app ErrorBoundary deeplink button
(Wave 2 Task 3) and the Phase 5 bug reporter will consume.
New:
- backend/core/links.py — PROJECT_REPO_URL + BLOB_MAIN resolver
(Tauri config first, pyproject fallback)
- backend/core/error_docs_map.py — lookup(error_class) → docs URL
- frontend/src/utils/errorDocsMap.ts (TS mirror with classifyError helper)
- tests/backend/core/test_links.py + test_error_docs_map.py
- frontend/src/utils/errorDocsMap.test.ts
Resolves checker B-6 (links.py ownership) and Open Question #3 (which fork
the deeplinks resolve to — the Tauri updater endpoint wins, which points
at the desktop app fork debpalash/OmniVoice-Studio).
The TS BASE constant is documented as the second hardcoded URL drift site;
the keys-sync test (`test_keys_match_python_map` equivalent) guards the
4-class taxonomy contract between Python + TS halves.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(ui): Settings → API Keys panel + ErrorBoundary docs deeplink
Wave 2 AUTH-03 UI half + ErrorBoundary deeplink wiring.
ErrorBoundary fallback now renders an "Open docs for this error" button
that classifies the thrown Error message (heuristic: pkg_resources → 401 /
HfHubHTTP → WebKit / white screen → quarantine / Gatekeeper) and opens the
matching docs anchor via Tauri shell.open (with a window.open fallback
in browser dev mode).
ApiKeysPanel consumes the Wave 1 resolver state endpoint:
- 3 source rows (App / Env var / HF CLI) with set/unset indicator,
masked token preview, whoami username + green check
- "Active" badge on whichever source is currently serving the cascade
- App-row only: Save (POST /api/settings/hf-token) +
Clear (DELETE with optional "also clear HF CLI" confirm dialog)
- "Test now" button refetches state (invalidates the resolver's
validation cache via the same endpoint hit)
Panel mounted in the existing Settings → Credentials tab; the legacy
HF_TOKEN row from CREDENTIAL_FIELDS is filtered out so the two paths
don't fight over the same key.
Threat T-02-02: the panel never displays the full token. The masked
value comes from the resolver state endpoint; the full token only
crosses the IPC boundary on Save (POST) and is cleared from local
state on success.
Closes AUTH-03 fully (Wave 1 backend + this Wave 2 UI).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(perf): INST-12 Disable torch.compile (Windows) toggle (backend + UI)
Wave 2 Task 4 — full INST-12 delivery per checker B-2/B-7 v0.3.0 fat-release
decision. Both the docs half (windows.md anchor, shipped in earlier commit)
and the runtime toggle are now in Phase 1.
Backend:
- backend/services/settings_store.py: adds get_text/set_text helpers for
non-secret config (refuses to write to the encrypted hf_token key).
- backend/api/routers/settings.py: GET + PUT
/api/settings/perf/torch-compile-disabled, both under the existing
loopback guard (threat T-02-04).
- backend/services/engine_env.py: new `build_engine_env()` helper that
centralises HF_TOKEN/YOUR_HF_TOKEN injection from the 3-source resolver
AND injects TORCH_COMPILE_DISABLE=1 when the flag is set on win32.
Phase 2 SubprocessBackend launchers should adopt the same helper.
- backend/services/sonitranslate.py: migrated to engine_env.build_engine_env()
while preserving the source-level `env["HF_TOKEN"]` sentinel that
test_sonitranslate_module_uses_resolver checks.
Frontend:
- frontend/src/components/settings/PerformancePanel.{jsx,css,test.jsx}:
toggle UI with the explainer for #65; renders disabled with a "not
applicable" badge on macOS/Linux.
- frontend/src/pages/Settings.jsx: mounts the panel into the Credentials
tab alongside the API Keys panel.
Tests:
- tests/backend/test_perf_settings.py: 7 backend tests (default state,
PUT persistence, T-02-04 non-loopback rejection, settings_store round-
trip, env injection on win32, NO injection on macOS/Linux, NO injection
when disabled).
- frontend PerformancePanel.test.jsx: 5 tests (renders from GET state,
PUT on toggle, disabled on non-Windows platforms, pre-enabled state).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(planning): Wave 2 SUMMARY + REQUIREMENTS status updates
- .planning/phases/01.../01-02-SUMMARY.md: full implementation report
per template (truths, commits, tests, deviations, drift-site
acknowledgments per W-3, launcher seam name for Phase 2,
taxonomy keys for Phase 5).
- .planning/REQUIREMENTS.md: flips Wave 2 closures to Done:
AUTH-03, INST-02, INST-03 (docs half), INST-06, INST-12,
DOCS-01..05.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Palash DebnathandClaude Opus 4.7
|
c32041289d |
Phase 1 Wave 3: AppImage launcher + .deb ffprobe + Docker LAN + Gatekeeper probe (closes #54, #56, #76, #80) (#93)
* fix(appimage): conditional WEBKIT_DISABLE_COMPOSITING_MODE launcher (#56) WebKitGTK 2.44.x and 2.46.x have a compositing-path regression on Wayland that blanks the AppImage's first paint on Fedora 44 / Ubuntu 24.04. Setting WEBKIT_DISABLE_COMPOSITING_MODE=1 forces the software fallback that works, but blindly setting it on healthy WebKit versions (2.48+) regresses those. This wave adds a conditional AppRun launcher that detects the WebKit version via pkg-config and only sets the env var on the broken ranges (plus a fail-safe when pkg-config is absent or the version is unknown). The launcher is injected into Tauri's AppImage staging dir via a beforeBundleCommand hook — see .planning/decisions/apprun-strategy.md for the spike outcome and rationale (Strategy B chosen). Phase 1 Wave 3 — Plan 01-03 Task 1. Closes #56 frontend half. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(deb): relocate bundled ffprobe out of /usr/bin to avoid conflicts (#76) Prior versions placed the bundled ffprobe at /usr/bin/ffprobe via Tauri's externalBin, which overwrites the system ffprobe on Ubuntu 26.04 and collides with apt-installed media-package ffprobe. Relocate the .deb-bundled ffprobe to /usr/lib/omnivoice-studio/bin/ffprobe via bundle.linux.deb.files, plus defensive maintainer scripts: - preinst: ensure target dir exists for upgrade flows - postinst: remove legacy /usr/bin/ffprobe ONLY when dpkg confirms our package owns it (never touches a user's distro ffprobe) - postrm: clean up the relocated path tree on purge/remove Rust side (tools.rs::resolve_ffprobe) now probes the new path on Linux, and backend spawn (backend.rs) carries both FFPROBE_PATH (legacy alias) and OMNIVOICE_FFPROBE_PATH (canonical) into the backend env. Python side (ffmpeg_utils.resolve_ffprobe) reads OMNIVOICE_FFPROBE_PATH first, falls back to FFPROBE_PATH, then to shutil.which("ffprobe"). 6 new unit tests cover the env-cascade resolution. Phase 1 Wave 3 — Plan 01-03 Task 2. Closes #76. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(frontend): centralised apiBase resolver for Docker LAN access (#80) Docker / LAN browser users hit the preview API at the LAN host's IP, not their local machine — the prior frontend/src/utils/media.js:20 hardcoded http://localhost:3900, which from a LAN client resolved to the client machine itself. Centralise via frontend/src/utils/apiBase.ts: 1. VITE_OMNIVOICE_API override (Docker compose / dev) always wins. 2. Tauri webview → http://localhost:3900 (unchanged behaviour). 3. Plain browser → ${window.location.protocol}//${window.location.hostname}:3900 (follows the page's origin — closes #80). 4. SSR / no-window → http://localhost:3900 (safe fallback). Grep-sweep confirmed media.js:20 was the only hardcode site (Assumption A4 in 01-RESEARCH.md verified). 6 new vitest cases cover the resolver. Phase 1 Wave 3 — Plan 01-03 Task 3. Closes #80 frontend half. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(backend): macOS Gatekeeper quarantine probe + INST-01 guard (#54) Adds backend/core/gatekeeper_detect.py which walks up from sys.executable to find the .app bundle and runs `xattr -l` to check for the quarantine extended attribute (com.apple.quarantine). On detection, the lifespan startup probe logs a structured warning and emits a system_error event through the existing event bus with error_class="GATEKEEPER_QUARANTINE", which Wave 2's React ErrorBoundary turns into a docs deeplink. Detection is informational only — we never auto-run `xattr -cr` (the app itself is quarantined and cannot fix its own state per Anti-Pattern in 01-RESEARCH.md). Users get a clear pointer to the workaround docs. GET /system/quarantine-status exposes the structured payload so the frontend can poll on first load. INST-01 (setuptools>=75.0 pin from PR #62) gains a PR-time guard in tests/backend/test_pyproject.py + a user-observable smoke check in scripts/smoke-test.sh (pkg_resources + whisperx import). 7 gatekeeper tests + 1 pyproject test added — all pass. Phase 1 Wave 3 — Plan 01-03 Task 4. Closes #54 backend half (Wave 2 owns the docs page + ErrorBoundary deeplink wiring). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Palash Debnath
|
4a6b978df9 |
Phase 1 Wave 1: HF token persistence + redactor (closes #35) (#91)
* feat(01-01): encrypted settings store + alembic migration (AUTH-02, T-01-01) Adds the SQLite-backed encrypted settings store that Phase 1 token resolver will read from. Closes the at-rest plaintext risk for HF tokens (T-01-01). - backend/services/settings_store.py: get_hf_token / set_hf_token / clear_hf_token using Fernet symmetric AEAD. Stored value column never contains the literal "hf_" substring. - backend/services/_secret_key.py: per-install Fernet key derived via scrypt(machine-id + 16-byte random salt). machine-id resolution covers macOS (ioreg IOPlatformUUID), Linux (/etc/machine-id and dbus fallback), Windows (HKLM Cryptography MachineGuid via winreg). Final fallback to hostname+user with a warn log. - backend/migrations/versions/0001_phase1_settings_table.py: alembic migration adding `settings(key, value, updated_at)`. Idempotent — checks for an existing table so fresh installs (where _BASE_SCHEMA already created it) and v0.2.7 upgrades both succeed. - backend/core/db.py: _BASE_SCHEMA grows the settings table for fresh installs; init_db() now runs `alembic upgrade head` after the CREATE. - backend/migrations/env.py: honours an externally-set sqlalchemy.url so tests can point alembic at a fixture DB; falls back to core.config DB_PATH for production. - pyproject.toml: cryptography>=41 added explicitly (RESEARCH.md Assumption A1 was checked at execute-time and proved false; the dep was not present transitively, so the install would fail without this). Tests (10 cases, all green): - Round-trip encryption + plaintext-leakage check (T-01-01 invariant) - Salt persistence across clear/set cycles - InvalidToken decrypt path returns None (Open Question #5 resolution) - Concurrent reads consistent under sqlite WAL - Alembic upgrade on a hand-built v0.2.7 fixture DB preserves all existing tables + seeded rows (CLAUDE.md backward-compat constraint) - Alembic downgrade -1 drops only the settings table Refs #35. * feat(01-01): 3-source HF token resolver + log redactor + 5 read sites patched Closes the #35 bug class (bare os.environ.get('HF_TOKEN') reads) by routing every backend HF-token consumer through one resolver, and mitigates T-01-02 (info disclosure via logs) by stripping `hf_[A-Za-z0-9]{30,}` substrings from every log record at the root logger. backend/services/token_resolver.py: - resolve(skip) — 3-source cascade (App → Env → HF-CLI), each source validated via huggingface_hub.whoami(); first valid wins. - on_401(active) — invalidate cache and re-resolve skipping the source that just 401'd (AUTH-06). - state() — three SourceState rows for the Settings UI: set, masked preview (hf_…<last 3>), whoami_user, whoami_ok. - save_app_token / clear_app_token — wraps settings_store + calls huggingface_hub.login(add_to_git_credential=False) per Pitfall #2. - 300-second whoami cache so repeated Settings-page renders don't hit the HF API. backend/core/logging_filter.py: - HFTokenRedactor(logging.Filter) — regex `hf_[A-Za-z0-9]{30,}` so real tokens are masked but `hf_hub` / `hf_token` literals survive. - install_redaction_filter() — idempotent attach to root + every handler. backend/main.py: install the redactor at startup, BEFORE the file handler is added. Re-installed after the file handler attaches so the handler-attached filter list includes it too. Read-side call sites patched (per Pitfall #1 — every HF token read must flow through token_resolver.resolve()): - backend/api/routers/dub_core.py:540 (the original #35 site) - backend/api/routers/system.py:38 (_has_hf_token notification) - backend/services/model_manager.py:480 (diarization pipeline auth) - backend/services/sonitranslate.py:143 (Popen env for SoniTranslate child) - backend/services/sonitranslate.py:217 (gradio_client predict call) New endpoint: - GET /system/hf-token/state — returns the 3-source cascade state with masked tokens for the Wave 2 Settings UI panel. Grep gate confirmed clean: zero `os.environ.get("HF_TOKEN")` reads remain outside token_resolver.py. Tests (17 new cases, all green): - tests/backend/services/test_token_resolver.py: priority cascade, 401 skip mid-resolve, on_401 fallback, state() shape, save+login invariant (add_to_git_credential=False), HUGGING_FACE_HUB_TOKEN alias acceptance. - tests/backend/core/test_logging_filter.py: msg + args redaction, multi-token redaction, non-string args pass-through, short-token literals preserved, install_redaction_filter idempotence. Refs #35. * feat(01-01): Settings hf-token API endpoints + subprocess env injection (AUTH-03/04) Backend half of the Wave 2 Settings → API Keys UI plus the AUTH-04 subprocess env-injection invariant. backend/api/routers/settings.py: - POST /api/settings/hf-token — body {token: str} → save_app_token - DELETE /api/settings/hf-token — also_clear_hf_cli query → clear_app_token - GET /api/settings/hf-token/state — same shape as token_resolver.state() All three are gated by `Depends(require_loopback)` at the router level (threat T-01-03 mitigation; non-loopback Host → 403). backend/main.py: router mounted alongside existing API routers. Subprocess env injection (AUTH-04, threat T-01-04 disposition=accept): - backend/services/sonitranslate.py already updated in Task 2 to read via token_resolver.resolve() and inject HF_TOKEN + YOUR_HF_TOKEN into the SoniTranslate child env block. - backend/services/gpu_sandbox.py: NOT patched — the GPU sandbox runs in-process TTS generation that uses the parent's already-loaded HF state. Adding env injection there is a no-op (parent and child share state via multiprocessing.Pipe before any HF API call). - backend/services/model_manager.py:480 (Task 2): resolves in-process, no subprocess crosses here. - backend/api/routers/exports.py: subprocess.Popen calls only spawn `open` / `explorer` / `xdg-open` — file-manager launchers with no HF needs. Skipped per Task 3 conservative-patching rule. So the canonical AUTH-04 site for this milestone is sonitranslate.py. Future SubprocessBackend work in Phase 2 will inherit the same pattern. Tests (8 new cases, all green): - tests/backend/test_engine_spawn_token.py * POST /hf-token loopback → 200 + state.active == "app" * POST /hf-token non-loopback → 403 ("loopback origin required") * DELETE /hf-token clears settings_store + state.active == None * GET /hf-token/state returns 3 source rows in priority order * GET /hf-token/state non-loopback → 403 * env block contains HF_TOKEN + YOUR_HF_TOKEN when resolver returns one * env block does NOT contain an injected empty HF_TOKEN when resolver returns None * source-level check that backend/services/sonitranslate.py still reads via token_resolver.resolve() (regression guard against silent reverts of the AUTH-04 wiring) Full Wave 1 test suite: 35/35 green. Phase 0 smoke tests still green. Refs #35. * docs(01-01): SUMMARY + STATE update for Phase 1 Wave 1 completion Records execution outcome of the 3-task plan: 10 files created, 9 modified, 35 new test cases, 5 read sites patched, grep gate clean. Documents the two Rule-3/Rule-2 deviations applied (cryptography dep, env.py URL override), the subprocess-launcher inventory for Phase 2, and the known stray edit to the main repo's pyproject.toml that needs a one- line user action to revert. Updates STATE.md current-position table, progress bar, and open TODOs to point at Wave 2 (Plan 01-02) and Wave 3 (Plan 01-03) as the next steps. |
||
|
Palash Debnath
|
651e63b7e9 |
P0 wave-1: security + correctness + Phase 2 foundation (#88)
P0 security + correctness fixes plus Phase 2 foundation work. 7 atomic commits, all CI green (Smoke + Tauri shell on macOS/Win/Linux + Tests). Code commits: - |
||
|
Palash Debnath
|
141546b8a7 |
fix: stabilize dub/diarization UI + production deployment + sonitranslate plumbing (#75)
Production deployment hardening, dub OOM recovery, new SoniTranslate sidecar engine, ASR backend expansion. - Dub generation OOM recovery: backend/api/routers/dub_generate.py:163-209 adds OOM detection + one retry with reduced nstep - New SoniTranslate sidecar engine: backend/api/routers/sonitranslate.py + backend/services/sonitranslate.py (subprocess-based dubbing pipeline, opt-in) - ASR backends expansion: backend/services/asr_backend.py adds NeMo Parakeet TDT, Moonshine, additional Whisper variants; new GET /system/asr-backends endpoint - Dub UI polish: tighter spacing in DubSegmentRow.css, DubTab.css Issue #78 (speaker diarization mis-assignment) NOT addressed by this PR — the bundled diarization changes are in the new SoniTranslate sidecar, not the existing pyannote pipeline. Keeping #78 open. No DB schema changes, no migration. Backward-compatible for existing user data. |
||
|
Palash Debnath
|
6825b8b0a9 |
Cross-platform bug bash + Stories tab + VRAM-aware GPU pool (#51)
First v0.3.x release on the Phase 0 cross-platform CI baseline. ## Cross-platform bug fixes (375ea4e) User-reported bugs from a Pinokio/Windows session: - Docker `compose --profile gpu up` no longer port-conflicts on 3900 — restored `profiles: ["cpu"]` that #49 wrongly reverted on CodeRabbit's advice. - Argos / pip install from the UI now works inside Docker — added `_in_virtualenv()` runtime check; `run_pip` injects `--system` automatically when on system Python. - Speaker diarization warning toast — when pyannote silently falls back to the silence-gap heuristic (missing HF_TOKEN, license not accepted, network blocked), `_diarize()` now returns `(segments, warning)`; `useDubWorkflow` renders an 8-second toast. ## Dub editor UX (d5df454) Six fixes per annotated screenshots: - Editable segment start times (`m:ss.s` or raw seconds; Esc reverts, Enter commits; rejects overlap with end). - Click a transcript row → seek the waveform/video (`WaveformTimeline` now forwardRef's `seekTo(time)`). - Speaker is datalist-backed (pulls from detected speaker clones; free text still allowed). - Scissors menu splits at cursor — uses live caret, then last caret, then sentence-boundary fallback. - Mouse-wheel scrolls the waveform; Cmd/Ctrl left alone for browser pinch-zoom. - Menu popover collision: added `avoidCollisions` + `collisionPadding=8` to Radix Content; removed `position: fixed` from `.ui-menu`. ## VRAM-aware GPU pool (73dbe18) `_gpu_pool` was hardcoded `ThreadPoolExecutor(max_workers=1)` since introduction — every TTS forward serialized through one thread. - CUDA / ROCm: `workers = clamp(1, free_GB // 2.5, 4)`. 16 GB card with ~14 GB free → 4 workers → ~4× throughput on multi-segment dubs. - MPS / CPU / unknown: 1 worker. - `OMNIVOICE_GPU_WORKERS` env var override (clamped 1..16). - Module `__getattr__` preserves the public `_gpu_pool` symbol for existing callers. ## Stories tab — wire-up + UX (f6bbc7a) The 264-line `StoriesEditor` component existed but was mounted nowhere. Now wired into NavRail + lazy-loaded on `mode === 'stories'`. Added Paste & Split panel (sentence-boundary chunking) and per-track `[pause 0.5s]` insertion. ## Stories — pauses + inline voice (edd3a1d) `frontend/src/utils/storyTokens.js` — tokenizer for `[pause X.Ys]` and `[voice:X]…[voice:default]` markers. Voice switches are stateful (carry forward). 13 new vitest cases (vitest now 24/24). ## Verified - 214 backend tests pass (3 skipped, 10 xfailed, 3 xpassed) - 23 router-smoke tests pass - 24/24 vitest cases pass (13 new) - All 7 Phase 0 CI checks green (Tauri shell + Smoke on macOS/Windows/Linux + Tests) 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Palash DebnathandClaude Opus 4.7
|
766e2f7284 |
Phase 0 — Gates: cross-platform CI matrix + regression fixture + release smoke (#71)
* docs: initialize OmniVoice stabilization milestone project * chore: add project config (yolo + balanced) * docs: domain research for stabilization milestone * docs: define v1 requirements for stabilization milestone * docs: add GGUF + singing engine spike requirements (Phase 4 new) * docs: roadmap revision + CLAUDE.md (7 phases, 62 reqs, +GGUF/SING spikes) * docs(phase-0): add Gates phase RESEARCH.md Phase 0 research synthesizes the cross-platform CI matrix, frozen omnivoice_data fixture, installer post-build smoke, SHA-256 checksum publishing, and PR-template extension into copy-paste-ready YAML and Python snippets composed entirely from existing in-repo patterns. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(phase-0): add Gates phase CONTEXT, PATTERNS, and PLAN Phase 0 — Gates is the hard pre-condition for v0.3.x stabilization. Lays cross-platform CI matrix (macos-14/windows-2022/ubuntu-22.04), regression fixture (≤200 KB), installer smoke on tag push, SHA-256 checksums in release body + per-OS SHA256SUMS-*.txt assets, PR template with RC cadence + fixture line, and the open-PR landing for #51. Plan covers GATE-01..06; structured into 7 slices (A–G) with explicit Slice C → Slice G dependency reordering so the new smoke-matrix lands on main before PR #51 (CONTEXT.md L86 interleave decision). Plan-checker iteration 2: APPROVED — all 3 BLOCKERs + 3 MAJORs from iteration 1 resolved (file truncation/Slice-G missing, GATE-06 sibling PR verification, Slice C ordering, Truth #5 wording, macOS Tauri WebView avoidance per Pitfall #5, Windows taskkill per Pitfall #2). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(00-gates): seed regression fixture (GATE-01) - scripts/seed-test-fixture.py — deterministic builder for tests/fixtures/omnivoice_data/ - wipes + rebuilds; fixed created_at=1700000000.0; all-zero PCM for byte-deterministic diffs - calls backend.core.db.init_db() directly (alembic versions/ is empty — see CONTEXT.md) - checkpoints WAL → DELETE on close so no -shm/-wal sidecars pollute git status - exits non-zero if fixture > 200 KB - tests/fixtures/omnivoice_data/{omnivoice.db, README.md} — 8-table empty DB + 1 voice_profiles row - tests/fixtures/omnivoice_data/voices/test-voice/{profile.json, sample.wav} — 1-sec 24 kHz mono silence - .gitignore — explicit allow-list (!tests/fixtures/omnivoice_data/**) so the existing omnivoice_data/, *.db, *.wav patterns don't hide the fixture from git Verifies: du = 144 KB on disk; sqlite_master lists 8 init_db tables + sqlite_sequence; voice_profiles has exactly 1 row id='test-voice'; 0 rows in generation_history. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(00-gates): add tests/smoke/test_boot_smoke.py (GATE-01) - tests/smoke/__init__.py — package marker so pytest treats tests/smoke/ as a module - tests/smoke/test_boot_smoke.py — 4 in-process FastAPI TestClient smoke tests: * test_health_returns_ok — /health returns 200 + {status:ok, device:...} * test_profiles_endpoint_lists_fixture_voice — /profiles surfaces the seeded test-voice row (validates OMNIVOICE_DATA_DIR wiring → DB_PATH → init_db schema) * test_system_info_includes_data_dir — /system/info resolves data_dir * test_history_endpoint_empty — /history reaches DB and returns [] Test isolation env vars (OMNIVOICE_MODEL=test, OMNIVOICE_DISABLE_FILE_LOG=1) set at module top BEFORE any backend import — pattern from tests/test_router_smoke.py. Fixture is copied to a per-session temp dir so the test never mutates the checked-in artifact (SQLite file-change counter + runtime subdirs like dub_jobs/ would otherwise dirty `git status` after every run). Failure mode: if tests/fixtures/omnivoice_data/ is missing, pytest.fail at import time with the regenerate command. - .gitignore — tighten the GATE-01 allow-list to ONLY the seed-produced files (README.md, omnivoice.db, voices/test-voice/profile.json, sample.wav). Prevents future runtime subdirs the backend may create under the fixture from being accidentally committed. Verifies: `uv run pytest tests/smoke/ -q --tb=short` → 4 passed in 1.31 s (target was < 30 s). `git status` clean after a test run. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(triage): record post-planning GitHub state — PR #62, new issues, OOS deferrals - GATE-06: mark #53 + #61 merged (2026-05-16); add #62 (Wave 1 quick wins) to gate set - INST-01: note PR #62 implements setuptools pin (closes #58) - INST-04: note PR #62 lands README docs for #56 workaround - INST-12: new requirement for #65 Windows Triton/torch.compile OOM (filed post-planning) - Out of Scope: defer #67/PR #68 (audio effects), #64 (custom model dir), PR #66 zh-CN (i18n milestone), #63 (empty-template bug) PR #62 is the user's own Wave 1 work landed as a separate PR while GSD planning ran in parallel. Merging it eliminates duplicate work in Phase 1. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci(00-gates): add cross-platform smoke matrix (GATE-02) - New smoke-matrix job on macos-14, windows-2022, ubuntu-22.04 - needs: test, fail-fast: false, timeout-minutes: 10 - Pinned actions: checkout@v4, setup-python@v5, setup-uv@v3 (cache enabled) - Per-OS ffmpeg + libsndfile install (brew/choco/apt via awalsh128 cache) - UV_HTTP_TIMEOUT=120, UV_HTTP_RETRIES=5 for restricted-network resilience - Narrow scope: uv run pytest tests/smoke/ -q --tb=short - Existing `test` and `tauri-cross-platform` jobs untouched Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: add workflow_dispatch to ci.yml so smoke-matrix can run on feature branches * feat(00-gates): add --health-check CLI flag to backend entrypoint (GATE-03) - argparse on __main__ block; --health-check boots uvicorn in a daemon thread and polls http://127.0.0.1:3900/health every 5s for up to 60s. - Prints 'OK — /health responded 200 after Ns' and exits 0 on first 200. - Prints 'FAIL — /health did not respond 200 within 60s' to stderr and exits 1 on timeout. Default invocation behavior unchanged. - No new deps (stdlib argparse/threading/time/urllib.request/sys + uvicorn). - Consumed by per-OS installer-smoke step in .github/workflows/release.yml. Verified locally: exits 0 in 5s against tests/fixtures/omnivoice_data/. * ci(00-gates): add per-OS installer smoke to release.yml (GATE-03) Adds three matrix-leg-specific steps after 'Build + release (Tauri)', each gated by runner.os with timeout-minutes: 5: - macOS (macos-14): hdiutil attach DMG → locate bundled Python backend inside *.app/Contents (NOT the Tauri WebView shell — RESEARCH Pitfall #5: WebView hangs on headless runners) → invoke --health-check → hdiutil detach. Falls back to *.app/Contents/Resources and hard-fails with a directory listing if no backend binary found. - Windows (windows-2022): msiexec /quiet install → find backend.exe under 'C:/Program Files/OmniVoice Studio' → invoke --health-check in background, wait, then taskkill //F //T //PID to cleanup orphaned PyInstaller child processes on port 3900 (RESEARCH Pitfall #2). - Linux (ubuntu-22.04): --appimage-extract (no FUSE on GH runners), locate binary or AppRun, run under xvfb-run -a. Bundle-only regressions (PyInstaller missing-module, Tauri sidecar path mismatch) are invisible to ci.yml's in-process smoke matrix — this step closes that gap before any release is published. Verified: YAML parses; all three steps present; gating + timeout correct; Pitfall #2/#5 mitigations preserved. * ci(00-gates): publish SHA-256 checksums in release body + as asset (GATE-05) - Add 'Compute SHA-256 checksums' step writing SHA256SUMS-<label>.txt per matrix leg using native shasum/sha256sum (Git Bash on Windows). - Add 'Append checksums to release + attach SHA256SUMS file' step using softprops/action-gh-release@v2 with append_body: true so the hashes land in the release body alongside tauri-action's content (not replacing it) and the file is uploaded as a release asset for 'shasum -c SHA256SUMS-<label>.txt' verification. - Both steps gated by 'github.event_name == push && refs/tags/v*' so workflow_dispatch dry-runs do not attempt to attach to a non-existent release (per CONTEXT.md L70 + RESEARCH Pitfall #7 deferral of any aggregate cross-leg SHA256SUMS job). - fail_on_unmatched_files: true to surface path-resolution errors loudly. * docs(00-gates): document RC cadence + regression-fixture check in PR template (GATE-04) * docs(setup): add HF token persistence guide for macOS/Windows/Linux (DOCS-05) Covers two persistent paths: - Method A — canonical ~/.cache/huggingface/token via huggingface-cli login - Method B — shell env var (~/.zshrc / ~/.bashrc / Windows User scope) Documents the v0.2.7 "session only" in-app behavior + notes that Phase 1 AUTH-03 will make in-app pastes write to the canonical file. Bundled with Phase 0 PR per user request. Strictly DOCS-05 scope — zero code changes, no engine touches. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * spec(auth): redesign HF token resolution as 3-source cascade with fallback (AUTH-01..06) Replaces the env_store.py file-based design with a SQLite-backed app store + cascade resolver that checks app → env var → ~/.cache/huggingface/token in priority order, with automatic fallback to next source on HTTP 401. User-explicit design decision: - App-stored token (SQLite settings table, AES-GCM encrypted) wins - Env var ($HF_TOKEN) second - Global huggingface-cli login file third - All three sources visible in Settings → API Keys with "Active" badge - Save action populates BOTH app store AND canonical HF file (defense in depth) New requirement: - AUTH-06 — on 401, auto-retry next source in cascade before erroring Also: traceability count corrected (62 → 74 — undercount at planning + INST-12 + AUTH-06 added post-planning). All 74 v1 reqs mapped. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(auth): backend recognizes HF token from canonical file, not just env var Two call sites were only checking $HF_TOKEN env var, missing the canonical ~/.cache/huggingface/token file written by `huggingface-cli login` (or the app's future Save action): - system.py `/system/info` `has_hf_token` flag — UI showed "No HF token" even when `huggingface-cli login` had populated the file. - model_manager.get_diarization_pipeline — pyannote diarization silently returned None when only the canonical file was set. This is the bug behind issue #35 (speaker diarization setup failure). Both fixes use the same pattern: env var > huggingface_hub.get_token() (which reads the canonical file). Adds a local _has_hf_token() helper to system.py with a comment marking it as prelude to the AUTH-01..06 cascade (Phase 1 token_resolver.py will layer SQLite app-store on top). Closes #35 sub-issue (canonical token invisible to diarization). Cross-cuts AUTH-02 + AUTH-06 design for Phase 1. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(dictation): make pill-widget mode reachable from GUI + scripts (INST-13) The dictation widget infrastructure shipped in PR #40 but was only reachable via the undocumented --pill CLI flag. Adds three discovery paths: 1. Tray menu: "Switch to Dictation Widget" (studio mode) — saves launch_as_widget=true to config, relaunches with --pill, exits current. Mirrors the existing "Open Studio" path in pill-mode tray. 2. Persistent config: AppConfig.launch_as_widget (bool, default false). Read at startup via load_config_pre_app() (uses dirs-next, no AppHandle required). CLI --pill still takes precedence when explicitly passed. 3. Tauri commands: get_launch_as_widget / set_launch_as_widget for the Phase 2 Settings UI to bind a checkbox to. 4. Scripts: bun desktop-prod:pill / desktop-prod:run:pill — forward --pill to the bundled app launch. macOS uses `open -n --args` to spawn fresh instance with the flag. Closes the GUI half of INST-13. Phase 2 closes the Settings UI half. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(dictation): show widget unconditionally on pill-mode launch + visible Suspense fallback Before: pill mode set up correctly but the widget window stayed hidden until ⌘⇧Space was pressed. New users saw absolutely nothing on launch (no main window, no dock icon, hidden widget) and assumed the app failed. If global-shortcut Accessibility permission wasn't granted, they had no path to discover the widget at all. Two changes: 1. lib.rs: in pill_mode_setup, explicitly show + position + focus the widget window after hiding main. With per-call error logging so we can diagnose failures (and a clear error log if widget window wasn't created at all — points at tauri.conf.json regression). 2. main-app.jsx: Suspense fallback was `null`, which combined with widget's transparent+decorations:false config made any lazy-import delay or failure invisible. Now renders a dark pill saying "Loading dictation…" so even if CaptureWidget lazy-import stalls, the user sees the window exists. Studio mode behavior unchanged — widget stays hidden until hotkey or tray click triggers it (existing show() call in the shortcut/ menu handlers is preserved). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(dictation): create widget window programmatically; Tauri 2 silently dropped config-array creation Root cause: declaring the widget window in tauri.conf.json's app.windows[] silently failed in Tauri 2 — get_webview_window("widget") returned None even though the config was syntactically valid. Probable culprit was the transparent + decorations:false + visible:false combo, but Tauri offered no error message either at startup or via webview_windows() enumeration. Diagnosed by adding webview_windows() enumeration logging at setup start (only ["main"] ever appeared) and a programmatic WebviewWindowBuilder fallback that surfaces real Result errors. Fix: - tauri.conf.json: widget entry now has `create: false` to make the config-vs-programmatic handoff explicit. - lib.rs setup(): call WebviewWindowBuilder::new(app, "widget", ...).build() with the exact same surface attributes the config used to declare. - capabilities/default.json: include "widget" in windows array so the new window inherits the same Tauri permissions as main. - tauri.conf.json: remove the invalid `"url": "/?window=widget"` field — WebviewUrl::App takes a path only, query strings aren't supported. Both windows now load index.html. - main-app.jsx: replace URL-query-based widget detection with getCurrentWindow().label === 'widget' via @tauri-apps/api/window. This is the Tauri 2-recommended pattern for multi-window apps and works regardless of URL routing. Closes the immediate UX bug behind the dictation widget being invisible. Builds cleanly + manually verified: pill widget visible on screen at top-center after `bun desktop-prod:pill`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Adam Łukawski
|
ba4cf6a35f |
fix: lazy-load ASR during desktop startup
* fix: lazy-load ASR during desktop startup * fix: handle ASR preload edge cases * fix: unload ASR backend on failure |
||
|
Palash DebnathandClaude Opus 4.7
|
e46b4e3d47 |
feat: import .srt subtitles to bypass Whisper (closes #52)
Closes #52. Users who already have correct, pre-synced subtitles can now skip ASR entirely — they upload a video as normal and then hit "Import .srt" instead of "Upload & Transcribe". The .srt cues populate the dub segment list directly, so the rest of the pipeline (translate, dub, export) just works. Backend - services/srt_parser.py: lenient SubRip parser. Tolerates BOM, CRLF, missing index numbers, dot-vs-comma ms separator, and overlap (shifts the later cue's start to the earlier's end rather than dropping). Skips cues with non-positive duration or empty bodies; reports counts so the UI can warn. - dub_core.py: new POST /dub/import-srt/{job_id} accepts the .srt file, parses it, clamps cues that run past the source media's duration, and replaces job["segments"]. Tries UTF-8 with BOM first, falls back to latin-1 for legacy Windows subs. Frontend - api/dub.ts: dubImportSrt helper with a typed response. - hooks/useDubWorkflow.js: handleDubImportSrt — sets segments, flips dubStep to 'editing', shows a toast with per-bucket counts (imported / skipped / overlap-shifted / clamped) so the user sees what happened. - pages/DubTab.jsx: "Import .srt" button next to "Upload & Transcribe" once a job exists, plus a smaller "Import .srt instead" affordance in the transcription-failure banner — the exact recovery path the reporter asked for. Tests - tests/test_srt_parser.py: 12 cases covering well-formed input, multi-line cues, dot-as-separator, BOM, CRLF, malformed cues, empty bodies, overlap shift, overlap-becomes-zero-drop, missing indices, empty input, and segment shape (sequential ids, speaker filler). pytest is now 226 passed (was 214); vitest unchanged at 11. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Palash DebnathandClaude Opus 4.7
|
a1ef66c321 |
Stability pass: DB leaks, App.jsx hooks refactor, desktop bootstrap (#49)
* fix: eliminate DB connection leaks, race conditions, and deprecated asyncio API ## DB Connection Leaks (P0) - Convert 38 raw get_db() calls to db_conn() context manager across 14 router files - Connections are now guaranteed to close even when exceptions are raised - profiles.py create_profile: clean up orphaned audio file if DB insert fails - profiles.py lock_profile: consolidate 3 separate conn.close() error paths ## Race Condition (P1) - Add _dub_jobs_lock (threading.Lock) to protect _dub_jobs dict in dub_pipeline.py - get_job/put_job now thread-safe for concurrent dub sessions ## asyncio Deprecation (P2) - Replace 23 asyncio.get_event_loop() calls with asyncio.get_running_loop() - Prevents DeprecationWarning on Python 3.12+ and future breakage on 3.14 ## Quick Fixes - gallery.py preview_voice: remove filesystem path from error response (P2) - dub_pipeline.py parse_vtt_segments: remove redundant `import re` inside loop (P3) - gallery.py _init_gallery_db: use db_conn() context manager (P2) * refactor: extract hooks, centralize isTauri, add pytest-cov ## Frontend - Extract useTTS hook (150 LOC) — TTS generation, streaming, audio ingestion - Extract useProfiles hook (219 LOC) — voice profile CRUD, lock/unlock, preview - Centralize isTauri detection: dialog.js, VoiceGallery.jsx, Settings.jsx now import from utils/media.js instead of 4 different detection patterns ## Backend - Add pytest-cov to dev dependencies - Baseline coverage: 39% across backend/ (214 tests pass) - Add .coverage to .gitignore * feat: add Vitest + checkJs, extract useDubWorkflow + useAppData hooks ## Frontend Testing (new) - Set up Vitest with jsdom environment + @testing-library/react - 11 tests: utils (isTauri, formatTime, constants) + Zustand store (mode, text, dubStep, pill) - Scripts: 'test' (vitest run), 'test:watch' (vitest), 'test:legacy' (node runner) ## App.jsx Decomposition (continued) - Extract useDubWorkflow hook (387 LOC) — upload, ingest, transcribe SSE, translate, generate SSE, abort, stop, cleanup - Extract useAppData hook (181 LOC) — data loading, localStorage persistence, WebSocket real-time updates, model-status pill management ## TypeScript checkJs - Enable checkJs: true in tsconfig.json for IDE-level type checking - 947 existing errors (informational, not blocking builds) - noImplicitAny remains false to avoid blocking * ci: add Vitest step, fix useProfiles duplicate state ## CI - Add 'Run Vitest (frontend)' step — runs 11 unit tests - Override --checkJs false in CI typecheck to avoid 947 pre-existing errors - Rename legacy test step for clarity ## Hooks - Fix useProfiles to accept loadProfiles from parent (useAppData) instead of managing its own duplicate profiles array * refactor: wire hooks into App.jsx — 2067 → 1129 LOC (-45%) App.jsx now delegates to extracted hooks instead of inline logic: - useAppData: data loading, localStorage, WebSocket, model pill - useProfiles: voice profile CRUD, lock/unlock, preview - useTTS: generation, streaming, audio ingestion - useDubWorkflow: upload, transcribe SSE, translate, generate SSE 988 lines removed. All handler logic lives in focused, independently testable hooks. Store selectors and render JSX stay in App.jsx as the shell. Verified: vite build clean, 11 frontend + 214 backend tests pass. * feat: show real-time percentage on model loading pill Backend: register hf_progress listener during _load_model_sync() so download/weight-loading tqdm events update _loading_detail with a progress percentage (0-99%). get_model_status() now includes a 'progress' field that the frontend polls. Frontend: useAppData reads msQuery.data.progress and calls setPillProgress() — the FloatingPill already renders the percentage text and progress bar width from this value. * fix: prevent FileNotFoundError in desktop bundle during model init transformers >=4.52 calls _can_set_experts_implementation() and _can_set_attn_implementation() during PreTrainedModel.__init__, which open the class source file via open(class_file). In a Tauri desktop bundle, module.__file__ points to a path that doesn't exist on disk, causing: FileNotFoundError: .../omnivoice/models/omnivoice.py Override both classmethods on OmniVoice to return static values without filesystem access. OmniVoice doesn't use MoE experts (return False), but does support flex/flash attn (return True). * fix: sync source dirs on every bootstrap, not just first run The Tauri bootstrap previously only copied omnivoice/ and backend/ to Application Support on the first run. Subsequent app updates kept using stale source files, preventing bug fixes from landing. Now ensure_venv_ready() always syncs both directories from the bundle resources before returning, even when the venv is healthy. This fixes the FileNotFoundError crash where the old omnivoice.py lacked the _can_set_experts_implementation override. * ui: premium setup wizard polish - Primary button: solid gradient fill with hover glow + lift + press - Stepper nav: connected pills with glow ring on active step - Welcome cards: glassmorphism with stagger-in animations, lucide icons, left-border accent strip, hover translate - Preflight panel: colored icon pill backgrounds, stagger-slide entrance - Step transitions: fade+slide animation via keyed wrapper - Footnote: shortened paths (~/ notation), Reveal in Finder button - Recommendation banner: gradient background with accent glow - Compact spacing throughout for denser, professional layout * fix: kill zombie backend on clean+retry bootstrap When clean_and_retry_bootstrap removes the project dir, any old uvicorn process still running from the deleted paths remains alive on port 3900. The subsequent retry_bootstrap sees the port is healthy and attaches to the zombie instead of re-bootstrapping. Now explicitly kill any process on the backend port after cleaning, before calling retry_bootstrap. * feat: integrate speaker clones into dubbing interface, sanitize system environment variables for subprocesses, and improve FFMPEG binary path resolution. * fix: restore docker compose default + drop dead setSeed call - deploy/docker-compose.yml: remove profiles: ["cpu"] from the default service so `docker compose up` matches the comment on line 5. With the profile present, no service auto-started. - frontend/src/App.jsx: drop the setSeed call in restoreHistory. The selector was never reintroduced after the App.jsx hooks split, and there is no seed state in the store — seeds are generated fresh per call in useTTS and only read from history items for display. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address CodeRabbit review — async detection, dub stream, bootstrap fail-fast - backend/services/tts_backend.py: invert async-context detection in _ensure_loaded. The previous code unconditionally caught its own diagnostic RuntimeError and then called asyncio.run() inside a running loop, masking the intended error message. - frontend/src/hooks/useDubWorkflow.js: require a terminal `done` event before reporting dub success. Without this, a dropped stream after partial progress would flip the UI to `done`, refresh history, and play the completion ping as if generation finished. - frontend/src/hooks/useDubWorkflow.js: restore the previous step when tasksCancel() fails. The UI was getting stuck in `stopping` forever on cancel errors. - frontend/src-tauri/src/bootstrap.rs: fail-fast when source sync fails after the existing directory has already been removed. The previous warn-and-continue path could leave the install with no backend/ or omnivoice/ sources and defer the failure to backend startup with a cryptic error. - backend/api/routers/generation.py: add `from e` to the ValueError → HTTPException re-raise (Ruff B904). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: preserve % suffix in TTS generation timer The 100ms timer in useTTS was rewriting generationTime to a plain elapsed-seconds string, which immediately wiped the "(xx%)" download suffix written on the next iteration of the response-body loop. The real-time percentage was flickering on/off as a result. Read the previous value inside the setter and reattach any existing percent suffix. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Palash Debnath
|
20ade687f6 |
fix: resolve open issues — Discord link, Docker crash, IndexTTS compat, engine tooltips (#47)
* fix: resolve 7 open GitHub issues (#46 #43 #42 #45 #44 #35 #4) #46 — Discord invite expired: - Replace discord.gg/aRRdVj3de7 with discord.gg/bzQavDfVV9 across README, CONTRIBUTING, EnterprisePage, LogsFooter #43 — Docker image crashes with 'No module named core': - Add PYTHONPATH=/app/backend to Dockerfile so bare imports resolve - Add sys.path safety net in backend/main.py (belt-and-suspenders) #42 — IndexTTS not compatible (transformers version conflict): - Catch ImportError + generic Exception in IndexTTS2Backend.is_available() - Return actionable error explaining transformers<5 vs >=5.3 conflict - Update install docs: recommend 'uv pip install -e .' not 'uv sync --all-extras' #45 — Improve pip install tooltips: - Add install_hint field to list_backends() API response - Show hints as tooltips on engine rows in Settings > Engines - Add models-row__hint CSS with hover reveal #44, #35, #4 — Response-only issues (need GitHub comments) * test: add 20 unit tests for issue batch fixes (#46 #43 #42 #45) Coverage: - Discord link sweep: parametrized per-file + repo-wide glob - Docker fix: sys.path insertion in main.py, PYTHONPATH in Dockerfile - IndexTTS: is_available() tuple shape, conflict detection mock, docstring - install_hint: presence, non-empty, registry coverage, backward compat - Regression: minimum engine count, all backends return (bool, str) * fix: address CodeRabbit review — voxcpm package name, bootstrap test isolation - Fix _INSTALL_HINTS: 'pip install voxcpm2' → 'pip install voxcpm' (correct PyPI name) - Replace test_core_config_importable with test_main_py_bootstrap_adds_backend_dir that validates main.py's preamble directly instead of relying on conftest.py - Add test_voxcpm_install_hint_uses_correct_package_name regression guard * fix: align install hints with backend reality (MOSS not on PyPI, VoxCPM supports CPU/MPS) - MOSS-TTS-Nano: not on PyPI, must install from GitHub repo - VoxCPM2: CPU/MPS supported, CUDA recommended (not required) |
||
|
Palash Debnath
|
545b39c912 |
feat: Scalar API docs, community health files, Quickstart cards (#41)
* feat: Scalar API docs, community health files, Quickstart cards, GHCR Docker Backend: - Replace Swagger UI with Scalar at /docs (scalar-fastapi) - Add OpenAI-compatible /v1/audio endpoints (openai_compat router) - Add TTS streaming endpoint (tts_stream router) - Add voice marketplace router (marketplace) - Update TTS backend registry Frontend: - Refine CaptureWidget, WaveformTimeline, App layout - CSS polish and index.css updates Community health: - SECURITY.md — vulnerability reporting policy - CODE_OF_CONDUCT.md — Contributor Covenant v2.1 - .github/FUNDING.yml — GitHub Sponsors - .github/ISSUE_TEMPLATE/ — bug report + feature request - .github/pull_request_template.md — PR checklist README: - Quickstart redesigned as 3-column progressive cards - Docker section updated with GHCR pull instructions - API Docs row added to service table Infra: - scalar-fastapi added to pyproject.toml + uv.lock - research/ added to .gitignore * refactor: clean up documentation and logging while enhancing desktop packaging dependencies and capture UI performance. * fix: address CodeRabbit review — streaming, escaping, thresholds Backend: - marketplace: stream zip entries via ZipFile.open()/copyfileobj, add 100MB upload cap, fix raise-from exception chaining (OOM prevention) - openai_compat: _encode_audio returns actual file ext so Content-Disposition matches real format; forward non-profile voices when DB row not found - tts_stream: send 'start' frame after generation so sample_rate is real; forward non-profile voices on DB miss - capture_ws: split MIN_BUFFER_BYTES into separate partial/final thresholds so short utterances (<2s) still get transcribed Frontend (Tauri): - lib.rs: tray 'dictate' now toggles start/stop based on widget visibility - commands.rs: XML-escape exe path in LaunchAgent plist, shell-quote in .desktop Exec line to prevent injection from special-char paths - CaptureWidget.css: fix Stylelint violations (empty lines, font-family quotes) |
||
|
Palash Debnath
|
a6aa9d79e6 |
feat: add CosyVoice 3 TTS backend, engine platform matrix, CONTRIBUTING.md (#39)
* feat: add CosyVoice 3 TTS backend, engine platform matrix, CONTRIBUTING.md - Add CosyVoiceBackend adapter to tts_backend.py (9 langs + 18 dialects, zero-shot voice cloning, instruct mode, Apache-2.0) - Fix VoxCPM2Backend.is_available() — remove incorrect hard CUDA gate; VoxCPM2 supports MPS (Apple Silicon) and CPU fallback - Add unified TTS Engines table to README with features + platform compat - Update FAQ to reflect current 6-engine Plugin SDK (was 'not yet') - Add CONTRIBUTING.md with dev setup, PR workflow, TTS plugin guide, code style conventions, and testing commands - Move Contributing section above FAQ in README * fix(pr): address CodeRabbit review feedback - README: Update 'Plugin SDK' to 'built-in backend registry' for clarity - tts_backend.py: Preserve full language codes for CosyVoice cross-lingual lookup before fallback |
||
|
Palash Debnath
|
0ecbf136e7 |
refactor: codebase cleanup & root folder reorganization (#38)
refactor: codebase cleanup & root folder reorganization |
||
|
Palash Debnath
|
0c1a3829d5 |
Fix transcription stream drops, IndexError, BrokenPipeError, FK constraint, and Tauri CSP
Fix transcription stream drops and Tauri CSP |
||
|
riyaaa-04
|
81c4b7d1ed | Fix transcription stream drops, IndexError, and Tauri CSP | ||
|
debpalash
|
41c23f6b3a | feat: enhance ASR performance and reliability with binary bundling, model warmup, sub-stage progress tracking, and optimized polling. | ||
|
Palash Debnath
|
8d11e19494 |
feat: dictation maturity + batch TTS pipeline + tests (#32)
Global Hotkey:
- Register ⌘+⇧+Space system-wide via tauri-plugin-global-shortcut
- Shows/focuses window and emits tray-dictate event from any app
Auto-Paste:
- enigo crate simulates ⌘V/Ctrl+V after transcription
- Text auto-pastes into whatever app was active before dictation
Streaming ASR:
- WebSocket endpoint /ws/transcribe for live partial transcription
- 2s buffer interval, configurable via OMNIVOICE_STREAM_INTERVAL
- CaptureButton streams audio chunks, shows italic partial text
- Falls back to HTTP POST if WebSocket unavailable
Batch TTS Pipeline:
- Replace stub worker with full pipeline:
extract → transcribe → translate → generate → mix → export
- Per-job progress tracking (stage, percent, current_lang, segment)
- GoogleTranslator integration via deep_translator
- Download endpoint GET /batch/download/{id}/{lang}
- BatchQueue UI rewritten: progress bars, cancel/delete, downloads
- Type-safe API client (api/batch.ts)
Tests:
- 23 tests for batch endpoints + streaming ASR helpers
- Lightweight fixtures that stub GPU deps
UX (earlier sessions):
- Dual-mode ASR (Turbo MLX + WhisperX Accurate)
- Enhanced download progress (speed, ETA, bytes)
- Status bar black flash fix
- Cold-start model preloading
- Full accessibility audit (ARIA, focus-visible)
- Compact UI layout improvements
- README updated with new features
|
||
|
debpalash
|
2cd1ab4fb9 |
feat: batched TTS, cold start, audiobook editor, context-aware pipeline
Batched TTS:
- Profile-grouped segment processing for cache locality
- CPU/GPU pipelining (ref audio load overlaps TTS inference)
- ~25-40% throughput improvement over sequential loop
- SegmentSpec container + generate_segments_batched() async API
Cold Start Optimization:
- Deferred torch + OmniVoice imports in model_manager.py
- Server starts in ~0.03s (was ~4s) — health/status respond immediately
- _lazy_torch() / _lazy_omnivoice() wrappers with singleton caching
- All downstream refs updated (idle_worker, free_vram, offload, restore)
Stories / Audiobook Editor:
- StoriesEditor component — multi-track with per-character voice assignment
- 7 character slots (Narrator + 6 characters) with color-coded dots
- Inline TTS preview per line via /dub/preview-segment endpoint
- Add/remove/reorder tracks, Generate All workflow
- Character stats footer (lines, characters, est. duration)
Context-Aware Pipeline:
- Video frame extraction via ffmpeg at segment midpoints
- Frame analysis: brightness, mood, complexity via PIL image stats
- Per-segment and global context (VideoContext container)
- get_segment_context() → natural-language TTS instruct hints
e.g. 'Speak with vibrant energy, dark atmosphere, fast-paced scene'
- POST /tools/video-context/{job_id} API endpoint
Roadmap: ALL items completed ✅
|
||
|
debpalash
|
b054249be2 |
feat: plugin SDK, GPU sandbox, waveform v2, accessibility
Plugin SDK: - Abstract TTSPlugin base class with register/discover pattern - Built-in plugins: ElevenLabs (cloud) + Bark (local) - Auto-discovery from backend/plugins/ directory - GET /tools/plugins API for frontend engine picker GPU Crash Sandbox: - Subprocess isolation for GPU-intensive operations - CUDA OOM / driver crash kills worker, not the server - Async wrapper with configurable timeout - Platform availability check Waveform Timeline v2: - Added MinimapPlugin (20px overview bar) - Added TimelinePlugin (time labels) - Keyboard shortcuts: J/K/L (rewind/play/forward), Space - Full ARIA labels on all controls - role=region, role=toolbar for assistive tech Accessibility: - ARIA labels on waveform controls, theme picker, capture button - role=radiogroup on theme dots - aria-checked state on theme selection - Keyboard hint icon (J/K/L) in waveform toolbar LLM Translation: already implemented (OpenAI provider in dub_translate) Roadmap: cleaned up, only batched TTS + vision items remain |
||
|
debpalash
|
e2f576f59e |
feat: MCP server + audio effects chain
MCP Server:
- Full Model Context Protocol server (backend/mcp_server.py)
- 5 tools: generate_speech, list_voices, list_personalities,
list_languages, check_health
- 2 resources: voice://{id}, history://recent
- stdio + SSE transports for Claude Desktop / Cursor / remote agents
- Example config: mcp.json
Audio Effects Chain:
- 6 presets: Broadcast, Cinematic, Podcast, Warm, Bright, Raw
- Configurable pipeline via apply_effects_chain() with pedalboard
- Effects: highpass, lowpass, compressor, reverb, noise_gate, eq, limiter
- GET /tools/effects API for frontend preset picker
- Graceful fallback when pedalboard isn't installed
|
||
|
Palash Debnath
|
34610ca091 |
feat: real-time WebSocket event bus + sidebar reactivity fixes (#27)
## Core Infrastructure - Add backend event bus (core/event_bus.py) — in-memory pub/sub with emit(), subscribe(), unsubscribe() - Add WebSocket endpoint /ws/events (api/routers/events.py) with 25s keepalive pings and auto-cleanup on disconnect - Add frontend hook useRealtimeEvents.js — single WS connection with exponential backoff reconnect (2s→60s) ## Backend Event Integration - projects.py: emit on create/update/delete - profiles.py: emit on create/update/lock/unlock/delete - dub_core.py: emit on clear/delete history - dub_pipeline.py: emit on save_job (every pipeline write) - exports.py: emit on export/record - generation.py: emit on generate/clear/delete - gallery.py: emit on save-as-profile/to-profile ## Frontend Improvements - Replace 45s polling interval with instant WS-based invalidation - Fix critical bug: apiModelStatus was undefined, causing loadAll() to loop forever — sidebar data never loaded on startup - Add websockets to main deps (was optional, got removed by uv sync) - Reduce model/status polling from 5s to 10s, disable background polling for logs - Add ReadinessChecklist and FloatingPill components - Default UI scale changed from S (1.0) to M (1.3) ## Dependencies - Add websockets>=16.0 to main dependencies for uvicorn WS support Closes #3 (native desktop app exists via Tauri) Closes #5 (Dockerfile already uses root bun.lock) Resolves #26 (Triton workaround documented) |
||
|
debpalash
|
393dd7e8b5 | fix: Fix frontend typecheck errors and raise TTS VRAM offload threshold to prevent CUDA OOM | ||
|
debpalash
|
f8b4673e1f | fix(ui): Fix segment row layout collapse, memory bugs, enterprise page, and UI enhancements | ||
|
debpalash
|
fc76e79ff8 |
feat: setup wizard, donate page, CI fixes, performance optimizations, and style extraction
- Implement donate page and migrate API fetching to react-query hooks - Add setup wizard for batch job management and voice clip editing - Refactor setup router into package (wizard, models, download sub-modules) - Fix 9 CI test failures from setup router refactor - Fix cross-device link error in prefs.py atomic writes - Fix event loop mismatch in export test fixtures - Modernize README with architecture diagram and 13 app screenshots - Defer per-segment disk writes in dub_generate for ~6s faster dubs - Extract 45 inline styles from Launchpad, KeyboardCheatsheet, DubSegmentRow - Add playwright dev dep and screenshot capture script |
||
|
debpalash
|
901eb040a8 | feat: add live bootstrap progress bars and log inspection to splash screen | ||
|
debpalash
|
4a8b06c25e | feat: implement structured progress tracking for model downloads and add local environment variable loading support. | ||
|
debpalashandClaude Opus 4.7
|
994c6cf065 |
feat(backend): setup wizard router, translation engines, export options, client-disconnect handling
- Add setup router (backend/api/routers/setup.py) for first-run wizard: system checks, engine probes, model downloads with progress - Add translation engines service with pluggable backends - Add utils/hf_progress for HuggingFace download progress streaming - Add PyInstaller runtime hooks (numpy compat, torch compiler disable) - Global exception handler short-circuits h11 LocalProtocolError and Starlette ClientDisconnect with HTTP 499 to silence noisy stack traces when users scrub or cancel video mid-stream - /dub/download-mp3 accepts bitrate query param (clamped 64–320kbps) - Refactor ASR/TTS backends, dub pipeline, engine management - Update backend.spec for PyInstaller packaging - Bump pyproject version to 0.2.0; refresh uv.lock Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
debpalash
|
52d68d05dc | refactor: update backend architecture, expand frontend state management, and synchronize voice-pro research modules. | ||
|
debpalash
|
6e89db0f77 | feat: add YouTube/URL ingestion support using yt-dlp and update UI with granular preparation progress tracking | ||
|
debpalashandClaude Opus 4.7
|
67328d04fe |
refactor: split backend into api/core/services/schemas, harden security + fd pressure, add searchable language picker, fix segment fragmentation
Backend:
- Split monolithic main.py into backend/{api/routers,core,schemas,services}
- core/db.py: allowlist-gated migrations, db_conn context manager (kills SQL injection on ALTER)
- core/tasks.py: lock-guarded listener add/remove/push, snapshot-before-iterate
- services/ffmpeg_utils.py: run_ffmpeg helper with concurrency semaphore, EAGAIN retry, guaranteed reap
- services/segmentation.py: Bengali/CJK/Arabic punctuation, ultra-short tier, stitch_adjacent_shorts,
bounded-loop merge; public clean_up_segments API
- services/model_manager.py: robust lock.locked() handling
- api/routers/dub_core.py: job_id traversal guard, thread-safe _active_procs, timeouts on ffmpeg/demucs,
POST /dub/cleanup-segments endpoint
- api/routers/dub_export.py: guarded SSE listener remove, ffmpeg timeouts via run_ffmpeg
- api/routers/exports.py: destination_path validation, safe source resolver, subprocess list-form
- api/routers/generation.py: contextlib.suppress on tempfile cleanup, db_conn usage, safe output-path helper
- api/routers/system.py: try/finally tmp cleanup, subprocess timeouts
- schemas/requests.py: TranslateSegment.id int->str to match hex segment IDs
- main.py: threading.Lock around crash log writes
Frontend:
- components/SearchableSelect.jsx: popover combobox with search, keyboard nav, popular+recent pins, 200-item cap
- App.jsx: wire SearchableSelect for dub language / ISO code / voice-gen language; Clean Up segments button;
fix blob URL leak (object-shaped prev in setter, unmount cleanup via ref)
- components/WaveformTimeline.jsx: explicit <video> detach instead of innerHTML='' to release decoder
- index.css: ss-* combobox styles matching Gruvbox theme
Tests:
- tests/test_segmentation.py (26 cases), test_dub_transcribe.py, test_dub_export_unique.py, conftest.py
Chore:
- .gitignore: exclude omnivoice.zip, /research/ reference clones
- Remove tracked stray root test scripts + crash_log.txt
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|