d91beef0fd314250d8d9b94de86dfea019a8bd96
22
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 debpalash
|
047e7d901e |
Merge remote-tracking branch 'origin/main' into fix/ghas-log-safety
# Conflicts: # CHANGELOG.md # backend/api/routers/dub_export.py # backend/api/routers/marketplace.py # backend/services/ffmpeg_utils.py # backend/services/sonitranslate.py |
||
|
debpalash
|
aeda504a03 | fix(security): authorize export filesystem sinks | ||
|
debpalash
|
854a637b83 | fix(security): make untrusted log values single-line | ||
|
Palash Debnath
|
5cab8e0149 |
feat: rename the product to VoiceStudio (previously OmniVoice-Studio)
Renames what users see. The app, the installers, the window title, the
docs and all 21 locales now say VoiceStudio, with "(previously
OmniVoice-Studio)" noted near the title of each doc surface so people
recognise it.
Deliberately NOT renamed, because renaming any of them silently breaks
an existing install — there is no legacy-path fallback anywhere in this
codebase:
- bundle identifier com.debpalash.omnivoice-studio (MSI UpgradeCode,
macOS TCC grants, managed venv, WebView localStorage, the
single-instance lock)
- data directories OmniVoice / .omnivoice and omnivoice.db
- the ~150 OMNIVOICE_* environment variables
- the X-OmniVoice-* HTTP headers (a wire protocol)
- the published Docker image paths
- the OmniVoice ENGINE, which is a model name and not this product
tests/test_identity_paths_survive_the_rename.py pins every one of those
so a future well-meaning sweep cannot orphan a user's library.
Linux .deb users install a new package name and should apt remove
omnivoice-studio; that note is in the changelog.
|
||
|
Palash Debnath
|
73ebd6518d |
fix(errors): six failures that reached users as raw text (#1262, #1256, #1251, #1247, #1257, #1254) (#1264)
* fix(errors): four failures that reached users as raw OS text (#1262, #1256, #1251, #1252) #1262 — a voice profile named in any non-latin-1 script 500'd every download endpoint with "'latin-1' codec can't encode characters in position 22-25". `attachment; filename="` is exactly 22 characters, so those were the first four characters of the user's own name. The sanitisers in front of the header filtered with str.isalnum(), which is True for every alphabetic script — they stripped punctuation and passed exactly what breaks the header. Ten sites, one RFC 6266 builder, plus a guard so an eleventh can't be hand-written. #1256 — a synth died on FileNotFoundError: 'ffprobe' and was reported as "an error OmniVoice doesn't recognize", on a Mac where the app's own ffprobe was resolvable the whole time. Our call sites pass explicit paths; a dependency shelling out by bare name does not. The resolved directories are now published on PATH, and the failure is classified either way. #1251 — "The paging file is too small" reached the user as a bare 500. It was already counted as an OOM, but that remedy (close apps, lighter engine) is wrong on a 32 GB machine — the fix is a Windows setting, and the hint now says which. Matched on the code in both the Python and Rust spellings. #1252/#1253 — deleting a dub mid-import crashed it with `ingest: 'mgw39lx3'`: str(KeyError) is the repr of the key. The pipeline blind-subscripted a job that DELETE /dub/history/{id} had popped minutes earlier. It now stops quietly, and no exception whose str() is a bare value can present itself that way again. * fix(engines): Unload 400'd, a wrong language said nothing, DRM was retried by hand (#1247, #1257, #1254) #1247 — list_loaded() advertises in-process engines as `engine:<id>` with "unloadable": true, but unload() only ever handled tts/diarization/sidecars. The panel was rendering a button for ids the dispatcher rejected. The engines already implement unload(); only the routing was missing. The contract test written for it immediately found a second instance — `capture-asr`, listed the same way with no branch either — which is why it enumerates the listing rather than hard-coding ids. #1257 — MLXAudioBackend.supported_languages() returns ["multi"] on the stated assumption that "each engine silently ignores languages it doesn't know". It doesn't; the library raises. So the picker offers all 646 languages and the rejection arrived as a bare list of 23 codes, naming neither the engine nor the way out. Enumerating each model's real language set would be a brittle map that goes stale every engine update — name the engine and the fix instead. #1254 — reported as intermittent: the same URL failed as DRM-protected, then succeeded on retry. Real DRM doesn't lapse; the player client varies. That is the same shape as the 403 case which already escalates through _YT_PLAYER_CLIENTS, so DRM now routes into it. If every client still refuses, the failure is classified instead of arriving as a raw yt-dlp line. * fix(review): close the delete race, narrow the tool match, sanitize the fallback Greptile P1 + CodeRabbit Major — verified real, and mine: splitting merge from save left a window where a delete lands between them, so the pending save UPSERTs the row straight back and a dub the user deleted reappears. Now one atomic step under _dub_jobs_lock, with both delete endpoints purging rows and memory under that same lock. That also fixed DELETE /dub/history, which deleted every row but evicted nothing — an in-flight job survived 'clear history' outright and re-saved itself on completion. CodeRabbit Minor (#1256): the media-tool match accepted any message ending in 'ffmpeg'/'ffprobe', so a missing FILE at /tmp/ffmpeg got the 'repair your media engine' remedy. Now requires the name unquoted-and-unqualified. CodeRabbit Major (#1262): `fallback` reached the header verbatim whenever the real name folded away entirely, walking past every guard the name goes through. Folded like the name. CodeRabbit Major (#1256): the PATH log printed resolved directories, and a user-set FFMPEG_PATH sits under their home. Logs a count now. CodeRabbit Minor (#1262): the subtitle-route assertion also passed against the pre-fix header; it now asserts filename*= too. Skipped: 'Highlights bullets must end with (#N)'. CLAUDE.md scopes that to the ### subsections; none of the seven pre-existing highlights carry refs, and tests/test_changelog_style.py encodes the rule already. * fix(review): the remaining unlocked save paths, an over-broad signature, two weak tests Greptile P1 — the mid-pipeline put_job + save_job pairs were still unlocked, so a clear-history landing between them left a ghost row behind the purge. Both go through put_and_save_job now; only the final completion gate decides whether a withdrawn job's work is kept. CodeRabbit Major (#1257) — 'unsupported language' as a bare prefix also matches 'Unsupported language model configuration', handing a model/config failure engine-switch advice it has no use for. The loose wordings now require the rejected thing to be a code or to end there. CodeRabbit Major (#1257) — the OOM test asserted on SOURCE TEXT, which passes even if the call is unreachable or its result discarded; #1224 taught this same lesson on this codebase. Both it and the language rewrite now drive the real _run_backend_inference with a raising backend. CodeRabbit Minor (#1257) — 'or "engine" in message' always passed, since the production template contains the word. Asserts the resolved class name now. CodeRabbit Major (#1252) — the delete-race test deleted the job BEFORE the merge, which only re-tested the absent case and would pass with the two steps still split. It now interleaves a real second thread against a slow save. CodeRabbit Major (#1256) — a hardcoded /tmp literal trips Ruff S108; built from tmp_path instead. * fix(review): a withdrawal must survive the job's first write CodeRabbit Major — the concern is real, though its suggested fix (gate the checkpoint on the job already existing) would break creation: an ingest's FIRST persistence is what creates the entry, so that gate would never pass. The actual defect is that dict membership cannot express 'withdrawn'. An absent key means 'not written yet' for a new job and 'deleted' for an established one — two opposite instructions from one signal. So a clear-history arriving before the first checkpoint was silently undone by that checkpoint recreating the row, and the run then persisted its result into history the user had just cleared. Tombstone it explicitly: the ingest declares itself in flight, a purge marks any in-flight id withdrawn, and both write paths refuse a withdrawn id. Released in , so it's bounded by concurrent ingests and can't poison a later run that reuses the id. That also fixed clear-history properly: a job with no row yet appears in no id list, so only an in-flight sweep can catch it. CodeRabbit Minor — my race test waited on an event that could not be set while the save held the lock, so it burned its full 2s timeout every run and synchronised nothing. It now waits for the purge thread to REACH the purge. * test(dub): the race test was not testing the race Caught by verifying fail-before rather than trusting the test: splitting merge from save — the exact resurrection bug — passed all 22 tests. The assertions checked WHAT happened (the save ran, the row was deleted, the job left memory) but never WHEN. A save landing after the delete is indistinguishable from one landing before if you only assert that both occurred — and 'after' is precisely the resurrection. Now recorded and asserted as an order. With merge+save atomic the purge cannot start until the save finishes, so the sequence is always save-then-delete; split them and it fails with ['delete', 'save']. That is the second time this test needed rewriting: v1 deleted the job before the merge and only re-checked the absent case, v2 interleaved a real thread but asserted the wrong thing. Both looked like tests. Also documents why the DB write sits inside the lock (atomicity beats a rare 5 s sqlite busy-timeout stall) and that no locked region calls another, so the non-reentrant lock cannot deadlock — verified by walking every locked region. * fix(dub): gate the withdrawal at save_job, not at its callers Greptile P1 — and the same class I'd already fixed, unfixed elsewhere. The withdrawal check sat in the two ingest helpers, but eight direct save_job call sites across dub generate / translate / export / core bypass those entirely. Deleting a dub mid-RENDER therefore still resurrected it, which is at least as likely as deleting mid-import. Moved the gate into save_job itself: one choke point, every caller inherits it, and the ninth cannot forget. That needs a re-entrant lock, since the atomic helpers call save_job while already holding it — a plain Lock would deadlock the backend, so a test pins the lock type and another exercises the nested path. Verified fail-before: removing the gate fails the new test. * fix(dub): the withdrawal only covered ingests, so it covered almost nothing Caught by testing the reported scenario directly instead of trusting a green suite: CI passed, 26 tests passed, and a dub deleted during a RENDER was still resurrected. The tombstone was scoped to in-flight ingests. But a dub is imported once and rendered many times, so the realistic delete lands during a render — long after its ingest ended — and end_ingest was CLEARING the tombstone at exactly that point. The rare case was protected and the common one left open. Now scoped to deletions, not ingests. Kept in a bounded LRU rather than cleared on completion, because there is no moment at which a delete stops mattering: any operation still holding that job can persist it. Re-importing an id is the only thing that legitimately revives it. Verified fail-before: the previous scoping fails three of the new tests. * refactor: move the dub delete-resurrection fix to its own PR (#1270) The six fixes left here are independent error-message changes that needed no corrections. The dub concurrency change needed five rounds, each finding something real in work that was already reviewed, tested and CI-green — the last of them being that the fix did not fix the reported case at all. Riding a release on that record is a bad trade, so it ships separately as #1270. This branch keeps #1262, #1256, #1251, #1247, #1257 and #1254; the KeyError message half goes with the dub PR, since it is that issue's other half. |
||
|
debpalashandClaude Fable 5
|
933743e336 |
fix: resolve open issue batch — #1172 #1173 #1174 #1185 #1186 #1188
- validate managed binaries before exec; 0-byte GGUF placeholders fail actionably instead of "Exec format error" (#1172) - KittenTTS: tokenizer-measured chunking to the ONNX 512-token cap; clear 400 for unspeakable input (#1173) - clean SIGTERM during weight load: shutdown-aware loader, benign cancelled-load classification, lifespan hardening, scoped log silencers (transformers load + alembic fileConfig) (#1174) - broken ASR deep-imports (lightning_fabric) mark the engine unavailable with a repair hint and fall through (#1185) - uv cache + managed Python follow the chosen install drive on Windows (cherry-picked cross-drive class fix + spaces/D: tests) (#1186) - adaptive silence-removal ladder for quiet clone references; localized actionable error for truly silent clips, all 21 locales (#1188) - CHANGELOG: consolidated Unreleased into the quiet one-liner style Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
debpalashandClaude Opus 4.8
|
9654677faa |
fix(dub): stop demucs crashing under the Windows SelectorEventLoop fallback
Dubbing's vocal-separation step died with `TypeError: An asyncio.Future,
a coroutine or an awaitable is required` whenever the backend ran on an
event loop without native async-subprocess support — notably the Windows
`SelectorEventLoop` that uvicorn forces under `--reload` (`bun desktop`).
On that loop `spawn_subprocess` returns a thread-backed `_AsyncCompatProc`
whose `.stderr` is a plain SYNC pipe, but `run_proc_streaming_stderr`
unconditionally did `await asyncio.wait_for(p.stderr.read(256), ...)`. A
sync `.read()` returns bytes, not a coroutine, so `wait_for` raised. The
extract step survived only because it uses `communicate()` (async-wrapped).
The wrapper now advertises `uses_sync_pipes`; the streaming reader keys off
it and, on that loop, runs the process to completion via the wrapper's async
`communicate()` and replays stderr as the same `('stderr', line)` events —
no live progress bar on the degraded loop, but demucs actually runs. The
native async path (Proactor/posix — every release build, macOS, Linux) is
untouched. Regression test covers the fallback (fails before, passes after)
and pins the native path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
mergetestandClaude Fable 5
|
155cebe911 |
fix(dub): diagnose export failures honestly; dodge the Windows 32K argv limit (#1152)
A Windows dub export died at ffmpeg *spawn* with [WinError 206] — the mux argv scales with tracks/segments and exceeded CreateProcess's 32,767- char limit — but the catch-all told the user their (8-char) filename was too long AND to check whether ffmpeg was installed. - explain_ffmpeg_failure() maps the three real failure modes to their own advice: argv-too-long (names the limit + the actual size, suggests fewer languages per export), ffmpeg-unlaunchable (the only case that suggests checking the install / FFMPEG_PATH), and ffmpeg-ran-and-failed (surfaces ffmpeg's stderr, no install advice). All three dub_export catch sites (video mux, audio export, MP3 encode) now use it. - run_ffmpeg() on Windows moves an oversized -filter_complex graph (the dominant argv consumer — the bed-mix/apad branches grow per track) into a -filter_complex_script temp file, so the spawn never hits the limit in the first place; the script file is removed after the run. Regression tests: tests/test_ffmpeg_failure_diagnosis.py. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
dc3527ab05 |
fix(dub): stereo, full-band music bed — separate the HQ extraction, pin the mix to stereo (#1138)
* fix(dub): stereo, full-band music bed — separate the HQ extraction, pin the mix to stereo Owner asked for a channels/Hz/samples comparison of a dub against its original to tune generation toward the source. The measurements found a class, not a knob: L/R correlation: original 0.754, dub 1.000 (mono in a stereo container) stereo width (S/M): 0.375 vs 0.003 LUFS: -17.8 vs -17.2 (already fine) Two stacked causes: 1. INGEST: Demucs separated audio.wav — the 16 kHz MONO extraction made for ASR. The music bed therefore inherited mono AND an 8 kHz bandwidth ceiling at its source (Demucs upsamples to 44.1 kHz internally, so the stems LOOKED like 44.1k stereo files while carrying neither). Ingest now extracts a second full-quality file (44.1 kHz stereo, pcm_s16le) just for separation; ASR keeps its 16 kHz mono file; Demucs cost is ~unchanged (it resampled to 44.1 kHz internally either way). Best-effort: if the HQ extraction fails, separation falls back to the ASR file — exactly the old behavior. The stem-move path follows the input's basename. 2. MIX: amix negotiates ONE channel layout across inputs, and the synthesized voice is mono — so even a true-stereo bed was collapsed at the mix. bed_mix_filter now pins BOTH legs to stereo (aformat=channel_layouts=stereo); upmixing the mono voice duplicates it dead-center, which is where dubbed dialogue belongs anyway. Verified with real ffmpeg: the new graph preserves a stereo bed's width through the mix (and the ingest test pins that demucs receives audio_hq.wav with -ac 2 -ar 44100 while ASR keeps -ac 1 -ar 16000). Both tests fail with their half of the fix reverted. Full suite: 2989 passed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(dub): pre-HQ stem caches are not reused (review) Greptile P1, real: the content-hash cache restores a previous job's stems for the same video and skips Demucs — so every video processed BEFORE the HQ-extraction change would keep its 16 kHz-mono-derived bed forever, and the fix would never apply to exactly the videos users re-upload to hear the difference. find_cached_job now requires the audio_hq.wav marker in the cached job dir; older candidates are skipped with a log line and separation reruns once at full quality. Regression test covers both directions of the gate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: mergetest <nizam4103@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
a7efaa24f7 |
fix(dub): background bed no longer plays quiet and muffled — cancel amix normalization, mix at 48 kHz (#1136)
* fix(dub): background bed no longer plays quiet and muffled — cancel amix normalization, mix at 48 kHz Reported live: "background music is so much not like the original." Two stacked fidelity bugs in every bed-mix site, measured with real ffmpeg on a real dub job: 1. LEVEL — ffmpeg's amix NORMALIZES its inputs, so the per-site weight strings meant "favor dialogue slightly" but actually played the music bed at ~57% of its original level (batch.py stacked an explicit volume=0.15 under the same normalization, leaving its bed near 8%). 2. BANDWIDTH — the voice track is synthesized at 24 kHz and amix negotiates one common rate, so the 44.1 kHz bed was silently downsampled to 24 kHz: everything above 12 kHz (cymbals, air, brightness) vanished. Six call sites carried six hand-rolled variants of the same filter string (dub_export x5, batch x1) with inconsistent input ordering — the same copy-divergence pattern that orphaned the clone-prompt cache (#1130). They now share one builder, services.ffmpeg_utils.bed_mix_filter(): both inputs resampled to 48 kHz before the mix, a compensating volume multiply that cancels amix's normalization exactly (the weights ARE the absolute gains: bed 0.9, voice 1.1), and a transparent peak limiter for the rare summed peak that full-scale mixing makes possible. Measured A/B on the reporting user's job (bed vs bed-through-mix, silent voice): 57% -> 90% of original level, 24 kHz -> 48 kHz output. The remaining -0.9 dB is deliberate dialogue headroom, one constant to change if policy shifts. Tests: the export command must carry the resample + compensation + limiter (fails on the old strings), builder label-uniqueness for multi-track graphs, and the existing export suites unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(dub): amix renormalizes when a stream ends — disable normalization instead of compensating for it Greptile P1 on this PR, confirmed real by measurement: amix's normalization is DYNAMIC — it rescales the remaining inputs whenever one ends. The previous commit cancelled it with a constant post-mix multiply, which is exact only while both streams are active; once the (even marginally shorter) voice track ends, the bed's internal scale jumps to 1.0 and the fixed multiply BOOSTS the tail music into the limiter. Measured on the real job with a deliberately short voice: bed at 90% while the voice runs, 189% after it ends. The original A/B used equal-length streams, which is why this never showed. Fix: amix normalize=0 (a plain sum) with per-input volume gains — levels are exact for the whole timeline regardless of stream lifetimes. Same measurement now: 90% / 90%. normalize= arrived in ffmpeg 5.x, and system-ffmpeg users can be older, where an unknown option rejects the whole graph (= no export at all). The builder probes `ffmpeg -h filter=amix` once per process and falls back to the compensated form on legacy builds — its tail quirk is the lesser evil next to a failed export, and every bundled/imageio tier ships 7.x. Tests: both paths pinned (normalize=0 + per-input gains on modern; the compensation multiply on legacy), probe monkeypatched per test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(dub): anchor the amix monkeypatches to the call chain — module aliases miss under random order Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: mergetest <nizam4103@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
5562aa16a7 |
feat(setup): media tools become invisible — bundled by default, controllable in Settings (#1071)
Most users should never learn what ffmpeg is. The Setup Wizard's SYSTEM
PREFLIGHT stops listing FFmpeg / FFprobe / yt-dlp as user-installed
requirements ("brew install ffmpeg…"): they are internal dependencies the
app provisions for itself. Genuine user facts (OS, RAM, disk, GPU,
network, Python) are untouched.
Backend
- New services/media_tools.py: per-tool status {version, path, origin:
sidecar|bundled|system|custom}; background acquisition of a pinned,
SHA-256-verified static ffmpeg+ffprobe build (immutable-commit fetch
from the same upstream the static-ffmpeg pip package uses — that
package itself was audited and rejected: mutable raw/main URL, no
checksums, writes into site-packages); binaries are `-version`-probed
via the existing _binary_runs before being trusted, installed under
DATA_DIR (update-surviving, frozen-build-safe), zero new Python deps.
- ffmpeg_utils resolution chain gains the acquired-bundled tier — and
ffprobe finally has a bundled tier at all (imageio-ffmpeg ships none),
closing the source-install gap.
- New /media-tools router (loopback-gated, same contract as
/system/set-env): status, acquire, {tool}/custom-path | use-system |
restore, ytdlp/update | restore. Overrides persist via the existing
env.FFMPEG_PATH / env.FFPROBE_PATH prefs convention — one store, no
competing controls.
- yt-dlp updates: audited in-venv pip/uv upgrade and rejected (venv is
uv-managed with no pip; yt-dlp is a locked dep, so the updater's
--inexact drift sync would revert it). Instead the newest wheel —
verified against PyPI's own sha256 — lands in a DATA_DIR overlay
prepended to sys.path at startup: survives app updates, works in
frozen builds, and "Restore tested version" is just deleting the
overlay. Gallery now runs yt-dlp via `python -m yt_dlp` (module, not
PATH) so the CLI can never be a user-install task either.
- /setup/preflight drops the three tool rows, carries a media_tools
verdict, and self-heals: kicks the bundled download in the background
when no tier resolves (never re-fires after a failure — the wizard's
card owns Retry). diagnose + the ffmpeg-missing notification now point
at Settings → Audio tools instead of package managers.
Frontend
- Wizard: new MediaEngineCard — renders NOTHING when the engine is ready,
a one-line progress while acquiring, and only on failure an actionable
card (Retry / Use a system copy / Choose file…).
- Settings → Audio tools (new category, System group): FFmpeg + FFprobe
rows with version, path, origin badge, Use system copy / Choose file… /
Restore bundled, header-level "Update bundled build"; yt-dlp row with
Update + Restore tested version (+ restart affordance). Package-manager
commands appear only as copyable prose, never executed.
- The FFmpeg-path override moved out of Settings → Network (pointer row
deep-links to Audio tools; no second writer of env.FFMPEG_PATH).
Notifications gain a settings-tab action type.
- All strings i18n (en + defaultValue), a11y labels on every control.
Tests: 29 new backend (origin classification, checksum/size/probe
rejection, override persistence, overlay update/restore, router gating +
route-shadowing) + preflight contract tests (tool rows gone, verdict
present, auto-acquire fires once); 14 new frontend (wizard hide/progress/
failure-card, Audio tools rows/badges/actions). Route snapshot
regenerated. Docs (macos/linux install, troubleshooting §7b) describe the
new reality in the same commit.
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
6140f888e1 |
fix(dub+win): dialect↔cinematic guidance loop + WinError 193 ffmpeg validation (#377)
* fix(dub): break the dialect↔cinematic guidance loop (#372, #373) - Cinematic toggle refuses the pick when no LLM endpoint is configured, pointing at Settings → Credentials → LLM endpoint - backend Fast fallback now syncs the quality toggle to 'fast' - the dialect warning no longer fires alongside the cinematic-no-LLM warning (the pair formed the loop), and both messages point at the LLM endpoint settings instead of each other Fixes #372 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ffmpeg): validate resolved ffmpeg/ffprobe actually runs — fall through on WinError 193 (#360, #361, #362) A corrupt or wrong-arch imageio-ffmpeg download (and WindowsApps alias stubs) passes os.path.isfile/shutil.which but explodes at spawn with '[WinError 193] %1 is not a valid Win32 application', killing transcription with an opaque 500. Every resolution step now probes the candidate with '-version' (cached per process), logs the rejected basename, and falls through to the next source. Fixes #362 Fixes #361 Fixes #360 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: mergetest <test@local> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
d6562d6f30 |
feat(dub): Smart Fit phase B — per-segment video retime export, drift absorption, fitted subtitles (#350)
* feat(dub): Smart Fit phase B — per-segment video retime export, drift absorption, fitted subtitles Executes the video side of the Smart Fit plans persisted by Phase A (job["fit_plans"], #347) at export and preview time. Backend: - services/video_retime.py (new, clean-room): two-tier retime executor. ≤48 chunks → the proven single-pass split/trim/setpts/concat filter_complex; above → batches of 40 chunks rendered to intermediate slices (identical libx264 medium/crf20 params, keyframe at t=0) joined losslessly with the concat demuxer. Slices are CFR-resampled (fps=) because setpts leaves VFR-ish timestamps that broke tpad and drifted a frame per retimed chunk on ffmpeg 7.x. Temp slices cleaned on success AND failure/abort. - Drift absorption: fitted track longer than retimed video → freeze-frame tail (tpad=stop_mode=clone) predicted into the last slice / single-pass graph, with residual mux-side tpad; video longer → silence-pad the dub audio chain (apad=whole_dur). ±50 ms tolerance. - VFR guard: probe r_frame_rate vs avg_frame_rate; normalise with fps= before trim/setpts; probe failure degrades gracefully. - Plan resolution: _video_retime_plan_for spans legacy video_stretch_plans (byte-identical resolution + command construction) and fit_plans, gated on the track's own timing_strategy so stale plans never retime a track re-generated under another strategy. - Fitted subtitles: /dub/srt + /dub/vtt accept ?lang= and serve cue times from fitted_segments for Smart Fit tracks; _write_burn_srt does the same for burn-in. burn_subs+retime is now allowed for smart_fit (burn runs AFTER the retime graph); still rejected for legacy stretch_video. - /dub/preview-video resolves the same plan so in-app preview matches export. - Fallback ladder: batch encode failure/timeouts → un-retimed export with a structured core.failure warning (X-Dub-Export-Warning header + job["last_export_warning"]); concat join rejection → one single-pass retry while ≤96 chunks; abort → 409 + proc kill via run_ffmpeg job_id registration (/dub/abort reaches export encodes now) + temp cleanup. Frontend: - Export drawer passes ?lang= on subtitle exports and shows an i18n'd re-encode cost note (~0.5–2× video length on CPU) when a retiming strategy is active — translated in all 21 locales. Tests: tests/test_smart_fit_export.py — plan resolution, batch math, graph parity + new stages, fitted-cue SRT/VTT/burn selection, burn policy, VFR detection; ffmpeg-gated integration renders both executor tiers (batch size forced to 2) and the real /dub/download endpoint, ffprobing durations within ±50 ms across both pad branches. All existing dub export/subtitle/preview/timing tests pass unchanged. Refs docs/competitive-analysis.md Action 1 (dub-length fitting v2); completes Smart Fit (Phase A = #347). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): sanitize Smart Fit retime work paths at every sink (CodeQL py/path-injection) The job_id-derived retime work path (retimed_*.mp4 / preview_retimed_*.tmp.mp4) flowed unguarded from dub_export into prepare_smart_fit_video / render_retimed_video and their derived slice/concat paths and ffmpeg argv. Apply the repo's proven inline realpath+startswith containment pattern (helpers/commonpath are not recognized — see #309/#328/#329/#348): - dub_export.py: validate work_path against DUB_DIR at both construction sites (export + preview) and pass the validated realpath onward. - video_retime.py: make both entry points self-defending — realpath + DUB_DIR containment on out_path/work_path before any derivation, raising RetimeError(stage="plan") on escape; slices_dir/slice_path/list_path and RetimeDecision.file_path now all derive from the sanitized value. DUB_DIR is read via module attribute so test fixtures reloading core.config work. - ffmpeg_utils.py: document that all caller-assembled argv paths are realpath-validated upstream. - tests: sandbox DUB_DIR in the executor integration tests (tmp_path) so the new guard sees the test workspace. No behavior change for valid (server-built) paths — the guard only fires on traversal. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(smart-fit): patch DUB_DIR on video_retime's own config ref — survives suite-wide reload The retime guard reads video_retime._config.DUB_DIR at call time; the sandbox fixture patched a fresh 'import core.config' instead. Another test reloads core.config in the full suite, so the two module refs diverged — the patch missed and the guard rejected the test's tmp paths (green in isolation, red in CI's full run). Patch the exact ref the guard dereferences. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(dub): resolve DUB_DIR live at call time in retime guards — survive full-suite reload The path-containment guards bound DUB_DIR via a module-level 'from core import config as _config'. Other tests importlib.reload() core.config (sandboxing OMNIVOICE_DATA_DIR), after which the guard checked containment against a stale DUB_DIR while dub_export built the path under the reloaded one — every retime path then 'escaped the dub workspace' (green file-alone, red full-suite: the 5 integration failures CI hit). Re-import DUB_DIR locally in each guard so it always reads the current sys.modules value; simplify the sandbox fixture to patch the canonical module. Verified: full backend suite green on the Smart Fit tests (the 2 remaining settings_store failures are pre-existing on main, unrelated — local data-dir artifact). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): clear CodeQL alerts on Smart Fit export — job_id allowlist, proc-registry decouple - py/path-injection (8, video_retime.py): validate job_id with a strict inline regex allowlist (re.fullmatch [A-Za-z0-9_-]{1,64}) at the entry of dub_download and dub_preview_video, before it reaches any filesystem path or ffmpeg argv. The existing realpath containment guards stay as defense-in-depth; the regex barrier is the sanitizer CodeQL recognizes through the service-module call chain. - py/log-injection (4): newline-strip job_id inline at the logger calls in ffmpeg_utils.run_ffmpeg and the two retime-fallback logger.error sites in dub_export. - py/empty-except (3): best-effort cleanup os.remove handlers now log the OSError at debug instead of bare pass (video_retime + both dub_export mux finally blocks; _discard_tmp too for consistency). - py/cyclic-import (2): break the dub_pipeline <-> ffmpeg_utils cycle for real — the subprocess registry (register_proc/unregister_proc/ kill_job_procs/has_active_procs + state) moves to a new stdlib-only leaf module services/proc_registry.py. ffmpeg_utils now imports it at module top (no lazy import); dub_pipeline re-exports every name so dub_core aliases and tests keep working unchanged. No behavior change for valid inputs; invalid job ids now get a clean 400 instead of a 404/containment error. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(dub): address #350 review — cancelled-vs-failed retime, logged best-effort excepts, redacted probe logs, narrowed test assert - rc<0 (killed by user cancel) now raises RetimeError(stage='aborted') instead of reporting an ordinary render failure (CodeRabbit) - best-effort cleanup/QC-event excepts log at debug instead of bare pass (CodeQL empty-except x3) - probe failure logs use basename, not full user paths (CodeRabbit/CodeQL) - test_render_cleans_slices_on_failure asserts RetimeError, not Exception Rebuttals (no change needed, see PR comment): fitted-cue subtitles track the fitted AUDIO timeline which is correct even on retime fallback; the planner only emits stretch ratios >1 so the early-exit guard is a true no-op check; '\'' is ffmpeg's own utility quoting for concat lists; has_active_procs is an intentional re-export (noqa'd). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: mergetest <test@local> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
4b21f82619 |
feat(dub): Smart Fit timing strategy — planner, fingerprints, generate path (phase A) (#347)
* feat(dub): Smart Fit planner, fit fingerprints, shared ffmpeg stretch helpers - services/fit_planner.py: pure, I/O-free planner for dub-length fitting v2 — slack absorption (gap guard), audio-only band (<=1.2x), geometric 50/50 audio/video split capped at 1.5x / 2.0x, residual overflow accounting, and a stretch_video-compatible video_plan + fitted timeline cursor. Clean-room reimplementation from a published description. - services/incremental.py: fit_fingerprint() over the fit params with the same _canon_value canonicalisation as segment hashes (#281 class). Fit params stay OUT of segment_fingerprint — a fit change re-mixes, never re-TTSes. - services/ffmpeg_utils.py: move _atempo_chain/_pitch_preserving_stretch out of the dub_generate router (lazy torch/numpy imports) so the Phase B export pipeline can reuse them; add probe_duration() ffprobe helper. - schemas/requests.py: timing_strategy gains "smart_fit"; optional fit_options knob overrides default server-side. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(dub): smart_fit branch in the generate path TTS loop unchanged (dur_s=None, natural-rate WAVs on disk). After the loop, plan_fit() decides per segment; the mix loop applies audio_rate via the pitch-preserving atempo pipe (linear-interp fallback), trims residual overflow with the existing fades, and places audio at the planned new_start on a fitted-length canvas. Truthful fit_status entries (audio_rate / video_ratio / overflow_s) feed the row badges. Persists job["fit_plans"][lang] = {plan (exact _build_video_stretch_filter_graph shape), fitted_segments (cue times from ACTUAL stretched sample positions), total/orig duration, params, fit_fp} and mirrors fit_fp on dubbed_tracks[lang]. video_stretch_plans untouched. Strategy-transition guard: job["seg_wav_kind"] records whether on-disk seg WAVs are natural or slot-squeezed; a smart_fit partial regen over slotted (or unknown) WAVs forces one full regen instead of double-compressing. Old strategies and old persisted jobs are byte-identical (all new reads via .get()). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ui): Smart Fit option in the dub timing picker (all 21 locales) - prefsSlice: TimingStrategy union gains 'smart_fit'; optional FitOptions overrides (null by default — backend defaults apply identically on every platform); persisted alongside timingStrategy. - DubTab: Segmented gains Smart Fit with i18n label + tooltip. - useDubWorkflow: sends fit_options only when set and strategy is smart_fit. Default strategy stays 'concise' — no default behaviour change on any platform. - locales: dub.timing_smart_fit{,_title} translated in all 21 languages. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(dub): fit planner unit + golden suites, smart_fit generate-path integration - test_fit_planner.py: threshold boundaries (0.9/1.0/1.2/1.21/4.0), cap saturation -> overflow, slack absorption incl. gap guard, last-segment tail, cursor monotonicity, allow_video_retime=False, video_plan fed straight into _build_video_stretch_filter_graph, fit_fingerprint canonicalisation (int vs float, omitted vs default — the #281 class) and a pinned stable digest. - tests/fixtures/fit_planner/*.json: 4 golden FitPlans; algorithm drift is a deliberate fixture diff, never a silent change. - test_smart_fit_generate.py: hermetic end-to-end runs (mock TTS, no ffmpeg) covering audio-only stretch, hybrid timeline growth + persisted plan shape, fit_options override, strict_slot->smart_fit forced regen then zero-TTS fit-only re-mix, and concise back-compat. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(competitive): dub-length fitting row reflects Smart Fit Phase A Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(incremental): mark fingerprint hashes usedforsecurity=False — dedup keys, not security (Bandit) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: mergetest <test@local> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Palash DebnathandClaude Opus 4.8
|
47729057bd |
chore(lint): remove unused imports + variables (ruff F401/F841) (#210)
Autofixes the genuine lint behind the CodeQL py/unused-import and py/unused-local-variable note-level alerts — actually removing the dead code rather than dismissing it. 68 safe fixes via 'ruff check --select F401,F841 --fix' across 29 backend files (dead stdlib/symbol imports like io/sys/json/torch/typing.Optional and unused locals). Only ruff's safe fixes applied — the 9 'unsafe' fixes and the audio_dsp numpy availability import were left untouched. Not touched: empty-except (needs per-site judgement, not autofixable); frontend js/unused-local-variable (eslint no-unused-vars has no autofix); the loopback-low-risk path/log/stack-trace alerts (real, left visible). Verified: full tests/ suite unchanged at 601 passed (the 2 test_supertonic3 failures are pre-existing on main, local .venv state, green in CI). Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Palash DebnathandClaude Opus 4.8
|
e69dcbb6b1 |
fix(win): subprocess spawns work under bun run dev (--reload) on Windows (#122) (#175)
issue #122 'Extract: Unknown Error' on Windows: 'bun run dev' fails, running backend+frontend separately works. Root cause: dev:api launches uvicorn with --reload, so use_subprocess=True, and uvicorn 0.42's asyncio_loop_factory EXPLICITLY forces the SelectorEventLoop on Windows in that case (passed as loop_factory to asyncio_run, overriding any policy). The SelectorEventLoop has no subprocess support -> asyncio.create_subprocess_exec raises NotImplementedError. 'python backend/main.py' (no reload) uses ProactorEventLoop -> works. So an event-loop-policy fix is futile; the thread fallback is the fix. The ffmpeg extract path already routed through _spawn_async's thread fallback (landed in #157), but several other spawn sites used raw create_subprocess_exec and stayed broken on the dev loop: - add public spawn_subprocess() (drop-in for create_subprocess_exec) that routes through _spawn_with_retry -> _spawn_async (NotImplementedError -> thread fallback + EAGAIN retry); native asyncio path unchanged on supported loops. - fix _spawn_thread_fallback to forward cwd/env/etc. to subprocess.Popen (was silently dropping them -- breaks sonitranslate's cwd= pip install). - convert raw spawns: dub_generate atempo, tools ffprobe, gallery yt-dlp (x2), sonitranslate install (x4). translation_engines already had its own fallback. - tests: NotImplementedError -> thread fallback; cwd forwarding; stdin input (atempo); native path unchanged. No behavior change off the broken loop (macOS/Linux/Windows-prod): the native asyncio subprocess is still used; the fallback only triggers on NotImplementedError. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
8fa54ccae4 |
l10n(zh-CN): full Chinese localization + Windows/settings fixes (absorbed from #66) (#157)
* @
fix: skip torch.compile on Windows where Triton is unavailable
torch.compile with mode="reduce-overhead" depends on Triton, which has no
official Windows support. On Windows the compile call succeeds but generates
code paths that crash at inference time with an OOM-like error
("Cannot find a working triton installation").
Check for Triton availability before compiling so TTS gracefully falls back
to eager mode on platforms without Triton.
Closes #65
SummerSec
@
* feat: comprehensive Chinese (zh-CN) localization for Settings and navigation
Add full Chinese (zh-CN) translation support across the frontend:
- NavRail, Launchpad, Clone/Design tabs, Settings (all tabs)
- Sidebar navigation labels, hero text, action cards, section headings
- Fix: Settings missing General tab in TABS array
- Fix: i18n locale not persisted after page reload (useEffect deps)
- Fix: NavRail key prop spreading into JSX elements
Co-authored-by: SummerSec
* fix: translate production override parameter labels (Speed, t_shift, etc.)
* fix: translate voice design category labels (Gender, Age, Pitch, etc.)
* feat: translate Transcriptions and Voice Gallery pages
* fix: translate gallery category names (Disney, Anime, etc.)
* feat: translate DubTab, personality presets, and voice design presets
* fix: remove duplicated emoji in personality name translations
* fix: correct preset translation keys to match actual preset IDs
* fix: filter natural language from personality instruct to prevent validation error
* fix: handle edge case where instruct has no valid tags
* chore: remove debug logging from personality instruct filter
* fix: address CodeRabbit review — importlib.util, English comment, grammar, theme, placeholder
* fix: localize selected category label in VoiceGallery header
* feat: translate remaining DubTab UI text (CAST, Generate Dub, Translate All, etc.)
* fix: improve ffmpeg detection on Windows, error messages, and yt-dlp download timeout
* feat: add proxy setting in Settings → General for downloading via proxy
* fix: improve ffmpeg detection on Windows, error messages, and yt-dlp download timeout
* feat: allow HTTP_PROXY/HTTPS_PROXY env vars via /system/set-env
* fix: support SOCKS5 proxy, also set ALL_PROXY env var
* fix: increase yt-dlp extractor retries for subtitle 429 errors
* feat: translate prep overlay stage labels (download, extract, demucs, scene)
* feat: translate BatchQueue, VoiceProfile, ToolsPage, Projects pages
* feat: translate SetupWizard, DonatePage, EnterprisePage + fix NotImplementedError handling
* feat: add ffmpeg status + manual path setting in Settings → General
* fix: validate ffmpeg path exists when user sets it manually
* fix: fall back to thread-based subprocess when asyncio raises NotImplementedError on Windows
* fix: pin setuptools<70 — ctranslate2 requires pkg_resources removed in 70+
* fix: translate transcribing overlay text
* fix: complete DubTab zh-CN localization + argostranslate preflight check
* fix: add cmn-Hans language code mapping for Google Translate
* fix: fall back to thread-based pip install on Windows when asyncio subprocess raises NotImplementedError
* feat: add DeepL/Microsoft/LLM credential fields to Settings
* feat(i18n): localize GlossaryPanel, DubSegmentTable, DubSegmentRow
* fix: Windows-safe log rotation handler avoids PermissionError on rename
* feat: persist proxy/FFmpeg/LLM/translation credentials, separate DeepL/Microsoft keys, add glossary max-height scroll and collapse
- /system/set-env writes env.* to prefs.json via prefs.set_()/delete()
- Backend startup reads env.* from prefs.json into os.environ (.setdefault)
- PERSISTENT_KEYS covers proxy, FFmpeg, LLM, DeepL/Microsoft keys
- DeepL uses DEEPL_API_KEY, Microsoft uses MICROSOFT_API_KEY (fallback TRANSLATE_API_KEY)
- DeepL/Microsoft _build_translator reads DEEPL_BASE_URL/MICROSOFT_BASE_URL
- Google/MyMemory/Microsoft translators bypass Windows registry proxy
- Frontend CREDENTIAL_GROUPS splits into 4 groups with password/text fields
- Glossary panel body max-height: 35vh + overflow-y: auto
- Glossary panel can be collapsed via ChevronDown button
- queryClient.invalidateQueries after save for immediate refresh
- SystemInfoResponse adds proxy_url, ffmpeg_ok, ffmpeg_path
* feat(i18n): localize ExportModal with zh-CN support
- Add useTranslation + replace ~50 hardcoded strings with t() calls
- Add exportModal namespace to en.json and zh-CN.json
- Cover presets, tracks, tabs, video/audio/subs/package tabs, license notice, and output footer
* fix: address PR #66 security review feedback
- Regenerate uv.lock against pypi.org (remove TUNA mirror URLs)
- Route HF_TOKEN through huggingface_hub.login() instead of prefs.json
- Add os.chmod(prefs_path, 0o600) for restricted file permissions
- Add warning logs to _WindowsSafeRotatingFileHandler bare except blocks
* docs: add Chinese translation README_CN.md
* docs: add link to Simplified Chinese translation in README.md
* docs: add English/Simplified Chinese cross-links between READMEs
* fix(l10n): restore clickable Discord/email footer links in EnterprisePage
The i18n extraction replaced main's clickable <button onClick=openExternal>
footer links with bare {t()} labels, dropping both the clickable behavior
and the literal Discord URL — which broke test_discord_link_updated
(EnterprisePage missing discord.gg/bzQavDfVV9). Restore both as clickable
links wrapping the translated label, with the hardcoded URL/mailto (URLs are
not translated). Keeps i18n, restores functional parity with main.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(l10n): no-hardcoded-CJK rule + enforce; clean dead LLM block; harden set-env
- Add 'Localization (hard rule)' to CLAUDE.md: no hardcoded non-English UI
text outside frontend/src/i18n/; functional CJK allowed via allowlist.
- New tests/test_no_hardcoded_cjk.py enforces it (allowlists text-processing
regexes, model/engine vocab & IDs, error matching, demo/eval data, fixtures).
- Settings.jsx: remove dead saveLlm block (Chinese toasts + unused llm* state,
flagged by CodeQL js/unused-local-variable); render language-picker native
names from new LANGUAGES export in i18n/index.ts instead of hardcoding.
- main.py: drop unused 'import shutil' (CodeQL py/unused-import).
- system.py: harden FFMPEG_PATH/FFPROBE_PATH set-env (reject control chars;
defense-in-depth for the py/path-injection finding). Endpoint stays
loopback-only — network sharing must never expose /system/set-env.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: drop unused ui import (Panel) + fix implicit str-concat in cjk test
Clears the two CodeQL notes introduced/attributed to this PR:
- Settings.jsx: remove unused 'Panel' from the '../ui' import (js/unused-local-variable).
- test_no_hardcoded_cjk.py: collapse multi-line message strings to single lines
(py/implicit-string-concatenation-in-list).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: SummerSec <summersec@qq.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Palash DebnathandClaude Opus 4.7
|
c32041289d |
Phase 1 Wave 3: AppImage launcher + .deb ffprobe + Docker LAN + Gatekeeper probe (closes #54, #56, #76, #80) (#93)
* fix(appimage): conditional WEBKIT_DISABLE_COMPOSITING_MODE launcher (#56) WebKitGTK 2.44.x and 2.46.x have a compositing-path regression on Wayland that blanks the AppImage's first paint on Fedora 44 / Ubuntu 24.04. Setting WEBKIT_DISABLE_COMPOSITING_MODE=1 forces the software fallback that works, but blindly setting it on healthy WebKit versions (2.48+) regresses those. This wave adds a conditional AppRun launcher that detects the WebKit version via pkg-config and only sets the env var on the broken ranges (plus a fail-safe when pkg-config is absent or the version is unknown). The launcher is injected into Tauri's AppImage staging dir via a beforeBundleCommand hook — see .planning/decisions/apprun-strategy.md for the spike outcome and rationale (Strategy B chosen). Phase 1 Wave 3 — Plan 01-03 Task 1. Closes #56 frontend half. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(deb): relocate bundled ffprobe out of /usr/bin to avoid conflicts (#76) Prior versions placed the bundled ffprobe at /usr/bin/ffprobe via Tauri's externalBin, which overwrites the system ffprobe on Ubuntu 26.04 and collides with apt-installed media-package ffprobe. Relocate the .deb-bundled ffprobe to /usr/lib/omnivoice-studio/bin/ffprobe via bundle.linux.deb.files, plus defensive maintainer scripts: - preinst: ensure target dir exists for upgrade flows - postinst: remove legacy /usr/bin/ffprobe ONLY when dpkg confirms our package owns it (never touches a user's distro ffprobe) - postrm: clean up the relocated path tree on purge/remove Rust side (tools.rs::resolve_ffprobe) now probes the new path on Linux, and backend spawn (backend.rs) carries both FFPROBE_PATH (legacy alias) and OMNIVOICE_FFPROBE_PATH (canonical) into the backend env. Python side (ffmpeg_utils.resolve_ffprobe) reads OMNIVOICE_FFPROBE_PATH first, falls back to FFPROBE_PATH, then to shutil.which("ffprobe"). 6 new unit tests cover the env-cascade resolution. Phase 1 Wave 3 — Plan 01-03 Task 2. Closes #76. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(frontend): centralised apiBase resolver for Docker LAN access (#80) Docker / LAN browser users hit the preview API at the LAN host's IP, not their local machine — the prior frontend/src/utils/media.js:20 hardcoded http://localhost:3900, which from a LAN client resolved to the client machine itself. Centralise via frontend/src/utils/apiBase.ts: 1. VITE_OMNIVOICE_API override (Docker compose / dev) always wins. 2. Tauri webview → http://localhost:3900 (unchanged behaviour). 3. Plain browser → ${window.location.protocol}//${window.location.hostname}:3900 (follows the page's origin — closes #80). 4. SSR / no-window → http://localhost:3900 (safe fallback). Grep-sweep confirmed media.js:20 was the only hardcode site (Assumption A4 in 01-RESEARCH.md verified). 6 new vitest cases cover the resolver. Phase 1 Wave 3 — Plan 01-03 Task 3. Closes #80 frontend half. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(backend): macOS Gatekeeper quarantine probe + INST-01 guard (#54) Adds backend/core/gatekeeper_detect.py which walks up from sys.executable to find the .app bundle and runs `xattr -l` to check for the quarantine extended attribute (com.apple.quarantine). On detection, the lifespan startup probe logs a structured warning and emits a system_error event through the existing event bus with error_class="GATEKEEPER_QUARANTINE", which Wave 2's React ErrorBoundary turns into a docs deeplink. Detection is informational only — we never auto-run `xattr -cr` (the app itself is quarantined and cannot fix its own state per Anti-Pattern in 01-RESEARCH.md). Users get a clear pointer to the workaround docs. GET /system/quarantine-status exposes the structured payload so the frontend can poll on first load. INST-01 (setuptools>=75.0 pin from PR #62) gains a PR-time guard in tests/backend/test_pyproject.py + a user-observable smoke check in scripts/smoke-test.sh (pkg_resources + whisperx import). 7 gatekeeper tests + 1 pyproject test added — all pass. Phase 1 Wave 3 — Plan 01-03 Task 4. Closes #54 backend half (Wave 2 owns the docs page + ErrorBoundary deeplink wiring). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Palash DebnathandClaude Opus 4.7
|
a1ef66c321 |
Stability pass: DB leaks, App.jsx hooks refactor, desktop bootstrap (#49)
* fix: eliminate DB connection leaks, race conditions, and deprecated asyncio API ## DB Connection Leaks (P0) - Convert 38 raw get_db() calls to db_conn() context manager across 14 router files - Connections are now guaranteed to close even when exceptions are raised - profiles.py create_profile: clean up orphaned audio file if DB insert fails - profiles.py lock_profile: consolidate 3 separate conn.close() error paths ## Race Condition (P1) - Add _dub_jobs_lock (threading.Lock) to protect _dub_jobs dict in dub_pipeline.py - get_job/put_job now thread-safe for concurrent dub sessions ## asyncio Deprecation (P2) - Replace 23 asyncio.get_event_loop() calls with asyncio.get_running_loop() - Prevents DeprecationWarning on Python 3.12+ and future breakage on 3.14 ## Quick Fixes - gallery.py preview_voice: remove filesystem path from error response (P2) - dub_pipeline.py parse_vtt_segments: remove redundant `import re` inside loop (P3) - gallery.py _init_gallery_db: use db_conn() context manager (P2) * refactor: extract hooks, centralize isTauri, add pytest-cov ## Frontend - Extract useTTS hook (150 LOC) — TTS generation, streaming, audio ingestion - Extract useProfiles hook (219 LOC) — voice profile CRUD, lock/unlock, preview - Centralize isTauri detection: dialog.js, VoiceGallery.jsx, Settings.jsx now import from utils/media.js instead of 4 different detection patterns ## Backend - Add pytest-cov to dev dependencies - Baseline coverage: 39% across backend/ (214 tests pass) - Add .coverage to .gitignore * feat: add Vitest + checkJs, extract useDubWorkflow + useAppData hooks ## Frontend Testing (new) - Set up Vitest with jsdom environment + @testing-library/react - 11 tests: utils (isTauri, formatTime, constants) + Zustand store (mode, text, dubStep, pill) - Scripts: 'test' (vitest run), 'test:watch' (vitest), 'test:legacy' (node runner) ## App.jsx Decomposition (continued) - Extract useDubWorkflow hook (387 LOC) — upload, ingest, transcribe SSE, translate, generate SSE, abort, stop, cleanup - Extract useAppData hook (181 LOC) — data loading, localStorage persistence, WebSocket real-time updates, model-status pill management ## TypeScript checkJs - Enable checkJs: true in tsconfig.json for IDE-level type checking - 947 existing errors (informational, not blocking builds) - noImplicitAny remains false to avoid blocking * ci: add Vitest step, fix useProfiles duplicate state ## CI - Add 'Run Vitest (frontend)' step — runs 11 unit tests - Override --checkJs false in CI typecheck to avoid 947 pre-existing errors - Rename legacy test step for clarity ## Hooks - Fix useProfiles to accept loadProfiles from parent (useAppData) instead of managing its own duplicate profiles array * refactor: wire hooks into App.jsx — 2067 → 1129 LOC (-45%) App.jsx now delegates to extracted hooks instead of inline logic: - useAppData: data loading, localStorage, WebSocket, model pill - useProfiles: voice profile CRUD, lock/unlock, preview - useTTS: generation, streaming, audio ingestion - useDubWorkflow: upload, transcribe SSE, translate, generate SSE 988 lines removed. All handler logic lives in focused, independently testable hooks. Store selectors and render JSX stay in App.jsx as the shell. Verified: vite build clean, 11 frontend + 214 backend tests pass. * feat: show real-time percentage on model loading pill Backend: register hf_progress listener during _load_model_sync() so download/weight-loading tqdm events update _loading_detail with a progress percentage (0-99%). get_model_status() now includes a 'progress' field that the frontend polls. Frontend: useAppData reads msQuery.data.progress and calls setPillProgress() — the FloatingPill already renders the percentage text and progress bar width from this value. * fix: prevent FileNotFoundError in desktop bundle during model init transformers >=4.52 calls _can_set_experts_implementation() and _can_set_attn_implementation() during PreTrainedModel.__init__, which open the class source file via open(class_file). In a Tauri desktop bundle, module.__file__ points to a path that doesn't exist on disk, causing: FileNotFoundError: .../omnivoice/models/omnivoice.py Override both classmethods on OmniVoice to return static values without filesystem access. OmniVoice doesn't use MoE experts (return False), but does support flex/flash attn (return True). * fix: sync source dirs on every bootstrap, not just first run The Tauri bootstrap previously only copied omnivoice/ and backend/ to Application Support on the first run. Subsequent app updates kept using stale source files, preventing bug fixes from landing. Now ensure_venv_ready() always syncs both directories from the bundle resources before returning, even when the venv is healthy. This fixes the FileNotFoundError crash where the old omnivoice.py lacked the _can_set_experts_implementation override. * ui: premium setup wizard polish - Primary button: solid gradient fill with hover glow + lift + press - Stepper nav: connected pills with glow ring on active step - Welcome cards: glassmorphism with stagger-in animations, lucide icons, left-border accent strip, hover translate - Preflight panel: colored icon pill backgrounds, stagger-slide entrance - Step transitions: fade+slide animation via keyed wrapper - Footnote: shortened paths (~/ notation), Reveal in Finder button - Recommendation banner: gradient background with accent glow - Compact spacing throughout for denser, professional layout * fix: kill zombie backend on clean+retry bootstrap When clean_and_retry_bootstrap removes the project dir, any old uvicorn process still running from the deleted paths remains alive on port 3900. The subsequent retry_bootstrap sees the port is healthy and attaches to the zombie instead of re-bootstrapping. Now explicitly kill any process on the backend port after cleaning, before calling retry_bootstrap. * feat: integrate speaker clones into dubbing interface, sanitize system environment variables for subprocesses, and improve FFMPEG binary path resolution. * fix: restore docker compose default + drop dead setSeed call - deploy/docker-compose.yml: remove profiles: ["cpu"] from the default service so `docker compose up` matches the comment on line 5. With the profile present, no service auto-started. - frontend/src/App.jsx: drop the setSeed call in restoreHistory. The selector was never reintroduced after the App.jsx hooks split, and there is no seed state in the store — seeds are generated fresh per call in useTTS and only read from history items for display. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address CodeRabbit review — async detection, dub stream, bootstrap fail-fast - backend/services/tts_backend.py: invert async-context detection in _ensure_loaded. The previous code unconditionally caught its own diagnostic RuntimeError and then called asyncio.run() inside a running loop, masking the intended error message. - frontend/src/hooks/useDubWorkflow.js: require a terminal `done` event before reporting dub success. Without this, a dropped stream after partial progress would flip the UI to `done`, refresh history, and play the completion ping as if generation finished. - frontend/src/hooks/useDubWorkflow.js: restore the previous step when tasksCancel() fails. The UI was getting stuck in `stopping` forever on cancel errors. - frontend/src-tauri/src/bootstrap.rs: fail-fast when source sync fails after the existing directory has already been removed. The previous warn-and-continue path could leave the install with no backend/ or omnivoice/ sources and defer the failure to backend startup with a cryptic error. - backend/api/routers/generation.py: add `from e` to the ValueError → HTTPException re-raise (Ruff B904). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: preserve % suffix in TTS generation timer The 100ms timer in useTTS was rewriting generationTime to a plain elapsed-seconds string, which immediately wiped the "(xx%)" download suffix written on the next iteration of the response-body loop. The real-time percentage was flickering on/off as a result. Read the previous value inside the setter and reattach any existing percent suffix. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Palash Debnath
|
545b39c912 |
feat: Scalar API docs, community health files, Quickstart cards (#41)
* feat: Scalar API docs, community health files, Quickstart cards, GHCR Docker Backend: - Replace Swagger UI with Scalar at /docs (scalar-fastapi) - Add OpenAI-compatible /v1/audio endpoints (openai_compat router) - Add TTS streaming endpoint (tts_stream router) - Add voice marketplace router (marketplace) - Update TTS backend registry Frontend: - Refine CaptureWidget, WaveformTimeline, App layout - CSS polish and index.css updates Community health: - SECURITY.md — vulnerability reporting policy - CODE_OF_CONDUCT.md — Contributor Covenant v2.1 - .github/FUNDING.yml — GitHub Sponsors - .github/ISSUE_TEMPLATE/ — bug report + feature request - .github/pull_request_template.md — PR checklist README: - Quickstart redesigned as 3-column progressive cards - Docker section updated with GHCR pull instructions - API Docs row added to service table Infra: - scalar-fastapi added to pyproject.toml + uv.lock - research/ added to .gitignore * refactor: clean up documentation and logging while enhancing desktop packaging dependencies and capture UI performance. * fix: address CodeRabbit review — streaming, escaping, thresholds Backend: - marketplace: stream zip entries via ZipFile.open()/copyfileobj, add 100MB upload cap, fix raise-from exception chaining (OOM prevention) - openai_compat: _encode_audio returns actual file ext so Content-Disposition matches real format; forward non-profile voices when DB row not found - tts_stream: send 'start' frame after generation so sample_rate is real; forward non-profile voices on DB miss - capture_ws: split MIN_BUFFER_BYTES into separate partial/final thresholds so short utterances (<2s) still get transcribed Frontend (Tauri): - lib.rs: tray 'dictate' now toggles start/stop based on widget visibility - commands.rs: XML-escape exe path in LaunchAgent plist, shell-quote in .desktop Exec line to prevent injection from special-char paths - CaptureWidget.css: fix Stylelint violations (empty lines, font-family quotes) |
||
|
debpalash
|
41c23f6b3a | feat: enhance ASR performance and reliability with binary bundling, model warmup, sub-stage progress tracking, and optimized polling. | ||
|
debpalashandClaude Opus 4.7
|
67328d04fe |
refactor: split backend into api/core/services/schemas, harden security + fd pressure, add searchable language picker, fix segment fragmentation
Backend:
- Split monolithic main.py into backend/{api/routers,core,schemas,services}
- core/db.py: allowlist-gated migrations, db_conn context manager (kills SQL injection on ALTER)
- core/tasks.py: lock-guarded listener add/remove/push, snapshot-before-iterate
- services/ffmpeg_utils.py: run_ffmpeg helper with concurrency semaphore, EAGAIN retry, guaranteed reap
- services/segmentation.py: Bengali/CJK/Arabic punctuation, ultra-short tier, stitch_adjacent_shorts,
bounded-loop merge; public clean_up_segments API
- services/model_manager.py: robust lock.locked() handling
- api/routers/dub_core.py: job_id traversal guard, thread-safe _active_procs, timeouts on ffmpeg/demucs,
POST /dub/cleanup-segments endpoint
- api/routers/dub_export.py: guarded SSE listener remove, ffmpeg timeouts via run_ffmpeg
- api/routers/exports.py: destination_path validation, safe source resolver, subprocess list-form
- api/routers/generation.py: contextlib.suppress on tempfile cleanup, db_conn usage, safe output-path helper
- api/routers/system.py: try/finally tmp cleanup, subprocess timeouts
- schemas/requests.py: TranslateSegment.id int->str to match hex segment IDs
- main.py: threading.Lock around crash log writes
Frontend:
- components/SearchableSelect.jsx: popover combobox with search, keyboard nav, popular+recent pins, 200-item cap
- App.jsx: wire SearchableSelect for dub language / ISO code / voice-gen language; Clean Up segments button;
fix blob URL leak (object-shaped prev in setter, unmount cleanup via ref)
- components/WaveformTimeline.jsx: explicit <video> detach instead of innerHTML='' to release decoder
- index.css: ss-* combobox styles matching Gruvbox theme
Tests:
- tests/test_segmentation.py (26 cases), test_dub_transcribe.py, test_dub_export_unique.py, conftest.py
Chore:
- .gitignore: exclude omnivoice.zip, /research/ reference clones
- Remove tracked stray root test scripts + crash_log.txt
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|