d91beef0fd314250d8d9b94de86dfea019a8bd96
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 debpalash
|
3a7368cb26 |
fix(watermark): route every synthetic-audio producer through one mark_synthetic chokepoint (#1169)
POST /v1/audio/speech returned synthetic audio without the AudioSeal provenance watermark while /generate marked the same text — and the audit behind #1169 found the same class of gap in five more producers. EU AI Act Art. 50(2) (applicable 2026-08-02, expressly carved out of the open-source exemption in Art. 2(12)) makes machine-readable marking of synthetic audio a provider obligation, so per-door coverage gaps are compliance bugs. Root cause: coverage grew call-site-by-call-site (three separate embed_watermark calls) with nothing forcing a new producer to opt in. The fix, per the whole-class rule: * services/watermark.py grows `mark_synthetic(wav, sr, *, context, force=False)` — THE named chokepoint (delegates to embed_watermark: pref-gated, no-op without AudioSeal, never raises / degrades to unmarked) — plus `will_mark()` for cache-key derivation. * Gaps closed at the tensor stage, before any encoding: - openai_compat `_run_tts` (the reported gap; all response_formats) - tts_stream /ws/tts (per-sentence, before PCM16 framing) - generation stream=true preview chunks (marked streamed copy; the saved take keeps its single whole-take mark in finalize) - batch dub pipeline (assembled track, before WAV write / aac mux) - longform chapter render — covers /audiobook, /longform/render (Stories), /audiobook/preview and /audiobook/resume; the chapter cache key now carries a watermark tag so stale unmarked cache entries can never be served for a marked-on render - dub preview-segment (docstring had declared it exempt) - archetype render (served preview + materialized profile reference) * Already-covered paths (generate finalize, dub segments, persona bundles) migrated onto the same chokepoint. * Documented non-producers/gaps instead of fake coverage: /stories/encode is a pure transcoder of user uploads (must NOT mark); the opt-in SoniTranslate sidecar synthesizes+muxes externally and is a documented provenance gap at its route. * Regression tests: tests/test_synthetic_audio_watermark_1169.py runs detect_watermark() on the actual response audio of every producing route (real watermark service, fake AudioSeal nets, fake engine; verified fail-before on the pre-fix tree — 11 of 13 fail). * Recurrence-proofing: tests/test_watermark_route_coverage.py structurally asserts every synthesis call site references mark_synthetic (justified allowlist), producers keep their call, and embed_watermark is never called outside the chokepoint. Settings toggle semantics are unchanged (the issue's two legal questions stay flagged for a lawyer, deliberately unanswered here). |
||
|
|
8281b7c798 |
fix(engines): dub and batch TTS honor the active-engine selection (#987)
* fix(engines): dub and batch TTS honor the active-engine selection — with a real capability gate, not a silent OmniVoice fallback
Dub generation and batch TTS hardcoded services.model_manager.get_model()
(OmniVoice) regardless of the engine picked in Settings → Engines. A user
selecting VoxCPM2 (or any other engine) still got OmniVoice output with no
error — the silent fallback IS the bug class, not just the one report.
Root-caused and fixed for the whole class:
- New `TTSBackend.supports_cloning` capability flag (default True) marks
engines that can only offer fixed preset voices — kittentts,
supertonic3, sherpa-onnx set it False. MLXAudioBackend exposes it as an
instance property (Kokoro doesn't clone, CSM does) since the adapter
multiplexes multiple models with different capabilities.
- `cloning_capable_engine_ids()` and a shared `resolve_generation_backend()`
helper in services/tts_backend.py centralize engine resolution
(id → is_available() → routing gate → optional cloning gate), mirroring
generation.py's /generate resolution instead of inventing a third
parallel mechanism. Both routers now standardize on the existing
get_active_tts_backend() cache (unload-on-switch already handled).
- dub_generate.py's two TTS-generate call sites (main run + OOM retry) and
the /dub/preview-segment route resolve once, up front, with
require_cloning=True — dub's ref_audio is populated for essentially
every real job, so an engine that can't clone fails the whole job with
one actionable message instead of mis-cloning per segment.
- batch.py resolves once per job, require_cloning only when voice_id is
pinned — an unpinned batch job runs fine on any engine.
- Applied the three pre-existing TODO(#312) comments: mastering now skips
via `applies_own_mastering` for both pipelines, matching generation.py.
Regression tests cover the capability-id list, the fail-fast gate (proving
no OmniVoice fallback), the success path on a selected non-OmniVoice
engine, batch's pinned-vs-unpinned voice_id behavior, and the mastering
skip for both pipelines. Three existing dub tests that mocked get_model()
directly were updated to mock the new resolver instead.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(engines): exclude model-dependent adapters from cloning_capable_engine_ids()
getattr(cls, "supports_cloning", True) at the CLASS level returns a
property descriptor object (always truthy) when the flag is an instance
@property, not a plain attribute — MLXAudioBackend uses exactly this
pattern because its cloning capability depends on which of its 7+ curated
models is loaded (only CSM clones; Kokoro etc. don't). Without this fix,
the dub/batch capability-gate error message would always recommend
'switch to mlx-audio' even when the user's configured MLX model can't
clone, sending them in a circle back to the same error.
isinstance(value, bool) distinguishes a resolved boolean from a
descriptor object, so mlx-audio is excluded from the suggestion list
until its actual per-instance capability can be checked (already handled
correctly by resolve_generation_backend()'s per-call instance check).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(changelog): engine-aware dub/batch entry (#987)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
|
d90cfde1bb |
feat(dub): per-language translations + per-track caches — switching languages stops destroying work (P1) (#958)
* feat(dub): per-language translations + per-track caches — switching languages stops destroying work (P1) Multi-language dubbing translated per language (#957) but stored everything in single-slot state, so tracks silently destroyed each other's work: P1.2 — per-language translation storage (additive): - Frontend keeps every translation in s.translations[langCode] alongside the legacy s.text slot (still = the shown language). Translate All writes both; the new store action switchDubLangCode swaps text through the map on a user-driven language switch (non-destructive; restore paths keep the plain setter); manual edits / restore-original update the current language's entry; merge joins per-language texts, split drops them. Rides project save/load inside dubSegments — legacy projects behave exactly as before. - Backend mirrors it as job["segments_i18n"] = {lang: {segKey: text}} (segKey = stable id, index for id-less legacy rows), written by _sync_job_segments; job["segments"] stays byte-identical for every existing consumer. /dub/srt|vtt?lang= and subtitle burn-in now emit THAT language's text when present — ExportModal's "all dubs" batch stops producing N identical files. Legacy jobs without the field fall back to today's output. P1.3 — per-track WAV cache + fingerprints: - Per-segment WAVs are language-keyed (seg_{lang}_{id}.wav). The partial-regen read path falls back to legacy seg_{id}.wav ONLY while the job has no other-language track — single-language jobs keep their whole on-disk cache; multi-track jobs stop splicing the last-generated language into the current track. Read-only endpoints (segment preview, clips zip) gained ?lang= with the permissive legacy fallback they always had. - Fingerprints include the track language (segment_fingerprint(track_lang=…), /tools/incremental lang=…) and live in job["seg_hashes_by_lang"]; the flat job["seg_hashes"] stays as the current track's mirror so the done event, history restore and older frontends read it unchanged. A legacy flat map is attributed to the job's last-generated language (dropped when unknown) and reads stale once — the safe direction. seg_wav_kind is per-track too. - The frontend stores fingerprints per language and judges "Regen N changed" against the ACTIVE track; project save/load and dub-history restore carry all tracks' hashes (segHashesByLang / seg_hashes_by_lang, additive). Tests: fail-before regression coverage — two-track regen never splices the other language's audio (sample-level assert on the mixed track), legacy single-track cache reuse + multi-track gate, per-lang seg_hashes with flat mirror + migration semantics, /dub/srt|vtt?lang= emitting different text per track with legacy fallbacks, per-lang burn-in, /tools/incremental lang scoping, and 14 frontend tests for translations round-trips, per-track fingerprints and legacy-project behaviour. Full backend + frontend suites, typecheck, lint and format:check green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(changelog): add per-language storage + per-track caches under [Unreleased] (#958) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: mergetest <test@local> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
48ae4dae1d |
fix(dub): re-dub honors transcript edits — fingerprints canonicalised, preview cache-busted, atomic mux (#281) (#329)
* fix(dub): re-dub honors transcript edits — fingerprints canonicalised, preview cache-busted, mux made atomic (#281) Three symptoms, three causes: 1. Edited line, unchanged result: the dubbed preview-video URL was identical across re-dubs, so the WebView kept serving the previous dub. A generation nonce now cache-busts the preview after every completed generation. 2. Preview stuck loading forever: overlapping preview requests ran ffmpeg against the same output path and the mtime cache check saw the half-written file as valid. The mux now runs under a per-path lock, writes to a temp file, and os.replace()s into place. 3. One edit re-dubs all lines: server-side fingerprints were computed from pydantic-parsed segments (defaults filled in) but recomputed client-side from raw dicts (keys omitted), so every segment always looked stale and incremental degraded to a full re-dub. Values are now canonicalised on the backend and the frontend builds generation inputs through one shared helper (utils/segments.js) for both the generate request and the incremental plan. Fixes #281 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Potential fix for pull request finding 'CodeQL / Uncontrolled data used in path expression' Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> * fix(dub): realpath containment for job-derived preview paths (CodeQL) Request-supplied job_id/lang flowed into the preview mux output path. Both now pass a realpath containment guard against DUB_DIR (the file's existing per-segment pattern) and lang is allowlist-validated before it lands in a filename. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(dub): inline the containment guard — CodeQL can't track it through a helper Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> |