Palash DebnathandClaude Opus 4.7
a5e1bb3c51
* docs(phase-5): research opt-in bug reporting
Phase 5 research: prefilled-URL GitHub Issues pattern, default-deny payload,
redaction layer, two-step consent UX, rate/dedup/recursion safeguards,
aggregation across Python/Rust/React error producers. Builds on Phase 1's
links.py + errorDocsMap deeplink infrastructure; uses already-installed
@tauri-apps/plugin-opener (^2.5.4). No new packages required.
Covers REPORT-01..12 with confidence levels, 8 pitfalls, subprocess-engine
error capture handoff to Phase 2, security domain mapped to ASVS, and
3-wave delivery plan (redactor + payload, consent UI, aggregation +
pre-submit search).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(phases): research for Phases 2, 3, 4, 6 (Engine + Supertonic + Spikes + Release)
* docs(stack): bump supertonic pin 1.2.3 → 1.3.1 (Phase 3 research finding)
* docs(phases): plan Phases 2-6 for v0.3.0 fat-milestone release
15 new plan files + 2 ADR decision docs across 5 phases. Combined with Phase 1's 3 plans, the v0.3.0 milestone now has 18 PLAN.md files covering all 7 phases (Phase 0 already complete via PR #71).
PHASE 2 (Engine Isolation — 4 plans):
- 02-01: SubprocessBackend primitive + echo sidecar POC + graceful is_available wrap (ENGINE-01/05)
- 02-02: _safe_torchaudio_save helper + migrate 11 WAV write sites + #48 regression (BUG-01)
- 02-03: IndexTTS sidecar entry + venv-probe bootstrap + IndexTTS2Backend rewire (ENGINE-02/03/04/07, closes #42)
- 02-04: Engine Compatibility Matrix UI + /engines/{id}/health route (ENGINE-06)
PHASE 3 (Supertonic-3 + Mirror — 2 plans):
- 03-01: Supertonic-3 engine on SubprocessBackend + SHA pin + license gate (TTS-01..06)
- 03-02: bootstrap.rs mirror cascade + UV_DEFAULT_INDEX migration + frozen enforcement + docs (INST-07..11)
PHASE 4 (Spike-first Adaptive & Specialty — 2 plans + 2 ADRs):
- 04-01: OmniVoice-GGUF hardware-adaptive engine + quant_map + bundled binaries (SPIKE-01, GGUF-01..06)
- 04-02: OmniVoice-Singing subclass + dub pipeline singing mode + segment detector (SPIKE-02, SING-01..05)
- SPIKE-01-gguf.md + SPIKE-02-singing.md ADRs in .planning/decisions/
PHASE 5 (Opt-in Bug Reporting — 3 plans):
- 05-01: Redactor + BugReporter + URL builder + rate/dedup/recursion safeguards + FastAPI router (REPORT-01/02/03/05/06/07/08/10/11)
- 05-02: BugReportDialog two-step consent + PrivacyPanel + ErrorBoundary integration + Rust panic hook (chained) (REPORT-01-Rust/04/09/12)
- 05-03: Dry-run vs 3 historical issues + cross-platform openUrl smoke + Phase 2 subprocess-errors handoff (REPORT-02 smoke, REPORT-03 expansion, REPORT-09)
PHASE 6 (Release + Retro — 4 plans):
- 06-01: rc1 prep — version bump across 4 sources + CHANGELOG + retro stub + PR-73-strategy doc (REL-01/03/06)
- 06-02: CI guards — workflow-parity actionlint + tag-shaped dry-run (Phase 0 retro options B + C; closes release-engineer gap)
- 06-03: PR #73 reimplementation (NOT rebase) — backend-split installer with mirror-cascade integration + pill-mode regression checkpoint
- 06-04: Execute the release — pre-tag gates + 4-OS clean-VM + 48h soak + tag + retro + 3 v0.4 deferral tracking issues (REL-01/02/03/04/05/06)
Scope decisions locked in plans (council session):
- SoniTranslate refactor DEFERRED to v0.4 (Phase 2 ships SubprocessBackend without migrating Soni)
- macOS notarization DEFERRED to v0.4 (Phase 6 ships xattr -cr automation per CLAUDE.md Key Decision #7)
- supertonic pin 1.2.3 → 1.3.1 (already committed in ba63733)
- SPIKE-01 and SPIKE-02 both GO; 13/13 Phase 4 reqs stay in scope
- PR #73 reimplemented, not rebased (93 commits behind main)
All 18 plans validated via gsd-sdk frontmatter.validate + verify.plan-structure.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4.2 KiB
SPIKE-02: Adopt ModelsLab/omnivoice-singing as singing variant of the existing engine
Status: Proposed (research-supported) — awaiting Phase 2 SubprocessBackend merge
Date: 2026-05-18
Decision-makers: [maintainer]
Related: ROADMAP Phase 4; REQUIREMENTS SING-01..05; .planning/phases/04-adaptive-specialty-engines-spike-first/04-RESEARCH.md
Context
ModelsLab/omnivoice-singing (HuggingFace, 1,053 downloads/month, verified 2026-05-18) is a finetune of k2-fsa/OmniVoice — same Apache-2.0 license, same Qwen3-0.6B backbone, same Higgs Audio v2 codec at 24 kHz mono, same omnivoice PyPI library (0.1.5, 2026-04-28) already shipping in OmniVoice Studio v0.2.7. Trained on additional singing + emotion-tagged data and activated by a [singing] text control tag at generation time.
OmniVoice's existing dub_pipeline.py runs Demucs to split source audio into vocal and instrumental stems and routes the vocal stem through the default TTS engine. Today this produces speech-like output even on sung source material, which is one of the loudest user complaints when dubbing music-adjacent content.
This decision is whether to integrate the singing finetune as a routed alternative for sung segments, with auto-detection + per-segment override.
Decision
GO with reduced scope — integrate per SING-01..05.
The integration shape is OmniVoiceSingingBackend(OmniVoiceBackend) — a ≤30-line subclass overriding id, display_name, the from_pretrained model ID, and auto-injecting the [singing] control tag in generate() unless the prompt already starts with a [-prefixed tag. The dubbing pipeline gains a "singing mode" toggle and a segment-routing path (vocal stem → singing engine for sung segments, vocal stem → default engine for spoken segments, instrumental stem preserved untouched). Segment detection uses a pitch-stability + energy heuristic on the Demucs vocal stem with per-segment user override in the dubbing UI.
SING-02's full per-segment routing depth is decided after a Wave 2 code-read of dub_pipeline.py: if the existing pipeline supports per-segment routing in ≤50 lines, ship it; if it would require >500 lines of refactor, descope to "singing mode applies to entire dubbing job" for v0.3 and defer per-segment to v0.4.
Consequences
Positive:
- Sung segments of dubbed content produce sung output (currently produces unsuitable speech-like output).
- Zero new Python dependencies — same
omnivoicelibrary already shipping. - ≤30-line backend subclass; no new engine architecture.
- Hardware footprint identical to existing
OmniVoiceBackend; runs anywhere the default engine already runs.
Negative / risk:
- Heuristic segmentation (pitch-stability + energy) is one-dimensional and misclassifies operatic / sustained-vowel speech and vibrato-heavy speech.
- Cross-language singing quality is acknowledged by the model card as "extrapolation with variable quality."
omnivoice-singingreturns garbled output if the[singing]tag is missing — automatic injection is load-bearing.
Mitigations:
- Per-segment override available in the dubbing UI before any segment is committed to a render (user owns the final route — SING-03 already requires this).
- SING-05 acceptance scoped to native-language singing pass; cross-language flagged as best-effort with model-card disclaimer surfaced in the engine card UI.
OmniVoiceSingingBackend.generate()always prepends[singing]unless the prompt already starts with[, allowing power users to compose[singing] [happy]etc. manually.- Model-based singing-vs-speech classifier explicitly deferred to v2 per REQUIREMENTS.md Out of Scope.
- License + model-card link surfaced in the engine card UI; first-use acceptance gates download (SING-04).
Sources
.planning/phases/04-adaptive-specialty-engines-spike-first/04-RESEARCH.md(this milestone's research)- https://huggingface.co/ModelsLab/omnivoice-singing (verified 2026-05-18)
- https://huggingface.co/k2-fsa/OmniVoice (upstream)
- https://pypi.org/project/omnivoice/ (0.1.5, 2026-04-28)
backend/services/tts_backend.py(existingOmniVoiceBackendreference)backend/services/dub_pipeline.py(existing dubbing pipeline — Wave 2 code-read target)