d91beef0fd314250d8d9b94de86dfea019a8bd96
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Palash DebnathandClaude Fable 5
|
2d5f2e800e |
feat(omnivoice): voice prompts that survive restarts + opt-in FlashInfer (~2.2x) (#1565)
* feat(omnivoice): port upstream VoiceClonePrompt persistence + FlashInfer opt-in Upstream k2-fsa teardown ports, verified with generated voice samples: - VoiceClonePrompt.save()/.load() (upstream format v1, weights_only-safe) on the vendored model, and a disk layer under the in-memory prompt LRU (DATA_DIR/prompt_cache, keyed by ref path+mtime+ref_text+preprocess, 32 newest kept, OMNIVOICE_PROMPT_DISK_CACHE=0 opts out). First generation of a session with a known voice skips the reference re-encode and any auto-transcription pass — verified across two real processes (encodes=1 then encodes=0, same voice). - omnivoice_flashinfer.py ported (packed CFG attention, fused kernels, optional CUDA graphs), schedule adapted to our num_step+1 divergence. Opt-in via OMNIVOICE_FLASHINFER=1|graph, CUDA-only, replaces torch.compile for the session; missing package / apply failure / runtime failure all degrade with a named reason (same #278 contract as compile: classify → unapply → retry once, session latch). Measured 2.20x at batch=1 on an RTX 4090 with byte-identical text and clean ASR round-trip. - Docs: OmniVoice guide gains instruct+reference combination semantics (consistent instruct stabilizes cloning, reference wins conflicts), inline pronunciation control (pinyin / CMU), prompt persistence, and corrects the 'no voice design' claim; performance.md documents both new env knobs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: point changelog entries at the real PR number (#1565) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pr): harden FlashInfer lifecycle + prompt-cache writes per review Bot harvest round 1 (#1565): unapply on apply-failure (half-patched model could crash the next render); pin eager-mode FlashInfer inference to one thread too — the attention plan and packed position ids are per-generation module state, so interleaved _gpu_pool workers would corrupt each other; restore the CAPTURED pre-apply attention impl (could be flash_attention_2) instead of assuming sdpa; unique tmp name per prompt-cache write; correct the _forward_logits layout docstring; resolve VoiceClonePrompt at test runtime; docs — Known limits keeps only the limitation, performance.md states the VRAM cost and scopes the fallback claim to classified kernel failures. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pr): round-2 review — publish only a fully restored model, redact latch reason, tighten CPU-persistence test Greptile: the runtime fallback now unapplies BEFORE swapping generate, so a concurrent render keeps queuing behind the thread-affinity wrapper while teardown mutates modules. CodeRabbit: FlashInfer failure reasons pass through core.failure.sanitize before latching/logging (wheel paths embed the user's home); the save-portability test now creates the tokens on CUDA when available and asserts the persisted payload itself is CPU-resident. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pr): fail-closed latch reason when the sanitizer itself breaks CodeQL empty-except + CodeRabbit round 3: if core.failure.sanitize raises, the raw reason (home paths, wheel paths) was latched anyway. Now only the exception class survives with a fixed redaction note; two regression tests (normal redaction + sanitizer failure). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Palash DebnathandClaude Opus 5
|
4f1135ac62 |
fix(compile): skip torch.compile when the torch lib path has whitespace (#1310)
* fix(compile): skip torch.compile when the torch lib path has whitespace Inductor passes the torch library directory to clang++/g++ as an unquoted -L flag, so a path containing a space splits into two arguments and the compile dies with "no such file or directory: 'Support/...'". The quoting bug is inside PyTorch and we cannot fix it — but a path we already know cannot compile is one we should not spend a compile attempt on. The cost was never a broken generation (eager mode is the documented fallback); it was a guaranteed-failing compile on every load, whose clang wreckage got swallowed by `except Exception: logger.info(...)` and then ate a chunk of the captured log tail. That is exactly how it surfaced in #1259, where it was not the actual fault but crowded out the output that was. Not platform-specific: macOS keeps app data under ~/Library/Application Support/ and a Windows profile is routinely C:/Users/First Last. OMNIVOICE_FORCE_TORCH_COMPILE=1 still overrides, consistent with the arch gate. An unreadable torch path fails open — no evidence is not evidence of a problem. Closes #1266 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(compile): redact the torch path in the skip reason; tighten the tests CodeRabbit Major: the reason string embedded the absolute torch lib path, and both log branches print it — so a home directory (and anything secret-shaped in the path) went into omnivoice.log and any pasted bug report. It now goes through core.failure.sanitize(), the same redaction every other user-facing failure text uses, with a basename fallback if that import ever fails. Also per review: the Settings gate is isolated in the test helper (it was reading the real settings_store, so a persisted perf.torch_compile_disabled=1 could have decided these tests instead of the path logic), and two logging contracts are now asserted rather than merely exercised — the forced-override warning, and the skip message naming OMNIVOICE_FORCE_TORCH_COMPILE. A skip the user cannot discover how to override is a dead end. DECLINED: CodeRabbit's Critical asks for torch.compile to be disabled by default on every platform "so the default is uniform". That would make every Linux CUDA user slower to satisfy a rule about USER-VISIBLE default behaviour — and torch.compile is not user-visible, it is an internal optimization whose absence shows up only as speed. The function has always diverged by host by design: device != "cuda" skips, and no-Triton skips (which is every Windows install). This gate adds no new divergence; it declines a compile that is GUARANTEED to fail on that host, which is the same shape as the existing arch gate. Disabling a working optimization everywhere would be the regression. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
debpalashandClaude Opus 4.8
|
791dae69db |
fix(rocm): stop force-routing every AMD GPU to the CPU (#1228)
The GPU compatibility gate built a CUDA-namespace tag from `get_device_capability()` (`sm_115` on a gfx1151 Strix Halo) and looked for it in `torch.cuda.get_arch_list()` — which on a ROCm wheel returns gfx *names* (`gfx1100`, `gfx1151`, …). The two namespaces can never intersect, so `check_device_compatibility()` returned False on every ROCm build and `get_best_device()` silently returned "cpu": torch saw the GPU, `torch.cuda.is_available()` was True, and the app ran on the CPU anyway. The comparison was copy-pasted in three places, all with the same bug, so it now lives once in `core.device_caps.arch_unsupported()` and branches on the build (gfx names on ROCm, sm_/compute_ tags on CUDA): - `model_manager.check_device_compatibility` — the CPU force-route, plus a ROCm-specific remedy instead of telling AMD users to install a cu128 wheel - `device_caps._probe` — the kernel-risk note that downgraded the routing badge - `engine_env._cuda_arch_supported_for_compile` — torch.compile off on all AMD `_configure_rocm_if_needed` also applied `HSA_OVERRIDE_GFX_VERSION` from a static map without checking whether the GPU needed it, remapping cards the installed build supports natively onto foreign kernels. It now applies only when the native GFX ID is genuinely absent from the arch list, and knows gfx1150/gfx1151 (Strix Point/Halo). Regression test: tests/test_rocm_arch_gate.py pins the reporter's host resolving to "cuda", a genuine ROCm mismatch still being caught, the CUDA path (#756 Blackwell fallback) unchanged, and the narrowed override. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
c0924f5eba |
fix(tts): torch.compile failures fall back to eager — generation never fails on unsupported GPUs (#278) (#327)
* fix(tts): torch.compile failures fall back to eager — generation never fails on unsupported GPUs (#278) On GPU architectures the bundled Triton doesn't support (e.g. Blackwell sm_120 / RTX 5060), the compiled model dies mid-generation inside the Dynamo/Inductor/Triton/cudagraph stack — previously surfaced as a fake 'ran out of memory' error and a dead Archetype preview. Now: - up-front arch gate: skip compile when the GPU's compute capability is not in this torch build's arch list (OMNIVOICE_FORCE_TORCH_COMPILE=1 overrides for PTX forward-compat setups) - runtime fallback: model.generate is wrapped once; a compile-stack failure (classified by exception chain: module, message, traceback paths — the cudagraph case is a bare AssertionError) logs a warning, restores the eager module, disables compile for the session, resets dynamo state, and retries eagerly. Non-compile errors propagate unchanged. - the /generate OOM handler no longer mislabels compile crashes as OOM and points users at the actual remedy. Fixes #278 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Potential fix for pull request finding 'CodeQL / Empty except' Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> * Potential fix for pull request finding 'CodeQL / Empty except' Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> * Update backend/api/routers/generation.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> Co-authored-by: mergetest <test@local> |
||
|
Palash DebnathandClaude Opus 4.8
|
898f41a57d |
fix(windows): gate torch.compile on Triton + ASR critical-path smoke (plan-02, closes #65) (#138)
* fix(windows): gate torch.compile on Triton availability (#129, closes #65) plan-02. torch.compile(mode="reduce-overhead") needs Triton at runtime; Triton has no Windows wheel, so the old `device=="cuda"`-only guard in model_manager.py failed on Windows+CUDA and surfaced as a confusing "OOM" (#65). Inference-time, hard to diagnose. - engine_env.should_torch_compile(device): requires CUDA + find_spec("triton") + the existing perf.torch_compile_disabled setting being off; logs the skip reason at INFO and falls back to eager. - model_manager.py call site uses it instead of the bare cuda check. - smoke-test.sh INST-02: import torch + ctranslate2 + whisperx (full ASR path) so a missing transitive dep fails the build instead of crashing mid- transcription (#116). Runs in the CI smoke-matrix on Win/macOS/Linux. setuptools>=75.0 (fix-sequence step 1) already pinned (#58). Linux/CUDA+Triton behaviour unchanged. Tests (TDD): tests/test_torch_compile_gate.py (4). Closes #65; addresses #129/#116. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(windows): also gate subprocess torch.compile on Triton (Greptile #138) Greptile flagged that the in-process gate left a parallel gap: engine subprocesses honour TORCH_COMPILE_DISABLE, but build_engine_env() only set it on the user's Performance toggle — so a Triton-absent host (Windows, or macOS) still exposed subprocess engines to the same crash this PR fixes in-process. - build_engine_env(): set TORCH_COMPILE_DISABLE=1 when the user disabled compile OR Triton is unavailable (find_spec), cross-platform — mirrors should_torch_compile(). Drops the Windows-only scoping (and the now-unused `import sys`). - Refreshed the stale module docstring. - 3 new tests cover the subprocess gate (triton-missing, triton-present, user-opt-out). 7/7 pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * revert(engine_env): keep subprocess TORCH_COMPILE_DISABLE user-driven Reverts the build_engine_env() broadening from the previous commit. Auto- disabling subprocess torch.compile on Triton-absence conflicts with a deliberate, tested contract (test_perf_settings: Windows + flag-off ⇒ no injection; non-Windows ⇒ never inject) — the subprocess var is intentionally under the user's explicit control. The #65 fix is the in-process should_torch_compile() gate (unchanged here), which IS automatic and fully tested. Pushing back on the subprocess auto-gate as a separate, deliberate contract change rather than forcing it through by rewriting established tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Palash DebnathandClaude Opus 4.7
|
715766cb04 |
Phase 1 Wave 2: per-OS install docs + Settings UI + error→docs deeplinks (#94)
* docs(install): per-OS install pages + drift validator + CI gate
Splits the 600-line README install section into self-contained per-OS docs
under docs/install/{macos,windows,linux,docker}.md plus a Top-10
troubleshooting index. Each OS doc is end-to-end: a user opens it and
reaches a working app following only commands inside that file.
Adds:
- docs/install/{macos,windows,linux,docker}.md (OS-specific install paths)
- docs/install/troubleshooting.md (top 10 install errors)
- docs/engines/cosyvoice.md (closes #55 docs half)
- docs/features/diarization.md (pyannote license flow)
- docs/setup/huggingface-token.md (3-source cascade guide)
- scripts/validate-install-docs.py (INST-06 docs-drift gate)
- tests/scripts/test_validate_install_docs.py (B-5: validator self-tests)
- .github/workflows/ci.yml step running the validator on every PR
Implements INST-02 (README routing), INST-03 (macOS Gatekeeper anchor),
INST-12 docs half (Windows torch-compile-oom anchor), DOCS-01..05.
The validator is a one-way diff: every `<!-- validate -->`-tagged line
in docs must appear in scripts/desktop-prod.sh after normalisation
(prompt-prefix strip, CRLF, trailing whitespace, blank-and-comment skip).
A `<!-- validate: skip -->` marker opts out for human-readability blocks.
Its own 10 unit tests catch regressions in the gate itself.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(deeplinks): links.py + error_docs_map (Python + TS mirror)
Adds the single source of truth for the project repo URL and the 4-class
error → docs taxonomy that both the in-app ErrorBoundary deeplink button
(Wave 2 Task 3) and the Phase 5 bug reporter will consume.
New:
- backend/core/links.py — PROJECT_REPO_URL + BLOB_MAIN resolver
(Tauri config first, pyproject fallback)
- backend/core/error_docs_map.py — lookup(error_class) → docs URL
- frontend/src/utils/errorDocsMap.ts (TS mirror with classifyError helper)
- tests/backend/core/test_links.py + test_error_docs_map.py
- frontend/src/utils/errorDocsMap.test.ts
Resolves checker B-6 (links.py ownership) and Open Question #3 (which fork
the deeplinks resolve to — the Tauri updater endpoint wins, which points
at the desktop app fork debpalash/OmniVoice-Studio).
The TS BASE constant is documented as the second hardcoded URL drift site;
the keys-sync test (`test_keys_match_python_map` equivalent) guards the
4-class taxonomy contract between Python + TS halves.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(ui): Settings → API Keys panel + ErrorBoundary docs deeplink
Wave 2 AUTH-03 UI half + ErrorBoundary deeplink wiring.
ErrorBoundary fallback now renders an "Open docs for this error" button
that classifies the thrown Error message (heuristic: pkg_resources → 401 /
HfHubHTTP → WebKit / white screen → quarantine / Gatekeeper) and opens the
matching docs anchor via Tauri shell.open (with a window.open fallback
in browser dev mode).
ApiKeysPanel consumes the Wave 1 resolver state endpoint:
- 3 source rows (App / Env var / HF CLI) with set/unset indicator,
masked token preview, whoami username + green check
- "Active" badge on whichever source is currently serving the cascade
- App-row only: Save (POST /api/settings/hf-token) +
Clear (DELETE with optional "also clear HF CLI" confirm dialog)
- "Test now" button refetches state (invalidates the resolver's
validation cache via the same endpoint hit)
Panel mounted in the existing Settings → Credentials tab; the legacy
HF_TOKEN row from CREDENTIAL_FIELDS is filtered out so the two paths
don't fight over the same key.
Threat T-02-02: the panel never displays the full token. The masked
value comes from the resolver state endpoint; the full token only
crosses the IPC boundary on Save (POST) and is cleared from local
state on success.
Closes AUTH-03 fully (Wave 1 backend + this Wave 2 UI).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(perf): INST-12 Disable torch.compile (Windows) toggle (backend + UI)
Wave 2 Task 4 — full INST-12 delivery per checker B-2/B-7 v0.3.0 fat-release
decision. Both the docs half (windows.md anchor, shipped in earlier commit)
and the runtime toggle are now in Phase 1.
Backend:
- backend/services/settings_store.py: adds get_text/set_text helpers for
non-secret config (refuses to write to the encrypted hf_token key).
- backend/api/routers/settings.py: GET + PUT
/api/settings/perf/torch-compile-disabled, both under the existing
loopback guard (threat T-02-04).
- backend/services/engine_env.py: new `build_engine_env()` helper that
centralises HF_TOKEN/YOUR_HF_TOKEN injection from the 3-source resolver
AND injects TORCH_COMPILE_DISABLE=1 when the flag is set on win32.
Phase 2 SubprocessBackend launchers should adopt the same helper.
- backend/services/sonitranslate.py: migrated to engine_env.build_engine_env()
while preserving the source-level `env["HF_TOKEN"]` sentinel that
test_sonitranslate_module_uses_resolver checks.
Frontend:
- frontend/src/components/settings/PerformancePanel.{jsx,css,test.jsx}:
toggle UI with the explainer for #65; renders disabled with a "not
applicable" badge on macOS/Linux.
- frontend/src/pages/Settings.jsx: mounts the panel into the Credentials
tab alongside the API Keys panel.
Tests:
- tests/backend/test_perf_settings.py: 7 backend tests (default state,
PUT persistence, T-02-04 non-loopback rejection, settings_store round-
trip, env injection on win32, NO injection on macOS/Linux, NO injection
when disabled).
- frontend PerformancePanel.test.jsx: 5 tests (renders from GET state,
PUT on toggle, disabled on non-Windows platforms, pre-enabled state).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(planning): Wave 2 SUMMARY + REQUIREMENTS status updates
- .planning/phases/01.../01-02-SUMMARY.md: full implementation report
per template (truths, commits, tests, deviations, drift-site
acknowledgments per W-3, launcher seam name for Phase 2,
taxonomy keys for Phase 5).
- .planning/REQUIREMENTS.md: flips Wave 2 closures to Done:
AUTH-03, INST-02, INST-03 (docs half), INST-06, INST-12,
DOCS-01..05.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|