Send individual jobs to GPUs on your other machines while everything else
stays local. Opt-in, off by default: with the toggle off there is no
listening socket, no certificate and no background loop.
Design follows remote/goal_v2.md, the council-revised goal doc. The
decisions that shaped the code, and why:
* A disconnect is an unknown outcome, not a failure. The original design
reassigned on disconnect while also describing the case where the worker
had already finished — following both guarantees duplicate execution. An
attempt now holds a grace window; a worker returning inside it commits
its result and no second attempt is ever made.
* At-least-once execution, exactly-once result commit. The result is
persisted BEFORE it is acknowledged, so a crash between the two cannot
silently lose a finished render.
* Deadlines are phased (accept -> model load -> execute -> deliver) and
liveness is a progress lease. The old fixed 30s execution budget was two
orders of magnitude below what this product actually does; silence is
the failure signal, not slowness.
* Capacity is derived from free VRAM, never configured: a static value
corrupts output under torch.compile thread affinity (#315) and aborts
the process on small cards (#567).
* A circuit breaker replaces the reliability-score/quarantine machinery,
which had no recovery path (no probation workload exists in a TTS
product) and penalised consumer networks for existing.
* Identity is a keypair the worker generates and never sends. A
server-assigned id is a name, not an authenticator, so revocation of one
would be theatre. Enrollment tokens are single-use and carry the control
plane's certificate fingerprint for pin-on-first-use.
Adds the domain core, scheduler, durable task store, gRPC transport,
worker agent, management API, Settings panel, and docs. Protobuf reserves
the tenant/trace/usage fields a hosted control plane would need, since
adding them later means upgrading a whole fleet.
Includes tests for the failure paths that matter: duplicate delivery,
stale-session fencing, reconnect reconciliation, grace expiry, breaker
attribution, and a real end-to-end TLS round trip.
The dictation hotkey could leave a blank dark square stuck on the desktop with no way to dismiss it. Three defects compounded: the tray listener's effect depended on [state], so it detached across an await on every state change and a press landing in that gap was lost; an idle pill renders null, so the window Rust had already shown was empty; and the opaque chrome background made that empty window a hard-edged square. Nothing could hide it — dismiss() is only reachable from the X button, Esc, or a post-session timer, none of which exist for a session that never started.
Fixed at the invariant rather than the call sites: the listener subscribes once for the component's lifetime, the widget window's chrome background is transparent, and an idle-but-visible window reconciles itself to hidden. The reconcile is polled (a dropped press changes no React state, so there is nothing to key an effect off) and aborts if its effect is torn down mid-check, so it can never hide a dictation that has just started.
Also in scope:
- The rename sweep had repointed three data-dir literals at a brand-named directory that does not exist, so smoke-test.sh verified a directory the backend never writes and desktop-prod.sh silently stopped clearing backend state on Windows. Both invisible on macOS, where they are usually run. A guard test now pins the assignments specifically.
- The dictation model picker's download sizes were wrong for all seven models, in both directions — Parakeet TDT v3 (the recommended default) understated 180 MB against an actual 670 MB, while the low-RAM fallbacks were overstated threefold, discouraging exactly the choice that would have helped. Measured from the published repos and pinned by a test.
- The 0.6B Parakeet models now decode on more threads, capped by host cores and still overridable.
- uninstall.ps1 gained a UTF-8 BOM (Windows PowerShell 5.1 mis-decodes its non-ASCII output without one), and sponsor.yml lost its last OmniVoice references.
Only the TTS model (~2.4 GB) is required on first run; ASR models are
per-platform curated picks (curated_on in models.yaml) installed on demand.
Every transcription surface returns a typed asr_model_missing error with a
one-click download CTA instead of silently pulling multi-GB Whisper weights.
Settings -> Models is a grouped, platform-aware catalog. New guided
permissions UX (wizard System Check + Settings -> Permissions + mic
pre-flight) with native mic-state checks and OS settings deep-links. New
parakeet-mlx engine brings Parakeet TDT v3 to Apple Silicon (language-gated
capture preference so multilingual dictation never regresses). Docs:
expressive-speech page, Flush/Unload + CPU-fallback triage, clone-length FAQ.
Hardening: preflight fails open for custom model pins, ROCm curation no
longer inherits NVIDIA picks, Windows mic probe reads the NonPackaged
consent key, CaptureWidget setup race fixed, offline-cache CI simulation
fixes so empty-cache runners stay green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(engines): bundle en_core_web_sm — no mid-generation GitHub download
Post-merge review finding on #1144 (valid): with pip present, misaki/spaCy's
first-use auto-download WORKS now — meaning the first English MLX-Audio
generation performs a raw GitHub release download that (a) bypasses the app's
entire HF-mirror/endpoint system (restricted-network users have no recourse
mid-generation) and (b) fails offline. Local-first says default features
shouldn't spring surprise outbound requests at generation time.
en_core_web_sm-3.8.0 is now a pinned URL dependency in pyproject/uv.lock
(~12 MB wheel): it arrives at install/update time via the normal dependency
flow (where network failures are visible and retried), survives drift-sync
by construction, and spacy.util.is_package() finds it so misaki never
triggers its downloader at all. The #1143 containment stays as the backstop
for any other CLI-shaped dependency.
Also clarifies the venv test per review: pytest's interpreter IS the
uv-synced venv in CI and the packaged app, so find_spec verifies the lock;
the test now also pins the bundled model.
Validated: uv sync --frozen clean; en_core_web_sm importable.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(engines): direct-URL dependency + frozen-bundle collection (review)
Two of three review P1s were real:
- Docker build break: `uv add` wrote a bare "en-core-web-sm" dependency with
the URL only in [tool.uv.sources] — Docker's `uv pip install --system .`
reads project metadata only, would resolve the bare name against PyPI
(where spaCy models don't exist), and the image build fails. Now a direct
"name @ url" dependency, the same form kittentts has always used, so every
installer (uv sync, pip, Docker) sees the same source. Re-locked;
uv sync --frozen clean.
- Frozen bundle: backend.spec ships mlx_audio, whose Kokoro path loads
en_core_web_sm DYNAMICALLY (spacy.load by name) — PyInstaller never sees
the import, so a frozen build would hit misaki's downloader at first
English generation. collect_all('en_core_web_sm') added inside the
mac-ARM block (plain data package, no nanobind hazard — the reason
collect_all is banned for mlx itself doesn't apply).
Declined with precedent: "hard-coded GitHub URL breaks restricted networks"
— kittentts has shipped as exactly this GitHub-release URL form in the same
dependency list since it was added; install-time GitHub fetches are the
project's accepted pattern (the bootstrap's gh-proxy mirror exists for
restricted networks), unlike mid-generation fetches, which this PR removes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: mergetest <nizam4103@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
New conservative, idempotent pre-TTS normalization pass
(services/text_normalization.py): strips zero-width/control junk, caps
pathological repeat runs, expands digits/times/ordinals/currency via
num2words (29 locales) and per-language abbreviation maps (EN/DE/ES/FR).
Wired once at each text-to-engine choke point — /generate, dub segments
(+ preview), and longform chapters — BEFORE the pronunciation dictionary
so user respellings stay the final say. Pref-gated
(text_normalization_enabled, default ON) with OMNIVOICE_TEXT_NORMALIZATION
env override; num2words promoted to a direct dependency.
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(net): SOCKS-proxy users can synthesize again — ship socksio, cache-first model resolution, degrade LLM clients (#959)
Under ALL_PROXY/HTTPS_PROXY=socks5:// without socksio installed, httpx
raises ImportError AT CLIENT CONSTRUCTION ("Using SOCKS proxy, but the
'socksio' package is not installed"). huggingface_hub's get_session()
builds exactly that client inside snapshot_download, so POST /generate
500'd with the bare message even for a fully installed model, and
preload_model's model_info probe hit the same error and silently
skipped warm-up. Latent since v0.3.5 — #947's fresh-process engine
spawning unmasked it in v0.3.10 by handing the user's proxy env
directly to a clean backend process.
Three layers, so the class (any session-construction failure) is dead,
not just the reported instance:
* Ship SOCKS support: socksio>=1.0 in [project] dependencies (pure
Python, MIT, zero transitive deps) AND in backend.spec hiddenimports
— httpx imports it lazily in try/except, so PyInstaller's tracer
misses it and the frozen installers would stay broken without the
explicit entry. uv.lock regenerated; `uv lock --check` and
`uv sync --frozen` (the Docker/release bootstrap semantics) verified.
* Cache-first model resolution: from_pretrained's snapshot resolution
extracted into _resolve_snapshot_dir() — local dir, else
snapshot_download(local_files_only=True) (a complete cache resolves
with NO HTTP session constructed), else the original network path.
preload_model's failed network probe now falls back to a cache-only
check and warms up anyway instead of silently skipping (honest log
either way).
* Class guards: resolve_skill_client wraps OpenAI() construction —
env-shaped construction failures degrade to the existing "LLM
unavailable" contract instead of 500ing the calling feature; and
core.failure learns SOCKS_PROXY_SUPPORT_MISSING with an actionable
hint, appended on the raw-string surfaces (global 500 handler,
model-install SSE) via the new append_hint().
Fail-before/pass-after verified by reverting the fix: 11 of the 12 new
tests fail pre-fix (the remaining one is the unchanged network-fallback
contract). 165 tests green across the touched suites.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(changelog): add SOCKS-proxy resilience under [Unreleased] (#966)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The desktop app's About panel, /health, /system/info, diagnostics, bug reports,
and exported persona/marketplace bundle metadata all read core.version.APP_VERSION.
In a synced env that resolves from package metadata (correct), so CI and the
lockstep test were green — but the PyInstaller-frozen backend has no omnivoice
.dist-info, hits PackageNotFoundError, and fell back to a hardcoded
APP_VERSION = "0.3.5". The version-bump job never touched that literal, so every
0.3.x desktop build has been reporting 0.3.5 regardless of its real version.
Fix (belt and suspenders, so it can't recur):
- backend.spec: copy_metadata('omnivoice') so importlib.metadata resolves in the
frozen build — the primary path now works there too.
- backend/core/version.py: resolution chain is metadata → pyproject (walked up,
correct for raw source checkouts) → a named _FALLBACK_VERSION literal as last
resort (no longer the only fallback).
- tests/test_app_version.py: _FALLBACK_VERSION joins the lockstep (now FIVE
sources); + a test that the fallback resolves to pyproject, + a test that
backend.spec copies the metadata (so the frozen path can't silently regress).
- release.yml version-bump: also bumps _FALLBACK_VERSION so the lockstep guard
never reddens main after a release.
Already-shipped binaries can't be fixed, but every build from here (tonight's
preview, the next stable) reports its real version. 5 version tests pass.
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The binary-filter hack from PR #13 broke Tauri's resource walker on
Linux:
resource path `.../dist/omnivoice-backend/_internal/libcufft.so.11`
doesn't exist
PyInstaller's accounting (hook-generated rerun manifests, resource
glob expansion) still referenced the files after they were filtered
out of `a.binaries`, so Tauri's build.rs saw a path that didn't
exist on disk. Removing the post-hoc filter drops that error.
Keep strip=True + optimize=2 — those alone should still shave hundreds
of MB from native libs + bytecode. If the CPU-only torch wheel (PR #11)
+ these two flags aren't enough to get under 2 GB on Linux/Windows,
the next step is splitting the backend into a separately-downloaded
payload rather than trying to force it into one installer asset.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* ci: opt JavaScript actions into Node 24 runtime
GH deprecates Node 20 for JavaScript actions on 2026-09-16. The
deprecation warning surfaces on every run right now. Setting
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true at workflow level makes
actions/checkout, actions/setup-*, astral-sh/setup-uv, and
oven-sh/setup-bun all run on Node 24 without bumping action versions.
This is a runtime override only — our own test script still pins
Node 22 via actions/setup-node@v4 (required for
--experimental-strip-types).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(release): strip symbols + filter CUDA/CUDA-provider binaries, report size
Previous slim pass (PR #11, CPU-only torch + module excludes) still left
the frozen backend above GH Releases' 2 GB per-asset cap. Two more levers:
1. strip=True on EXE + COLLECT. Strips debug symbols from ELF/Mach-O
native libraries. libtorch_cpu.so and friends drop ~25-30%. No-op on
Windows (MSVC stores symbols in separate .pdb files).
2. optimize=2 in Analysis. Compiles embedded bytecode with -OO:
docstrings + assertions removed. ~50-80 MB off the PYZ archive.
3. Post-hoc binary filter after collect_all. Even with nvidia wheels
excluded as Python modules, collect_all('torch')/('onnxruntime') can
still pull the CUDA-runtime shared libs via their linker hints.
Pattern-match them out of a.binaries before PYZ.
4. Log bundle size after freeze so CI runs can be compared without
downloading artifacts.
If this round still overshoots 2 GB, the next step is splitting the
payload (thin installer + post-install download of the Python bundle).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Linux .deb upload and Windows MSI build both hit GitHub Releases'
hard 2147483648-byte asset cap because the frozen backend was ~2.2 GB
on Linux/Windows. Root causes + fixes:
- PyPI's default torch/torchaudio wheels bundle the full CUDA runtime
(~1.8 GB of libcuda*, libcublas*, libcudnn*, libcufft*, libcusparse*,
etc.). We ship CPU-only inference from the desktop binary; GPU is
surfaced only when a user-installed driver is detected at runtime.
Re-install torch from download.pytorch.org/whl/cpu for the Linux and
Windows matrix jobs before PyInstaller freezes. macOS wheels don't
include CUDA so they skip this step.
- Expand backend.spec excludes: torch subpackages we never touch at
inference time (torch.distributed, torch._dynamo, torch._inductor,
torch._export, torch.testing, torch.onnx, torch.ao, torch.fx.
experimental, torch._functorch, torch.utils.tensorboard,
torch.utils.benchmark), torchaudio.prototype, and heavy pyproject
deps the backend never imports (gradio, tensorboardX, webdataset,
s3prl, funasr, pedalboard). Also drop test trees that collect_all
sweeps up (scipy.special.tests, numpy.f2py.tests, etc.).
Expected bundle size after trim: ~600-900 MB uncompressed on Linux /
Windows, well under the 2 GB cap for .deb and MSI.
Model weights were never bundled — they already download on first run
via the HF cache when the user hits the Dub / TTS / ASR flows. So no
user-visible behaviour changes; the app just ships without the libs
required for CUDA builds, which weren't callable on those runners
anyway.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
velixio