The compute-time error told users to raise a generation timeout that had no control anywhere in the app — the only knob was an environment variable, and on Windows the docs explicitly warn against the usual way of setting one. Both budgets are now editable in Settings under Performance & Device, persisted and applied on the next start.
Two defects found in review and fixed here rather than shipped: an explicit universal budget silently overrode a separately saved CPU budget, so the CPU row would have looked like it worked and done nothing; and a value already set in the environment shadowed the saved preference while the panel still reported success. A shadowed row now says so instead. Long-input warnings also fire on Apple Silicon, which gets the accelerated budget and was the device in one of the duplicate reports.
Fixes#1787. Closes the reports tracked in #1774 and #1778.
Exports and every other native-picker action 403'd with "Invalid or expired desktop authorization" whenever Tauri and the backend resolved different data directories — a dev backend spawned without the OMNIVOICE_* env, a custom data folder, or portable mode. Tauri now takes the directory the running backend advertises, so the two processes cannot disagree, falling back to its own resolution when the backend is unreachable. One resolver covers all six capability kinds. Also decodes JSON escapes when reading that field, which Windows paths depend on, requires an absolute path so a relative data dir cannot recreate the same split, and keeps the 403 and its log line free of filesystem paths. Fixes#1781.
Synchronize VoiceStudio release metadata, lockfiles, installers, container references, documentation, and the dated v0.5.2 changelog after all planned fixes landed.
Prepare the tested main branch for the v0.5.1 patch release with synchronized version sources, mirrors, lockfiles, release notes, and install guidance.
Closes#1687.
Classify and journal local and remote streaming generation failures, return actionable scrubbed guidance, and keep exception details, tokens, and user paths out of logs and NDJSON responses.
PUT/DELETE /profiles (rename, delete, revoke consent) and the history/export
mutators are sync FastAPI endpoints: their bodies run in threadpool workers
where asyncio.get_running_loop() raises, so event_bus.emit() hit the
RuntimeError branch and silently dropped the "profiles" event. The UI only
refetches the voice list on that event, so after a rename the list kept stale
state, and a reload during that window could land on an empty panel (no
retry on the initial load either) — which reads to a user as "all my voices
are gone" even though nothing was deleted.
emit() now captures the serving loop in subscribe() and hands off from
foreign threads via call_soon_threadsafe (async callers are unchanged).
Also: the initial list loads in useAppData retry until FIRST success via
retryInitialLoad — a WS-triggered reload failure still keeps the previous
list, but the first load has nothing to keep. Loaders gained {rethrow: true}
for the initial path so the retry actually engages (they swallow errors by
design elsewhere); an integration test pins that wiring.
Tests: tests/test_event_bus_thread_emit.py fails on the old emit (verified
by stashing the fix) and passes with it; a live two-instance probe confirmed
PUT rename → WS event arrives on the fixed build and never on the original.
* feat(settings): compute-device override — auto | CUDA | ROCm | XPU | MPS | CPU
Auto-detect stays the default; the override kills the 'auto-detect picked
wrong' issue class. Applied at the single choke point (_probe()'s family
selection) so routing, get_best_device(), and every badge inherit it.
Resolution: OMNIVOICE_DEVICE env > Settings pick (prefs.json) > auto (#981
pattern). An override can steer, never invent hardware: a family the host
lacks is noted and ignored; cpu is always honorable. Applies at next
backend start (host caps are immutable per process — same restart contract
as the rest of the Performance tab, RestartBadge shown).
GET/PUT /api/settings/compute-device (admin-gated) reports resolved vs
applied so the panel shows restart-required truthfully and disables itself
under an env pin instead of pretending.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(changelog): entry for the compute-device override (#1557)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(device-override): harvest — the override reaches CT2 ASR, full i18n, honest edge states
- _ctranslate2_cuda_ok() and the ASR sidecar now gate on the probe's family,
so a cpu pin (or ROCm host) can never hand CTranslate2 a CUDA device —
the override reaches every CT2 loader through one shared gate
- override_ignored exposed by the API and shown by the panel (env pin naming
a device this machine lacks: auto is in effect, restart won't change it)
- all 8 panel strings + 5 device-family labels translated into all 21
locales; failed saves keep their error visible through the re-sync
- test isolation: cleanup drops OMNIVOICE_DEVICE before re-probing so no
overridden caps leak into later tests; panel tests wait for loaded state
- xpu/intel search keywords; oxfmt formatting
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(device-override): round 2 — fail-safe probe fallbacks, complete i18n, combined pin state
- a broken capability probe now means CPU everywhere (CT2 gate + ASR
sidecar) — never a torch-derived guess that would bypass a cpu pin or
re-open #1529 on ROCm; regression test added
- env-pinned AND not-detected shows both facts in one subtitle
- device_load_failed/perf_save_failed translated into all 21 locales;
CJK/th/vi/ar strings no longer say literal 'Auto'
- test_ctranslate2_never_gets_cuda_on_a_rocm_build pins the probe family
(it was order-dependent on the lru_cache before)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(test): pin the probe family in the faster-whisper OOM-fallback test
Same class as the rocm-build test: it mocked torch but not the probe the
new override gate consults first, so on a cpu-family CI host the CUDA
fallback chain under test was unreachable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(startup): bind the socket in ~1s and narrate startup step by step
The structural fix for the "can't reach the local backend" class (~1 in 5
of every issue ever filed): uvicorn served nothing until torch import
(10-20s cold), the 30-router fan-out, an import-time DB migration, the
cuDNN preload, and alembic all finished — every slow or fragile step
rendered as an unexplained dead backend.
main.py now keeps module scope fast and defers the heavy work:
- _phase_a_build (executor thread): prefs/env restore + #963 migration,
yt-dlp overlay, cuDNN preload, torchaudio, model_manager, router
imports — order preserved, literal imports so PyInstaller still traces.
- _phase_a_finalize (event loop, no awaits → atomic wrt requests):
include_router, mounts, MCP, SPA, openapi bust.
- _phase_b: the old lifespan startup body; handles on app.state so
shutdown survives a startup that never finished.
- Eager mode (pytest / OMNIVOICE_EAGER_INIT=1) runs everything at import
— byte-equivalent behavior for the ~100 lifespan-less TestClient sites
and for embedders (dump_api_routes, probe boot runner opt in).
While starting: /health answers 503 with the current step, new
/startup/progress serves the full ledger (always 200), and
StartupGateMiddleware 503s everything else with the [starting] marker
(same skip-the-Report-button convention as [shutting_down]). A deferred
failure keeps import-crash semantics: traceback to stderr → shell crash
forensics, run sentinel stays uncleared, exit 1 names the failed step.
Shell: startup_progress() probe (marker-header-gated so a foreign
responder can't narrate the splash) feeds per-step log lines into the
launch poll and the supervisor's reconnect wait. --health-check absorbs
the deferred init (60→180s); --diagnose runs Phase A up front so it
still sees restored prefs. Docker HEALTHCHECK semantics unchanged
(curl -f fails on 503 exactly as it did on connection-refused).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(startup): join the Phase A thread on shutdown; async fail-path sleep
Bot-review harvest on #1550: cancelling the deferred-startup task cannot
stop the executor thread inside Phase A's blocking imports — shutdown now
waits (bounded, only when a build started and hasn't finished) on a
thread-completion event so interpreter teardown can't race a mid-import
(#1000 class). The failure path's last-poll beat is now awaited, not
time.sleep — a blocking sleep froze the very loop that beat exists to let
serve. Also: dump_api_routes forces eager (assignment, not setdefault),
and the integration test's child gets DEVNULL instead of an undrained
pipe that could wedge a cold boot.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(startup): close the Phase A submission race; CodeQL nits
Review finds on #1550: shutdown could sample _phase_a_started unset
while the executor callable was queued-but-not-running, skipping the
thread join. started is now set BEFORE submission, the submission is
shielded so a cancel can't strand a queued callable that would never set
_phase_a_finished, and the wrapper sets finished on every exit including
the already-built early return. Contract pinned by
test_phase_a_thread_join_contract. Plus explanatory comments on the new
bare excepts and a consistent return in the gate's websocket branch.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
resolve_within split candidate paths on os.sep alone. Windows accepts /
as a real separator but os.sep is \ there, so a persisted sub-path such
as "job_123/out.mp4" stayed a single component, failed the
basename-equality check, and raised UnsafePath — while the identical
value split cleanly and resolved on POSIX. A data directory written on
Linux or by the Docker deployment and then opened by the Windows desktop
app hit exactly that.
Split on both separator families instead, which is what the comment
above the split already states the code intends. This is not a
loosening: every component still goes through the same basename / "." /
".." / empty rejection, and the commonpath containment check and symlink
resolution below are unchanged. POSIX behaviour is unchanged too — a
backslash is already rejected there as a foreign separator before the
split runs.
This also restores real coverage of the symlink-escape guard on Windows.
test_resolve_within_rejects_symlink_escape asserts through
"link/secret.wav", which previously raised at component validation
before reaching the containment check it exists to cover, so it passed
for the wrong reason. It now matches on the reason.
* chore: bump to 0.5.0
Owner-requested minor bump. package.json is the source of truth; the
three mirrors (Cargo.toml, pyproject.toml, _FALLBACK_VERSION), the three
lockfiles and the branding pin move in lockstep, and the accumulated
Unreleased section becomes the curated 0.5.0 release notes — quiet
Highlights first, one-liner subsections after, duplicates folded.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: fresh README for the VoiceStudio era; 0.5.0 notes wear the release
README: 551 lines from 656 — a What's-new-in-0.5.0 section with real
captures, the engine tables corrected to the actual 16 TTS registrations,
a stale Settings path and a broken Colab link fixed, roadmap/FAQ/credits
trimmed to what earns its place.
Release notes: the quick-switch GIF and catalogue/gallery screenshots,
captured from the running app during the pre-bump test pass, embedded
after the Highlights; #1542's gallery work and the ffmpeg CI fallback
recorded in their subsections.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: the feature inventory's Remote Model Downloads mention survives the README trim
check-docs-drift requires every docs/features.yaml name verbatim in the
README; the overhaul folded the phrase away. It now lives in the remote
workers feature line.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: one blockquote, and an architecture claim that survives the opt-ins
CodeRabbit on #1540: MD028 blank line inside adjacent blockquotes, and
'every layer is on your machine' contradicted the opt-in remote paths
documented two sections away — it now states local-by-default with the
opt-ins named.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(gallery): save gallery voices as profiles, with validated audio references
Work-in-progress lifted from the concurrent gallery session at the
owner's request (its uncommitted working tree, preserved verbatim from
base 92b1ee5d; safety snapshot remains at rescue/gallery-wip):
- gallery voices can be saved as local profiles: audio is copied into
the profile store with content-addressed filenames, existing profiles
are detected and refreshed only when the source clip changed
- backend/core/audio_validation.py: symlink-rejecting, root-contained
resolution for persisted profile WAV references, with tests
- archetype/community routers and the Voice Gallery UI updated for the
save-as-profile handoff (spec: docs/specs/longform/26-gallery-use-handoff.md)
- locale updates for the new gallery strings across all 21 files
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: drop a stray local screenshot script that rode in with the tree copy
* fix(community): explain the tolerated Content-Length parse failure; drop an unused import
CodeQL on #1542: the empty except now says why it is safe (the streamed
byte counter enforces the same cap regardless), and the test file loses
an unused Path import.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(gallery): review findings — copy outside the write lock, no stale completions
CodeRabbit on #1542, all findings addressed:
- the profile-audio copy stages to a .part temp BEFORE BEGIN IMMEDIATE
and publishes via atomic os.replace inside it — other backend writers
no longer block for the duration of an audio copy; a mid-copy failure
leaves no temp droppings and no profile row (both pinned by tests)
- VoiceGallery async ops carry per-operation generation tokens: a
preview or save-as-profile that resolves after unmount (or after a
newer operation) can no longer play audio, redirect into a workspace,
or touch state — three fail-before regression tests
- VoiceGalleryActions imports the page at test runtime; the e2e locator
uses a stable data-testid instead of a translated string; symlink
tests skip cleanly where the OS can't create symlinks; the changelog
line carries its PR ref
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci: static ffmpeg fallback when the chocolatey feed is down
Third feed outage to break a PR run (2026-07-20, 2026-07-28, today —
three attempts, three 'installed 0/1'). Chocolatey is a distribution
channel, not the dependency: after the retry loop exhausts, fetch the
static gyan.dev build from its GitHub release mirror and put it on
PATH — same binary, no feed in the path. URL verified live (HTTP 200).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(security): replace persistent admin keys with sessions
Exchange the remote administrator key once for bounded, revocable credentials. Canonicalize backend principals, enforce cookie CSRF and exact origins, and use path-bound one-use WebSocket tickets.
Migrate the bundled UI away from durable master-key storage and credential-bearing URLs. Add unit, integration, static-hygiene, and production-browser regressions plus synchronized operator documentation.
* docs: link session hardening to PR 1528
* fix(security): key session indexes with process pepper
Use HMAC-SHA-256 instead of an unkeyed digest for in-memory session and WebSocket-ticket indexes. This preserves constant-size lookup identifiers, makes copied records unusable without the process pepper, and resolves CodeQL's weak sensitive-data hash finding.
* fix(auth): align empty bearer migration precedence
Centralize the Authorization-channel presence decision with canonical principal parsing. Bearer followed only by spaces now remains an empty channel during legacy cookie migration, while unsupported or invalid explicit credentials stay authoritative and fail closed.
* fix(security): harden admin session review boundaries
* fix(security): derive key generations with HKDF
* fix(auth): anchor the admin-session store so module reloads cannot fork it
test_master_exchange_does_not_bypass_pin_on_normal_routes failed in full-suite
runs: test_mcp_bindings' client fixture purges the services.* tree from
sys.modules and reloads main, so api.routers.auth re-imported a fresh
services.admin_sessions (new AdminSessionStore) while core.auth kept its
import-time reference to the old one — the exchange issued the cookie into
one store and the middleware resolved it against another, turning the
expected "PIN required" into "API key required".
Root cause is the class of bug, not the one test: a process-global auth
store defined as a bare module-level singleton forks under importlib.reload
or purge-and-reimport. Fix at the source: admin_session_store now resolves
through a synthetic sys.modules anchor (_omnivoice_admin_session_store_anchor)
that reloads never re-execute and package-prefix purges never match, so every
copy of the module shares the one per-process store. No consumer or behavior
changes.
Regression test reproduces both fork vectors (in-place reload and
sys.modules purge + fresh import) and asserts previously issued sessions
still resolve and the store identity is preserved; it fails before this fix
and passes after.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(auth): honor X-Forwarded-Proto for CSRF origin and Secure cookies behind TLS proxies
Behind Tailscale Serve (docs/remote-gpu.md) or any TLS-terminating proxy,
the browser talks https while the backend hop stays http, so exact-origin
CSRF compared an https Origin against an http expectation and rejected
every legitimate request, and the session cookie shipped without Secure.
uvicorn's ProxyHeadersMiddleware only rewrites the scope for loopback
peers, which misses Docker and any non-loopback proxy topology.
New core.csrf.effective_scheme derives the client-facing scheme: resolved
scope first (uvicorn's trusted-proxy rewrite wins), then an upgrade-only
read of X-Forwarded-Proto's first value — https/wss promotes http to
https, everything else is ignored, and a genuine TLS hop can never be
downgraded. Used by both the destination-origin comparison and
auth._secure_cookie so the WS-ticket/logout CSRF paths and the cookie
Secure flag agree. Spoofing gains nothing: the host:port half of the
origin tuple is untouched, browsers cannot attach the header cross-site
without a preflight this API never grants, and forging it on plain http
only adds Secure (the browser then drops the cookie — self-harm only).
Regression tests: proxied https origin accepted (origin check, Secure
flag, logout), comma-separated chains, scope-fallback path, spoofed
header still rejects cross-origin, cannot downgrade real https, junk
values ignored.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(auth): consume the stored admin key only after a successful exchange
A remote-backend user upgrading with their backend unreachable lost the
only stored copy of OMNIVOICE_API_KEY: every migration path deleted the
durable ov_api_key BEFORE the session exchange settled, stranding them
until they recovered the key from the server box. Close the whole class:
- client.ts bootstrap: read the legacy key, exchange first, and remove
the durable copy only after the exchange succeeds; on failure the key
stays so the next launch retries the migration (auth gate still rises).
- authSession.ts exchangeApiKey: move removeLegacyMaster from before the
fetch to the cookie/bearer success paths — the key never coexists with
a live session, but a rejected or hung exchange no longer consumes it.
- remoteBackendProbe.ts configuredRemoteBackend: stop wiping the key on
every app mount.
- RemoteBackendPanel: a connection test or an aborted save no longer
wipes the pending key; only disabling the remote backend discards it.
- prefKeys.js: ov_api_key moves from PREF_KEYS to PRESERVED_KEYS —
factory reset preserves the pending connection credential exactly like
ov_backend_url; the successful migration is what deletes it.
Tighten the credential-hygiene static guard to match: it accepted
sessionStorage.setItem('ov_api_key', …) — the exact class it exists to
close. The guard now flags .setItem(<master key>) on any storage
receiver, quote style, or injected-store alias, with a self-test pinning
what it catches and what stays legal.
Fail-before/pass-after regression tests: backend unreachable retains the
key and the next bootstrap retries it; a successful exchange removes it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* perf(auth): make session validation occupancy-independent
* test(auth): catch optional master-key storage calls
* feat(docs): add PR control document for bultodepapas in VoiceStudio
* docs: keep the PR tracking board in the fork; credit the changelog line
The pr-control document is excellent process discipline, but it is the
contributor's own operational board (their inventory, their update
commands) — it lives naturally in their fork, and docs/agents/ here is
context every repo agent loads. Removed with appreciation; the changelog
line gains its contributor credit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: debpalash <4178343+debpalash@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* refactor(launchpad): quieter, borderless design refresh
The launchpad carried decoration from an earlier direction: icon chips,
corner-hung count badges, a permanently visible filled arrow, uppercase
mono card titles, and a dotted stipple divider — plus a frame that had
been invisible since the app-wide border tokens were zeroed.
Rework it around what the borderless direction actually implies:
- Feature tiles get a whisper-faint surface instead of a dead frame, and
read as three bands (bare glyph + count / title + arrow / description).
`--card-hue` is spent sparingly — the glyph at rest, the surface, count
and arrow only once raised. Titles move to sans sentence case; counts
are plain tabular numerals. Lift softened 4px -> 2px, coloured glow ->
neutral shadow, plus an explicit focus ring and a staggered entrance.
- Hero drops the boxed "646" pill and the filled A/B-Compare button for
quiet type, with a hairline standing in for the separation.
- Section labels trade the dotted stipple for a single fading hairline;
rows are transparent until hover and reveal "Open" on hover/focus (it
stays in the DOM, so AT and keyboard always reach it).
- Hero, tiles, recent files, callout and project lists now share one
1180px column — previously only the top half was capped, so lists ran
edge-to-edge on a wide display while the deck stayed centred.
Two bugs found and fixed while doing it:
- Buttons that had `border border-solid border-transparent` removed fell
back to the UA default border and rendered a visible 1px outline. They
now carry `border-0` explicitly.
- `.lp-animate` used `animation-fill-mode: both`, so after the entrance
it kept owning `transform` — and animation-origin declarations outrank
normal ones, which silently killed the card hover lift. Now `backwards`,
which still holds the from-state through the stagger delay.
Also drops CSS the page has not rendered since #904: the cursor-spotlight
layer, the breath ring, and the per-card waveform strip.
Verified with headless renders at 1600/1280/940 and the empty state.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(dictation): decode Wayland portal signals and show the capture pill
The GlobalShortcuts portal declares Activated/Deactivated as
(o session, s shortcut_id, t timestamp, a{sv} options). We decoded the
timestamp as u32, so zbus rejected every signal with
Signature mismatch: got `(osta{sv})`, expected `(osua{sv})`
and the press was dropped as an invalid signal. Registration succeeded
and the desktop even reported the bound chord back, so the hotkey looked
wired up while doing nothing at all — on every Wayland compositor, for
the whole life of the feature (#1490). Decode the 64-bit timestamp, and
keep the 32-bit spelling as a fallback so a non-conforming portal
degrades to working rather than to silence.
With presses arriving, the second half of the failure showed: nothing
had shown the widget window since it became a hidden recorder host, so a
capture ran with no pill on screen — and a mic or Accessibility failure
rendered into a window nobody could see. Add show_dictation_pill, which
bottom-centres the capsule on the monitor under the pointer and shows it
without taking focus (Windows keeps SW_SHOWNOACTIVATE so paste still
lands in the user's document), and call it from the widget for every
state but idle. Wayland denies clients their own placement, so the
compositor picks the spot there; the pill still appears.
dispatch_dictation_capture now logs whether a press was emitted or
queued — a press that reaches Rust and produces nothing was otherwise
indistinguishable from one the compositor never delivered.
Tests: portal signals decode at both timestamp widths (the 64-bit case
fails before this change with the exact production error); pill
placement centres, respects a second monitor's origin, and clamps rather
than going off-screen; the widget shows for a state needing the user,
stays hidden while idle, and never shows for a press that arrives while
dictation is disabled.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: sync in-progress workspace changes
Uncommitted work already in the tree, checkpointed so the branch matches
the local machine:
- Remote GPU workers: join-from-the-app flow, one-time secrets, QR join
codes, a Compute control in the status bar, and the device-list
Workers panel (#1516)
- Model Catalogue workspace, with Settings pointing at it
- Settings sidebar search and keyboard navigation
- Demo assets for dubbing, dictation and voice design, plus the scripts
that render them
- Backend: validation-error handling, ASR request-path degradation, and
the accompanying tests
- CHANGELOG entries for the above and for the Wayland dictation fix
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(tests): follow Engines to the Model Catalogue, and green the sweep
- test_supertonic3 asserted the license gate points at "Settings" while
the engine now names Model Catalogue → Engines, which is where the
accept button actually lives. The assertion follows the move; what it
pins is unchanged — the hint must name a place the user can reach it.
- Carries the CJK allowlist entries for the rendered dub bundle (#1517)
and the regenerated route snapshot for /workers/agent (#1516), both of
which this branch inherits from the workspace sync.
- docs/install/linux.md: the dictation capsule is bottom-anchored
everywhere except Wayland, where the protocol gives applications no
say in their placement. Documented rather than left as a surprise
(CodeRabbit).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* ci: stop a flaky dependency fetch from failing green runs
en-core-web-sm resolves to a direct GitHub release URL, and github.com
intermittently answers `http2 error: refused stream before processing
any application logic`. uv's own three retries all land within the same
few seconds and fail together, so the whole job dies on a dependency
that has nothing to do with the change under test — it cost #1518 and
#1517 an otherwise-green run tonight.
Two changes: back off between whole `uv sync` attempts, which is what
actually clears it, and pass --no-sync to the pytest steps. `uv run`
re-resolves the environment before running, so every test step was a
fresh chance to hit the same fetch even though the install step had
already synced — that is exactly how #1518 failed, in the isolated
backend/tests step, with all 5467 tests already passed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* ci: one retry seam for every uv sync, not just the job that failed last
en-core-web-sm resolves to a direct GitHub *release* URL rather than a
package index, and github.com intermittently answers `http2 error:
refused stream before processing any application logic`. uv's own
retries all land inside the same ~10 seconds and fail together, so a job
dies on a dependency unrelated to the change under test. Tonight that
cost four otherwise-green runs across #1515, #1517 and #1518 — and the
first fix only covered the Tests job, so the next failure simply moved
to Smoke (Linux), which syncs separately.
The fetch is per-job, so the fix has to be per-job: scripts/uv-sync-retry.sh
backs off between whole attempts (15s, 45s, 90s) and every workflow that
syncs now goes through it — ci.yml (tests + the platform matrix),
release.yml, security.yml, evals.yml. It still fails loudly after four
attempts, so a genuinely broken lockfile is not disguised as a flake.
The Tests job also lacked the UV_HTTP_TIMEOUT / UV_HTTP_RETRIES the smoke
matrix has always set, which is part of why it was the one that kept
dying; it has them now.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(ci): pin the Intel-Mac contract by intent, not by command spelling
test_ci_verifies_intel_mac_as_the_documented_remote_only_host asserted
the literal line `run: uv sync --extra pockettts`, so routing every sync
through scripts/uv-sync-retry.sh read as a broken Intel-Mac contract. The
contract it exists to protect is that the pockettts extra installs ONLY
on backend_supported legs — which the regex now pins, while leaving how
the sync is invoked free to change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* ci: keep every uv run out of the resolver, and bound the retry budget
CodeRabbit, #1517:
- `uv run` re-resolves before running, so the smoke suite, the
worker-artifact tests, the release test run and the eval run were each
a fresh chance to hit the flaky direct-URL fetch outside the retry
loop. All of them pass --no-sync now; the environment is already
synced by the step that owns the retries. security.yml's
`uv run --with pip-audit` is deliberately left alone — it layers an
ephemeral package rather than running the project's own tests.
- The retry count multiplied uv's own budget (UV_HTTP_RETRIES=5 with a
120 s timeout on the smoke matrix). Three attempts and 60 s of total
backoff outlast the refusals actually observed while staying well
inside the jobs' timeout-minutes.
- The Intel-Mac contract test pinned the smoke command literally too, so
--no-sync tripped it exactly like the sync line did. Same fix: assert
the contract (smoke runs only on backend_supported legs), not its
spelling.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three conflicts, all additive on both sides — resolved by keeping both
rather than choosing, since either side's entries were real shipped work:
* CHANGELOG.md — remote-GPU entries against branding, IndexTTS 2.5 and
the recording-input work
* setup/download.py — the per-target progress reset against main's
active-install tracking; both belong in the same finally block
* docs/features.yaml — the remote-worker and model docs against
docs/branding.md
Backend 5349 passed, frontend 1871 passed. `bun install --frozen-lockfile`
reports no changes, so the Docker build sees the same tree CI does.
The remote-GPU line, verified on hardware rather than asserted.
**Dubbing renders on the worker.** dub_generate.py dispatches the coarse
`dub_segments` operation through the gateway, following the audiobook
pattern: per-unit local fallback after consecutive remote failures, one
aggregated notice rather than one per segment. A 40-minute dub that loses
its worker at segment 200 degrades instead of producing 200 error rows.
**An out-of-date worker is now refused by name.** This was the worst
defect in the plan and it was silent: an un-upgraded worker registered
cleanly, then ignored `inputs` and rendered a clone with NO reference
audio — returned as success. A plausible wrong result with nothing
anywhere to surface it. Workers now declare features, and one missing
them is turned away with the features named and `no task was run`.
Verified live: a worker one commit behind was correctly refused.
**"Offline" and "cannot run this" are different facts.** Asking a live
worker for an engine it lacks answered "is offline or cannot be reached.
Wake the selected worker" — while that worker reported ready, one free
slot and 3.6 ms latency. The user was sent to wake a machine that was
already awake. The scheduler now distinguishes absent from present-but-
incapable, and names the engine rather than the operation, because the
engine is the thing a user can install.
**An engine with no catalog entry is no longer hidden.** A `repo_ids`
non-emptiness check had been implemented as a runtime filter, so a worker
silently refused to advertise any engine lacking a models.yaml entry —
which is four registered engines, including CosyVoice. Users with those
already installed would have lost remote support with only a log line.
Empty `repo_ids` now means "not downloadable here", never "not runnable".
**And a script so this stops being done by hand.**
scripts/verify-remote-worker.sh runs the per-phase acceptance checks
against a live worker, non-destructively. Its preconditions are the
mistakes that cost the most time: exactly one listener on the control
port (two instances silently shared it), and never detecting the worker
with a pgrep pattern that matches the ssh shell running it.
Its first real run found the dubbing picker claiming remote placement.
That turned out to be the CHECK being stale, not the picker — the port
had landed since it was written. It now asserts self-consistency instead:
the picker may claim remote only for an operation the control plane
actually advertises as remotely producible, which cannot rot the next
time an op is ported.
Backend 5291 passed, frontend 1812 passed. Acceptance script: no
automated failures across Phases 4-8 on an RTX 4090. Four checks remain
MANUAL by design — true airplane mode, concurrent downloads, killing a
worker mid-audiobook, and the model-list UI — and are reported as
unverified rather than passed.
Three phases that only make sense together: a job that names a worker,
a worker that reports honestly what it can actually run, and the small
defects that made both lie.
**Pinning** (Phase 1). `pinned_worker_id` is now honoured in both places
that choose a worker — `eligible_workers` and `select_worker` build
independent lists, so applying it to one silently leaked work onto
whichever machine was least busy. The pin persists across a restart via
an additive column, deliberately not alembic (justified in the code, per
the precedent already in db.py): quitting mid-render used to drop it
without a word. `max_attempts=1` was rejected as the mechanism — it makes
the FIRST failure terminal, including the penalty-free ones a stale
advisory view produces routinely.
Cancel now actually reaches the worker. `WorkerServicer.cancel` had zero
callers, so cancelling released the slot while the GPU thread kept
running, and a late result could resurrect the task as COMPLETED —
`commit_result` assigned that state directly, bypassing the transition
table where CANCELLED is terminal by construction.
**Honest capabilities** (Phase 4). A worker now probes whether weights
are actually present, and a job stops BEFORE dispatch with a typed 409
naming the model and the machine, instead of failing mid-task. The probe
fails OPEN: `is_cached`/`cache_is_complete` cannot see a user-managed
clone outside the HF layout, so only a positive "absent" refuses.
Refusing an engine that works today would break the compatibility
promise. `pool.supports` deliberately still ignores `downloaded` — had it
not, the scheduler would drop the worker and answer with a terminal
NO_CAPABLE_WORKER, which tells the user to check their install when the
truth is one download away. The frontend no longer offers "Report this
bug" for that state; it offers the download.
Catalog tags resolve against the TARGET's OS/arch/backend, not this
machine's. From a Mac control plane, a CUDA worker's model list was
showing the mlx-community repos it cannot run and hiding the ones it
needs.
**And the quiet ones** (Phase 0 leftovers): a model's human label rides
its own proto field so renaming it cannot orphan breaker history; an
empty model_id no longer forks the capacity slot key into two slots for
one model; the idle sweep cannot evict an engine out from under a live
LOCAL render.
Verified on real hardware, which is the only verification that has ever
caught anything here: 2025 characters, default settings, routed to an
RTX 4090 over the wire — 100% GPU utilisation on the remote box, 119.6 s
of 24 kHz audio returned in 16.6 s, 5.7 MB delivered out of band through
the artifact path rather than the control stream.
Backend 5259 passed, frontend 1808 passed.
Palash Debnath