upstream-archive/VoiceStudio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: preserve release notes and qualify packaged Metal availability

Browse Source
This commit is contained in:
Palash Debnath
2026-09-17 12:18:02 +05:30
parent be1be89a69 18c543484b
commit a9d1464acb
154 changed files with 5235 additions and 2191 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.claude/skills/omnivoice/SKILL.md → .claude/skills/omnivoice/LEGACY.md
+6 -6
View File
@@ -1,11 +1,6 @@
---
name: omnivoice
description: Legacy VoiceStudio skill alias for existing Claude installations. Generate local speech, discover saved voices, and transcribe audio through the running VoiceStudio backend.
---
# VoiceStudio compatibility entry
The current cross-agent package is [voicestudio](../../../skills/voicestudio/SKILL.md).
The current cross-agent package is [voicestudio](../../../../skills/voicestudio/SKILL.md).
For new installations use `npx skills add debpalash/VoiceStudio --skill voicestudio`.
Use the running backend at the user's configured address (default
@@ -27,3 +22,8 @@ never disable authentication to make an example work.
Source and current setup documentation:
https://github.com/debpalash/VoiceStudio
This archived entry is not an installable skill. Existing installations should
remove the old `omnivoice` / `oss-maintainer` entries and install `voicestudio` /
`voicestudio-maintainer` from the canonical repository. Legacy helpers remain
for existing users; the Electron supervisor is the preferred launcher.
.github/CONTRIBUTING.md
+20 -10
View File
@@ -56,7 +56,17 @@ bun install
bun run dev
```
This starts both services:
This launches Electron with hot reload. Its runtime supervisor manages backend setup
and startup; do not launch a second backend. See [Electron setup](../electron/README.md).
```bash
bun run build # build Electron
bun run start # launch the built Electron app
bun run dist # package locally without publishing
bun run dev:web # legacy browser UI + backend
```
The legacy browser command starts both services:
| Service | URL | What it does |
|---------|-----|---|
@@ -71,29 +81,29 @@ cause doesn't scroll away with the terminal. The same death is also reported
as a crash notice in the UI the next time the backend starts (see
[docs/install/troubleshooting.md §14c](docs/install/troubleshooting.md)).
### Desktop App (Tauri)
### Legacy Desktop App (Tauri)
```bash
bun run desktop # dev: hot-reload Tauri shell + backend
bun run desktop-prod # production: builds, bundles the backend, then launches
bun run tauri # legacy dev: hot-reload Tauri shell + backend
bun run tauri:desktop-prod # legacy production: builds, bundles the backend, then launches
```
Both run `uv sync` first (so the Python backend env is set up) and start the
backend automatically — you do **not** start it separately. Use the exact script
names: there is no `desktop=prod` (note the **hyphen** in `desktop-prod`).
`desktop-prod` is Windows-aware (auto-detects bash/git; see `scripts/desktop-prod.mjs`).
names: there is no `desktop=prod` (note the **hyphen** in `tauri:desktop-prod`).
`tauri:desktop-prod` is Windows-aware (auto-detects bash/git; see `scripts/desktop-prod.mjs`).
Requires [Rust](https://rustup.rs/) and platform-specific Tauri dependencies — see the [Tauri prerequisites](https://v2.tauri.app/start/prerequisites/).
After installing Rust with rustup (or `uv` with its installer), a terminal that
was already open still has the old `PATH`. The desktop launchers (`bun desktop`,
`bun desktop-prod`, `bun desktop-fresh`) detect this and add `~/.cargo/bin` /
was already open still has the old `PATH`. The desktop launchers (`bun tauri`,
`bun tauri:desktop-prod`, `bun tauri:desktop-fresh`) detect this and add `~/.cargo/bin` /
`~/.local/bin` for that run, printing a one-line note; to make it permanent,
open a new terminal, or on macOS/Linux load Cargo into the current one:
```bash
source "$HOME/.cargo/env"
bun desktop
bun run tauri
```
If Rust is genuinely not installed, the launchers stop up front with the
@@ -310,7 +320,7 @@ that — the agent recalls the architecture, conventions, and your past findings
instead of re-reading the tree each time. [**memxt**](https://github.com/debpalash/memxt)
(100% local, MCP-based, built by this project's maintainer) exists for exactly
this; any MCP memory server works. Pair it with the repo's agent skill —
`npx skills add debpalash/omnivoice-studio` — so your agent knows the project's
`npx skills add debpalash/VoiceStudio` — so your agent knows the project's
hard rules from the first prompt.
## Quality gates your PR must pass
.github/workflows/electron-release.yml
+242
View File
@@ -0,0 +1,242 @@
name: Electron desktop release
# Builds are safe by default. Only an explicit publish dispatch exposes a release.
on:
push:
tags: ['v*']
workflow_dispatch:
inputs:
release_tag:
description: "Existing version tag to package using this workflow from main (optional)"
type: string
default: ''
publish:
description: "Publish the tagged Electron release after all platforms pass"
type: boolean
default: false
allow_unsigned:
description: "Explicitly accept unsigned/unnotarized Electron installers and documented updater limitations"
type: boolean
default: false
permissions:
contents: read
concurrency:
group: electron-release-${{ inputs.release_tag || github.ref_name }}
cancel-in-progress: false
env:
RELEASE_REF: ${{ inputs.release_tag && format('refs/tags/{0}', inputs.release_tag) || github.ref }}
jobs:
validate:
# The transition tag is assembled after the manual Tauri draft succeeds.
if: github.event_name == 'workflow_dispatch' || github.ref_name != vars.TAURI_SUNSET_TAG
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ env.RELEASE_REF }}
- name: Require an exact version tag
env:
REF: ${{ env.RELEASE_REF }}
WORKFLOW_REF: ${{ github.ref }}
RELEASE_TAG_OVERRIDE: ${{ inputs.release_tag }}
ALLOW_UNSIGNED: ${{ inputs.allow_unsigned }}
DISPATCH_ACTOR: ${{ github.actor }}
RERUN_ACTOR: ${{ github.triggering_actor }}
OWNER: ${{ github.repository_owner }}
run: |
if [ "$ALLOW_UNSIGNED" = true ]; then
test "$DISPATCH_ACTOR" = "$OWNER" && test "$RERUN_ACTOR" = "$OWNER" || {
echo "Only the repository owner may accept unsigned installers"; exit 1;
}
fi
if [ -n "$RELEASE_TAG_OVERRIDE" ]; then
test "$WORKFLOW_REF" = refs/heads/main || { echo "Tag overrides require the workflow from main"; exit 1; }
fi
VERSION=$(node -p "require('./frontend/package.json').version")
test "$REF" = "refs/tags/v$VERSION" || { echo "Select the exact version tag"; exit 1; }
test "$(git rev-parse HEAD)" = "$(git rev-parse "$REF^{commit}")" || { echo "Checkout does not match the release tag"; exit 1; }
package:
needs: validate
runs-on: ${{ matrix.runner }}
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
include:
- runner: ubuntu-24.04
platform: linux
arch: x64
target: x86_64-unknown-linux-gnu
flags: --linux --x64
- runner: windows-2022
platform: win32
arch: x64
target: x86_64-pc-windows-msvc
flags: --win --x64
- runner: macos-15
platform: darwin
arch: arm64
target: aarch64-apple-darwin
flags: --mac --arm64
- runner: macos-15-intel
platform: darwin
arch: x64
target: x86_64-apple-darwin
flags: --mac --x64
defaults:
run:
shell: bash
env:
VOICESTUDIO_RUST_TARGET: ${{ matrix.target }}
VOICESTUDIO_UPDATE_CHANNEL: electron-stable-${{ matrix.platform }}-${{ matrix.arch }}
CSC_IDENTITY_AUTO_DISCOVERY: 'false'
steps:
- uses: actions/checkout@v4
with:
ref: ${{ env.RELEASE_REF }}
- uses: actions/setup-node@v4
with:
node-version: '22'
- uses: oven-sh/setup-bun@v2
with:
bun-version: '1.4.2'
- uses: dtolnay/rust-toolchain@6bed0761d98439e5a578e2877258200ad565ba87 # stable
with:
targets: ${{ matrix.target }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: native/desktop-bridge -> target
key: electron-${{ matrix.target }}
- uses: astral-sh/setup-uv@v6
with:
version: '0.12.13'
enable-cache: false
- name: Linux native dependencies
if: runner.os == 'Linux'
run: |
sudo apt-get update
sudo apt-get install -y libasound2-dev libxdo-dev libxtst-dev libx11-dev libxkbcommon-dev libwayland-dev libssl-dev pkg-config xvfb
- name: Bundle pinned uv for the host architecture
run: |
node --input-type=module <<'NODE'
import { execFileSync } from 'node:child_process';
import { mkdirSync, copyFileSync, chmodSync } from 'node:fs';
import { join } from 'node:path';
const expected = process.env.VOICESTUDIO_RUST_TARGET;
const targets = { 'linux-x64': 'x86_64-unknown-linux-gnu', 'win32-x64': 'x86_64-pc-windows-msvc', 'darwin-arm64': 'aarch64-apple-darwin', 'darwin-x64': 'x86_64-apple-darwin' };
if (targets[`${process.platform}-${process.arch}`] !== expected) throw new Error('Runner architecture does not match package target');
const source = execFileSync(process.platform === 'win32' ? 'where.exe' : 'which', ['uv'], { encoding: 'utf8' }).trim().split(/\r?\n/)[0];
const dir = 'frontend/src-tauri/binaries';
mkdirSync(dir, { recursive: true });
const destination = join(dir, `uv-${expected}${process.platform === 'win32' ? '.exe' : ''}`);
copyFileSync(source, destination);
if (process.platform !== 'win32') chmodSync(destination, 0o755);
NODE
- name: Install locked dependencies
run: bun install --frozen-lockfile
- name: Validate and build Electron
run: bun run check:electron
- name: Package without publishing
env:
CSC_LINK: ${{ secrets.ELECTRON_CSC_LINK }}
CSC_KEY_PASSWORD: ${{ secrets.ELECTRON_CSC_KEY_PASSWORD }}
working-directory: electron
run: |
# An empty CSC_LINK is interpreted as the working directory by the
# signer. Omit absent credentials rather than passing empty strings.
if [ -z "${CSC_LINK:-}" ]; then
unset CSC_LINK CSC_KEY_PASSWORD
fi
bun x electron-builder --config electron-builder.config.mjs ${{ matrix.flags }} --publish never
node tests/packaging-contract.mjs --artifact
node tests/update-package-contract.mjs --platform ${{ matrix.platform }} --arch ${{ matrix.arch }}
- name: Verify macOS signing and notarization before publication
if: inputs.publish == true && inputs.allow_unsigned != true && matrix.platform == 'darwin'
run: |
APP=$(find electron/release -maxdepth 2 -name VoiceStudio.app -type d -print -quit)
test -n "$APP"
codesign --verify --deep --strict "$APP"
spctl --assess --type execute --verbose=2 "$APP"
- name: Verify Windows installer signature before publication
if: inputs.publish == true && inputs.allow_unsigned != true && matrix.platform == 'win32'
shell: pwsh
run: |
$installers = @(Get-ChildItem electron/release/VoiceStudio-Electron-*.exe)
if ($installers.Count -eq 0) { throw "No installer to verify" }
foreach ($installer in $installers) {
$signature = Get-AuthenticodeSignature $installer.FullName
if ($signature.Status -ne 'Valid') { throw "Installer signature is not trusted: $($installer.Name)" }
}
- name: Packaged startup smoke test
working-directory: electron
run: |
if [ "$RUNNER_OS" = Linux ]; then
xvfb-run -a node tests/packaged-smoke.mjs --setup
else
node tests/packaged-smoke.mjs --setup
fi
- name: Save installers and updater metadata for review
uses: actions/upload-artifact@v4
with:
name: electron-release-${{ matrix.platform }}-${{ matrix.arch }}
retention-days: 14
if-no-files-found: error
path: |
electron/release/VoiceStudio-Electron-*
electron/release/electron-*.yml
release:
needs: package
runs-on: ubuntu-latest
permissions:
contents: write
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAG: ${{ inputs.release_tag || github.ref_name }}
SUNSET_TAG: ${{ vars.TAURI_SUNSET_TAG }}
PUBLISH: ${{ inputs.publish }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ env.RELEASE_REF }}
- uses: actions/download-artifact@v4
with:
pattern: electron-release-*
merge-multiple: true
path: release-assets
- name: Validate all platforms before creating a release
run: |
python3 scripts/prepare_electron_release.py --assets release-assets --tag "$TAG"
- name: Preserve the final Tauri updater feeds
run: |
test -n "$SUNSET_TAG" || { echo "Set TAURI_SUNSET_TAG before releasing"; exit 1; }
# The transition tag already holds its own final Tauri feeds.
# Later releases carry copies pointing to the immutable sunset payloads.
gh release download "$SUNSET_TAG" --pattern latest.json --dir release-assets
gh release download "$SUNSET_TAG" --pattern latest-user.json --dir release-assets
python3 scripts/prepare_electron_release.py --assets release-assets --tag "$TAG" --sunset-tag "$SUNSET_TAG"
- name: Disclose explicitly accepted unsigned artifacts
if: inputs.allow_unsigned == true
run: |
cat >> release-assets/RELEASE_NOTES.md <<'EOF'
### Electron installer trust
These Electron installers are unsigned or ad-hoc signed and are not Apple-notarized.
Windows/macOS may show trust warnings. macOS automatic updates are unverified;
use manual installer updates. Tauri updater signatures remain independently verified.
EOF
- name: Create or update draft
run: |
if ! gh release view "$TAG" >/dev/null 2>&1; then
gh release create "$TAG" --verify-tag --draft --title "$TAG — VoiceStudio" --notes-file release-assets/RELEASE_NOTES.md
fi
test "$(gh release view "$TAG" --json isDraft --jq .isDraft)" = true || { echo "Refusing to replace a published release"; exit 1; }
gh release edit "$TAG" --notes-file release-assets/RELEASE_NOTES.md
find release-assets -maxdepth 1 -type f ! -name RELEASE_NOTES.md -print0 | xargs -0 gh release upload "$TAG" --clobber
- name: Publish only when explicitly requested
if: github.event_name == 'workflow_dispatch' && inputs.publish == true
run: gh release edit "$TAG" --draft=false --latest
.github/workflows/release.yml
+29 -19
View File
@@ -27,27 +27,22 @@
# each to surface PyInstaller/Tauri issues that never showed up locally on
# macOS — iterate on CI.
name: Desktop Release
name: Tauri sunset (manual only)
# Legacy workflow: run once on the final Tauri version tag.
# Electron releases are owned by electron-release.yml.
on:
push:
tags: ['v*']
schedule:
# 07:00 UTC daily — rolling `preview` prerelease from `main`. The
# preview-gate job no-ops the matrix when main hasn't moved in a day.
- cron: '0 7 * * *'
workflow_dispatch:
inputs:
draft:
description: "Create as draft release (tag push only)"
required: false
description: "Keep the final Tauri release draft until Electron artifacts are ready"
default: "true"
publish_preview:
description: "Publish a rolling 'preview' prerelease (updater Preview channel). Previews ALWAYS build from main — dispatching from any other branch fails the preview-gate."
required: false
description: "Legacy compatibility input; previews are retired"
type: boolean
default: false
permissions:
contents: write # needed to attach artifacts + updater manifest to GH Release
@@ -76,6 +71,16 @@ jobs:
name: Tests (backend + frontend)
runs-on: ubuntu-22.04
steps:
- name: Require the designated final Tauri tag
env:
SUNSET_TAG: ${{ vars.TAURI_SUNSET_TAG }}
REF: ${{ github.ref }}
PREVIEW: ${{ inputs.publish_preview }}
run: |
test -n "$SUNSET_TAG" || { echo "Set TAURI_SUNSET_TAG to the final v* tag first"; exit 1; }
test "$REF" = "refs/tags/$SUNSET_TAG"
test "$PREVIEW" != "true"
- uses: actions/checkout@v4
- name: Setup Python 3.11
@@ -476,6 +481,9 @@ jobs:
fi
{
echo 'body<<RELEASE_BODY_EOF'
echo '## Final Tauri update'
echo 'VoiceStudio desktop is moving to Electron. This is the last Tauri release. Back up your data and install Electron separately: https://github.com/debpalash/VoiceStudio/blob/main/docs/electron-migration.md'
echo
echo "$BODY"
echo 'RELEASE_BODY_EOF'
} >> "$GITHUB_OUTPUT"
@@ -667,6 +675,7 @@ jobs:
includeUpdaterJson: true
- name: Build + publish Electron desktop
if: false # Electron is released independently by electron-release.yml.
shell: bash
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@@ -973,7 +982,7 @@ jobs:
# Writes SHA256SUMS-<label>.txt, attached to the release below. The
# release-notes-checksums job puts every leg's file into the notes.
- name: Compute SHA-256 checksums
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
if: startsWith(github.ref, 'refs/tags/v') && (github.event_name == 'push' || github.event_name == 'workflow_dispatch')
id: checksums
shell: bash
run: |
@@ -1032,7 +1041,7 @@ jobs:
# decision, so both belong to the single release-notes-checksums job
# that runs after the whole matrix (see there for why).
- name: Attach SHA256SUMS file
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
if: startsWith(github.ref, 'refs/tags/v') && (github.event_name == 'push' || github.event_name == 'workflow_dispatch')
shell: bash
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@@ -1058,7 +1067,7 @@ jobs:
# missing, so a failed platform leaves the release a draft.
release-notes-checksums:
needs: [build, repair-updater-manifest, uninstall-scripts]
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
if: startsWith(github.ref, 'refs/tags/v') && (github.event_name == 'push' || github.event_name == 'workflow_dispatch')
runs-on: ubuntu-22.04
timeout-minutes: 10
permissions:
@@ -1067,6 +1076,7 @@ jobs:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ github.repository }}
TAG: ${{ github.ref_name }}
KEEP_DRAFT: ${{ inputs.draft }}
steps:
- name: Write every platform's checksums into the notes, then publish
shell: bash
@@ -1091,7 +1101,9 @@ jobs:
done
[ "$missing" = 0 ] || exit 1
gh release edit "$TAG" --repo "$REPO" --notes-file "$WORK/notes.md"
if [[ "$TAG" == *-* ]]; then
if [[ "$KEEP_DRAFT" == "true" ]]; then
echo "Final Tauri draft verified; Electron publication owns the transition."
elif [[ "$TAG" == *-* ]]; then
gh release edit "$TAG" --repo "$REPO" --draft=false --prerelease
else
gh release edit "$TAG" --repo "$REPO" --draft=false --latest
@@ -1121,9 +1133,7 @@ jobs:
# can leave the in-app updater with four 404 feeds.
electron-publish-contract:
needs: [build, preview-gate]
if: >-
needs.build.result == 'success' &&
(needs.preview-gate.outputs.is_preview == 'true' || startsWith(github.ref, 'refs/tags/v'))
if: false # Electron release workflow owns this contract.
runs-on: ubuntu-22.04
permissions:
contents: read
@@ -1200,7 +1210,7 @@ jobs:
uninstall-scripts:
needs: [build]
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
if: startsWith(github.ref, 'refs/tags/v') && (github.event_name == 'push' || github.event_name == 'workflow_dispatch')
runs-on: ubuntu-22.04
permissions:
contents: write
CHANGELOG.md
+44 -1
View File
@@ -10,6 +10,20 @@ the frozen-backend fallback mirror it for their toolchains.
**Highlights**
- Electron packaging can recover without changing a release tag
### CI
- Handle missing Electron signing credentials and retry packaging fixes without moving release tags (#2157)
### CI
- Require the pinned Apple Silicon GGUF build to pass and document runtime preflight conditions (#2115) — thanks @LMGXENON and @martinezpl!
## [0.5.3] — 2026-09-17
**Highlights**
- The README is shorter, with a new Electron UI tour and refreshed screenshots (#2129)
- Support pages feature cleaner donation cards, with a workspace support shortcut and sponsor footer with hover cards and email inquiries (#2129)
@@ -27,6 +41,10 @@ the frozen-backend fallback mirror it for their toolchains.
### Changed
- Electron becomes the default source desktop, with artifact-only packaging rehearsals and a separate final Tauri update path (#2157)
- Installable agent skills use current VoiceStudio names and Electron workflows (#2157)
- README clarifies the Electron transition while keeping desktop contributions welcome (#2153) — thanks @cyberspace-cs!
- Electron first run uses four simple steps with model packs, optional advanced controls and skippable dictation setup (#2129)
- Model Catalogue is one page: a setup summary (speech, transcription, dictation, language model) on top, one TTS / ASR / LLM switch, and each family's downloadable weights listed under its engines; the separate Models pane and the Settings → Voice → Engines / Models signposts are gone, the models directory and voice previews moved to Settings → Storage and the HF mirror to Network (#2013)
@@ -35,13 +53,36 @@ the frozen-backend fallback mirror it for their toolchains.
### Fixed
- Apple Silicon Metal builds of omnivoice.cpp are documented as working clean and verified without continue-on-error in CI (#2105) — thanks @martinezpl!
- Keep demo playback aligned across languages, preserve worker GPU metrics, and restrict unsigned releases to owner dispatches (#2157)
- Desktop integration checks cover current dubbing safeguards, navigation, and the linked engine catalog (#2157)
- Tauri and Electron now share native dictation, watch-folder, and Wayland shortcut contracts; focused paste stays ordered and first-run uv stays pinned at 0.12.13 (#2122)
- Dubbing demos synchronize playheads without simultaneous playback and let you open a sample in the editor (#2131)
- macOS desktop sidebar clears the traffic lights, uses a narrower collapsed rail, and places notifications and device controls with more space (#2126)
- Dubbing timelines keep short segments proportional, support zoom, and remove timestamp-confirmed duplicate ASR context (#2129)
- Dubbing translation shares the agent footer with live logs, validated output, cancellation and contextual retries (#2129)
- Agent dubbing translation saves a custom tone and adaptation prompt and preserves it during timing rewrites (#2129)
- Dubbing preserves original sound outside dialogue and mixes separated background only beneath replacement speech (#2129)
- Dubbing repairs missing speech caches, rejects incomplete output, avoids oversized speaker references, and fits full speech without early clipping (#2129)
- Workspace sidebars have a working right-edge resize handle, allow 40% more width, remember their size, and keep video controls inside the preview (#2129)
- Pressing Play while a video is loading starts playback when it is ready instead of reporting playback unavailable (#2129)
- Video previews show their thumbnail before playback, including the source video in Dub (#2129)
- Linux and Windows workspace headers consistently expand and collapse the sidebar, with the app logo at the top of the collapsed rail (#2129)
- Stopping a process on macOS no longer fails with "Operation not permitted" when it was already exiting (#2032)
- A YouTube link blocked by its "not a bot" check now says how to attach signed-in cookies in Dub, instead of quoting yt-dlp's command-line flags (#2036, #2034)
- An engine that fails to start now says whether it timed out, crashed (with its exit code and last output) or answered wrongly, instead of "did not signal ready: None" (#2037, #2026)
- Transcribing an M4A file with PyTorch Whisper works, instead of failing with "Format not recognised" (#2042, #2039)
- PyTorch Whisper runs on 6 GB NVIDIA cards instead of falling back to CPU, because its memory check now fits the model it loads (#2044, #2041)
- MCP tools wait as long as the backend does, so a long transcription no longer fails at 120 s with an empty error (#2043, #2040)
- Generating on an older NVIDIA GPU (Tesla T4, and other pre-Ampere cards) no longer kills the backend on the first request — CUDA graphs are not captured below sm_80 (#2135)
- "Disable torch.compile" in Settings → Performance now works on macOS and Linux, not only Windows; it was greyed out on the platforms that needed it (#2135)
- Setting `TORCH_COMPILE_DISABLE=1` in the environment now actually disables torch.compile, for the in-process engine and engine subprocesses alike (#2135)
- A backend killed by a native crash now leaves the faulting thread's stack in `backend_err.log` instead of exiting silently (#2135)
### CI
@@ -140,6 +181,8 @@ the frozen-backend fallback mirror it for their toolchains.
### Added
- Remote-worker metrics distinguish unavailable readings from zero and keep probes off the control loop (#2155)
- The audiobook result is now a synced-lyrics player: chapter text follows playback with the current word highlighted and click-to-seek, timed from the render's own chapter durations with a karaoke-style even split — no ASR pass, fully local (#1766) — thanks @mvanhorn!
- The dub CAST strip expands into a project-level casting board: drag voice chips (clone profiles, design presets, Default) onto speaker rows — or pick from a keyboard listbox — writing the same per-speaker cast fields as the existing dropdowns (#1767) — thanks @mvanhorn!
- Studio's new Convert method turns a dropped or recorded clip into an existing voice profile's voice, with optional source-duration matching (#1765) — thanks @mvanhorn!
README.md
+26 -16
View File
@@ -5,7 +5,6 @@
<a href="https://trendshift.io/repositories/28176?utm_source=repository-badge&amp;utm_medium=badge&amp;utm_campaign=badge-repository-28176" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/repositories/28176" alt="VoiceStudio ranking on Trendshift" width="220" height="48" /></a>
</p>
<p><strong>Open source voice cloning and workflow engine. Build local.</strong></p>
<p>Clone voices, dub videos, dictate, and create audiobooks with local AI.</p>
<p>
<a href="https://voicestudio.sh/?utm_source=github&utm_medium=readme&utm_campaign=project">Website</a> ·
<a href="https://github.com/debpalash/VoiceStudio/releases/latest">Download</a> ·
@@ -20,22 +19,23 @@
<a href="LICENSE"><img src="https://img.shields.io/badge/license-AGPL--3.0-blue" alt="AGPL-3.0" /></a>
</p>
</div>
<img width="2628" height="1950" alt="screenshot-2026-09-16_17-21-37" src="https://github.com/user-attachments/assets/b474497d-a453-49a3-a2dd-f023ec6b7659" />
![A tour of the Electron app: voice cloning, voice design, dubbing, and model management](docs/media/electron/voicestudio.gif)
## Create with VoiceStudio
## Your voice. Your workflow.
- **Clone & design voices** — use a reference recording or describe the voice you imagine.
- **Dub video** — transcribe, translate, assign speakers, and edit timed speech.
- **Dictate anywhere** — record, transcribe, and copy text with a floating recording widget.
- **Tell longer stories** — create multi-voice scripts, audiobooks, and batch jobs.
- **Choose your models** — manage speech and transcription engines, languages, and compute devices.
| Create | Produce | Connect |
| :--- | :--- | :--- |
| Clone a voice or design your own | Dub videos with timed speech | Local API & MCP for agents |
| Dictate with a floating widget | Stories, audiobooks & batch jobs | Optional remote workers |
Start with **VoiceStudio** (default, powered by k2-fsa/OmniVoice), or choose another engine.
Start with **VoiceStudio** (default, powered by k2-fsa/OmniVoice), or choose another engine. [Features & engine catalog](docs/feature-catalog.md).
Local workflows run on your hardware. Remote services are optional; usage analytics requires consent.
<details>
<summary><strong>Explore the workspaces</strong> · Clone, dub, design & models</summary>
<table>
<tr>
<td><img src="docs/media/electron/voice-cloning.png" alt="Electron voice cloning workspace with the bundled demo voice" width="100%" /></td>
@@ -49,6 +49,10 @@ Local workflows run on your hardware. Remote services are optional; usage analyt
<tr><td align="center">Voice design</td><td align="center">Local models</td></tr>
</table>
<img width="2628" height="1950" alt="VoiceStudio desktop workspace" src="https://github.com/user-attachments/assets/b474497d-a453-49a3-a2dd-f023ec6b7659" />
</details>
## Get started
Download from [Releases](https://github.com/debpalash/VoiceStudio/releases/latest), then follow your platform guide:
@@ -57,17 +61,21 @@ Download from [Releases](https://github.com/debpalash/VoiceStudio/releases/lates
Open **Voice cloning**, choose a voice or add a clean reference recording, enter your text, and generate. Install the required model when prompted. Hardware needs vary by engine; see [performance](docs/performance.md).
**Run the Electron preview from source:**
<details>
<summary><strong>Run the Electron preview from source</strong></summary>
```bash
git clone https://github.com/debpalash/VoiceStudio.git
cd VoiceStudio
bun install
cd electron
bun run dev
```
See [Electron setup](electron/README.md) for prerequisites and backend configuration. VoiceStudio is in active development; report bugs through [GitHub Issues](https://github.com/debpalash/VoiceStudio/issues).
See [Electron setup](electron/README.md) for prerequisites and backend configuration.
</details>
> **Electron is the primary desktop app.** The next desktop release ships Electron, with one final Tauri sunset update. Bug reports and contributions remain welcome; include the app version and whether you use Electron or Tauri.
## Documentation
@@ -78,13 +86,15 @@ See [Electron setup](electron/README.md) for prerequisites and backend configura
| Integrations | [Local API](docs/speech-platform.md) · [MCP](docs/mcp.md) · [Examples](examples/README.md) |
| Development | [Contributing](.github/CONTRIBUTING.md) · [Electron](electron/README.md) · [Changelog](CHANGELOG.md) |
Agent skills: `npx skills add debpalash/VoiceStudio` — choose **voicestudio** for audio workflows or **oss-maintainer** for repository maintenance.
Agent skills: `npx skills add debpalash/VoiceStudio` — choose **voicestudio** for audio workflows or **voicestudio-maintainer** for repository maintenance.
## Support VoiceStudio
## Sponsors
[Ko-fi](https://ko-fi.com/debpalash) · [PayPal](https://paypal.me/palashCoder) · [Sponsor the project](SPONSORS.md) · [Partnerships](mailto:partner@voicestudio.sh)
<a href="https://forms.gle/2PYCvd39hbwijzX37"><img src="docs/media/sponsor-slot.svg" alt="Your brand — apply for a featured VoiceStudio sponsor slot" width="640" /></a>
**Put your brand where people build with voice.** Explore paid placements in the app footer, integrations directory, documentation, and README. [Apply to partner](https://forms.gle/2PYCvd39hbwijzX37) or [email us](mailto:partner@voicestudio.sh).
**Become a featured partner.** [Apply for a paid placement](https://forms.gle/2PYCvd39hbwijzX37) · [Email us](mailto:partner@voicestudio.sh)
Support development: [Ko-fi](https://ko-fi.com/debpalash) · [PayPal](https://paypal.me/palashCoder) · [Sponsorship details](SPONSORS.md)
## License & responsible use
backend/api/routers/settings.py
+10 -4
View File
@@ -156,7 +156,7 @@ def set_performance_profile(body: _PerformanceProfileBody):
class _TorchCompileBody(BaseModel):
enabled: bool = Field(..., description="True to set TORCH_COMPILE_DISABLE=1 on engine subprocesses")
enabled: bool = Field(..., description="True to disable torch.compile (eager mode) for the engine")
def _torch_compile_state() -> dict:
@@ -170,15 +170,21 @@ def _torch_compile_state() -> dict:
@router.get("/perf/torch-compile-disabled")
def get_torch_compile_disabled():
"""Return the current torch.compile-disabled toggle + the runtime platform.
UI uses the platform to render the toggle disabled (with an explainer)
on non-Windows hosts, since the OOM is Windows-specific (issue #65)."""
`platform` is still reported (clients may show it), but since #2135 the
toggle is live on every host: it used to be rendered disabled off Windows
on the assumption that only #65's Windows OOM needed it, which left the
Linux/CUDA reporter of #2135 with no way to switch off the compile that
was killing their backend.
"""
return _torch_compile_state()
@router.put("/perf/torch-compile-disabled")
def set_torch_compile_disabled(body: _TorchCompileBody):
"""Persist the toggle. Honoured by `services.engine_env.build_engine_env()`
which injects TORCH_COMPILE_DISABLE=1 on Windows when enabled."""
(subprocess engines) and `services.engine_env.should_torch_compile()`
(in-process), on every platform since #2135."""
from services import settings_store
try:
backend/core/crash_diagnostics.py
+59
View File
@@ -0,0 +1,59 @@
"""Native-crash diagnostics for the backend process (#2135).
A crash inside torch/CUDA — graph capture, a driver fault, an allocator abort —
kills the interpreter below the level any ``except`` can reach. #2135's reporter
saw exactly that: the backend "simply exited" mid-``/generate`` with no Python
traceback, no HTTP response, and ``ConnectionRefused`` on the next ``/health``.
There was nothing in the logs to diagnose because nothing in Python ever ran
again.
``faulthandler`` installs handlers for the fatal signals (SIGSEGV, SIGABRT,
SIGBUS, SIGFPE, SIGILL) that print every thread's Python stack to stderr on the
way down. That is the difference between "the process vanished" and a named
frame pointing at the engine call that killed it.
This is strictly a diagnostic: it does not prevent the crash, and it must never
be the reason startup fails.
"""
from __future__ import annotations
import os
_DISABLE_ENV = "OMNIVOICE_DISABLE_FAULTHANDLER"
_TRUTHY = frozenset({"1", "true", "yes", "on"})
def _disabled() -> bool:
return os.environ.get(_DISABLE_ENV, "").strip().lower() in _TRUTHY
def enable_fault_handler(stderr=None) -> bool:
"""Arm fatal-signal tracebacks. Returns True when armed.
Call as early as possible — before torch is imported — so a crash during
model load is covered too. Honours ``OMNIVOICE_DISABLE_FAULTHANDLER=1`` for
hosts whose outer supervisor installs its own handlers.
Args:
stderr: optional file object to write dumps to. Defaults to the real
``sys.stderr`` (→ ``backend_err.log``). faulthandler keeps the
underlying fd, so the object must stay open for the process
lifetime.
Never raises: a frozen build with a detached stderr, or a platform without
the signals, degrades to "no crash dump" rather than a failed boot.
"""
if _disabled():
return False
try:
import faulthandler
# all_threads=True: the fatal frame is routinely on a GPU-pool or
# compile worker, not whichever thread happens to take the signal.
if stderr is not None:
faulthandler.enable(file=stderr, all_threads=True)
else:
faulthandler.enable(all_threads=True)
return True
except Exception:
return False
backend/core/version.py
+1 -1
View File
@@ -24,7 +24,7 @@ from pathlib import Path
# tests/test_app_version.py::test_all_version_files_in_lockstep and bumped by
# release.yml's version-bump job, so it stays equal to
# pyproject/tauri.conf/Cargo/package.json.
_FALLBACK_VERSION = "0.5.2"
_FALLBACK_VERSION = "0.5.3"
def _fallback_version() -> str:
backend/engines/omnivoice_gguf/README.md
+3 -2
View File
@@ -115,8 +115,9 @@ silently hanging on a Gatekeeper-killed spawn.
The macOS Apple Silicon Metal build compiles cleanly with `-DGGML_METAL=ON`
at the pinned `omnivoice.cpp` SHA (#2105), enabling GPU-accelerated GGUF
voice cloning on Apple Silicon out of the box with `VoiceStudioBackend`
remaining available as an in-process fallback.
voice cloning when the packaged binary passes preflight and is permitted by
macOS. Missing binaries, placeholders, or Gatekeeper rejection leave
`VoiceStudioBackend` available as the in-process fallback.
## Smoke test
backend/main.py
+7
View File
@@ -9,6 +9,13 @@ _backend_dir = os.path.dirname(os.path.abspath(__file__))
if _backend_dir not in sys.path:
sys.path.insert(0, _backend_dir)
# #2135: arm fatal-signal tracebacks before anything heavy is imported, so a
# native crash inside torch/CUDA leaves a named frame in backend_err.log
# instead of a silently vanished process. See core/crash_diagnostics.py.
from core.crash_diagnostics import enable_fault_handler # noqa: E402
enable_fault_handler()
# PyInstaller re-executes this entry module when the frozen backend binary is
# launched. Nested operation supervisors therefore dispatch here, before math,
# logging, FastAPI, torch, or any application initialization. Source launches
backend/services/engine_env.py
+92 -15
View File
@@ -17,7 +17,6 @@ from __future__ import annotations
import importlib.util
import logging
import os
import sys
from typing import Optional
logger = logging.getLogger("omnivoice.engine_env")
@@ -29,6 +28,53 @@ _TORCH_COMPILE_KEY = "perf.torch_compile_disabled"
# (e.g. a brand-new architecture running through PTX forward-compat).
_FORCE_COMPILE_ENV = "OMNIVOICE_FORCE_TORCH_COMPILE"
# #2135: the environment escape hatches that torch itself honours. `main.py`
# sets TORCH_COMPILE_DISABLE/TORCHDYNAMO_DISABLE on win32, `build_engine_env`
# injects TORCH_COMPILE_DISABLE into engine subprocesses, and
# `docs/install/windows.md` tells users to export it — but the in-process gate
# below never read them, so an operator who set the documented variable still
# got a compiled model (and, on a cudagraph mode, a native crash they could not
# turn off). Reading them here makes one knob mean one thing everywhere.
_COMPILE_DISABLE_ENVS = (
"TORCH_COMPILE_DISABLE",
"TORCHDYNAMO_DISABLE",
"TORCHINDUCTOR_DISABLE",
)
_TRUTHY = frozenset({"1", "true", "yes", "on"})
def _env_compile_disabled() -> Optional[str]:
"""The name of the first set-and-truthy compile-disable env var, else None.
Mirrors torch's own reading of these variables so the app's decision and
torch's behaviour cannot disagree — the state the reporter in #2135 hit,
where the log said "torch.compile applied" while TORCH_COMPILE_DISABLE=1
was exported.
"""
for name in _COMPILE_DISABLE_ENVS:
if os.environ.get(name, "").strip().lower() in _TRUTHY:
return name
return None
def _settings_db_path() -> str:
"""The settings DB the compile toggle is actually read from (best-effort).
Logged alongside the toggle because #2135's reporter had three
`omnivoice.db` files on the box and edited one the backend never opened;
naming the path turns "the setting doesn't work" into a one-line diagnosis.
"""
try:
from core.config import DB_PATH
from core.scrub import scrub_text
return scrub_text(str(DB_PATH))
except Exception:
return "<unknown>"
# #278: set (with a reason) the first time torch.compile — or *running* the
# compiled model — fails at runtime in this process. Once set, every later
# load in the same session goes straight to eager instead of re-tripping the
@@ -251,6 +297,15 @@ def should_torch_compile(device: str) -> bool:
"""
if device != "cuda":
return False
# #2135: honoured before every other gate — an explicit env opt-out is the
# user's most direct statement of intent, and it must hold on every
# platform (the reporter was on Linux, where this used to be ignored).
disabled_by = _env_compile_disabled()
if disabled_by is not None:
logger.info(
"torch.compile skipped: %s is set — using eager mode.", disabled_by,
)
return False
if importlib.util.find_spec("triton") is None:
logger.info("torch.compile skipped: Triton unavailable — using eager mode.")
return False
@@ -258,8 +313,18 @@ def should_torch_compile(device: str) -> bool:
from services import settings_store
if settings_store.get_text(_TORCH_COMPILE_KEY, "0") == "1":
logger.info("torch.compile skipped: disabled in Settings (Performance).")
logger.info(
"torch.compile skipped: disabled in Settings (Performance) [%s].",
_settings_db_path(),
)
return False
# #2135: say which DB answered "not disabled". Without this the only
# observable outcome of a toggle that never reached the running
# backend is a log line saying compile was applied anyway.
logger.debug(
"torch.compile: %s not set in %s — compile remains eligible.",
_TORCH_COMPILE_KEY, _settings_db_path(),
)
except Exception:
logger.exception("should_torch_compile: settings read failed; proceeding")
if _compile_runtime_failure is not None:
@@ -328,19 +393,31 @@ def build_engine_env(
except Exception:
logger.exception("build_engine_env: token resolver failed (non-fatal)")
# INST-12: TORCH_COMPILE_DISABLE on Windows when the user opted in.
# The flag is a Windows-only escape hatch — torch.compile OOMs the same
# Triton kernel cache differently on macOS/Linux, so injecting on those
# platforms would just slow the engine for no gain. (The in-process
# should_torch_compile() gate handles the automatic Triton-absence case;
# the subprocess var stays user-driven by design — see test_perf_settings.)
if sys.platform.startswith("win"):
try:
from services import settings_store
# INST-12 (#65), widened to every platform by #2135: TORCH_COMPILE_DISABLE
# when the user opted in. This was win32-only on the theory that
# torch.compile only misbehaves on Windows (no Triton wheel). #2135 is the
# counter-example — a Linux/CUDA host where compile crashes the engine —
# and a Settings toggle that silently does nothing on the user's platform
# is worse than no toggle at all. Cost when enabled on Linux/macOS is a
# slower engine, which is exactly what the user asked for by enabling it.
try:
from services import settings_store
if settings_store.get_text(_TORCH_COMPILE_KEY, "0") == "1":
env["TORCH_COMPILE_DISABLE"] = "1"
except Exception:
logger.exception("build_engine_env: torch_compile_disabled read failed")
if settings_store.get_text(_TORCH_COMPILE_KEY, "0") == "1":
env["TORCH_COMPILE_DISABLE"] = "1"
except Exception:
logger.exception("build_engine_env: torch_compile_disabled read failed")
# #2135: an env opt-out on the parent must reach the child too. Without
# this a user who exported TORCH_COMPILE_DISABLE=1 got an eager parent and
# a compiled sidecar — the inconsistency that made the flag look ignored.
disabled_by = _env_compile_disabled()
if disabled_by is not None:
if env.get("TORCH_COMPILE_DISABLE") != "1":
logger.debug(
"build_engine_env: %s is set — disabling torch.compile in the "
"engine subprocess too.", disabled_by,
)
env["TORCH_COMPILE_DISABLE"] = "1"
return env
backend/services/model_manager.py
+65 -4
View File
@@ -1793,6 +1793,64 @@ _TORCH_COMPILE_MODE = "reduce-overhead"
# would not.
_CUDAGRAPH_COMPILE_MODES = frozenset({"reduce-overhead", "max-autotune"})
# ── #2135: CUDA-graph capture needs Ampere or newer ─────────────────────────
# On a Turing T4 (sm_75) the cudagraph mode above took the whole backend
# process down on the first generate — no Python traceback, no HTTP response,
# just a dead PID (the native capture aborts below the interpreter, so neither
# the #278 eager fallback nor any `except` can see it). The graph *capture* is
# the risky part, not Inductor: dropping to the non-cudagraph "default" mode
# keeps the compiled kernels (and most of the speedup) while removing the
# crash surface. Ampere (sm_80) is the floor because that is where the app has
# actual passing evidence; anything older takes the conservative path.
_CUDAGRAPH_MIN_CAPABILITY = (8, 0)
# Escape hatch in the other direction, for operators benchmarking on old GPUs.
_FORCE_CUDAGRAPH_ENV = "OMNIVOICE_FORCE_CUDAGRAPH"
def _resolve_compile_mode() -> str:
"""The ``torch.compile`` mode to use on this GPU (#2135).
Returns the configured cudagraph mode on Ampere+, and the non-cudagraph
``"default"`` on older architectures where graph capture has been observed
to abort the process. Fails *safe* (→ "default") only when we positively
identify a pre-Ampere device; any probe error keeps the configured mode so
a weird torch build doesn't silently lose the optimization.
"""
if _TORCH_COMPILE_MODE not in _CUDAGRAPH_COMPILE_MODES:
return _TORCH_COMPILE_MODE
if os.environ.get(_FORCE_CUDAGRAPH_ENV, "").strip().lower() in {"1", "true", "yes", "on"}:
logger.warning(
"%s=1 — keeping torch.compile mode %r on a GPU where CUDA-graph "
"capture is not known-good (#2135).",
_FORCE_CUDAGRAPH_ENV, _TORCH_COMPILE_MODE,
)
return _TORCH_COMPILE_MODE
try:
import torch
if not torch.cuda.is_available():
return _TORCH_COMPILE_MODE
capability = torch.cuda.get_device_capability(0)
except Exception:
logger.debug("compile-mode capability probe failed; keeping %r",
_TORCH_COMPILE_MODE, exc_info=True)
return _TORCH_COMPILE_MODE
if tuple(capability) >= _CUDAGRAPH_MIN_CAPABILITY:
return _TORCH_COMPILE_MODE
try:
device_name = torch.cuda.get_device_name(0)
except Exception:
device_name = "this GPU"
logger.info(
"torch.compile mode %r downgraded to 'default' on %s (sm_%d%d): CUDA-graph "
"capture below sm_%d%d has been seen to abort the backend process (#2135). "
"Compiled kernels are still used. Set %s=1 to override.",
_TORCH_COMPILE_MODE, device_name, capability[0], capability[1],
_CUDAGRAPH_MIN_CAPABILITY[0], _CUDAGRAPH_MIN_CAPABILITY[1],
_FORCE_CUDAGRAPH_ENV,
)
return "default"
_compiled_inference_executor: "ThreadPoolExecutor | None" = None
_compiled_inference_thread_ident: "int | None" = None
@@ -2586,8 +2644,11 @@ def _load_model_sync():
if not flashinfer_applied and should_torch_compile(device):
_set_loading("compiling", "Compiling model (torch.compile)…")
# #2135: resolved per-GPU — pre-Ampere drops to the
# non-cudagraph mode rather than risking a native abort.
compile_mode = _resolve_compile_mode()
try:
_model.llm = torch.compile(_model.llm, mode=_TORCH_COMPILE_MODE)
_model.llm = torch.compile(_model.llm, mode=compile_mode)
except Exception as compile_exc:
# #278: compile is an optimization, never a point of
# failure — keep the eager model and remember the failure
@@ -2604,7 +2665,7 @@ def _load_model_sync():
# archs, #278). Wrap generate so that falls back to eager
# instead of failing the generation.
_install_compile_fallback(_model)
if _TORCH_COMPILE_MODE in _CUDAGRAPH_COMPILE_MODES:
if compile_mode in _CUDAGRAPH_COMPILE_MODES:
# #315: reduce-overhead uses CUDA graphs, whose
# captured state is thread-local. Pin all inference to
# one dedicated thread so a later render dispatched to
@@ -2615,9 +2676,9 @@ def _load_model_sync():
logger.info(
"torch.compile mode %r uses CUDA graphs — compiled-model "
"inference pinned to a single dedicated thread (#315).",
_TORCH_COMPILE_MODE,
compile_mode,
)
logger.info("torch.compile applied.")
logger.info("torch.compile applied (mode=%r).", compile_mode)
except Exception as e:
logger.info("torch.compile skipped: %s", e)
backend/worker/capacity.py
+14 -1
View File
@@ -150,7 +150,11 @@ class WorkerCapacity:
worker_id: str
max_concurrent_tasks: int = 1
active_tasks: int = 0
free_memory_bytes: int = 0
# ``None`` means the worker could not query VRAM. It is distinct from a
# real zero-byte reading, which means the device is completely occupied.
free_memory_bytes: Optional[int] = None
cpu_percent: Optional[float] = None
gpu_utilization_percent: Optional[float] = None
backend: str = ""
resident_models: set[str] = field(default_factory=set)
slots: dict[str, ModelSlot] = field(default_factory=dict)
@@ -287,6 +291,8 @@ class WorkerCapacity:
available_slots: int,
resident_models: Optional[set[str]] = None,
free_memory_bytes: Optional[int] = None,
cpu_percent: Optional[float] = None,
gpu_utilization_percent: Optional[float] = None,
now: Optional[float] = None,
) -> None:
"""Adopt a heartbeat snapshot. The worker is the source of truth for
@@ -307,6 +313,10 @@ class WorkerCapacity:
self.resident_models = set(resident_models)
if free_memory_bytes is not None:
self.free_memory_bytes = free_memory_bytes
if cpu_percent is not None:
self.cpu_percent = max(0.0, min(100.0, float(cpu_percent)))
if gpu_utilization_percent is not None:
self.gpu_utilization_percent = max(0.0, min(100.0, float(gpu_utilization_percent)))
# Parks are released on a timer, and by the worker restarting — never
# by the worker's own load report.
#
@@ -330,6 +340,9 @@ class WorkerCapacity:
"zombie_tasks": self.zombie_tasks,
"available_slots": self.available_slots,
"resident_models": sorted(self.resident_models),
"free_memory_bytes": self.free_memory_bytes,
"cpu_percent": self.cpu_percent,
"gpu_utilization_percent": self.gpu_utilization_percent,
}
backend/worker/pool.py
+4
View File
@@ -305,6 +305,8 @@ class WorkerPool:
available_slots: int,
resident_models: Optional[set[str]] = None,
free_memory_bytes: Optional[int] = None,
cpu_percent: Optional[float] = None,
gpu_utilization_percent: Optional[float] = None,
latency_ms: Optional[float] = None,
now: Optional[float] = None,
) -> Optional[ConnectedWorker]:
@@ -319,6 +321,8 @@ class WorkerPool:
available_slots=available_slots,
resident_models=resident_models,
free_memory_bytes=free_memory_bytes,
cpu_percent=cpu_percent,
gpu_utilization_percent=gpu_utilization_percent,
)
return worker
backend/worker/protocol/gen/worker_v1_pb2.py
+70 -70
View File
File diff suppressed because one or more lines are too long
backend/worker/protocol/gen/worker_v1_pb2.pyi
+4 -2
View File
@@ -194,20 +194,22 @@ class RegisterResponse(_message.Message):
def __init__(self, envelope: _Optional[_Union[Envelope, _Mapping]] = ..., worker_id: _Optional[str] = ..., session_token: _Optional[str] = ..., session_epoch: _Optional[int] = ..., protocol_version: _Optional[int] = ..., session_expires_at_unix: _Optional[int] = ..., heartbeat_interval_seconds: _Optional[int] = ..., authoritative_in_flight: _Optional[_Iterable[_Union[TaskRef, _Mapping]]] = ..., error: _Optional[_Union[Error, _Mapping]] = ...) -> None: ...
class Heartbeat(_message.Message):
__slots__ = ("envelope", "active_tasks", "available_slots", "resident_models", "free_memory_bytes", "cpu_percent")
__slots__ = ("envelope", "active_tasks", "available_slots", "resident_models", "free_memory_bytes", "cpu_percent", "gpu_utilization_percent")
ENVELOPE_FIELD_NUMBER: _ClassVar[int]
ACTIVE_TASKS_FIELD_NUMBER: _ClassVar[int]
AVAILABLE_SLOTS_FIELD_NUMBER: _ClassVar[int]
RESIDENT_MODELS_FIELD_NUMBER: _ClassVar[int]
FREE_MEMORY_BYTES_FIELD_NUMBER: _ClassVar[int]
CPU_PERCENT_FIELD_NUMBER: _ClassVar[int]
GPU_UTILIZATION_PERCENT_FIELD_NUMBER: _ClassVar[int]
envelope: Envelope
active_tasks: int
available_slots: int
resident_models: _containers.RepeatedScalarFieldContainer[str]
free_memory_bytes: int
cpu_percent: float
def __init__(self, envelope: _Optional[_Union[Envelope, _Mapping]] = ..., active_tasks: _Optional[int] = ..., available_slots: _Optional[int] = ..., resident_models: _Optional[_Iterable[str]] = ..., free_memory_bytes: _Optional[int] = ..., cpu_percent: _Optional[float] = ...) -> None: ...
gpu_utilization_percent: float
def __init__(self, envelope: _Optional[_Union[Envelope, _Mapping]] = ..., active_tasks: _Optional[int] = ..., available_slots: _Optional[int] = ..., resident_models: _Optional[_Iterable[str]] = ..., free_memory_bytes: _Optional[int] = ..., cpu_percent: _Optional[float] = ..., gpu_utilization_percent: _Optional[float] = ...) -> None: ...
class TaskAccepted(_message.Message):
__slots__ = ("ref", "envelope")
backend/worker/protocol/worker_v1.proto
+3 -5
View File
@@ -228,11 +228,9 @@ message Heartbeat {
uint32 active_tasks = 2;
uint32 available_slots = 3;
repeated string resident_models = 4;
uint64 free_memory_bytes = 5;
double cpu_percent = 6;
// GPU utilisation is deliberately absent: unobtainable on Apple without
// sudo powermetrics and absent on CUDA without a new NVML dependency.
// Slots + queue depth are the load signals (goal_v2.md A11).
optional uint64 free_memory_bytes = 5;
optional double cpu_percent = 6;
optional double gpu_utilization_percent = 7;
}
message TaskAccepted { TaskRef ref = 1; Envelope envelope = 2; }
backend/worker/routing.py
+23
View File
@@ -117,6 +117,13 @@ class Target:
latency_ms: float = 0.0
active_tasks: int = 0
max_tasks: int = 0
cpu_percent: Optional[float] = None
free_memory_bytes: Optional[int] = None
system_memory_bytes: int = 0
cpu_count: int = 0
gpu_name: str = ""
gpu_memory_bytes: int = 0
gpu_utilization_percent: Optional[float] = None
@property
def is_local(self) -> bool:
@@ -135,6 +142,13 @@ class Target:
"latency_ms": round(self.latency_ms, 1),
"active_tasks": self.active_tasks,
"max_tasks": self.max_tasks,
"cpu_percent": self.cpu_percent,
"free_memory_bytes": self.free_memory_bytes,
"system_memory_bytes": self.system_memory_bytes,
"cpu_count": self.cpu_count,
"gpu_name": self.gpu_name,
"gpu_memory_bytes": self.gpu_memory_bytes,
"gpu_utilization_percent": self.gpu_utilization_percent,
}
@@ -192,6 +206,8 @@ def list_targets(control_plane=None) -> list[Target]:
pool = getattr(control_plane, "pool", None) if control_plane.running else None
for record in enrolled:
live = pool.get(record.id) if pool is not None else None
host = record.host or {}
gpu = (host.get("gpus") or [{}])[0]
connected = live is not None and not live.stale()
available, detail = _availability(record, live, pool)
targets.append(
@@ -207,6 +223,13 @@ def list_targets(control_plane=None) -> list[Target]:
latency_ms=live.latency_ms if live else 0.0,
active_tasks=live.capacity.active_tasks if live else 0,
max_tasks=live.capacity.max_concurrent_tasks if live else 0,
cpu_percent=live.capacity.cpu_percent if live else None,
free_memory_bytes=live.capacity.free_memory_bytes if live else None,
system_memory_bytes=int(host.get("system_memory_bytes") or 0),
cpu_count=int(host.get("cpu_count") or 0),
gpu_name=str(gpu.get("model") or ""),
gpu_memory_bytes=int(gpu.get("memory_bytes") or 0),
gpu_utilization_percent=live.capacity.gpu_utilization_percent if live else None,
)
)
return targets
backend/worker/transport/client.py
+84
View File
@@ -43,6 +43,8 @@ import platform
import random
import socket
import sys
import threading
from concurrent.futures import Future
from dataclasses import dataclass, field
from typing import Awaitable, Callable, Optional, Protocol
@@ -73,6 +75,33 @@ _FALLBACK_MODEL_LOAD_SECONDS = 1800.0
# see _oversized_result_error for why that has to be a failure and not a retry.
MAX_MESSAGE_BYTES = 8 * 1024 * 1024
def _heartbeat_resources() -> tuple[Optional[float], Optional[int], Optional[float]]:
"""Sample cheap host telemetry without making a heartbeat depend on CUDA."""
cpu_percent = free_memory_bytes = gpu_utilization_percent = None
try:
import psutil
cpu_percent = float(psutil.cpu_percent(interval=None))
except Exception:
logger.debug("Could not sample worker CPU usage", exc_info=True)
try:
import torch
if torch.cuda.is_available():
free_memory_bytes = int(torch.cuda.mem_get_info()[0])
except Exception:
logger.debug("Could not sample worker free VRAM", exc_info=True)
try:
import pynvml
pynvml.nvmlInit()
handle = pynvml.nvmlDeviceGetHandleByIndex(0)
gpu_utilization_percent = float(pynvml.nvmlDeviceGetUtilizationRates(handle).gpu)
except Exception:
logger.debug("Could not sample worker GPU usage", exc_info=True)
return cpu_percent, free_memory_bytes, gpu_utilization_percent
# Room left for result_json, the ref, and protobuf framing when a payload does
# ride inline. The inline decision is made on the payload alone, so without a
# reserve a payload sized exactly at the frame cap would overflow it.
@@ -339,6 +368,11 @@ class WorkerClient:
self._running: dict[str, asyncio.Task] = {}
self._keepalives: dict[str, asyncio.Task] = {}
self._maintenance: set[asyncio.Task] = set()
self._telemetry: tuple[Optional[float], Optional[int], Optional[float]] = (None, None, None)
# A driver query can hang indefinitely. Keep that one query owned
# rather than cancelling its awaiter and starting a fresh thread at
# every heartbeat.
self._telemetry_task: Optional[asyncio.Future] = None
self._prewarms: dict[str, asyncio.Task] = {}
self._prewarm_cancellations: dict[str, asyncio.Task] = {}
self._epoch = 0
@@ -683,10 +717,59 @@ class WorkerClient:
async def _heartbeat_loop(self, interval: float) -> None:
while True:
await asyncio.sleep(interval)
await self._refresh_telemetry()
await self._send(self.heartbeat_message())
async def _refresh_telemetry(self) -> None:
"""Publish completed samples and retain one non-blocking probe.
CUDA/NVML calls may wedge in a driver. A timed ``to_thread`` await
only cancels the awaiter, leaving that thread alive; retaining this
task prevents later heartbeats from accumulating more blocked probes.
"""
if not self._accepting_assignments or self._stop.is_set():
return
task = self._telemetry_task
if task is not None and task.done():
try:
sampled = task.result()
except Exception:
logger.debug("Could not sample worker telemetry", exc_info=True)
else:
# A partial failed sample must not erase an independent last
# good value. Presence on the heartbeat remains honest until
# that individual metric can next be measured.
self._telemetry = tuple(
current if value is None else value
for current, value in zip(self._telemetry, sampled)
)
self._telemetry_task = None
if self._telemetry_task is None:
# Read-only driver probes cannot be interrupted. Keep one across
# reconnects, outside assignment drain and the shared executor
# (whose shutdown would otherwise wait forever for a wedged driver).
result = Future()
self._telemetry_task = asyncio.wrap_future(result)
def sample() -> None:
try:
result.set_result(_heartbeat_resources())
except Exception:
logger.debug("Could not sample worker telemetry", exc_info=True)
result.set_result((None, None, None))
threading.Thread(
target=sample, name="worker-telemetry-probe", daemon=True,
).start()
def heartbeat_message(self) -> pb.WorkerMessage:
"""Build the worker's current liveness/capacity frame."""
cpu_percent, free_memory_bytes, gpu_utilization_percent = self._telemetry
telemetry = {}
if cpu_percent is not None: telemetry["cpu_percent"] = cpu_percent
if free_memory_bytes is not None: telemetry["free_memory_bytes"] = free_memory_bytes
if gpu_utilization_percent is not None: telemetry["gpu_utilization_percent"] = gpu_utilization_percent
return pb.WorkerMessage(
heartbeat=pb.Heartbeat(
active_tasks=len(self._running),
@@ -694,6 +777,7 @@ class WorkerClient:
0, self.config.max_concurrent_tasks - len(self._running)
),
resident_models=self._resident_models(),
**telemetry,
)
)
backend/worker/transport/server.py
+3 -1
View File
@@ -2068,7 +2068,9 @@ class WorkerServicer(pb_grpc.WorkerServiceServicer):
active_tasks=active_tasks,
available_slots=available_slots,
resident_models=set(beat.resident_models),
free_memory_bytes=beat.free_memory_bytes,
free_memory_bytes=beat.free_memory_bytes if beat.HasField("free_memory_bytes") else None,
cpu_percent=beat.cpu_percent if beat.HasField("cpu_percent") else None,
gpu_utilization_percent=beat.gpu_utilization_percent if beat.HasField("gpu_utilization_percent") else None,
)
self._queue_heartbeat_touch(session)
return
bin/README.md
+3 -2
View File
@@ -24,8 +24,9 @@ scripts/build-omnivoice-tts.sh --platform <slug> --commit-sha <40hex>
See `.github/workflows/build-omnivoice-tts.yml` `build-omnivoice-tts` job
for the CI matrix that produces these artifacts. Apple Silicon (`macos-14`)
builds cleanly with `-DGGML_METAL=ON` at the pinned SHA (#2105), enabling
hardware-accelerated Metal inference without falling back to the in-process
`VoiceStudioBackend`.
hardware-accelerated Metal inference when the packaged artifact passes binary
preflight and macOS permits execution. Missing or blocked binaries retain the
in-process `VoiceStudioBackend` fallback.
## Placeholder note
bun.lock
+1 -1
View File
@@ -75,7 +75,7 @@
},
"frontend": {
"name": "omnivoice-studio",
"version": "0.5.2",
"version": "0.5.3",
"dependencies": {
"@fontsource-variable/inter": "^5.3.0",
"@fontsource-variable/source-serif-4": "^5.3.0",
docs/RELEASING.md
+62 -26
View File
@@ -71,26 +71,18 @@ git tag vX.Y.Z
git push origin vX.Y.Z
```
The `Desktop Release` workflow fires on tag push. It builds four targets in parallel on GitHub Actions runners:
`electron-release.yml` builds Linux x64, Windows x64, macOS arm64 and macOS
x64 installers with updater metadata and packaged startup checks. Ordinary tag
pushes create drafts; a tag-scoped manual dispatch with `publish=true` publishes
after all four targets pass. Signing checks apply by default.
| Target | Runner | Artifact |
|---|---|---|
| macOS Apple Silicon | macos-14 | `.dmg` + updater `.app.tar.gz` |
| macOS Intel | macos-13 | `.dmg` + updater `.app.tar.gz` |
| Windows x64 | windows-2022 | `.msi`, machine-wide and per-user, each with its updater `.sig` |
| Linux x64 | ubuntu-22.04 | `.AppImage` + updater `.AppImage.sig` |
Each runner signs the updater payload with the stored `TAURI_SIGNING_PRIVATE_KEY`, merges into a single `latest.json`, and attaches everything to the draft release.
Workflow runtime: **~20-40 minutes** (PyInstaller + four platform builds). Follow progress at:
`https://github.com/debpalash/VoiceStudio/actions`
The release stays a draft while the platforms build. Once every platform, the
updater-manifest repair and the uninstall scripts are done, the
`release-notes-checksums` job writes all four platforms' checksums into the
notes and publishes it, with no manual step. A failed platform leaves the
release a draft, so nothing half-built goes public. Existing clients detect
the update on their next launch.
For the one-time transition tag, set `TAURI_SUNSET_TAG`, dispatch `release.yml`
on that tag with `draft=true`, and wait for its final Tauri installers and signed
updater feeds. Then dispatch `electron-release.yml` on the same tag. Automatic
Electron builds are skipped for this tag to avoid racing the Tauri draft.
Keep the release draft until both builds and their checks have passed.
See [Electron transition](#electron-transition-next-desktop-release) below for
signing requirements and the explicit owner-only unsigned exception.
## 5b. Deployment channels — all must ship (hard rule, owner-set 2026-07-16)
@@ -100,8 +92,9 @@ bug to fix immediately, not backlog.
| Channel | Source | Produced by | How to verify |
|---|---|---|---|
| GitHub Release: installers + signed `latest.json` (**Stable** updater channel) | the `vX.Y.Z` tag | `release.yml` on tag push | Release page has dmg (arm+intel), msi (machine-wide and per-user), AppImage, `latest.json` and `latest-user.json`; body = the CHANGELOG section (not the auto-generated fallback), followed by per-platform checksums and a **Contributors** avatar strip (owner + every PR author for the tag — the `contributors-strip` job) |
| **Preview** updater channel (rolling `preview` prerelease) | **`main` only** | `release.yml` nightly cron / manual dispatch | preview `latest.json` uses main's version when it is ahead; otherwise it advances the stable patch, then appends `-N` so it semver-sorts above stable |
| GitHub Release: Electron installers and updater manifests | the `vX.Y.Z` tag | `electron-release.yml`, explicit publish dispatch | All four platforms, Electron manifests, SHA256SUMS.txt, versioned CHANGELOG notes; retained Tauri feeds point to the final Tauri tag |
| Final Tauri installers and signed updater feeds | `TAURI_SUNSET_TAG` | `release.yml`, manual dispatch only | Both macOS architectures, Windows system/user installers, Linux AppImage, signed `latest.json` and `latest-user.json` |
| Desktop preview channel | frozen during transition | no scheduled publishing | Existing preview assets remain available; new desktop previews are paused |
| GHCR CUDA image: `:X.Y.Z`, `:X.Y`, `:stable` | the tag | `docker.yml` on tag push | `docker manifest inspect ghcr.io/debpalash/omnivoice-studio:X.Y.Z` |
| GHCR ROCm image: `:X.Y.Z-rocm`, `:X.Y-rocm`, `:stable-rocm` | the tag | `docker.yml` on tag push | same, with `-rocm` suffix |
| Docker Hub mirror of **all** the above tags | the tag | `docker.yml` (gated on `DOCKERHUB_*` secrets) | tag list at hub.docker.com/r/palashdeb/omnivoice-studio/tags |
@@ -109,11 +102,8 @@ bug to fix immediately, not backlog.
| Rolling Docker previews: `:latest`, `:main`, `:rocm` | **`main` only** | `docker.yml` on every main push | tag timestamps move with main |
**Preview/RC policy:** there are no RC tags (beta cadence — see CLAUDE.md).
The preview channel *is* the release candidate, and it **always builds from
`main`** — the preview-gate in `release.yml` refuses `publish_preview` from
any other branch, and the rolling Docker tags track `main` by construction.
To get users testing a fix: merge to `main`, then cut a preview. Never a
side-branch build.
Rolling Docker previews always build from `main`. Desktop preview publication
is paused during the Electron transition; never publish a side-branch preview.
## 6. Expect-to-fail-first-time on Windows and Linux
@@ -161,3 +151,49 @@ before Tauri uploads them again. A macOS retry also replaces that architecture's
versionless updater archive. Other versions, sibling platforms, and updater
manifests remain intact. Inventory or deletion permission/network failures stop
the job instead of hiding an upload collision.
## Electron transition (next desktop release)
Electron is the primary desktop distribution. electron-release.yml builds Linux
x64, Windows x64, macOS arm64 and macOS x64, checks packaged startup and updater
artifacts, then creates a draft. Publishing requires a tag-scoped manual dispatch
with publish=true. Tag pushes never publish automatically. electron-build.yml
remains the artifact-only rehearsal; run it before tagging.
Set TAURI_SUNSET_TAG to the final Tauri version tag. Run the manual release.yml
on that tag first; it rejects other refs. Automatic Tauri builds and scheduled
previews are retired. Keep the transition release draft until Electron on the
same tag completes. Electron requires the final signed latest.json and
latest-user.json assets; subsequent releases copy those feeds without changing
their immutable sunset payload URLs. Retain the sunset release and its assets.
Write versioned CHANGELOG notes before release. Review all four platform builds,
checksums, signing requirements and docs/electron-migration.md. Existing Electron
artifact names and app IDs remain stable for updater compatibility. This pipeline
ships stable releases; rolling preview publication is paused during transition.
Preparation is not proof of cross-platform packaging, signing, migration, or a
real installed update hop. Record those results before release. Keep Tauri source
and shared assets until remaining Electron resource references are relocated.
No tag, version bump, or publishing is authorized by workflow preparation alone.
Electron signing uses ELECTRON_CSC_LINK and ELECTRON_CSC_KEY_PASSWORD secrets.
Without them rehearsal/draft artifacts are unsigned or ad-hoc signed. Publishing
checks macOS signing/notarization and Windows Authenticode signatures by default.
The owner may explicitly choose the existing unsigned-release policy by dispatching
with `allow_unsigned=true` (both dispatch and rerun actors must be the repository owner); the release notes then disclose OS trust warnings and
unverified macOS automatic updates. Never select this exception without the owner's
choice. Tauri's signing keys do not sign Electron packages.
For the transition tag, automatic Electron release jobs are skipped. Build the
manual Tauri sunset draft first, then dispatch Electron on the same tag after
its signed updater feeds exist. Later tags build Electron automatically.
If a packaging-workflow fix is needed after tagging, keep the release tag
immutable. Merge and validate the workflow fix on main, then dispatch
`electron-release.yml` from main with `release_tag=vX.Y.Z`. Validation and every
packaging/release job check out that exact tag; only the workflow comes from
main. Empty signing secrets are omitted from the builder environment so drafts
and explicitly accepted unsigned builds do not interpret the working directory
as a certificate. Publication still requires `publish=true` and the same guards.
docs/STRUCTURE.md
+2 -2
View File
@@ -39,7 +39,7 @@ VoiceStudio/
│ │ └── setup/ first-run wizard, model download
│ ├── core/ config, db, job queue, event bus, auth/CSRF, path security,
│ │ opt-in analytics, version, diagnostics
│ ├── services/ 85 modules of business logic — TTS, dubbing pipeline,
│ ├── services/ 86 modules of business logic — TTS, dubbing pipeline,
│ │ audio DSP, GPU gateway, engine routing, model lifecycle
│ ├── engines/ per-engine adapters: indextts, supertonic3, confucius4,
│ │ dots_tts, moss_tts_v15, pockettts, audiocpp,
@@ -103,7 +103,7 @@ VoiceStudio/
│
├── .agents/skills/ ⟵ canonical skill copies (vite, fastapi-python), pinned by
│ skills-lock.json — followed by path, never symlinked
├── skills/ ⟵ skills this repo publishes (omnivoice, oss-maintainer)
├── skills/ ⟵ skills this repo publishes (voicestudio, voicestudio-maintainer)
│
├── infra/ ⟵ edge/deploy workers (not the Docker deploy path)
│ └── install-redirect/ voicestudio.sh/install — UA-sniffing installer worker
docs/electron-dubbing.md
+6
View File
@@ -1,5 +1,11 @@
# Electron dubbing workspace
The idle workspace includes an original/dubbed demo comparison with compact
player controls. Sync playheads aligns positions without starting both videos.
Sample transcript edits are retained per language while the demo is mounted;
they do not regenerate the prerecorded audio. Edit on the dubbed card imports
that sample video into the normal upload/transcription and editing workflow.
Open Dub from the cloning sidebar or command search. Upload or drop audio/video, or explicitly submit a video URL;
preparation completes before transcription starts. The editor shows source text,
editable translated text, and per-segment voice/timing controls. Translation uses
docs/electron-macos-shell.md
+19
View File
@@ -0,0 +1,19 @@
# macOS desktop shell
The expanded sidebar reserves space for the native traffic lights and app name.
The collapsed sidebar is 64 px wide, with its toggle below the traffic lights
and its right divider beginning below the 72 px header region.
Notifications appear at the top right, with space reserved before the bell.
The notification menu opens downward and remains available while notification
data loads. Settings uses an icon in the macOS sidebar footer; Local device
sits beside it and opens the device and compute-target menu. The expanded
sidebar retains the Local device label.
Windows and Linux retain their existing notification and device placement.
The notification control follows workspace headers in document order so their
native drag regions cannot consume its mouse clicks. On macOS, run
`node tests/native-bell-repro.mjs` from `electron/` against the dev renderer
to verify a real system mouse click (requires Swift and Accessibility access).
Browser automation alone bypasses native titlebar hit testing.
docs/electron-migration.md
+20
View File
@@ -0,0 +1,20 @@
# Moving to Electron
The next desktop release uses Electron. Tauri receives one final sunset update;
subsequent releases build only Electron. Existing Tauri downloads remain available.
1. Find your Tauri data directory in Settings and back up the entire directory
while the app is closed. Keep reference audio stored outside it too.
2. Install Electron for your platform. Keep Tauri and its data until verification.
Do not run both apps against the same data directory.
3. Check Electron's configured data location before generating. Use its supported
storage/backend configuration to select the existing data directory.
4. Verify voices, projects, history, and model locations. Generate a short test
clip before removing the old app.
The shells share the backend, but shell preferences and credentials are not
guaranteed to migrate. Recheck devices, shortcuts, theme, backend address and
permissions. No automatic installer-to-installer migration is provided.
The final Tauri updater feeds retain signed Tauri payloads at immutable URLs.
A Tauri updater must never receive an Electron installer.
docs/electron-performance.md
+2 -2
View File
@@ -1,10 +1,10 @@
# Electron compute and performance settings
Settings > Compute device exposes the existing device override, Windows torch.compile workaround, generation time budgets, and hardware readouts.
Settings > Compute device exposes the existing device override, the torch.compile workaround, generation time budgets, and hardware readouts.
Device choices come from the backend's detected families plus Auto. The chosen preference and currently active family are displayed separately. Environment-pinned choices are disabled, an ignored unavailable override is explained, and a changed preference shows its actual restart requirement. Failed saves keep the last confirmed state. Nothing automatically restarts the backend or changes the active model.
The torch.compile workaround uses the same platform gate as Tauri: Windows can opt in; other platforms retain their working optimization. Generation budgets preserve separate GPU and CPU limits, validate the existing positive/21600-second range, and keep edits during refetches. An externally overridden budget reports that fact instead of implying the saved value will take effect after restart. Hardware RAM/VRAM readouts poll only while this view is mounted.
The torch.compile workaround matches Tauri: since #2135 it is selectable on every platform, because the compile failures it works around are not Windows-only. Generation budgets preserve separate GPU and CPU limits, validate the existing positive/21600-second range, and keep edits during refetches. An externally overridden budget reports that fact instead of implying the saved value will take effect after restart. Hardware RAM/VRAM readouts poll only while this view is mounted.
During synthesis, the fixed-width primary action polls the existing model-status contract and names the active runtime phase: starting the AI runtime, loading weights, warming speech recognition, optimizing the model, generating, or receiving audio. Model-load percentage and elapsed time share the reserved status line, and the progress track switches from model loading to streamed audio delivery without moving the controls.
docs/feature-catalog.md
+52
View File
@@ -0,0 +1,52 @@
# Features and engines
Engine availability depends on installed models, hardware, and configured providers.
## Features
- **Voice Cloning**
- **Voice Design**
- **Video Dubbing**
- **Dictation Widget**
- **Vocal Isolation**
- **Speaker Diarization**
- **Batch Queue**
- **MCP Server**
- **AI Watermark**
- **Local-first**
- **GPU Auto-Detect**
- **Remote Model Downloads**
- **Extensible**
## Speech generation
- **VoiceStudio** (default, powered by k2-fsa/OmniVoice)
- omnivoice-subprocess — [Guide](engines/omnivoice-subprocess.md)
- CosyVoice 3 — [Guide](engines/cosyvoice.md)
- KittenTTS
- MLX-Audio
- VoxCPM2
- MOSS-TTS-Nano
- gpt-sovits
- sherpa-onnx
- **IndexTTS 2.5** ⚡ — [Guide](engines/indextts.md)
- omnivoice-gguf
- supertonic3
- **MOSS-TTS-v1.5** — [Guide](engines/moss-tts-v15.md)
- **dots.tts** — [Guide](engines/dots-tts.md)
- **Confucius4-TTS** — [Guide](engines/confucius4-tts.md)
- pockettts
- audiocpp — [Guide](engines/audio-cpp.md)
## Transcription
- **WhisperX** (default)
- Faster-Whisper
- MLX Whisper
- PyTorch Whisper
- Parakeet TDT
- Parakeet TDT v3 (MLX)
- Moonshine
- FunASR
- **sherpa-onnx** (live dictation)
- **OpenAI-compatible** ⚠️ configured server
docs/features.yaml
+1
View File
@@ -1,3 +1,4 @@
catalog: docs/feature-catalog.md
# Canonical feature inventory — the single source of truth that the daily
# docs-drift job (.github/workflows/docs-drift.yml) diffs against README.md,
# docs/, and the engine registries via scripts/check-docs-drift.py.
docs/hardware-notes-tesla-t4.md
+2 -2
View File
@@ -53,8 +53,8 @@ that the app already runs the "fast" preset unless you override it via `/generat
| dtype | `torch.float16` hardcoded for the `omnivoice` engine (`model_manager.py`) — correct for Turing (no bf16 tensor cores this generation). No env var override for this engine specifically (ASR engines have `ASR_COMPUTE_TYPE`; `dots_tts`/`indextts` have their own precision vars; `omnivoice` doesn't). |
| Attention | `sdpa`, selected automatically since `flash_attn` isn't installed (`_supports_flash_attn_2=True` is declared but the package itself is absent) — safe on T4. |
| int8 | No int8 path for this engine (ASR's CTranslate2 `int8` and `sherpa-onnx`'s int8 ONNX models are separate/unrelated). |
| CUDA Graphs | No direct API usage in the app. Reachable indirectly via `torch.compile(mode="reduce-overhead")`, which the app attempts **by default** on this GPU (T4/sm_75 isn't in the framework's compile-exclusion list, unlike newer/Blackwell GPUs). The numbers above were measured with `TORCH_COMPILE_DISABLE=1` for a clean eager baseline. |
| torch.compile | Attempted by default on T4 (see above) — not evaluated further here. |
| CUDA Graphs | **Not used on T4 any more (#2135).** Reachable only indirectly via `torch.compile(mode="reduce-overhead")`, which the app used to attempt by default here — and which killed the backend process outright on the first `/generate` (no traceback, no HTTP response). The app now picks the compile mode per GPU and drops to the non-cudagraph `default` mode below sm_80. `OMNIVOICE_FORCE_CUDAGRAPH=1` restores the old behaviour for benchmarking. |
| torch.compile | Still attempted on T4, in `default` mode — compiled Inductor kernels, no graph capture. Disable entirely with Settings → Performance → "Disable torch.compile" or `TORCH_COMPILE_DISABLE=1`. |
## VRAM
docs/install/linux.md
+24 -7
View File
@@ -1,5 +1,22 @@
# VoiceStudio — Install on Linux
## Electron desktop (current)
From the repository root, install Bun and uv, then run:
```sh
bun install
bun run dev
```
Use `bun run desktop-prod` to build and launch Electron, or `bun run dist`
to create local installers without publishing. The app manages its backend.
See [Electron setup](../../electron/README.md) and [migration notes](../electron-migration.md).
## Legacy Tauri installation and troubleshooting
The instructions below apply to the sunset Tauri app and existing Tauri installers.
This page is self-contained: follow it top to bottom and you'll end up with a
working VoiceStudio install on a Debian / Ubuntu / Fedora / Arch host.
@@ -28,7 +45,7 @@ Everything above, plus the toolchain:
`sudo dnf install python3.11` on Fedora, or already installed on Arch.
- **Bun** — `curl -fsSL https://bun.sh/install | bash`.
- **Rust / Cargo** — `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh` or via your package manager (e.g., `sudo apt install rustc cargo`).
If you use rustup, reopen the shell or source `"$HOME/.cargo/env"` before running `bun run desktop-prod`.
If you use rustup, reopen the shell or source `"$HOME/.cargo/env"` before running `bun run tauri:desktop-prod`.
- **GTK/WebKit deps** for the Tauri shell:
```bash
@@ -66,10 +83,10 @@ git clone https://github.com/debpalash/VoiceStudio.git
cd VoiceStudio
bun install
source "$HOME/.cargo/env" # only needed in a shell opened before rustup finished
bun desktop # development build with hot reload
bun tauri # development build with hot reload
```
Use `bun run desktop-prod` instead when you need to build and launch the
Use `bun run tauri:desktop-prod` instead when you need to build and launch the
production bundle. Both commands create the Python environment via `uv`, sync
dependencies, and start the backend automatically; do not start the backend in
a second terminal.
@@ -87,7 +104,7 @@ pkg-config --exists \
&& echo "Tauri system libraries are ready"
```
`bun desktop` also checks the native `libxdo` linker input and GStreamer's
`bun tauri` also checks the native `libxdo` linker input and GStreamer's
`autoaudiosink` before starting. The latter is required even if you do not plan
to record: WebKitGTK 2.52 aborts its renderer when a page creates an audio
element without that plugin, which otherwise turns a running app blank. The
@@ -256,7 +273,7 @@ If you are on v0.4.0 or older, either update or build from source:
git clone https://github.com/debpalash/VoiceStudio.git
cd VoiceStudio
bun install
bun run desktop-prod
bun run tauri:desktop-prod
```
Tracking issues: [#62](https://github.com/debpalash/VoiceStudio/issues/62),
@@ -389,9 +406,9 @@ reinstall and left the CPU-only CUDA build in place).
**2. Environment variable (existing installs / headless / source).** Set
`OMNIVOICE_TORCH_VARIANT=rocm` before launching — the next bootstrap performs
the same ROCm reinstall. Source installs honour it too:
`OMNIVOICE_TORCH_VARIANT=rocm bun run desktop` swaps torch right after
`OMNIVOICE_TORCH_VARIANT=rocm bun run tauri` swaps torch right after
`uv sync` and launches the backend without re-syncing, so the wheel is not
reverted on the next start (#1665). Without the variable, `bun run desktop`
reverted on the next start (#1665). Without the variable, `bun run tauri`
restores the lockfile's CUDA build — a hand-swapped ROCm wheel does not
survive it. `OMNIVOICE_TORCH_INDEX=<url>` overrides the wheel
index when you need a different ROCm version — e.g. AMD publishes newer
docs/install/macos.md
+19 -2
View File
@@ -1,5 +1,22 @@
# VoiceStudio — Install on macOS
## Electron desktop (current)
From the repository root, install Bun and uv, then run:
```sh
bun install
bun run dev
```
Use `bun run desktop-prod` to build and launch Electron, or `bun run dist`
to create local installers without publishing. The app manages its backend.
See [Electron setup](../../electron/README.md) and [migration notes](../electron-migration.md).
## Legacy Tauri installation and troubleshooting
The instructions below apply to the sunset Tauri app and existing Tauri installers.
This page is self-contained: follow it top to bottom and you'll end up with a
working VoiceStudio install on macOS (Apple Silicon).
@@ -36,7 +53,7 @@ Everything above, plus the toolchain:
- **Python 3.11+** — `brew install python@3.11` (or use `pyenv` / the system Python if you already have ≥3.11).
- **Bun** — `curl -fsSL https://bun.sh/install | bash`.
- **Rust / Cargo** — `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh` or `brew install rust`.
If you use rustup, reopen the terminal or source `"$HOME/.cargo/env"` before running `bun run desktop-prod`.
If you use rustup, reopen the terminal or source `"$HOME/.cargo/env"` before running `bun run tauri:desktop-prod`.
FFmpeg/FFprobe and yt-dlp are **not** prerequisites on any install path: the
app resolves them itself (a static build ships with the Python environment;
@@ -63,7 +80,7 @@ Or manually:
git clone https://github.com/debpalash/VoiceStudio.git
cd VoiceStudio
bun install
bun run desktop-prod
bun run tauri:desktop-prod
```
The first launch builds the Tauri shell, creates the Python venv via `uv`,
docs/install/troubleshooting.md
+36
View File
@@ -248,6 +248,42 @@ peak memory footprint that exceeds free VRAM. Windows-only quirk.
**Linked issue:** [#65](https://github.com/debpalash/VoiceStudio/issues/65)
## 5a. Backend dies on the first `/generate` (older NVIDIA GPUs, e.g. Tesla T4)
**Symptom:** the backend starts fine, `/health` reports your GPU, the model
preloads — and then the first generation request returns
`RemoteDisconnected: Remote end closed connection without response`. Every call
after it gets `ConnectionRefused`, because the backend process is gone. No
Python traceback is printed.
**Cause:** `torch.compile(mode="reduce-overhead")` captures CUDA graphs. On
pre-Ampere cards (Turing sm_75 / Volta sm_70 — the Tesla T4 on Google Colab is
the common case) that capture can abort the process from inside the native CUDA
library. It happens below the interpreter, so no `except` in the app can catch
it and nothing is logged.
**Fix:** update — VoiceStudio now selects the compile mode per GPU and does not
capture CUDA graphs below sm_80, so this should no longer happen. If you still
see a crash in the generate path on any GPU, turn compilation off entirely:
- **In the app:** Settings → Performance → **"Disable torch.compile"**.
- **From the CLI / from source:** `TORCH_COMPILE_DISABLE=1` before launching.
This is honoured on every platform and by every engine, in-process or
sidecar.
**Getting a traceback:** the backend now arms `faulthandler`, so a native crash
writes the faulting thread's Python stack to `backend_err.log` on the way down.
Include that stack when reporting — without it a native crash is unattributable.
(`OMNIVOICE_DISABLE_FAULTHANDLER=1` turns it off.)
**Extra containment:** to keep a crashing engine from taking the API down with
it, run the engine in a killable child process — select
**OmniVoice (subprocess-isolated)** in Settings → Engines, or
`OMNIVOICE_TTS_BACKEND=omnivoice-subprocess`. The parent then returns an HTTP
error and respawns the sidecar instead of dying.
**Linked issue:** [#2135](https://github.com/debpalash/VoiceStudio/issues/2135)
## 5b. RTX 50-series (Blackwell, sm_120): backend crashes during `ml_imports`
**Symptom:** on an RTX 5070 / 5070 Ti / 5080 / 5090, the backend never becomes
docs/install/windows.md
+34 -13
View File
@@ -1,5 +1,22 @@
# VoiceStudio — Install on Windows
## Electron desktop (current)
From the repository root, install Bun and uv, then run:
```sh
bun install
bun run dev
```
Use `bun run desktop-prod` to build and launch Electron, or `bun run dist`
to create local installers without publishing. The app manages its backend.
See [Electron setup](../../electron/README.md) and [migration notes](../electron-migration.md).
## Legacy Tauri installation and troubleshooting
The instructions below apply to the sunset Tauri app and existing Tauri installers.
This page is self-contained: follow it top to bottom and you'll end up with a
working VoiceStudio install on Windows 10 / 11 (x64).
@@ -20,7 +37,7 @@ by the app itself on first launch. No toolchain needed.
Everything above, plus the toolchain:
- **Git for Windows** — `winget install --id Git.Git -e`. Needed for
`git clone`, and it includes **Git Bash**, which `bun run desktop-prod`
`git clone`, and it includes **Git Bash**, which `bun run tauri:desktop-prod`
uses to run its build-and-launch script. Without it, `desktop-prod` stops
with an error telling you to install it.
- **Python 3.11+** — `winget install Python.Python.3.11` (or download from
@@ -32,7 +49,7 @@ Everything above, plus the toolchain:
- **Bun** — `powershell -c "irm bun.sh/install.ps1 | iex"`.
- **FFmpeg** — `winget install Gyan.FFmpeg`.
- **Rust / Cargo** — `winget install Rust.Rustup` or download `rustup-init.exe` from [rustup.rs](https://rustup.rs/).
After installing Rustup, close and reopen PowerShell before running `bun run desktop-prod`.
After installing Rustup, close and reopen PowerShell before running `bun run tauri:desktop-prod`.
## GPU support on Windows
@@ -64,17 +81,17 @@ Or manually:
git clone https://github.com/debpalash/VoiceStudio.git
cd VoiceStudio
bun install
bun run desktop-prod
bun run tauri:desktop-prod
```
The first launch creates the Python venv via `uv`, syncs deps, and downloads
model weights. The splash screen shows progress.
> **Note:** `bun run desktop-prod` runs a bash script under the hood. You can
> **Note:** `bun run tauri:desktop-prod` runs a bash script under the hood. You can
> launch it from PowerShell or cmd as shown — it finds Git Bash automatically
> (installed with Git for Windows, see Prerequisites). If no Git Bash is
> found, it prints instructions instead of failing silently. Alternatives
> that don't need bash: `bun run desktop` (dev mode) or the pre-built MSI
> that don't need bash: `bun run tauri` (dev mode) or the pre-built MSI
> below.
## Install (pre-built MSI)
@@ -274,21 +291,25 @@ synthesise call. On machines with <16 GB VRAM, that compile step can OOM
failed`.
**The one-click fix:** open **Settings → Performance** in the app and toggle
**"Disable torch.compile (Windows)"** on. That sets the
`TORCH_COMPILE_DISABLE=1` env var on every engine subprocess VoiceStudio spawns,
which falls back to the eager-mode kernel path. You'll lose a few percent of
peak throughput in exchange for the engine actually loading.
**"Disable torch.compile"** on. That sets the `TORCH_COMPILE_DISABLE=1` env var
on every engine subprocess VoiceStudio spawns and forces the in-process engine
to eager mode as well. You'll lose a few percent of peak throughput in exchange
for the engine actually loading.
**From the CLI / from source:** set the env var manually before launching:
```powershell
$env:TORCH_COMPILE_DISABLE = "1"
bun run desktop-prod
bun run tauri:desktop-prod
```
This setting is a no-op on macOS and Linux (the OOM is Windows-specific —
the `torch.compile` kernel cache behaves differently on the other platforms).
Tracking issue: [#65](https://github.com/debpalash/VoiceStudio/issues/65).
The OOM this section describes is Windows-specific, but the toggle itself works
on **every** platform — it used to be greyed out elsewhere, which left Linux and
macOS users with no way to switch off a `torch.compile` that was breaking their
engine. Tracking issues:
[#65](https://github.com/debpalash/VoiceStudio/issues/65) (this OOM) and
[#2135](https://github.com/debpalash/VoiceStudio/issues/2135) (the same toggle
on Linux/CUDA).
## Hugging Face token (optional but recommended)
docs/media/sponsor-slot.svg
+10
View File
@@ -0,0 +1,10 @@
<svg xmlns="http://www.w3.org/2000/svg" width="640" height="104" viewBox="0 0 640 104">
<title>Your brand — partner with VoiceStudio</title>
<desc>Apply for a paid featured placement in the app, integrations directory, and README.</desc>
<rect x="1" y="1" width="638" height="102" rx="16" fill="#191420" stroke="#574051"/>
<circle cx="52" cy="52" r="23" fill="#30212e" stroke="#d3869b" stroke-width="1.5"/>
<text x="52" y="60" text-anchor="middle" fill="#f5eaf2" font-family="Arial,sans-serif" font-size="26" font-weight="700">?</text>
<text x="94" y="43" fill="#f5eaf2" font-family="Arial,sans-serif" font-size="19" font-weight="600">Your brand, where people build with voice.</text>
<text x="94" y="69" fill="#c0adbd" font-family="Arial,sans-serif" font-size="13">App placement · Integration page · README exposure</text>
<path d="M590 61l18-18m-15 0h15v15" fill="none" stroke="#d3869b" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"/>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 982 B

docs/performance.md
+14 -3
View File
@@ -102,9 +102,20 @@ where the runtime check says it can work (a CUDA device with Triton importable
and a supported GPU architecture) and skipped automatically everywhere else —
MPS, CPU, and the typical Windows install (Triton ships no Windows wheel).
The one user-facing control is Settings → Performance → "Disable
torch.compile" (shown on Windows), for the rare setup where a partial Triton
install makes the probe pass but the compile attempt itself crash — see
[Windows install notes](install/windows.md).
torch.compile", available on every platform, for the setup where the probe
passes but the compile attempt itself misbehaves — a partial Triton install,
or a GPU whose compiled kernels crash the engine. Setting
`TORCH_COMPILE_DISABLE=1` (or `TORCHDYNAMO_DISABLE=1`) in the environment does
the same thing and is honoured by both the in-process engine and every engine
subprocess. See [Windows install notes](install/windows.md).
On CUDA the compile **mode** is chosen per GPU: Ampere (sm_80) and newer use
`reduce-overhead`, which captures CUDA graphs; older cards (Turing/Volta, e.g.
the Tesla T4) fall back to the plain `default` mode, because graph capture was
observed to abort the whole backend process there
([#2135](https://github.com/debpalash/VoiceStudio/issues/2135)). They still get
compiled Inductor kernels. `OMNIVOICE_FORCE_CUDAGRAPH=1` restores the
cudagraph mode if you want to benchmark it.
## Warnings before a slow generation
docs/remote-workers.md
+6
View File
@@ -396,3 +396,9 @@ would disrupt the machine or network, including airplane mode, simultaneous
downloads, and stopping a worker during an audiobook, are printed as exact
`MANUAL` steps and are never reported as passed automatically. A failed
precondition or automated check exits non-zero.
Remote compute targets show available CPU/GPU usage and free VRAM. Unavailable
metrics are omitted; a transient sampling failure retains the last successful
reading. Telemetry runs off the control loop with at most one probe per worker
client, retained across reconnects. Read-only probes never block task draining or
shutdown; a stuck driver probe cannot accumulate more threads.
electron/README.md
+8 -16
View File
@@ -1,19 +1,11 @@
# VoiceStudio — Electron shell (preview)
# VoiceStudio — Electron desktop app
An Electron rewrite of the desktop shell, built page by page. Today it ships
**Voice cloning** only; the Tauri app in `frontend/` remains the product.
Electron is the primary desktop app for voice cloning, stories, dubbing,
transcription, voice design, and workflows. Tauri is retained only for its final
sunset update; see [migration notes](../docs/electron-migration.md).
Both shells talk to the same local FastAPI backend (`backend/`, port 3900), so
voices, history and installed engines are shared. Nothing leaves the machine.
The Electron UI follows T3 Code's styling foundation: shadcn Base UI Mira,
Zinc light surfaces, near-black dark surfaces, blue actions, system fonts,
compact controls, and translucent popovers/dialogs. Shared palette roles live in
`src/renderer/src/styles/t3-theme.css`; app geometry and surface utilities live in
`styles/globals.css`. The palette is adapted from
[T3 Code](https://github.com/pingdotgg/t3code/blob/main/apps/web/src/index.css)
under the MIT license (see `T3CODE-LICENSE.txt`).
Light/dark switching is local; T3's theme editor and theme library are not included.
The runtime supervisor manages the local FastAPI backend. Network integrations
and remote workers require configuration; local generation stays on your machine.
## Stack
@@ -28,7 +20,7 @@ Light/dark switching is local; T3's theme editor and theme library are not inclu
## Run it
```sh
cd electron
# From the repository root
bun install
bun run dev # electron-vite: main + preload + renderer with HMR
```
@@ -57,7 +49,7 @@ app-relative `/api/...`:
```sh
bun run typecheck # tsgo, both projects
bun run check # vp: format + lint + types
bun run check:electron # types, tests, build, packaging contract
bun run test # vitest (jsdom)
bun run build # electron-vite build → out/
bun run dist # + electron-builder → release/
electron/electron-builder.config.mjs
+4 -4
View File
@@ -94,13 +94,13 @@ export default {
'utf8',
).match(/<key>NSMicrophoneUsageDescription<\/key>\s*<string>([^<]+)<\/string>/)[1],
},
target: [
{ target: 'dmg', arch: ['arm64', 'x64'] },
{ target: 'zip', arch: ['arm64', 'x64'] },
],
// The CLI matrix selects one architecture per runner and updater feed.
target: ['dmg', 'zip'],
category: 'public.app-category.productivity',
},
linux: {
// Linux targets rewrite ${arch} to x86_64/amd64; feeds use Node's x64.
artifactName: 'VoiceStudio-Electron-${version}-linux-x64.${ext}',
icon: '../frontend/src-tauri/icons/icon.png',
syncDesktopName: true,
target: ['AppImage', 'deb'],
electron/src/main/models-directory-authorization.test.ts
+3 -3
View File
@@ -1,4 +1,4 @@
import { mkdtemp, readFile, rm } from 'node:fs/promises';
import { mkdtemp, readFile, realpath, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { afterEach, expect, it } from 'vitest';
@@ -16,14 +16,14 @@ it('mints one-shot models-directory capabilities only after a writable directory
const selected = join(root, 'model cache');
const result = await authorizeModelsDirectory(dataDir, selected);
expect(result.path).toBe(selected);
expect(result.path).toBe(await realpath(selected));
const payload = JSON.parse(
await readFile(join(dataDir, '.path-authorizations', `${result.authorization}.json`), 'utf8'),
);
expect(payload).toEqual({
token: result.authorization,
kind: 'models_dir',
path: selected,
path: await realpath(selected),
});
});
electron/src/main/runtime-project.test.ts
+3 -3
View File
@@ -101,7 +101,7 @@ describe('packaged runtime setup', () => {
'installing_deps',
'verifying',
]);
expect(run.mock.calls).toHaveLength(3);
expect(run.mock.calls).toHaveLength(process.platform === 'darwin' ? 2 : 3);
expect(await runtimeReady(bundle, project)).toBe(true);
await writeFile(join(bundle, 'uv.lock'), 'updated dependencies');
expect(await runtimeReady(bundle, project)).toBe(false);
@@ -232,7 +232,7 @@ describe('packaged runtime setup', () => {
expect(fetch).not.toHaveBeenCalled();
expect(run.mock.calls[0]?.[0]).toBe(executable);
});
it('installs and validates cuDNN 8 compatibility on a CUDA runtime', async () => {
it.skipIf(process.platform === 'darwin')('installs and validates cuDNN 8 compatibility on a CUDA runtime', async () => {
const { bundle, project } = await fixture();
const sitePackages = join(project, '.venv', 'Lib', 'site-packages');
const run = vi.fn(async (command: string, args: string[]) => {
@@ -279,7 +279,7 @@ describe('packaged runtime setup', () => {
expect(compatInstall?.[1]).toContain(join(sitePackages, 'cudnn8_compat'));
expect(await runtimeReady(bundle, project)).toBe(true);
});
it('keeps a CUDA runtime incomplete when the compatibility wheel is partial', async () => {
it.skipIf(process.platform === 'darwin')('keeps a CUDA runtime incomplete when the compatibility wheel is partial', async () => {
const { bundle, project } = await fixture();
const sitePackages = join(project, '.venv', 'Lib', 'site-packages');
const run = vi.fn(async (command: string, args: string[]) => {
electron/src/renderer/src/components/app-shell/app-shell.tsx
+22 -1
View File
@@ -4,15 +4,26 @@ import { CommandPalette } from '@/components/command-palette';
import { Outlet, useRouterState } from '@tanstack/react-router';
import { BackendGate } from '../backend-gate';
import { RepairAgentDock } from './repair-agent-dock';
import { isMac } from '../bridge';
import { cn } from '@/lib/utils';
import { useBackendStatus } from '@/hooks/use-backend-status';
import { SystemNotifications } from './system-notifications';
export function AppShell() {
const backend = useBackendStatus();
const pathname = useRouterState({
select: (state) => state.location.pathname,
});
const settings = pathname.startsWith('/settings');
const macWorkspace = isMac() && !settings;
const SettingsWorkspace = pathname === '/settings/openapi' ? 'div' : 'main';
return (
<div className="app-surface relative flex h-full flex-col bg-background text-foreground">
<div
className={cn(
'app-surface relative flex h-full flex-col bg-background text-foreground',
macWorkspace && 'macos-notification-safe-area',
)}
>
<div className="flex min-h-0 flex-1">
{settings ? (
<>
@@ -41,6 +52,16 @@ export function AppShell() {
</BackendGate>
)}
</div>
{/* Native drag-region hit testing follows document order. Keep this
no-drag control after every workspace titlebar, outside BackendGate. */}
{macWorkspace && (
<div
data-slot="macos-system-notifications"
className="app-no-drag fixed top-3.5 right-3.5 z-50"
>
<SystemNotifications enabled={backend.stage === 'ready'} titlebar />
</div>
)}
</div>
);
}
electron/src/renderer/src/components/app-shell/sponsor-footer.tsx
+3 -2
View File
@@ -1,3 +1,4 @@
import { runRendererTask } from '@/lib/global-error-recovery';
import { VOICE_AI_DIRECTORY } from '../../../../../../frontend/src/config/voice-ai-directory';
import './sponsor-footer.css';
import { useNavigate } from '@tanstack/react-router';
@@ -166,7 +167,7 @@ export function SponsorFooter() {
type="button"
className="sponsor-catalog-toggle"
aria-label={t('integrationCatalog.title')}
onClick={() => void navigate({ to: '/integrations' })}
onClick={() => runRendererTask('Open integrations', () => navigate({ to: '/integrations' }))}
title={t('integrationCatalog.title')}
>
<BlocksIcon aria-hidden="true" className="size-4" />
@@ -314,7 +315,7 @@ export function SponsorFooter() {
aria-label={t('supportPlans.remove')}
className={linkClass + ' justify-center px-2'}
onClick={() =>
void navigate({ to: '/settings/support', search: { compare: true } })
runRendererTask('Open support', () => navigate({ to: '/settings/support', search: { compare: true } }))
}
/>
}
electron/src/renderer/src/components/app-shell/status-bar.tsx
+146 -57
View File
@@ -1,4 +1,4 @@
import { useRef, useState } from 'react';
import { useRef, useState, type ReactNode } from 'react';
import { Tooltip, TooltipTrigger, TooltipContent } from '@/components/ui/tooltip';
import { useTranslationEngines } from '@/features/settings/translation-settings';
import { Link } from '@tanstack/react-router';
@@ -79,6 +79,13 @@ function boundedPercent(value: number, total = 100) {
return Math.max(0, Math.min(100, (value / total) * 100));
}
function formatBytes(bytes: number) {
const gib = Math.max(0, bytes) / 1024 ** 3;
return new Intl.NumberFormat(undefined, {
style: 'unit', unit: 'gigabyte', maximumFractionDigits: gib >= 10 ? 0 : 1,
}).format(gib);
}
function DeviceMetric({
Icon,
label,
@@ -222,7 +229,15 @@ function EngineTip({
);
}
export function StatusBar({ compact = false }: { compact?: boolean }) {
export function StatusBar({
compact = false,
inline = false,
footerLeading,
}: {
compact?: boolean;
inline?: boolean;
footerLeading?: ReactNode;
}) {
const { t } = useTranslation();
const [expanded, setExpanded] = useState(false);
const [deviceOpen, setDeviceOpen] = useState(false);
@@ -477,7 +492,7 @@ export function StatusBar({ compact = false }: { compact?: boolean }) {
</div>
<ComputeTargetChoices data={computeTarget.data} />
{activeRemoteTarget ? (
<div className="space-y-1 rounded-lg border border-border/55 bg-muted/20 px-3 py-2 text-xs">
<div className="space-y-3 rounded-lg border border-border/55 bg-muted/20 p-2.5 text-xs">
<p className="truncate text-foreground/85" title={activeRemoteTarget.endpoint}>
{activeRemoteTarget.endpoint}
</p>
@@ -491,6 +506,42 @@ export function StatusBar({ compact = false }: { compact?: boolean }) {
{activeRemoteTarget.active_tasks}/{activeRemoteTarget.max_tasks}
</span>
</div>
{activeRemoteTarget.cpu_percent != null && (
<DeviceMetric
Icon={CpuIcon}
label={t('settings.device_family_cpu')}
value={`${Math.round(activeRemoteTarget.cpu_percent)}%`}
percent={activeRemoteTarget.cpu_percent}
/>
)}
{activeRemoteTarget.gpu_name &&
(activeRemoteTarget.gpu_utilization_percent != null ||
(activeRemoteTarget.free_memory_bytes != null && activeRemoteTarget.gpu_memory_bytes > 0)) && (
<DeviceMetric
Icon={MonitorUpIcon}
label={t('settings.device_family_gpu')}
value={
[
activeRemoteTarget.gpu_utilization_percent != null
? `${Math.round(activeRemoteTarget.gpu_utilization_percent)}%`
: null,
activeRemoteTarget.free_memory_bytes != null && activeRemoteTarget.gpu_memory_bytes > 0
? `${formatBytes(
activeRemoteTarget.gpu_memory_bytes - activeRemoteTarget.free_memory_bytes,
)} / ${formatBytes(activeRemoteTarget.gpu_memory_bytes)}`
: null,
]
.filter((value): value is string => value != null)
.join(' · ')
}
percent={
activeRemoteTarget.free_memory_bytes != null && activeRemoteTarget.gpu_memory_bytes > 0
? boundedPercent(activeRemoteTarget.gpu_memory_bytes - activeRemoteTarget.free_memory_bytes, activeRemoteTarget.gpu_memory_bytes)
: activeRemoteTarget.gpu_utilization_percent ?? 0
}
detail={activeRemoteTarget.gpu_name}
/>
)}
</div>
) : deviceUsage.isError ? (
<button
@@ -661,72 +712,88 @@ export function StatusBar({ compact = false }: { compact?: boolean }) {
: 'modelSettings.unavailable',
},
];
const iconDevicePopover = (
<Popover open={deviceOpen} onOpenChange={setDeviceOpen}>
<PopoverTrigger
render={
<button
type="button"
aria-label={`${deviceLabel}: ${deviceStageText}`}
className={cn(engineLinkClass, 'h-7 w-full', !compact && 'justify-start gap-2 px-2')}
/>
}
>
<CpuIcon className={engineIconClass} aria-hidden="true" />
{!compact && <span className="min-w-0 truncate">{deviceLabel}</span>}
<span
className={cn(
compact
? 'absolute inset-x-2 bottom-0.5 h-0.5 rounded-full'
: 'ml-auto size-1.5 shrink-0 rounded-full',
deviceDot,
)}
aria-hidden="true"
/>
</PopoverTrigger>
{deviceContent}
</Popover>
);
if (compact) {
return (
<footer className="shrink-0 border-t border-border/50 px-1.5 py-2 text-muted-foreground">
<Popover open={deviceOpen} onOpenChange={setDeviceOpen}>
<PopoverTrigger
render={
<button
type="button"
aria-label={`${deviceLabel}: ${deviceStageText}`}
className={cn(engineLinkClass, 'w-full')}
/>
}
>
<CpuIcon className={engineIconClass} aria-hidden="true" />
<span
className={cn('absolute inset-x-2 bottom-0.5 h-0.5 rounded-full', deviceDot)}
aria-hidden="true"
/>
</PopoverTrigger>
{deviceContent}
</Popover>
<footer
className={cn(
'shrink-0 text-muted-foreground',
inline ? 'contents' : 'border-t border-border/50 px-1.5 py-2',
)}
>
{iconDevicePopover}
</footer>
);
}
return (
<footer className="@container/engines w-full min-w-0 max-w-full border-t border-border/50 px-3 py-1.5 text-[length:var(--text-caption)] text-muted-foreground">
<div>
<div className="flex items-center gap-0.5">
<Popover open={deviceOpen} onOpenChange={setDeviceOpen}>
<PopoverTrigger
render={
<button
type="button"
aria-label={`${deviceLabel}: ${deviceStageText}`}
className="group/status flex min-w-0 flex-1 items-center gap-2 rounded-md px-2 py-1 text-left outline-none transition-[background-color,box-shadow,backdrop-filter] duration-150 hover:bg-sidebar-accent/65 hover:backdrop-blur-xl hover:shadow-[inset_0_1px_0_rgb(255_255_255/8%),0_5px_14px_rgb(0_0_0/10%)] hover:ring-1 hover:ring-inset hover:ring-sidebar-border/60 focus-visible:ring-2 focus-visible:ring-ring"
{!footerLeading && (
<div className="flex items-center gap-0.5">
<Popover open={deviceOpen} onOpenChange={setDeviceOpen}>
<PopoverTrigger
render={
<button
type="button"
aria-label={`${deviceLabel}: ${deviceStageText}`}
className="group/status flex min-w-0 flex-1 items-center gap-2 rounded-md px-2 py-1 text-left outline-none transition-[background-color,box-shadow,backdrop-filter] duration-150 hover:bg-sidebar-accent/65 hover:backdrop-blur-xl hover:shadow-[inset_0_1px_0_rgb(255_255_255/8%),0_5px_14px_rgb(0_0_0/10%)] hover:ring-1 hover:ring-inset hover:ring-sidebar-border/60 focus-visible:ring-2 focus-visible:ring-ring"
/>
}
>
<span
className={cn('size-1.5 shrink-0 rounded-full', deviceDot)}
aria-hidden="true"
/>
}
<CpuIcon className="size-3.5 shrink-0" aria-hidden="true" />
<span className="min-w-0 flex-1 truncate" role="status">
{deviceLabel}
</span>
</PopoverTrigger>
{deviceContent}
</Popover>
<button
type="button"
aria-label={expanded ? t('paneActions.collapse') : t('modelSettings.models')}
aria-expanded={expanded}
aria-controls="sidebar-engine-details"
onClick={() => setExpanded((value) => !value)}
className="flex size-7 shrink-0 items-center justify-center rounded-md outline-none transition-colors hover:bg-sidebar-accent/65 focus-visible:ring-2 focus-visible:ring-ring"
>
<span
className={cn('size-1.5 shrink-0 rounded-full', deviceDot)}
<ChevronDownIcon
className={cn(
'size-3.5 transition-transform duration-150 motion-reduce:transition-none',
expanded && 'rotate-180',
)}
aria-hidden="true"
/>
<CpuIcon className="size-3.5 shrink-0" aria-hidden="true" />
<span className="min-w-0 flex-1 truncate" role="status">
{deviceLabel}
</span>
</PopoverTrigger>
{deviceContent}
</Popover>
<button
type="button"
aria-label={expanded ? t('paneActions.collapse') : t('modelSettings.models')}
aria-expanded={expanded}
aria-controls="sidebar-engine-details"
onClick={() => setExpanded((value) => !value)}
className="flex size-7 shrink-0 items-center justify-center rounded-md outline-none transition-colors hover:bg-sidebar-accent/65 focus-visible:ring-2 focus-visible:ring-ring"
>
<ChevronDownIcon
className={cn(
'size-3.5 transition-transform duration-150 motion-reduce:transition-none',
expanded && 'rotate-180',
)}
aria-hidden="true"
/>
</button>
</div>
</button>
</div>
)}
<div
ref={tipAnchor}
className="mt-0.5 grid w-full min-w-0 grid-cols-6 gap-1 rounded-lg border border-border/50 bg-sidebar-accent/25 p-1"
@@ -809,6 +876,28 @@ export function StatusBar({ compact = false }: { compact?: boolean }) {
<PerformanceProfile tooltipAnchor={tipAnchor} />
</div>
)}
{footerLeading && (
<div className="mt-1.5 flex items-center gap-1 border-t border-border/50 pt-1.5">
{footerLeading}
<div className="min-w-0 flex-1">{iconDevicePopover}</div>
<button
type="button"
aria-label={expanded ? t('paneActions.collapse') : t('modelSettings.models')}
aria-expanded={expanded}
aria-controls="sidebar-engine-details"
onClick={() => setExpanded((value) => !value)}
className="ml-auto flex size-7 shrink-0 items-center justify-center rounded-md outline-none transition-colors hover:bg-sidebar-accent/65 focus-visible:ring-2 focus-visible:ring-ring"
>
<ChevronDownIcon
className={cn(
'size-3.5 transition-transform duration-150 motion-reduce:transition-none',
expanded && 'rotate-180',
)}
aria-hidden="true"
/>
</button>
</div>
)}
</div>
</footer>
);
electron/src/renderer/src/components/app-shell/system-notifications.test.tsx
+27
View File
@@ -46,6 +46,13 @@ describe('SystemNotifications desktop updates', () => {
beforeEach(() => {
mocks.navigate.mockReset();
mocks.listener = undefined;
mocks.state = {
status: 'available',
currentVersion: '0.5.2',
availableVersion: '0.5.3',
channel: 'stable',
progress: 0,
} as UpdateState;
});
afterEach(cleanup);
@@ -62,4 +69,24 @@ describe('SystemNotifications desktop updates', () => {
fireEvent.click(await screen.findByText('common.open'));
await waitFor(() => expect(mocks.navigate).toHaveBeenCalledWith({ to: '/settings/updates' }));
});
test('shows unavailable instead of endless loading when the backend is offline', async () => {
mocks.state = {
status: 'idle',
currentVersion: '0.5.2',
channel: 'stable',
progress: 0,
} as UpdateState;
render(
<QueryClientProvider client={new QueryClient()}>
<SystemNotifications enabled={false} titlebar />
</QueryClientProvider>,
);
const trigger = await screen.findByRole('button', { name: 'modelSettings.unavailable' });
expect(trigger).toBeEnabled();
expect(trigger).toHaveClass('app-no-drag');
fireEvent.click(trigger);
expect(await screen.findByText('modelSettings.unavailable')).toBeVisible();
});
});
electron/src/renderer/src/components/app-shell/system-notifications.tsx
+27 -19
View File
@@ -83,9 +83,11 @@ function LevelIcon({ level }: { level: SystemNotification['level'] }) {
export function SystemNotifications({
enabled,
compact = false,
titlebar = false,
}: {
enabled: boolean;
compact?: boolean;
titlebar?: boolean;
}) {
const { t } = useTranslation();
const navigate = useNavigate();
@@ -120,24 +122,20 @@ export function SystemNotifications({
return {
id: `desktop-update-${update.availableVersion}`,
level: 'info',
title: t(
update.status === 'downloaded' ? 'update.ready' : 'update.available',
{ version: update.availableVersion },
),
title: t(update.status === 'downloaded' ? 'update.ready' : 'update.available', {
version: update.availableVersion,
}),
message: t('update.safety'),
action: { type: 'navigate', target: '/settings/updates', label: t('common.open') },
persistent: true,
};
}, [t, update]);
const visible = useMemo(
() => {
const backend = (query.data?.notifications ?? []).filter(
(note) => note.level === 'error' || !dismissed.includes(note.id),
);
return updateNotification ? [updateNotification, ...backend] : backend;
},
[dismissed, query.data?.notifications, updateNotification],
);
const visible = useMemo(() => {
const backend = (query.data?.notifications ?? []).filter(
(note) => note.level === 'error' || !dismissed.includes(note.id),
);
return updateNotification ? [updateNotification, ...backend] : backend;
}, [dismissed, query.data?.notifications, updateNotification]);
const dismiss = (id: string) => {
const next = [...dismissed.filter((item) => item !== id), id].slice(-50);
@@ -178,7 +176,7 @@ export function SystemNotifications({
.map((note) => note.title || note.message)
.filter(Boolean)
.join('. ') ||
t(query.isError ? 'common.error' : query.isPending ? 'preferences.loading' : 'logs.all_clear');
t(!enabled ? 'modelSettings.unavailable' : query.isError ? 'common.error' : query.isPending ? 'preferences.loading' : 'logs.all_clear');
return (
<Popover>
<PopoverTrigger
@@ -186,9 +184,11 @@ export function SystemNotifications({
<Button
variant="ghost"
size="icon-xs"
className="relative shrink-0 text-muted-foreground hover:text-foreground"
className={cn(
'relative shrink-0 text-muted-foreground hover:text-foreground',
titlebar && 'app-no-drag',
)}
aria-label={triggerLabel}
disabled={!enabled && !updateNotification}
/>
}
>
@@ -206,11 +206,19 @@ export function SystemNotifications({
)}
</PopoverTrigger>
<PopoverContent
side={compact ? 'right' : 'top'}
align="start"
side={titlebar ? 'bottom' : compact ? 'right' : 'top'}
align={titlebar ? 'end' : 'start'}
className="max-h-[min(28rem,calc(100vh-2rem))] w-[min(22rem,calc(100vw-2rem))] space-y-1 overflow-y-auto p-1.5"
>
{!query.isPending && !query.isError && visible.length === 0 && (
{!enabled && visible.length === 0 && (
<p className="px-3 py-4 text-center text-xs text-muted-foreground">{t('modelSettings.unavailable')}</p>
)}
{enabled && query.isPending && visible.length === 0 && (
<p className="px-3 py-4 text-center text-xs text-muted-foreground">
{t('preferences.loading')}
</p>
)}
{enabled && !query.isPending && !query.isError && visible.length === 0 && (
<p className="px-3 py-4 text-center text-xs text-muted-foreground">
{t('logs.all_clear')}
</p>
electron/src/renderer/src/components/app-shell/workspace-sidebar.tsx
+124 -37
View File
@@ -1,31 +1,84 @@
import { Link } from '@tanstack/react-router';
import { Link, useRouterState } from '@tanstack/react-router';
import { useTranslation } from 'react-i18next';
import { PanelLeftIcon, PanelLeftOpenIcon, SettingsIcon } from 'lucide-react';
import { brandIcon, brandArtwork } from '@/lib/brand';
import { getBridge, isMac } from '@/components/bridge';
import { isMac } from '@/components/bridge';
import { cn } from '@/lib/utils';
import { Button, buttonVariants } from '@/components/ui/button';
import { usePaneResize } from '@/hooks/use-pane-resize';
import { useWorkspace } from '@/lib/store/workspace';
import { setWorkspace, useWorkspace } from '@/lib/store/workspace';
import { VoicesSidebar } from '@/features/clone/voices-sidebar';
import { WorkspaceNavigation } from './workspace-menu';
import { StatusBar } from './status-bar';
import { SystemNotifications } from './system-notifications';
import { useBackendStatus } from '@/hooks/use-backend-status';
import { useWorkspaceSidebarState } from './use-workspace-sidebar';
import { useState, useSyncExternalStore } from 'react';
const SECONDARY_ROUTES = new Set([
'/stories',
'/audiobook',
'/tools',
'/batch',
'/gallery',
'/personas',
'/projects',
'/dub',
'/design',
'/transcriptions',
]);
// A local-controls pane needs enough room for the actual workspace. At the
// default desktop window, preserve navigation as a rail and restore the full
// voice library automatically once both it and a local-controls pane leave a
// useful editing canvas. Browser zoom and Windows display scaling are included
// in the CSS viewport width, so this threshold also covers high-DPI layouts.
const COMPACT_QUERY = '(max-width: 1680px)';
function routeHasSecondarySidebar(pathname: string): boolean {
const normalized = pathname.replace(/\/+$/, '') || '/';
return [...SECONDARY_ROUTES].some(
(route) => normalized === route || normalized.startsWith(`${route}/`),
);
}
function routeOwnsVoiceLibrary(pathname: string): boolean {
const normalized = pathname.replace(/\/+$/, '') || '/';
return normalized === '/personas' || normalized.startsWith('/personas/');
}
function useCompactViewport(): boolean {
return useSyncExternalStore(
(notify) => {
const query = window.matchMedia(COMPACT_QUERY);
query.addEventListener('change', notify);
return () => query.removeEventListener('change', notify);
},
() => window.matchMedia(COMPACT_QUERY).matches,
() => false,
);
}
export function WorkspaceSidebar() {
const backend = useBackendStatus();
const showCompactBrand = ['win32', 'linux'].includes(getBridge()?.app.platform ?? '');
const { t } = useTranslation();
const mac = isMac();
const { libraryOpen, libraryTab } = useWorkspace();
const { compact, compactViewport, forceExpanded, secondaryWorkspace, setOpen } =
useWorkspaceSidebarState();
const pathname = useRouterState({ select: (state) => state.location.pathname });
const compactViewport = useCompactViewport();
const compactContext = `${pathname}:${compactViewport}`;
const [expandedContext, setExpandedContext] = useState<string | null>(null);
const forceExpanded = expandedContext === compactContext;
const ownsVoiceLibrary = routeOwnsVoiceLibrary(pathname);
const compact =
!libraryOpen ||
((ownsVoiceLibrary || (compactViewport && routeHasSecondarySidebar(pathname))) &&
!forceExpanded);
const secondaryWorkspace = routeHasSecondarySidebar(pathname);
const setLibraryOpen = (libraryOpen: boolean) => setWorkspace({ libraryOpen });
const sidebarResize = usePaneResize({
storageKey: 'voicestudio.library-width',
side: 'left',
minimum: 220,
initial: 256,
minimum: mac ? 288 : 220,
initial: mac ? 288 : 256,
maximum: 360,
reserve: compactViewport && secondaryWorkspace && forceExpanded ? 520 : 640,
enabled: libraryOpen,
@@ -36,43 +89,61 @@ export function WorkspaceSidebar() {
<aside
aria-label={t('clone.saved_profiles')}
data-slot="compact-main-sidebar"
className="brand-sidebar relative isolate grid h-dvh min-h-0 w-12 shrink-0 grid-rows-[auto_minmax(0,1fr)_auto_auto] overflow-hidden border-r border-border/50 bg-sidebar"
className={cn(
'brand-sidebar relative isolate grid h-dvh min-h-0 shrink-0 grid-rows-[auto_minmax(0,1fr)_auto_auto] overflow-hidden bg-sidebar',
mac ? 'w-16' : 'w-12 border-r border-border/50',
)}
>
{showCompactBrand ? (
<div className="workspace-titlebar flex w-full shrink-0 items-center justify-center">
<img src={brandIcon} alt={t('app.name')} className="size-6 shrink-0" />
</div>
) : (
{mac && (
<span
aria-hidden="true"
data-slot="compact-sidebar-divider"
className="pointer-events-none absolute top-[72px] right-0 bottom-0 w-px bg-border/50"
/>
)}
<div
className={cn(
'workspace-titlebar flex shrink-0 justify-center',
mac ? 'min-h-[72px] items-end pb-1' : 'h-12 items-center',
)}
>
{!mac ? <img src={brandIcon} alt={t('app.name')} className="size-6 shrink-0" /> : (
<Button
variant="ghost"
size="icon-sm"
aria-label={t('clone.toggle_sidebar')}
aria-expanded={false}
onClick={() => {
setOpen(true);
setExpandedContext(compactContext);
setLibraryOpen(true);
}}
className={cn(
'workspace-titlebar h-auto w-full shrink-0 rounded-none outline-none focus-visible:ring-2 focus-visible:ring-ring',
isMac() && 'pt-5',
)}
className="shrink-0 outline-none focus-visible:ring-2 focus-visible:ring-ring"
>
<PanelLeftOpenIcon className="size-5" aria-hidden="true" />
</Button>
)}
<WorkspaceNavigation compact />
<div className="flex min-w-0 flex-col items-center">
<StatusBar compact />
<SystemNotifications enabled={backend.stage === 'ready'} compact />
)}
</div>
<div className="flex h-[var(--workspace-footer-height)] shrink-0 items-center justify-center border-t border-border/50">
<WorkspaceNavigation compact />
{!mac && <StatusBar compact />}
<div
className={cn(
'shrink-0 border-t border-border/50 py-2',
mac ? 'grid grid-cols-2 items-center gap-1 px-1' : 'flex flex-col items-center gap-1',
)}
>
<Link
to="/settings"
aria-label={t('nav.settings')}
title={t('nav.settings')}
className={buttonVariants({ variant: 'ghost', size: 'icon-sm' })}
className={buttonVariants({
variant: 'ghost',
size: mac ? 'icon-xs' : 'icon-sm',
})}
>
<SettingsIcon />
</Link>
{mac && <StatusBar compact inline />}
{!mac && <SystemNotifications enabled={backend.stage === 'ready'} compact />}
</div>
</aside>
)}
@@ -92,7 +163,7 @@ export function WorkspaceSidebar() {
<header
className={cn(
'workspace-titlebar flex shrink-0 items-center gap-2 px-4',
isMac() && 'pl-20',
mac && 'pl-24',
)}
>
<Link
@@ -108,7 +179,8 @@ export function WorkspaceSidebar() {
size="icon-sm"
aria-label={t('common.close')}
onClick={() => {
setOpen(false);
setExpandedContext(null);
setLibraryOpen(false);
}}
>
<PanelLeftIcon />
@@ -122,14 +194,29 @@ export function WorkspaceSidebar() {
<VoicesSidebar key={libraryTab} initialTab={libraryTab} />
<div className="flex min-w-0 shrink-0 flex-col border-t border-border/50">
<WorkspaceNavigation />
<StatusBar />
<div className="flex h-[var(--workspace-footer-height)] items-center justify-between gap-2 border-t border-border/50 px-3">
<Link to="/settings" className={buttonVariants({ variant: 'ghost', size: 'sm' })}>
<SettingsIcon />
{t('nav.settings')}
</Link>
<SystemNotifications enabled={backend.stage === 'ready'} />
</div>
<StatusBar
footerLeading={
mac ? (
<Link
to="/settings"
aria-label={t('nav.settings')}
title={t('nav.settings')}
className={buttonVariants({ variant: 'ghost', size: 'icon-xs' })}
>
<SettingsIcon />
</Link>
) : undefined
}
/>
{!mac && (
<div className="flex items-center justify-between gap-2 border-t border-border/50 px-3 py-2">
<Link to="/settings" className={buttonVariants({ variant: 'ghost', size: 'sm' })}>
<SettingsIcon />
{t('nav.settings')}
</Link>
<SystemNotifications enabled={backend.stage === 'ready'} />
</div>
)}
</div>
</aside>
)}
electron/src/renderer/src/components/video-player.tsx
+91 -75
View File
@@ -37,6 +37,7 @@ export const VideoPlayer = memo(function VideoPlayer({
onPause,
onSeeked,
onCanPlay,
controls = 'full',
}: {
src: MediaPlayerProps['src'];
source?: string;
@@ -48,6 +49,7 @@ export const VideoPlayer = memo(function VideoPlayer({
onPause?: MediaPlayerProps['onPause'];
onSeeked?: MediaPlayerProps['onSeeked'];
onCanPlay?: MediaPlayerProps['onCanPlay'];
controls?: 'full' | 'compact';
}) {
const localPlayerRef = useRef<MediaPlayerInstance>(null);
const player = externalPlayerRef ?? localPlayerRef;
@@ -76,12 +78,14 @@ export const VideoPlayer = memo(function VideoPlayer({
loaders={videoLoaders}
className="relative aspect-video [&_[data-remotion-canvas]]:h-full [&_[data-remotion-canvas]]:w-full [&_[data-remotion-container]]:h-full [&_[data-remotion-container]]:w-full [&_video]:h-full [&_video]:w-full [&_iframe]:h-full [&_iframe]:w-full"
>
<Poster
alt=""
className="absolute inset-0 h-full w-full object-contain opacity-0 data-[visible]:opacity-100 data-[hidden]:hidden"
/>
<Poster alt="" className="absolute inset-0 h-full w-full object-contain opacity-0 data-[visible]:opacity-100 data-[hidden]:hidden" />
</MediaProvider>
<VideoControls player={player} source={source} sourceIdentity={sourceIdentity} />
<VideoControls
player={player}
source={source}
sourceIdentity={sourceIdentity}
compact={controls === 'compact'}
/>
</StudioMediaPlayer>
);
});
@@ -89,10 +93,12 @@ function VideoControls({
player,
source,
sourceIdentity,
compact,
}: {
player: React.RefObject<MediaPlayerInstance | null>;
source: string;
sourceIdentity: string;
compact: boolean;
}) {
const { t } = useTranslation();
const remote = useMediaRemote(player);
@@ -191,32 +197,36 @@ function VideoControls({
<PauseIcon />
)}
</Button>
<Button
size="icon-sm"
variant="ghost"
className="hover:bg-white/15 hover:text-white"
aria-label={`${t('player.seek')} -10s`}
onClick={() => {
if (player.current) player.current.currentTime = Math.max(0, time - 10);
}}
>
<RotateCcwIcon />
</Button>
<Button
size="icon-sm"
variant="ghost"
className="hover:bg-white/15 hover:text-white"
aria-label={`${t('player.seek')} +10s`}
onClick={() => {
if (player.current)
player.current.currentTime = Math.min(
Number.isFinite(duration) ? duration : time + 10,
time + 10,
);
}}
>
<RotateCwIcon />
</Button>
{!compact && (
<>
<Button
size="icon-sm"
variant="ghost"
className="hover:bg-white/15 hover:text-white"
aria-label={`${t('player.seek')} -10s`}
onClick={() => {
if (player.current) player.current.currentTime = Math.max(0, time - 10);
}}
>
<RotateCcwIcon />
</Button>
<Button
size="icon-sm"
variant="ghost"
className="hover:bg-white/15 hover:text-white"
aria-label={`${t('player.seek')} +10s`}
onClick={() => {
if (player.current)
player.current.currentTime = Math.min(
Number.isFinite(duration) ? duration : time + 10,
time + 10,
);
}}
>
<RotateCwIcon />
</Button>
</>
)}
<Button
size="icon-sm"
variant="ghost"
@@ -229,53 +239,59 @@ function VideoControls({
>
{muted ? <VolumeXIcon /> : <Volume2Icon />}
</Button>
<input
type="range"
aria-label={t('player.volume')}
min={0}
max={1}
step="0.05"
value={muted ? 0 : volume}
className="hidden h-1 w-14 shrink-0 cursor-pointer appearance-none rounded-full bg-white/25 accent-primary [&::-webkit-slider-thumb]:size-3 [&::-webkit-slider-thumb]:appearance-none [&::-webkit-slider-thumb]:rounded-full [&::-webkit-slider-thumb]:bg-white @min-[420px]/player:block"
onInput={(event) => {
if (player.current) {
player.current.muted = false;
player.current.volume = Number(event.currentTarget.value);
}
}}
/>
<Button
size="xs"
variant="ghost"
className="min-w-10 px-1.5 text-[10px] tabular-nums hover:bg-white/15 hover:text-white"
aria-label={t('clone.speed')}
onClick={() => {
const rates = [0.75, 1, 1.25, 1.5, 2];
const next = rates[(rates.indexOf(playbackRate) + 1) % rates.length];
setPlaybackRate(next);
if (player.current) player.current.playbackRate = next;
}}
>
{playbackRate}×
</Button>
{!compact && (
<>
<input
type="range"
aria-label={t('player.volume')}
min={0}
max={1}
step="0.05"
value={muted ? 0 : volume}
className="hidden h-1 w-14 shrink-0 cursor-pointer appearance-none rounded-full bg-white/25 accent-primary [&::-webkit-slider-thumb]:size-3 [&::-webkit-slider-thumb]:appearance-none [&::-webkit-slider-thumb]:rounded-full [&::-webkit-slider-thumb]:bg-white sm:block"
onInput={(event) => {
if (player.current) {
player.current.muted = false;
player.current.volume = Number(event.currentTarget.value);
}
}}
/>
<Button
size="xs"
variant="ghost"
className="min-w-10 px-1.5 text-[10px] tabular-nums hover:bg-white/15 hover:text-white"
aria-label={t('clone.speed')}
onClick={() => {
const rates = [0.75, 1, 1.25, 1.5, 2];
const next = rates[(rates.indexOf(playbackRate) + 1) % rates.length];
setPlaybackRate(next);
if (player.current) player.current.playbackRate = next;
}}
>
{playbackRate}×
</Button>
</>
)}
<span className="ml-auto whitespace-nowrap text-[10px] tabular-nums">
{formatClock(time)} / {formatClock(duration)}
</span>
<Button
size="icon-sm"
variant="ghost"
disabled={!canFullscreen}
aria-label={t(fullscreen ? 'player.exit_fullscreen' : 'player.fullscreen')}
className="hover:bg-white/15 hover:text-white"
onClick={() => {
const action = fullscreen
? player.current?.exitFullscreen()
: player.current?.enterFullscreen();
void action?.catch(() => setFailed(true));
}}
>
{fullscreen ? <MinimizeIcon /> : <MaximizeIcon />}
</Button>
{!compact && (
<Button
size="icon-sm"
variant="ghost"
disabled={!canFullscreen}
aria-label={t(fullscreen ? 'player.exit_fullscreen' : 'player.fullscreen')}
className="hover:bg-white/15 hover:text-white"
onClick={() => {
const action = fullscreen
? player.current?.exitFullscreen()
: player.current?.enterFullscreen();
void action?.catch(() => setFailed(true));
}}
>
{fullscreen ? <MinimizeIcon /> : <MaximizeIcon />}
</Button>
)}
</div>
</div>
);
electron/src/renderer/src/features/dub/dub-page.tsx
+18 -4
View File
@@ -1677,8 +1677,8 @@ export function DubPage() {
/>
)}
{!session.segments.length && (!session.recovery || busy) && (
<div className="mx-auto flex min-h-[28rem] w-full max-w-5xl flex-col items-center justify-center px-6 text-center">
<div className="mb-5 flex size-14 items-center justify-center rounded-2xl bg-primary/10 text-primary">
<div className="mx-auto flex min-h-[24rem] w-full max-w-5xl flex-col items-center justify-center px-6 text-center">
<div className="mb-3 flex size-12 items-center justify-center rounded-2xl bg-primary/10 text-primary">
{busy ? (
<LoaderCircleIcon className="size-6 animate-spin motion-reduce:animate-none" />
) : (
@@ -1696,9 +1696,23 @@ export function DubPage() {
{t('dub.supported_formats')}
</p>
{!session.jobId && !demoDismissed && (
<div className="mt-6 w-full">
<div className="mt-4 w-full">
<DubbingDemo
onTry={() => input.current?.click()}
onEdit={async ({ path, filename }) => {
try {
const response = await apiFetch(path);
const blob = await response.blob();
setPreview('original');
await uploadDub(
new File([blob], filename, {
type: blob.type || 'video/mp4',
}),
);
} catch (error) {
toast.error(describeError(error));
}
}}
onDismiss={() => {
setDemoDismissed(true);
try {
@@ -1710,7 +1724,7 @@ export function DubPage() {
/>
</div>
)}
<ol className="mt-6 grid w-full grid-cols-3 gap-2 text-left max-sm:grid-cols-1">
<ol className="mt-4 grid w-full grid-cols-3 gap-2 text-left max-sm:grid-cols-1">
{[
['1', 'dub.upload_transcribe'],
['2', 'dub.translate'],
electron/src/renderer/src/features/dub/dubbing-demo.test.tsx
+181
View File
@@ -0,0 +1,181 @@
import { useLayoutEffect } from 'react';
import { cleanup, fireEvent, render, screen } from '@testing-library/react';
import { afterEach, expect, it, vi } from 'vitest';
const mocks = vi.hoisted(() => ({
api: vi.fn(),
videoProps: [] as Array<{ controls?: string; source: string }>,
players: new Map<
string,
{
currentTime: number;
paused: boolean;
play: ReturnType<typeof vi.fn>;
pause: ReturnType<typeof vi.fn>;
}
>(),
}));
vi.mock('react-i18next', () => ({ useTranslation: () => ({ t: (key: string) => key }) }));
vi.mock('@/lib/api/client', () => ({ apiJson: mocks.api, apiPath: (path: string) => path }));
vi.mock('@/components/video-player', () => ({
VideoPlayer: ({ controls, playerRef, onPlay, onPause, onCanPlay, source, src }: Record<string, any>) => {
mocks.videoProps.push({ controls, source });
let player = mocks.players.get(source);
if (!player) {
player = {
currentTime: 0,
paused: true,
play: vi.fn(async () => {
player!.paused = false;
}),
pause: vi.fn(async () => {
player!.paused = true;
}),
};
mocks.players.set(source, player);
}
playerRef.current = player;
useLayoutEffect(() => {
player!.currentTime = 0;
onCanPlay?.({});
}, [src.src]);
return (
<button
type="button"
aria-label={source}
onDoubleClick={() => { player!.paused = true; onPause?.({}); }}
onClick={() => {
player!.paused = false;
onPlay?.({});
}}
/>
);
},
}));
import { DubbingDemo } from './dubbing-demo';
const manifest = {
source: {
code: 'en',
label: 'English',
video: 'source.mp4',
script: 'Original script',
},
dubbed: [
{
code: 'es',
label: 'Español',
video: 'dubbed_es.mp4',
script: 'Spanish script',
dir: 'ltr' as const,
},
{
code: 'fr',
label: 'Français',
video: 'dubbed_fr.mp4',
script: 'French script',
dir: 'ltr' as const,
},
],
};
afterEach(() => {
cleanup();
vi.clearAllMocks();
mocks.players.clear();
mocks.videoProps.length = 0;
});
function mount(onEdit = vi.fn()) {
mocks.api.mockResolvedValue(manifest);
render(<DubbingDemo onDismiss={vi.fn()} onTry={vi.fn()} onEdit={onEdit} />);
}
it('keeps A/B playback exclusive while synchronizing the peer playhead', async () => {
mount();
const sourceButton = await screen.findByRole('button', {
name: 'dubbing-demo-comparison-demo.original_tag',
});
const source = mocks.players.get('dubbing-demo-comparison-demo.original_tag')!;
const dubbed = mocks.players.get('dubbing-demo-comparison-demo.dubbed_tag')!;
expect(mocks.videoProps.slice(0, 2).map(({ controls }) => controls)).toEqual([
'compact',
'compact',
]);
source.currentTime = 4.25;
fireEvent.click(sourceButton);
expect(dubbed.currentTime).toBe(4.25);
expect(dubbed.play).not.toHaveBeenCalled();
});
it('keeps editable transcript drafts for each sample language', async () => {
mount();
const transcripts = await screen.findAllByRole('textbox');
expect(transcripts).toHaveLength(2);
fireEvent.change(transcripts[1]!, { target: { value: 'Edited Spanish script' } });
fireEvent.click(screen.getByRole('button', { name: 'Français' }));
expect(screen.getAllByRole('textbox')[1]).toHaveValue('French script');
fireEvent.change(screen.getAllByRole('textbox')[1]!, {
target: { value: 'Edited French script' },
});
fireEvent.click(screen.getByRole('button', { name: 'Español' }));
expect(screen.getAllByRole('textbox')[1]).toHaveValue('Edited Spanish script');
});
it('opens the selected dubbed sample in the editor', async () => {
const onEdit = vi.fn();
mount(onEdit);
fireEvent.click(await screen.findByRole('button', { name: 'clone.edit Español' }));
expect(onEdit).toHaveBeenCalledWith({
path: '/demo_audio/demo/dubbing/dubbed_es.mp4',
filename: 'dubbed_es.mp4',
});
});
it('continues from the outgoing playhead when switching samples', async () => {
mount();
const original = await screen.findByRole('button', {name: 'dubbing-demo-comparison-demo.original_tag'});
const translated = screen.getByRole('button', {name: 'dubbing-demo-comparison-demo.dubbed_tag'});
fireEvent.click(original);
const source = mocks.players.get('dubbing-demo-comparison-demo.original_tag')!;
const dubbed = mocks.players.get('dubbing-demo-comparison-demo.dubbed_tag')!;
source.currentTime = 7.25;
fireEvent.click(translated);
expect(dubbed.currentTime).toBe(7.25);
expect(source.pause).toHaveBeenCalled();
});
it('copies the final paused position to the other sample', async () => {
mount();
const original = await screen.findByRole('button', {name: 'dubbing-demo-comparison-demo.original_tag'});
fireEvent.click(original);
await Promise.resolve();
const source = mocks.players.get('dubbing-demo-comparison-demo.original_tag')!;
source.currentTime = 5;
fireEvent.doubleClick(original);
expect(mocks.players.get('dubbing-demo-comparison-demo.dubbed_tag')!.currentTime).toBe(5);
});
it('preserves the active dubbed position when its language source resets', async () => {
mount();
const translated = await screen.findByRole('button', { name: 'dubbing-demo-comparison-demo.dubbed_tag' });
fireEvent.click(translated);
await Promise.resolve();
const dubbed = mocks.players.get('dubbing-demo-comparison-demo.dubbed_tag')!;
const original = mocks.players.get('dubbing-demo-comparison-demo.original_tag')!;
dubbed.currentTime = 9.5;
fireEvent.click(screen.getByRole('button', { name: 'Français' }));
expect(dubbed.currentTime).toBe(9.5);
fireEvent.click(translated);
expect(original.currentTime).toBe(9.5);
});
electron/src/renderer/src/features/dub/dubbing-demo.tsx
+196 -61
View File
@@ -1,11 +1,20 @@
import { useEffect, useRef, useState } from 'react';
import { useTranslation } from 'react-i18next';
import { FilmIcon, LoaderCircleIcon, PlayIcon, XIcon } from 'lucide-react';
import {
FilmIcon,
LoaderCircleIcon,
PencilIcon,
PlayIcon,
RotateCcwIcon,
XIcon,
} from 'lucide-react';
import type { MediaPlayerInstance } from '@/components/media-player';
import { VideoPlayer } from '@/components/video-player';
import { Button } from '@/components/ui/button';
import { Switch } from '@/components/ui/switch';
import { Textarea } from '@/components/ui/textarea';
import { apiJson, apiPath } from '@/lib/api/client';
import { runRendererTask } from '@/lib/global-error-recovery';
interface DemoManifest {
source: {
@@ -26,15 +35,32 @@ interface DemoManifest {
const DEMO_BASE = '/demo_audio/demo/dubbing';
const PLAYBACK_GROUP = 'dubbing-demo-comparison';
export function DubbingDemo({ onDismiss, onTry }: { onDismiss: () => void; onTry: () => void }) {
interface EditableDemoVideo {
path: string;
filename: string;
}
export function DubbingDemo({
onDismiss,
onTry,
onEdit,
}: {
onDismiss: () => void;
onTry: () => void;
onEdit: (sample: EditableDemoVideo) => void | Promise<void>;
}) {
const { t } = useTranslation();
const [manifest, setManifest] = useState<DemoManifest | null>(null);
const [failed, setFailed] = useState(false);
const [language, setLanguage] = useState('es');
const [synchronized, setSynchronized] = useState(true);
const [scripts, setScripts] = useState<Record<string, string>>({});
const [editingVideo, setEditingVideo] = useState(false);
const sourcePlayer = useRef<MediaPlayerInstance>(null);
const dubbedPlayer = useRef<MediaPlayerInstance>(null);
const mirroring = useRef(false);
const activePlayer = useRef<MediaPlayerInstance | null>(null);
const pendingDubbedPosition = useRef<number | null>(null);
useEffect(() => {
const controller = new AbortController();
@@ -44,6 +70,11 @@ export function DubbingDemo({ onDismiss, onTry }: { onDismiss: () => void; onTry
return () => controller.abort();
}, []);
useEffect(() => {
void sourcePlayer.current?.pause().catch(() => {});
void dubbedPlayer.current?.pause().catch(() => {});
}, [language]);
if (failed) return null;
if (!manifest)
return (
@@ -56,17 +87,14 @@ export function DubbingDemo({ onDismiss, onTry }: { onDismiss: () => void; onTry
const dubbed = manifest.dubbed.find((item) => item.code === language) || manifest.dubbed[0];
if (!dubbed) return null;
const mirror = (
const synchronizePosition = (
from: MediaPlayerInstance | null,
to: MediaPlayerInstance | null,
action: 'play' | 'pause' | 'seek',
) => {
if (!synchronized || mirroring.current || !from || !to) return;
if (!synchronized || pendingDubbedPosition.current !== null || mirroring.current || !from || !to) return;
mirroring.current = true;
try {
to.currentTime = from.currentTime;
if (action === 'play' && to.paused) void to.play().catch(() => {});
if (action === 'pause' && !to.paused) void to.pause();
} finally {
queueMicrotask(() => {
mirroring.current = false;
@@ -75,6 +103,8 @@ export function DubbingDemo({ onDismiss, onTry }: { onDismiss: () => void; onTry
};
const card = (
channel: 'A' | 'B',
code: string,
label: string,
tag: string,
video: string,
@@ -82,34 +112,114 @@ export function DubbingDemo({ onDismiss, onTry }: { onDismiss: () => void; onTry
player: React.RefObject<MediaPlayerInstance | null>,
peer: React.RefObject<MediaPlayerInstance | null>,
direction?: 'ltr' | 'rtl',
) => (
<article className="min-w-0 space-y-2">
<div className="flex items-center gap-2 text-xs font-medium">
<span>{label}</span>
<span className="text-[10px] uppercase tracking-wide text-muted-foreground">{tag}</span>
</div>
<VideoPlayer
playerRef={player}
playbackGroup={PLAYBACK_GROUP}
load="eager"
src={{ src: apiPath(`${DEMO_BASE}/${video}`), type: 'video/mp4' }}
source={`${PLAYBACK_GROUP}-${tag}`}
onPlay={() => mirror(player.current, peer.current, 'play')}
onPause={() => mirror(player.current, peer.current, 'pause')}
onSeeked={() => mirror(player.current, peer.current, 'seek')}
/>
<p
dir={direction}
className="line-clamp-3 rounded-lg bg-background/35 p-2 text-left text-xs leading-5 text-muted-foreground"
>
{script}
</p>
</article>
);
editableVideo?: EditableDemoVideo,
) => {
const value = scripts[code] ?? script;
const edited = value !== script;
return (
<article className="min-w-0 overflow-hidden rounded-2xl border border-border/55 bg-background/25 shadow-[inset_0_1px_0_color-mix(in_oklab,var(--foreground)_4%,transparent)]">
<div className="flex items-center gap-2.5 px-3 py-2.5 text-xs font-medium">
<span className="grid size-6 shrink-0 place-items-center rounded-md bg-primary/12 font-mono text-[10px] font-semibold text-primary">
{channel}
</span>
<span>{label}</span>
<span className="text-[10px] uppercase tracking-[0.08em] text-muted-foreground">
{tag}
</span>
<div className="ml-auto flex items-center gap-1">
{edited && (
<Button
size="icon-xs"
variant="ghost"
className="text-muted-foreground"
aria-label={`${t('preferences.reset')} ${label}`}
title={t('preferences.reset')}
onClick={() =>
setScripts((current) => {
const next = { ...current };
delete next[code];
return next;
})
}
>
<RotateCcwIcon />
</Button>
)}
{editableVideo && (
<Button
size="xs"
variant="ghost"
disabled={editingVideo}
aria-label={`${t('clone.edit')} ${label}`}
onClick={() => {
setEditingVideo(true);
runRendererTask('Edit dubbing demo', async () => {
try { await onEdit(editableVideo); }
finally { setEditingVideo(false); }
});
}}
>
{editingVideo ? (
<LoaderCircleIcon className="animate-spin motion-reduce:animate-none" />
) : (
<PencilIcon />
)}
{t('clone.edit')}
</Button>
)}
</div>
</div>
<div className="px-2.5">
<VideoPlayer
playerRef={player}
controls="compact"
load="eager"
src={{ src: apiPath(`${DEMO_BASE}/${video}`), type: 'video/mp4' }}
source={`${PLAYBACK_GROUP}-${tag}`}
onCanPlay={() => {
if (channel !== 'B' || pendingDubbedPosition.current === null || !player.current) return;
player.current.currentTime = pendingDubbedPosition.current;
pendingDubbedPosition.current = null;
if (activePlayer.current === player.current) synchronizePosition(player.current, peer.current);
}}
onPlay={() => {
const incoming = player.current;
const outgoing = activePlayer.current;
if (synchronized && incoming && outgoing && incoming !== outgoing) {
incoming.currentTime = pendingDubbedPosition.current ?? outgoing.currentTime;
}
activePlayer.current = incoming;
void peer.current?.pause().catch(() => {});
synchronizePosition(incoming, peer.current);
}}
onPause={() => {
if (activePlayer.current === player.current) synchronizePosition(player.current, peer.current);
}}
onSeeked={() => synchronizePosition(player.current, peer.current)}
/>
</div>
<label className="sr-only" htmlFor={`dubbing-demo-script-${code}`}>
{label} — {t('dub.transcript')}
</label>
<Textarea
id={`dubbing-demo-script-${code}`}
dir={direction}
value={value}
rows={3}
spellCheck
onChange={(event) => {
const nextValue = event.currentTarget.value;
setScripts((current) => ({ ...current, [code]: nextValue }));
}}
className="m-2.5 mt-3 h-24 min-h-20 max-h-40 w-[calc(100%-1.25rem)] resize-y rounded-xl border-border/45 bg-background/45 text-xs leading-5 text-muted-foreground [field-sizing:fixed] focus-visible:text-foreground"
/>
</article>
);
};
return (
<section className="glass-panel w-full space-y-4 rounded-2xl border border-border/60 bg-card/35 p-4 text-left shadow-sm">
<header className="flex flex-wrap items-center gap-3">
<section className="glass-panel @container/dubbing-demo w-full space-y-3 rounded-2xl border border-border/60 bg-card/35 p-4 text-left shadow-sm">
<header className="grid grid-cols-[auto_minmax(0,1fr)_auto] items-center gap-3">
<span className="flex size-8 items-center justify-center rounded-lg bg-primary/10 text-primary">
<FilmIcon className="size-4" />
</span>
@@ -117,21 +227,57 @@ export function DubbingDemo({ onDismiss, onTry }: { onDismiss: () => void; onTry
<h3 className="text-sm font-medium">{t('demo.dubbing_title')}</h3>
<p className="text-xs text-muted-foreground">{t('demo.dubbing_picker')}</p>
</div>
<label className="inline-flex items-center gap-2 text-xs text-muted-foreground">
<div className="flex items-center gap-1.5">
<Button
size="sm"
aria-label={t('demo.dubbing_cta')}
title={t('demo.dubbing_cta')}
onClick={onTry}
>
<PlayIcon />
<span className="hidden @min-[560px]:inline">{t('demo.dubbing_cta')}</span>
</Button>
<Button
size="icon-xs"
variant="ghost"
aria-label={t('demo.dubbing_dismiss')}
onClick={onDismiss}
>
<XIcon />
</Button>
</div>
</header>
<div
role="group"
aria-label={t('demo.dubbing_picker')}
className="flex flex-wrap items-center gap-1.5 rounded-xl border border-border/45 bg-background/30 p-1.5"
>
{manifest.dubbed.map((item) => (
<Button
key={item.code}
size="xs"
variant={item.code === dubbed.code ? 'secondary' : 'ghost'}
aria-pressed={item.code === dubbed.code}
onClick={() => {
if (item.code === language) return;
pendingDubbedPosition.current = pendingDubbedPosition.current ??
(synchronized ? activePlayer.current?.currentTime : undefined) ??
dubbedPlayer.current?.currentTime ?? 0;
setLanguage(item.code);
}}
>
{item.label}
</Button>
))}
<label className="ml-auto inline-flex items-center gap-2 px-1.5 text-xs text-muted-foreground">
{t('demo.dubbing_sync')}
<Switch checked={synchronized} onCheckedChange={setSynchronized} />
</label>
<Button
size="icon-xs"
variant="ghost"
aria-label={t('demo.dubbing_dismiss')}
onClick={onDismiss}
>
<XIcon />
</Button>
</header>
<div className="grid gap-4 min-[1100px]:grid-cols-2">
</div>
<div className="grid gap-3 min-[1100px]:grid-cols-2">
{card(
'A',
manifest.source.code,
manifest.source.label,
t('demo.original_tag'),
manifest.source.video,
@@ -140,6 +286,8 @@ export function DubbingDemo({ onDismiss, onTry }: { onDismiss: () => void; onTry
dubbedPlayer,
)}
{card(
'B',
dubbed.code,
dubbed.label,
t('demo.dubbed_tag'),
dubbed.video,
@@ -147,25 +295,12 @@ export function DubbingDemo({ onDismiss, onTry }: { onDismiss: () => void; onTry
dubbedPlayer,
sourcePlayer,
dubbed.dir,
{
path: `${DEMO_BASE}/${dubbed.video}`,
filename: dubbed.video,
},
)}
</div>
<footer className="flex flex-wrap items-center gap-1.5">
{manifest.dubbed.map((item) => (
<Button
key={item.code}
size="xs"
variant={item.code === dubbed.code ? 'secondary' : 'ghost'}
aria-pressed={item.code === dubbed.code}
onClick={() => setLanguage(item.code)}
>
{item.label}
</Button>
))}
<Button size="sm" className="ml-auto" onClick={onTry}>
<PlayIcon />
{t('demo.dubbing_cta')}
</Button>
</footer>
</section>
);
}
electron/src/renderer/src/features/integrations/integrations-page.tsx
+2 -1
View File
@@ -1,3 +1,4 @@
import { runRendererTask } from '@/lib/global-error-recovery';
import { BlocksIcon, ExternalLinkIcon, SearchIcon, SparklesIcon } from 'lucide-react';
import { useMemo, useState } from 'react';
import { useTranslation } from 'react-i18next';
@@ -142,7 +143,7 @@ export function IntegrationsPage() {
<button
type="button"
key={entry.url}
onClick={() => void navigate({ to: '/integrations/$slug', params: { slug: integrationSlug(entry.name) } })}
onClick={() => runRendererTask('Open integration', () => navigate({ to: '/integrations/$slug', params: { slug: integrationSlug(entry.name) } }))}
className="integration-card"
>
<div className="integration-card-top">
electron/src/renderer/src/features/settings/performance-settings.tsx
+5 -7
View File
@@ -469,22 +469,20 @@ function CompileSetting() {
signal,
}),
});
const windows = query.data?.platform === 'win32';
// #2135: live on every platform. This used to be gated to win32 (matching
// the Tauri UI), which left the Linux/CUDA reporter unable to switch off the
// torch.compile that was killing their backend.
return (
<SettingsSection icon={CpuIcon} title={t('settings.perf_title')}>
<SettingsRow
id="torch-compile"
title={t('settings.perf_torch_compile')}
description={
query.data
? t(windows ? 'settings.perf_torch_compile_note' : 'settings.perf_torch_compile_na')
: undefined
}
description={query.data ? t('settings.perf_torch_compile_note') : undefined}
>
<Switch
aria-label={t('settings.perf_torch_compile')}
checked={!!query.data?.enabled}
disabled={!windows || action.busy || query.isPending}
disabled={action.busy || query.isPending}
onCheckedChange={(enabled) =>
void action.run(async () => {
const state = await apiJson('/api/settings/perf/torch-compile-disabled', {
electron/src/renderer/src/features/settings/support-settings.tsx
+2 -1
View File
@@ -1,3 +1,4 @@
import { runRendererTask } from '@/lib/global-error-recovery';
import {
ArrowUpRightIcon,
CheckIcon,
@@ -230,7 +231,7 @@ export function SupportSettings() {
<p>{t('supportPlans.unavailable')}</p>
<button
type="button"
onClick={() => void navigate({ to: '/settings/support', search: { compare: true } })}
onClick={() => runRendererTask('Open support', () => navigate({ to: '/settings/support', search: { compare: true } }))}
className="mt-4 flex min-h-11 items-center gap-2 text-sm font-medium text-primary hover:underline focus-visible:outline-2 focus-visible:outline-primary"
>
{t('supportPlans.title')}
electron/src/renderer/src/hooks/use-compute-target.ts
+7
View File
@@ -18,6 +18,13 @@ export interface ComputeTarget {
latency_ms: number;
active_tasks: number;
max_tasks: number;
cpu_percent: number | null;
free_memory_bytes: number | null;
system_memory_bytes: number;
cpu_count: number;
gpu_name: string;
gpu_memory_bytes: number;
gpu_utilization_percent: number | null;
}
export interface ComputeTargetState {
electron/src/renderer/src/i18n/locales/ar.json
+2 -2
View File
@@ -1050,7 +1050,7 @@
"perf_title": "الأداء",
"generate_timeout_cpu_note": "تُستخدم عند تشغيل التوليد على المعالج المركزي (CPU). بمجرد حفظها هنا، تحكم دائمًا في توليد فئة المعالج المركزي — بمعزل عن ميزانية المسرَّع أعلاه.",
"device_family_rocm": "AMD GPU (ROCm)",
"perf_torch_compile": "تعطيل torch.compile (Windows)",
"perf_torch_compile": "تعطيل torch.compile",
"generate_timeout_shadowed_note": "يوجد متغيّر بيئة خارج VoiceStudio (الصدفة، ملف .env، أو الحاوية) يُحدّد هذه القيمة حاليًا — القيمة المحفوظة هنا تُتجاهل إلى أن تتم إزالة ذلك المتغيّر.",
"device_family_cpu": "CPU",
"device_family_gpu": "GPU",
@@ -1973,7 +1973,7 @@
"dictation_replay": "إعادة التشغيل",
"dictation_transcribing": "جارٍ النسخ…",
"dubbing_title": "شاهد الدبلجة أثناء العمل",
"dubbing_sync": "التشغيل المتزامن",
"dubbing_sync": "مزامنة موضع التشغيل",
"dubbing_picker": "جرب لغة أخرى:",
"dubbing_cta": "قم بتشغيل هذا على الفيديو الخاص بك →",
"dubbing_loading": "جارٍ تحميل العرض التوضيحي للدبلجة...",
electron/src/renderer/src/i18n/locales/de.json
+2 -2
View File
@@ -1042,7 +1042,7 @@
"perf_title": "Leistung",
"generate_timeout_cpu_note": "Wird verwendet, wenn die Generierung auf der CPU läuft. Einmal hier gespeichert, bestimmt dieser Wert immer die CPU-Generierung — unabhängig vom beschleunigten Budget oben.",
"device_family_rocm": "AMD GPU (ROCm)",
"perf_torch_compile": "torch.compile deaktivieren (Windows)",
"perf_torch_compile": "torch.compile deaktivieren",
"generate_timeout_shadowed_note": "Eine Umgebungsvariable außerhalb von VoiceStudio (Shell, .env-Datei oder Container) legt diesen Wert derzeit fest — der hier gespeicherte Wert wird ignoriert, bis diese Variable entfernt wird.",
"device_family_cpu": "CPU",
"device_family_gpu": "GPU",
@@ -1965,7 +1965,7 @@
"dictation_replay": "Wiederholung",
"dictation_transcribing": "Transkribieren…",
"dubbing_title": "Erleben Sie Synchronisation in Aktion",
"dubbing_sync": "Synchronisierte Wiedergabe",
"dubbing_sync": "Abspielposition synchronisieren",
"dubbing_picker": "Versuchen Sie es mit einer anderen Sprache:",
"dubbing_cta": "Führen Sie dies in Ihrem eigenen Video aus →",
"dubbing_loading": "Synchronisationsdemo wird geladen…",
electron/src/renderer/src/i18n/locales/en.json
+2 -2
View File
@@ -1218,7 +1218,7 @@
"perf_title": "Performance",
"generate_timeout_cpu_note": "Used when generation runs on the CPU. Once saved here, this always governs CPU-family generation — independent of the accelerated budget above.",
"device_family_rocm": "AMD GPU (ROCm)",
"perf_torch_compile": "Disable torch.compile (Windows)",
"perf_torch_compile": "Disable torch.compile",
"generate_timeout_shadowed_note": "An environment variable outside VoiceStudio (shell, .env file, or container) is currently setting this — your saved value here is ignored until that variable is removed.",
"device_family_cpu": "CPU",
"device_family_gpu": "GPU",
@@ -2014,7 +2014,7 @@
"dictation_replay": "Replay",
"dictation_transcribing": "Transcribing…",
"dubbing_title": "See dubbing in action",
"dubbing_sync": "Synced playback",
"dubbing_sync": "Sync playheads",
"dubbing_picker": "Try another language:",
"dubbing_cta": "Run this on your own video →",
"dubbing_loading": "Loading dubbing demo…",
electron/src/renderer/src/i18n/locales/es.json
+2 -2
View File
@@ -1044,7 +1044,7 @@
"perf_title": "Rendimiento",
"generate_timeout_cpu_note": "Se usa cuando la generación se ejecuta en la CPU. Una vez guardado aquí, este valor siempre rige la generación en CPU, independientemente del presupuesto acelerado de arriba.",
"device_family_rocm": "AMD GPU (ROCm)",
"perf_torch_compile": "Desactivar torch.compile (Windows)",
"perf_torch_compile": "Desactivar torch.compile",
"generate_timeout_shadowed_note": "Una variable de entorno externa a VoiceStudio (shell, archivo .env o contenedor) está estableciendo este valor actualmente — el valor guardado aquí se ignora hasta que se elimine esa variable.",
"device_family_cpu": "CPU",
"device_family_gpu": "GPU",
@@ -1967,7 +1967,7 @@
"dictation_replay": "Reproducir",
"dictation_transcribing": "Transcribiendo…",
"dubbing_title": "Ver doblaje en acción",
"dubbing_sync": "Reproducción sincronizada",
"dubbing_sync": "Sincronizar posición",
"dubbing_picker": "Prueba con otro idioma:",
"dubbing_cta": "Ejecute esto en su propio video →",
"dubbing_loading": "Cargando demostración de doblaje…",
electron/src/renderer/src/i18n/locales/fr.json
+2 -2
View File
@@ -1044,7 +1044,7 @@
"perf_title": "Performances",
"generate_timeout_cpu_note": "Utilisé lorsque la génération s'exécute sur le processeur. Une fois enregistrée ici, cette valeur régit toujours la génération sur processeur — indépendamment du budget accéléré ci-dessus.",
"device_family_rocm": "AMD GPU (ROCm)",
"perf_torch_compile": "Désactiver torch.compile (Windows)",
"perf_torch_compile": "Désactiver torch.compile",
"generate_timeout_shadowed_note": "Une variable d'environnement extérieure à VoiceStudio (shell, fichier .env ou conteneur) définit actuellement cette valeur — la valeur enregistrée ici est ignorée tant que cette variable n'est pas supprimée.",
"device_family_cpu": "CPU",
"device_family_gpu": "GPU",
@@ -1967,7 +1967,7 @@
"dictation_replay": "Rejouer",
"dictation_transcribing": "Transcription…",
"dubbing_title": "Voir le doublage en action",
"dubbing_sync": "Lecture synchronisée",
"dubbing_sync": "Synchroniser la position",
"dubbing_picker": "Essayez une autre langue :",
"dubbing_cta": "Exécutez ceci sur votre propre vidéo →",
"dubbing_loading": "Chargement de la démo de doublage…",
electron/src/renderer/src/i18n/locales/hi.json
+2 -2
View File
@@ -1042,7 +1042,7 @@
"perf_title": "प्रदर्शन",
"generate_timeout_cpu_note": "जब जनरेशन CPU पर चलता है, तब इस्तेमाल होता है। यहाँ सेव होते ही, यह हमेशा CPU जनरेशन को नियंत्रित करता है — ऊपर के एक्सेलरेटेड बजट से स्वतंत्र।",
"device_family_rocm": "AMD GPU (ROCm)",
"perf_torch_compile": "torch.compile बंद करें (Windows)",
"perf_torch_compile": "torch.compile बंद करें",
"generate_timeout_shadowed_note": "VoiceStudio के बाहर का एक एनवायरनमेंट वेरिएबल (शेल, .env फ़ाइल, या कंटेनर) फ़िलहाल यह मान सेट कर रहा है — यहाँ सेव किया गया मान तब तक अनदेखा किया जाएगा जब तक वह वेरिएबल हटाया न जाए।",
"device_family_cpu": "CPU",
"device_family_gpu": "GPU",
@@ -1965,7 +1965,7 @@
"dictation_replay": "पुनः चलाएँ",
"dictation_transcribing": "प्रतिलेखन...",
"dubbing_title": "डबिंग क्रिया देखें",
"dubbing_sync": "समन्वयित प्लेबैक",
"dubbing_sync": "प्लेबैक स्थिति सिंक करें",
"dubbing_picker": "दूसरी भाषा आज़माएँ:",
"dubbing_cta": "इसे अपने वीडियो पर चलाएँ →",
"dubbing_loading": "डबिंग डेमो लोड हो रहा है...",
electron/src/renderer/src/i18n/locales/id.json
+2 -2
View File
@@ -1042,7 +1042,7 @@
"perf_title": "Performa",
"generate_timeout_cpu_note": "Digunakan saat proses generasi berjalan di CPU. Setelah disimpan di sini, nilai ini selalu mengatur generasi CPU — terlepas dari anggaran yang dipercepat di atas.",
"device_family_rocm": "AMD GPU (ROCm)",
"perf_torch_compile": "Nonaktifkan torch.compile (Windows)",
"perf_torch_compile": "Nonaktifkan torch.compile",
"generate_timeout_shadowed_note": "Variabel lingkungan di luar VoiceStudio (shell, file .env, atau kontainer) saat ini sedang mengatur nilai ini — nilai yang disimpan di sini diabaikan sampai variabel tersebut dihapus.",
"device_family_cpu": "CPU",
"device_family_gpu": "GPU",
@@ -1965,7 +1965,7 @@
"dictation_replay": "Putar ulang",
"dictation_transcribing": "Mentranskripsikan…",
"dubbing_title": "Lihat aksi sulih suara",
"dubbing_sync": "Pemutaran yang disinkronkan",
"dubbing_sync": "Sinkronkan posisi",
"dubbing_picker": "Coba bahasa lain:",
"dubbing_cta": "Jalankan ini di video Anda sendiri →",
"dubbing_loading": "Memuat demo sulih suara…",
electron/src/renderer/src/i18n/locales/it.json
+2 -2
View File
@@ -1044,7 +1044,7 @@
"perf_title": "Prestazioni",
"generate_timeout_cpu_note": "Usato quando la generazione viene eseguita sulla CPU. Una volta salvato qui, questo valore governa sempre la generazione su CPU, indipendentemente dal budget accelerato sopra.",
"device_family_rocm": "AMD GPU (ROCm)",
"perf_torch_compile": "Disabilita torch.compile (Windows)",
"perf_torch_compile": "Disabilita torch.compile",
"generate_timeout_shadowed_note": "Una variabile d'ambiente esterna a VoiceStudio (shell, file .env o container) sta attualmente impostando questo valore — il valore salvato qui viene ignorato finché quella variabile non viene rimossa.",
"device_family_cpu": "CPU",
"device_family_gpu": "GPU",
@@ -1967,7 +1967,7 @@
"dictation_replay": "Riproduci",
"dictation_transcribing": "Trascrizione…",
"dubbing_title": "Guarda il doppiaggio in azione",
"dubbing_sync": "Riproduzione sincronizzata",
"dubbing_sync": "Sincronizza posizione",
"dubbing_picker": "Prova un'altra lingua:",
"dubbing_cta": "Eseguilo sul tuo video →",
"dubbing_loading": "Caricamento demo del doppiaggio…",
electron/src/renderer/src/i18n/locales/ja.json
+2 -2
View File
@@ -1042,7 +1042,7 @@
"perf_title": "パフォーマンス",
"generate_timeout_cpu_note": "生成が CPU 上で実行される場合に使用されます。ここで保存すると、この値は上のアクセラレーテッド予算とは関係なく、常に CPU での生成を管理します。",
"device_family_rocm": "AMD GPU (ROCm)",
"perf_torch_compile": "torch.compile を無効にする (Windows)",
"perf_torch_compile": "torch.compile を無効にする",
"generate_timeout_shadowed_note": "VoiceStudio の外部にある環境変数（シェル、.env ファイル、またはコンテナ）が現在この値を設定しています — その変数が削除されるまで、ここで保存した値は無視されます。",
"device_family_cpu": "CPU",
"device_family_gpu": "GPU",
@@ -1965,7 +1965,7 @@
"dictation_replay": "リプレイ",
"dictation_transcribing": "文字起こし中…",
"dubbing_title": "吹き替えの様子をご覧ください",
"dubbing_sync": "同期再生",
"dubbing_sync": "再生位置を同期",
"dubbing_picker": "別の言語を試してください:",
"dubbing_cta": "これを自分のビデオで実行します→",
"dubbing_loading": "ダビングデモを読み込み中…",
electron/src/renderer/src/i18n/locales/ko.json
+2 -2
View File
@@ -1119,7 +1119,7 @@
"perf_title": "성능",
"generate_timeout_cpu_note": "생성이 CPU에서 실행될 때 사용됩니다. 여기에 저장하면 위의 가속 예산과 관계없이 이 값이 항상 CPU 생성을 관리합니다.",
"device_family_rocm": "AMD GPU (ROCm)",
"perf_torch_compile": "torch.compile 비활성화 (Windows)",
"perf_torch_compile": "torch.compile 비활성화",
"generate_timeout_shadowed_note": "VoiceStudio 외부의 환경 변수(셸, .env 파일 또는 컨테이너)가 현재 이 값을 설정하고 있습니다 — 해당 변수가 제거될 때까지 여기에 저장한 값은 무시됩니다.",
"device_family_cpu": "CPU",
"device_family_gpu": "GPU",
@@ -1965,7 +1965,7 @@
"dictation_replay": "재생",
"dictation_transcribing": "받아쓰는 중…",
"dubbing_title": "실제 더빙 보기",
"dubbing_sync": "동기화된 재생",
"dubbing_sync": "재생 위치 동기화",
"dubbing_picker": "다른 언어를 사용해 보세요:",
"dubbing_cta": "자신의 비디오에서 이것을 실행 →",
"dubbing_loading": "더빙 데모 로드 중…",
electron/src/renderer/src/i18n/locales/nl.json
+2 -2
View File
@@ -1042,7 +1042,7 @@
"perf_title": "Prestaties",
"generate_timeout_cpu_note": "Gebruikt wanneer de generatie op de CPU draait. Eenmaal hier opgeslagen, bepaalt deze waarde altijd de CPU-generatie — onafhankelijk van het versnelde budget hierboven.",
"device_family_rocm": "AMD GPU (ROCm)",
"perf_torch_compile": "torch.compile uitschakelen (Windows)",
"perf_torch_compile": "torch.compile uitschakelen",
"generate_timeout_shadowed_note": "Een omgevingsvariabele buiten VoiceStudio (shell, .env-bestand of container) stelt deze waarde momenteel in — de hier opgeslagen waarde wordt genegeerd totdat die variabele wordt verwijderd.",
"device_family_cpu": "CPU",
"device_family_gpu": "GPU",
@@ -1965,7 +1965,7 @@
"dictation_replay": "Opnieuw afspelen",
"dictation_transcribing": "Transcriberen…",
"dubbing_title": "Zie nasynchronisatie in actie",
"dubbing_sync": "Gesynchroniseerd afspelen",
"dubbing_sync": "Afspeelpositie synchroniseren",
"dubbing_picker": "Probeer een andere taal:",
"dubbing_cta": "Voer dit uit op uw eigen video →",
"dubbing_loading": "Dubdemo laden…",
electron/src/renderer/src/i18n/locales/pl.json
+2 -2
View File
@@ -1046,7 +1046,7 @@
"perf_title": "Wydajność",
"generate_timeout_cpu_note": "Używany, gdy generowanie działa na CPU. Po zapisaniu tutaj ta wartość zawsze rządzi generowaniem na CPU — niezależnie od przyspieszonego budżetu powyżej.",
"device_family_rocm": "AMD GPU (ROCm)",
"perf_torch_compile": "Wyłącz torch.compile (Windows)",
"perf_torch_compile": "Wyłącz torch.compile",
"generate_timeout_shadowed_note": "Zmienna środowiskowa spoza VoiceStudio (powłoka, plik .env lub kontener) obecnie ustawia tę wartość — zapisana tu wartość jest ignorowana, dopóki ta zmienna nie zostanie usunięta.",
"device_family_cpu": "CPU",
"device_family_gpu": "GPU",
@@ -1969,7 +1969,7 @@
"dictation_replay": "Odtwórz ponownie",
"dictation_transcribing": "Transkrypcja…",
"dubbing_title": "Zobacz dubbing w akcji",
"dubbing_sync": "Zsynchronizowane odtwarzanie",
"dubbing_sync": "Synchronizuj pozycję",
"dubbing_picker": "Spróbuj innego języka:",
"dubbing_cta": "Uruchom to na swoim własnym filmie →",
"dubbing_loading": "Ładowanie wersji demonstracyjnej kopiowania…",
electron/src/renderer/src/i18n/locales/pt.json
+2 -2
View File
@@ -1044,7 +1044,7 @@
"perf_title": "Desempenho",
"generate_timeout_cpu_note": "Usado quando a geração é executada na CPU. Depois de salvo aqui, este valor sempre rege a geração em CPU — independentemente do orçamento acelerado acima.",
"device_family_rocm": "AMD GPU (ROCm)",
"perf_torch_compile": "Desativar torch.compile (Windows)",
"perf_torch_compile": "Desativar torch.compile",
"generate_timeout_shadowed_note": "Uma variável de ambiente externa ao VoiceStudio (shell, arquivo .env ou contêiner) está definindo este valor no momento — o valor salvo aqui é ignorado até que essa variável seja removida.",
"device_family_cpu": "CPU",
"device_family_gpu": "GPU",
@@ -1967,7 +1967,7 @@
"dictation_replay": "Repetir",
"dictation_transcribing": "Transcrevendo…",
"dubbing_title": "Veja a dublagem em ação",
"dubbing_sync": "Reprodução sincronizada",
"dubbing_sync": "Sincronizar posição",
"dubbing_picker": "Tente outro idioma:",
"dubbing_cta": "Execute isso em seu próprio vídeo →",
"dubbing_loading": "Carregando demonstração de dublagem…",
electron/src/renderer/src/i18n/locales/ru.json
+2 -2
View File
@@ -1046,7 +1046,7 @@
"perf_title": "Производительность",
"generate_timeout_cpu_note": "Используется, когда генерация выполняется на CPU. После сохранения здесь это значение всегда определяет генерацию на CPU — независимо от ускоренного бюджета выше.",
"device_family_rocm": "AMD GPU (ROCm)",
"perf_torch_compile": "Отключить torch.compile (Windows)",
"perf_torch_compile": "Отключить torch.compile",
"generate_timeout_shadowed_note": "Переменная окружения вне VoiceStudio (оболочка, файл .env или контейнер) сейчас задаёт это значение — сохранённое здесь значение игнорируется, пока эта переменная не будет удалена.",
"device_family_cpu": "CPU",
"device_family_gpu": "GPU",
@@ -1969,7 +1969,7 @@
"dictation_replay": "Повтор",
"dictation_transcribing": "Расшифровка…",
"dubbing_title": "Посмотрите дубляж в действии",
"dubbing_sync": "Синхронизированное воспроизведение",
"dubbing_sync": "Синхронизировать позицию",
"dubbing_picker": "Попробуйте другой язык:",
"dubbing_cta": "Запустите это на своем собственном видео →",
"dubbing_loading": "Загрузка демо-версии дубляжа…",
electron/src/renderer/src/i18n/locales/sv.json
+2 -2
View File
@@ -1042,7 +1042,7 @@
"perf_title": "Prestanda",
"generate_timeout_cpu_note": "Används när genereringen körs på CPU:n. När den väl sparats här styr detta värde alltid CPU-generering — oberoende av den accelererade budgeten ovan.",
"device_family_rocm": "AMD GPU (ROCm)",
"perf_torch_compile": "Inaktivera torch.compile (Windows)",
"perf_torch_compile": "Inaktivera torch.compile",
"generate_timeout_shadowed_note": "En miljövariabel utanför VoiceStudio (skal, .env-fil eller container) anger just nu detta värde — värdet som sparats här ignoreras tills den variabeln tas bort.",
"device_family_cpu": "CPU",
"device_family_gpu": "GPU",
@@ -1965,7 +1965,7 @@
"dictation_replay": "Spela om",
"dictation_transcribing": "Transkriberar...",
"dubbing_title": "Se dubbning i aktion",
"dubbing_sync": "Synkroniserad uppspelning",
"dubbing_sync": "Synka uppspelningsposition",
"dubbing_picker": "Prova ett annat språk:",
"dubbing_cta": "Kör detta på din egen video →",
"dubbing_loading": "Laddar dubbningsdemo...",
electron/src/renderer/src/i18n/locales/th.json
+2 -2
View File
@@ -1042,7 +1042,7 @@
"perf_title": "ประสิทธิภาพ",
"generate_timeout_cpu_note": "ใช้เมื่อการสร้างทำงานบน CPU เมื่อบันทึกที่นี่แล้ว ค่านี้จะควบคุมการสร้างบน CPU เสมอ — โดยไม่ขึ้นกับงบแบบเร่งความเร็วด้านบน",
"device_family_rocm": "AMD GPU (ROCm)",
"perf_torch_compile": "ปิดใช้ torch.compile (Windows)",
"perf_torch_compile": "ปิดใช้ torch.compile",
"generate_timeout_shadowed_note": "มีตัวแปรสภาพแวดล้อมภายนอก VoiceStudio (เชลล์, ไฟล์ .env หรือคอนเทนเนอร์) กำลังตั้งค่านี้อยู่ — ค่าที่บันทึกไว้ที่นี่จะถูกละเว้นจนกว่าตัวแปรนั้นจะถูกลบออก",
"device_family_cpu": "CPU",
"device_family_gpu": "GPU",
@@ -1965,7 +1965,7 @@
"dictation_replay": "เล่นซ้ำ",
"dictation_transcribing": "กำลังถอดเสียง...",
"dubbing_title": "ดูการดำเนินการพากย์",
"dubbing_sync": "การเล่นแบบซิงโครไนซ์",
"dubbing_sync": "ซิงค์ตำแหน่งการเล่น",
"dubbing_picker": "ลองภาษาอื่น:",
"dubbing_cta": "เรียกใช้สิ่งนี้ในวิดีโอของคุณเอง →",
"dubbing_loading": "กำลังโหลดการสาธิตการพากย์...",
electron/src/renderer/src/i18n/locales/tr.json
+2 -2
View File
@@ -1042,7 +1042,7 @@
"perf_title": "Performans",
"generate_timeout_cpu_note": "Üretim CPU üzerinde çalıştığında kullanılır. Burada kaydedildikten sonra bu değer, yukarıdaki hızlandırılmış bütçeden bağımsız olarak her zaman CPU üretimini yönetir.",
"device_family_rocm": "AMD GPU (ROCm)",
"perf_torch_compile": "torch.compile devre dışı (Windows)",
"perf_torch_compile": "torch.compile devre dışı",
"generate_timeout_shadowed_note": "VoiceStudio dışındaki bir ortam değişkeni (kabuk, .env dosyası veya konteyner) şu anda bu değeri ayarlıyor — o değişken kaldırılana kadar burada kaydedilen değer yok sayılır.",
"device_family_cpu": "CPU",
"device_family_gpu": "GPU",
@@ -1965,7 +1965,7 @@
"dictation_replay": "Tekrar oynat",
"dictation_transcribing": "Metne dönüştürülüyor…",
"dubbing_title": "Dublajı çalışırken görün",
"dubbing_sync": "Senkronize oynatma",
"dubbing_sync": "Oynatma konumunu eşitle",
"dubbing_picker": "Başka bir dil deneyin:",
"dubbing_cta": "Bunu kendi videonuzda çalıştırın →",
"dubbing_loading": "Dublaj demosu yükleniyor…",
electron/src/renderer/src/i18n/locales/uk.json
+2 -2
View File
@@ -1046,7 +1046,7 @@
"perf_title": "Продуктивність",
"generate_timeout_cpu_note": "Використовується, коли генерація виконується на CPU. Після збереження тут це значення завжди керує генерацією на CPU — незалежно від прискореного бюджету вище.",
"device_family_rocm": "AMD GPU (ROCm)",
"perf_torch_compile": "Вимкнути torch.compile (Windows)",
"perf_torch_compile": "Вимкнути torch.compile",
"generate_timeout_shadowed_note": "Змінна середовища поза VoiceStudio (оболонка, файл .env або контейнер) наразі задає це значення — збережене тут значення ігнорується, доки цю змінну не видалять.",
"device_family_cpu": "CPU",
"device_family_gpu": "GPU",
@@ -1969,7 +1969,7 @@
"dictation_replay": "Повтор",
"dictation_transcribing": "Транскрибування…",
"dubbing_title": "Перегляньте дубляж у дії",
"dubbing_sync": "Синхронізоване відтворення",
"dubbing_sync": "Синхронізувати позицію",
"dubbing_picker": "Спробуйте іншу мову:",
"dubbing_cta": "Запустіть це на власному відео →",
"dubbing_loading": "Завантаження демонстрації дубляжу…",
electron/src/renderer/src/i18n/locales/vi.json
+2 -2
View File
@@ -1042,7 +1042,7 @@
"perf_title": "Hiệu suất",
"generate_timeout_cpu_note": "Dùng khi quá trình tạo chạy trên CPU. Sau khi lưu ở đây, giá trị này luôn chi phối việc tạo trên CPU — không phụ thuộc vào ngân sách tăng tốc ở trên.",
"device_family_rocm": "AMD GPU (ROCm)",
"perf_torch_compile": "Tắt torch.compile (Windows)",
"perf_torch_compile": "Tắt torch.compile",
"generate_timeout_shadowed_note": "Một biến môi trường bên ngoài VoiceStudio (shell, tệp .env hoặc container) hiện đang đặt giá trị này — giá trị đã lưu ở đây sẽ bị bỏ qua cho đến khi biến đó được gỡ bỏ.",
"device_family_cpu": "CPU",
"device_family_gpu": "GPU",
@@ -1965,7 +1965,7 @@
"dictation_replay": "Phát lại",
"dictation_transcribing": "Phiên âm…",
"dubbing_title": "Xem lồng tiếng đang hoạt động",
"dubbing_sync": "Phát lại được đồng bộ hóa",
"dubbing_sync": "Đồng bộ vị trí phát",
"dubbing_picker": "Hãy thử một ngôn ngữ khác:",
"dubbing_cta": "Chạy cái này trên video của riêng bạn →",
"dubbing_loading": "Đang tải bản demo lồng tiếng…",
electron/src/renderer/src/i18n/locales/zh-CN.json
+2 -2
View File
@@ -1123,7 +1123,7 @@
"perf_title": "性能",
"generate_timeout_cpu_note": "当生成运行在 CPU 上时使用。一旦在此处保存，该值将始终控制 CPU 生成——与上面的加速预算无关。",
"device_family_rocm": "AMD GPU (ROCm)",
"perf_torch_compile": "禁用 torch.compile (Windows)",
"perf_torch_compile": "禁用 torch.compile",
"generate_timeout_shadowed_note": "VoiceStudio 之外的一个环境变量（终端、.env 文件或容器）当前正在设置此值——在移除该变量之前，此处保存的值会被忽略。",
"device_family_cpu": "CPU",
"device_family_gpu": "GPU",
@@ -1969,7 +1969,7 @@
"dictation_replay": "重放",
"dictation_transcribing": "转录中…",
"dubbing_title": "查看配音效果",
"dubbing_sync": "同步播放",
"dubbing_sync": "同步播放位置",
"dubbing_picker": "试试其他语言：",
"dubbing_cta": "在你自己的视频上运行 →",
"dubbing_loading": "正在加载配音演示...",
electron/src/renderer/src/i18n/locales/zh-TW.json
+2 -2
View File
@@ -1042,7 +1042,7 @@
"perf_title": "效能",
"generate_timeout_cpu_note": "當生成執行於 CPU 上時使用。一旦在此處儲存，此值將始終控制 CPU 生成 — 與上方的加速預算無關。",
"device_family_rocm": "AMD GPU (ROCm)",
"perf_torch_compile": "停用 torch.compile (Windows)",
"perf_torch_compile": "停用 torch.compile",
"generate_timeout_shadowed_note": "VoiceStudio 之外的一個環境變數（終端機、.env 檔案或容器）目前正在設定此值 — 在移除該變數之前，此處儲存的值會被忽略。",
"device_family_cpu": "CPU",
"device_family_gpu": "GPU",
@@ -1965,7 +1965,7 @@
"dictation_replay": "重播",
"dictation_transcribing": "正在抄寫…",
"dubbing_title": "看實際配音",
"dubbing_sync": "同步播放",
"dubbing_sync": "同步播放位置",
"dubbing_picker": "嘗試另一種語言：",
"dubbing_cta": "在您自己的影片上運行此→",
"dubbing_loading": "正在載入配音演示...",
electron/src/renderer/src/lib/themes/t3-palettes.ts
+6 -6
View File
@@ -202,9 +202,9 @@ export const T3_CHAT_THEME: ThemeDefinition = {
textMuted: 'oklch(0.880303 0.03077 342.696)',
border: 'oklch(0.266943 0.015262 302.425)',
input: 'oklch(0.266817 0.02897 344.461)',
focus: 'oklch(0.591646 0.217985 0.584)',
accent: 'oklch(0.460685 0.185347 4.099)',
accentForeground: 'oklch(0.901233 0.057189 343.694)',
focus: '#f09bb8',
accent: '#f09bb8',
accentForeground: '#291d29',
secondary: 'oklch(0.313674 0.030572 310.061)',
secondaryForeground: 'oklch(0.848252 0.038248 307.961)',
muted: 'oklch(0.360924 0.021469 316.83)',
@@ -225,9 +225,9 @@ export const T3_CHAT_THEME: ThemeDefinition = {
accentSurfaceForeground: 'oklch(0.964695 0.009139 341.803)',
messageSurface: 'oklch(0.273791 0.025541 309.079)',
messageForeground: 'oklch(0.949872 0.021269 306.838)',
messageAction: 'oklch(0.460685 0.185347 4.099)',
messageActionForeground: 'oklch(0.901233 0.057189 343.694)',
messageActionHover: 'oklch(0.458754 0.184639 3.857)',
messageAction: '#f09bb8',
messageActionForeground: '#291d29',
messageActionHover: '#ffb2cc',
codeBackground: 'oklch(0.22813 0.020366 307.469)',
codeForeground: 'oklch(0.848703 0.064239 306.645)',
sidebar: 'oklch(0.185778 0.019368 322.159)',
electron/src/renderer/src/styles/globals.css
+3
View File
@@ -176,6 +176,9 @@
.native-controls-right {
padding-right: 148px;
}
.macos-notification-safe-area main .workspace-titlebar {
padding-right: 3.5rem;
}
/* Keep the Lucide family visually consistent across controls and pane headings. */
@layer base {
electron/tests/fixtures/native-bell-host.cjs
Vendored
+11
View File
@@ -0,0 +1,11 @@
const { app, BrowserWindow } = require('electron');
app.whenReady().then(() => {
const window = new BrowserWindow({
width: 1100,
height: 760,
x: 60,
y: 60,
titleBarStyle: 'hidden',
});
window.loadURL('about:blank');
});
electron/tests/fixtures/native-click.swift
Vendored
+7
View File
@@ -0,0 +1,7 @@
import CoreGraphics
import Foundation
let point = CGPoint(x: Double(CommandLine.arguments[1])!, y: Double(CommandLine.arguments[2])!)
CGEvent(mouseEventSource: nil, mouseType: .leftMouseDown, mouseCursorPosition: point, mouseButton: .left)?.post(tap: .cghidEventTap)
usleep(80000)
CGEvent(mouseEventSource: nil, mouseType: .leftMouseUp, mouseCursorPosition: point, mouseButton: .left)?.post(tap: .cghidEventTap)
electron/tests/live-route-health.mjs
+1
View File
@@ -18,6 +18,7 @@ const routes = [
'/audiobook',
'/projects',
'/tools',
'/integrations',
'/settings/general',
'/settings/appearance',
'/settings/models',
electron/tests/native-bell-repro.mjs
+67
View File
@@ -0,0 +1,67 @@
import { _electron as electron } from 'playwright';
import { execFileSync } from 'node:child_process';
import { mkdtempSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { fileURLToPath } from 'node:url';
if (process.platform !== 'darwin')
throw new Error('Requires macOS and Accessibility permission for native mouse events.');
const scratch = mkdtempSync(join(tmpdir(), 'vs-native-bell-'));
const click = join(scratch, 'native-click');
execFileSync('swiftc', [
fileURLToPath(new URL('./fixtures/native-click.swift', import.meta.url)),
'-o',
click,
]);
const app = await electron.launch({
args: [fileURLToPath(new URL('./fixtures/native-bell-host.cjs', import.meta.url))],
});
try {
const page = await app.firstWindow();
await page.addInitScript(() => {
localStorage.setItem('voicestudio.setup.complete.v1', '1');
window.voicestudio = {
app: {
platform: 'darwin',
version: 'test',
onNavigate: () => () => {},
onPersistenceFlush: () => () => {},
},
repair: {
list: async () => [],
getState: async () => ({ status: 'idle', output: '', workspaceAvailable: false }),
onEvent: () => () => {},
},
};
});
await page.goto((process.env.VOICESTUDIO_UI_URL || 'http://localhost:3902') + '/#/clone');
const bell = page.locator('[data-slot=macos-system-notifications] button');
await bell.waitFor();
const rect = await bell.boundingBox();
const bounds = await app.evaluate(({ BrowserWindow }) => {
const w = BrowserWindow.getAllWindows()[0];
w.show();
w.focus();
return w.getContentBounds();
});
await page.waitForTimeout(500);
execFileSync(click, [
String(bounds.x + rect.x + rect.width / 2),
String(bounds.y + rect.y + rect.height / 2),
]);
await page.waitForTimeout(500);
const nativeOpened = await page.locator('[data-slot=popover-content][data-open]').isVisible();
console.log(JSON.stringify({ nativeOpened }));
if (!nativeOpened) {
await bell.click();
console.log(
JSON.stringify({
automatedOpened: await page.locator('[data-slot=popover-content][data-open]').isVisible(),
}),
);
process.exitCode = 1;
}
} finally {
await app.close();
rmSync(scratch, { recursive: true, force: true });
}
electron/tests/route-layout-smoke.mjs
+1
View File
@@ -15,6 +15,7 @@ const routes = [
'/batch',
'/projects',
'/tools',
'/integrations',
'/settings',
'/settings/general',
'/settings/appearance',
electron/tests/sidebar-layout-smoke.mjs
+123 -2
View File
@@ -3,7 +3,10 @@ import assert from 'node:assert/strict';
import { mkdtempSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
const browser = await chromium.launch({ channel: 'msedge', headless: true });
const browser = await chromium.launch({
...(process.env.PLAYWRIGHT_BUNDLED === '1' ? {} : { channel: 'msedge' }),
headless: true,
});
const page = await browser.newPage();
const out = mkdtempSync(join(tmpdir(), 'voicestudio-sidebar-'));
const ui = process.env.VOICESTUDIO_UI_URL || 'http://localhost:3912';
@@ -79,8 +82,126 @@ try {
await compactMain.waitFor();
}
}
const macPage = await browser.newPage();
try {
await macPage.addInitScript(() => {
localStorage.setItem('voicestudio.setup.complete.v1', '1');
Object.defineProperty(window, 'voicestudio', {
value: {
app: {
version: 'test',
platform: 'darwin',
isDev: true,
onNavigate: () => () => {},
onPersistenceFlush: () => () => {},
},
repair: {
list: async () => [],
getState: async () => ({
status: 'idle',
output: '',
workspaceAvailable: false,
}),
onEvent: () => () => {},
},
},
});
});
await macPage.goto(ui + '/#/clone');
const macSidebar = macPage.locator('aside').first();
const macNotifications = macPage.locator('[data-slot=macos-system-notifications]');
const macNotificationBounds = await macNotifications.boundingBox();
assert.ok(
macNotificationBounds &&
macNotificationBounds.y < 20 &&
macNotificationBounds.x + macNotificationBounds.width >=
(await macPage.evaluate(() => window.innerWidth)) - 20,
'macOS notifications must sit in the top-right titlebar corner',
);
const titlebarActions = await macPage.locator('main .workspace-titlebar button').all();
const titlebarActionBounds = (
await Promise.all(titlebarActions.map((action) => action.boundingBox()))
).filter(Boolean);
assert.ok(
macNotificationBounds &&
titlebarActionBounds.length > 0 && titlebarActionBounds.every(
(bounds) => bounds.x + bounds.width <= macNotificationBounds.x - 12,
),
'macOS titlebar actions must leave space before notifications',
);
await macNotifications.getByRole('button').first().click();
await macPage.locator('[data-slot=popover-content][data-open]').waitFor({ state: 'visible' });
await macPage.keyboard.press('Escape');
const brandLink = macSidebar.getByRole('link', { name: 'VoiceStudio', exact: true });
const brandBounds = await brandLink.boundingBox();
assert.ok(brandBounds && brandBounds.x >= 96, 'macOS brand must clear the traffic lights');
assert.ok(
await brandLink
.locator('span')
.evaluate((element) => element.scrollWidth <= element.clientWidth),
'macOS titlebar must show the complete VoiceStudio wordmark',
);
const expandedSettings = macSidebar.getByRole('link', { name: 'Settings', exact: true });
const expandedDevice = macSidebar.getByRole('button', { name: /Local device/ });
const expandedSettingsBounds = await expandedSettings.boundingBox();
const expandedDeviceBounds = await expandedDevice.boundingBox();
assert.equal((await expandedSettings.innerText()).trim(), '');
assert.ok(
expandedSettingsBounds &&
expandedDeviceBounds &&
expandedDeviceBounds.x > expandedSettingsBounds.x,
'expanded macOS Local device must sit right of icon-only Settings',
);
await expandedDevice.click();
await macPage.locator('[data-slot=popover-content][data-open]').waitFor({ state: 'visible' });
await macPage.keyboard.press('Escape');
await macSidebar.getByRole('button', { name: 'Close', exact: true }).click();
const compactMacSidebar = macPage.locator('[data-slot=compact-main-sidebar]');
await compactMacSidebar.waitFor();
assert.equal(Math.round((await compactMacSidebar.boundingBox()).width), 64);
const compactDividerBounds = await compactMacSidebar
.locator('[data-slot=compact-sidebar-divider]')
.boundingBox();
assert.ok(
compactDividerBounds && compactDividerBounds.y >= 72,
'macOS compact-sidebar divider must begin below the titlebar',
);
const compactToggleBounds = await compactMacSidebar
.getByRole('button', { name: 'Toggle Sidebar', exact: true })
.boundingBox();
assert.ok(
compactToggleBounds && compactToggleBounds.y >= 32,
'macOS compact-sidebar toggle must sit below the traffic lights',
);
const compactSettingsBounds = await compactMacSidebar
.getByRole('link', { name: 'Settings', exact: true })
.boundingBox();
const compactDeviceBounds = await compactMacSidebar
.getByRole('button', { name: /Local device/ })
.boundingBox();
assert.ok(
compactSettingsBounds &&
compactDeviceBounds &&
compactDeviceBounds.x > compactSettingsBounds.x &&
Math.abs(
compactDeviceBounds.y +
compactDeviceBounds.height / 2 -
(compactSettingsBounds.y + compactSettingsBounds.height / 2),
) <= 1,
`macOS Local device must sit to the right of Settings: ${JSON.stringify({ compactSettingsBounds, compactDeviceBounds })}`,
);
const workspaceTitleBounds = await macPage
.getByRole('heading', { name: 'Voice cloning' })
.boundingBox();
assert.ok(
workspaceTitleBounds && workspaceTitleBounds.x >= 88,
'macOS workspace title must clear the traffic lights',
);
} finally {
await macPage.close();
}
console.log(
'Sidebar compact/expanded, 9 destinations, 6 engine links, visible models, non-duplicated Profiles, settings visibility and navigation passed. ' +
'Sidebar compact/expanded, macOS titlebar clearance, 9 destinations, 6 engine links, visible models, non-duplicated Profiles, settings visibility and navigation passed. ' +
out,
);
} finally {
frontend/package.json
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "omnivoice-studio",
"version": "0.5.2",
"version": "0.5.3",
"private": true,
"license": "AGPL-3.0-only",
"type": "module",
frontend/src-tauri/Cargo.lock
Generated
+1 -1
View File
@@ -3093,7 +3093,7 @@ dependencies = [
[[package]]
name = "omnivoice-studio"
version = "0.5.2"
version = "0.5.3"
dependencies = [
"arboard",
"cap-std",
frontend/src-tauri/Cargo.toml
+1 -1
View File
@@ -4,7 +4,7 @@
# launcher's pkill matches `omnivoice-studio` and must never match a user's
# installed app. Renaming it would collapse that distinction.
name = "omnivoice-studio"
version = "0.5.2"
version = "0.5.3"
description = "VoiceStudio – AI voice cloning & dubbing desktop app"
authors = ["Debpalash"]
license = "AGPL-3.0-only"
frontend/src-tauri/src/dictation_output.rs
+70 -12
View File
@@ -38,6 +38,7 @@ pub struct DictationOutput {
#[derive(Default)]
struct Inner {
owner_pid: Option<u32>,
next_session_id: AtomicU64,
operation: Mutex<()>,
state: Mutex<OutputState>,
@@ -81,6 +82,21 @@ enum ClipboardSnapshot {
}
impl DictationOutput {
/// Native-helper hosts identify the UI process so tray capture excludes its windows.
/// In-process callers retain the existing default (the current process).
pub fn for_owner(owner_pid: u32) -> Self {
Self {
inner: Arc::new(Inner {
owner_pid: Some(owner_pid),
..Inner::default()
}),
}
}
fn owner_pid(&self) -> u32 {
self.inner.owner_pid.unwrap_or_else(std::process::id)
}
/// Remember focus on tray mouse-down, before the menu itself can become
/// foreground. Tauri does not emit tray pointer events on Linux; X11 can
/// still capture `_NET_ACTIVE_WINDOW` at the menu action, while Wayland
@@ -97,7 +113,7 @@ impl DictationOutput {
// can hold the operation lock while another application becomes
// foreground, so acquiring it first would capture the wrong target.
let captured_at = Instant::now();
let target = capture().filter(|target| !target.belongs_to_current_process());
let target = capture().filter(|target| !target.belongs_to_process(self.owner_pid()));
if let Ok(mut state) = self.inner.state.lock() {
state.tray_target = target.map(|target| (target, captured_at));
}
@@ -124,7 +140,7 @@ impl DictationOutput {
.then(capture)
.flatten()
.filter(|target| {
origin == CaptureOrigin::Shortcut || !target.belongs_to_current_process()
origin == CaptureOrigin::Shortcut || !target.belongs_to_process(self.owner_pid())
});
let mut state = self
.inner
@@ -278,6 +294,12 @@ impl DictationOutput {
return Ok(DeliveryOutcome::Copied);
}
self.schedule_restore(session_id, generation, text.to_owned());
// SendInput/CGEventPost/XTest enqueue keystrokes; success does not mean
// the target has consumed its paste. Keep the operation lock through
// the same consumption window used by clipboard restoration, so a
// second utterance cannot replace the clipboard or reset modifier state
// (Windows AttachThreadInput) while the first Ctrl/Cmd+V is queued.
thread::sleep(CLIPBOARD_CONSUME_DELAY);
Ok(DeliveryOutcome::Inserted)
}
@@ -777,8 +799,8 @@ struct PlatformTarget {
#[cfg(target_os = "macos")]
impl PlatformTarget {
fn belongs_to_current_process(&self) -> bool {
self.pid == std::process::id() as i32
fn belongs_to_process(&self, owner_pid: u32) -> bool {
self.pid == owner_pid as i32
}
}
@@ -818,8 +840,8 @@ struct PlatformTarget {
#[cfg(target_os = "windows")]
impl PlatformTarget {
fn belongs_to_current_process(&self) -> bool {
self.pid == std::process::id()
fn belongs_to_process(&self, owner_pid: u32) -> bool {
self.pid == owner_pid
}
}
@@ -864,12 +886,23 @@ fn activate_target(target: &PlatformTarget) -> bool {
return false;
}
let current_thread = GetCurrentThreadId();
let attached = target_thread != current_thread
&& AttachThreadInput(current_thread, target_thread, true).as_bool();
let foreground = GetForegroundWindow();
let foreground_thread = if foreground.0.is_null() {
0
} else {
GetWindowThreadProcessId(foreground, None)
};
// Windows grants foreground activation through the thread that owns the
// current foreground window. The recorder can become foreground when its
// Stop button is clicked, so attaching to the destination thread does not
// transfer that right and SetForegroundWindow can silently fail.
let attached = foreground_thread != 0
&& foreground_thread != current_thread
&& AttachThreadInput(current_thread, foreground_thread, true).as_bool();
let _ = BringWindowToTop(hwnd);
let requested = SetForegroundWindow(hwnd).as_bool();
if attached {
let _ = AttachThreadInput(current_thread, target_thread, false);
let _ = AttachThreadInput(current_thread, foreground_thread, false);
}
if !requested {
return false;
@@ -906,8 +939,8 @@ struct PlatformTarget {
#[cfg(target_os = "linux")]
impl PlatformTarget {
fn belongs_to_current_process(&self) -> bool {
self.pid == Some(std::process::id())
fn belongs_to_process(&self, owner_pid: u32) -> bool {
self.pid == Some(owner_pid)
}
}
@@ -1049,7 +1082,7 @@ struct PlatformTarget;
#[cfg(not(any(target_os = "macos", target_os = "windows", target_os = "linux")))]
impl PlatformTarget {
fn belongs_to_current_process(&self) -> bool {
fn belongs_to_process(&self, _owner_pid: u32) -> bool {
false
}
}
@@ -1144,6 +1177,7 @@ impl DictationOutput {
return Ok(DeliveryOutcome::Copied);
}
self.schedule_restore(session_id, generation, text.to_owned());
thread::sleep(CLIPBOARD_CONSUME_DELAY);
return Ok(DeliveryOutcome::Inserted);
}
LinuxTool::Ydotool => {
@@ -1155,6 +1189,7 @@ impl DictationOutput {
return Ok(DeliveryOutcome::Copied);
}
self.schedule_restore(session_id, generation, text.to_owned());
thread::sleep(CLIPBOARD_CONSUME_DELAY);
return Ok(DeliveryOutcome::Inserted);
}
}
@@ -1357,6 +1392,29 @@ mod tests {
use std::thread;
use std::time::{Duration, Instant};
#[cfg(target_os = "windows")]
#[test]
fn helper_tray_capture_excludes_the_owner_window() {
let owner = std::process::id().wrapping_add(1);
let output = DictationOutput::for_owner(owner);
let session = output.begin_session_with(
CaptureOrigin::Tray,
|| {
Some(super::PlatformTarget {
hwnd: 1,
pid: owner,
})
},
true,
);
let state = output.inner.state.lock().unwrap();
let active = state.active.as_ref().unwrap();
assert_eq!(active.id, session);
assert!(active.target.is_none());
assert!(active.clipboard_only);
assert_eq!(DictationOutput::default().owner_pid(), std::process::id());
}
#[test]
fn clipboard_restore_never_overwrites_a_new_user_copy() {
assert!(clipboard_still_staged(
frontend/src-tauri/src/lib.rs
+3
View File
@@ -23,8 +23,11 @@ pub mod tools;
pub mod uninstall;
pub mod updater_channel;
pub mod watch_folder;
mod watch_folder_core;
#[cfg(target_os = "linux")]
pub mod wayland_shortcut;
#[cfg(target_os = "linux")]
pub mod wayland_shortcut_core;
use std::collections::{HashMap, VecDeque};
use std::process::Child;
frontend/src-tauri/src/tools.rs
+1 -1
View File
@@ -653,7 +653,7 @@ fn assign_job_and_resume(
// Version of the Astral `uv` binary we download at first run when no system
// uv is on PATH. Pinned for reproducibility — bump alongside the uv.lock
// when the toolchain needs a newer uv.
pub const UV_VERSION: &str = "0.11.7";
pub const UV_VERSION: &str = "0.12.13";
// Version of BtbN/FFmpeg-Builds we download for Linux/Windows ffmpeg first-
// run setup. The string appears *twice* in each URL (once as the release tag,
frontend/src-tauri/src/watch_folder.rs
+19 -645
View File
@@ -1,402 +1,24 @@
//! Batch watch-folder IPC: native folder pick, polling scan, and upload.
//!
//! The watcher lives entirely on the client side of the app: the webview asks
//! this module (over Tauri IPC) for directory listings and asks Rust to stream
//! a settled file through the existing `POST /batch/enqueue` multipart route.
//! The Python backend only ever sees uploaded bytes — filesystem paths never
//! ride an HTTP request (same posture as `commands::authorize_host_path`).
//!
//! Access model: the folder is picked in a native dialog inside this process
//! and registered under a random session token, together with a `cap_std`
//! directory HANDLE opened at pick time. Scan/read commands resolve entries
//! relative to that handle — the pathname is never re-resolved for *access*,
//! so swapping the directory (or any component of its path) for a
//! symlink/junction later cannot redirect the watcher, on any OS. The stored
//! pathname is re-resolved only by the liveness/identity check, which stops
//! the watcher loudly when the folder is deleted, moved, or replaced. The
//! webview cannot point the commands at an arbitrary path; reads are confined
//! to files sitting directly in the folder the user explicitly picked this
//! session (non-recursive by design).
//!
//! Holding the handle must not lock the user's folder: `cap_std` opens
//! directories on Windows WITHOUT `FILE_SHARE_DELETE` (it pins the pathname
//! for its own path-based helpers), which would make Explorer refuse to
//! rename or delete a watched folder until the watch is stopped — a
//! Windows-only behaviour the other two platforms don't have. The handle is
//! therefore opened here with the full share mode (`open_dir_handle`), so
//! replacing the folder behaves identically everywhere: the OS allows it, the
//! next poll's identity check fails, and the UI stops the watcher.
use std::collections::HashMap;
use std::fs;
use std::io::{self, Read};
use std::path::{Path, PathBuf};
use std::sync::{Mutex, OnceLock};
use std::time::UNIX_EPOCH;
#[cfg(not(windows))]
use cap_std::ambient_authority;
use cap_std::fs::Dir;
use serde::Serialize;
//! Tauri adapter for shared capability-scoped batch watch folders.
pub use crate::watch_folder_core::{WatchEntry, WatchFolderSelection, WatchFolderUploadReply};
use tauri_plugin_dialog::DialogExt;
/// An authorized watch folder: the directory handle everything resolves
/// against, plus the identity the folder had when the user picked it. The
/// handle is the security boundary (operations can never leave it); the
/// identity check is the LIVENESS signal — when the folder is deleted, moved,
/// or replaced, token resolution fails loudly and the UI stops the watcher
/// instead of polling silently forever.
struct WatchedDir {
/// Fully-resolved directory path captured at pick time.
canonical: PathBuf,
/// Filesystem identity (device, inode) captured at pick time.
#[cfg(unix)]
identity: (u64, u64),
/// Filesystem identity (volume serial, file index) captured at pick time.
#[cfg(windows)]
identity: (u32, u64),
/// Directory handle captured at pick time — all scans/reads go through it.
handle: Dir,
}
#[cfg(unix)]
fn dir_identity(meta: &fs::Metadata) -> (u64, u64) {
use std::os::unix::fs::MetadataExt;
(meta.dev(), meta.ino())
}
#[cfg(windows)]
fn dir_identity(dir: &Dir) -> Result<(u32, u64), String> {
use std::os::windows::io::AsRawHandle;
use windows::Win32::Foundation::HANDLE;
use windows::Win32::Storage::FileSystem::{
GetFileInformationByHandle, BY_HANDLE_FILE_INFORMATION,
};
let mut info = BY_HANDLE_FILE_INFORMATION::default();
// SAFETY: `dir` owns a live directory handle for the duration of this
// call, and `info` is a valid writable output buffer.
unsafe { GetFileInformationByHandle(HANDLE(dir.as_raw_handle()), &mut info) }
.map_err(|_| "Selected watch folder identity could not be read".to_string())?;
Ok((
info.dwVolumeSerialNumber,
((info.nFileIndexHigh as u64) << 32) | info.nFileIndexLow as u64,
))
}
/// Open a directory handle for capability-scoped access.
///
/// Unix: `cap_std`'s own ambient open. Windows: the same
/// `FILE_FLAG_BACKUP_SEMANTICS` directory open `cap_std` performs, but with
/// `FILE_SHARE_DELETE` included so the user can still rename/delete the folder
/// while it is watched (see the module docs). Child opens stay handle-relative
/// (`CreateFileAtW` / `NtCreateFile` with a root directory) so confinement is
/// unaffected; only the liveness check observes the rename, which is the
/// intended signal.
#[cfg(not(windows))]
fn open_dir_handle(dir: &Path) -> io::Result<Dir> {
Dir::open_ambient_dir(dir, ambient_authority())
}
#[cfg(windows)]
fn open_dir_handle(dir: &Path) -> io::Result<Dir> {
use std::os::windows::fs::OpenOptionsExt;
use windows::Win32::Storage::FileSystem::{
FILE_FLAG_BACKUP_SEMANTICS, FILE_SHARE_DELETE, FILE_SHARE_READ, FILE_SHARE_WRITE,
};
let file = fs::OpenOptions::new()
.read(true)
.custom_flags(FILE_FLAG_BACKUP_SEMANTICS.0)
.share_mode((FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE).0)
.open(dir)?;
if !file.metadata()?.is_dir() {
return Err(io::Error::other("not a directory"));
}
Ok(Dir::from_std_file(file))
}
fn authorize_watched_dir(dir: &Path) -> Result<WatchedDir, String> {
let canonical = fs::canonicalize(dir)
.map_err(|e| format!("Selected watch folder could not be resolved: {e}"))?;
if !canonical.is_dir() {
return Err("Selected watch folder is not a directory".into());
}
let handle = open_dir_handle(&canonical)
.map_err(|e| format!("Selected watch folder could not be opened: {e}"))?;
#[cfg(unix)]
let identity = dir_identity(
&fs::metadata(&canonical)
.map_err(|e| format!("Selected watch folder could not be inspected: {e}"))?,
);
#[cfg(windows)]
let identity = dir_identity(&handle)?;
Ok(WatchedDir {
canonical,
#[cfg(unix)]
identity,
#[cfg(windows)]
identity,
handle,
})
}
/// Re-verify a watched folder's identity: the stored path must still resolve
/// to the same canonical target (and, on unix, the same device+inode). A
/// deleted, moved, replaced, or recreated directory fails here, which is what
/// stops the watcher loudly in the UI. Reads never depend on this check for
/// confinement — they go through the pinned handle regardless.
fn verify_watched_dir(watched: &WatchedDir) -> Result<(), String> {
let canonical_now = fs::canonicalize(&watched.canonical)
.map_err(|_| "Watched folder is no longer accessible".to_string())?;
if canonical_now != watched.canonical {
return Err("Watched folder changed identity".into());
}
#[cfg(unix)]
{
let meta = fs::metadata(&canonical_now)
.map_err(|_| "Watched folder is no longer accessible".to_string())?;
if dir_identity(&meta) != watched.identity {
return Err("Watched folder changed identity".into());
}
}
#[cfg(windows)]
{
let current = open_dir_handle(&canonical_now)
.map_err(|_| "Watched folder is no longer accessible".to_string())?;
if dir_identity(&current)? != watched.identity {
return Err("Watched folder changed identity".into());
}
}
Ok(())
}
fn registry() -> &'static Mutex<HashMap<String, WatchedDir>> {
static WATCHED: OnceLock<Mutex<HashMap<String, WatchedDir>>> = OnceLock::new();
WATCHED.get_or_init(|| Mutex::new(HashMap::new()))
}
#[derive(Serialize)]
pub struct WatchFolderSelection {
token: String,
path: String,
}
#[derive(Serialize)]
pub struct WatchEntry {
name: String,
size: u64,
/// Modification time in ms since the Unix epoch (0 when unavailable).
mtime: u64,
}
fn new_token() -> Result<String, String> {
let mut random = [0_u8; 32];
getrandom::fill(&mut random).map_err(|e| format!("Secure randomness unavailable: {e}"))?;
Ok(random.iter().map(|b| format!("{b:02x}")).collect())
}
/// Resolve a session token to a clone of its pinned directory handle,
/// re-verifying the folder's liveness/identity on every access.
fn registered_dir(token: &str) -> Result<Dir, String> {
let map = registry()
.lock()
.map_err(|_| "Watch-folder registry poisoned".to_string())?;
let watched = map
.get(token)
.ok_or_else(|| "Watch folder is not authorized".to_string())?;
verify_watched_dir(watched)?;
watched
.handle
.try_clone()
.map_err(|e| format!("Watched folder handle could not be reused: {e}"))
}
/// A directory entry name must be a single plain path component — anything
/// that could climb out of the watched folder is rejected. (The `cap_std`
/// handle would also refuse an escape; this keeps the error crisp and the
/// contract explicit.)
fn validate_entry_name(name: &str) -> Result<(), String> {
if name.is_empty()
|| name == "."
|| name == ".."
|| name.contains('/')
|| name.contains('\\')
|| name.chars().any(|c| c.is_control())
{
return Err("Invalid watch-folder entry name".into());
}
Ok(())
}
fn mtime_ms(meta: &fs::Metadata) -> u64 {
meta.modified()
.ok()
.and_then(|t| t.duration_since(UNIX_EPOCH).ok())
.map(|d| d.as_millis() as u64)
.unwrap_or(0)
}
fn cap_mtime_ms(meta: &cap_std::fs::Metadata) -> u64 {
meta.modified()
.ok()
.and_then(|t| t.into_std().duration_since(UNIX_EPOCH).ok())
.map(|d| d.as_millis() as u64)
.unwrap_or(0)
}
/// Non-recursive listing of the regular files in the watched folder (name,
/// size, mtime), resolved through the pinned handle. Symlinks are skipped
/// outright — the read path cannot follow them out of the folder anyway, so
/// listing them would only produce entries that can never be ingested.
fn scan_dir(dir: &Dir) -> Result<Vec<WatchEntry>, String> {
let mut entries = Vec::new();
let read = dir
.entries()
.map_err(|e| format!("Watched folder is unreadable: {e}"))?;
for item in read.flatten() {
let Ok(file_type) = item.file_type() else {
continue;
};
if !file_type.is_file() {
continue;
}
let Ok(meta) = item.metadata() else { continue };
let Ok(name) = item.file_name().into_string() else {
continue; // non-UTF-8 names can't round-trip through IPC; skip
};
entries.push(WatchEntry {
name,
size: meta.len(),
mtime: cap_mtime_ms(&meta),
});
}
Ok(entries)
}
/// Open a settled watched file through the pinned directory handle. A symlink
/// outside the folder cannot be opened, and the returned reader revalidates
/// size+mtime around every network read so a mutation aborts the upload.
fn open_watched_reader(
dir: &Dir,
name: &str,
expected_size: u64,
expected_mtime: u64,
) -> Result<SnapshotReader, String> {
validate_entry_name(name)?;
let file = dir
.open(name)
.map_err(|e| format!("Watched file could not be opened: {e}"))?
.into_std();
let meta = file
.metadata()
.map_err(|e| format!("Watched file could not be inspected: {e}"))?;
if !meta.is_file() {
return Err("Watched entry is not a regular file".into());
}
if meta.len() != expected_size || mtime_ms(&meta) != expected_mtime {
return Err("Watched file changed after it was scanned".into());
}
Ok(SnapshotReader {
file,
expected_size,
expected_mtime,
})
}
#[derive(Debug)]
struct SnapshotReader {
file: fs::File,
expected_size: u64,
expected_mtime: u64,
}
impl SnapshotReader {
fn validate(&self) -> io::Result<()> {
let meta = self.file.metadata()?;
if !meta.is_file()
|| meta.len() != self.expected_size
|| mtime_ms(&meta) != self.expected_mtime
{
return Err(io::Error::other("watched file changed during upload"));
}
Ok(())
}
}
impl Read for SnapshotReader {
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
self.validate()?;
let read = self.file.read(buf)?;
self.validate()?;
Ok(read)
}
}
/// Open the native folder picker and register the chosen directory for this
/// session. Returns `None` when the user cancels.
#[tauri::command]
pub async fn watch_folder_pick(
app: tauri::AppHandle,
) -> Result<Option<WatchFolderSelection>, String> {
let picked = app
.dialog()
app.dialog()
.file()
.blocking_pick_folder()
.and_then(|value| value.into_path().ok());
let Some(dir) = picked else {
return Ok(None);
};
if !dir.is_absolute() || !dir.is_dir() {
return Err("Selected watch folder is not a directory".into());
}
let watched = authorize_watched_dir(&dir)?;
let display = watched.canonical.to_string_lossy().into_owned();
let token = new_token()?;
registry()
.lock()
.map_err(|_| "Watch-folder registry poisoned".to_string())?
.insert(token.clone(), watched);
Ok(Some(WatchFolderSelection {
token,
path: display,
}))
.and_then(|value| value.into_path().ok())
.map(|dir| crate::watch_folder_core::register(&dir))
.transpose()
}
/// List the files currently sitting in the watched folder (non-recursive).
#[tauri::command]
pub fn watch_folder_scan(token: String) -> Result<Vec<WatchEntry>, String> {
scan_dir(&registered_dir(&token)?)
crate::watch_folder_core::scan(token)
}
#[derive(Serialize)]
pub struct WatchFolderUploadReply {
status: u16,
body: serde_json::Value,
}
fn video_mime(name: &str) -> &'static str {
match Path::new(name)
.extension()
.and_then(|ext| ext.to_str())
.unwrap_or_default()
.to_ascii_lowercase()
.as_str()
{
"mp4" | "m4v" => "video/mp4",
"mov" => "video/quicktime",
"mkv" => "video/x-matroska",
"webm" => "video/webm",
"avi" => "video/x-msvideo",
"mpg" | "mpeg" => "video/mpeg",
"wmv" => "video/x-ms-wmv",
_ => "application/octet-stream",
}
}
/// Stream one settled watched file directly from its pinned OS handle to the
/// loopback backend. Keeping bytes out of WebView IPC avoids an O(file size)
/// renderer allocation for multi-gigabyte videos.
#[tauri::command]
pub async fn watch_folder_enqueue(
token: String,
@@ -407,272 +29,24 @@ pub async fn watch_folder_enqueue(
voice_id: Option<String>,
preserve_bg: bool,
) -> Result<WatchFolderUploadReply, String> {
let dir = registered_dir(&token)?;
let reader = open_watched_reader(&dir, &name, expected_size, expected_mtime)?;
let mime = video_mime(&name);
let url = format!("http://127.0.0.1:{}/batch/enqueue", crate::backend_port());
let port = crate::backend_port();
tauri::async_runtime::spawn_blocking(move || {
let part = reqwest::blocking::multipart::Part::reader_with_length(reader, expected_size)
.file_name(name)
.mime_str(mime)
.map_err(|_| "Watched file type could not be prepared".to_string())?;
let mut form = reqwest::blocking::multipart::Form::new()
.part("video", part)
.text("langs", langs.join(","))
.text("preserve_bg", preserve_bg.to_string());
if let Some(voice_id) = voice_id.filter(|value| !value.is_empty()) {
form = form.text("voice_id", voice_id);
}
let response = reqwest::blocking::Client::builder()
.no_proxy()
.connect_timeout(std::time::Duration::from_secs(5))
.build()
.map_err(|_| "Watch-folder upload client could not start".to_string())?
.post(url)
.multipart(form)
.send()
.map_err(|_| "Watch-folder upload failed".to_string())?;
let status = response.status().as_u16();
let body = response
.json::<serde_json::Value>()
.map_err(|_| "Watch-folder backend returned an invalid response".to_string())?;
Ok(WatchFolderUploadReply { status, body })
crate::watch_folder_core::enqueue(
port,
token,
name,
expected_size,
expected_mtime,
langs,
voice_id,
preserve_bg,
)
})
.await
.map_err(|_| "Watch-folder upload task failed".to_string())?
}
/// Drop a watch-folder authorization (watcher stopped or component unmounted).
#[tauri::command]
pub fn watch_folder_forget(token: String) {
if let Ok(mut map) = registry().lock() {
map.remove(&token);
}
}
#[cfg(test)]
mod tests {
use super::{
authorize_watched_dir, mtime_ms, open_dir_handle, open_watched_reader, scan_dir,
validate_entry_name, verify_watched_dir, Dir,
};
use std::fs;
use std::io::Read;
use std::path::PathBuf;
fn temp_watch_dir(tag: &str) -> PathBuf {
let dir = std::env::temp_dir().join(format!("vs-watch-{tag}-{}", std::process::id()));
let _ = fs::remove_dir_all(&dir);
fs::create_dir_all(&dir).unwrap();
dir
}
fn open_handle(dir: &std::path::Path) -> Dir {
open_dir_handle(dir).unwrap()
}
fn snapshot(path: &std::path::Path) -> (u64, u64) {
let meta = fs::metadata(path).unwrap();
(meta.len(), mtime_ms(&meta))
}
#[test]
fn entry_names_must_be_single_components() {
assert!(validate_entry_name("clip.mp4").is_ok());
assert!(validate_entry_name("weird name (1).MOV").is_ok());
for bad in [
"",
".",
"..",
"a/b.mp4",
"a\\b.mp4",
"..\\up.mp4",
"x\n.mp4",
] {
assert!(validate_entry_name(bad).is_err(), "accepted {bad:?}");
}
}
#[test]
fn scan_lists_regular_files_with_size_and_mtime_and_skips_dirs() {
let dir = temp_watch_dir("scan");
fs::create_dir_all(dir.join("nested")).unwrap();
fs::write(dir.join("a.mp4"), b"12345").unwrap();
fs::write(dir.join("notes.txt"), b"x").unwrap();
let mut entries = scan_dir(&open_handle(&dir)).unwrap();
entries.sort_by(|a, b| a.name.cmp(&b.name));
let names: Vec<&str> = entries.iter().map(|e| e.name.as_str()).collect();
// Directories are skipped; filtering to *videos* is the frontend's job.
assert_eq!(names, ["a.mp4", "notes.txt"]);
assert_eq!(entries[0].size, 5);
assert!(entries[0].mtime > 0);
let _ = fs::remove_dir_all(&dir);
}
#[test]
fn snapshot_reader_streams_the_exact_bytes() {
let dir = temp_watch_dir("stream");
fs::write(dir.join("clip.mp4"), b"0123456789").unwrap();
let (size, mtime) = snapshot(&dir.join("clip.mp4"));
let handle = open_handle(&dir);
let mut reader = open_watched_reader(&handle, "clip.mp4", size, mtime).unwrap();
let mut whole = Vec::new();
reader.read_to_end(&mut whole).unwrap();
assert_eq!(whole, b"0123456789");
let _ = fs::remove_dir_all(&dir);
}
#[test]
fn reads_are_bound_to_the_settled_snapshot() {
let dir = temp_watch_dir("snapshot");
fs::write(dir.join("clip.mp4"), b"settled bytes").unwrap();
let (size, mtime) = snapshot(&dir.join("clip.mp4"));
let handle = open_handle(&dir);
// The file is replaced after the scan settled → the read must refuse
// rather than upload bytes the tracker never saw stabilize.
fs::write(dir.join("clip.mp4"), b"replaced with something longer").unwrap();
let err = open_watched_reader(&handle, "clip.mp4", size, mtime).unwrap_err();
assert!(err.contains("changed"), "unexpected error: {err}");
let _ = fs::remove_dir_all(&dir);
}
#[test]
fn snapshot_reader_aborts_when_file_changes_during_stream() {
let dir = temp_watch_dir("mid-stream-change");
fs::write(dir.join("clip.mp4"), b"settled bytes").unwrap();
let (size, mtime) = snapshot(&dir.join("clip.mp4"));
let handle = open_handle(&dir);
let mut reader = open_watched_reader(&handle, "clip.mp4", size, mtime).unwrap();
fs::write(dir.join("clip.mp4"), b"different-length bytes").unwrap();
let mut byte = [0_u8; 1];
assert!(reader.read(&mut byte).is_err());
let _ = fs::remove_dir_all(&dir);
}
#[test]
fn authorization_pins_the_directory_identity() {
let dir = temp_watch_dir("identity");
let watched = authorize_watched_dir(&dir).unwrap();
// Untouched directory verifies fine…
assert!(verify_watched_dir(&watched).is_ok());
// …and a directory that disappears after authorization is refused.
// The removal itself must succeed WHILE the handle is held: a watch
// that locked the user's folder against deletion (Windows sharing
// violation, OS error 32) would be a Windows-only behaviour.
fs::remove_dir_all(&dir).unwrap();
assert!(verify_watched_dir(&watched).is_err());
}
#[test]
fn a_watched_folder_can_be_renamed_by_the_user_while_watched() {
// Cross-platform contract: holding the pinned handle never blocks the
// user from moving the folder (Explorer/Finder/mv). The liveness
// check is what notices — it must refuse, not the OS.
let dir = temp_watch_dir("rename-while-watched");
let moved = dir.with_extension("moved");
let _ = fs::remove_dir_all(&moved);
let watched = authorize_watched_dir(&dir).unwrap();
fs::rename(&dir, &moved).unwrap();
assert!(verify_watched_dir(&watched).is_err());
// The pinned handle still points at the ORIGINAL directory object.
fs::write(moved.join("clip.mp4"), b"x").unwrap();
let names: Vec<String> = scan_dir(&watched.handle)
.unwrap()
.into_iter()
.map(|e| e.name)
.collect();
assert_eq!(names, ["clip.mp4"]);
drop(watched);
let _ = fs::remove_dir_all(&moved);
}
#[cfg(unix)]
#[test]
fn a_directory_swapped_for_a_symlink_is_refused_and_never_followed() {
let dir = temp_watch_dir("dir-swap");
let elsewhere = temp_watch_dir("dir-swap-target");
fs::write(elsewhere.join("clip.mp4"), b"outside").unwrap();
let (size, mtime) = snapshot(&elsewhere.join("clip.mp4"));
let watched = authorize_watched_dir(&dir).unwrap();
assert!(verify_watched_dir(&watched).is_ok());
// Replace the authorized directory itself with a symlink pointing
// somewhere else. Token resolution refuses (identity check)…
fs::remove_dir_all(&dir).unwrap();
std::os::unix::fs::symlink(&elsewhere, &dir).unwrap();
let err = verify_watched_dir(&watched).unwrap_err();
assert!(err.contains("identity"), "unexpected error: {err}");
// …and even the pinned handle cannot reach the swap target: it still
// points at the ORIGINAL (now unlinked) directory, which is empty.
assert!(scan_dir(&watched.handle).unwrap().is_empty());
assert!(open_watched_reader(&watched.handle, "clip.mp4", size, mtime).is_err());
let _ = fs::remove_file(&dir);
let _ = fs::remove_dir_all(&elsewhere);
}
#[cfg(unix)]
#[test]
fn a_recreated_directory_at_the_same_path_is_refused() {
let dir = temp_watch_dir("dir-recreate");
let watched = authorize_watched_dir(&dir).unwrap();
fs::remove_dir_all(&dir).unwrap();
fs::create_dir_all(&dir).unwrap(); // same path, different inode
assert!(verify_watched_dir(&watched).is_err());
let _ = fs::remove_dir_all(&dir);
}
#[cfg(windows)]
#[test]
fn a_replaced_directory_at_the_same_windows_path_is_refused() {
// Same pathname, different directory object (volume serial + file
// index): the pathname check alone would pass, the identity must not.
let dir = temp_watch_dir("windows-dir-replace");
let moved = dir.with_extension("moved");
let _ = fs::remove_dir_all(&moved);
let watched = authorize_watched_dir(&dir).unwrap();
fs::rename(&dir, &moved).unwrap();
fs::create_dir_all(&dir).unwrap();
let err = verify_watched_dir(&watched).unwrap_err();
assert!(err.contains("identity"), "unexpected error: {err}");
drop(watched);
let _ = fs::remove_dir_all(&dir);
let _ = fs::remove_dir_all(&moved);
}
#[cfg(unix)]
#[test]
fn symlinks_are_never_followed_out_of_the_folder() {
let dir = temp_watch_dir("symlink");
let secret = std::env::temp_dir().join(format!("vs-secret-{}", std::process::id()));
fs::write(&secret, b"outside the folder").unwrap();
std::os::unix::fs::symlink(&secret, dir.join("evil.mp4")).unwrap();
let meta = fs::metadata(dir.join("evil.mp4")).unwrap();
let handle = open_handle(&dir);
// Even with a "correct" snapshot of the symlink target, opening it
// through the capability handle refuses: resolution may not escape
// the watched folder.
let err =
open_watched_reader(&handle, "evil.mp4", meta.len(), mtime_ms(&meta)).unwrap_err();
assert!(
err.contains("could not be opened"),
"unexpected error: {err}"
);
// And the scanner never lists it in the first place.
assert!(scan_dir(&handle).unwrap().is_empty());
let _ = fs::remove_dir_all(&dir);
let _ = fs::remove_file(&secret);
}
crate::watch_folder_core::forget(token);
}
frontend/src-tauri/src/watch_folder_core.rs
+749
View File
@@ -0,0 +1,749 @@
//! Shared capability-scoped batch watch-folder scan and streaming upload.
//!
//! The watcher lives entirely on the client side of the app: the webview asks
//! this module (over Tauri IPC) for directory listings and asks Rust to stream
//! a settled file through the existing `POST /batch/enqueue` multipart route.
//! The Python backend only ever sees uploaded bytes — filesystem paths never
//! ride an HTTP request (same posture as `commands::authorize_host_path`).
//!
//! Access model: the folder is picked in a native dialog inside this process
//! and registered under a random session token, together with a `cap_std`
//! directory HANDLE opened at pick time. Scan/read commands resolve entries
//! relative to that handle — the pathname is never re-resolved for *access*,
//! so swapping the directory (or any component of its path) for a
//! symlink/junction later cannot redirect the watcher, on any OS. The stored
//! pathname is re-resolved only by the liveness/identity check, which stops
//! the watcher loudly when the folder is deleted, moved, or replaced. The
//! webview cannot point the commands at an arbitrary path; reads are confined
//! to files sitting directly in the folder the user explicitly picked this
//! session (non-recursive by design).
//!
//! Holding the handle must not lock the user's folder: `cap_std` opens
//! directories on Windows WITHOUT `FILE_SHARE_DELETE` (it pins the pathname
//! for its own path-based helpers), which would make Explorer refuse to
//! rename or delete a watched folder until the watch is stopped — a
//! Windows-only behaviour the other two platforms don't have. The handle is
//! therefore opened here with the full share mode (`open_dir_handle`), so
//! replacing the folder behaves identically everywhere: the OS allows it, the
//! next poll's identity check fails, and the UI stops the watcher.
use std::collections::HashMap;
use std::fs;
use std::io::{self, Read};
use std::path::{Path, PathBuf};
use std::sync::{Mutex, OnceLock};
use std::time::UNIX_EPOCH;
#[cfg(not(windows))]
use cap_std::ambient_authority;
use cap_std::fs::Dir;
use serde::Serialize;
/// An authorized watch folder: the directory handle everything resolves
/// against, plus the identity the folder had when the user picked it. The
/// handle is the security boundary (operations can never leave it); the
/// identity check is the LIVENESS signal — when the folder is deleted, moved,
/// or replaced, token resolution fails loudly and the UI stops the watcher
/// instead of polling silently forever.
struct WatchedDir {
/// Fully-resolved directory path captured at pick time.
canonical: PathBuf,
/// Filesystem identity (device, inode) captured at pick time.
#[cfg(unix)]
identity: (u64, u64),
/// Filesystem identity (volume serial, file index) captured at pick time.
#[cfg(windows)]
identity: (u32, u64),
/// Directory handle captured at pick time — all scans/reads go through it.
handle: Dir,
}
#[cfg(unix)]
fn dir_identity(meta: &fs::Metadata) -> (u64, u64) {
use std::os::unix::fs::MetadataExt;
(meta.dev(), meta.ino())
}
#[cfg(windows)]
fn dir_identity(dir: &Dir) -> Result<(u32, u64), String> {
use std::os::windows::io::AsRawHandle;
use windows::Win32::Foundation::HANDLE;
use windows::Win32::Storage::FileSystem::{
GetFileInformationByHandle, BY_HANDLE_FILE_INFORMATION,
};
let mut info = BY_HANDLE_FILE_INFORMATION::default();
// SAFETY: `dir` owns a live directory handle for the duration of this
// call, and `info` is a valid writable output buffer.
unsafe { GetFileInformationByHandle(HANDLE(dir.as_raw_handle()), &mut info) }
.map_err(|_| "Selected watch folder identity could not be read".to_string())?;
Ok((
info.dwVolumeSerialNumber,
((info.nFileIndexHigh as u64) << 32) | info.nFileIndexLow as u64,
))
}
/// Open a directory handle for capability-scoped access.
///
/// Unix: `cap_std`'s own ambient open. Windows: the same
/// `FILE_FLAG_BACKUP_SEMANTICS` directory open `cap_std` performs, but with
/// `FILE_SHARE_DELETE` included so the user can still rename/delete the folder
/// while it is watched (see the module docs). Child opens stay handle-relative
/// (`CreateFileAtW` / `NtCreateFile` with a root directory) so confinement is
/// unaffected; only the liveness check observes the rename, which is the
/// intended signal.
#[cfg(not(windows))]
fn open_dir_handle(dir: &Path) -> io::Result<Dir> {
Dir::open_ambient_dir(dir, ambient_authority())
}
#[cfg(windows)]
fn open_dir_handle(dir: &Path) -> io::Result<Dir> {
use std::os::windows::fs::OpenOptionsExt;
use windows::Win32::Storage::FileSystem::{
FILE_FLAG_BACKUP_SEMANTICS, FILE_SHARE_DELETE, FILE_SHARE_READ, FILE_SHARE_WRITE,
};
let file = fs::OpenOptions::new()
.read(true)
.custom_flags(FILE_FLAG_BACKUP_SEMANTICS.0)
.share_mode((FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE).0)
.open(dir)?;
if !file.metadata()?.is_dir() {
return Err(io::Error::other("not a directory"));
}
Ok(Dir::from_std_file(file))
}
fn authorize_watched_dir(dir: &Path) -> Result<WatchedDir, String> {
let canonical = fs::canonicalize(dir)
.map_err(|e| format!("Selected watch folder could not be resolved: {e}"))?;
if !canonical.is_dir() {
return Err("Selected watch folder is not a directory".into());
}
let handle = open_dir_handle(&canonical)
.map_err(|e| format!("Selected watch folder could not be opened: {e}"))?;
#[cfg(unix)]
let identity = dir_identity(
&fs::metadata(&canonical)
.map_err(|e| format!("Selected watch folder could not be inspected: {e}"))?,
);
#[cfg(windows)]
let identity = dir_identity(&handle)?;
Ok(WatchedDir {
canonical,
#[cfg(unix)]
identity,
#[cfg(windows)]
identity,
handle,
})
}
/// Re-verify a watched folder's identity: the stored path must still resolve
/// to the same canonical target (and, on unix, the same device+inode). A
/// deleted, moved, replaced, or recreated directory fails here, which is what
/// stops the watcher loudly in the UI. Reads never depend on this check for
/// confinement — they go through the pinned handle regardless.
fn verify_watched_dir(watched: &WatchedDir) -> Result<(), String> {
let canonical_now = fs::canonicalize(&watched.canonical)
.map_err(|_| "Watched folder is no longer accessible".to_string())?;
if canonical_now != watched.canonical {
return Err("Watched folder changed identity".into());
}
#[cfg(unix)]
{
let meta = fs::metadata(&canonical_now)
.map_err(|_| "Watched folder is no longer accessible".to_string())?;
if dir_identity(&meta) != watched.identity {
return Err("Watched folder changed identity".into());
}
}
#[cfg(windows)]
{
let current = open_dir_handle(&canonical_now)
.map_err(|_| "Watched folder is no longer accessible".to_string())?;
if dir_identity(&current)? != watched.identity {
return Err("Watched folder changed identity".into());
}
}
Ok(())
}
fn registry() -> &'static Mutex<HashMap<String, WatchedDir>> {
static WATCHED: OnceLock<Mutex<HashMap<String, WatchedDir>>> = OnceLock::new();
WATCHED.get_or_init(|| Mutex::new(HashMap::new()))
}
#[derive(Serialize)]
pub struct WatchFolderSelection {
token: String,
path: String,
}
#[derive(Serialize)]
pub struct WatchEntry {
name: String,
size: u64,
/// Modification time in ms since the Unix epoch (0 when unavailable).
mtime: u64,
}
fn new_token() -> Result<String, String> {
let mut random = [0_u8; 32];
getrandom::fill(&mut random).map_err(|e| format!("Secure randomness unavailable: {e}"))?;
Ok(random.iter().map(|b| format!("{b:02x}")).collect())
}
/// Resolve a session token to a clone of its pinned directory handle,
/// re-verifying the folder's liveness/identity on every access.
fn registered_dir(token: &str) -> Result<Dir, String> {
let map = registry()
.lock()
.map_err(|_| "Watch-folder registry poisoned".to_string())?;
let watched = map
.get(token)
.ok_or_else(|| "Watch folder is not authorized".to_string())?;
verify_watched_dir(watched)?;
watched
.handle
.try_clone()
.map_err(|e| format!("Watched folder handle could not be reused: {e}"))
}
/// A directory entry name must be a single plain path component — anything
/// that could climb out of the watched folder is rejected. (The `cap_std`
/// handle would also refuse an escape; this keeps the error crisp and the
/// contract explicit.)
fn validate_entry_name(name: &str) -> Result<(), String> {
if name.is_empty()
|| name == "."
|| name == ".."
|| name.contains('/')
|| name.contains('\\')
|| name.chars().any(|c| c.is_control())
{
return Err("Invalid watch-folder entry name".into());
}
Ok(())
}
fn mtime_ms(meta: &fs::Metadata) -> u64 {
meta.modified()
.ok()
.and_then(|t| t.duration_since(UNIX_EPOCH).ok())
.map(|d| d.as_millis() as u64)
.unwrap_or(0)
}
fn cap_mtime_ms(meta: &cap_std::fs::Metadata) -> u64 {
meta.modified()
.ok()
.and_then(|t| t.into_std().duration_since(UNIX_EPOCH).ok())
.map(|d| d.as_millis() as u64)
.unwrap_or(0)
}
/// Non-recursive listing of the regular files in the watched folder (name,
/// size, mtime), resolved through the pinned handle. Symlinks are skipped
/// outright — the read path cannot follow them out of the folder anyway, so
/// listing them would only produce entries that can never be ingested.
fn scan_dir(dir: &Dir) -> Result<Vec<WatchEntry>, String> {
let mut entries = Vec::new();
let read = dir
.entries()
.map_err(|e| format!("Watched folder is unreadable: {e}"))?;
for item in read.flatten() {
let Ok(file_type) = item.file_type() else {
continue;
};
if !file_type.is_file() {
continue;
}
let Ok(meta) = item.metadata() else { continue };
let Ok(name) = item.file_name().into_string() else {
continue; // non-UTF-8 names can't round-trip through IPC; skip
};
entries.push(WatchEntry {
name,
size: meta.len(),
mtime: cap_mtime_ms(&meta),
});
}
Ok(entries)
}
/// Open a settled watched file through the pinned directory handle. A symlink
/// outside the folder cannot be opened, and the returned reader revalidates
/// size+mtime around every network read so a mutation aborts the upload.
fn open_watched_reader(
dir: &Dir,
name: &str,
expected_size: u64,
expected_mtime: u64,
) -> Result<SnapshotReader, String> {
validate_entry_name(name)?;
let file = dir
.open(name)
.map_err(|e| format!("Watched file could not be opened: {e}"))?
.into_std();
let meta = file
.metadata()
.map_err(|e| format!("Watched file could not be inspected: {e}"))?;
if !meta.is_file() {
return Err("Watched entry is not a regular file".into());
}
if meta.len() != expected_size || mtime_ms(&meta) != expected_mtime {
return Err("Watched file changed after it was scanned".into());
}
Ok(SnapshotReader {
file,
expected_size,
expected_mtime,
})
}
#[derive(Debug)]
struct SnapshotReader {
file: fs::File,
expected_size: u64,
expected_mtime: u64,
}
impl SnapshotReader {
fn validate(&self) -> io::Result<()> {
let meta = self.file.metadata()?;
if !meta.is_file()
|| meta.len() != self.expected_size
|| mtime_ms(&meta) != self.expected_mtime
{
return Err(io::Error::other("watched file changed during upload"));
}
Ok(())
}
}
impl Read for SnapshotReader {
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
self.validate()?;
let read = self.file.read(buf)?;
self.validate()?;
Ok(read)
}
}
/// Register a folder selected by the owning desktop shell's native dialog.
/// This function is not exposed to renderer IPC with an arbitrary path.
pub fn register(dir: &Path) -> Result<WatchFolderSelection, String> {
if !dir.is_absolute() || !dir.is_dir() {
return Err("Selected watch folder is not a directory".into());
}
let watched = authorize_watched_dir(dir)?;
let display = watched.canonical.to_string_lossy().into_owned();
let token = new_token()?;
registry()
.lock()
.map_err(|_| "Watch-folder registry poisoned".to_string())?
.insert(token.clone(), watched);
Ok(WatchFolderSelection {
token,
path: display,
})
}
/// List the files currently sitting in the watched folder (non-recursive).
pub fn scan(token: String) -> Result<Vec<WatchEntry>, String> {
scan_dir(&registered_dir(&token)?)
}
#[derive(Serialize)]
pub struct WatchFolderUploadReply {
status: u16,
body: serde_json::Value,
}
fn video_mime(name: &str) -> &'static str {
match Path::new(name)
.extension()
.and_then(|ext| ext.to_str())
.unwrap_or_default()
.to_ascii_lowercase()
.as_str()
{
"mp4" | "m4v" => "video/mp4",
"mov" => "video/quicktime",
"mkv" => "video/x-matroska",
"webm" => "video/webm",
"avi" => "video/x-msvideo",
"mpg" | "mpeg" => "video/mpeg",
"wmv" => "video/x-ms-wmv",
_ => "application/octet-stream",
}
}
fn batch_endpoint(backend_url: &str) -> Result<reqwest::Url, String> {
let mut url = reqwest::Url::parse(backend_url)
.map_err(|_| "Invalid watch-folder backend URL".to_string())?;
if !matches!(url.scheme(), "http" | "https")
|| url.host_str().is_none()
|| !url.username().is_empty()
|| url.password().is_some()
|| url.query().is_some()
|| url.fragment().is_some()
|| !matches!(url.path(), "" | "/")
{
return Err("Invalid watch-folder backend URL".into());
}
url.set_path("/batch/enqueue");
Ok(url)
}
fn validate_upload_authorization(url: &reqwest::Url, authorization: Option<&str>) -> Result<(), String> {
let host = url.host_str().unwrap_or("").trim_start_matches('[').trim_end_matches(']');
let loopback = host.eq_ignore_ascii_case("localhost")
|| host.parse::<std::net::IpAddr>().is_ok_and(|ip| ip.is_loopback());
if url.scheme() == "http" && !loopback && authorization.is_some_and(|v| !v.is_empty()) {
return Err("Credentialed watch-folder uploads require HTTPS outside loopback".into());
}
Ok(())
}
/// Stream one settled watched file directly from its pinned OS handle to the
/// selected backend. Keeping bytes out of WebView IPC avoids an O(file size)
/// renderer allocation for multi-gigabyte videos.
pub fn enqueue_to(
backend_url: String,
authorization: Option<String>,
token: String,
name: String,
expected_size: u64,
expected_mtime: u64,
langs: Vec<String>,
voice_id: Option<String>,
preserve_bg: bool,
) -> Result<WatchFolderUploadReply, String> {
let dir = registered_dir(&token)?;
let reader = open_watched_reader(&dir, &name, expected_size, expected_mtime)?;
let mime = video_mime(&name);
let url = batch_endpoint(&backend_url)?;
validate_upload_authorization(&url, authorization.as_deref())?;
let part = reqwest::blocking::multipart::Part::reader_with_length(reader, expected_size)
.file_name(name)
.mime_str(mime)
.map_err(|_| "Watched file type could not be prepared".to_string())?;
let mut form = reqwest::blocking::multipart::Form::new()
.part("video", part)
.text("langs", langs.join(","))
.text("preserve_bg", preserve_bg.to_string());
if let Some(voice_id) = voice_id.filter(|value| !value.is_empty()) {
form = form.text("voice_id", voice_id);
}
let client = reqwest::blocking::Client::builder()
.no_proxy()
.connect_timeout(std::time::Duration::from_secs(5))
.build()
.map_err(|_| "Watch-folder upload client could not start".to_string())?;
let mut request = client.post(url).multipart(form);
if let Some(value) = authorization.filter(|value| !value.is_empty()) {
request = request.header(reqwest::header::AUTHORIZATION, value);
}
let response = request
.send()
.map_err(|_| "Watch-folder upload failed".to_string())?;
let status = response.status().as_u16();
let body = response
.json::<serde_json::Value>()
.map_err(|_| "Watch-folder backend returned an invalid response".to_string())?;
Ok(WatchFolderUploadReply { status, body })
}
#[allow(dead_code)] // Used by the Tauri adapter; the Electron helper calls enqueue_to.
pub fn enqueue(
backend_port: u16,
token: String,
name: String,
expected_size: u64,
expected_mtime: u64,
langs: Vec<String>,
voice_id: Option<String>,
preserve_bg: bool,
) -> Result<WatchFolderUploadReply, String> {
enqueue_to(
format!("http://127.0.0.1:{backend_port}"),
None,
token,
name,
expected_size,
expected_mtime,
langs,
voice_id,
preserve_bg,
)
}
/// Drop a watch-folder authorization (watcher stopped or component unmounted).
pub fn forget(token: String) {
if let Ok(mut map) = registry().lock() {
map.remove(&token);
}
}
#[cfg(test)]
mod tests {
use super::{
authorize_watched_dir, batch_endpoint, mtime_ms, open_dir_handle, open_watched_reader,
scan_dir, validate_entry_name, verify_watched_dir, Dir,
};
use std::fs;
use std::io::Read;
use std::path::PathBuf;
fn temp_watch_dir(tag: &str) -> PathBuf {
let dir = std::env::temp_dir().join(format!("vs-watch-{tag}-{}", std::process::id()));
let _ = fs::remove_dir_all(&dir);
fs::create_dir_all(&dir).unwrap();
dir
}
fn open_handle(dir: &std::path::Path) -> Dir {
open_dir_handle(dir).unwrap()
}
fn snapshot(path: &std::path::Path) -> (u64, u64) {
let meta = fs::metadata(path).unwrap();
(meta.len(), mtime_ms(&meta))
}
#[test]
fn entry_names_must_be_single_components() {
assert!(validate_entry_name("clip.mp4").is_ok());
assert!(validate_entry_name("weird name (1).MOV").is_ok());
for bad in [
"",
".",
"..",
"a/b.mp4",
"a\\b.mp4",
"..\\up.mp4",
"x\n.mp4",
] {
assert!(validate_entry_name(bad).is_err(), "accepted {bad:?}");
}
}
#[test]
fn backend_endpoint_accepts_remote_https_without_credential_injection() {
assert_eq!(
batch_endpoint("https://gpu-box.example:3900")
.unwrap()
.as_str(),
"https://gpu-box.example:3900/batch/enqueue"
);
for bad in [
"file:///tmp/backend",
"https://user:secret@gpu-box.example:3900",
"https://gpu-box.example:3900/other",
"https://gpu-box.example:3900?token=secret",
] {
assert!(batch_endpoint(bad).is_err(), "accepted {bad:?}");
}
}
#[test]
fn scan_lists_regular_files_with_size_and_mtime_and_skips_dirs() {
let dir = temp_watch_dir("scan");
fs::create_dir_all(dir.join("nested")).unwrap();
fs::write(dir.join("a.mp4"), b"12345").unwrap();
fs::write(dir.join("notes.txt"), b"x").unwrap();
let mut entries = scan_dir(&open_handle(&dir)).unwrap();
entries.sort_by(|a, b| a.name.cmp(&b.name));
let names: Vec<&str> = entries.iter().map(|e| e.name.as_str()).collect();
// Directories are skipped; filtering to *videos* is the frontend's job.
assert_eq!(names, ["a.mp4", "notes.txt"]);
assert_eq!(entries[0].size, 5);
assert!(entries[0].mtime > 0);
let _ = fs::remove_dir_all(&dir);
}
#[test]
fn snapshot_reader_streams_the_exact_bytes() {
let dir = temp_watch_dir("stream");
fs::write(dir.join("clip.mp4"), b"0123456789").unwrap();
let (size, mtime) = snapshot(&dir.join("clip.mp4"));
let handle = open_handle(&dir);
let mut reader = open_watched_reader(&handle, "clip.mp4", size, mtime).unwrap();
let mut whole = Vec::new();
reader.read_to_end(&mut whole).unwrap();
assert_eq!(whole, b"0123456789");
let _ = fs::remove_dir_all(&dir);
}
#[test]
fn reads_are_bound_to_the_settled_snapshot() {
let dir = temp_watch_dir("snapshot");
fs::write(dir.join("clip.mp4"), b"settled bytes").unwrap();
let (size, mtime) = snapshot(&dir.join("clip.mp4"));
let handle = open_handle(&dir);
// The file is replaced after the scan settled → the read must refuse
// rather than upload bytes the tracker never saw stabilize.
fs::write(dir.join("clip.mp4"), b"replaced with something longer").unwrap();
let err = open_watched_reader(&handle, "clip.mp4", size, mtime).unwrap_err();
assert!(err.contains("changed"), "unexpected error: {err}");
let _ = fs::remove_dir_all(&dir);
}
#[test]
fn snapshot_reader_aborts_when_file_changes_during_stream() {
let dir = temp_watch_dir("mid-stream-change");
fs::write(dir.join("clip.mp4"), b"settled bytes").unwrap();
let (size, mtime) = snapshot(&dir.join("clip.mp4"));
let handle = open_handle(&dir);
let mut reader = open_watched_reader(&handle, "clip.mp4", size, mtime).unwrap();
fs::write(dir.join("clip.mp4"), b"different-length bytes").unwrap();
let mut byte = [0_u8; 1];
assert!(reader.read(&mut byte).is_err());
let _ = fs::remove_dir_all(&dir);
}
#[test]
fn authorization_pins_the_directory_identity() {
let dir = temp_watch_dir("identity");
let watched = authorize_watched_dir(&dir).unwrap();
// Untouched directory verifies fine…
assert!(verify_watched_dir(&watched).is_ok());
// …and a directory that disappears after authorization is refused.
// The removal itself must succeed WHILE the handle is held: a watch
// that locked the user's folder against deletion (Windows sharing
// violation, OS error 32) would be a Windows-only behaviour.
fs::remove_dir_all(&dir).unwrap();
assert!(verify_watched_dir(&watched).is_err());
}
#[test]
fn a_watched_folder_can_be_renamed_by_the_user_while_watched() {
// Cross-platform contract: holding the pinned handle never blocks the
// user from moving the folder (Explorer/Finder/mv). The liveness
// check is what notices — it must refuse, not the OS.
let dir = temp_watch_dir("rename-while-watched");
let moved = dir.with_extension("moved");
let _ = fs::remove_dir_all(&moved);
let watched = authorize_watched_dir(&dir).unwrap();
fs::rename(&dir, &moved).unwrap();
assert!(verify_watched_dir(&watched).is_err());
// The pinned handle still points at the ORIGINAL directory object.
fs::write(moved.join("clip.mp4"), b"x").unwrap();
let names: Vec<String> = scan_dir(&watched.handle)
.unwrap()
.into_iter()
.map(|e| e.name)
.collect();
assert_eq!(names, ["clip.mp4"]);
drop(watched);
let _ = fs::remove_dir_all(&moved);
}
#[cfg(unix)]
#[test]
fn a_directory_swapped_for_a_symlink_is_refused_and_never_followed() {
let dir = temp_watch_dir("dir-swap");
let elsewhere = temp_watch_dir("dir-swap-target");
fs::write(elsewhere.join("clip.mp4"), b"outside").unwrap();
let (size, mtime) = snapshot(&elsewhere.join("clip.mp4"));
let watched = authorize_watched_dir(&dir).unwrap();
assert!(verify_watched_dir(&watched).is_ok());
// Replace the authorized directory itself with a symlink pointing
// somewhere else. Token resolution refuses (identity check)…
fs::remove_dir_all(&dir).unwrap();
std::os::unix::fs::symlink(&elsewhere, &dir).unwrap();
let err = verify_watched_dir(&watched).unwrap_err();
assert!(err.contains("identity"), "unexpected error: {err}");
// …and even the pinned handle cannot reach the swap target: it still
// points at the ORIGINAL (now unlinked) directory, which is empty.
assert!(scan_dir(&watched.handle).unwrap().is_empty());
assert!(open_watched_reader(&watched.handle, "clip.mp4", size, mtime).is_err());
let _ = fs::remove_file(&dir);
let _ = fs::remove_dir_all(&elsewhere);
}
#[cfg(unix)]
#[test]
fn a_recreated_directory_at_the_same_path_is_refused() {
let dir = temp_watch_dir("dir-recreate");
let watched = authorize_watched_dir(&dir).unwrap();
fs::remove_dir_all(&dir).unwrap();
fs::create_dir_all(&dir).unwrap(); // same path, different inode
assert!(verify_watched_dir(&watched).is_err());
let _ = fs::remove_dir_all(&dir);
}
#[cfg(windows)]
#[test]
fn a_replaced_directory_at_the_same_windows_path_is_refused() {
// Same pathname, different directory object (volume serial + file
// index): the pathname check alone would pass, the identity must not.
let dir = temp_watch_dir("windows-dir-replace");
let moved = dir.with_extension("moved");
let _ = fs::remove_dir_all(&moved);
let watched = authorize_watched_dir(&dir).unwrap();
fs::rename(&dir, &moved).unwrap();
fs::create_dir_all(&dir).unwrap();
let err = verify_watched_dir(&watched).unwrap_err();
assert!(err.contains("identity"), "unexpected error: {err}");
drop(watched);
let _ = fs::remove_dir_all(&dir);
let _ = fs::remove_dir_all(&moved);
}
#[cfg(unix)]
#[test]
fn symlinks_are_never_followed_out_of_the_folder() {
let dir = temp_watch_dir("symlink");
let secret = std::env::temp_dir().join(format!("vs-secret-{}", std::process::id()));
fs::write(&secret, b"outside the folder").unwrap();
std::os::unix::fs::symlink(&secret, dir.join("evil.mp4")).unwrap();
let meta = fs::metadata(dir.join("evil.mp4")).unwrap();
let handle = open_handle(&dir);
// Even with a "correct" snapshot of the symlink target, opening it
// through the capability handle refuses: resolution may not escape
// the watched folder.
let err =
open_watched_reader(&handle, "evil.mp4", meta.len(), mtime_ms(&meta)).unwrap_err();
assert!(
err.contains("could not be opened"),
"unexpected error: {err}"
);
// And the scanner never lists it in the first place.
assert!(scan_dir(&handle).unwrap().is_empty());
let _ = fs::remove_dir_all(&dir);
let _ = fs::remove_file(&secret);
}
}
#[cfg(test)]
mod transport_tests {
#[test]
fn credentials_require_tls_except_loopback() {
for base in ["http://127.0.0.1:3900", "http://[::1]:3900", "http://localhost:3900", "https://gpu.example"] {
assert!(super::validate_upload_authorization(&super::batch_endpoint(base).unwrap(), Some("Bearer secret")).is_ok());
}
for base in ["http://gpu.example", "http://192.168.1.2:3900", "http://localhost.evil.test"] {
let url = super::batch_endpoint(base).unwrap();
assert!(super::validate_upload_authorization(&url, Some("Bearer secret")).is_err());
assert!(super::validate_upload_authorization(&url, None).is_ok());
}
}
}
frontend/src-tauri/src/wayland_shortcut.rs
+18 -785
View File
@@ -1,645 +1,35 @@
//! Wayland global shortcut support through xdg-desktop-portal.
//!
//! `tauri-plugin-global-shortcut` uses `global-hotkey`, whose Linux backend is
//! X11-only. Under XWayland its registration can still return `Ok(())`, but a
//! native Wayland compositor never sends it key events. The portal is the
//! compositor-owned, permission-aware API for this job.
use std::collections::HashMap;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{mpsc, Mutex};
use std::time::Duration;
//! Tauri adapter for the shared compositor-owned shortcut implementation.
pub use crate::wayland_shortcut_core::is_wayland_session;
use std::sync::Arc;
use tauri::Manager;
use zbus::{
blocking::{Connection, Proxy},
zvariant::{OwnedObjectPath, OwnedValue, Str},
};
const DESKTOP_DESTINATION: &str = "org.freedesktop.portal.Desktop";
const DESKTOP_PATH: &str = "/org/freedesktop/portal/desktop";
const GLOBAL_SHORTCUTS_INTERFACE: &str = "org.freedesktop.portal.GlobalShortcuts";
const REQUEST_INTERFACE: &str = "org.freedesktop.portal.Request";
const SESSION_INTERFACE: &str = "org.freedesktop.portal.Session";
const REGISTRY_INTERFACE: &str = "org.freedesktop.host.portal.Registry";
const SHORTCUT_ID: &str = "voice-dictation";
static REQUEST_SEQUENCE: AtomicU64 = AtomicU64::new(1);
const PORTAL_LISTENER_TIMEOUT: Duration = Duration::from_secs(5);
const PORTAL_RESPONSE_TIMEOUT: Duration = Duration::from_secs(60);
type VariantMap = HashMap<String, OwnedValue>;
#[derive(Clone)]
struct PortalRegistration {
connection: Connection,
session: OwnedObjectPath,
}
#[derive(Default)]
pub struct PortalShortcutState {
active: Mutex<Option<PortalRegistration>>,
revision: AtomicU64,
}
pub struct PortalShortcutState(crate::wayland_shortcut_core::PortalShortcutState);
impl PortalShortcutState {
pub fn replace(&self, app: tauri::AppHandle, accelerator: String) -> Result<String, String> {
let revision = self.reserve();
self.replace_reserved(app, accelerator, revision)
}
pub fn reserve(&self) -> u64 {
self.revision.fetch_add(1, Ordering::SeqCst) + 1
self.0.reserve()
}
fn is_current(&self, revision: u64) -> bool {
self.revision.load(Ordering::SeqCst) == revision
pub fn replace(&self, app: tauri::AppHandle, accelerator: String) -> Result<String, String> {
self.0.replace(
Arc::new(move |pressed| {
crate::dispatch_dictation_capture(&app, if pressed { "start" } else { "stop" })
}),
accelerator,
)
}
pub fn replace_reserved(
&self,
app: tauri::AppHandle,
accelerator: String,
revision: u64,
) -> Result<String, String> {
// Bind the replacement first. A declined consent dialog or unavailable
// portal therefore leaves the working shortcut and saved preference
// untouched.
let (registration, display) = bind(&accelerator)?;
if !self.is_current(revision) {
let _ = close_session(&registration);
return Err("shortcut registration was superseded by a newer request".into());
}
let mut active = match self.active.lock() {
Ok(active) => active,
Err(_) => {
let _ = close_session(&registration);
return Err("portal shortcut lock poisoned".into());
}
};
if !self.is_current(revision) {
drop(active);
let _ = close_session(&registration);
return Err("shortcut registration was superseded by a newer request".into());
}
if let Err(error) = start_listener(app, registration.clone()) {
let _ = close_session(&registration);
return Err(error);
}
let previous = active.replace(registration);
drop(active);
if let Some(previous) = previous {
if let Err(error) = close_session(&previous) {
log::warn!("Could not close the previous Wayland shortcut session: {error}");
}
}
Ok(display)
}
}
const DESKTOP_ID: &str = "com.debpalash.omnivoice-studio";
fn user_entry_path() -> Option<std::path::PathBuf> {
dirs_next::data_dir().map(|dir| {
dir.join("applications")
.join(format!("{DESKTOP_ID}.desktop"))
})
}
/// A packaged (system-dir) entry — deb installs manage their own; never touch.
fn system_entry_exists() -> bool {
let filename = format!("{DESKTOP_ID}.desktop");
std::env::var_os("XDG_DATA_DIRS")
.map(|dirs| {
std::env::split_paths(&dirs)
.any(|dir| dir.join("applications").join(&filename).is_file())
})
.unwrap_or_else(|| {
["/usr/local/share", "/usr/share"].iter().any(|dir| {
std::path::Path::new(dir)
.join("applications")
.join(&filename)
.is_file()
})
})
}
/// The `[Desktop Entry]` group's Exec target, unquoted. `None` when the main
/// group has no usable Exec line — which GLib treats the same as a missing
/// program. Scoped to the main group deliberately: a `[Desktop Action …]`
/// group carries its own `Exec=`, and accepting it would retain an entry GLib
/// still cannot resolve (CodeRabbit, #1526).
fn entry_exec_target(content: &str) -> Option<std::path::PathBuf> {
let mut in_main_group = false;
let mut exec = None;
for line in content.lines() {
let line = line.trim_start();
if line.starts_with('[') {
in_main_group = line == "[Desktop Entry]";
continue;
}
if in_main_group {
if let Some(value) = line.strip_prefix("Exec=") {
exec = Some(value);
break;
}
}
}
let raw = exec?.trim();
let unquoted = raw
.strip_prefix('"')
.and_then(|rest| rest.split('"').next())
.unwrap_or_else(|| raw.split_whitespace().next().unwrap_or(raw));
if unquoted.is_empty() {
return None;
}
Some(std::path::PathBuf::from(unquoted))
}
/// Whether a user-local identity entry must be rewritten before the portal
/// will accept it.
///
/// GLib refuses to resolve a desktop entry whose Exec program does not exist
/// (`GDesktopAppInfo` returns NULL), and the portal then rejects the bind with
/// "App info not found" — the shortcut silently dies for the whole session.
/// A dev entry pointing at a `target/debug` binary goes stale exactly this
/// way: a `cargo clean`, a moved checkout, or anything that relocates the
/// binary breaks system-wide dictation with only a log line to show for it.
fn entry_needs_rewrite(content: &str, exec_exists: impl Fn(&std::path::Path) -> bool) -> bool {
match entry_exec_target(content) {
Some(target) => !exec_exists(&target),
None => true,
}
}
fn desktop_exec_path() -> Result<std::path::PathBuf, String> {
// AppImage's current_exe() points inside its transient mount. APPIMAGE is
// the stable launcher path the desktop entry must retain.
if let Some(path) = std::env::var_os("APPIMAGE").filter(|path| !path.is_empty()) {
return Ok(path.into());
}
std::env::current_exe().map_err(|error| format!("could not locate VoiceStudio: {error}"))
}
fn desktop_exec_value(path: &std::path::Path) -> String {
let escaped = path
.to_string_lossy()
.replace('\\', "\\\\")
.replace('"', "\\\"")
.replace('`', "\\`")
.replace('$', "\\$");
format!("\"{escaped}\"")
}
/// The host portal resolves un-sandboxed apps through their desktop entry.
/// Deb packages already install one; dev builds and standalone AppImages may
/// not. Add an invisible identity entry only when none exists.
fn ensure_desktop_identity() -> Result<(), String> {
if system_entry_exists() {
return Ok(());
}
let path = user_entry_path().ok_or("could not locate the user data directory")?;
if let Ok(existing) = std::fs::read_to_string(&path) {
if !entry_needs_rewrite(&existing, |target| target.exists()) {
return Ok(());
}
// Stale: GLib returns NULL for an entry whose Exec is gone, and the
// portal then refuses the bind ("App info not found"). Rewrite with
// where the app actually is NOW. The user dir with our app id is ours
// to manage — packaged entries live in the system dirs handled above.
log::info!(
"Wayland portal identity at {} points at a missing program — rewriting",
path.display()
);
}
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)
.map_err(|error| format!("could not create applications directory: {error}"))?;
}
let entry = format!(
"[Desktop Entry]\nType=Application\nName=VoiceStudio\nExec={}\nTerminal=false\nNoDisplay=true\nStartupWMClass=VoiceStudio\nX-VoiceStudio-Generated=true\n",
desktop_exec_value(&desktop_exec_path()?)
);
std::fs::write(&path, entry)
.map_err(|error| format!("could not create {}: {error}", path.display()))?;
log::info!("Installed Wayland portal identity at {}", path.display());
Ok(())
}
pub fn is_wayland_session() -> bool {
std::env::var("XDG_SESSION_TYPE")
.map(|kind| kind.eq_ignore_ascii_case("wayland"))
.unwrap_or(false)
|| std::env::var_os("WAYLAND_DISPLAY").is_some()
}
/// Convert Tauri's cross-platform accelerator spelling to the portal format.
/// The portal may still let the user choose a different chord in its consent
/// dialog, so an unknown spelling is deliberately omitted rather than guessed.
fn portal_trigger(accelerator: &str) -> Option<String> {
let mut modifiers: Vec<String> = Vec::new();
let mut key = None;
for part in accelerator
.split('+')
.map(str::trim)
.filter(|part| !part.is_empty())
{
match part.to_ascii_lowercase().as_str() {
"cmdorctrl" | "commandorcontrol" | "ctrl" | "control" => {
if !modifiers.iter().any(|modifier| modifier == "CTRL") {
modifiers.push("CTRL".into());
}
}
"shift" => modifiers.push("SHIFT".into()),
"alt" | "option" => modifiers.push("ALT".into()),
"cmd" | "command" | "super" | "meta" => modifiers.push("LOGO".into()),
_ if key.is_none() => key = xkb_key_name(part),
_ => return None,
}
}
let key = key?;
if modifiers.is_empty() {
return None;
}
modifiers.push(key);
Some(modifiers.join("+"))
}
fn xkb_key_name(key: &str) -> Option<String> {
let lower = key.to_ascii_lowercase();
if let Some(letter) = lower.strip_prefix("key") {
if letter.len() == 1
&& letter
.chars()
.all(|character| character.is_ascii_alphabetic())
{
return Some(letter.to_owned());
}
}
if let Some(digit) = lower.strip_prefix("digit") {
if digit.len() == 1 && digit.chars().all(|character| character.is_ascii_digit()) {
return Some(digit.to_owned());
}
}
if lower.len() == 1
&& lower
.chars()
.all(|character| character.is_ascii_alphanumeric())
{
return Some(lower);
}
Some(
match lower.as_str() {
"space" => "space",
"enter" | "return" => "Return",
"escape" | "esc" => "Escape",
"tab" => "Tab",
"backspace" => "BackSpace",
"delete" => "Delete",
"insert" => "Insert",
"home" => "Home",
"end" => "End",
"pageup" => "Page_Up",
"pagedown" => "Page_Down",
"arrowup" | "up" => "Up",
"arrowdown" | "down" => "Down",
"arrowleft" | "left" => "Left",
"arrowright" | "right" => "Right",
"minus" => "minus",
"equal" => "equal",
"comma" => "comma",
"period" => "period",
"slash" => "slash",
"semicolon" => "semicolon",
"quote" | "apostrophe" => "apostrophe",
"bracketleft" => "bracketleft",
"bracketright" => "bracketright",
"backslash" => "backslash",
"backquote" | "grave" => "grave",
_ if lower.strip_prefix('f').is_some_and(|digits| {
digits
.parse::<u8>()
.is_ok_and(|number| (1..=35).contains(&number))
}) =>
{
return Some(key.to_ascii_uppercase());
}
_ => return None,
}
.into(),
)
}
fn variant_string(value: &str) -> OwnedValue {
OwnedValue::from(Str::from(value))
}
fn trigger_description(shortcuts: Vec<(String, VariantMap)>) -> Option<String> {
shortcuts
.into_iter()
.find(|(id, _)| id == SHORTCUT_ID)
.and_then(|(_, mut properties)| properties.remove("trigger_description"))
.and_then(|value| String::try_from(value).ok())
.filter(|description| !description.trim().is_empty())
}
fn request_path(connection: &Connection, token: &str) -> Result<OwnedObjectPath, String> {
let sender = connection
.unique_name()
.ok_or("session bus did not assign a unique name")?
.as_str()
.trim_start_matches(':')
.replace('.', "_");
OwnedObjectPath::try_from(format!("{DESKTOP_PATH}/request/{sender}/{token}"))
.map_err(|error| format!("invalid portal request path: {error}"))
}
fn response_for<F>(connection: &Connection, token: &str, call: F) -> Result<VariantMap, String>
where
F: FnOnce() -> Result<OwnedObjectPath, zbus::Error>,
{
// Subscribe before making the request: a fast portal is allowed to answer
// immediately after returning the request handle.
let expected = request_path(connection, token)?;
let listener_connection = connection.clone();
let listener_path = expected.clone();
let (ready_tx, ready_rx) = mpsc::sync_channel(1);
let (response_tx, response_rx) = mpsc::sync_channel(1);
std::thread::Builder::new()
.name("wayland-portal-response".into())
.spawn(move || {
let request = match Proxy::new(
&listener_connection,
DESKTOP_DESTINATION,
listener_path.as_str(),
REQUEST_INTERFACE,
) {
Ok(request) => request,
Err(error) => {
let _ = ready_tx.send(Err(format!("portal request listener: {error}")));
return;
}
};
let mut responses = match request.receive_signal("Response") {
Ok(responses) => responses,
Err(error) => {
let _ = ready_tx.send(Err(format!("portal response listener: {error}")));
return;
}
};
if ready_tx.send(Ok(())).is_err() {
return;
}
let response = responses
.next()
.ok_or_else(|| "portal closed before answering the shortcut request".to_string());
let _ = response_tx.send(response);
})
.map_err(|error| format!("could not start the portal response listener: {error}"))?;
receive_with_timeout(
&ready_rx,
PORTAL_LISTENER_TIMEOUT,
"portal response listener",
)?;
let returned = call().map_err(|error| format!("portal request failed: {error}"))?;
if returned != expected {
return Err(format!(
"portal returned unexpected request path {returned} (expected {expected})"
));
}
let message = match receive_with_timeout(
&response_rx,
PORTAL_RESPONSE_TIMEOUT,
"portal shortcut request",
) {
Ok(message) => message,
Err(error) => {
if let Ok(request) = Proxy::new(
connection,
DESKTOP_DESTINATION,
expected.as_str(),
REQUEST_INTERFACE,
) {
let _ = request.call::<_, _, ()>("Close", &());
}
return Err(error);
}
};
let (code, results): (u32, VariantMap) = message
.body()
.deserialize()
.map_err(|error| format!("invalid portal response: {error}"))?;
if code != 0 {
return Err(format!(
"portal shortcut request was declined (response {code})"
));
}
Ok(results)
}
fn receive_with_timeout<T>(
receiver: &mpsc::Receiver<Result<T, String>>,
timeout: Duration,
operation: &str,
) -> Result<T, String> {
match receiver.recv_timeout(timeout) {
Ok(result) => result,
Err(mpsc::RecvTimeoutError::Timeout) => Err(format!(
"{operation} timed out after {} seconds",
timeout.as_secs()
)),
Err(mpsc::RecvTimeoutError::Disconnected) => {
Err(format!("{operation} stopped before completing"))
}
}
}
fn bind(accelerator: &str) -> Result<(PortalRegistration, String), String> {
ensure_desktop_identity()?;
let connection = Connection::session()
.map_err(|error| format!("could not connect to the desktop portal: {error}"))?;
// GNOME's host portal uses the installed desktop entry to associate this
// un-sandboxed process with its desktop id.
let registry = Proxy::new(
&connection,
DESKTOP_DESTINATION,
DESKTOP_PATH,
REGISTRY_INTERFACE,
)
.map_err(|error| format!("could not open the portal registry: {error}"))?;
let registry_options: VariantMap = HashMap::new();
if let Err(error) = registry.call::<_, _, ()>("Register", &(DESKTOP_ID, registry_options)) {
// Development builds and portable AppImages may not have a desktop
// entry for the host registry to resolve. Portal v1 does not require
// this handshake, so continue and let CreateSession be authoritative.
log::warn!("Wayland portal host registration skipped: {error}");
}
let portal = Proxy::new(
&connection,
DESKTOP_DESTINATION,
DESKTOP_PATH,
GLOBAL_SHORTCUTS_INTERFACE,
)
.map_err(|error| format!("could not open the global-shortcuts portal: {error}"))?;
let process = std::process::id();
let sequence = REQUEST_SEQUENCE.fetch_add(1, Ordering::Relaxed);
let create_token = format!("vs_create_{process}_{sequence}");
let session_token = format!("vs_session_{process}_{sequence}");
let mut create_options = VariantMap::new();
create_options.insert("handle_token".into(), variant_string(&create_token));
create_options.insert(
"session_handle_token".into(),
variant_string(&session_token),
);
let mut create_results = response_for(&connection, &create_token, || {
portal.call("CreateSession", &(create_options,))
})?;
let session_value = create_results
.remove("session_handle")
.ok_or("portal did not return a shortcut session")?;
// The portal specification declares an object path, but deployed portal
// versions historically returned a string. Accept both wire formats.
let session = match session_value
.try_clone()
.ok()
.and_then(|value| OwnedObjectPath::try_from(value).ok())
{
Some(path) => path,
None => {
let path = String::try_from(session_value).map_err(|error| {
format!("portal returned an invalid shortcut session handle: {error}")
})?;
OwnedObjectPath::try_from(path)
.map_err(|error| format!("portal returned an invalid session path: {error}"))?
}
};
let mut shortcut_info = VariantMap::new();
shortcut_info.insert("description".into(), variant_string("VoiceStudio"));
if let Some(trigger) = portal_trigger(&accelerator) {
shortcut_info.insert("preferred_trigger".into(), variant_string(&trigger));
}
let shortcuts = vec![(SHORTCUT_ID.to_string(), shortcut_info)];
let bind_token = format!("vs_bind_{process}_{sequence}");
let mut bind_options = VariantMap::new();
bind_options.insert("handle_token".into(), variant_string(&bind_token));
let mut bind_results = response_for(&connection, &bind_token, || {
portal.call(
"BindShortcuts",
&(session.clone(), shortcuts, "", bind_options),
self.0.replace_reserved(
Arc::new(move |pressed| {
crate::dispatch_dictation_capture(&app, if pressed { "start" } else { "stop" })
}),
accelerator,
revision,
)
})?;
let display = bind_results
.remove("shortcuts")
.and_then(|value| Vec::<(String, VariantMap)>::try_from(value).ok())
.and_then(trigger_description)
.unwrap_or_else(|| crate::dictation_shortcut::display_accelerator(accelerator));
drop(portal);
drop(registry);
Ok((
PortalRegistration {
connection,
session,
},
display,
))
}
fn close_session(registration: &PortalRegistration) -> Result<(), String> {
let session = Proxy::new(
&registration.connection,
DESKTOP_DESTINATION,
registration.session.as_str(),
SESSION_INTERFACE,
)
.map_err(|error| format!("could not open the shortcut session: {error}"))?;
session
.call::<_, _, ()>("Close", &())
.map_err(|error| format!("could not close the shortcut session: {error}"))
}
/// Read the session handle and shortcut id out of an `Activated`/`Deactivated`
/// signal.
///
/// The portal declares `(o session_handle, s shortcut_id, t timestamp,
/// a{sv} options)` — the timestamp is **64-bit**. Deserializing the body into a
/// `u32` field fails zbus' signature check, so every key press was discarded as
/// an invalid signal and dictation never started on any Wayland compositor. The
/// 32-bit spelling stays as a fallback so a non-conforming portal degrades to
/// working rather than to silence.
fn shortcut_signal_target(message: &zbus::Message) -> Result<(OwnedObjectPath, String), String> {
let body = message.body();
if let Ok((session, shortcut_id, _timestamp, _options)) =
body.deserialize::<(OwnedObjectPath, String, u64, VariantMap)>()
{
return Ok((session, shortcut_id));
}
body.deserialize::<(OwnedObjectPath, String, u32, VariantMap)>()
.map(|(session, shortcut_id, _timestamp, _options)| (session, shortcut_id))
.map_err(|error| error.to_string())
}
fn listen(app: tauri::AppHandle, registration: PortalRegistration) -> Result<(), String> {
let portal = Proxy::new(
&registration.connection,
DESKTOP_DESTINATION,
DESKTOP_PATH,
GLOBAL_SHORTCUTS_INTERFACE,
)
.map_err(|error| format!("could not open the global-shortcuts portal: {error}"))?;
log::info!("Wayland dictation shortcut registered through xdg-desktop-portal");
let mut signals = portal
.receive_all_signals()
.map_err(|error| format!("could not listen for portal shortcuts: {error}"))?;
for message in &mut signals {
let header = message.header();
let member = header
.member()
.map(|name| name.as_str().to_owned())
.unwrap_or_default();
if member != "Activated" && member != "Deactivated" {
continue;
}
let (signal_session, shortcut_id) = match shortcut_signal_target(&message) {
Ok(target) => target,
Err(error) => {
log::warn!("Invalid Wayland shortcut signal: {error}");
continue;
}
};
if signal_session != registration.session || shortcut_id != SHORTCUT_ID {
continue;
}
if member == "Activated" {
log::info!("Wayland shortcut pressed: dictation start");
crate::dispatch_dictation_capture(&app, "start");
} else {
log::info!("Wayland shortcut released: dictation stop");
crate::dispatch_dictation_capture(&app, "stop");
}
}
Err("global-shortcuts portal closed the session".into())
}
fn start_listener(app: tauri::AppHandle, registration: PortalRegistration) -> Result<(), String> {
std::thread::Builder::new()
.name("wayland-global-shortcut".into())
.spawn(move || {
if let Err(error) = listen(app, registration) {
log::info!("Wayland shortcut listener stopped: {error}");
}
})
.map(|_| ())
.map_err(|error| format!("failed to start Wayland shortcut listener: {error}"))
}
pub fn register_initial(app: tauri::AppHandle, accelerator: String, revision: u64) {
@@ -657,160 +47,3 @@ pub fn register_initial(app: tauri::AppHandle, accelerator: String, revision: u6
log::error!("Failed to start Wayland shortcut setup: {error}");
}
}
#[cfg(test)]
mod tests {
use super::{
desktop_exec_value, portal_trigger, receive_with_timeout, shortcut_signal_target,
trigger_description, variant_string, PortalShortcutState, VariantMap,
GLOBAL_SHORTCUTS_INTERFACE, SHORTCUT_ID,
};
use std::collections::HashMap;
use std::path::Path;
use std::sync::mpsc;
use std::time::Duration;
use zbus::zvariant::OwnedObjectPath;
#[test]
fn converts_tauri_accelerators_to_portal_triggers() {
assert_eq!(
portal_trigger("CmdOrCtrl+Shift+Space").as_deref(),
Some("CTRL+SHIFT+space")
);
assert_eq!(
portal_trigger("Alt+Control+K").as_deref(),
Some("ALT+CTRL+k")
);
assert_eq!(
portal_trigger("Super+PageUp").as_deref(),
Some("LOGO+Page_Up")
);
assert_eq!(
portal_trigger("Ctrl+BracketLeft").as_deref(),
Some("CTRL+bracketleft")
);
assert_eq!(portal_trigger("Cmd+Digit1").as_deref(), Some("LOGO+1"));
}
#[test]
fn rejects_modifier_free_or_ambiguous_accelerators() {
assert_eq!(portal_trigger("Space"), None);
assert_eq!(portal_trigger("Ctrl+K+L"), None);
}
#[test]
fn a_stale_identity_entry_is_rewritten() {
// The class from 2026-08-13: the entry's Exec pointed at a binary that
// had been moved. GLib then resolves the entry to NULL and the portal
// refuses the bind with "App info not found" — system-wide dictation
// silently dead for the whole session.
let stale = "[Desktop Entry]\nType=Application\nExec=/gone/omnivoice-studio\n";
assert!(super::entry_needs_rewrite(stale, |_| false));
let healthy = "[Desktop Entry]\nType=Application\nExec=\"/opt/VoiceStudio.AppImage\"\n";
assert!(!super::entry_needs_rewrite(healthy, |path| {
path == std::path::Path::new("/opt/VoiceStudio.AppImage")
}));
}
#[test]
fn exec_targets_parse_quoted_legacy_and_missing_lines() {
use super::entry_exec_target;
// Current writer: quoted.
assert_eq!(
entry_exec_target("[Desktop Entry]\nExec=\"/tmp/Voice Studio/app\"\n").as_deref(),
Some(std::path::Path::new("/tmp/Voice Studio/app"))
);
// Pre-quoting entries from older builds still parse.
assert_eq!(
entry_exec_target("[Desktop Entry]\nExec=/home/u/target/debug/omnivoice-studio\n")
.as_deref(),
Some(std::path::Path::new("/home/u/target/debug/omnivoice-studio"))
);
// No Exec at all resolves to NULL in GLib — treat as needing rewrite.
assert_eq!(entry_exec_target("[Desktop Entry]\nType=Application\n"), None);
assert!(super::entry_needs_rewrite("[Desktop Entry]\n", |_| true));
// An action group's Exec is NOT the entry's Exec: GLib still resolves
// the entry to NULL without a main-group Exec, so accepting this would
// keep exactly the stale entry the rewrite exists to replace.
let action_only = "[Desktop Entry]\nType=Application\n[Desktop Action new]\nExec=/bin/true\n";
assert_eq!(entry_exec_target(action_only), None);
assert!(super::entry_needs_rewrite(action_only, |_| true));
}
#[test]
fn desktop_exec_paths_are_quoted_and_escaped() {
assert_eq!(
desktop_exec_value(Path::new("/tmp/Voice Studio/$build")),
"\"/tmp/Voice Studio/\\$build\""
);
}
#[test]
fn uses_the_portals_effective_trigger_description() {
let mut properties = HashMap::new();
properties.insert("trigger_description".into(), variant_string("Meta+Shift+V"));
assert_eq!(
trigger_description(vec![("voice-dictation".into(), properties)]).as_deref(),
Some("Meta+Shift+V")
);
}
#[test]
fn newer_rebinds_supersede_in_flight_registration() {
let state = PortalShortcutState::default();
let startup = state.reserve();
let changed = state.reserve();
assert!(!state.is_current(startup));
assert!(state.is_current(changed));
}
fn shortcut_signal<T>(timestamp: T) -> zbus::Message
where
T: serde::Serialize + zbus::zvariant::Type,
{
let session = OwnedObjectPath::try_from("/org/freedesktop/portal/desktop/session/1").unwrap();
zbus::Message::signal(
super::DESKTOP_PATH,
GLOBAL_SHORTCUTS_INTERFACE,
"Activated",
)
.unwrap()
.build(&(session, SHORTCUT_ID, timestamp, VariantMap::new()))
.unwrap()
}
/// The portal spells the timestamp `t`; a `u32` field made zbus reject every
/// signal, which silently killed Wayland dictation.
#[test]
fn reads_portal_signals_with_a_64_bit_timestamp() {
let message = shortcut_signal(1_786_563_484_746_u64);
let (session, shortcut_id) = shortcut_signal_target(&message)
.expect("64-bit timestamps are the portal's declared spelling");
assert_eq!(
session.as_str(),
"/org/freedesktop/portal/desktop/session/1"
);
assert_eq!(shortcut_id, SHORTCUT_ID);
}
#[test]
fn still_reads_a_32_bit_timestamp_from_a_nonconforming_portal() {
let message = shortcut_signal(42_u32);
let (_session, shortcut_id) = shortcut_signal_target(&message)
.expect("a 32-bit timestamp must not drop the key press");
assert_eq!(shortcut_id, SHORTCUT_ID);
}
#[test]
fn portal_response_wait_is_bounded() {
let (_sender, receiver) = mpsc::channel::<Result<(), String>>();
let error = receive_with_timeout(
&receiver,
Duration::from_millis(1),
"portal shortcut request",
)
.unwrap_err();
assert!(error.contains("timed out"));
}
}
frontend/src-tauri/src/wayland_shortcut_core.rs
+854
View File
@@ -0,0 +1,854 @@
//! Wayland global shortcut support through xdg-desktop-portal.
//!
//! `tauri-plugin-global-shortcut` uses `global-hotkey`, whose Linux backend is
//! X11-only. Under XWayland its registration can still return `Ok(())`, but a
//! native Wayland compositor never sends it key events. The portal is the
//! compositor-owned, permission-aware API for this job.
use std::collections::HashMap;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{mpsc, Mutex};
use std::time::Duration;
use std::sync::Arc;
type Callback = Arc<dyn Fn(bool) + Send + Sync>;
use zbus::{
blocking::{Connection, Proxy},
zvariant::{OwnedObjectPath, OwnedValue, Str},
};
const DESKTOP_DESTINATION: &str = "org.freedesktop.portal.Desktop";
const DESKTOP_PATH: &str = "/org/freedesktop/portal/desktop";
const GLOBAL_SHORTCUTS_INTERFACE: &str = "org.freedesktop.portal.GlobalShortcuts";
const REQUEST_INTERFACE: &str = "org.freedesktop.portal.Request";
const SESSION_INTERFACE: &str = "org.freedesktop.portal.Session";
const REGISTRY_INTERFACE: &str = "org.freedesktop.host.portal.Registry";
const SHORTCUT_ID: &str = "voice-dictation";
static REQUEST_SEQUENCE: AtomicU64 = AtomicU64::new(1);
const PORTAL_LISTENER_TIMEOUT: Duration = Duration::from_secs(5);
const PORTAL_RESPONSE_TIMEOUT: Duration = Duration::from_secs(60);
type VariantMap = HashMap<String, OwnedValue>;
#[derive(Clone)]
struct PortalRegistration {
connection: Connection,
session: OwnedObjectPath,
}
#[derive(Default)]
pub struct PortalShortcutState {
active: Mutex<Option<PortalRegistration>>,
revision: AtomicU64,
identity: DesktopIdentity,
}
impl PortalShortcutState {
pub fn for_desktop(desktop_id: &'static str, executable: std::path::PathBuf) -> Self {
Self {
identity: DesktopIdentity {
desktop_id,
executable: Some(executable),
},
..Self::default()
}
}
pub fn close(&self) {
self.reserve();
if let Ok(mut active) = self.active.lock() {
if let Some(previous) = active.take() {
let _ = close_session(&previous);
}
}
}
pub fn replace(&self, callback: Callback, accelerator: String) -> Result<String, String> {
let revision = self.reserve();
self.replace_reserved(callback, accelerator, revision)
}
pub fn reserve(&self) -> u64 {
self.revision.fetch_add(1, Ordering::SeqCst) + 1
}
fn is_current(&self, revision: u64) -> bool {
self.revision.load(Ordering::SeqCst) == revision
}
pub fn replace_reserved(
&self,
callback: Callback,
accelerator: String,
revision: u64,
) -> Result<String, String> {
// Bind the replacement first. A declined consent dialog or unavailable
// portal therefore leaves the working shortcut and saved preference
// untouched.
let (registration, display) = bind(&accelerator, &self.identity)?;
if !self.is_current(revision) {
let _ = close_session(&registration);
return Err("shortcut registration was superseded by a newer request".into());
}
let mut active = match self.active.lock() {
Ok(active) => active,
Err(_) => {
let _ = close_session(&registration);
return Err("portal shortcut lock poisoned".into());
}
};
if !self.is_current(revision) {
drop(active);
let _ = close_session(&registration);
return Err("shortcut registration was superseded by a newer request".into());
}
if let Err(error) = start_listener(callback, registration.clone()) {
let _ = close_session(&registration);
return Err(error);
}
let previous = active.replace(registration);
drop(active);
if let Some(previous) = previous {
if let Err(error) = close_session(&previous) {
log::warn!("Could not close the previous Wayland shortcut session: {error}");
}
}
Ok(display)
}
}
struct DesktopIdentity {
desktop_id: &'static str,
executable: Option<std::path::PathBuf>,
}
impl Default for DesktopIdentity {
fn default() -> Self {
Self {
desktop_id: "com.debpalash.omnivoice-studio",
executable: None,
}
}
}
fn user_entry_path(desktop_id: &str) -> Option<std::path::PathBuf> {
dirs_next::data_dir().map(|dir| {
dir.join("applications")
.join(format!("{desktop_id}.desktop"))
})
}
/// A packaged (system-dir) entry — deb installs manage their own; never touch.
fn system_entry_exists(desktop_id: &str) -> bool {
let filename = format!("{desktop_id}.desktop");
std::env::var_os("XDG_DATA_DIRS")
.map(|dirs| {
std::env::split_paths(&dirs)
.any(|dir| dir.join("applications").join(&filename).is_file())
})
.unwrap_or_else(|| {
["/usr/local/share", "/usr/share"].iter().any(|dir| {
std::path::Path::new(dir)
.join("applications")
.join(&filename)
.is_file()
})
})
}
/// The `[Desktop Entry]` group's Exec target, unquoted. `None` when the main
/// group has no usable Exec line — which GLib treats the same as a missing
/// program. Scoped to the main group deliberately: a `[Desktop Action …]`
/// group carries its own `Exec=`, and accepting it would retain an entry GLib
/// still cannot resolve (CodeRabbit, #1526).
fn entry_exec_target(content: &str) -> Option<std::path::PathBuf> {
let mut in_main_group = false;
let mut exec = None;
for line in content.lines() {
let line = line.trim_start();
if line.starts_with('[') {
in_main_group = line == "[Desktop Entry]";
continue;
}
if in_main_group {
if let Some(value) = line.strip_prefix("Exec=") {
exec = Some(value);
break;
}
}
}
let raw = exec?.trim();
let unquoted = raw
.strip_prefix('"')
.and_then(|rest| rest.split('"').next())
.unwrap_or_else(|| raw.split_whitespace().next().unwrap_or(raw));
if unquoted.is_empty() {
return None;
}
Some(std::path::PathBuf::from(unquoted))
}
/// Whether a user-local identity entry must be rewritten before the portal
/// will accept it.
///
/// GLib refuses to resolve a desktop entry whose Exec program does not exist
/// (`GDesktopAppInfo` returns NULL), and the portal then rejects the bind with
/// "App info not found" — the shortcut silently dies for the whole session.
/// A dev entry pointing at a `target/debug` binary goes stale exactly this
/// way: a `cargo clean`, a moved checkout, or anything that relocates the
/// binary breaks system-wide dictation with only a log line to show for it.
fn entry_needs_rewrite(content: &str, exec_exists: impl Fn(&std::path::Path) -> bool) -> bool {
match entry_exec_target(content) {
Some(target) => !exec_exists(&target),
None => true,
}
}
fn desktop_exec_path() -> Result<std::path::PathBuf, String> {
// AppImage's current_exe() points inside its transient mount. APPIMAGE is
// the stable launcher path the desktop entry must retain.
if let Some(path) = std::env::var_os("APPIMAGE").filter(|path| !path.is_empty()) {
return Ok(path.into());
}
std::env::current_exe().map_err(|error| format!("could not locate VoiceStudio: {error}"))
}
fn desktop_exec_value(path: &std::path::Path) -> String {
let escaped = path
.to_string_lossy()
.replace('\\', "\\\\")
.replace('"', "\\\"")
.replace('`', "\\`")
.replace('$', "\\$");
format!("\"{escaped}\"")
}
/// The host portal resolves un-sandboxed apps through their desktop entry.
/// Deb packages already install one; dev builds and standalone AppImages may
/// not. Add an invisible identity entry only when none exists.
fn ensure_desktop_identity(identity: &DesktopIdentity) -> Result<(), String> {
if system_entry_exists(identity.desktop_id) {
return Ok(());
}
let path =
user_entry_path(identity.desktop_id).ok_or("could not locate the user data directory")?;
if let Ok(existing) = std::fs::read_to_string(&path) {
if !entry_needs_rewrite(&existing, |target| target.exists()) {
return Ok(());
}
// Stale: GLib returns NULL for an entry whose Exec is gone, and the
// portal then refuses the bind ("App info not found"). Rewrite with
// where the app actually is NOW. The user dir with our app id is ours
// to manage — packaged entries live in the system dirs handled above.
log::info!(
"Wayland portal identity at {} points at a missing program — rewriting",
path.file_name().unwrap_or_default().to_string_lossy()
);
}
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)
.map_err(|error| format!("could not create applications directory: {error}"))?;
}
let entry = format!(
"[Desktop Entry]\nType=Application\nName=VoiceStudio\nExec={}\nTerminal=false\nNoDisplay=true\nStartupWMClass=VoiceStudio\nX-VoiceStudio-Generated=true\n",
desktop_exec_value(&identity.executable.clone().map(Ok).unwrap_or_else(desktop_exec_path)?)
);
std::fs::write(&path, entry)
.map_err(|error| format!("could not create {}: {error}", path.file_name().unwrap_or_default().to_string_lossy()))?;
log::info!("Installed Wayland portal identity at {}", path.file_name().unwrap_or_default().to_string_lossy());
Ok(())
}
pub fn is_wayland_session() -> bool {
std::env::var("XDG_SESSION_TYPE")
.map(|kind| kind.eq_ignore_ascii_case("wayland"))
.unwrap_or(false)
|| std::env::var_os("WAYLAND_DISPLAY").is_some()
}
/// Convert Tauri's cross-platform accelerator spelling to the portal format.
/// The portal may still let the user choose a different chord in its consent
/// dialog, so an unknown spelling is deliberately omitted rather than guessed.
fn portal_trigger(accelerator: &str) -> Option<String> {
let mut modifiers: Vec<String> = Vec::new();
let mut key = None;
for part in accelerator
.split('+')
.map(str::trim)
.filter(|part| !part.is_empty())
{
match part.to_ascii_lowercase().as_str() {
"cmdorctrl" | "commandorcontrol" | "ctrl" | "control" => {
if !modifiers.iter().any(|modifier| modifier == "CTRL") {
modifiers.push("CTRL".into());
}
}
"shift" => modifiers.push("SHIFT".into()),
"alt" | "option" => modifiers.push("ALT".into()),
"cmd" | "command" | "super" | "meta" => modifiers.push("LOGO".into()),
_ if key.is_none() => key = xkb_key_name(part),
_ => return None,
}
}
let key = key?;
if modifiers.is_empty() {
return None;
}
modifiers.push(key);
Some(modifiers.join("+"))
}
fn xkb_key_name(key: &str) -> Option<String> {
let lower = key.to_ascii_lowercase();
if let Some(letter) = lower.strip_prefix("key") {
if letter.len() == 1
&& letter
.chars()
.all(|character| character.is_ascii_alphabetic())
{
return Some(letter.to_owned());
}
}
if let Some(digit) = lower.strip_prefix("digit") {
if digit.len() == 1 && digit.chars().all(|character| character.is_ascii_digit()) {
return Some(digit.to_owned());
}
}
if lower.len() == 1
&& lower
.chars()
.all(|character| character.is_ascii_alphanumeric())
{
return Some(lower);
}
Some(
match lower.as_str() {
"space" => "space",
"enter" | "return" => "Return",
"escape" | "esc" => "Escape",
"tab" => "Tab",
"backspace" => "BackSpace",
"delete" => "Delete",
"insert" => "Insert",
"home" => "Home",
"end" => "End",
"pageup" => "Page_Up",
"pagedown" => "Page_Down",
"arrowup" | "up" => "Up",
"arrowdown" | "down" => "Down",
"arrowleft" | "left" => "Left",
"arrowright" | "right" => "Right",
"minus" => "minus",
"equal" => "equal",
"comma" => "comma",
"period" => "period",
"slash" => "slash",
"semicolon" => "semicolon",
"quote" | "apostrophe" => "apostrophe",
"bracketleft" => "bracketleft",
"bracketright" => "bracketright",
"backslash" => "backslash",
"backquote" | "grave" => "grave",
_ if lower.strip_prefix('f').is_some_and(|digits| {
digits
.parse::<u8>()
.is_ok_and(|number| (1..=35).contains(&number))
}) =>
{
return Some(key.to_ascii_uppercase());
}
_ => return None,
}
.into(),
)
}
fn variant_string(value: &str) -> OwnedValue {
OwnedValue::from(Str::from(value))
}
fn trigger_description(shortcuts: Vec<(String, VariantMap)>) -> Option<String> {
shortcuts
.into_iter()
.find(|(id, _)| id == SHORTCUT_ID)
.and_then(|(_, mut properties)| properties.remove("trigger_description"))
.and_then(|value| String::try_from(value).ok())
.filter(|description| !description.trim().is_empty())
}
fn request_path(connection: &Connection, token: &str) -> Result<OwnedObjectPath, String> {
let sender = connection
.unique_name()
.ok_or("session bus did not assign a unique name")?
.as_str()
.trim_start_matches(':')
.replace('.', "_");
OwnedObjectPath::try_from(format!("{DESKTOP_PATH}/request/{sender}/{token}"))
.map_err(|error| format!("invalid portal request path: {error}"))
}
fn response_for<F>(connection: &Connection, token: &str, call: F) -> Result<VariantMap, String>
where
F: FnOnce() -> Result<OwnedObjectPath, zbus::Error>,
{
// Subscribe before making the request: a fast portal is allowed to answer
// immediately after returning the request handle.
let expected = request_path(connection, token)?;
let listener_connection = connection.clone();
let listener_path = expected.clone();
let (ready_tx, ready_rx) = mpsc::sync_channel(1);
let (response_tx, response_rx) = mpsc::sync_channel(1);
std::thread::Builder::new()
.name("wayland-portal-response".into())
.spawn(move || {
let request = match Proxy::new(
&listener_connection,
DESKTOP_DESTINATION,
listener_path.as_str(),
REQUEST_INTERFACE,
) {
Ok(request) => request,
Err(error) => {
let _ = ready_tx.send(Err(format!("portal request listener: {error}")));
return;
}
};
let mut responses = match request.receive_signal("Response") {
Ok(responses) => responses,
Err(error) => {
let _ = ready_tx.send(Err(format!("portal response listener: {error}")));
return;
}
};
if ready_tx.send(Ok(())).is_err() {
return;
}
let response = responses
.next()
.ok_or_else(|| "portal closed before answering the shortcut request".to_string());
let _ = response_tx.send(response);
})
.map_err(|error| format!("could not start the portal response listener: {error}"))?;
receive_with_timeout(
&ready_rx,
PORTAL_LISTENER_TIMEOUT,
"portal response listener",
)?;
let returned = call().map_err(|error| format!("portal request failed: {error}"))?;
if returned != expected {
return Err(format!(
"portal returned unexpected request path {returned} (expected {expected})"
));
}
let message = match receive_with_timeout(
&response_rx,
PORTAL_RESPONSE_TIMEOUT,
"portal shortcut request",
) {
Ok(message) => message,
Err(error) => {
if let Ok(request) = Proxy::new(
connection,
DESKTOP_DESTINATION,
expected.as_str(),
REQUEST_INTERFACE,
) {
let _ = request.call::<_, _, ()>("Close", &());
}
return Err(error);
}
};
let (code, results): (u32, VariantMap) = message
.body()
.deserialize()
.map_err(|error| format!("invalid portal response: {error}"))?;
if code != 0 {
return Err(format!(
"portal shortcut request was declined (response {code})"
));
}
Ok(results)
}
fn receive_with_timeout<T>(
receiver: &mpsc::Receiver<Result<T, String>>,
timeout: Duration,
operation: &str,
) -> Result<T, String> {
match receiver.recv_timeout(timeout) {
Ok(result) => result,
Err(mpsc::RecvTimeoutError::Timeout) => Err(format!(
"{operation} timed out after {} seconds",
timeout.as_secs()
)),
Err(mpsc::RecvTimeoutError::Disconnected) => {
Err(format!("{operation} stopped before completing"))
}
}
}
fn bind(
accelerator: &str,
identity: &DesktopIdentity,
) -> Result<(PortalRegistration, String), String> {
ensure_desktop_identity(identity)?;
let connection = Connection::session()
.map_err(|error| format!("could not connect to the desktop portal: {error}"))?;
// GNOME's host portal uses the installed desktop entry to associate this
// un-sandboxed process with its desktop id.
let registry = Proxy::new(
&connection,
DESKTOP_DESTINATION,
DESKTOP_PATH,
REGISTRY_INTERFACE,
)
.map_err(|error| format!("could not open the portal registry: {error}"))?;
let registry_options: VariantMap = HashMap::new();
if let Err(error) =
registry.call::<_, _, ()>("Register", &(identity.desktop_id, registry_options))
{
// Development builds and portable AppImages may not have a desktop
// entry for the host registry to resolve. Portal v1 does not require
// this handshake, so continue and let CreateSession be authoritative.
log::warn!("Wayland portal host registration skipped: {error}");
}
let portal = Proxy::new(
&connection,
DESKTOP_DESTINATION,
DESKTOP_PATH,
GLOBAL_SHORTCUTS_INTERFACE,
)
.map_err(|error| format!("could not open the global-shortcuts portal: {error}"))?;
let process = std::process::id();
let sequence = REQUEST_SEQUENCE.fetch_add(1, Ordering::Relaxed);
let create_token = format!("vs_create_{process}_{sequence}");
let session_token = format!("vs_session_{process}_{sequence}");
let mut create_options = VariantMap::new();
create_options.insert("handle_token".into(), variant_string(&create_token));
create_options.insert(
"session_handle_token".into(),
variant_string(&session_token),
);
let mut create_results = response_for(&connection, &create_token, || {
portal.call("CreateSession", &(create_options,))
})?;
let session_value = create_results
.remove("session_handle")
.ok_or("portal did not return a shortcut session")?;
// The portal specification declares an object path, but deployed portal
// versions historically returned a string. Accept both wire formats.
let session = match session_value
.try_clone()
.ok()
.and_then(|value| OwnedObjectPath::try_from(value).ok())
{
Some(path) => path,
None => {
let path = String::try_from(session_value).map_err(|error| {
format!("portal returned an invalid shortcut session handle: {error}")
})?;
OwnedObjectPath::try_from(path)
.map_err(|error| format!("portal returned an invalid session path: {error}"))?
}
};
let mut shortcut_info = VariantMap::new();
shortcut_info.insert("description".into(), variant_string("VoiceStudio"));
if let Some(trigger) = portal_trigger(&accelerator) {
shortcut_info.insert("preferred_trigger".into(), variant_string(&trigger));
}
let shortcuts = vec![(SHORTCUT_ID.to_string(), shortcut_info)];
let bind_token = format!("vs_bind_{process}_{sequence}");
let mut bind_options = VariantMap::new();
bind_options.insert("handle_token".into(), variant_string(&bind_token));
let mut bind_results = response_for(&connection, &bind_token, || {
portal.call(
"BindShortcuts",
&(session.clone(), shortcuts, "", bind_options),
)
})?;
let display = bind_results
.remove("shortcuts")
.and_then(|value| Vec::<(String, VariantMap)>::try_from(value).ok())
.and_then(trigger_description)
.unwrap_or_else(|| display_accelerator(accelerator));
drop(portal);
drop(registry);
Ok((
PortalRegistration {
connection,
session,
},
display,
))
}
fn close_session(registration: &PortalRegistration) -> Result<(), String> {
let session = Proxy::new(
&registration.connection,
DESKTOP_DESTINATION,
registration.session.as_str(),
SESSION_INTERFACE,
)
.map_err(|error| format!("could not open the shortcut session: {error}"))?;
session
.call::<_, _, ()>("Close", &())
.map_err(|error| format!("could not close the shortcut session: {error}"))
}
/// Read the session handle and shortcut id out of an `Activated`/`Deactivated`
/// signal.
///
/// The portal declares `(o session_handle, s shortcut_id, t timestamp,
/// a{sv} options)` — the timestamp is **64-bit**. Deserializing the body into a
/// `u32` field fails zbus' signature check, so every key press was discarded as
/// an invalid signal and dictation never started on any Wayland compositor. The
/// 32-bit spelling stays as a fallback so a non-conforming portal degrades to
/// working rather than to silence.
fn shortcut_signal_target(message: &zbus::Message) -> Result<(OwnedObjectPath, String), String> {
let body = message.body();
if let Ok((session, shortcut_id, _timestamp, _options)) =
body.deserialize::<(OwnedObjectPath, String, u64, VariantMap)>()
{
return Ok((session, shortcut_id));
}
body.deserialize::<(OwnedObjectPath, String, u32, VariantMap)>()
.map(|(session, shortcut_id, _timestamp, _options)| (session, shortcut_id))
.map_err(|error| error.to_string())
}
fn listen(callback: Callback, registration: PortalRegistration) -> Result<(), String> {
let portal = Proxy::new(
&registration.connection,
DESKTOP_DESTINATION,
DESKTOP_PATH,
GLOBAL_SHORTCUTS_INTERFACE,
)
.map_err(|error| format!("could not open the global-shortcuts portal: {error}"))?;
log::info!("Wayland dictation shortcut registered through xdg-desktop-portal");
let mut signals = portal
.receive_all_signals()
.map_err(|error| format!("could not listen for portal shortcuts: {error}"))?;
for message in &mut signals {
let header = message.header();
let member = header
.member()
.map(|name| name.as_str().to_owned())
.unwrap_or_default();
if member != "Activated" && member != "Deactivated" {
continue;
}
let (signal_session, shortcut_id) = match shortcut_signal_target(&message) {
Ok(target) => target,
Err(error) => {
log::warn!("Invalid Wayland shortcut signal: {error}");
continue;
}
};
if signal_session != registration.session || shortcut_id != SHORTCUT_ID {
continue;
}
if member == "Activated" {
log::info!("Wayland shortcut pressed: dictation start");
callback(true);
} else {
log::info!("Wayland shortcut released: dictation stop");
callback(false);
}
}
Err("global-shortcuts portal closed the session".into())
}
fn start_listener(callback: Callback, registration: PortalRegistration) -> Result<(), String> {
std::thread::Builder::new()
.name("wayland-global-shortcut".into())
.spawn(move || {
if let Err(error) = listen(callback, registration) {
log::info!("Wayland shortcut listener stopped: {error}");
}
})
.map(|_| ())
.map_err(|error| format!("failed to start Wayland shortcut listener: {error}"))
}
#[cfg(test)]
mod tests {
use super::{
desktop_exec_value, portal_trigger, receive_with_timeout, shortcut_signal_target,
trigger_description, variant_string, PortalShortcutState, VariantMap,
GLOBAL_SHORTCUTS_INTERFACE, SHORTCUT_ID,
};
use std::collections::HashMap;
use std::path::Path;
use std::sync::mpsc;
use std::time::Duration;
use zbus::zvariant::OwnedObjectPath;
#[test]
fn converts_tauri_accelerators_to_portal_triggers() {
assert_eq!(
portal_trigger("CmdOrCtrl+Shift+Space").as_deref(),
Some("CTRL+SHIFT+space")
);
assert_eq!(
portal_trigger("Alt+Control+K").as_deref(),
Some("ALT+CTRL+k")
);
assert_eq!(
portal_trigger("Super+PageUp").as_deref(),
Some("LOGO+Page_Up")
);
assert_eq!(
portal_trigger("Ctrl+BracketLeft").as_deref(),
Some("CTRL+bracketleft")
);
assert_eq!(portal_trigger("Cmd+Digit1").as_deref(), Some("LOGO+1"));
}
#[test]
fn rejects_modifier_free_or_ambiguous_accelerators() {
assert_eq!(portal_trigger("Space"), None);
assert_eq!(portal_trigger("Ctrl+K+L"), None);
}
#[test]
fn a_stale_identity_entry_is_rewritten() {
// The class from 2026-08-13: the entry's Exec pointed at a binary that
// had been moved. GLib then resolves the entry to NULL and the portal
// refuses the bind with "App info not found" — system-wide dictation
// silently dead for the whole session.
let stale = "[Desktop Entry]\nType=Application\nExec=/gone/omnivoice-studio\n";
assert!(super::entry_needs_rewrite(stale, |_| false));
let healthy = "[Desktop Entry]\nType=Application\nExec=\"/opt/VoiceStudio.AppImage\"\n";
assert!(!super::entry_needs_rewrite(healthy, |path| {
path == std::path::Path::new("/opt/VoiceStudio.AppImage")
}));
}
#[test]
fn exec_targets_parse_quoted_legacy_and_missing_lines() {
use super::entry_exec_target;
// Current writer: quoted.
assert_eq!(
entry_exec_target("[Desktop Entry]\nExec=\"/tmp/Voice Studio/app\"\n").as_deref(),
Some(std::path::Path::new("/tmp/Voice Studio/app"))
);
// Pre-quoting entries from older builds still parse.
assert_eq!(
entry_exec_target("[Desktop Entry]\nExec=/home/u/target/debug/omnivoice-studio\n")
.as_deref(),
Some(std::path::Path::new(
"/home/u/target/debug/omnivoice-studio"
))
);
// No Exec at all resolves to NULL in GLib — treat as needing rewrite.
assert_eq!(
entry_exec_target("[Desktop Entry]\nType=Application\n"),
None
);
assert!(super::entry_needs_rewrite("[Desktop Entry]\n", |_| true));
// An action group's Exec is NOT the entry's Exec: GLib still resolves
// the entry to NULL without a main-group Exec, so accepting this would
// keep exactly the stale entry the rewrite exists to replace.
let action_only =
"[Desktop Entry]\nType=Application\n[Desktop Action new]\nExec=/bin/true\n";
assert_eq!(entry_exec_target(action_only), None);
assert!(super::entry_needs_rewrite(action_only, |_| true));
}
#[test]
fn desktop_exec_paths_are_quoted_and_escaped() {
assert_eq!(
desktop_exec_value(Path::new("/tmp/Voice Studio/$build")),
"\"/tmp/Voice Studio/\\$build\""
);
}
#[test]
fn uses_the_portals_effective_trigger_description() {
let mut properties = HashMap::new();
properties.insert("trigger_description".into(), variant_string("Meta+Shift+V"));
assert_eq!(
trigger_description(vec![("voice-dictation".into(), properties)]).as_deref(),
Some("Meta+Shift+V")
);
}
#[test]
fn newer_rebinds_supersede_in_flight_registration() {
let state = PortalShortcutState::default();
let startup = state.reserve();
let changed = state.reserve();
assert!(!state.is_current(startup));
assert!(state.is_current(changed));
}
fn shortcut_signal<T>(timestamp: T) -> zbus::Message
where
T: serde::Serialize + zbus::zvariant::Type,
{
let session =
OwnedObjectPath::try_from("/org/freedesktop/portal/desktop/session/1").unwrap();
zbus::Message::signal(super::DESKTOP_PATH, GLOBAL_SHORTCUTS_INTERFACE, "Activated")
.unwrap()
.build(&(session, SHORTCUT_ID, timestamp, VariantMap::new()))
.unwrap()
}
/// The portal spells the timestamp `t`; a `u32` field made zbus reject every
/// signal, which silently killed Wayland dictation.
#[test]
fn reads_portal_signals_with_a_64_bit_timestamp() {
let message = shortcut_signal(1_786_563_484_746_u64);
let (session, shortcut_id) = shortcut_signal_target(&message)
.expect("64-bit timestamps are the portal's declared spelling");
assert_eq!(
session.as_str(),
"/org/freedesktop/portal/desktop/session/1"
);
assert_eq!(shortcut_id, SHORTCUT_ID);
}
#[test]
fn still_reads_a_32_bit_timestamp_from_a_nonconforming_portal() {
let message = shortcut_signal(42_u32);
let (_session, shortcut_id) = shortcut_signal_target(&message)
.expect("a 32-bit timestamp must not drop the key press");
assert_eq!(shortcut_id, SHORTCUT_ID);
}
#[test]
fn portal_response_wait_is_bounded() {
let (_sender, receiver) = mpsc::channel::<Result<(), String>>();
let error = receive_with_timeout(
&receiver,
Duration::from_millis(1),
"portal shortcut request",
)
.unwrap_err();
assert!(error.contains("timed out"));
}
}
fn display_accelerator(accelerator: &str) -> String {
accelerator
.split('+')
.map(|part| match part.to_ascii_lowercase().as_str() {
"cmdorctrl" | "commandorcontrol" | "ctrl" | "control" => "Ctrl",
"cmd" | "command" | "meta" | "super" => "Super",
"alt" | "option" => "Alt",
"shift" => "Shift",
_ => part,
})
.collect::<Vec<_>>()
.join("+")
}
frontend/src-tauri/tauri.conf.json
+1 -1
View File
@@ -46,7 +46,7 @@
}
],
"security": {
"csp": "default-src 'self' 'unsafe-inline' 'unsafe-eval'; connect-src 'self' ipc://localhost http://localhost:* ws://localhost:* http://127.0.0.1:* ws://127.0.0.1:* blob: data:; media-src 'self' blob: data: http://localhost:* http://127.0.0.1:* asset: https://asset.localhost; img-src 'self' blob: data: asset: https://asset.localhost http://localhost:* http://127.0.0.1:* https://fonts.gstatic.com; font-src 'self' data: https://fonts.googleapis.com https://fonts.gstatic.com; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com;",
"csp": "default-src 'self' 'unsafe-inline' 'unsafe-eval'; connect-src 'self' ipc://localhost http://localhost:* ws://localhost:* http://127.0.0.1:* ws://127.0.0.1:* https://eu.i.posthog.com blob: data:; media-src 'self' blob: data: http://localhost:* http://127.0.0.1:* asset: https://asset.localhost; img-src 'self' blob: data: asset: https://asset.localhost http://localhost:* http://127.0.0.1:* https://fonts.gstatic.com; font-src 'self' data: https://fonts.googleapis.com https://fonts.gstatic.com; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com;",
"assetProtocol": {
"enable": true,
"scope": [
frontend/src/components/settings/PerformancePanel.jsx
+23 -29
View File
@@ -1,13 +1,16 @@
/**
* Settings → Performance panel (Wave 2 INST-12 UI half).
*
* Toggles the `Disable torch.compile (Windows)` setting that backend
* engine launchers read via `services.engine_env.build_engine_env()`.
* Toggles the `Disable torch.compile` setting that backend engine
* launchers read via `services.engine_env.build_engine_env()`.
*
* The toggle is disabled (with an explainer tooltip) on non-Windows
* platforms — torch.compile OOMs the same Triton kernel cache
* differently on macOS / Linux, so toggling it there would just slow
* the engine for no gain (issue #65).
* Usable on every platform since #2135. It was previously disabled
* outside Windows on the theory that torch.compile only misbehaves
* there (issue #65, the Triton kernel-cache OOM). #2135 is the
* counter-example: a Linux/CUDA host whose engine was killed by
* torch.compile, where the one control that would have stopped it was
* greyed out. A toggle the affected user cannot reach is not a
* safeguard.
*
* Endpoints:
* GET /api/settings/perf/torch-compile-disabled
@@ -25,7 +28,6 @@ import RestartBadge from './RestartBadge';
export default function PerformancePanel() {
const { t } = useTranslation();
const [enabled, setEnabled] = useState(false);
const [platform, setPlatform] = useState(null);
const [loading, setLoading] = useState(false);
const [saving, setSaving] = useState(false);
const [error, setError] = useState(null);
@@ -36,7 +38,6 @@ export default function PerformancePanel() {
try {
const data = await apiJson('/api/settings/perf/torch-compile-disabled');
setEnabled(Boolean(data?.enabled));
setPlatform(data?.platform ?? null);
} catch (e) {
setError(
e?.message ||
@@ -51,8 +52,6 @@ export default function PerformancePanel() {
refresh();
}, [refresh]);
const isWindows = platform === 'win32';
const onToggle = async (next) => {
setSaving(true);
setError(null);
@@ -94,28 +93,16 @@ export default function PerformancePanel() {
<RestartBadge />
</>
}
subtitle={
!isWindows
? platform === null
? '…'
: t('settings.perf_torch_compile_na', {
defaultValue: 'Windows only — not needed on this platform',
})
: undefined
}
note={
isWindows
? t('settings.perf_torch_compile_note', {
defaultValue: 'Falls back to eager mode — fixes Triton OOM on <16 GB GPUs.',
})
: undefined
}
note={t('settings.perf_torch_compile_note', {
defaultValue: 'Falls back to eager mode — fixes Triton OOM on <16 GB GPUs.',
})}
hint={
<Trans
i18nKey="settings.perf_torch_compile_hint"
defaults="Workaround for <issueLink>#65</issueLink> — Windows users may hit Triton / <code>torch.compile</code> OOM during model load on GPUs with less than 16 GB VRAM. Enabling this sets <code>TORCH_COMPILE_DISABLE=1</code> on engine subprocesses, which falls back to eager mode. macOS and Linux are unaffected."
defaults="Falls back to eager mode by setting <code>TORCH_COMPILE_DISABLE=1</code> on the engine. Turn this on if model load or generation fails with a Triton / <code>torch.compile</code> error, or if the backend dies mid-generation — see <issueLink>#65</issueLink> (Windows OOM on GPUs under 16 GB) and <crashLink>#2135</crashLink> (CUDA-graph crash on older NVIDIA GPUs). Slower, but it always works."
components={{
// Trans injects the link text ("#65") from the translation string.
// Trans injects each link's text ("#65" / "#2135") from the
// translation string.
issueLink: (
<a
href="https://github.com/debpalash/VoiceStudio/issues/65"
@@ -123,6 +110,13 @@ export default function PerformancePanel() {
rel="noopener noreferrer"
/>
),
crashLink: (
<a
href="https://github.com/debpalash/VoiceStudio/issues/2135"
target="_blank"
rel="noopener noreferrer"
/>
),
code: <code />,
}}
/>
@@ -131,7 +125,7 @@ export default function PerformancePanel() {
<SettingsToggle
checked={enabled}
onChange={onToggle}
disabled={!isWindows || saving || loading}
disabled={saving || loading}
aria-label={toggleLabel}
data-testid="torch-compile-toggle"
/>
frontend/src/components/settings/PerformancePanel.test.jsx
+8 -8
View File
@@ -55,7 +55,7 @@ describe('PerformancePanel', () => {
});
});
it('renders disabled with badge on non-Windows platforms (darwin)', async () => {
it('allows eager fallback on macOS', async () => {
global.fetch = mockFetchSequence({
status: 200,
body: { enabled: false, platform: 'darwin' },
@@ -63,9 +63,9 @@ describe('PerformancePanel', () => {
render(<PerformancePanel />);
await waitFor(() => {
const toggle = screen.getByTestId('torch-compile-toggle');
expect(toggle).toBeDisabled();
expect(toggle).not.toBeDisabled();
});
expect(screen.getByText(/not needed on this platform/i)).toBeInTheDocument();
expect(screen.queryByText(/not needed on this platform/i)).not.toBeInTheDocument();
});
it('renders every user-facing string through i18n (en fallback)', async () => {
@@ -77,10 +77,10 @@ describe('PerformancePanel', () => {
await waitFor(() => screen.getByTestId('torch-compile-toggle'));
// Section title + row label resolve from settings.perf_* keys.
expect(screen.getByText('Performance')).toBeInTheDocument();
expect(screen.getByText(/Disable torch\.compile \(Windows\)/)).toBeInTheDocument();
expect(screen.getByText(/Falls back to eager mode/)).toBeInTheDocument();
expect(screen.getByText(/Disable torch\.compile/)).toBeInTheDocument();
expect(screen.getByText(/Uses eager execution/)).toBeInTheDocument();
expect(screen.getByTestId('torch-compile-toggle')).toHaveAccessibleName(
/Disable torch\.compile \(Windows\)/,
/Disable torch\.compile/,
);
});
@@ -91,7 +91,7 @@ describe('PerformancePanel', () => {
expect(screen.getByRole('alert')).toHaveTextContent(/boom|Failed to load/i);
});
it('renders disabled on linux platform', async () => {
it('allows eager fallback on Linux', async () => {
global.fetch = mockFetchSequence({
status: 200,
body: { enabled: false, platform: 'linux' },
@@ -99,7 +99,7 @@ describe('PerformancePanel', () => {
render(<PerformancePanel />);
await waitFor(() => {
const toggle = screen.getByTestId('torch-compile-toggle');
expect(toggle).toBeDisabled();
expect(toggle).not.toBeDisabled();
});
});
frontend/src/config/integration-catalog.js
+192 -12
View File
@@ -4,13 +4,181 @@ import genericLogo from '../assets/integrations/integration-generic.svg';
import { VOICE_AI_DIRECTORY } from './voice-ai-directory';
const groups = [
['automation', ['n8n|https://n8n.io', 'Zapier|https://zapier.com', 'Make|https://www.make.com', 'Pipedream|https://pipedream.com', 'IFTTT|https://ifttt.com', 'Activepieces|https://www.activepieces.com', 'Windmill|https://www.windmill.dev', 'Temporal|https://temporal.io', 'Prefect|https://www.prefect.io', 'Dagster|https://dagster.io', 'Airflow|https://airflow.apache.org', 'Node-RED|https://nodered.org', 'Huginn|https://github.com/huginn/huginn', 'Trigger.dev|https://trigger.dev', 'Inngest|https://www.inngest.com', 'Pipedream Connect|https://pipedream.com/connect', 'Workato|https://www.workato.com', 'Tray.ai|https://tray.ai', 'Retool Workflows|https://retool.com/workflows', 'Bardeen|https://www.bardeen.ai'] ],
['comms', ['Twilio|https://www.twilio.com', 'Plivo|https://www.plivo.com', 'Telnyx|https://telnyx.com', 'SignalWire|https://signalwire.com', 'Vonage|https://www.vonage.com/communications-apis', 'Bandwidth|https://www.bandwidth.com', 'Sinch|https://sinch.com', 'MessageBird|https://bird.com', 'Agora|https://www.agora.io', 'Daily|https://www.daily.co', 'LiveKit|https://livekit.io', 'Stream|https://getstream.io', 'Retell AI|https://www.retellai.com', 'Vapi|https://vapi.ai', 'Bland AI|https://www.bland.ai', 'ElevenLabs Conversational AI|https://elevenlabs.io/conversational-ai', 'Deepgram Voice Agent|https://deepgram.com/voice-agents', 'SIP.js|https://sipjs.com', 'Asterisk|https://www.asterisk.org', 'FreeSWITCH|https://signalwire.com/freeswitch'] ],
['agents', ['OpenAI Agents|https://platform.openai.com/docs/guides/agents', 'Anthropic Claude|https://www.anthropic.com', 'Claude Code|https://docs.anthropic.com/en/docs/claude-code', 'Codex CLI|https://github.com/openai/codex', 'GitHub Copilot|https://github.com/features/copilot', 'Cursor|https://www.cursor.com', 'Windsurf|https://windsurf.com', 'Cline|https://github.com/cline/cline', 'Roo Code|https://github.com/RooCodeInc/Roo-Code', 'Aider|https://aider.chat', 'OpenHands|https://github.com/All-Hands-AI/OpenHands', 'Goose|https://block.github.io/goose', 'OpenCode|https://opencode.ai', 'Continue|https://www.continue.dev', 'Amazon Q Developer|https://aws.amazon.com/q/developer', 'Google Jules|https://jules.google', 'Devin|https://devin.ai', 'Replit Agent|https://replit.com/ai', 'Warp AI|https://www.warp.dev/ai', 'Pi|https://pi.dev'] ],
['mcp', ['Model Context Protocol|https://modelcontextprotocol.io', 'MCP Registry|https://registry.modelcontextprotocol.io', 'GitHub MCP Server|https://github.com/github/github-mcp-server', 'Filesystem MCP|https://github.com/modelcontextprotocol/servers', 'Postgres MCP|https://github.com/modelcontextprotocol/servers', 'Brave Search MCP|https://github.com/modelcontextprotocol/servers', 'Slack MCP|https://github.com/modelcontextprotocol/servers', 'Notion MCP|https://github.com/makenotion/notion-mcp-server', 'Linear MCP|https://github.com/jerhadf/linear-mcp-server', 'Figma MCP|https://github.com/GLips/Figma-Context-MCP', 'Playwright MCP|https://github.com/microsoft/playwright-mcp', 'Browserbase MCP|https://github.com/browserbase/mcp-server-browserbase', 'Sentry MCP|https://github.com/getsentry/sentry-mcp', 'Cloudflare MCP|https://github.com/cloudflare/mcp-server-cloudflare', 'AWS MCP|https://github.com/awslabs/mcp', 'Google Cloud MCP|https://github.com/GoogleCloudPlatform/mcp', 'Docker MCP Catalog|https://www.docker.com/products/mcp-catalog', 'MCPJam|https://www.mcpjam.com', 'Smithery|https://smithery.ai', 'PulseMCP|https://www.pulsemcp.com'] ],
['developer', ['GitHub|https://github.com', 'GitLab|https://gitlab.com', 'Bitbucket|https://bitbucket.org', 'GitHub Actions|https://github.com/features/actions', 'GitHub Container Registry|https://ghcr.io', 'Docker|https://www.docker.com', 'Kubernetes|https://kubernetes.io', 'Vercel|https://vercel.com', 'Netlify|https://www.netlify.com', 'Cloudflare Workers|https://workers.cloudflare.com', 'Fly.io|https://fly.io', 'Railway|https://railway.app', 'Render|https://render.com', 'Supabase|https://supabase.com', 'Neon|https://neon.tech', 'Turso|https://turso.tech', 'PlanetScale|https://planetscale.com', 'Sentry|https://sentry.io', 'PostHog|https://posthog.com', 'Grafana|https://grafana.com'] ],
['data', ['OpenAI API|https://platform.openai.com', 'Hugging Face|https://huggingface.co', 'Ollama|https://ollama.com', 'LM Studio|https://lmstudio.ai', 'vLLM|https://vllm.ai', 'llama.cpp|https://github.com/ggml-org/llama.cpp', 'Replicate|https://replicate.com', 'Together AI|https://www.together.ai', 'Groq|https://groq.com', 'Cerebras|https://www.cerebras.ai', 'Fireworks AI|https://fireworks.ai', 'Cohere|https://cohere.com', 'Mistral AI|https://mistral.ai', 'Google Gemini API|https://ai.google.dev', 'Perplexity API|https://www.perplexity.ai', 'AssemblyAI|https://www.assemblyai.com', 'Deepgram|https://deepgram.com', 'Qdrant|https://qdrant.tech', 'Weaviate|https://weaviate.io', 'Pinecone|https://www.pinecone.io'] ],
['productivity', ['Slack|https://slack.com', 'Discord|https://discord.com', 'Microsoft Teams|https://www.microsoft.com/microsoft-teams', 'Telegram|https://telegram.org', 'Notion|https://www.notion.so', 'Linear|https://linear.app', 'Jira|https://www.atlassian.com/software/jira', 'Trello|https://trello.com', 'Asana|https://asana.com', 'ClickUp|https://clickup.com', 'Google Drive|https://drive.google.com', 'Dropbox|https://www.dropbox.com', 'OneDrive|https://www.microsoft.com/microsoft-365/onedrive', 'Gmail|https://gmail.com', 'Google Calendar|https://calendar.google.com', 'Microsoft Outlook|https://outlook.live.com', 'Airtable|https://airtable.com', 'HubSpot|https://www.hubspot.com', 'Salesforce|https://www.salesforce.com', 'Intercom|https://www.intercom.com'] ],
[
'automation',
[
'n8n|https://n8n.io',
'Zapier|https://zapier.com',
'Make|https://www.make.com',
'Pipedream|https://pipedream.com',
'IFTTT|https://ifttt.com',
'Activepieces|https://www.activepieces.com',
'Windmill|https://www.windmill.dev',
'Temporal|https://temporal.io',
'Prefect|https://www.prefect.io',
'Dagster|https://dagster.io',
'Airflow|https://airflow.apache.org',
'Node-RED|https://nodered.org',
'Huginn|https://github.com/huginn/huginn',
'Trigger.dev|https://trigger.dev',
'Inngest|https://www.inngest.com',
'Pipedream Connect|https://pipedream.com/connect',
'Workato|https://www.workato.com',
'Tray.ai|https://tray.ai',
'Retool Workflows|https://retool.com/workflows',
'Bardeen|https://www.bardeen.ai',
],
],
[
'comms',
[
'Twilio|https://www.twilio.com',
'Plivo|https://www.plivo.com',
'Telnyx|https://telnyx.com',
'SignalWire|https://signalwire.com',
'Vonage|https://www.vonage.com/communications-apis',
'Bandwidth|https://www.bandwidth.com',
'Sinch|https://sinch.com',
'MessageBird|https://bird.com',
'Agora|https://www.agora.io',
'Daily|https://www.daily.co',
'LiveKit|https://livekit.io',
'Stream|https://getstream.io',
'Retell AI|https://www.retellai.com',
'Vapi|https://vapi.ai',
'Bland AI|https://www.bland.ai',
'ElevenLabs Conversational AI|https://elevenlabs.io/conversational-ai',
'Deepgram Voice Agent|https://deepgram.com/voice-agents',
'SIP.js|https://sipjs.com',
'Asterisk|https://www.asterisk.org',
'FreeSWITCH|https://signalwire.com/freeswitch',
],
],
[
'agents',
[
'OpenAI Agents|https://platform.openai.com/docs/guides/agents',
'Anthropic Claude|https://www.anthropic.com',
'Claude Code|https://docs.anthropic.com/en/docs/claude-code',
'Codex CLI|https://github.com/openai/codex',
'GitHub Copilot|https://github.com/features/copilot',
'Cursor|https://www.cursor.com',
'Windsurf|https://windsurf.com',
'Cline|https://github.com/cline/cline',
'Roo Code|https://github.com/RooCodeInc/Roo-Code',
'Aider|https://aider.chat',
'OpenHands|https://github.com/All-Hands-AI/OpenHands',
'Goose|https://block.github.io/goose',
'OpenCode|https://opencode.ai',
'Continue|https://www.continue.dev',
'Amazon Q Developer|https://aws.amazon.com/q/developer',
'Google Jules|https://jules.google',
'Devin|https://devin.ai',
'Replit Agent|https://replit.com/ai',
'Warp AI|https://www.warp.dev/ai',
'Pi|https://pi.dev',
],
],
[
'mcp',
[
'Model Context Protocol|https://modelcontextprotocol.io',
'MCP Registry|https://registry.modelcontextprotocol.io',
'GitHub MCP Server|https://github.com/github/github-mcp-server',
'Filesystem MCP|https://github.com/modelcontextprotocol/servers',
'Postgres MCP|https://github.com/modelcontextprotocol/servers',
'Brave Search MCP|https://github.com/modelcontextprotocol/servers',
'Slack MCP|https://github.com/modelcontextprotocol/servers',
'Notion MCP|https://github.com/makenotion/notion-mcp-server',
'Linear MCP|https://github.com/jerhadf/linear-mcp-server',
'Figma MCP|https://github.com/GLips/Figma-Context-MCP',
'Playwright MCP|https://github.com/microsoft/playwright-mcp',
'Browserbase MCP|https://github.com/browserbase/mcp-server-browserbase',
'Sentry MCP|https://github.com/getsentry/sentry-mcp',
'Cloudflare MCP|https://github.com/cloudflare/mcp-server-cloudflare',
'AWS MCP|https://github.com/awslabs/mcp',
'Google Cloud MCP|https://github.com/GoogleCloudPlatform/mcp',
'Docker MCP Catalog|https://www.docker.com/products/mcp-catalog',
'MCPJam|https://www.mcpjam.com',
'Smithery|https://smithery.ai',
'PulseMCP|https://www.pulsemcp.com',
],
],
[
'developer',
[
'GitHub|https://github.com',
'GitLab|https://gitlab.com',
'Bitbucket|https://bitbucket.org',
'GitHub Actions|https://github.com/features/actions',
'GitHub Container Registry|https://ghcr.io',
'Docker|https://www.docker.com',
'Kubernetes|https://kubernetes.io',
'Vercel|https://vercel.com',
'Netlify|https://www.netlify.com',
'Cloudflare Workers|https://workers.cloudflare.com',
'Fly.io|https://fly.io',
'Railway|https://railway.app',
'Render|https://render.com',
'Supabase|https://supabase.com',
'Neon|https://neon.tech',
'Turso|https://turso.tech',
'PlanetScale|https://planetscale.com',
'Sentry|https://sentry.io',
'PostHog|https://posthog.com',
'Grafana|https://grafana.com',
],
],
[
'data',
[
'OpenAI API|https://platform.openai.com',
'Hugging Face|https://huggingface.co',
'Ollama|https://ollama.com',
'LM Studio|https://lmstudio.ai',
'vLLM|https://vllm.ai',
'llama.cpp|https://github.com/ggml-org/llama.cpp',
'Replicate|https://replicate.com',
'Together AI|https://www.together.ai',
'Groq|https://groq.com',
'Cerebras|https://www.cerebras.ai',
'Fireworks AI|https://fireworks.ai',
'Cohere|https://cohere.com',
'Mistral AI|https://mistral.ai',
'Google Gemini API|https://ai.google.dev',
'Perplexity API|https://www.perplexity.ai',
'AssemblyAI|https://www.assemblyai.com',
'Deepgram|https://deepgram.com',
'Qdrant|https://qdrant.tech',
'Weaviate|https://weaviate.io',
'Pinecone|https://www.pinecone.io',
],
],
[
'productivity',
[
'Slack|https://slack.com',
'Discord|https://discord.com',
'Microsoft Teams|https://www.microsoft.com/microsoft-teams',
'Telegram|https://telegram.org',
'Notion|https://www.notion.so',
'Linear|https://linear.app',
'Jira|https://www.atlassian.com/software/jira',
'Trello|https://trello.com',
'Asana|https://asana.com',
'ClickUp|https://clickup.com',
'Google Drive|https://drive.google.com',
'Dropbox|https://www.dropbox.com',
'OneDrive|https://www.microsoft.com/microsoft-365/onedrive',
'Gmail|https://gmail.com',
'Google Calendar|https://calendar.google.com',
'Microsoft Outlook|https://outlook.live.com',
'Airtable|https://airtable.com',
'HubSpot|https://www.hubspot.com',
'Salesforce|https://www.salesforce.com',
'Intercom|https://www.intercom.com',
],
],
];
const categoryDetails = {
@@ -25,14 +193,26 @@ const categoryDetails = {
export const INTEGRATION_CATALOG = [
...VOICE_AI_DIRECTORY.map((entry) => ({ ...entry, category: 'comms', featured: false })),
...groups.flatMap(([category, items]) => items.map((item) => {
const [name, url] = item.split('|');
return { name, url, logoUrl: genericLogo, detailKeys: categoryDetails[category], category, featured: false };
})),
...groups.flatMap(([category, items]) =>
items.map((item) => {
const [name, url] = item.split('|');
return {
name,
url,
logoUrl: genericLogo,
detailKeys: categoryDetails[category],
category,
featured: false,
};
}),
),
];
export function integrationSlug(name) {
return name.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-|-$/g, '');
return name
.toLowerCase()
.replace(/[^a-z0-9]+/g, '-')
.replace(/^-|-$/g, '');
}
export function getIntegrationBySlug(slug) {

Some files were not shown because too many files have changed in this diff Show More