Merge remote-tracking branch 'origin/main' into fix/1900-attempt-id
# Conflicts: # CHANGELOG.md
This commit is contained in:
Palash Debnath
@@ -10,6 +10,7 @@ the frozen-backend fallback mirror it for their toolchains.
|
||||
|
||||
**Highlights**
|
||||
- A pronunciation entry that is stored but not applied yet says so, instead of looking like it did not match (#1949)
|
||||
- A bare 500 report now names the backend error class, so two unrelated faults stop filing the same issue (#1773)
|
||||
- The first-run install log is kept on disk instead of vanishing with the setup screen (#1847)
|
||||
- `bun run desktop` reclaims port 3900 from a backend the app itself left running, instead of refusing to start (#1974)
|
||||
- A dictation shortcut another app already owns now says so, instead of silently doing nothing (#1858)
|
||||
@@ -84,10 +85,13 @@ the frozen-backend fallback mirror it for their toolchains.
|
||||
|
||||
### Docs
|
||||
|
||||
- Docker quick starts now explain the AMD64-only images and direct Apple Silicon users to the native macOS app (#1921) — thanks @yangfan-yf-yf!
|
||||
- audio.cpp (Breeze-TTS-2) is now a documented opt-in engine: prebuilt binary install, explicit GGUF download, voice modes, and the weights' research/non-commercial terms (#1891)
|
||||
- `docs/STRUCTURE.md` describes the tree as it is today, and a test now keeps its counts honest (#1981) — thanks @Dawcraft!
|
||||
|
||||
### Fixed
|
||||
|
||||
- Windows contributors can run the test suite without Developer Mode: tests that create a symlink now skip instead of failing with `WinError 1314` (#1990)
|
||||
- The first-run setup screen no longer mislabels a step when the bootstrap restarts itself: Rust now says which attempt each stage and log line belongs to, instead of the screen guessing from a once-a-second poll (#1900)
|
||||
|
||||
- Windows desktop launches no longer freeze at "Loading ML runtime (PyTorch)": the parent-liveness watchdog polls the stdin pipe instead of leaving a read pending, which deadlocked numpy's OpenBLAS initializer (#1952)
|
||||
|
||||
@@ -78,7 +78,7 @@ Download a package from the [latest release](https://github.com/debpalash/VoiceS
|
||||
| macOS 13.3+ | Apple Silicon DMG | [Install on macOS](docs/install/macos.md) |
|
||||
| Windows 10/11 | x64 MSI; choose the current-user build when listed to install without admin access | [Install on Windows](docs/install/windows.md#install-pre-built-msi) |
|
||||
| Linux | AppImage, x86_64 with glibc 2.39+ | [Install on Linux](docs/install/linux.md) |
|
||||
| Docker | CUDA, ROCm, CPU, and worker-only GPU profiles | [Run with Docker](docs/install/docker.md) |
|
||||
| Docker | Linux/AMD64 images; CUDA, ROCm, CPU, and worker-only GPU profiles | [Run with Docker](docs/install/docker.md) |
|
||||
|
||||
First launch creates a managed Python environment and downloads the default model. Later launches reuse both.
|
||||
|
||||
@@ -87,6 +87,11 @@ First launch creates a managed Python environment and downloads the default mode
|
||||
|
||||
### Quick Docker run
|
||||
|
||||
The published images are **`linux/amd64` only**. On Apple Silicon, use the
|
||||
[native macOS app](docs/install/macos.md) for GPU acceleration. ARM64 hosts
|
||||
should read the [architecture requirements](docs/install/docker.md#architecture)
|
||||
before pulling an image.
|
||||
|
||||
```bash
|
||||
docker run -d -p 127.0.0.1:3900:3900 -v omnivoice-data:/app/omnivoice_data --name voicestudio palashdeb/omnivoice-studio:stable
|
||||
```
|
||||
|
||||
@@ -14,12 +14,21 @@ cloning, and cinematic video dubbing — fully local, with no cloud API keys or
|
||||
|
||||
|
||||
|
||||
VoiceStudio runs entirely on your own hardware (CUDA / MPS / ROCm / CPU
|
||||
VoiceStudio runs entirely on your own hardware (CUDA / ROCm / CPU
|
||||
auto-detect) — nothing is sent to the cloud. This image is the **headless
|
||||
web-server build**: a FastAPI backend serving a pre-built React UI over HTTP, so
|
||||
you can run it on a homelab box, a GPU server, or anywhere Docker runs and open
|
||||
you can run it on an AMD64 homelab box or GPU server and open
|
||||
the UI in a browser.
|
||||
|
||||
**Architecture:** published images are **`linux/amd64` only**; there is no
|
||||
native ARM64 image. On Apple Silicon, use the
|
||||
[native macOS app](https://github.com/debpalash/VoiceStudio/blob/main/docs/install/macos.md)
|
||||
for Apple GPU acceleration; the Linux container cannot access the Mac's Apple
|
||||
GPU through MPS or MLX. Other ARM64 hosts need an AMD64 server or CPU emulation,
|
||||
which can be much slower. See the
|
||||
[architecture requirements](https://github.com/debpalash/VoiceStudio/blob/main/docs/install/docker.md#architecture)
|
||||
before pulling an image.
|
||||
|
||||
> The Tauri desktop app's auto-updater and update-channel toggle are
|
||||
> **desktop-only** and do not apply to this image — to update, pull a newer tag
|
||||
> and recreate the container.
|
||||
@@ -136,7 +145,7 @@ are mirrored on GHCR at
|
||||
- **📦 Batch Queue** — drop 50 videos and walk away; per-job progress.
|
||||
- **🤖 MCP Server** — drive VoiceStudio from Claude, Cursor, or any MCP client.
|
||||
- **🛡️ AI Watermark** — invisible AudioSeal (Meta) marking that survives compression.
|
||||
- **⚡ GPU Auto-Detect** — CUDA · MPS · ROCm · CPU, with auto-offload on ≤8 GB cards.
|
||||
- **⚡ GPU Auto-Detect** — CUDA · ROCm · CPU, with auto-offload on ≤8 GB cards.
|
||||
- **🧩 Extensible** — subclass `TTSBackend` to add any engine in ~50 lines.
|
||||
|
||||
Multiple TTS engines ship out of the box (IndexTTS, CosyVoice, Supertonic-3, and
|
||||
|
||||
+109
-48
@@ -7,39 +7,71 @@ Every folder has a single job. Every file at the root earns its place.
|
||||
```
|
||||
VoiceStudio/
|
||||
│
|
||||
├── README.md ⟵ user-facing overview
|
||||
├── CHANGELOG.md ⟵ release history
|
||||
├── LICENSE
|
||||
├── README.md / README_CN.md ⟵ user-facing overview (English / Chinese)
|
||||
├── CHANGELOG.md ⟵ release history; release.yml extracts the tag's section verbatim
|
||||
├── CLAUDE.md / AGENTS.md ⟵ the working contract for AI agents — keep the two in sync
|
||||
├── LICENSE, LICENSE-NOTICE.md, SPONSORS.md
|
||||
│
|
||||
├── pyproject.toml ⟵ Python project manifest
|
||||
├── pyproject.toml ⟵ Python project manifest (+ pytest / lint config)
|
||||
├── uv.lock ⟵ Python lockfile
|
||||
├── package.json ⟵ monorepo manifest (Bun workspaces + Turborepo)
|
||||
├── bun.lock ⟵ JS lockfile
|
||||
├── bun.lock ⟵ JS lockfile — repo-root, covers frontend/ too
|
||||
├── turbo.json ⟵ turborepo pipeline
|
||||
│
|
||||
├── .coderabbit.yaml ⟵ CodeRabbit PR review config (fed CLAUDE.md)
|
||||
├── greptile.json ⟵ Greptile PR review config (fed CLAUDE.md)
|
||||
├── skills-lock.json ⟵ pins the sources + hashes of .agents/skills/
|
||||
├── .gitleaks.toml ⟵ secret-scan config
|
||||
├── .gitmodules ⟵ omnivoice-gallery submodule
|
||||
├── .python-version
|
||||
├── .dockerignore ⟵ Docker build context filter
|
||||
├── backend.spec ⟵ pyinstaller spec (stays at root by pyinstaller convention)
|
||||
├── alembic.ini ⟵ DB migration config (stays at root by alembic convention)
|
||||
│
|
||||
├── .env ⟵ user config; gitignored, .env.example is the template
|
||||
├── .gitignore
|
||||
├── .gitignore ⟵ a repo-local .env stays ignored, but user config is NOT
|
||||
│ kept here: the durable env file is ~/.config/omnivoice/env
|
||||
│ (backend/core/user_env.py), written by the Settings panel
|
||||
│
|
||||
├── backend/ ⟵ FastAPI server
|
||||
│ ├── main.py
|
||||
│ ├── api/routers/ HTTP endpoints (thin)
|
||||
│ ├── core/ config, db, task queue, metrics
|
||||
│ ├── services/ business logic
|
||||
│ └── schemas/ pydantic request/response shapes
|
||||
│ ├── main.py the one entry point; its boot order is load-bearing —
|
||||
│ │ read the comments before reordering anything
|
||||
│ ├── api/routers/ 39 routers, auto-included; thin HTTP/WS surface
|
||||
│ │ └── setup/ first-run wizard, model download
|
||||
│ ├── core/ config, db, job queue, event bus, auth/CSRF, path security,
|
||||
│ │ opt-in analytics, version, diagnostics
|
||||
│ ├── services/ 78 modules of business logic — TTS, dubbing pipeline,
|
||||
│ │ audio DSP, GPU gateway, engine routing, model lifecycle
|
||||
│ ├── engines/ per-engine adapters: indextts, supertonic3, confucius4,
|
||||
│ │ dots_tts, moss_tts_v15, pockettts, audiocpp,
|
||||
│ │ omnivoice_gguf, omnivoice_subprocess, _asr_sidecar, _echo
|
||||
│ ├── worker/ remote / distributed workers — scheduler, pool, routing,
|
||||
│ │ breaker, capacity, plus protocol/ and inbound/
|
||||
│ ├── mcp_shim/ MCP server entry point (docs/mcp.md)
|
||||
│ ├── speech_client/ speech sidecar client entry point
|
||||
│ ├── schemas/ pydantic request/response shapes
|
||||
│ ├── migrations/versions/ alembic revisions — every schema change goes through here
|
||||
│ ├── plugins/ plugin drop-in point (see services/plugin_sdk.py)
|
||||
│ ├── hooks/ pyinstaller runtime hooks
|
||||
│ ├── config/models.yaml model catalogue
|
||||
│ └── tests/ the isolated pytest session — see "Where tests live"
|
||||
│
|
||||
├── frontend/ ⟵ React 19 + Vite + Tauri desktop
|
||||
│ ├── package.json THE app version — every other version file mirrors it
|
||||
│ ├── src/
|
||||
│ │ ├── pages/ one file per top-level view
|
||||
│ │ ├── components/ reusable UI
|
||||
│ │ ├── api/ typed API clients
|
||||
│ │ ├── store/ Zustand slices
|
||||
│ │ ├── components/ reusable UI (+ audiobook/ clone/ dub/ gallery/ settings/ …)
|
||||
│ │ ├── ui/, lib/ shared primitives and helpers
|
||||
│ │ ├── api/ typed API clients, one per router group
|
||||
│ │ ├── store/ Zustand slices (+ persisted-state migrations)
|
||||
│ │ ├── hooks/ custom React hooks
|
||||
│ │ └── utils/
|
||||
│ ├── src-tauri/ Rust desktop shell
|
||||
│ │ ├── i18n/locales/ the ONLY home for user-facing strings
|
||||
│ │ ├── config/, data/, assets/, utils/
|
||||
│ │ └── test/ vitest setup + visual-test helpers
|
||||
│ ├── e2e/, e2e-perf/, e2e-prod/ Playwright suites: functional, perf, packaged bundle
|
||||
│ ├── src-tauri/ Rust desktop shell — backend spawn/bootstrap, updater
|
||||
│ │ │ channel, dictation shortcut, crash/reset/uninstall
|
||||
│ │ ├── capabilities/, icons/, wix/, debian/, appimage/ packaging inputs
|
||||
│ │ └── tests/
|
||||
│ └── public/
|
||||
│
|
||||
├── omnivoice/ ⟵ the underlying TTS model package
|
||||
@@ -51,46 +83,59 @@ VoiceStudio/
|
||||
│ ├── training/
|
||||
│ └── utils/
|
||||
│
|
||||
├── tests/ ⟵ all tests live here, no exceptions
|
||||
│ ├── conftest.py
|
||||
│ ├── test_api.py
|
||||
│ ├── test_dub_*.py
|
||||
│ ├── test_job_queue.py
|
||||
│ ├── test_segmentation.py
|
||||
│ └── frontend/ Node-based frontend tests
|
||||
├── tests/ ⟵ the main pytest session (testpaths in pyproject.toml)
|
||||
│ ├── conftest.py hermetic OMNIVOICE_DATA_DIR — never touches real app state
|
||||
│ ├── backend/, scripts/ mirrors of the source trees they cover
|
||||
│ ├── smoke/ fast end-to-end checks (own CI job, HF_HUB_OFFLINE=1)
|
||||
│ ├── evals/ quality evals (evals.yml)
|
||||
│ ├── probe/, fixtures/
|
||||
│ └── frontend/ Node-based frontend tests (legacy; vitest is the default)
|
||||
│
|
||||
├── scripts/ ⟵ dev / build / release shell + python scripts
|
||||
│ ├── install.sh universal installer (macOS/Linux/WSL)
|
||||
│ ├── install.ps1 universal installer (Windows)
|
||||
│ ├── run.sh universal launcher
|
||||
├── scripts/ ⟵ dev / build / release scripts (shell, python, mjs)
|
||||
│ ├── install.sh / install.ps1 universal installers
|
||||
│ ├── desktop-*.mjs dev, prod and fresh desktop launchers
|
||||
│ ├── smoke-test.sh end-to-end validation
|
||||
│ └── desktop-prod.sh production desktop build
|
||||
|
||||
│ ├── check-docs-drift.py the docs-drift.yml checker (docs/features.yaml is canonical)
|
||||
│ └── build-omnivoice-tts.sh builds the bin/ sidecars
|
||||
│
|
||||
├── bin/ ⟵ prebuilt omnivoice-tts sidecars, one per platform
|
||||
│
|
||||
├── .agents/skills/ ⟵ canonical skill copies (vite, fastapi-python), pinned by
|
||||
│ skills-lock.json — followed by path, never symlinked
|
||||
├── skills/ ⟵ skills this repo publishes (omnivoice, oss-maintainer)
|
||||
│
|
||||
├── infra/ ⟵ edge/deploy workers (not the Docker deploy path)
|
||||
│ └── install-redirect/ voicestudio.sh/install — UA-sniffing installer worker
|
||||
│
|
||||
├── deploy/ ⟵ Docker deployment configs
|
||||
│ ├── Dockerfile single-stage CUDA image
|
||||
│ └── docker-compose.yml one-click local deployment
|
||||
│ ├── Dockerfile CUDA by default; CI builds the ROCm variant from the same
|
||||
│ │ file via BASE_IMAGE / GPU_FLAVOR overrides
|
||||
│ ├── docker-compose.yml one-click local deployment
|
||||
│ ├── torch-constraints.txt pinned torch resolution for the image
|
||||
│ └── dockerhub-overview.md synced to the Docker Hub overview page at release
|
||||
│
|
||||
├── docs/ ⟵ developer docs, screenshots, branding
|
||||
│ ├── ROADMAP.md where this project is going
|
||||
│ ├── STRUCTURE.md you are here
|
||||
│ ├── mcp.json MCP config template
|
||||
│ ├── preview.png README hero image
|
||||
│ ├── logo.png, logo.svg branding assets
|
||||
│ ├── screenshot-*.png feature screenshots
|
||||
│ ├── languages.md
|
||||
│ ├── training.md
|
||||
│ ├── data_preparation.md
|
||||
│ ├── evaluation.md
|
||||
│ └── voice-design.md
|
||||
│ ├── RELEASING.md the release checklist — every deployment channel
|
||||
│ ├── features.yaml canonical feature inventory (drives docs-drift.yml)
|
||||
│ ├── adr/ architecture decision records
|
||||
│ ├── agents/ agent-facing docs (issue tracker, triage labels, domain)
|
||||
│ ├── engines/, dubbing/, install/, setup/, migration/, features/, playbooks/, specs/
|
||||
│ ├── media/, screenshot-*.png, preview.png, logo.*
|
||||
│ └── languages.md, training.md, data_preparation.md, evaluation.md, voice-design.md
|
||||
│
|
||||
├── examples/ ⟵ runnable demos + sample inputs
|
||||
├── examples/ ⟵ runnable demos + sample inputs (agentic/, speech-platform/)
|
||||
│
|
||||
├── notebooks/ ⟵ OmniVoice_Studio_Colab.ipynb
|
||||
│
|
||||
├── omnivoice-gallery/ ⟵ git submodule — the published voice gallery
|
||||
│
|
||||
├── omnivoice_data/ ⟵ Docker bind-mount target (gitignored)
|
||||
│ DB + HF cache live here when running via compose
|
||||
│
|
||||
├── .github/workflows/ ⟵ ci, docker, release, security, docs-drift, evals,
|
||||
│ install-smoke, build-omnivoice-tts
|
||||
└── .git/
|
||||
```
|
||||
|
||||
@@ -98,11 +143,22 @@ VoiceStudio/
|
||||
|
||||
1. **Nothing at the root is a runtime artifact.** Outputs, temp files, local DBs, crash logs — all go to `~/Library/Application Support/OmniVoice/` (or the OS equivalent), *never* into the repo. The one exception is `omnivoice_data/`, which exists as a bind-mount anchor for Docker.
|
||||
|
||||
2. **No ad-hoc scripts at the root.** One-off debug scripts live in `scripts/`. Tests live in `tests/`. Benchmarks live in `scripts/benchmarks/` (when we create them).
|
||||
2. **No ad-hoc scripts at the root.** One-off debug scripts live in `scripts/`. Tests live in one of the three homes below, never at the root.
|
||||
|
||||
3. **Each subdirectory owns one concern.** If you can't describe what goes in a directory in one sentence, it's wrong.
|
||||
|
||||
4. **Every package has a manifest.** `backend/`, `frontend/`, `omnivoice/` each have their own deps declared via `pyproject.toml` / `package.json` — they are independently testable.
|
||||
4. **Every package has a manifest.** `backend/`, `frontend/`, `omnivoice/` each have their own deps declared via `pyproject.toml` / `package.json` — they are independently testable. The JS lockfile is the **repo-root** `bun.lock` (Bun workspace), and `deploy/Dockerfile` installs from it with `--frozen-lockfile`.
|
||||
|
||||
## Where tests live
|
||||
|
||||
Three homes, each with its own runner. CI runs all three inside the single `test` job in
|
||||
`ci.yml`, as separate steps. The split is deliberate, not drift:
|
||||
|
||||
| Home | Runner | Why it's separate |
|
||||
|---|---|---|
|
||||
| `tests/` | `pytest tests/` — the `testpaths` default | The main suite. Its `conftest.py` points `OMNIVOICE_DATA_DIR` at a throwaway dir so a run can never touch the developer's real app state (#878). |
|
||||
| `backend/tests/` | `pytest backend/tests/` — its own pytest session (the `Run pytest (backend/tests, isolated)` step) | Runs as an isolated session against `backend/`'s bare imports. Its `conftest.py` sets the same hermetic data dir; **never** reintroduce module-level `sys.modules` stubs there — they leak process-wide at collection time and poison mixed runs. |
|
||||
| `frontend/src/**/*.test.{js,jsx,ts,tsx}` | `bun run test` (vitest, jsdom) | Co-located with the component under test. `frontend/e2e*/` hold the Playwright suites; `tests/frontend/` is the older `node:test` set. |
|
||||
|
||||
## What lives where
|
||||
|
||||
@@ -110,10 +166,14 @@ VoiceStudio/
|
||||
|---|---|
|
||||
| User-facing product code | `backend/`, `frontend/` |
|
||||
| The TTS model (independent of the studio) | `omnivoice/` |
|
||||
| A new TTS/ASR engine adapter | `backend/engines/<engine>/` |
|
||||
| Everything executable but not user-facing | `scripts/` |
|
||||
| Tests | `tests/` |
|
||||
| Prebuilt platform sidecars | `bin/` |
|
||||
| Python tests | `tests/` (or `backend/tests/` when the isolated session is required) |
|
||||
| Frontend unit tests | next to the component, as `*.test.jsx` |
|
||||
| Developer + user docs (Markdown) | `docs/` |
|
||||
| Architecture decision records (ADRs) | `docs/adr/` |
|
||||
| Agent-facing docs | `docs/agents/` |
|
||||
| Runnable demos and sample data | `examples/` |
|
||||
| Runtime data (never committed) | `~/Library/Application Support/OmniVoice/` on Mac |
|
||||
|
||||
@@ -137,10 +197,10 @@ Removed in the 2026-07-12 cleanup pass (all preserved in git history):
|
||||
| Dir | Why it was there | Where it went |
|
||||
|---|---|---|
|
||||
| `.planning/` (74 files) | GSD-era planning archive: phases, quick plans, issue clusters. The GSD workflow was retired 2026-07-08. | Deleted; the four load-bearing decision docs moved to `docs/adr/`. |
|
||||
| `specs/` | spec-kit specs for features 001–007 — all shipped. | Deleted. |
|
||||
| `specs/` | spec-kit specs for features 001–007 — all shipped. | Deleted; `docs/specs/` is the current home. |
|
||||
| `design/` | ASCII mockups of the pre-React target UX, superseded by the shipped app. | Deleted. |
|
||||
| `research/` | Archived legacy Gradio UI + April-2026 competitor notes. | Deleted. |
|
||||
| `.agents/` | Rules for a third-party agent tool no longer in use. | Deleted. |
|
||||
| `.agents/` | Rules for a third-party agent tool no longer in use. | Deleted — then reintroduced with a different job: `.agents/skills/` now holds the canonical skill copies pinned by `skills-lock.json`. |
|
||||
|
||||
## Scaling path (proposed, not yet executed)
|
||||
|
||||
@@ -169,12 +229,13 @@ VoiceStudio/
|
||||
- `backend.spec` (`['backend/main.py']`, `pathex=['.']`)
|
||||
- `frontend/src-tauri/tauri.*.conf.json` sidecar paths
|
||||
- every import that reads `from backend.main import …` (tests, scripts)
|
||||
- `frontend/package.json` as the version source of truth, and the mirrors that track it
|
||||
|
||||
Migrate when adding the second `apps/*` or the second `packages/*`. Not before.
|
||||
|
||||
## Conventions
|
||||
|
||||
- **Filenames:** snake_case for Python, kebab-case or PascalCase for JS/TS components, lowercase for Markdown.
|
||||
- **Tests mirror source paths.** `backend/services/dub_pipeline.py` → `tests/services/test_dub_pipeline.py`.
|
||||
- **Tests mirror source paths** where a mirror exists: `tests/backend/` mirrors `api/ core/ engines/ services/`, so `backend/services/ffmpeg_utils.py` → `tests/backend/services/test_ffmpeg_utils.py`. Everything else stays flat — `tests/backend/test_*.py` for backend-wide cases, `tests/test_*.py` for cross-cutting ones. A React component's test sits next to the component.
|
||||
- **One-off scripts** go into `scripts/` with a descriptive name, not `test_*.py` at the root.
|
||||
- **New top-level directories** require a PR that updates *this file*.
|
||||
|
||||
@@ -7,6 +7,25 @@ it in a normal browser.
|
||||
**Official images:** [`ghcr.io/debpalash/omnivoice-studio`](https://github.com/debpalash/VoiceStudio/pkgs/container/omnivoice-studio)
|
||||
and [`palashdeb/omnivoice-studio` on Docker Hub](https://hub.docker.com/r/palashdeb/omnivoice-studio) — same images, same tags.
|
||||
|
||||
## Architecture
|
||||
|
||||
The published images are **`linux/amd64` (x86-64) only**, including `:stable`,
|
||||
`:latest`, and the ROCm variants. There is no native `linux/arm64` image.
|
||||
On an ARM64 host, pulling without an explicit platform can fail with
|
||||
`no matching manifest for linux/arm64/v8 in the manifest list entries`.
|
||||
|
||||
- **Apple Silicon (M-series Macs):** use the [native macOS app](macos.md),
|
||||
which supports Apple GPU acceleration. The Linux container cannot access
|
||||
the Mac's Apple GPU through MPS or MLX.
|
||||
- **AMD64 emulation on ARM64 (including Apple Silicon):** if your Docker
|
||||
installation supports it, place `--platform linux/amd64` **before the image
|
||||
name** in both `docker pull` and `docker run` from the CPU instructions
|
||||
below. This is an emulated CPU option, not native
|
||||
ARM64 support; inference can be much slower and is not a GPU workaround.
|
||||
Without emulation, use an AMD64 server for this Docker deployment.
|
||||
|
||||
## Image tags
|
||||
|
||||
> **Image ↔ version mapping**
|
||||
>
|
||||
> | Tag | What you get |
|
||||
@@ -237,6 +256,33 @@ docker compose -f deploy/docker-compose.yml --profile gpu up -d
|
||||
docker compose -f deploy/docker-compose.yml --profile rocm up -d
|
||||
```
|
||||
|
||||
> **ARM64 hosts:** Compose has no per-command `--platform` flag, so the
|
||||
> override that works for `docker pull` and `docker run` does not reach it.
|
||||
> Set `DOCKER_DEFAULT_PLATFORM=linux/amd64` in the shell you run Compose from,
|
||||
> or the image resolves to the ARM64 manifest that does not exist and fails
|
||||
> with `no matching manifest for linux/arm64/v8`. Only the CPU profile makes
|
||||
> sense under emulation — it is not a GPU workaround.
|
||||
>
|
||||
> ```bash
|
||||
> export DOCKER_DEFAULT_PLATFORM=linux/amd64
|
||||
> docker compose -f deploy/docker-compose.yml --profile cpu pull
|
||||
> docker compose -f deploy/docker-compose.yml --profile cpu up -d
|
||||
> ```
|
||||
>
|
||||
> In PowerShell, set both the administrator key and the platform for the
|
||||
> session before running the same two commands:
|
||||
>
|
||||
> ```powershell
|
||||
> $env:OMNIVOICE_API_KEY = python -c "import secrets; print(secrets.token_urlsafe(32))"
|
||||
> $env:DOCKER_DEFAULT_PLATFORM = 'linux/amd64'
|
||||
> docker compose -f deploy/docker-compose.yml --profile cpu pull
|
||||
> docker compose -f deploy/docker-compose.yml --profile cpu up -d
|
||||
> ```
|
||||
>
|
||||
> Either way the setting lives only in that shell and the processes it starts.
|
||||
> The [architecture limits](#architecture) still apply: this is emulated CPU
|
||||
> inference, not native ARM64 support.
|
||||
|
||||
The `docker-compose.yml` shipped in `deploy/` defaults to `127.0.0.1:3900`
|
||||
on the host. The backend inside the container binds to `0.0.0.0` so the
|
||||
host port mapping can forward — the host-side `127.0.0.1` binding is what
|
||||
|
||||
@@ -236,11 +236,29 @@ if (typeof window !== 'undefined') {
|
||||
export class ApiError extends Error {
|
||||
status?: number;
|
||||
detail?: unknown;
|
||||
constructor(message: string, init: { status?: number; detail?: unknown } = {}) {
|
||||
/**
|
||||
* The backend exception type behind an unclassified failure.
|
||||
*
|
||||
* The 500 handler puts `error_class` in the response body, but nothing
|
||||
* lifted it onto the Error — so the auto bug reporter, which reads the
|
||||
* Error, filed "VoiceStudio hit an internal error; check the backend log"
|
||||
* and nothing else. Every such report looked identical and none could be
|
||||
* triaged (#1773).
|
||||
*
|
||||
* #1956 did this for the streaming path. This is the classic path, which
|
||||
* had been carrying the datum on the wire the whole time.
|
||||
*/
|
||||
errorClass?: string;
|
||||
constructor(
|
||||
message: string,
|
||||
init: { status?: number; detail?: unknown; errorClass?: string } = {},
|
||||
) {
|
||||
super(message);
|
||||
this.name = 'ApiError';
|
||||
this.status = init.status;
|
||||
this.detail = init.detail;
|
||||
this.errorClass =
|
||||
typeof init.errorClass === 'string' && init.errorClass ? init.errorClass : undefined;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -577,9 +595,16 @@ export async function apiFetch(path: string, opts: ApiFetchOptions = {}): Promis
|
||||
typeof detail === 'string'
|
||||
? detail
|
||||
: ((detail as { message?: string })?.message ?? JSON.stringify(detail));
|
||||
// The backend names the exception type in `error_class` on its 500s.
|
||||
// Lifting it here is what lets the bug report say which failure it was.
|
||||
const errorClass =
|
||||
detail && typeof detail === 'object'
|
||||
? (detail as { error_class?: unknown }).error_class
|
||||
: undefined;
|
||||
throw new ApiError(`${res.status} ${res.statusText}: ${msg}`, {
|
||||
status: res.status,
|
||||
detail,
|
||||
errorClass: typeof errorClass === 'string' ? errorClass : undefined,
|
||||
});
|
||||
}
|
||||
return res;
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
/**
|
||||
* #1773 — an unclassified 500 must name the failure it actually was.
|
||||
*
|
||||
* The backend's 500 handler has always put `error_class` in the response body,
|
||||
* but nothing lifted it onto the Error object. The auto bug reporter reads the
|
||||
* Error, so it filed "VoiceStudio hit an internal error; check the backend log
|
||||
* for details." and nothing else — every such report identical, none of them
|
||||
* triageable.
|
||||
*
|
||||
* #1956 fixed the streaming path the same way. This is the classic path, which
|
||||
* had been carrying the datum on the wire the whole time.
|
||||
*/
|
||||
import { describe, it, expect } from 'vitest';
|
||||
|
||||
import { ApiError } from '../api/client';
|
||||
import { buildBugReportUrl } from '../utils/bugReport';
|
||||
|
||||
describe('ApiError carries the backend error class', () => {
|
||||
it('keeps a string class', () => {
|
||||
const err = new ApiError('500 Internal Server Error: boom', {
|
||||
status: 500,
|
||||
detail: { detail: 'boom', error_class: 'MemoryError' },
|
||||
errorClass: 'MemoryError',
|
||||
});
|
||||
expect(err.errorClass).toBe('MemoryError');
|
||||
});
|
||||
|
||||
it('leaves it undefined when the backend sent none', () => {
|
||||
// Not every failure has one — a 404 or a validation error carries no class,
|
||||
// and inventing an empty string would put a blank line in every report.
|
||||
const err = new ApiError('404 Not Found: nope', { status: 404, detail: 'nope' });
|
||||
expect(err.errorClass).toBeUndefined();
|
||||
});
|
||||
|
||||
it('ignores a non-string class rather than stringifying it', () => {
|
||||
const err = new ApiError('500', { status: 500, errorClass: { nope: true } });
|
||||
expect(err.errorClass).toBeUndefined();
|
||||
});
|
||||
|
||||
it('reaches the bug report', async () => {
|
||||
// The whole point: the report is what a maintainer reads.
|
||||
const err = new ApiError('500 Internal Server Error: internal error', {
|
||||
status: 500,
|
||||
errorClass: 'FileNotFoundError',
|
||||
});
|
||||
const body = decodeURIComponent(await buildBugReportUrl({ error: err }));
|
||||
expect(body).toContain('Backend error class: FileNotFoundError');
|
||||
});
|
||||
|
||||
it('two unrelated 500s stop producing the same report', async () => {
|
||||
const a = new ApiError('500 Internal Server Error: internal error', {
|
||||
status: 500,
|
||||
errorClass: 'MemoryError',
|
||||
});
|
||||
const b = new ApiError('500 Internal Server Error: internal error', {
|
||||
status: 500,
|
||||
errorClass: 'PermissionError',
|
||||
});
|
||||
const [ra, rb] = await Promise.all([
|
||||
buildBugReportUrl({ error: a }),
|
||||
buildBugReportUrl({ error: b }),
|
||||
]);
|
||||
expect(decodeURIComponent(ra)).not.toBe(decodeURIComponent(rb));
|
||||
});
|
||||
});
|
||||
@@ -274,7 +274,7 @@ submit "Hacker News" "https://news.ycombinator.com/submitlink
|
||||
echo ""
|
||||
echo "${YELLOW}━━━ 12. Extended Wayback Machine Archives ━━━${NC}"
|
||||
EXTRA_PAGES=(
|
||||
"https://github.com/debpalash/VoiceStudio/blob/main/STRUCTURE.md"
|
||||
"https://github.com/debpalash/VoiceStudio/blob/main/docs/STRUCTURE.md"
|
||||
"https://github.com/debpalash/VoiceStudio/blob/main/LICENSE"
|
||||
"https://github.com/debpalash/VoiceStudio/graphs/contributors"
|
||||
"https://github.com/debpalash/VoiceStudio/network/dependents"
|
||||
|
||||
@@ -510,12 +510,14 @@ def test_package_filename_default_and_override(monkeypatch, app_modules):
|
||||
assert bootstrap.package_filename() == "breeze-tts-2-bf16.gguf"
|
||||
|
||||
|
||||
def test_materialize_hf_symlink_keeps_gguf_suffix_without_copy(tmp_path, app_modules):
|
||||
def test_materialize_hf_symlink_keeps_gguf_suffix_without_copy(
|
||||
tmp_path, app_modules, symlink_or_skip,
|
||||
):
|
||||
bootstrap = app_modules.bootstrap
|
||||
blob = tmp_path / "content-addressed-blob"
|
||||
blob.write_bytes(b"GGUF test payload")
|
||||
snapshot = tmp_path / "breeze-tts-2-q8_0.gguf"
|
||||
snapshot.symlink_to(blob)
|
||||
symlink_or_skip(snapshot, blob)
|
||||
|
||||
materialized = bootstrap._materialize_gguf_cache_path(snapshot)
|
||||
|
||||
@@ -533,16 +535,18 @@ def test_materialize_rejects_extensionless_model(tmp_path, app_modules):
|
||||
bootstrap._materialize_gguf_cache_path(model)
|
||||
|
||||
|
||||
def test_materialize_replaces_preexisting_symlink_alias(tmp_path, app_modules):
|
||||
def test_materialize_replaces_preexisting_symlink_alias(
|
||||
tmp_path, app_modules, symlink_or_skip,
|
||||
):
|
||||
bootstrap = app_modules.bootstrap
|
||||
blob = tmp_path / "content-addressed-blob"
|
||||
blob.write_bytes(b"GGUF test payload")
|
||||
snapshot = tmp_path / "breeze-tts-2-q8_0.gguf"
|
||||
snapshot.symlink_to(blob)
|
||||
symlink_or_skip(snapshot, blob)
|
||||
alias = snapshot.with_name(
|
||||
f".{snapshot.stem}-{bootstrap.HF_MODEL_REVISION[:12]}.audiocpp.gguf"
|
||||
)
|
||||
alias.symlink_to(blob)
|
||||
symlink_or_skip(alias, blob)
|
||||
|
||||
materialized = bootstrap._materialize_gguf_cache_path(snapshot)
|
||||
|
||||
@@ -552,7 +556,7 @@ def test_materialize_replaces_preexisting_symlink_alias(tmp_path, app_modules):
|
||||
|
||||
|
||||
def test_materialize_cross_filesystem_symlink_links_beside_target(
|
||||
tmp_path, monkeypatch, app_modules,
|
||||
tmp_path, monkeypatch, app_modules, symlink_or_skip,
|
||||
):
|
||||
bootstrap = app_modules.bootstrap
|
||||
source_dir = tmp_path / "source"
|
||||
@@ -562,7 +566,7 @@ def test_materialize_cross_filesystem_symlink_links_beside_target(
|
||||
link_dir = tmp_path / "link"
|
||||
link_dir.mkdir()
|
||||
snapshot = link_dir / "custom.gguf"
|
||||
snapshot.symlink_to(blob)
|
||||
symlink_or_skip(snapshot, blob)
|
||||
real_link = os.link
|
||||
calls = 0
|
||||
|
||||
@@ -584,13 +588,13 @@ def test_materialize_cross_filesystem_symlink_links_beside_target(
|
||||
|
||||
|
||||
def test_file_override_materializes_hf_style_symlink(
|
||||
tmp_path, monkeypatch, app_modules,
|
||||
tmp_path, monkeypatch, app_modules, symlink_or_skip,
|
||||
):
|
||||
bootstrap = app_modules.bootstrap
|
||||
blob = tmp_path / "blob"
|
||||
blob.write_bytes(b"GGUF test payload")
|
||||
model = tmp_path / "custom.gguf"
|
||||
model.symlink_to(blob)
|
||||
symlink_or_skip(model, blob)
|
||||
monkeypatch.setenv("OMNIVOICE_AUDIOCPP_MODEL", str(model))
|
||||
|
||||
resolved = bootstrap.resolve_model_file()
|
||||
@@ -601,7 +605,7 @@ def test_file_override_materializes_hf_style_symlink(
|
||||
|
||||
|
||||
def test_directory_override_materializes_hf_style_symlink(
|
||||
tmp_path, monkeypatch, app_modules,
|
||||
tmp_path, monkeypatch, app_modules, symlink_or_skip,
|
||||
):
|
||||
bootstrap = app_modules.bootstrap
|
||||
blob = tmp_path / "blob"
|
||||
@@ -609,7 +613,7 @@ def test_directory_override_materializes_hf_style_symlink(
|
||||
model_dir = tmp_path / "models"
|
||||
model_dir.mkdir()
|
||||
model = model_dir / bootstrap.DEFAULT_PACKAGE
|
||||
model.symlink_to(blob)
|
||||
symlink_or_skip(model, blob)
|
||||
monkeypatch.setenv("OMNIVOICE_AUDIOCPP_MODEL", str(model_dir))
|
||||
|
||||
resolved = bootstrap.resolve_model_file()
|
||||
|
||||
@@ -570,3 +570,23 @@ def _restore_config_paths_after_reload():
|
||||
for const, value in before.items():
|
||||
if isinstance(getattr(mod, const, None), str) and getattr(mod, const) != value:
|
||||
setattr(mod, const, value)
|
||||
|
||||
|
||||
# ── Symlinks on a stock Windows checkout ───────────────────────────────────
|
||||
# Creating a symlink on Windows needs SeCreateSymbolicLinkPrivilege, which a
|
||||
# normal user account does not hold unless Developer Mode is on. Hosted CI
|
||||
# runs elevated, so unguarded `Path.symlink_to` / `os.symlink` calls pass
|
||||
# there and hand a Windows contributor a suite that fails on their machine
|
||||
# for reasons that have nothing to do with their change (WinError 1314). Tests
|
||||
# that need a real symlink take this fixture, so the environment that cannot
|
||||
# make one skips instead of erroring. Coverage still holds: the full pytest
|
||||
# job runs on Linux.
|
||||
@pytest.fixture
|
||||
def symlink_or_skip():
|
||||
def _make(link, target, *, target_is_directory: bool = False):
|
||||
try:
|
||||
os.symlink(target, link, target_is_directory=target_is_directory)
|
||||
except (OSError, NotImplementedError) as exc: # pragma: no cover - Windows-only
|
||||
pytest.skip(f"symlinks unavailable in this environment: {exc}")
|
||||
|
||||
return _make
|
||||
|
||||
@@ -209,13 +209,13 @@ def test_export_404_when_source_gone(client, outputs_dir, tmp_path, authorize_de
|
||||
|
||||
|
||||
def test_export_symlink_inside_outputs_pointing_outside_is_rejected(
|
||||
client, outputs_dir, tmp_path, authorize_destination
|
||||
client, outputs_dir, tmp_path, authorize_destination, symlink_or_skip
|
||||
):
|
||||
# A symlink planted in OUTPUTS_DIR must not let /export read arbitrary
|
||||
# files: realpath resolves it outside the root, failing containment.
|
||||
secret = tmp_path / "secret.txt"
|
||||
secret.write_bytes(b"credentials")
|
||||
(outputs_dir / "innocent.wav").symlink_to(secret)
|
||||
symlink_or_skip(outputs_dir / "innocent.wav", secret)
|
||||
|
||||
r = client.post("/export", json={
|
||||
"source_filename": "innocent.wav",
|
||||
|
||||
@@ -334,12 +334,12 @@ def test_clear_temp_removes_only_app_owned_entries(tmp_path):
|
||||
assert (tmp / "keep.txt").exists()
|
||||
|
||||
|
||||
def test_clear_temp_unlinks_symlinks_without_following(tmp_path):
|
||||
def test_clear_temp_unlinks_symlinks_without_following(tmp_path, symlink_or_skip):
|
||||
tmp = tmp_path / "tmp"
|
||||
target = tmp_path / "precious"
|
||||
_write(str(target / "data.bin"), 50)
|
||||
os.makedirs(tmp, exist_ok=True)
|
||||
os.symlink(str(target), str(tmp / "omnivoice_link"))
|
||||
symlink_or_skip(str(tmp / "omnivoice_link"), str(target), target_is_directory=True)
|
||||
|
||||
res = storage_report.clear_temp(str(tmp))
|
||||
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
"""`docs/STRUCTURE.md` carries counts of the trees it describes.
|
||||
|
||||
Counts rot silently: the doc shipped "79 modules of business logic" when
|
||||
`backend/services/` held 78, and nothing failed. A number in a doc that no
|
||||
test reads is a number that is wrong the week after it is written, so the
|
||||
counts are pinned here rather than trusted to review attention.
|
||||
|
||||
Per CLAUDE.md's token-economy rule, mechanical claims like this belong in a
|
||||
deterministic test, not in agent or reviewer effort. When you add a router or
|
||||
a service, update the doc — this test tells you which line.
|
||||
"""
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
REPO = Path(__file__).resolve().parents[1]
|
||||
STRUCTURE = REPO / "docs" / "STRUCTURE.md"
|
||||
|
||||
|
||||
def _modules(directory: Path) -> set:
|
||||
"""Importable modules directly under `directory`: flat `*.py` files plus
|
||||
subpackages. `__init__.py` is packaging, not a module of its own."""
|
||||
names = {p.stem for p in directory.glob("*.py") if p.stem != "__init__"}
|
||||
names |= {p.name for p in directory.iterdir() if (p / "__init__.py").is_file()}
|
||||
return names
|
||||
|
||||
|
||||
def _documented(pattern: str) -> int:
|
||||
text = STRUCTURE.read_text(encoding="utf-8")
|
||||
match = re.search(pattern, text)
|
||||
assert match, f"docs/STRUCTURE.md no longer states a count for {pattern!r}"
|
||||
return int(match.group(1))
|
||||
|
||||
|
||||
def test_router_count_matches_the_tree():
|
||||
actual = len(_modules(REPO / "backend" / "api" / "routers"))
|
||||
assert _documented(r"(\d+) routers, auto-included") == actual, (
|
||||
f"docs/STRUCTURE.md says N routers; backend/api/routers/ has {actual}"
|
||||
)
|
||||
|
||||
|
||||
def test_service_count_matches_the_tree():
|
||||
actual = len(_modules(REPO / "backend" / "services"))
|
||||
assert _documented(r"(\d+) modules of business logic") == actual, (
|
||||
f"docs/STRUCTURE.md says N services; backend/services/ has {actual}"
|
||||
)
|
||||
|
||||
|
||||
def test_every_engine_adapter_is_listed():
|
||||
engines = REPO / "backend" / "engines"
|
||||
actual = {p.name for p in engines.iterdir() if (p / "__init__.py").is_file()}
|
||||
text = STRUCTURE.read_text(encoding="utf-8")
|
||||
listed = text[text.index("per-engine adapters:") :][:400]
|
||||
missing = sorted(name for name in actual if name not in listed)
|
||||
assert not missing, f"docs/STRUCTURE.md does not list engine adapter(s): {missing}"
|
||||
@@ -0,0 +1,114 @@
|
||||
"""Every symlink a test creates must be able to skip instead of erroring.
|
||||
|
||||
Creating a symlink on Windows requires SeCreateSymbolicLinkPrivilege, which a
|
||||
normal account does not hold without Developer Mode. Hosted CI runs elevated,
|
||||
so an unguarded `Path.symlink_to` / `os.symlink` passes there and fails only
|
||||
on a contributor's Windows checkout, with `WinError 1314` and no connection to
|
||||
their change. Five tests in `test_audiocpp_backend.py` plus one each in
|
||||
`test_exports_api.py` and `test_storage_report.py` shipped exactly that.
|
||||
|
||||
`tests/conftest.py` provides the `symlink_or_skip` fixture. This test keeps new
|
||||
call sites on it — a mechanical rule, so it lives in CI rather than in reviewer
|
||||
attention (CLAUDE.md token economy).
|
||||
|
||||
Guarded means one of: the `symlink_or_skip` fixture, a call inside a `try`
|
||||
(the module already handles the failure), or a helper that skips on its own.
|
||||
"""
|
||||
|
||||
import ast
|
||||
from pathlib import Path
|
||||
|
||||
TESTS = Path(__file__).resolve().parent
|
||||
|
||||
# Helpers that already skip on failure themselves; calls inside them are the
|
||||
# guard, not a violation.
|
||||
GUARD_FUNCTIONS = {"_symlink_or_skip", "_make", "symlink_or_skip"}
|
||||
|
||||
|
||||
def _creates_symlink(node: ast.AST) -> bool:
|
||||
if not isinstance(node, ast.Call):
|
||||
return False
|
||||
func = node.func
|
||||
if isinstance(func, ast.Attribute):
|
||||
if func.attr == "symlink_to":
|
||||
return True
|
||||
if func.attr == "symlink" and isinstance(func.value, ast.Name) and func.value.id == "os":
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _has_skipif(decorators) -> bool:
|
||||
for decorator in decorators:
|
||||
for node in ast.walk(decorator):
|
||||
if isinstance(node, ast.Attribute) and node.attr in ("skipif", "skip"):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _module_is_skippable(tree: ast.Module) -> bool:
|
||||
"""A module-level `pytestmark = pytest.mark.skipif(...)` guards every test
|
||||
in the file, so a symlink call inside one is already conditional."""
|
||||
for node in tree.body:
|
||||
if not isinstance(node, ast.Assign):
|
||||
continue
|
||||
names = {t.id for t in node.targets if isinstance(t, ast.Name)}
|
||||
if "pytestmark" in names and _has_skipif([node.value]):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _calls_a_guard(function: ast.AST) -> bool:
|
||||
"""The test already ran a skipping helper, so reaching a later raw call
|
||||
means the environment demonstrably supports symlinks."""
|
||||
for node in ast.walk(function):
|
||||
if isinstance(node, ast.Call) and isinstance(node.func, ast.Name):
|
||||
if node.func.id in GUARD_FUNCTIONS:
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _unguarded(path: Path) -> list:
|
||||
tree = ast.parse(path.read_text(encoding="utf-8"))
|
||||
if _module_is_skippable(tree):
|
||||
return []
|
||||
parents = {}
|
||||
for parent in ast.walk(tree):
|
||||
for child in ast.iter_child_nodes(parent):
|
||||
parents[child] = parent
|
||||
|
||||
bad = []
|
||||
for node in ast.walk(tree):
|
||||
if not _creates_symlink(node):
|
||||
continue
|
||||
cursor = node
|
||||
guarded = False
|
||||
while cursor in parents:
|
||||
cursor = parents[cursor]
|
||||
if isinstance(cursor, ast.Try):
|
||||
guarded = True
|
||||
break
|
||||
if isinstance(cursor, (ast.FunctionDef, ast.AsyncFunctionDef)):
|
||||
if cursor.name in GUARD_FUNCTIONS or _has_skipif(cursor.decorator_list):
|
||||
guarded = True
|
||||
break
|
||||
if cursor.name.startswith("test_") and _calls_a_guard(cursor):
|
||||
guarded = True
|
||||
break
|
||||
if not guarded:
|
||||
bad.append(node.lineno)
|
||||
return bad
|
||||
|
||||
|
||||
def test_no_unguarded_symlink_creation_in_tests():
|
||||
offenders = {}
|
||||
for path in sorted(TESTS.rglob("test_*.py")):
|
||||
if path.name == Path(__file__).name:
|
||||
continue
|
||||
lines = _unguarded(path)
|
||||
if lines:
|
||||
offenders[str(path.relative_to(TESTS.parent))] = lines
|
||||
assert not offenders, (
|
||||
"These tests create a symlink without a way to skip, so they fail with "
|
||||
"WinError 1314 on a stock Windows checkout. Take the `symlink_or_skip` "
|
||||
f"fixture from tests/conftest.py instead: {offenders}"
|
||||
)
|
||||
Reference in New Issue
Block a user