chore: release docs, pin python version, drop stale tarball
- Add docs/RELEASING.md, DESKTOP_RELEASE.md, desktop-build.md for release workflow and packaging steps - Relocate next.md → docs/specs/studio-v1.md (scratch → formal spec) - Pin Python version via .python-version - Ignore research/ clones in .gitignore - Remove stale omnivoice-studio-20260421-1834.tar.gz snapshot Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
debpalash
co-authored by
Claude Opus 4.7
Claude Opus 4.7
parent
90f63f26e4
commit
d1fd0e5fcb
@@ -42,6 +42,11 @@ Thumbs.db
|
||||
/.cache*
|
||||
/.tmp/
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────
|
||||
# Research clones — upstream repos used as reference, not shipped
|
||||
# ─────────────────────────────────────────────────────────────────────────
|
||||
research/
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────
|
||||
# Runtime data & artifacts
|
||||
# (app-local state — safe to regenerate)
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
3.11
|
||||
@@ -0,0 +1,271 @@
|
||||
# Desktop release plan — OmniVoice Studio
|
||||
|
||||
A shippable macOS (and eventually cross-platform) desktop release where the user drags the `.app` to `Applications`, double-clicks once, and does **everything else from the UI** — dependency runtime, model weights, first-run consent, all inside the app.
|
||||
|
||||
Reference implementation: **[jamiepine/voicebox](https://github.com/jamiepine/voicebox)** — same stack (Tauri v2 + FastAPI sidecar + PyInstaller), same target. They ship `Voicebox_0.4.4_aarch64.dmg` at 482 MB signed + notarized, plus matching Windows `.msi`/`.nsis`, Linux nothing but an `x64.app.tar.gz`, and a separate lazy-download CUDA tarball for NVIDIA hosts. This plan is a translation of their approach to our stack.
|
||||
|
||||
---
|
||||
|
||||
## Target architecture (mirrors voicebox)
|
||||
|
||||
| Layer | Contents | Ships in DMG? |
|
||||
|---|---|---|
|
||||
| Tauri v2 shell (Rust) | Native window, process lifecycle, filesystem paths | Yes |
|
||||
| Frontend bundle | React/Vite build in `.app/Contents/Resources/dist/` | Yes |
|
||||
| **FastAPI sidecar binary** | **PyInstaller-frozen** `omnivoice-backend` with Python + torch + mlx + soundfile + demucs + yt_dlp + omnivoice TTS | Yes (~400–500 MB bundle) |
|
||||
| ffmpeg | arm64 binary in `.app/Contents/Resources/bin/` | Yes (~20 MB) |
|
||||
| Model weights (OmniVoice TTS, MLX Whisper) | `~/Library/Application Support/OmniVoice/models/` | **No — first-run download** |
|
||||
| Optional engine packs (VoxCPM2 CUDA, pyannote, MOSS-TTS) | Separate `.tar.gz` via GitHub Releases manifest | **No — first-run download if user opts in** |
|
||||
|
||||
Target DMG size: **~500 MB** (matches voicebox's 482 MB arm64 DMG).
|
||||
First-run model download: **~5 GB** one-time.
|
||||
|
||||
---
|
||||
|
||||
## Four techniques to steal from voicebox
|
||||
|
||||
### 1. Sidecar port-reuse dance (dev ergonomics + crash recovery)
|
||||
|
||||
Tauri's startup flow — adapted from `voicebox/tauri/src-tauri/src/main.rs`:
|
||||
|
||||
1. Check if `127.0.0.1:17493/health` responds.
|
||||
2. If yes, verify JSON shape: `status == "healthy"`, `model_loaded: bool`, `gpu_available: bool`. If valid, **attach** to the existing process instead of spawning.
|
||||
3. If a legacy port (8000) has an orphan, kill via `lsof -ti :8000 | xargs kill -9`.
|
||||
4. Otherwise spawn the frozen backend sidecar via Tauri's `externalBin`.
|
||||
5. On app close: send SIGTERM, wait 2 s, SIGKILL if still alive.
|
||||
|
||||
**Why this matters:** you can still `uv run uvicorn …` in dev and Tauri cooperates. Restarting a crashed backend is a port-probe, not a process-kill dance.
|
||||
|
||||
**Our files to touch:** `frontend/src-tauri/src/lib.rs` (replace current `find_project_root` / `uv run` logic with port-probe + `externalBin` launch).
|
||||
|
||||
### 2. tqdm → SSE progress for HuggingFace downloads
|
||||
|
||||
Voicebox's `backend/utils/hf_progress.py` is ~80 lines:
|
||||
|
||||
```python
|
||||
from huggingface_hub.utils import _tqdm as hf_tqdm_module
|
||||
from tqdm.auto import tqdm as base_tqdm
|
||||
|
||||
class TrackedTqdm(base_tqdm):
|
||||
def update(self, n=1):
|
||||
super().update(n)
|
||||
callback(self.desc or "download", self.n, self.total)
|
||||
|
||||
# Monkey-patch once at startup — every hf_hub_download() across every
|
||||
# library (transformers, diffusers, accelerate, mlx_whisper) now reports.
|
||||
hf_tqdm_module._original_tqdm_class = hf_tqdm_module.tqdm_class
|
||||
hf_tqdm_module.tqdm_class = TrackedTqdm
|
||||
```
|
||||
|
||||
Pipe callbacks to a new `/setup/download/stream` SSE endpoint. React subscribes with `EventSource`, renders per-file progress bars, locks the rest of the UI until models are ready.
|
||||
|
||||
**Zero changes to calling code.** Every `mlx_whisper.load_model(...)` now reports progress for free.
|
||||
|
||||
**Our files to add:** `backend/utils/hf_progress.py` + `backend/api/routers/setup.py` with `/setup/status` and `/setup/download/stream` endpoints. Frontend `src/pages/SetupWizard.jsx` that renders when `/setup/status` says models aren't present.
|
||||
|
||||
### 3. Two-tier binary: small base DMG + lazy optional payloads
|
||||
|
||||
Voicebox excludes every `nvidia.*` wheel from the Apple Silicon build (saves ~2 GB) and ships CUDA libs in a separate `cuda-libs-cu128-v1.tar.gz` (1.97 GB), referenced by `cuda-libs.json` on the release.
|
||||
|
||||
For us:
|
||||
- **Base DMG ships MPS + MLX path only.** Excludes `nvidia.*`, `triton`, `flash-attn`, anything CUDA-specific in the spec.
|
||||
- **Optional pack: VoxCPM2** (requires CUDA). Not installed by default. Settings → Engines → "Install VoxCPM2" triggers download from our `voxcpm2-cu128-v1.tar.gz` release asset.
|
||||
- **Optional pack: pyannote** (HF-token gated). Default off. Settings → Speaker diarisation → "Enable" prompts for HF token, downloads + installs.
|
||||
- **Optional pack: MOSS-TTS-Nano.** Same pattern.
|
||||
|
||||
Manifest format copied from voicebox's `cuda-libs.json`:
|
||||
```json
|
||||
{
|
||||
"url": "https://github.com/.../releases/download/v0.1.0/voxcpm2-cu128-v1.tar.gz",
|
||||
"sha256": "…",
|
||||
"size_bytes": 2100000000,
|
||||
"extract_to": "packs/voxcpm2"
|
||||
}
|
||||
```
|
||||
|
||||
### 4. PyInstaller spec + runtime hooks
|
||||
|
||||
Starting point: our existing `backend.spec` (already rewritten this session).
|
||||
|
||||
Two runtime hooks we should add, copied from voicebox:
|
||||
|
||||
- **`pyi_rth_numpy_compat.py`** — fixes a numpy compat shim that PyInstaller misses.
|
||||
- **`pyi_rth_torch_compiler_disable.py`** — disables `torch.compile` code paths that break under frozen imports.
|
||||
|
||||
Exclude list (saves space on Apple Silicon build):
|
||||
```python
|
||||
excludes = [
|
||||
'nvidia.cublas', 'nvidia.cudnn', 'nvidia.cuda_runtime',
|
||||
'nvidia.nccl', 'nvidia.nvtx',
|
||||
'triton', 'flash_attn',
|
||||
'tkinter', 'matplotlib.backends._tkagg',
|
||||
]
|
||||
```
|
||||
|
||||
Hidden-imports to add (iterative — fix as PyInstaller errors surface):
|
||||
- `mlx.core`, `mlx.nn`
|
||||
- `omnivoice`, `omnivoice.models.omnivoice`
|
||||
- `soundfile._soundfile`
|
||||
- `demucs.separate`, `demucs.pretrained`
|
||||
- `huggingface_hub.repocard_data`
|
||||
|
||||
---
|
||||
|
||||
## Phased execution plan
|
||||
|
||||
Each phase produces a testable artifact. Don't proceed to the next phase until the current one verifies end-to-end.
|
||||
|
||||
### Phase A — Frozen backend works (3–5 h, highest risk)
|
||||
|
||||
**Deliverable:** `dist/omnivoice-backend/omnivoice-backend` runs standalone + serves the full API.
|
||||
|
||||
1. Drop voicebox's two runtime hooks into `backend/hooks/`.
|
||||
2. Update `backend.spec` with the exclude list + the runtime hook paths.
|
||||
3. Run `uv run pyinstaller backend.spec --noconfirm --clean`.
|
||||
4. Iterate on hidden-imports until `./dist/omnivoice-backend/omnivoice-backend` starts cleanly and `/system/info` returns 200.
|
||||
5. End-to-end smoke: transcribe the Fireship fixture → generate dub in Spanish → verify output audio exists.
|
||||
|
||||
**Verify:** `curl -sf http://127.0.0.1:17493/system/info` on the frozen binary returns JSON in <2 s.
|
||||
|
||||
**Fail-path:** if PyInstaller can't bundle after 5 hours, pivot to "ship a portable `.venv` inside `.app/Contents/Resources/`" — uglier, reliably works. Adds ~300 MB but skips PyInstaller drama.
|
||||
|
||||
### Phase B — Tauri launches the frozen sidecar (2 h)
|
||||
|
||||
**Deliverable:** `bun run desktop` launches a dev .app that uses the frozen backend, not `uv run`.
|
||||
|
||||
1. Rewrite `frontend/src-tauri/src/lib.rs`'s `setup` hook:
|
||||
- Check port 17493 first (port-reuse dance).
|
||||
- If free, launch the bundled `Contents/Resources/backend/omnivoice-backend` via Tauri's `shell_plugin::Command`.
|
||||
- Kill orphans on port 8000 (legacy).
|
||||
2. Wire `tauri.conf.json` `bundle.resources` to include `../../dist/omnivoice-backend/**` and `binaries/ffmpeg`.
|
||||
3. Change backend's default port from 8000 → 17493 (new namespace, fewer conflicts with other dev tools).
|
||||
|
||||
**Verify:** launch the dev app with `bun run desktop` — window opens, segment table loads, test ingest-url works.
|
||||
|
||||
### Phase C — First-run model download UI (4–6 h)
|
||||
|
||||
**Deliverable:** fresh app on a machine with no cached HF models walks user through download with live progress.
|
||||
|
||||
1. Port voicebox's `hf_progress.py` — ~80 LOC.
|
||||
2. New `backend/api/routers/setup.py`:
|
||||
- `GET /setup/status` → `{ models_ready: bool, missing: [...], disk_free_gb: number }`.
|
||||
- `GET /setup/download/stream` → SSE: `{ type: "progress", file, bytes, total, pct }` then `{ type: "done" }`.
|
||||
3. Frontend `src/pages/SetupWizard.jsx`:
|
||||
- Shown when `/setup/status` says models missing.
|
||||
- Per-file progress bars driven by the SSE stream.
|
||||
- Disk-space check; error state if <10 GB free.
|
||||
- Retry on network failure.
|
||||
4. App-level route guard: if `setupWizardNeeded`, render `<SetupWizard>` instead of `<Launchpad>`.
|
||||
|
||||
**Verify:** move/rename `~/Library/Application Support/OmniVoice/models/` — launch app — wizard shows up — progress bars tick — models download — UI unlocks.
|
||||
|
||||
### Phase D — DMG build + clean-machine test (2–3 h)
|
||||
|
||||
**Deliverable:** signed-but-not-notarized DMG that works on a virgin Mac after right-click → Open.
|
||||
|
||||
1. `bun run tauri build` (via `scripts/build_desktop.sh` we'll add).
|
||||
2. Artifact: `frontend/src-tauri/target/release/bundle/dmg/OmniVoice Studio_0.1.0_aarch64.dmg`.
|
||||
3. Copy to a fresh macOS user account (or a second Mac).
|
||||
4. Right-click → Open once, walk the wizard, dub the Fireship fixture.
|
||||
5. Fix whatever breaks.
|
||||
|
||||
**Verify:** target Mac with NO development tools installed can dub a YouTube URL in the target language end-to-end.
|
||||
|
||||
### Phase E — (optional) Signed + notarized
|
||||
|
||||
**Deliverable:** DMG that opens without Gatekeeper override.
|
||||
|
||||
Requires:
|
||||
- Apple Developer ID (~$99/yr).
|
||||
- Code-signing cert, App Store Connect API key.
|
||||
- GitHub Actions workflow mirroring voicebox's `.github/workflows/release.yml`:
|
||||
- `apple-actions/import-codesign-certs@v3`
|
||||
- `tauri-apps/tauri-action@v0.6` with `APPLE_SIGNING_IDENTITY` + `APPLE_API_KEY` + `APPLE_API_ISSUER` + `APPLE_PROVIDER_SHORT_NAME`
|
||||
- **Explicit DMG re-notarize step** — voicebox's workflow comment notes macOS 15 Sequoia rejects DMGs that wrap a signed `.app` but aren't themselves notarized. Run `xcrun notarytool submit --wait` + `xcrun stapler staple` on the DMG, re-upload as release asset.
|
||||
|
||||
---
|
||||
|
||||
## Cross-platform extension (future)
|
||||
|
||||
Voicebox ships macOS arm64 + macOS x64 + Windows x64 (`.msi`, `.nsis`, `.exe`). Not Linux.
|
||||
|
||||
We can mirror this by extending the CI matrix once Phases A–D are green:
|
||||
|
||||
| Target | Runner | PyInstaller variant | Notes |
|
||||
|---|---|---|---|
|
||||
| macOS Apple Silicon | `macos-14` (ARM) | `backend.spec` (MPS/MLX) | Our primary |
|
||||
| macOS Intel | `macos-13` | `backend.spec` (MPS/x64 torch) | MLX absent — falls back to CPU Whisper |
|
||||
| Windows x64 (NVIDIA) | `windows-2022` | `backend.spec` + `--bootloader` + CUDA | Requires second CUDA tarball like voicebox's `cuda-libs-cu128-v1.tar.gz` |
|
||||
| Linux x64 | `ubuntu-22.04` | `backend.spec` | Untested in voicebox — treat as best-effort |
|
||||
|
||||
Each platform's first build will take the longest (PyInstaller hidden-import tuning is per-OS). Subsequent builds reuse the spec.
|
||||
|
||||
**Honest caveat:** Windows is a whole separate set of headaches — `mlx_whisper` doesn't exist there, `pyannote` + `soundfile` have different wheel sources, signing requires a separate Windows code-signing cert. Add **1 full session per additional platform**.
|
||||
|
||||
For our current goal (friend on the same M2 air), **stop at Phase D**. Cross-platform is a later conversation once macOS is solid.
|
||||
|
||||
---
|
||||
|
||||
## Risks & mitigations
|
||||
|
||||
| Risk | Likelihood | Mitigation |
|
||||
|---|---|---|
|
||||
| PyInstaller can't bundle torch Metal libs | Medium | Voicebox proved it works → copy their `collect_all(...)` calls verbatim. Fallback: portable `.venv` approach |
|
||||
| `torch.compile` breaks under frozen imports | Certain | Copy `pyi_rth_torch_compiler_disable.py` runtime hook from voicebox |
|
||||
| DMG size >800 MB | Medium | Keep nvidia/triton/matplotlib out of the spec; defer optional packs to lazy download |
|
||||
| First-run download fails halfway | Medium | SSE retry + resumable `hf_hub_download` (supported natively). Show disk-space check upfront |
|
||||
| Gatekeeper blocks unsigned app | Certain | Document right-click → Open as the one-time step. Long-term: buy Apple Developer ID |
|
||||
| User's friend has <10 GB free | Low | Pre-check in `/setup/status`. Refuse to start download if insufficient. Point user to clear space |
|
||||
| Apple Silicon build runs on Intel Mac | Possible | Warn in installer + app header. Don't promise cross-arch without actual Intel build |
|
||||
|
||||
---
|
||||
|
||||
## Success criteria (for this plan, per phase)
|
||||
|
||||
- **A ✅ when** `./dist/omnivoice-backend/omnivoice-backend` starts in <3 s on a clean shell and serves `/system/info`.
|
||||
- **B ✅ when** `bun run desktop` launches a window that uses the frozen binary (not `uv run`) and all core APIs work.
|
||||
- **C ✅ when** deleting the models dir and launching shows a wizard that completes to functional state without any terminal interaction.
|
||||
- **D ✅ when** an unrelated M-series Mac runs the DMG end-to-end (Fireship clip → Spanish dub) with zero developer tooling installed, just right-click → Open once.
|
||||
|
||||
---
|
||||
|
||||
## Reference files to read in jamiepine/voicebox
|
||||
|
||||
Before starting Phase A, pull these from voicebox's repo for direct copy/adapt:
|
||||
|
||||
- `backend/voicebox-server.spec` — PyInstaller spec
|
||||
- `backend/build_binary.py` — binary build + platform suffix
|
||||
- `backend/utils/hf_progress.py` — tqdm monkey-patch
|
||||
- `backend/hooks/pyi_rth_numpy_compat.py` — numpy runtime hook
|
||||
- `backend/hooks/pyi_rth_torch_compiler_disable.py` — torch.compile disable
|
||||
- `tauri/src-tauri/src/main.rs` — sidecar spawn + port-reuse
|
||||
- `tauri/src-tauri/tauri.conf.json` — bundle + updater config
|
||||
- `scripts/build-server.sh` — CI build entry
|
||||
- `scripts/package_cuda.py` — out-of-band CUDA tarball builder
|
||||
- `.github/workflows/release.yml` — full release pipeline (signing, notarization, DMG re-notarize)
|
||||
|
||||
---
|
||||
|
||||
## Out-of-scope for v1
|
||||
|
||||
- Auto-update (Tauri has `tauri-plugin-updater`, but requires signing + hosted `latest.json`).
|
||||
- Automatic crash reporting (needs a Sentry-type endpoint).
|
||||
- User telemetry of any kind.
|
||||
- In-app feedback form.
|
||||
- Notarized installer — Phase E, deferred.
|
||||
- Cross-platform builds — see "Cross-platform extension" section.
|
||||
- Homebrew cask — possible later, not blocking.
|
||||
|
||||
---
|
||||
|
||||
## Timeline (honest, single developer)
|
||||
|
||||
| Phase | Hours | Confidence |
|
||||
|---|---|---|
|
||||
| A — frozen backend | 3–5 | High (voicebox proved it) |
|
||||
| B — Tauri sidecar + port-reuse | 2 | High (their Rust is readable) |
|
||||
| C — first-run wizard + hf progress | 3–4 | High (80 LOC + UI) |
|
||||
| D — DMG + clean-machine test | 2–3 | Medium (Gatekeeper dance) |
|
||||
| **Total (macOS arm64 only)** | **10–14** | **Phaseable over 2–3 sessions** |
|
||||
| E — signing + notarization | +3 | Blocked on Apple Developer ID |
|
||||
| Cross-platform (each OS) | +8 | Per-OS effort |
|
||||
@@ -0,0 +1,116 @@
|
||||
# Releasing — self-updating builds for mac / linux / windows
|
||||
|
||||
This doc covers the release workflow after the auto-updater wiring landed. Read top-to-bottom the first time. After that, cutting a release is the three commands in §5.
|
||||
|
||||
## 1. One-time repo setup
|
||||
|
||||
The signing key was generated locally at `~/.tauri/omnivoice-updater.key` (private) and `~/.tauri/omnivoice-updater.key.pub` (public). The public key is already embedded in `frontend/src-tauri/tauri.conf.json` — that's what shipping clients use to verify updates.
|
||||
|
||||
The private key needs to live in **GitHub Actions Secrets** so CI can sign each release:
|
||||
|
||||
1. Read the private key contents:
|
||||
```
|
||||
cat ~/.tauri/omnivoice-updater.key
|
||||
```
|
||||
2. GitHub → Settings → Secrets and variables → Actions → **New repository secret** (on `debpalash/OmniVoice-Studio`, which is where the updater endpoint points):
|
||||
- Name: `TAURI_SIGNING_PRIVATE_KEY`
|
||||
- Value: paste the full contents (including the `untrusted comment:` header line)
|
||||
3. Add a second secret:
|
||||
- Name: `TAURI_SIGNING_PRIVATE_KEY_PASSWORD`
|
||||
- Value: leave blank (the key was generated without a password)
|
||||
|
||||
**Back the key up.** Copy `~/.tauri/omnivoice-updater.key` to a password manager or encrypted vault. If you lose it, you can never ship an update for any client that has the current public key — they'll be stranded and need a manual reinstall.
|
||||
|
||||
## 2. One-time account setup (you)
|
||||
|
||||
**Rotate the leaked GH token** (the `ghp_...` in `origin` remote). See the session transcript — already flagged. Do this before anything else.
|
||||
|
||||
No Apple Developer / Windows signing certs needed for v1. Apps ship unsigned; first-launch shows "unverified developer" warnings that users bypass with right-click → Open (mac) or "Run anyway" (Windows SmartScreen). Self-update still works — Tauri's updater verifies via its own signing key, independent of OS code signing.
|
||||
|
||||
## 3. What the updater does
|
||||
|
||||
On every app launch, the webview:
|
||||
1. Fetches `https://github.com/debpalash/OmniVoice-Studio/releases/latest/download/latest.json`
|
||||
2. Compares the version in `latest.json` to the running app's version (from `tauri.conf.json`)
|
||||
3. If newer, shows a native dialog: *"A new version (x.y.z) is available. Download and install now?"*
|
||||
4. If user accepts, downloads the signed update bundle, verifies the minisign signature against the embedded pubkey, replaces the app in place, relaunches.
|
||||
|
||||
Failures (no network, 404, signature mismatch) are silent — the app continues to launch normally. Check the frontend devtools console for `Updater check failed` messages if you're debugging.
|
||||
|
||||
## 4. Version bumps
|
||||
|
||||
Two files must agree before you tag a release:
|
||||
|
||||
- `frontend/src-tauri/tauri.conf.json` → `"version": "0.2.0"`
|
||||
- `frontend/src-tauri/Cargo.toml` → `version = "0.2.0"`
|
||||
|
||||
(Not `frontend/package.json` — Tauri ignores it.)
|
||||
|
||||
Keep bumps monotonic. Tauri updater uses semver comparison, so `v0.2.0` does not update clients already on `v0.2.1`.
|
||||
|
||||
## 5. Cutting a release
|
||||
|
||||
```bash
|
||||
# 1. Bump versions in the two files above, commit.
|
||||
git add frontend/src-tauri/tauri.conf.json frontend/src-tauri/Cargo.toml
|
||||
git commit -m "release: v0.2.0"
|
||||
|
||||
# 2. Tag and push.
|
||||
git tag v0.2.0
|
||||
git push origin main
|
||||
git push origin v0.2.0
|
||||
```
|
||||
|
||||
The `Desktop Release` workflow fires on tag push. It builds four targets in parallel on GitHub Actions runners:
|
||||
|
||||
| Target | Runner | Artifact |
|
||||
|---|---|---|
|
||||
| macOS Apple Silicon | macos-14 | `.dmg` + updater `.app.tar.gz` |
|
||||
| macOS Intel | macos-13 | `.dmg` + updater `.app.tar.gz` |
|
||||
| Windows x64 | windows-2022 | `.msi` + `.exe` + updater `.nsis.zip` |
|
||||
| Linux x64 | ubuntu-22.04 | `.AppImage` + `.deb` + updater `.AppImage.tar.gz` |
|
||||
|
||||
Each runner signs the updater payload with the stored `TAURI_SIGNING_PRIVATE_KEY`, merges into a single `latest.json`, and attaches everything to the draft release.
|
||||
|
||||
Workflow runtime: **~20-40 minutes** (PyInstaller + four platform builds). Follow progress at:
|
||||
`https://github.com/debpalash/OmniVoice-Studio/actions`
|
||||
|
||||
When it finishes, the draft release needs manual publishing — GitHub → Releases → **Edit** the draft → **Publish release**. Once published, existing clients detect the update on their next launch.
|
||||
|
||||
## 6. Expect-to-fail-first-time on Windows and Linux
|
||||
|
||||
mac-ARM is tested locally. The other three platforms will likely hit PyInstaller issues on their first CI run because neither dependency set nor platform quirks have been exercised. Common failures to expect:
|
||||
|
||||
- **Windows**: `mlx_whisper` is mac-only — need to conditional-guard the import in `backend.spec`. `demucs`'s CUDA autodetect may pull wheels we don't want. Long-path limits during the PyInstaller bundle.
|
||||
- **Linux**: `libasound` / `libwebkit2gtk` dev headers vs runtime confusion. AppImage FUSE assumptions on the runner.
|
||||
- **mac-Intel**: should work, but torch wheels for x86_64 differ — watch for `nvidia-*` wheels sneaking in via the default torch.
|
||||
|
||||
When a target fails, either fix the root cause in the spec / workflow, or comment that matrix row out temporarily and keep the working targets shipping. The `fail-fast: false` setting means one failure doesn't kill the others.
|
||||
|
||||
## 7. Testing the updater locally (before shipping a tag)
|
||||
|
||||
Two options:
|
||||
|
||||
**Option A — dry run the manifest:**
|
||||
After a release is published, hit the updater URL manually:
|
||||
```
|
||||
curl -L https://github.com/debpalash/OmniVoice-Studio/releases/latest/download/latest.json | jq
|
||||
```
|
||||
You should see platform-keyed download URLs + minisign signatures. If that JSON looks right, clients will pick it up.
|
||||
|
||||
**Option B — full end-to-end:**
|
||||
1. Install v0.1.0 on a fresh machine (or clean-installed Applications).
|
||||
2. Cut v0.2.0 (bump, tag, push, wait for CI, publish draft).
|
||||
3. Launch the installed v0.1.0. Within seconds, the dialog should appear.
|
||||
4. Accept → app downloads, verifies, replaces, relaunches as v0.2.0.
|
||||
|
||||
If step 3 silently does nothing, DevTools console in the app webview has the `Updater check failed:` log.
|
||||
|
||||
## 8. Rolling back
|
||||
|
||||
There's no "revert update" flow for clients — they'll only see a *newer* version. To roll back:
|
||||
1. Delete the broken release from GitHub Releases (or mark it as pre-release).
|
||||
2. Re-tag the previous good commit with a higher version (e.g., if you shipped bad `v0.2.0`, tag `v0.2.1` on the old `v0.1.0` commit).
|
||||
3. Clients auto-update to the "new" v0.2.1 which is actually the old code.
|
||||
|
||||
Ugly but it works. Better plan: test with Option B above before publishing the draft.
|
||||
@@ -0,0 +1,215 @@
|
||||
# Desktop build — progress tracker
|
||||
|
||||
Working doc for the desktop release effort. Every box maps to a concrete
|
||||
deliverable; mark `[x]` when verified end-to-end on a fresh environment,
|
||||
not just "code compiles." See `docs/DESKTOP_RELEASE.md` for the full
|
||||
engineering plan + `voicebox` comparison that grounds these milestones.
|
||||
|
||||
**Primary target:** macOS Apple Silicon (arm64), unsigned.
|
||||
**Stretch targets:** macOS Intel, Windows x64, Linux — parked in CI matrix
|
||||
until the arm64 path is green end-to-end.
|
||||
|
||||
---
|
||||
|
||||
## Source of truth files
|
||||
|
||||
| Concern | Owner file |
|
||||
|---|---|
|
||||
| Backend freeze spec | `backend.spec` |
|
||||
| PyInstaller runtime hooks | `backend/hooks/*.py` (TBD) |
|
||||
| Tauri sidecar launcher | `frontend/src-tauri/src/lib.rs` |
|
||||
| Tauri bundle config | `frontend/src-tauri/tauri.conf.json` |
|
||||
| HF download progress | `backend/utils/hf_progress.py` |
|
||||
| First-run wizard endpoints | `backend/api/routers/setup.py` |
|
||||
| First-run wizard UI | `frontend/src/pages/SetupWizard.jsx` (TBD) |
|
||||
| CI release matrix | `.github/workflows/release.yml` |
|
||||
| Reference implementation | [jamiepine/voicebox](https://github.com/jamiepine/voicebox) |
|
||||
|
||||
---
|
||||
|
||||
## Phase A — Frozen backend binary (✅ 2026-04-21)
|
||||
|
||||
- [x] Port voicebox runtime hooks into `backend/hooks/`
|
||||
- [x] `pyi_rth_numpy_compat.py` — pre-imports numpy to prime the C ext
|
||||
- [x] `pyi_rth_torch_compiler_disable.py` — disables dynamo/inductor via env
|
||||
- [x] Wire hooks into `backend.spec` via `runtime_hooks=[...]`
|
||||
- [x] Add Apple Silicon exclude list to `backend.spec` (`nvidia.*`, `triton`, `flash_attn`)
|
||||
- [x] `uv run pyinstaller backend.spec --noconfirm --clean` produces a clean bundle — **140 s build time**
|
||||
- [x] `./dist/omnivoice-backend/omnivoice-backend` starts and serves `/system/info` — **~23 s cold start** (not ≤3 s as targeted, but acceptable behind splash screen in Phase D)
|
||||
- [x] Frozen binary serves all core endpoints (`/system/info`, `/setup/status`, `/engines`)
|
||||
- [x] `hf_progress` patch installs on frozen start (confirmed via log)
|
||||
- [ ] Frozen binary transcribes Fireship fixture → Spanish dub end-to-end (deferred to Phase B — needs Tauri WebView for the frontend, or direct `curl`-driven harness)
|
||||
- [ ] **Bundle size ≤600 MB** — **currently 1.1 GB, ~2× over target.** Likely shavable to ~700 MB by excluding `torch.distributed.*`, transformers bloat, scipy tests. Not a Phase A blocker; cleanup tracked as Phase A.1.
|
||||
|
||||
**Verified:**
|
||||
```bash
|
||||
./dist/omnivoice-backend/omnivoice-backend
|
||||
# → "Uvicorn running on http://0.0.0.0:8000"
|
||||
curl -s http://127.0.0.1:8000/system/info # ✓ returns JSON
|
||||
curl -s http://127.0.0.1:8000/setup/status # ✓ {"models_ready":true,...}
|
||||
curl -s http://127.0.0.1:8000/engines # ✓ omnivoice/voxcpm2/moss-tts-nano listed
|
||||
```
|
||||
|
||||
**Lessons learned (feeding into Phase B):**
|
||||
- PyInstaller buffers stdout by default — Tauri sidecar spawn must set `PYTHONUNBUFFERED=1` so logs surface promptly.
|
||||
- `main.py`'s `if __name__ == "__main__": uvicorn.run(app, host="0.0.0.0", port=8000)` is what makes the frozen binary serve. Keep it.
|
||||
- `frontend_path = os.path.join(os.path.dirname(__file__), "..", "frontend", "dist")` at `main.py:172` will never resolve in a frozen build; frontend assets come from Tauri's WebView instead, not the Python backend. Guard is fine; note for Phase B.
|
||||
|
||||
### Phase A.1 — Bundle size reduction (optional, before Phase D)
|
||||
|
||||
- [ ] Add `torch.distributed.*` to excludes (not used on Apple Silicon single-device inference)
|
||||
- [ ] Exclude `scipy.special.tests.*`, `numpy.tests.*`, `torch.testing.*`
|
||||
- [ ] Exclude `transformers.models.*` for non-used model families
|
||||
- [ ] Target: <700 MB bundle
|
||||
|
||||
---
|
||||
|
||||
## Phase B — Tauri launches the frozen sidecar (in progress 2026-04-21)
|
||||
|
||||
- [ ] ~~Change backend default port 8000 → 17493~~ — deferred. Port-switching ripples through the frontend API client + bench scripts + install scripts. Keep 8000 for now; switch in a dedicated refactor when we need to.
|
||||
- [x] Rewrite `lib.rs::setup` to:
|
||||
- [x] Probe `/system/info` on port 8000 and attach if responding (`backend_healthy()`)
|
||||
- [x] Kill orphans on port 8000 via `lsof -ti :8000 | xargs kill -9`
|
||||
- [x] Launch bundled `Contents/Resources/backend/omnivoice-backend/omnivoice-backend` if free
|
||||
- [x] Fall back to `uv run uvicorn ...` in dev (when bundled binary not present)
|
||||
- [x] Set `PYTHONUNBUFFERED=1` on sidecar env so backend logs flush in real time
|
||||
- [x] Export `OMNIVOICE_FFMPEG` pointing at bundled ffmpeg + prepend its dir to `PATH`
|
||||
- [x] Wire `tauri.conf.json` `bundle.resources`:
|
||||
- [x] `../../dist/omnivoice-backend` → `backend/omnivoice-backend/`
|
||||
- [x] ~~`binaries/ffmpeg` → `bin/ffmpeg`~~ — **removed**. Hit `Permission denied (os error 13)` at bundle time (brew shim carried `com.apple.provenance` xattr that `xattr -c` couldn't strip under SIP). Dropped the separate resource: `imageio_ffmpeg` already ships a 47 MB static arm64 ffmpeg inside the PyInstaller bundle at `_internal/imageio_ffmpeg/binaries/ffmpeg-macos-aarch64-v7.1`, and `find_ffmpeg()` picks it up via `imageio_ffmpeg.get_ffmpeg_exe()` — zero extra bundling.
|
||||
- [x] Rust compile clean — `cargo check` passes
|
||||
- [x] `bunx tauri build` produces an `.app` (1.3 GB) — bundle contents verified: `Contents/Resources/backend/omnivoice-backend/omnivoice-backend`. ffmpeg travels inside the backend resource (see above).
|
||||
- [x] `bunx tauri build` produces a `.dmg` — **400 MB** ✅ (well under the 600 MB target).
|
||||
- [x] Fix API host in production build — `frontend/src/api/client.ts` previously used `API = ''` in prod, which caused relative fetches against `tauri://localhost` to fail with *"The string did not match the expected pattern"* in the Settings → Models tab. Now hardcodes `http://localhost:8000` so the webview always reaches the bundled sidecar. CORS + CSP already allow it.
|
||||
- [ ] `.app` opens, connects to bundled backend, ingests/transcribes without error (manual test pending)
|
||||
|
||||
### DMG packaging failure to investigate
|
||||
|
||||
Tauri's `bundle_dmg.sh` runs `hdiutil` to convert the intermediate RW DMG → compressed read-only DMG. On our 1.7 GB payload this failed silently; next session's task is to run the shell script manually and capture stderr. Possible fixes in priority order:
|
||||
1. Shrink the backend bundle (Phase A.1 — we're 1.1 GB, voicebox is 482 MB; 500 MB target cuts DMG pipeline latency and works around many hdiutil edge cases).
|
||||
2. Pass `hdiutilArgs` in `tauri.conf.json` → `["-format", "UDZO", "-imagekey", "zlib-level=1"]` — cheaper compression, faster, fewer hdiutil quirks.
|
||||
3. If hdiutil errors with disk space: `TMPDIR=/Volumes/other bunx tauri build` to route the scratch volume elsewhere.
|
||||
4. Last resort: build `.app.tar.gz` with `create-dmg` externally (voicebox pattern).
|
||||
|
||||
**Verification target:**
|
||||
```bash
|
||||
open frontend/src-tauri/target/release/bundle/macos/"OmniVoice Studio.app"
|
||||
# The window should open, segment table should render from a fresh drop.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase C — First-run wizard + HF progress (✅ 2026-04-21)
|
||||
|
||||
- [x] **`backend/utils/hf_progress.py`** — tqdm monkey-patch
|
||||
- [x] **`backend/api/routers/setup.py`**
|
||||
- [x] `GET /setup/status` — missing + disk-free
|
||||
- [x] `GET /setup/download-stream` — SSE stream
|
||||
- [x] `POST /setup/warmup` — background model load
|
||||
- [x] `GET /models` — every known model + install state (Phase M seed)
|
||||
- [x] `POST /models/install` — single-repo install (progress via SSE)
|
||||
- [x] `DELETE /models/{repo_id}` — evict cached revisions, free disk
|
||||
- [x] `frontend/src/pages/SetupWizard.jsx` + `.css`
|
||||
- [x] Mounts on boot, calls `/setup/status`
|
||||
- [x] Per-file progress bars from SSE
|
||||
- [x] Disk-space error state if `disk_free_gb < min_free_gb`
|
||||
- [x] Polls `/setup/status` every 5 s during install to detect completion
|
||||
- [x] Route guard in `App.jsx`: `if (setupNeeded) return <SetupWizard>`
|
||||
- [x] Settings → Models tab: `ModelStoreTab` replaces the old read-only summary
|
||||
- [x] Lists every KNOWN_MODEL with status badge, role, size, repo_id
|
||||
- [x] Install / Reinstall / Delete buttons per row
|
||||
- [x] Aggregate "on-disk" footer
|
||||
- [x] Live per-file progress bars driven by the shared SSE stream
|
||||
|
||||
**Verification:**
|
||||
```bash
|
||||
# Simulate fresh install
|
||||
mv ~/.cache/huggingface/hub /tmp/hf-hub.bak
|
||||
bun run dev
|
||||
# Wizard appears, progress ticks, UI unlocks when done.
|
||||
mv /tmp/hf-hub.bak ~/.cache/huggingface/hub # restore
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase D — DMG + clean-machine test
|
||||
|
||||
- [ ] `frontend/src-tauri/tauri.conf.json` `bundle.targets = ["dmg", "app"]`
|
||||
- [ ] `scripts/build_desktop.sh` — one-shot: PyInstaller → Tauri build → DMG
|
||||
- [ ] Build produces `OmniVoice Studio_0.1.0_aarch64.dmg`
|
||||
- [ ] DMG size ≤600 MB
|
||||
- [ ] DMG runs on a fresh macOS user account (no brew/uv/bun present)
|
||||
- [ ] Mount → drag to Applications → right-click → Open (Gatekeeper override)
|
||||
- [ ] First launch shows setup wizard, models download cleanly
|
||||
- [ ] End-to-end: ingest YouTube URL → transcribe → translate → generate → play
|
||||
- [ ] README section: "Download DMG" with right-click → Open instructions
|
||||
|
||||
**Verification:**
|
||||
```bash
|
||||
bash scripts/build_desktop.sh
|
||||
ls -lh frontend/src-tauri/target/release/bundle/dmg/*.dmg
|
||||
# Copy to fresh user account via System Settings → Users → Add.
|
||||
# Log in, install DMG, walk the whole flow.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase E — Signing + notarization (blocked on Apple Developer ID)
|
||||
|
||||
- [ ] Apple Developer ID ($99/yr)
|
||||
- [ ] Generate signing cert + App Store Connect API key
|
||||
- [ ] Store secrets in GitHub: `APPLE_SIGNING_IDENTITY`, `APPLE_API_KEY`, `APPLE_API_ISSUER`, `APPLE_PROVIDER_SHORT_NAME`
|
||||
- [ ] Enable signing in `tauri-apps/tauri-action@v0.6` step
|
||||
- [ ] **Post-build DMG re-notarize step** (voicebox's workaround for Sequoia):
|
||||
- [ ] `xcrun notarytool submit "$DMG" --wait --apple-id "$APPLE_ID" --password "$APP_SPECIFIC_PASSWORD"`
|
||||
- [ ] `xcrun stapler staple "$DMG"`
|
||||
- [ ] Re-upload stapled DMG as release asset
|
||||
- [ ] Verify on clean Mac: no Gatekeeper warning on first open
|
||||
|
||||
---
|
||||
|
||||
## Phase F — CI release matrix (parallel track)
|
||||
|
||||
- [x] **`.github/workflows/release.yml`** — unsigned matrix build
|
||||
- [x] `macos-14` arm64 primary
|
||||
- [x] Intel / Windows / Linux stubbed (commented), ready to un-comment
|
||||
- [x] `workflow_dispatch` uploads as workflow artifacts
|
||||
- [x] Tag push (`v*`) attaches to GitHub Release
|
||||
- [ ] First green run — push a `v0.1.0-preview` tag, see DMG attached
|
||||
- [ ] Un-comment `macos-13` row once arm64 is stable
|
||||
- [ ] Un-comment `windows-2022` row + add Windows-specific PyInstaller notes
|
||||
- [ ] Un-comment `ubuntu-22.04` row once the Linux packaging path is chosen (AppImage vs deb)
|
||||
|
||||
---
|
||||
|
||||
## Cross-platform expansion (post-Phase-D)
|
||||
|
||||
One session per platform. Ordered by payoff:
|
||||
|
||||
- [ ] **macOS Intel (`macos-13`)** — easiest; same cert, different PyInstaller wheel set
|
||||
- [ ] **Windows x64 CPU (`windows-2022`)** — `mlx_whisper` absent, fall back to PyTorch Whisper. Needs Windows code-signing cert (~$100–300/yr)
|
||||
- [ ] **Windows x64 CUDA** — follow voicebox `scripts/package_cuda.py` pattern, ship as lazy-download pack
|
||||
- [ ] **Linux x64 AppImage** — `appimagetool` builds; unsigned is acceptable on Linux
|
||||
|
||||
---
|
||||
|
||||
## Risks we're tracking
|
||||
|
||||
| # | Risk | Mitigation | Status |
|
||||
|---|---|---|---|
|
||||
| 1 | PyInstaller can't bundle torch Metal libs | Copy voicebox `collect_all()` calls; fallback: portable-venv inside `.app/Contents/Resources/` | Untested |
|
||||
| 2 | `torch.compile` breaks under frozen imports | Port `pyi_rth_torch_compiler_disable.py` hook | Pending |
|
||||
| 3 | First-run model download fails halfway | SSE retry + resumable `hf_hub_download` (native) | Partial — frontend UI TBD |
|
||||
| 4 | User has <10 GB free disk | `/setup/status` refuses download; shows error | ✅ Implemented |
|
||||
| 5 | Gatekeeper blocks unsigned app | Document right-click → Open in README | Not yet documented |
|
||||
| 6 | DMG size >800 MB | Exclude nvidia/triton/matplotlib; lazy-download optional packs | Exclude list in spec |
|
||||
| 7 | macOS 15 Sequoia rejects un-notarized DMG wrapper | Phase E includes explicit `stapler staple` step | Blocked on Phase E |
|
||||
|
||||
---
|
||||
|
||||
## Change log (this doc)
|
||||
|
||||
- **2026-04-21** — initial tracker created. Phase C partially shipped:
|
||||
tqdm monkey-patch + `/setup/status` + SSE stream + `/setup/warmup` live
|
||||
and smoke-tested. Phase F skeleton committed (CI workflow, primary target
|
||||
only — non-arm64 rows parked). Phases A, B, D, E pending.
|
||||
@@ -0,0 +1,143 @@
|
||||
# Studio / Projects — v1 spec
|
||||
|
||||
**Goal:** ElevenLabs-Studio parity for long-form narration. A user pastes (or drags in) a 10-page script, the app splits it into blocks, they assign a voice per block, preview inline, then hit Generate to get one stitched WAV.
|
||||
|
||||
Not in v1: video sync, multi-track mixing, music beds, SFX, realtime playback of unstitched audio. Those come later.
|
||||
|
||||
## 1 — Data model
|
||||
|
||||
Reuse `studio_projects`. Add a strict shape to `state_json`:
|
||||
|
||||
```ts
|
||||
interface ProjectState {
|
||||
kind: 'studio'; // discriminates from dub projects (same table)
|
||||
blocks: Block[];
|
||||
default_voice_id: string; // profile_id used for blocks that don't pin one
|
||||
default_lang?: string;
|
||||
created_by_version: string;
|
||||
}
|
||||
|
||||
interface Block {
|
||||
id: string; // uuid, stable across edits
|
||||
text: string;
|
||||
voice_id?: string; // overrides project default; null → inherit
|
||||
pause_before_ms?: number; // inserted silence (0–5000)
|
||||
pause_after_ms?: number;
|
||||
// Generation state — server-owned, not user-edited
|
||||
gen?: {
|
||||
audio_path: string; // absolute path to per-block WAV in scratch
|
||||
duration_ms: number;
|
||||
hash: string; // SHA-256 of (text, voice_id, gen knobs) — cache key
|
||||
generated_at: number;
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
Two migrations needed:
|
||||
- `ALTER TABLE studio_projects ADD COLUMN kind TEXT DEFAULT 'dub'` so we can filter studio vs dub projects in list views.
|
||||
- A `project_block_cache` table keyed by `hash` so re-opening a project replays existing audio without re-generating. (Can skip for v1 and just store paths inside `state_json.blocks[*].gen` — single-writer, no concurrent edits.)
|
||||
|
||||
## 2 — Backend endpoints
|
||||
|
||||
Only two new routes; everything else reuses existing generation:
|
||||
|
||||
```
|
||||
POST /studio/projects/{id}/blocks/{block_id}/generate
|
||||
body: { text, voice_id, knobs? } (knobs = same FormData shape as /generate)
|
||||
response: { audio_path, duration_ms, hash }
|
||||
Internally calls the same TTS pipeline /generate does, just writes to
|
||||
scratch under projects/{id}/ instead of the global history dir.
|
||||
|
||||
POST /studio/projects/{id}/stitch
|
||||
body: { include_block_ids?: string[] } // defaults to all
|
||||
response: { audio_path, duration_ms }
|
||||
Reads blocks in order, inserts silence for pause_before/after_ms,
|
||||
concatenates via ffmpeg, returns final WAV.
|
||||
```
|
||||
|
||||
Plus extend the existing `PUT /projects/{id}` to accept the new `state_json` shape — zero code change since it's already JSON blob passthrough.
|
||||
|
||||
Everything else (list, create, delete, the profiles endpoint for voice picker) is already built.
|
||||
|
||||
## 3 — UI shape
|
||||
|
||||
One new route: `/studio/:projectId`. Lazy-load like you do for CloneDesignTab.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ ← Projects [project title, inline-edit] [Export WAV] │
|
||||
├────────────────┬────────────────────────────────────────────────┤
|
||||
│ │ │
|
||||
│ BLOCK LIST │ BLOCK EDITOR (selected block) │
|
||||
│ (left rail) │ │
|
||||
│ │ ┌─────────────────────────────────────────┐ │
|
||||
│ ┌───────────┐ │ │ [textarea — the text for this block] │ │
|
||||
│ │ Block 1 ▸ │ │ │ │ │
|
||||
│ │ "In a…" │ │ └─────────────────────────────────────────┘ │
|
||||
│ │ [voice:A] │ │ │
|
||||
│ │ ▶ 0:12 │ │ Voice: [SearchableSelect — profiles ▾] │
|
||||
│ └───────────┘ │ Pause before: [___] ms │
|
||||
│ ┌───────────┐ │ Pause after: [___] ms │
|
||||
│ │ Block 2 ▸ │ │ │
|
||||
│ │ "The…" │ │ [Generate block] [Preview] [⚙ advanced ▾] │
|
||||
│ │ [voice:B] │ │ │
|
||||
│ │ ▶ 0:08 │ │ ── Generated audio ────────────────────── │
|
||||
│ └───────────┘ │ [waveform] [▶ play] [hash: 3f2a…] │
|
||||
│ ┌───────────┐ │ │
|
||||
│ │ + add │ │ │
|
||||
│ └───────────┘ │ │
|
||||
│ │ │
|
||||
├────────────────┴────────────────────────────────────────────────┤
|
||||
│ TRANSPORT │
|
||||
│ [▶ play all] [Regenerate stale (3)] [Stitch & export WAV] │
|
||||
│ ▓▓▓▓▓▓▓▓▓░░░░░░░░ block 3 of 12 · 1:42 / 6:10 │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Key UX calls:
|
||||
- **Paste-to-split**: a user pasting a long script should get auto-split into blocks on paragraph breaks (reuse `backend/services/subtitle_segmenter.py` — it already does sentence boundaries). No manual block-creation for v1.
|
||||
- **Stale indicator**: if `block.gen.hash` ≠ `hash(current text+voice+knobs)`, show a 🔄 badge. "Regenerate stale" button in transport acts on all stale blocks in parallel (backend already parallel-safe via `loop.create_task`).
|
||||
- **Voice inheritance**: blank voice on a block = use project default. Drop-down shows "↳ Default (Voice A)" so it's obvious.
|
||||
- **No timeline ruler in v1.** Blocks are a vertical list, not a horizontal timeline. Horizontal timeline with per-block length visualization is v2 — it's a lot of UI work and users don't need it until they're composing with music beds.
|
||||
|
||||
## 4 — Reusable primitives (already built)
|
||||
|
||||
| Thing | Where | Reuse as |
|
||||
|---|---|---|
|
||||
| `SearchableSelect` | `frontend/src/components/` | voice picker |
|
||||
| `WaveformTimeline` | `frontend/src/components/` | per-block playback |
|
||||
| Profiles API (`listProfiles`) | `frontend/src/api/profiles.ts` | voice dropdown source |
|
||||
| TTS generation | `backend/api/routers/generation.py` (`/generate`) | per-block generate |
|
||||
| Subtitle segmenter | `backend/services/subtitle_segmenter.py` | paste-to-split |
|
||||
| SSE progress | `backend/utils/hf_progress.py` | stream generation status |
|
||||
| ffmpeg concat | `backend/services/ffmpeg_utils.py` | stitching |
|
||||
|
||||
## 5 — Golden path (flow the user walks)
|
||||
|
||||
1. Home → Projects → **New Studio Project** → auto-creates id, lands in `/studio/:id`
|
||||
2. Paste 5 paragraphs of text → auto-split into 5 blocks, all assigned to project default voice
|
||||
3. Click block 3 → change voice to a second profile (narrator → character)
|
||||
4. Click **Regenerate stale** → backend fires 5 parallel `/studio/.../generate` calls, progress streams back
|
||||
5. Click **▶ play all** → client plays each block's audio in sequence with silences (no stitch needed for preview)
|
||||
6. **Stitch & export WAV** → backend concatenates, returns file path, Tauri reveals in Finder
|
||||
|
||||
## 6 — Week-of-work breakdown
|
||||
|
||||
| Day | Work |
|
||||
|---|---|
|
||||
| **Day 1** | Backend: schema migration, `/studio/projects/{id}/blocks/{block_id}/generate` route, `/studio/projects/{id}/stitch` route. Reuse `subtitle_segmenter` for paste-split. |
|
||||
| **Day 2** | Frontend: `StudioPage.jsx` skeleton + routing + new-project creation. Block list left rail. Block editor right panel (text + voice + pauses). |
|
||||
| **Day 3** | Per-block generate wiring + stale-hash detection + transport bar + parallel regen. Reuse WaveformTimeline for preview. |
|
||||
| **Day 4** | Play-all sequencing (client-side — Web Audio queue), stitch-and-export. Toast + Finder reveal. |
|
||||
| **Day 5** | Paste-to-split, keyboard shortcuts (⌘↩ regen, ⌘S save, ⌘E export), empty states, polish. |
|
||||
| **Day 6** | Ship-test: on the fresh-install DMG, create a 10-block project, swap voices, export. Fix whatever breaks. |
|
||||
| **Day 7** | Buffer / docs / cut a release. |
|
||||
|
||||
## 7 — Explicitly out of scope for v1
|
||||
|
||||
- Horizontal time-ruler / waveform-scrubbing timeline
|
||||
- Inter-block transitions (crossfade, duck)
|
||||
- Music / SFX beds
|
||||
- Multi-track (parallel voice layers)
|
||||
- Import from `.docx` / `.fdx` (screenwriter formats)
|
||||
- Shareable project links (local-first — skip)
|
||||
Binary file not shown.
Reference in New Issue
Block a user