fix(imports): preserve legacy text encodings in Electron and web

2026-09-17 12:27:56 +05:30
parent d9f51da3c6 18c543484b
commit 8538702ecb
996 changed files with 165093 additions and 4156 deletions
@@ -1,5 +1,22 @@
 # VoiceStudio — Install on Linux

+## Electron desktop (current)
+
+From the repository root, install Bun and uv, then run:
+
+```sh
+bun install
+bun run dev
+```
+
+Use `bun run desktop-prod` to build and launch Electron, or `bun run dist`
+to create local installers without publishing. The app manages its backend.
+See [Electron setup](../../electron/README.md) and [migration notes](../electron-migration.md).
+
+## Legacy Tauri installation and troubleshooting
+
+The instructions below apply to the sunset Tauri app and existing Tauri installers.
+
 This page is self-contained: follow it top to bottom and you'll end up with a
 working VoiceStudio install on a Debian / Ubuntu / Fedora / Arch host.

@@ -28,7 +45,7 @@ Everything above, plus the toolchain:
  `sudo dnf install python3.11` on Fedora, or already installed on Arch.
 - **Bun** — `curl -fsSL https://bun.sh/install | bash`.
 - **Rust / Cargo** — `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh` or via your package manager (e.g., `sudo apt install rustc cargo`).
-  If you use rustup, reopen the shell or source `"$HOME/.cargo/env"` before running `bun run desktop-prod`.
+  If you use rustup, reopen the shell or source `"$HOME/.cargo/env"` before running `bun run tauri:desktop-prod`.
 - **GTK/WebKit deps** for the Tauri shell:

  ```bash
@@ -66,10 +83,10 @@ git clone https://github.com/debpalash/VoiceStudio.git
 cd VoiceStudio
 bun install
 source "$HOME/.cargo/env"  # only needed in a shell opened before rustup finished
-bun desktop               # development build with hot reload
+bun tauri               # development build with hot reload
 ```

-Use `bun run desktop-prod` instead when you need to build and launch the
+Use `bun run tauri:desktop-prod` instead when you need to build and launch the
 production bundle. Both commands create the Python environment via `uv`, sync
 dependencies, and start the backend automatically; do not start the backend in
 a second terminal.
@@ -87,7 +104,7 @@ pkg-config --exists \
  && echo "Tauri system libraries are ready"
 ```

-`bun desktop` also checks the native `libxdo` linker input and GStreamer's
+`bun tauri` also checks the native `libxdo` linker input and GStreamer's
 `autoaudiosink` before starting. The latter is required even if you do not plan
 to record: WebKitGTK 2.52 aborts its renderer when a page creates an audio
 element without that plugin, which otherwise turns a running app blank. The
@@ -256,7 +273,7 @@ If you are on v0.4.0 or older, either update or build from source:
 git clone https://github.com/debpalash/VoiceStudio.git
 cd VoiceStudio
 bun install
-bun run desktop-prod
+bun run tauri:desktop-prod
 ```

 Tracking issues: [#62](https://github.com/debpalash/VoiceStudio/issues/62),
@@ -389,9 +406,9 @@ reinstall and left the CPU-only CUDA build in place).
 **2. Environment variable (existing installs / headless / source).** Set
 `OMNIVOICE_TORCH_VARIANT=rocm` before launching — the next bootstrap performs
 the same ROCm reinstall. Source installs honour it too:
-`OMNIVOICE_TORCH_VARIANT=rocm bun run desktop` swaps torch right after
+`OMNIVOICE_TORCH_VARIANT=rocm bun run tauri` swaps torch right after
 `uv sync` and launches the backend without re-syncing, so the wheel is not
-reverted on the next start (#1665). Without the variable, `bun run desktop`
+reverted on the next start (#1665). Without the variable, `bun run tauri`
 restores the lockfile's CUDA build — a hand-swapped ROCm wheel does not
 survive it. `OMNIVOICE_TORCH_INDEX=<url>` overrides the wheel
 index when you need a different ROCm version — e.g. AMD publishes newer
@@ -1,5 +1,22 @@
 # VoiceStudio — Install on macOS

+## Electron desktop (current)
+
+From the repository root, install Bun and uv, then run:
+
+```sh
+bun install
+bun run dev
+```
+
+Use `bun run desktop-prod` to build and launch Electron, or `bun run dist`
+to create local installers without publishing. The app manages its backend.
+See [Electron setup](../../electron/README.md) and [migration notes](../electron-migration.md).
+
+## Legacy Tauri installation and troubleshooting
+
+The instructions below apply to the sunset Tauri app and existing Tauri installers.
+
 This page is self-contained: follow it top to bottom and you'll end up with a
 working VoiceStudio install on macOS (Apple Silicon).

@@ -36,7 +53,7 @@ Everything above, plus the toolchain:
 - **Python 3.11+** — `brew install python@3.11` (or use `pyenv` / the system Python if you already have ≥3.11).
 - **Bun** — `curl -fsSL https://bun.sh/install | bash`.
 - **Rust / Cargo** — `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh` or `brew install rust`.
-  If you use rustup, reopen the terminal or source `"$HOME/.cargo/env"` before running `bun run desktop-prod`.
+  If you use rustup, reopen the terminal or source `"$HOME/.cargo/env"` before running `bun run tauri:desktop-prod`.

 FFmpeg/FFprobe and yt-dlp are **not** prerequisites on any install path: the
 app resolves them itself (a static build ships with the Python environment;
@@ -63,7 +80,7 @@ Or manually:
 git clone https://github.com/debpalash/VoiceStudio.git
 cd VoiceStudio
 bun install
-bun run desktop-prod
+bun run tauri:desktop-prod
 ```

 The first launch builds the Tauri shell, creates the Python venv via `uv`,
@@ -248,6 +248,42 @@ peak memory footprint that exceeds free VRAM. Windows-only quirk.

 **Linked issue:** [#65](https://github.com/debpalash/VoiceStudio/issues/65)

+## 5a. Backend dies on the first `/generate` (older NVIDIA GPUs, e.g. Tesla T4)
+
+**Symptom:** the backend starts fine, `/health` reports your GPU, the model
+preloads — and then the first generation request returns
+`RemoteDisconnected: Remote end closed connection without response`. Every call
+after it gets `ConnectionRefused`, because the backend process is gone. No
+Python traceback is printed.
+
+**Cause:** `torch.compile(mode="reduce-overhead")` captures CUDA graphs. On
+pre-Ampere cards (Turing sm_75 / Volta sm_70 — the Tesla T4 on Google Colab is
+the common case) that capture can abort the process from inside the native CUDA
+library. It happens below the interpreter, so no `except` in the app can catch
+it and nothing is logged.
+
+**Fix:** update — VoiceStudio now selects the compile mode per GPU and does not
+capture CUDA graphs below sm_80, so this should no longer happen. If you still
+see a crash in the generate path on any GPU, turn compilation off entirely:
+
+- **In the app:** Settings → Performance → **"Disable torch.compile"**.
+- **From the CLI / from source:** `TORCH_COMPILE_DISABLE=1` before launching.
+  This is honoured on every platform and by every engine, in-process or
+  sidecar.
+
+**Getting a traceback:** the backend now arms `faulthandler`, so a native crash
+writes the faulting thread's Python stack to `backend_err.log` on the way down.
+Include that stack when reporting — without it a native crash is unattributable.
+(`OMNIVOICE_DISABLE_FAULTHANDLER=1` turns it off.)
+
+**Extra containment:** to keep a crashing engine from taking the API down with
+it, run the engine in a killable child process — select
+**OmniVoice (subprocess-isolated)** in Settings → Engines, or
+`OMNIVOICE_TTS_BACKEND=omnivoice-subprocess`. The parent then returns an HTTP
+error and respawns the sidecar instead of dying.
+
+**Linked issue:** [#2135](https://github.com/debpalash/VoiceStudio/issues/2135)
+
 ## 5b. RTX 50-series (Blackwell, sm_120): backend crashes during `ml_imports`

 **Symptom:** on an RTX 5070 / 5070 Ti / 5080 / 5090, the backend never becomes
@@ -1112,3 +1148,7 @@ remove the app binary itself are in
 [docs/install/uninstall.md](uninstall.md).

 **Linked issue:** [#1089](https://github.com/debpalash/VoiceStudio/issues/1089)
+
+### Subtitle and manuscript encodings
+
+Electron and web imports accept UTF-8, UTF-16 with a byte-order mark, and Windows-1252 text. The same decoder rules apply to uploaded subtitles and audiobook manuscripts; legacy punctuation is preserved.
@@ -15,6 +15,10 @@ and ships a script that finds and removes them for you (with a dry-run first).
 owns with its real size, lets you opt in (separately) to the shared Hugging Face
 model cache, asks you to type `DELETE`, then removes everything and quits.

+The Electron build finishes deletion through its signed desktop helper after the
+window exits, because Chromium keeps parts of its profile directory locked while
+the app is open. The same ownership checks and model-cache opt-in still apply.
+
 This is the right path if you installed the **.dmg / .msi / AppImage** — you
 don't have the repo, so the script below isn't available to you.

@@ -1,5 +1,22 @@
 # VoiceStudio — Install on Windows

+## Electron desktop (current)
+
+From the repository root, install Bun and uv, then run:
+
+```sh
+bun install
+bun run dev
+```
+
+Use `bun run desktop-prod` to build and launch Electron, or `bun run dist`
+to create local installers without publishing. The app manages its backend.
+See [Electron setup](../../electron/README.md) and [migration notes](../electron-migration.md).
+
+## Legacy Tauri installation and troubleshooting
+
+The instructions below apply to the sunset Tauri app and existing Tauri installers.
+
 This page is self-contained: follow it top to bottom and you'll end up with a
 working VoiceStudio install on Windows 10 / 11 (x64).

@@ -20,7 +37,7 @@ by the app itself on first launch. No toolchain needed.
 Everything above, plus the toolchain:

 - **Git for Windows** — `winget install --id Git.Git -e`. Needed for
-  `git clone`, and it includes **Git Bash**, which `bun run desktop-prod`
+  `git clone`, and it includes **Git Bash**, which `bun run tauri:desktop-prod`
  uses to run its build-and-launch script. Without it, `desktop-prod` stops
  with an error telling you to install it.
 - **Python 3.11+** — `winget install Python.Python.3.11` (or download from
@@ -32,7 +49,7 @@ Everything above, plus the toolchain:
 - **Bun** — `powershell -c "irm bun.sh/install.ps1 | iex"`.
 - **FFmpeg** — `winget install Gyan.FFmpeg`.
 - **Rust / Cargo** — `winget install Rust.Rustup` or download `rustup-init.exe` from [rustup.rs](https://rustup.rs/).
-  After installing Rustup, close and reopen PowerShell before running `bun run desktop-prod`.
+  After installing Rustup, close and reopen PowerShell before running `bun run tauri:desktop-prod`.

 ## GPU support on Windows

@@ -64,17 +81,17 @@ Or manually:
 git clone https://github.com/debpalash/VoiceStudio.git
 cd VoiceStudio
 bun install
-bun run desktop-prod
+bun run tauri:desktop-prod
 ```

 The first launch creates the Python venv via `uv`, syncs deps, and downloads
 model weights. The splash screen shows progress.

-> **Note:** `bun run desktop-prod` runs a bash script under the hood. You can
+> **Note:** `bun run tauri:desktop-prod` runs a bash script under the hood. You can
 > launch it from PowerShell or cmd as shown — it finds Git Bash automatically
 > (installed with Git for Windows, see Prerequisites). If no Git Bash is
 > found, it prints instructions instead of failing silently. Alternatives
-> that don't need bash: `bun run desktop` (dev mode) or the pre-built MSI
+> that don't need bash: `bun run tauri` (dev mode) or the pre-built MSI
 > below.

 ## Install (pre-built MSI)
@@ -274,21 +291,25 @@ synthesise call. On machines with <16 GB VRAM, that compile step can OOM
 failed`.

 **The one-click fix:** open **Settings → Performance** in the app and toggle
-**"Disable torch.compile (Windows)"** on. That sets the
-`TORCH_COMPILE_DISABLE=1` env var on every engine subprocess VoiceStudio spawns,
-which falls back to the eager-mode kernel path. You'll lose a few percent of
-peak throughput in exchange for the engine actually loading.
+**"Disable torch.compile"** on. That sets the `TORCH_COMPILE_DISABLE=1` env var
+on every engine subprocess VoiceStudio spawns and forces the in-process engine
+to eager mode as well. You'll lose a few percent of peak throughput in exchange
+for the engine actually loading.

 **From the CLI / from source:** set the env var manually before launching:

 ```powershell
 $env:TORCH_COMPILE_DISABLE = "1"
-bun run desktop-prod
+bun run tauri:desktop-prod
 ```

-This setting is a no-op on macOS and Linux (the OOM is Windows-specific —
-the `torch.compile` kernel cache behaves differently on the other platforms).
-Tracking issue: [#65](https://github.com/debpalash/VoiceStudio/issues/65).
+The OOM this section describes is Windows-specific, but the toggle itself works
+on **every** platform — it used to be greyed out elsewhere, which left Linux and
+macOS users with no way to switch off a `torch.compile` that was breaking their
+engine. Tracking issues:
+[#65](https://github.com/debpalash/VoiceStudio/issues/65) (this OOM) and
+[#2135](https://github.com/debpalash/VoiceStudio/issues/2135) (the same toggle
+on Linux/CUDA).

 ## Hugging Face token (optional but recommended)