fix(asr): secure configured endpoints and refresh guidance (#1751)
Refreshes README and linked docs with accurate installation, platform, privacy, API, and model-license guidance; documents local gigastt; and pins OpenAI-compatible ASR traffic to the configured secure origin. Closes #1736.
This commit is contained in:
@@ -22,12 +22,14 @@ the frozen-backend fallback mirror it for their toolchains.
|
||||
|
||||
### Docs
|
||||
|
||||
- Local gigastt is now documented as a supported OpenAI-compatible ASR endpoint, with loopback privacy distinguished from remote servers (#1736) — thanks @ekhodzitsky!
|
||||
- The CosyVoice guide now states that packaged builds have no one-click runtime installer and records the exact readiness checks exposed by [Discussion 1631](https://github.com/debpalash/VoiceStudio/discussions/1631).
|
||||
- A production private-API guide now covers pinned containers, root credentials, network isolation, streaming proxies, health checks, upgrades, and benchmark evidence (#1720)
|
||||
- RX 6700 XT/gfx1031 over WSL2 ROCDXG is now explicitly unverified until a published end-to-end GPU workload proves the mapped path (#1716)
|
||||
|
||||
### Fixed
|
||||
|
||||
- OpenAI-compatible ASR now requires HTTPS outside loopback and refuses redirects so audio stays on the configured origin (#1736)
|
||||
- Windows isolated engines now retain direct Job ownership without an extra Python supervisor process that can deadlock the child loader (#1734)
|
||||
- The setup splash now waits through the backend's full startup budget instead of reporting slow Windows CUDA initialization as stuck after two minutes (#1749)
|
||||
- Dubbing jobs can now reuse every source-language code produced by automatic ASR detection without a 400 error on the next upload (#1737)
|
||||
|
||||
+10
-4
@@ -10,10 +10,10 @@ Copyright 2024-present Palash Debnath and VoiceStudio contributors.
|
||||
|
||||
VoiceStudio is **free and open-source software, licensed under the GNU
|
||||
Affero General Public License, Version 3 (AGPL-3.0)**. You are free to use,
|
||||
copy, modify, and redistribute it — and that **includes commercial and internal
|
||||
business use**: run the app, use its outputs commercially, sell the audio you
|
||||
produce with it, provide professional/client services with it, and deploy it
|
||||
within your organization.
|
||||
copy, modify, and redistribute it. That **includes commercial and internal
|
||||
business use** of the application itself. Model weights, tokenizers, and other
|
||||
third-party assets retain their own terms; this application license does not
|
||||
grant or summarize rights under those separate terms.
|
||||
|
||||
Because this is the **Affero** GPL, one additional obligation applies: if you
|
||||
modify VoiceStudio and make that modified version available to others over
|
||||
@@ -41,6 +41,12 @@ is **separately licensed under Apache License 2.0** by its upstream authors and
|
||||
is not relicensed here. Apache License 2.0 is compatible with, and may be
|
||||
combined under, the GNU AGPL-3.0. See `pyproject.toml`.
|
||||
|
||||
Downloaded model weights are not relicensed by VoiceStudio. The default
|
||||
`k2-fsa/OmniVoice` model card identifies its code as Apache-2.0 and pretrained
|
||||
weights as CC-BY-NC. Its `audio_tokenizer/LICENSE` contains separate Boson
|
||||
Higgs Audio 2 and Meta Llama community terms. A commercial license for
|
||||
VoiceStudio-owned code does not replace any of those terms.
|
||||
|
||||
Third-party dependencies retain their own licenses. See `Cargo.lock`,
|
||||
`bun.lock`, and `uv.lock` for the resolved set.
|
||||
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
<div align="center">
|
||||
<a href="https://trendshift.io/repositories/28176?utm_source=repository-badge&utm_medium=badge&utm_campaign=badge-repository-28176" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/repositories/28176" alt="debpalash%2FVoiceStudio | Trendshift" width="250" height="55" /></a>
|
||||
<a href="https://trendshift.io/repositories/28176?utm_source=repository-badge&utm_medium=badge&utm_campaign=badge-repository-28176" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/repositories/28176" alt="VoiceStudio ranking on Trendshift" width="250" height="55" /></a>
|
||||
|
||||
<img src="docs/logo.png" alt="VoiceStudio logo" width="120" height="120" />
|
||||
<h1>VoiceStudio</h1>
|
||||
<p><sub>Previously OmniVoice-Studio</sub></p>
|
||||
<h3>Local voice cloning, dubbing, dictation, and long-form audio.</h3>
|
||||
<p>16 TTS engines · 11 ASR engines · 646-language catalogue · macOS, Windows, and Linux</p>
|
||||
<p><strong>Local-first.</strong> No account, API key, subscription, or usage meter for the core workflow.</p>
|
||||
<h3>Clone voices, dub video, dictate, and produce long-form audio on your own hardware.</h3>
|
||||
<p>16 TTS engines · 11 ASR engines · 646-language catalogue · macOS, Windows, Linux, and Docker</p>
|
||||
<p>No account, API key, subscription, or usage meter for the local workflow.</p>
|
||||
|
||||
<p>
|
||||
<a href="#install">Install</a> ·
|
||||
@@ -38,7 +38,7 @@
|
||||
</div>
|
||||
|
||||
> [!WARNING]
|
||||
> **Active beta.** Use the [latest release](https://github.com/debpalash/VoiceStudio/releases/latest) for stable work or `main` for current fixes. Report problems through [GitHub Issues](https://github.com/debpalash/VoiceStudio/issues).
|
||||
> **Active beta.** Use the [latest release](https://github.com/debpalash/VoiceStudio/releases/latest) for stable work. `main` contains the newest fixes and may change between releases. Report problems through [GitHub Issues](https://github.com/debpalash/VoiceStudio/issues).
|
||||
|
||||
## At a glance
|
||||
|
||||
@@ -51,28 +51,30 @@
|
||||
| **Compute** | CUDA · Apple Silicon MPS/MLX · ROCm on Linux · CPU · optional remote workers |
|
||||
| **Interfaces** | Desktop app · local REST/SSE/WebSocket API · OpenAI-compatible audio API · MCP Server |
|
||||
| **Storage** | Voices, projects, settings, and outputs stay on the machine by default |
|
||||
| **License** | AGPL-3.0; optional engines keep their own model licenses |
|
||||
| **License** | AGPL-3.0 application; downloaded models keep their upstream terms |
|
||||
|
||||
<a id="install"></a>
|
||||
|
||||
## Install
|
||||
|
||||
Download a package from the [latest release](https://github.com/debpalash/VoiceStudio/releases/latest), then follow the platform guide.
|
||||
|
||||
| Platform | Package | Guide |
|
||||
|---|---|---|
|
||||
| macOS 13.3+ | DMG, Apple Silicon | [Install on macOS](docs/install/macos.md) |
|
||||
| Windows 10/11 | MSI, x64 | [Install on Windows](docs/install/windows.md) |
|
||||
| macOS 13.3+ | Apple Silicon DMG | [Install on macOS](docs/install/macos.md) |
|
||||
| Windows 10/11 | x64 MSI; choose the current-user build when listed to install without admin access | [Install on Windows](docs/install/windows.md#install-pre-built-msi) |
|
||||
| Linux | AppImage, x86_64 with glibc 2.39+ | [Install on Linux](docs/install/linux.md) |
|
||||
| Docker | CUDA, ROCm, or CPU; worker-only GPU profiles | [Run with Docker](docs/install/docker.md) |
|
||||
| Docker | CUDA, ROCm, CPU, and worker-only GPU profiles | [Run with Docker](docs/install/docker.md) |
|
||||
|
||||
Download packages from the [latest release](https://github.com/debpalash/VoiceStudio/releases/latest). First launch creates a managed Python environment and downloads the default model. Later launches reuse both.
|
||||
First launch creates a managed Python environment and downloads the default model. Later launches reuse both.
|
||||
|
||||
> [!NOTE]
|
||||
> On macOS, first launch needs a one-time right-click → **Open** approval. Intel Macs cannot run the local Python backend; use a [remote backend](docs/install/macos.md) instead.
|
||||
> On macOS, first launch needs a one-time right-click, then **Open** approval. Intel Macs cannot run the local Python backend; use a [remote backend](docs/install/macos.md) instead.
|
||||
|
||||
### First voice
|
||||
|
||||
1. Launch VoiceStudio and open **Voice Cloning**.
|
||||
2. Add a clean voice sample. Three seconds works; 5–15 seconds usually gives a better prompt.
|
||||
2. Add a clean voice sample. Three seconds works; 5 to 15 seconds usually gives a better prompt.
|
||||
3. Enter text, choose a language, then select **Generate**.
|
||||
|
||||
### Run from source
|
||||
@@ -159,7 +161,7 @@ Requirements vary by engine. These values cover the default local workflow.
|
||||
| **Disk** | 10 GB free | 20 GB+ SSD |
|
||||
| **GPU** | Optional; CPU mode is supported | NVIDIA CUDA or Apple Silicon |
|
||||
| **VRAM** | 4 GB when using a GPU | 8 GB+; large optional engines need more |
|
||||
| **Python from source** | 3.11+ | 3.11–3.12 |
|
||||
| **Python from source** | 3.11+ | 3.11 or 3.12 |
|
||||
|
||||
ROCm is Linux-only and opt-in. Windows AMD/Ryzen AI uses CPU. Systems with limited VRAM offload work to CPU when required. See [performance](docs/performance.md), [benchmarks](docs/benchmarks.md), and [engine disk usage](docs/engines/disk-usage.md).
|
||||
|
||||
@@ -175,22 +177,22 @@ Engine support is capability-specific. Check cloning, language, platform, memory
|
||||
|
||||
| Engine | Languages | Clone | Instruct | Linux | macOS ARM | Windows | License |
|
||||
|---|:---:|:---:|:---:|:---:|:---:|:---:|---|
|
||||
| **VoiceStudio** (default, powered by k2-fsa/OmniVoice) | 600+ | Yes | Yes | CUDA/CPU | MPS | CUDA/CPU | [AGPL-3.0](LICENSE) app · [Apache-2.0](LICENSE-NOTICE.md) model |
|
||||
| **VoiceStudio** (default, powered by k2-fsa/OmniVoice) | 600+ | Yes | Yes | CUDA/CPU | MPS | CUDA/CPU | [AGPL-3.0](LICENSE) app · [Apache-2.0 code, CC-BY-NC weights](https://huggingface.co/k2-fsa/OmniVoice#license)³ |
|
||||
| **CosyVoice 3** | 9 + 18 dialects | Yes | Yes | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
|
||||
| **GPT-SoVITS** | 5 | Yes | — | CUDA/CPU | — | CUDA/CPU | MIT |
|
||||
| **GPT-SoVITS** | 5 | Yes | No | CUDA/CPU | No | CUDA/CPU | MIT |
|
||||
| **VoxCPM2** | 30 | Yes | Yes | CUDA/CPU | MPS | CUDA/CPU | Apache-2.0 |
|
||||
| **MOSS-TTS-Nano** | 20 | Yes | — | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
|
||||
| **KittenTTS** | English | — | — | CPU | CPU | CPU | MIT |
|
||||
| **MLX-Audio** | Model-dependent | Varies | Varies | — | MLX | — | Varies |
|
||||
| **Sherpa-ONNX** | 20+ | — | — | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
|
||||
| **IndexTTS 2.5** ⚡ | ZH · EN · JA · ES · AR | Yes | — | CUDA/CPU | CPU | CUDA/CPU | Bilibili model license¹ |
|
||||
| **OmniVoice GGUF** ⚡ | 600+ | Yes | Yes | CUDA/CPU | MPS/CPU | CUDA/CPU | [AGPL-3.0](LICENSE) app · [Apache-2.0](LICENSE-NOTICE.md) model |
|
||||
| **OmniVoice (subprocess)** ⚡ | 600+ | Yes | Yes | CUDA/CPU | MPS | CUDA/CPU | [AGPL-3.0](LICENSE) app · [Apache-2.0](LICENSE-NOTICE.md) model |
|
||||
| **PocketTTS** ⚡ | EN · FR · DE · PT · IT · ES | Yes | — | CPU | CPU | CPU | CC-BY-4.0, gated² |
|
||||
| **Supertonic 3** ⚡ | 31 | — | — | CPU | CPU | CPU | OpenRAIL-M |
|
||||
| **MOSS-TTS-v1.5** ⚡ | 31 | Yes | — | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
|
||||
| **dots.tts** ⚡ | 24 | Yes | — | CUDA/CPU | CPU | — | Apache-2.0 |
|
||||
| **Confucius4-TTS** ⚡ | 14 | Yes | — | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
|
||||
| **MOSS-TTS-Nano** | 20 | Yes | No | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
|
||||
| **KittenTTS** | English | No | No | CPU | CPU | CPU | MIT |
|
||||
| **MLX-Audio** | Model-dependent | Varies | Varies | No | MLX | No | Varies |
|
||||
| **Sherpa-ONNX** | 20+ | No | No | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
|
||||
| **IndexTTS 2.5** ⚡ | ZH · EN · JA · ES · AR | Yes | No | CUDA/CPU | CPU | CUDA/CPU | Bilibili model license¹ |
|
||||
| **OmniVoice GGUF** ⚡ | 600+ | Yes | Yes | CUDA/CPU | MPS/CPU | CUDA/CPU | [AGPL-3.0](LICENSE) app · [review the derivative model terms](https://huggingface.co/Serveurperso/OmniVoice-GGUF#license)³ |
|
||||
| **OmniVoice (subprocess)** ⚡ | 600+ | Yes | Yes | CUDA/CPU | MPS | CUDA/CPU | [AGPL-3.0](LICENSE) app · [Apache-2.0 code, CC-BY-NC weights](https://huggingface.co/k2-fsa/OmniVoice#license)³ |
|
||||
| **PocketTTS** ⚡ | EN · FR · DE · PT · IT · ES | Yes | No | CPU | CPU | CPU | CC-BY-4.0, gated² |
|
||||
| **Supertonic 3** ⚡ | 31 | No | No | CPU | CPU | CPU | OpenRAIL-M |
|
||||
| **MOSS-TTS-v1.5** ⚡ | 31 | Yes | No | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
|
||||
| **dots.tts** ⚡ | 24 | Yes | No | CUDA/CPU | CPU | No | Apache-2.0 |
|
||||
| **Confucius4-TTS** ⚡ | 14 | Yes | No | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
|
||||
|
||||
⚡ Installed or registered on demand.
|
||||
|
||||
@@ -198,6 +200,8 @@ Engine support is capability-specific. Check cloning, language, platform, memory
|
||||
|
||||
² PocketTTS shows its gated-access and CC-BY-4.0 terms before first use.
|
||||
|
||||
³ The OmniVoice snapshot also includes an audio tokenizer under separate [Boson Higgs Audio 2 and Meta Llama community terms](https://huggingface.co/k2-fsa/OmniVoice/blob/main/audio_tokenizer/LICENSE). VoiceStudio's application license does not replace model or tokenizer terms.
|
||||
|
||||
Clone-less engines cannot preserve a reference speaker in dubbing or pinned-voice batch jobs. VoiceStudio rejects those jobs instead of silently changing engines. Heavy engines have separate memory and platform limits; check their engine guide first.
|
||||
|
||||
<a id="asr-engines"></a>
|
||||
@@ -216,7 +220,7 @@ Clone-less engines cannot preserve a reference speaker in dubbing or pinned-voic
|
||||
| **Moonshine** | `moonshine` | English | Low-power, low-latency ONNX |
|
||||
| **FunASR** | `funasr` | 50+ | VAD and inline diarization |
|
||||
| **sherpa-onnx** (live dictation) | `sherpa-onnx-asr` | Model-dependent | Streaming CPU dictation |
|
||||
| **OpenAI-compatible** ⚠️ remote | `openai-compat-asr` | Server-dependent | Qwen3-ASR or another compatible endpoint; audio leaves the machine |
|
||||
| **OpenAI-compatible** ⚠️ configured server | `openai-compat-asr` | Server-dependent | Local gigastt/Qwen3-ASR or a remote endpoint; audio goes only to that server |
|
||||
|
||||
WhisperX and Faster-Whisper retry with `int8` when efficient `float16` is unavailable. Pin `ASR_COMPUTE_TYPE=int8` or `float32` only if automatic selection still fails.
|
||||
|
||||
@@ -251,8 +255,8 @@ FastAPI backend
|
||||
|
||||
- The desktop talks to a loopback-only backend on `localhost:3900`.
|
||||
- Loopback API calls need no server key. Remote access requires a share PIN or API key.
|
||||
- Remote workers and OpenAI-compatible ASR are opt-in. The UI identifies when audio leaves the machine.
|
||||
- Analytics is off until consent. If enabled, it sends allowlisted, content-free usage metadata—not text, audio, file names, or projects.
|
||||
- Remote workers and OpenAI-compatible ASR are opt-in. Loopback ASR may use HTTP and keeps audio on the machine; non-loopback endpoints require HTTPS, and redirects are not followed.
|
||||
- Analytics is off until consent. If enabled, it sends allowlisted, content-free usage metadata. It never sends text, audio, file names, or projects.
|
||||
|
||||
<a id="api"></a>
|
||||
|
||||
@@ -287,12 +291,11 @@ with client.audio.speech.with_streaming_response.create(
|
||||
response.stream_to_file("speech.wav")
|
||||
```
|
||||
|
||||
The bundled Rust control sidecar also lets Herdr, coding agents, VS Code,
|
||||
desktop apps, and TUIs trigger the existing system-wide dictation flow or reuse
|
||||
its safe native insertion. See the [speech platform guide](docs/speech-platform.md).
|
||||
The full API reference is in **Settings → OpenAPI Reference**. For LAN,
|
||||
Tailscale, or proxy access, read [API authentication](docs/api-auth.md) before
|
||||
exposing the backend.
|
||||
The bundled Rust control sidecar lets Herdr, coding agents, VS Code, desktop apps,
|
||||
and TUIs trigger the system-wide dictation flow or reuse its native text
|
||||
insertion. See the [speech platform guide](docs/speech-platform.md). The full API
|
||||
reference is in **Settings → OpenAPI Reference**. For LAN, Tailscale, or proxy
|
||||
access, read [API authentication](docs/api-auth.md) before exposing the backend.
|
||||
|
||||
### Agent skills
|
||||
|
||||
@@ -337,19 +340,19 @@ Apple Silicon is supported with MPS and MLX options. Intel Macs cannot run the l
|
||||
<details>
|
||||
<summary><strong>How much VRAM do I need?</strong></summary>
|
||||
|
||||
A GPU is optional. Use 4 GB VRAM as the minimum for accelerated work and 8 GB+ for the default multi-stage workflow. Large optional engines can require 12–16 GB or more. Check the [benchmarks](docs/benchmarks.md) and engine guide.
|
||||
A GPU is optional. Use 4 GB VRAM as the minimum for accelerated work and 8 GB+ for the default multi-stage workflow. Large optional engines can require 12 to 16 GB or more. Check the [benchmarks](docs/benchmarks.md) and engine guide.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Why does a longer reference clip not always improve the clone?</strong></summary>
|
||||
|
||||
Cloning is zero-shot: the clip is a prompt, not training data. Use 5–15 seconds of one speaker, close to the microphone, without music, noise, or reverb. Match the tone and pace you want in the output. For training, see [data preparation](docs/data_preparation.md) and [training](docs/training.md).
|
||||
Cloning is zero-shot: the clip is a prompt, not training data. Use 5 to 15 seconds of one speaker, close to the microphone, without music, noise, or reverb. Match the tone and pace you want in the output. For training, see [data preparation](docs/data_preparation.md) and [training](docs/training.md).
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Can I use generated audio commercially?</strong></summary>
|
||||
|
||||
Yes under VoiceStudio's AGPL-3.0 terms. Optional engines and model weights may use different licenses; review the selected engine's license before commercial use.
|
||||
VoiceStudio's application license does not restrict generated audio, but it does not grant rights under a model's separate terms. The default OmniVoice repository labels its pretrained weights CC-BY-NC and includes a tokenizer under separate community terms. Review the selected model terms before commercial use.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
@@ -379,9 +382,9 @@ VoiceStudio is free and has no paid tier. Donations fund development and infrast
|
||||
|
||||
## License
|
||||
|
||||
VoiceStudio is licensed under [AGPL-3.0](LICENSE). You may run it, modify it, use it internally, and sell generated audio. If you modify VoiceStudio and provide that modified version as a network service, AGPL requires you to offer the corresponding source under the same license. A commercial license is available for proprietary embedding; contact **VoiceStudio@palash.dev**. See [LICENSE-NOTICE.md](LICENSE-NOTICE.md) for the plain-language scope.
|
||||
VoiceStudio is licensed under [AGPL-3.0](LICENSE). You may run it, modify it, and use it internally. The application license itself does not restrict selling generated audio, but downloaded model and tokenizer terms may. If you modify VoiceStudio and provide that modified version as a network service, AGPL requires you to offer the corresponding source under the same license. A commercial license for VoiceStudio-owned code is available for proprietary embedding; it does not relicense third-party models. Contact **VoiceStudio@palash.dev**. See [LICENSE-NOTICE.md](LICENSE-NOTICE.md) for the plain-language scope.
|
||||
|
||||
Optional engines and downloaded models retain their own licenses. The bundled `omnivoice/` model remains Apache-2.0 upstream.
|
||||
Optional engines and downloaded models retain their own licenses. The bundled `omnivoice/` Python code is Apache-2.0 upstream; the default downloaded weights and audio tokenizer use separate terms.
|
||||
|
||||
## Acknowledgments
|
||||
|
||||
|
||||
@@ -1035,9 +1035,10 @@ def set_asr_openai_compat(body: _ASROpenAICompatBody):
|
||||
from services import asr_backend, settings_store
|
||||
|
||||
if body.base_url is not None:
|
||||
url = body.base_url.strip().rstrip("/")
|
||||
if url and not url.startswith(("http://", "https://")):
|
||||
raise HTTPException(status_code=400, detail="Base URL must start with http(s)://")
|
||||
try:
|
||||
url = asr_backend.normalize_openai_compat_asr_base_url(body.base_url)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
settings_store.set_text(asr_backend._ASR_OPENAI_COMPAT_BASE_URL_KEY, url)
|
||||
if body.model is not None:
|
||||
settings_store.set_text(
|
||||
|
||||
@@ -24,6 +24,7 @@ faster-whisper because it's available on every platform we ship to).
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import ipaddress
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
@@ -31,6 +32,7 @@ import contextlib
|
||||
import threading
|
||||
import time
|
||||
import weakref
|
||||
from urllib.parse import urlsplit
|
||||
from utils.containment import contain_system_exit
|
||||
|
||||
from abc import ABC, abstractmethod
|
||||
@@ -2000,6 +2002,42 @@ _ASR_OPENAI_COMPAT_MODEL_KEY = "asr.openai_compat.model"
|
||||
_ASR_OPENAI_COMPAT_SECRET_NAME = "asr_openai_compat_key"
|
||||
|
||||
|
||||
def normalize_openai_compat_asr_base_url(value: str) -> str:
|
||||
"""Normalize a safe ASR endpoint, allowing plain HTTP only on loopback."""
|
||||
base = (value or "").strip().rstrip("/")
|
||||
if not base:
|
||||
return ""
|
||||
try:
|
||||
parsed = urlsplit(base)
|
||||
_ = parsed.port
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise ValueError("Invalid OpenAI-compatible ASR base URL") from exc
|
||||
scheme = parsed.scheme.lower()
|
||||
if (
|
||||
scheme not in {"http", "https"}
|
||||
or not parsed.hostname
|
||||
or parsed.username is not None
|
||||
or parsed.password is not None
|
||||
or parsed.query
|
||||
or parsed.fragment
|
||||
):
|
||||
raise ValueError(
|
||||
"OpenAI-compatible ASR base URL must be a credential-free HTTP(S) URL"
|
||||
)
|
||||
host = parsed.hostname.lower()
|
||||
loopback = host == "localhost"
|
||||
if not loopback:
|
||||
try:
|
||||
address = ipaddress.ip_address(host)
|
||||
address = getattr(address, "ipv4_mapped", None) or address
|
||||
loopback = address.is_loopback
|
||||
except ValueError:
|
||||
loopback = False
|
||||
if scheme == "http" and not loopback:
|
||||
raise ValueError("Non-loopback OpenAI-compatible ASR endpoints require HTTPS")
|
||||
return base
|
||||
|
||||
|
||||
def resolve_openai_compat_asr_base_url() -> str:
|
||||
from services import settings_store
|
||||
return (
|
||||
@@ -2063,7 +2101,7 @@ def probe_openai_compat_server(
|
||||
maps to a translated message:
|
||||
|
||||
not_configured no base URL anywhere
|
||||
invalid_url base URL without an http(s):// scheme
|
||||
invalid_url malformed URL or non-loopback HTTP endpoint
|
||||
ok 2xx — ``model_found`` says whether the configured
|
||||
model appears in the server's list (None = unknown)
|
||||
ok_no_models 404/405/501 — reachable, but no /models endpoint
|
||||
@@ -2078,7 +2116,7 @@ def probe_openai_compat_server(
|
||||
|
||||
from core.scrub import scrub_text
|
||||
|
||||
base = (base_url if base_url is not None else resolve_openai_compat_asr_base_url()).strip().rstrip("/")
|
||||
configured_base = base_url if base_url is not None else resolve_openai_compat_asr_base_url()
|
||||
mdl = (model if model is not None else resolve_openai_compat_asr_model()).strip()
|
||||
if api_key is None:
|
||||
key = resolve_openai_compat_asr_api_key()
|
||||
@@ -2094,9 +2132,11 @@ def probe_openai_compat_server(
|
||||
"model_found": None,
|
||||
"detail": None,
|
||||
}
|
||||
if not base:
|
||||
if not configured_base.strip():
|
||||
return out
|
||||
if not base.startswith(("http://", "https://")):
|
||||
try:
|
||||
base = normalize_openai_compat_asr_base_url(configured_base)
|
||||
except ValueError:
|
||||
out["status"] = "invalid_url"
|
||||
return out
|
||||
|
||||
@@ -2107,7 +2147,7 @@ def probe_openai_compat_server(
|
||||
try:
|
||||
with httpx.Client(
|
||||
timeout=httpx.Timeout(timeout_s, connect=min(5.0, timeout_s)),
|
||||
follow_redirects=True,
|
||||
follow_redirects=False,
|
||||
) as client:
|
||||
resp = client.get(f"{base}/models", headers=headers)
|
||||
except httpx.TimeoutException as exc:
|
||||
@@ -2170,13 +2210,20 @@ class OpenAICompatASRBackend(ASRBackend):
|
||||
gpu_compat = ("cpu",) # network client only — no local compute
|
||||
|
||||
def __init__(self):
|
||||
self._base_url = resolve_openai_compat_asr_base_url()
|
||||
self._base_url = normalize_openai_compat_asr_base_url(
|
||||
resolve_openai_compat_asr_base_url()
|
||||
)
|
||||
self._model = resolve_openai_compat_asr_model()
|
||||
|
||||
@classmethod
|
||||
def is_available(cls) -> tuple[bool, str]:
|
||||
if not resolve_openai_compat_asr_base_url():
|
||||
base_url = resolve_openai_compat_asr_base_url()
|
||||
if not base_url:
|
||||
return False, "Configure a server endpoint in Model Catalogue → Engines"
|
||||
try:
|
||||
normalize_openai_compat_asr_base_url(base_url)
|
||||
except ValueError as exc:
|
||||
return False, str(exc)
|
||||
try:
|
||||
import openai # noqa: F401
|
||||
except ImportError:
|
||||
@@ -2184,13 +2231,18 @@ class OpenAICompatASRBackend(ASRBackend):
|
||||
return True, "ready"
|
||||
|
||||
def _client(self):
|
||||
from openai import OpenAI
|
||||
from openai import DefaultHttpxClient, OpenAI
|
||||
api_key = resolve_openai_compat_asr_api_key() or "not-needed"
|
||||
# max_retries=0: mirrors llm_skills.resolve_skill_client — a
|
||||
# rate-limited/slow server retrying inside the SDK would blow past
|
||||
# whatever bounded timeout the caller (dub transcribe, dictation)
|
||||
# expects from a single call.
|
||||
return OpenAI(base_url=self._base_url, api_key=api_key, max_retries=0)
|
||||
return OpenAI(
|
||||
base_url=self._base_url,
|
||||
api_key=api_key,
|
||||
max_retries=0,
|
||||
http_client=DefaultHttpxClient(follow_redirects=False),
|
||||
)
|
||||
|
||||
def transcribe(self, audio_path: str, *, word_timestamps: bool = True) -> dict:
|
||||
logger.info(
|
||||
|
||||
@@ -54,7 +54,7 @@ approval), [Windows](../install/windows.md), [Linux](../install/linux.md),
|
||||
| Moonshine | [moonshine](moonshine.md) | CPU | edge/low-power, no timestamps | `pip install` (see guide) |
|
||||
| FunASR (SenseVoice) | [funasr](funasr.md) | CUDA · CPU | 50+ languages, inline diarization | `pip install funasr` |
|
||||
| Sherpa-ONNX dictation | [sherpa-onnx-asr](sherpa-onnx-asr.md) | CPU | live streaming dictation | curated model download |
|
||||
| OpenAI-compatible (remote) | [openai-compatible-asr](openai-compatible-asr.md) | network | offloading to a server (audio leaves the machine) | Model Catalogue |
|
||||
| OpenAI-compatible (local or remote) | [openai-compatible-asr](openai-compatible-asr.md) | network | a configured endpoint; loopback stays local | Model Catalogue |
|
||||
|
||||
Speaker diarization is not an engine registry of its own — the dub pipeline
|
||||
uses pyannote (HF-gated; see [diarization](../features/diarization.md)) and
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
# VoiceStudio — OpenAI-Compatible Remote ASR
|
||||
# VoiceStudio: OpenAI-Compatible ASR
|
||||
|
||||
Point transcription at **any** server exposing an OpenAI-compatible
|
||||
`POST /v1/audio/transcriptions` endpoint — LM Studio or a llama.cpp-style
|
||||
local server, a self-hosted Qwen3-ASR/FunASR/SenseVoice box on your network,
|
||||
Groq, or OpenAI's own Whisper API. Unlike every other ASR engine, this one
|
||||
runs no model locally: it's a pure network client, so it needs no install
|
||||
and claims no GPU.
|
||||
`POST /v1/audio/transcriptions` endpoint: gigastt, LM Studio, or a
|
||||
llama.cpp-style server on the same machine; a self-hosted
|
||||
Qwen3-ASR/FunASR/SenseVoice box on your network; Groq; or OpenAI's Whisper
|
||||
API. VoiceStudio is a pure client in this mode, so the configured server owns
|
||||
model installation and compute.
|
||||
|
||||
## Setup
|
||||
|
||||
@@ -39,13 +39,18 @@ picks local engines, and the app works fully with this engine unconfigured.
|
||||
|
||||
| Server | Server URL | Model | API key |
|
||||
| --- | --- | --- | --- |
|
||||
| [gigastt](https://github.com/ekhodzitsky/gigastt) (local Russian specialist) | `http://127.0.0.1:9876/v1` | `gigaam-v3-rnnt` | none |
|
||||
| LM Studio (local) | `http://localhost:1234/v1` | the model name shown in LM Studio | none |
|
||||
| llama.cpp / whisper.cpp server (local) | `http://localhost:8080/v1` | whatever the server loads (often ignored) | none |
|
||||
| speaches / faster-whisper-server (local) | `http://localhost:8000/v1` | e.g. `Systran/faster-whisper-large-v3` | none |
|
||||
| Self-hosted Qwen3-ASR / FunASR (LAN box) | `http://<host>:8000/v1` | your deployment's model id | if you enabled auth |
|
||||
| Self-hosted Qwen3-ASR / FunASR (LAN box) | `https://<host>:8000/v1` | your deployment's model id | if you enabled auth |
|
||||
| Groq | `https://api.groq.com/openai/v1` | `whisper-large-v3` | required |
|
||||
| OpenAI | `https://api.openai.com/v1` | `whisper-1` | required |
|
||||
|
||||
Plain HTTP is accepted only for exact loopback hosts such as `localhost`,
|
||||
`127.0.0.1`, and `::1`. Every non-loopback endpoint must use HTTPS. VoiceStudio
|
||||
does not follow redirects from transcription or connection-probe requests.
|
||||
|
||||
Local servers vary in which endpoints they implement — if **Test
|
||||
connection** reports the server is reachable but doesn't list models,
|
||||
transcription may still work; run a small dictation or dub-transcribe to
|
||||
@@ -60,8 +65,7 @@ path returns word-level timestamps — that's not part of this API.
|
||||
|
||||
## Privacy note
|
||||
|
||||
Unlike every other ASR engine in VoiceStudio, audio sent through this backend
|
||||
leaves your machine — to whatever server **you** configured, and nowhere
|
||||
else. If that's a self-hosted server on your own network, nothing leaves
|
||||
your control; if it's a third-party API (Groq, OpenAI's, or someone
|
||||
else's), review their data handling before sending anything sensitive.
|
||||
Audio goes only to the server **you** configure. A loopback URL such as the
|
||||
gigastt example keeps it on the same machine and may use HTTP. LAN and public
|
||||
endpoints require HTTPS, and redirects are not followed. Review the configured
|
||||
server's data handling before sending anything sensitive.
|
||||
|
||||
+1
-1
@@ -82,7 +82,7 @@ asr_engines:
|
||||
- id: sherpa-onnx-asr
|
||||
readme: "**sherpa-onnx** (live dictation)"
|
||||
- id: openai-compat-asr
|
||||
readme: "**OpenAI-compatible** ⚠️ remote"
|
||||
readme: "**OpenAI-compatible** ⚠️ configured server"
|
||||
|
||||
# Doc files that must exist (the install path users are sent to).
|
||||
docs:
|
||||
|
||||
@@ -7,8 +7,8 @@ working VoiceStudio install on a Debian / Ubuntu / Fedora / Arch host.
|
||||
|
||||
### Using the AppImage
|
||||
|
||||
- **Linux x86_64** with a desktop session (X11 or Wayland) capable of running
|
||||
a Tauri / WebKitGTK app.
|
||||
- **Linux x86_64 with glibc 2.39+** and a desktop session (X11 or Wayland)
|
||||
capable of running a Tauri / WebKitGTK app.
|
||||
- **~10 GB free disk** for the app, its Python environment, and model weights.
|
||||
- Optional: an **NVIDIA driver** for CUDA GPU acceleration — the app runs
|
||||
CPU-only without one. For AMD GPUs see [AMD GPU (ROCm)](#amd-gpu-rocm).
|
||||
|
||||
@@ -117,6 +117,17 @@ def test_available_once_base_url_configured(asr_mod, ss):
|
||||
assert ok is True
|
||||
|
||||
|
||||
def test_insecure_stored_url_cannot_bypass_settings_validation(asr_mod, ss):
|
||||
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://asr.example/v1")
|
||||
|
||||
ok, reason = asr_mod.OpenAICompatASRBackend.is_available()
|
||||
|
||||
assert ok is False
|
||||
assert "require HTTPS" in reason
|
||||
with pytest.raises(ValueError, match="require HTTPS"):
|
||||
asr_mod.OpenAICompatASRBackend()
|
||||
|
||||
|
||||
# ── response adaptation ─────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@@ -180,6 +191,9 @@ def test_client_disables_sdk_retries(asr_mod, ss, monkeypatch, tmp_path):
|
||||
audio.write_bytes(b"RIFF....WAVEfmt ")
|
||||
asr_mod.OpenAICompatASRBackend().transcribe(str(audio))
|
||||
assert captured_kwargs[0]["max_retries"] == 0
|
||||
transport = captured_kwargs[0]["http_client"]
|
||||
assert transport.follow_redirects is False
|
||||
transport.close()
|
||||
|
||||
|
||||
# ── settings endpoints ───────────────────────────────────────────────────────
|
||||
@@ -235,6 +249,22 @@ def test_rejects_a_base_url_without_scheme(settings_mod):
|
||||
)
|
||||
|
||||
|
||||
def test_rejects_plain_http_for_non_loopback_server(settings_mod):
|
||||
from fastapi import HTTPException
|
||||
|
||||
with pytest.raises(HTTPException, match="require HTTPS"):
|
||||
settings_mod.set_asr_openai_compat(
|
||||
settings_mod._ASROpenAICompatBody(base_url="http://asr.example/v1")
|
||||
)
|
||||
|
||||
|
||||
def test_accepts_https_for_non_loopback_server(settings_mod):
|
||||
state = settings_mod.set_asr_openai_compat(
|
||||
settings_mod._ASROpenAICompatBody(base_url="https://asr.example/v1/")
|
||||
)
|
||||
assert state["base_url"] == "https://asr.example/v1"
|
||||
|
||||
|
||||
def test_registered_in_backend_list(asr_mod):
|
||||
assert "openai-compat-asr" in asr_mod._REGISTRY
|
||||
assert asr_mod._REGISTRY["openai-compat-asr"] is asr_mod.OpenAICompatASRBackend
|
||||
@@ -249,16 +279,16 @@ def test_engine_reads_fresh_config_per_call(asr_mod, ss):
|
||||
next transcribe — the backend is instantiated fresh per call
|
||||
(get_active_asr_backend) and reads settings_store in __init__, so no
|
||||
backend restart is ever required after a config change."""
|
||||
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://old:1/v1")
|
||||
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "https://old.example/v1")
|
||||
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_MODEL_KEY, "old-model")
|
||||
first = asr_mod.OpenAICompatASRBackend()
|
||||
assert first._base_url == "http://old:1/v1"
|
||||
assert first._base_url == "https://old.example/v1"
|
||||
assert first._model == "old-model"
|
||||
|
||||
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://new:2/v1")
|
||||
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "https://new.example/v1")
|
||||
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_MODEL_KEY, "new-model")
|
||||
second = asr_mod.OpenAICompatASRBackend()
|
||||
assert second._base_url == "http://new:2/v1"
|
||||
assert second._base_url == "https://new.example/v1"
|
||||
assert second._model == "new-model"
|
||||
|
||||
|
||||
@@ -320,6 +350,17 @@ def test_probe_rejects_schemeless_url_before_any_network(asr_mod, monkeypatch):
|
||||
assert out == {**out, "ok": False, "status": "invalid_url"}
|
||||
|
||||
|
||||
def test_probe_rejects_non_loopback_http_before_any_network(asr_mod, monkeypatch):
|
||||
import httpx
|
||||
|
||||
def _boom(**kw): # pragma: no cover — must never be constructed
|
||||
raise AssertionError("network client constructed for an insecure URL")
|
||||
|
||||
monkeypatch.setattr(httpx, "Client", _boom)
|
||||
out = asr_mod.probe_openai_compat_server(base_url="http://asr.example/v1")
|
||||
assert out == {**out, "ok": False, "status": "invalid_url"}
|
||||
|
||||
|
||||
def test_probe_ok_reports_latency_and_model_found(asr_mod, ss, monkeypatch):
|
||||
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://localhost:8080/v1/")
|
||||
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_MODEL_KEY, "qwen3-asr")
|
||||
@@ -335,6 +376,7 @@ def test_probe_ok_reports_latency_and_model_found(asr_mod, ss, monkeypatch):
|
||||
assert out["model_found"] is True
|
||||
assert isinstance(out["latency_ms"], float)
|
||||
assert captured["url"] == "http://localhost:8080/v1/models" # trailing / trimmed
|
||||
assert captured["client_kwargs"]["follow_redirects"] is False
|
||||
# No key configured → the probe must not invent an Authorization header.
|
||||
assert "Authorization" not in captured["headers"]
|
||||
|
||||
|
||||
Reference in New Issue
Block a user