fix(asr): secure configured endpoints and refresh guidance (#1751)

Refreshes README and linked docs with accurate installation, platform, privacy, API, and model-license guidance; documents local gigastt; and pins OpenAI-compatible ASR traffic to the configured secure origin. Closes #1736.
This commit is contained in:
Palash Debnath
2026-09-02 04:41:36 +05:30
committed by GitHub
parent c9a468e500
commit 08569397d3
10 changed files with 187 additions and 77 deletions
+2
View File
@@ -22,12 +22,14 @@ the frozen-backend fallback mirror it for their toolchains.
### Docs
- Local gigastt is now documented as a supported OpenAI-compatible ASR endpoint, with loopback privacy distinguished from remote servers (#1736) — thanks @ekhodzitsky!
- The CosyVoice guide now states that packaged builds have no one-click runtime installer and records the exact readiness checks exposed by [Discussion 1631](https://github.com/debpalash/VoiceStudio/discussions/1631).
- A production private-API guide now covers pinned containers, root credentials, network isolation, streaming proxies, health checks, upgrades, and benchmark evidence (#1720)
- RX 6700 XT/gfx1031 over WSL2 ROCDXG is now explicitly unverified until a published end-to-end GPU workload proves the mapped path (#1716)
### Fixed
- OpenAI-compatible ASR now requires HTTPS outside loopback and refuses redirects so audio stays on the configured origin (#1736)
- Windows isolated engines now retain direct Job ownership without an extra Python supervisor process that can deadlock the child loader (#1734)
- The setup splash now waits through the backend's full startup budget instead of reporting slow Windows CUDA initialization as stuck after two minutes (#1749)
- Dubbing jobs can now reuse every source-language code produced by automatic ASR detection without a 400 error on the next upload (#1737)
+10 -4
View File
@@ -10,10 +10,10 @@ Copyright 2024-present Palash Debnath and VoiceStudio contributors.
VoiceStudio is **free and open-source software, licensed under the GNU
Affero General Public License, Version 3 (AGPL-3.0)**. You are free to use,
copy, modify, and redistribute it — and that **includes commercial and internal
business use**: run the app, use its outputs commercially, sell the audio you
produce with it, provide professional/client services with it, and deploy it
within your organization.
copy, modify, and redistribute it. That **includes commercial and internal
business use** of the application itself. Model weights, tokenizers, and other
third-party assets retain their own terms; this application license does not
grant or summarize rights under those separate terms.
Because this is the **Affero** GPL, one additional obligation applies: if you
modify VoiceStudio and make that modified version available to others over
@@ -41,6 +41,12 @@ is **separately licensed under Apache License 2.0** by its upstream authors and
is not relicensed here. Apache License 2.0 is compatible with, and may be
combined under, the GNU AGPL-3.0. See `pyproject.toml`.
Downloaded model weights are not relicensed by VoiceStudio. The default
`k2-fsa/OmniVoice` model card identifies its code as Apache-2.0 and pretrained
weights as CC-BY-NC. Its `audio_tokenizer/LICENSE` contains separate Boson
Higgs Audio 2 and Meta Llama community terms. A commercial license for
VoiceStudio-owned code does not replace any of those terms.
Third-party dependencies retain their own licenses. See `Cargo.lock`,
`bun.lock`, and `uv.lock` for the resolved set.
+44 -41
View File
@@ -1,12 +1,12 @@
<div align="center">
<a href="https://trendshift.io/repositories/28176?utm_source=repository-badge&amp;utm_medium=badge&amp;utm_campaign=badge-repository-28176" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/repositories/28176" alt="debpalash%2FVoiceStudio | Trendshift" width="250" height="55" /></a>
<a href="https://trendshift.io/repositories/28176?utm_source=repository-badge&amp;utm_medium=badge&amp;utm_campaign=badge-repository-28176" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/repositories/28176" alt="VoiceStudio ranking on Trendshift" width="250" height="55" /></a>
<img src="docs/logo.png" alt="VoiceStudio logo" width="120" height="120" />
<h1>VoiceStudio</h1>
<p><sub>Previously OmniVoice-Studio</sub></p>
<h3>Local voice cloning, dubbing, dictation, and long-form audio.</h3>
<p>16 TTS engines · 11 ASR engines · 646-language catalogue · macOS, Windows, and Linux</p>
<p><strong>Local-first.</strong> No account, API key, subscription, or usage meter for the core workflow.</p>
<h3>Clone voices, dub video, dictate, and produce long-form audio on your own hardware.</h3>
<p>16 TTS engines · 11 ASR engines · 646-language catalogue · macOS, Windows, Linux, and Docker</p>
<p>No account, API key, subscription, or usage meter for the local workflow.</p>
<p>
<a href="#install">Install</a> ·
@@ -38,7 +38,7 @@
</div>
> [!WARNING]
> **Active beta.** Use the [latest release](https://github.com/debpalash/VoiceStudio/releases/latest) for stable work or `main` for current fixes. Report problems through [GitHub Issues](https://github.com/debpalash/VoiceStudio/issues).
> **Active beta.** Use the [latest release](https://github.com/debpalash/VoiceStudio/releases/latest) for stable work. `main` contains the newest fixes and may change between releases. Report problems through [GitHub Issues](https://github.com/debpalash/VoiceStudio/issues).
## At a glance
@@ -51,28 +51,30 @@
| **Compute** | CUDA · Apple Silicon MPS/MLX · ROCm on Linux · CPU · optional remote workers |
| **Interfaces** | Desktop app · local REST/SSE/WebSocket API · OpenAI-compatible audio API · MCP Server |
| **Storage** | Voices, projects, settings, and outputs stay on the machine by default |
| **License** | AGPL-3.0; optional engines keep their own model licenses |
| **License** | AGPL-3.0 application; downloaded models keep their upstream terms |
<a id="install"></a>
## Install
Download a package from the [latest release](https://github.com/debpalash/VoiceStudio/releases/latest), then follow the platform guide.
| Platform | Package | Guide |
|---|---|---|
| macOS 13.3+ | DMG, Apple Silicon | [Install on macOS](docs/install/macos.md) |
| Windows 10/11 | MSI, x64 | [Install on Windows](docs/install/windows.md) |
| macOS 13.3+ | Apple Silicon DMG | [Install on macOS](docs/install/macos.md) |
| Windows 10/11 | x64 MSI; choose the current-user build when listed to install without admin access | [Install on Windows](docs/install/windows.md#install-pre-built-msi) |
| Linux | AppImage, x86_64 with glibc 2.39+ | [Install on Linux](docs/install/linux.md) |
| Docker | CUDA, ROCm, or CPU; worker-only GPU profiles | [Run with Docker](docs/install/docker.md) |
| Docker | CUDA, ROCm, CPU, and worker-only GPU profiles | [Run with Docker](docs/install/docker.md) |
Download packages from the [latest release](https://github.com/debpalash/VoiceStudio/releases/latest). First launch creates a managed Python environment and downloads the default model. Later launches reuse both.
First launch creates a managed Python environment and downloads the default model. Later launches reuse both.
> [!NOTE]
> On macOS, first launch needs a one-time right-click **Open** approval. Intel Macs cannot run the local Python backend; use a [remote backend](docs/install/macos.md) instead.
> On macOS, first launch needs a one-time right-click, then **Open** approval. Intel Macs cannot run the local Python backend; use a [remote backend](docs/install/macos.md) instead.
### First voice
1. Launch VoiceStudio and open **Voice Cloning**.
2. Add a clean voice sample. Three seconds works; 515 seconds usually gives a better prompt.
2. Add a clean voice sample. Three seconds works; 5 to 15 seconds usually gives a better prompt.
3. Enter text, choose a language, then select **Generate**.
### Run from source
@@ -159,7 +161,7 @@ Requirements vary by engine. These values cover the default local workflow.
| **Disk** | 10 GB free | 20 GB+ SSD |
| **GPU** | Optional; CPU mode is supported | NVIDIA CUDA or Apple Silicon |
| **VRAM** | 4 GB when using a GPU | 8 GB+; large optional engines need more |
| **Python from source** | 3.11+ | 3.113.12 |
| **Python from source** | 3.11+ | 3.11 or 3.12 |
ROCm is Linux-only and opt-in. Windows AMD/Ryzen AI uses CPU. Systems with limited VRAM offload work to CPU when required. See [performance](docs/performance.md), [benchmarks](docs/benchmarks.md), and [engine disk usage](docs/engines/disk-usage.md).
@@ -175,22 +177,22 @@ Engine support is capability-specific. Check cloning, language, platform, memory
| Engine | Languages | Clone | Instruct | Linux | macOS ARM | Windows | License |
|---|:---:|:---:|:---:|:---:|:---:|:---:|---|
| **VoiceStudio** (default, powered by k2-fsa/OmniVoice) | 600+ | Yes | Yes | CUDA/CPU | MPS | CUDA/CPU | [AGPL-3.0](LICENSE) app · [Apache-2.0](LICENSE-NOTICE.md) model |
| **VoiceStudio** (default, powered by k2-fsa/OmniVoice) | 600+ | Yes | Yes | CUDA/CPU | MPS | CUDA/CPU | [AGPL-3.0](LICENSE) app · [Apache-2.0 code, CC-BY-NC weights](https://huggingface.co/k2-fsa/OmniVoice#license |
| **CosyVoice 3** | 9 + 18 dialects | Yes | Yes | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
| **GPT-SoVITS** | 5 | Yes | | CUDA/CPU | | CUDA/CPU | MIT |
| **GPT-SoVITS** | 5 | Yes | No | CUDA/CPU | No | CUDA/CPU | MIT |
| **VoxCPM2** | 30 | Yes | Yes | CUDA/CPU | MPS | CUDA/CPU | Apache-2.0 |
| **MOSS-TTS-Nano** | 20 | Yes | | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
| **KittenTTS** | English | | | CPU | CPU | CPU | MIT |
| **MLX-Audio** | Model-dependent | Varies | Varies | | MLX | | Varies |
| **Sherpa-ONNX** | 20+ | | | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
| **IndexTTS 2.5** ⚡ | ZH · EN · JA · ES · AR | Yes | | CUDA/CPU | CPU | CUDA/CPU | Bilibili model license¹ |
| **OmniVoice GGUF** ⚡ | 600+ | Yes | Yes | CUDA/CPU | MPS/CPU | CUDA/CPU | [AGPL-3.0](LICENSE) app · [Apache-2.0](LICENSE-NOTICE.md) model |
| **OmniVoice (subprocess)** ⚡ | 600+ | Yes | Yes | CUDA/CPU | MPS | CUDA/CPU | [AGPL-3.0](LICENSE) app · [Apache-2.0](LICENSE-NOTICE.md) model |
| **PocketTTS** ⚡ | EN · FR · DE · PT · IT · ES | Yes | | CPU | CPU | CPU | CC-BY-4.0, gated² |
| **Supertonic 3** ⚡ | 31 | | | CPU | CPU | CPU | OpenRAIL-M |
| **MOSS-TTS-v1.5** ⚡ | 31 | Yes | | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
| **dots.tts** ⚡ | 24 | Yes | | CUDA/CPU | CPU | | Apache-2.0 |
| **Confucius4-TTS** ⚡ | 14 | Yes | | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
| **MOSS-TTS-Nano** | 20 | Yes | No | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
| **KittenTTS** | English | No | No | CPU | CPU | CPU | MIT |
| **MLX-Audio** | Model-dependent | Varies | Varies | No | MLX | No | Varies |
| **Sherpa-ONNX** | 20+ | No | No | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
| **IndexTTS 2.5** ⚡ | ZH · EN · JA · ES · AR | Yes | No | CUDA/CPU | CPU | CUDA/CPU | Bilibili model license¹ |
| **OmniVoice GGUF** ⚡ | 600+ | Yes | Yes | CUDA/CPU | MPS/CPU | CUDA/CPU | [AGPL-3.0](LICENSE) app · [review the derivative model terms](https://huggingface.co/Serveurperso/OmniVoice-GGUF#license |
| **OmniVoice (subprocess)** ⚡ | 600+ | Yes | Yes | CUDA/CPU | MPS | CUDA/CPU | [AGPL-3.0](LICENSE) app · [Apache-2.0 code, CC-BY-NC weights](https://huggingface.co/k2-fsa/OmniVoice#license |
| **PocketTTS** ⚡ | EN · FR · DE · PT · IT · ES | Yes | No | CPU | CPU | CPU | CC-BY-4.0, gated² |
| **Supertonic 3** ⚡ | 31 | No | No | CPU | CPU | CPU | OpenRAIL-M |
| **MOSS-TTS-v1.5** ⚡ | 31 | Yes | No | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
| **dots.tts** ⚡ | 24 | Yes | No | CUDA/CPU | CPU | No | Apache-2.0 |
| **Confucius4-TTS** ⚡ | 14 | Yes | No | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
⚡ Installed or registered on demand.
@@ -198,6 +200,8 @@ Engine support is capability-specific. Check cloning, language, platform, memory
² PocketTTS shows its gated-access and CC-BY-4.0 terms before first use.
³ The OmniVoice snapshot also includes an audio tokenizer under separate [Boson Higgs Audio 2 and Meta Llama community terms](https://huggingface.co/k2-fsa/OmniVoice/blob/main/audio_tokenizer/LICENSE). VoiceStudio's application license does not replace model or tokenizer terms.
Clone-less engines cannot preserve a reference speaker in dubbing or pinned-voice batch jobs. VoiceStudio rejects those jobs instead of silently changing engines. Heavy engines have separate memory and platform limits; check their engine guide first.
<a id="asr-engines"></a>
@@ -216,7 +220,7 @@ Clone-less engines cannot preserve a reference speaker in dubbing or pinned-voic
| **Moonshine** | `moonshine` | English | Low-power, low-latency ONNX |
| **FunASR** | `funasr` | 50+ | VAD and inline diarization |
| **sherpa-onnx** (live dictation) | `sherpa-onnx-asr` | Model-dependent | Streaming CPU dictation |
| **OpenAI-compatible** ⚠️ remote | `openai-compat-asr` | Server-dependent | Qwen3-ASR or another compatible endpoint; audio leaves the machine |
| **OpenAI-compatible** ⚠️ configured server | `openai-compat-asr` | Server-dependent | Local gigastt/Qwen3-ASR or a remote endpoint; audio goes only to that server |
WhisperX and Faster-Whisper retry with `int8` when efficient `float16` is unavailable. Pin `ASR_COMPUTE_TYPE=int8` or `float32` only if automatic selection still fails.
@@ -251,8 +255,8 @@ FastAPI backend
- The desktop talks to a loopback-only backend on `localhost:3900`.
- Loopback API calls need no server key. Remote access requires a share PIN or API key.
- Remote workers and OpenAI-compatible ASR are opt-in. The UI identifies when audio leaves the machine.
- Analytics is off until consent. If enabled, it sends allowlisted, content-free usage metadata—not text, audio, file names, or projects.
- Remote workers and OpenAI-compatible ASR are opt-in. Loopback ASR may use HTTP and keeps audio on the machine; non-loopback endpoints require HTTPS, and redirects are not followed.
- Analytics is off until consent. If enabled, it sends allowlisted, content-free usage metadata. It never sends text, audio, file names, or projects.
<a id="api"></a>
@@ -287,12 +291,11 @@ with client.audio.speech.with_streaming_response.create(
response.stream_to_file("speech.wav")
```
The bundled Rust control sidecar also lets Herdr, coding agents, VS Code,
desktop apps, and TUIs trigger the existing system-wide dictation flow or reuse
its safe native insertion. See the [speech platform guide](docs/speech-platform.md).
The full API reference is in **Settings → OpenAPI Reference**. For LAN,
Tailscale, or proxy access, read [API authentication](docs/api-auth.md) before
exposing the backend.
The bundled Rust control sidecar lets Herdr, coding agents, VS Code, desktop apps,
and TUIs trigger the system-wide dictation flow or reuse its native text
insertion. See the [speech platform guide](docs/speech-platform.md). The full API
reference is in **Settings → OpenAPI Reference**. For LAN, Tailscale, or proxy
access, read [API authentication](docs/api-auth.md) before exposing the backend.
### Agent skills
@@ -337,19 +340,19 @@ Apple Silicon is supported with MPS and MLX options. Intel Macs cannot run the l
<details>
<summary><strong>How much VRAM do I need?</strong></summary>
A GPU is optional. Use 4 GB VRAM as the minimum for accelerated work and 8 GB+ for the default multi-stage workflow. Large optional engines can require 1216 GB or more. Check the [benchmarks](docs/benchmarks.md) and engine guide.
A GPU is optional. Use 4 GB VRAM as the minimum for accelerated work and 8 GB+ for the default multi-stage workflow. Large optional engines can require 12 to 16 GB or more. Check the [benchmarks](docs/benchmarks.md) and engine guide.
</details>
<details>
<summary><strong>Why does a longer reference clip not always improve the clone?</strong></summary>
Cloning is zero-shot: the clip is a prompt, not training data. Use 515 seconds of one speaker, close to the microphone, without music, noise, or reverb. Match the tone and pace you want in the output. For training, see [data preparation](docs/data_preparation.md) and [training](docs/training.md).
Cloning is zero-shot: the clip is a prompt, not training data. Use 5 to 15 seconds of one speaker, close to the microphone, without music, noise, or reverb. Match the tone and pace you want in the output. For training, see [data preparation](docs/data_preparation.md) and [training](docs/training.md).
</details>
<details>
<summary><strong>Can I use generated audio commercially?</strong></summary>
Yes under VoiceStudio's AGPL-3.0 terms. Optional engines and model weights may use different licenses; review the selected engine's license before commercial use.
VoiceStudio's application license does not restrict generated audio, but it does not grant rights under a model's separate terms. The default OmniVoice repository labels its pretrained weights CC-BY-NC and includes a tokenizer under separate community terms. Review the selected model terms before commercial use.
</details>
<details>
@@ -379,9 +382,9 @@ VoiceStudio is free and has no paid tier. Donations fund development and infrast
## License
VoiceStudio is licensed under [AGPL-3.0](LICENSE). You may run it, modify it, use it internally, and sell generated audio. If you modify VoiceStudio and provide that modified version as a network service, AGPL requires you to offer the corresponding source under the same license. A commercial license is available for proprietary embedding; contact **VoiceStudio@palash.dev**. See [LICENSE-NOTICE.md](LICENSE-NOTICE.md) for the plain-language scope.
VoiceStudio is licensed under [AGPL-3.0](LICENSE). You may run it, modify it, and use it internally. The application license itself does not restrict selling generated audio, but downloaded model and tokenizer terms may. If you modify VoiceStudio and provide that modified version as a network service, AGPL requires you to offer the corresponding source under the same license. A commercial license for VoiceStudio-owned code is available for proprietary embedding; it does not relicense third-party models. Contact **VoiceStudio@palash.dev**. See [LICENSE-NOTICE.md](LICENSE-NOTICE.md) for the plain-language scope.
Optional engines and downloaded models retain their own licenses. The bundled `omnivoice/` model remains Apache-2.0 upstream.
Optional engines and downloaded models retain their own licenses. The bundled `omnivoice/` Python code is Apache-2.0 upstream; the default downloaded weights and audio tokenizer use separate terms.
## Acknowledgments
+4 -3
View File
@@ -1035,9 +1035,10 @@ def set_asr_openai_compat(body: _ASROpenAICompatBody):
from services import asr_backend, settings_store
if body.base_url is not None:
url = body.base_url.strip().rstrip("/")
if url and not url.startswith(("http://", "https://")):
raise HTTPException(status_code=400, detail="Base URL must start with http(s)://")
try:
url = asr_backend.normalize_openai_compat_asr_base_url(body.base_url)
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
settings_store.set_text(asr_backend._ASR_OPENAI_COMPAT_BASE_URL_KEY, url)
if body.model is not None:
settings_store.set_text(
+61 -9
View File
@@ -24,6 +24,7 @@ faster-whisper because it's available on every platform we ship to).
from __future__ import annotations
import asyncio
import ipaddress
import logging
import os
import re
@@ -31,6 +32,7 @@ import contextlib
import threading
import time
import weakref
from urllib.parse import urlsplit
from utils.containment import contain_system_exit
from abc import ABC, abstractmethod
@@ -2000,6 +2002,42 @@ _ASR_OPENAI_COMPAT_MODEL_KEY = "asr.openai_compat.model"
_ASR_OPENAI_COMPAT_SECRET_NAME = "asr_openai_compat_key"
def normalize_openai_compat_asr_base_url(value: str) -> str:
"""Normalize a safe ASR endpoint, allowing plain HTTP only on loopback."""
base = (value or "").strip().rstrip("/")
if not base:
return ""
try:
parsed = urlsplit(base)
_ = parsed.port
except (TypeError, ValueError) as exc:
raise ValueError("Invalid OpenAI-compatible ASR base URL") from exc
scheme = parsed.scheme.lower()
if (
scheme not in {"http", "https"}
or not parsed.hostname
or parsed.username is not None
or parsed.password is not None
or parsed.query
or parsed.fragment
):
raise ValueError(
"OpenAI-compatible ASR base URL must be a credential-free HTTP(S) URL"
)
host = parsed.hostname.lower()
loopback = host == "localhost"
if not loopback:
try:
address = ipaddress.ip_address(host)
address = getattr(address, "ipv4_mapped", None) or address
loopback = address.is_loopback
except ValueError:
loopback = False
if scheme == "http" and not loopback:
raise ValueError("Non-loopback OpenAI-compatible ASR endpoints require HTTPS")
return base
def resolve_openai_compat_asr_base_url() -> str:
from services import settings_store
return (
@@ -2063,7 +2101,7 @@ def probe_openai_compat_server(
maps to a translated message:
not_configured no base URL anywhere
invalid_url base URL without an http(s):// scheme
invalid_url malformed URL or non-loopback HTTP endpoint
ok 2xx ``model_found`` says whether the configured
model appears in the server's list (None = unknown)
ok_no_models 404/405/501 reachable, but no /models endpoint
@@ -2078,7 +2116,7 @@ def probe_openai_compat_server(
from core.scrub import scrub_text
base = (base_url if base_url is not None else resolve_openai_compat_asr_base_url()).strip().rstrip("/")
configured_base = base_url if base_url is not None else resolve_openai_compat_asr_base_url()
mdl = (model if model is not None else resolve_openai_compat_asr_model()).strip()
if api_key is None:
key = resolve_openai_compat_asr_api_key()
@@ -2094,9 +2132,11 @@ def probe_openai_compat_server(
"model_found": None,
"detail": None,
}
if not base:
if not configured_base.strip():
return out
if not base.startswith(("http://", "https://")):
try:
base = normalize_openai_compat_asr_base_url(configured_base)
except ValueError:
out["status"] = "invalid_url"
return out
@@ -2107,7 +2147,7 @@ def probe_openai_compat_server(
try:
with httpx.Client(
timeout=httpx.Timeout(timeout_s, connect=min(5.0, timeout_s)),
follow_redirects=True,
follow_redirects=False,
) as client:
resp = client.get(f"{base}/models", headers=headers)
except httpx.TimeoutException as exc:
@@ -2170,13 +2210,20 @@ class OpenAICompatASRBackend(ASRBackend):
gpu_compat = ("cpu",) # network client only — no local compute
def __init__(self):
self._base_url = resolve_openai_compat_asr_base_url()
self._base_url = normalize_openai_compat_asr_base_url(
resolve_openai_compat_asr_base_url()
)
self._model = resolve_openai_compat_asr_model()
@classmethod
def is_available(cls) -> tuple[bool, str]:
if not resolve_openai_compat_asr_base_url():
base_url = resolve_openai_compat_asr_base_url()
if not base_url:
return False, "Configure a server endpoint in Model Catalogue → Engines"
try:
normalize_openai_compat_asr_base_url(base_url)
except ValueError as exc:
return False, str(exc)
try:
import openai # noqa: F401
except ImportError:
@@ -2184,13 +2231,18 @@ class OpenAICompatASRBackend(ASRBackend):
return True, "ready"
def _client(self):
from openai import OpenAI
from openai import DefaultHttpxClient, OpenAI
api_key = resolve_openai_compat_asr_api_key() or "not-needed"
# max_retries=0: mirrors llm_skills.resolve_skill_client — a
# rate-limited/slow server retrying inside the SDK would blow past
# whatever bounded timeout the caller (dub transcribe, dictation)
# expects from a single call.
return OpenAI(base_url=self._base_url, api_key=api_key, max_retries=0)
return OpenAI(
base_url=self._base_url,
api_key=api_key,
max_retries=0,
http_client=DefaultHttpxClient(follow_redirects=False),
)
def transcribe(self, audio_path: str, *, word_timestamps: bool = True) -> dict:
logger.info(
+1 -1
View File
@@ -54,7 +54,7 @@ approval), [Windows](../install/windows.md), [Linux](../install/linux.md),
| Moonshine | [moonshine](moonshine.md) | CPU | edge/low-power, no timestamps | `pip install` (see guide) |
| FunASR (SenseVoice) | [funasr](funasr.md) | CUDA · CPU | 50+ languages, inline diarization | `pip install funasr` |
| Sherpa-ONNX dictation | [sherpa-onnx-asr](sherpa-onnx-asr.md) | CPU | live streaming dictation | curated model download |
| OpenAI-compatible (remote) | [openai-compatible-asr](openai-compatible-asr.md) | network | offloading to a server (audio leaves the machine) | Model Catalogue |
| OpenAI-compatible (local or remote) | [openai-compatible-asr](openai-compatible-asr.md) | network | a configured endpoint; loopback stays local | Model Catalogue |
Speaker diarization is not an engine registry of its own — the dub pipeline
uses pyannote (HF-gated; see [diarization](../features/diarization.md)) and
+16 -12
View File
@@ -1,11 +1,11 @@
# VoiceStudio OpenAI-Compatible Remote ASR
# VoiceStudio: OpenAI-Compatible ASR
Point transcription at **any** server exposing an OpenAI-compatible
`POST /v1/audio/transcriptions` endpoint LM Studio or a llama.cpp-style
local server, a self-hosted Qwen3-ASR/FunASR/SenseVoice box on your network,
Groq, or OpenAI's own Whisper API. Unlike every other ASR engine, this one
runs no model locally: it's a pure network client, so it needs no install
and claims no GPU.
`POST /v1/audio/transcriptions` endpoint: gigastt, LM Studio, or a
llama.cpp-style server on the same machine; a self-hosted
Qwen3-ASR/FunASR/SenseVoice box on your network; Groq; or OpenAI's Whisper
API. VoiceStudio is a pure client in this mode, so the configured server owns
model installation and compute.
## Setup
@@ -39,13 +39,18 @@ picks local engines, and the app works fully with this engine unconfigured.
| Server | Server URL | Model | API key |
| --- | --- | --- | --- |
| [gigastt](https://github.com/ekhodzitsky/gigastt) (local Russian specialist) | `http://127.0.0.1:9876/v1` | `gigaam-v3-rnnt` | none |
| LM Studio (local) | `http://localhost:1234/v1` | the model name shown in LM Studio | none |
| llama.cpp / whisper.cpp server (local) | `http://localhost:8080/v1` | whatever the server loads (often ignored) | none |
| speaches / faster-whisper-server (local) | `http://localhost:8000/v1` | e.g. `Systran/faster-whisper-large-v3` | none |
| Self-hosted Qwen3-ASR / FunASR (LAN box) | `http://<host>:8000/v1` | your deployment's model id | if you enabled auth |
| Self-hosted Qwen3-ASR / FunASR (LAN box) | `https://<host>:8000/v1` | your deployment's model id | if you enabled auth |
| Groq | `https://api.groq.com/openai/v1` | `whisper-large-v3` | required |
| OpenAI | `https://api.openai.com/v1` | `whisper-1` | required |
Plain HTTP is accepted only for exact loopback hosts such as `localhost`,
`127.0.0.1`, and `::1`. Every non-loopback endpoint must use HTTPS. VoiceStudio
does not follow redirects from transcription or connection-probe requests.
Local servers vary in which endpoints they implement — if **Test
connection** reports the server is reachable but doesn't list models,
transcription may still work; run a small dictation or dub-transcribe to
@@ -60,8 +65,7 @@ path returns word-level timestamps — that's not part of this API.
## Privacy note
Unlike every other ASR engine in VoiceStudio, audio sent through this backend
leaves your machine — to whatever server **you** configured, and nowhere
else. If that's a self-hosted server on your own network, nothing leaves
your control; if it's a third-party API (Groq, OpenAI's, or someone
else's), review their data handling before sending anything sensitive.
Audio goes only to the server **you** configure. A loopback URL such as the
gigastt example keeps it on the same machine and may use HTTP. LAN and public
endpoints require HTTPS, and redirects are not followed. Review the configured
server's data handling before sending anything sensitive.
+1 -1
View File
@@ -82,7 +82,7 @@ asr_engines:
- id: sherpa-onnx-asr
readme: "**sherpa-onnx** (live dictation)"
- id: openai-compat-asr
readme: "**OpenAI-compatible** ⚠️ remote"
readme: "**OpenAI-compatible** ⚠️ configured server"
# Doc files that must exist (the install path users are sent to).
docs:
+2 -2
View File
@@ -7,8 +7,8 @@ working VoiceStudio install on a Debian / Ubuntu / Fedora / Arch host.
### Using the AppImage
- **Linux x86_64** with a desktop session (X11 or Wayland) capable of running
a Tauri / WebKitGTK app.
- **Linux x86_64 with glibc 2.39+** and a desktop session (X11 or Wayland)
capable of running a Tauri / WebKitGTK app.
- **~10 GB free disk** for the app, its Python environment, and model weights.
- Optional: an **NVIDIA driver** for CUDA GPU acceleration — the app runs
CPU-only without one. For AMD GPUs see [AMD GPU (ROCm)](#amd-gpu-rocm).
+46 -4
View File
@@ -117,6 +117,17 @@ def test_available_once_base_url_configured(asr_mod, ss):
assert ok is True
def test_insecure_stored_url_cannot_bypass_settings_validation(asr_mod, ss):
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://asr.example/v1")
ok, reason = asr_mod.OpenAICompatASRBackend.is_available()
assert ok is False
assert "require HTTPS" in reason
with pytest.raises(ValueError, match="require HTTPS"):
asr_mod.OpenAICompatASRBackend()
# ── response adaptation ─────────────────────────────────────────────────────
@@ -180,6 +191,9 @@ def test_client_disables_sdk_retries(asr_mod, ss, monkeypatch, tmp_path):
audio.write_bytes(b"RIFF....WAVEfmt ")
asr_mod.OpenAICompatASRBackend().transcribe(str(audio))
assert captured_kwargs[0]["max_retries"] == 0
transport = captured_kwargs[0]["http_client"]
assert transport.follow_redirects is False
transport.close()
# ── settings endpoints ───────────────────────────────────────────────────────
@@ -235,6 +249,22 @@ def test_rejects_a_base_url_without_scheme(settings_mod):
)
def test_rejects_plain_http_for_non_loopback_server(settings_mod):
from fastapi import HTTPException
with pytest.raises(HTTPException, match="require HTTPS"):
settings_mod.set_asr_openai_compat(
settings_mod._ASROpenAICompatBody(base_url="http://asr.example/v1")
)
def test_accepts_https_for_non_loopback_server(settings_mod):
state = settings_mod.set_asr_openai_compat(
settings_mod._ASROpenAICompatBody(base_url="https://asr.example/v1/")
)
assert state["base_url"] == "https://asr.example/v1"
def test_registered_in_backend_list(asr_mod):
assert "openai-compat-asr" in asr_mod._REGISTRY
assert asr_mod._REGISTRY["openai-compat-asr"] is asr_mod.OpenAICompatASRBackend
@@ -249,16 +279,16 @@ def test_engine_reads_fresh_config_per_call(asr_mod, ss):
next transcribe the backend is instantiated fresh per call
(get_active_asr_backend) and reads settings_store in __init__, so no
backend restart is ever required after a config change."""
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://old:1/v1")
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "https://old.example/v1")
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_MODEL_KEY, "old-model")
first = asr_mod.OpenAICompatASRBackend()
assert first._base_url == "http://old:1/v1"
assert first._base_url == "https://old.example/v1"
assert first._model == "old-model"
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://new:2/v1")
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "https://new.example/v1")
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_MODEL_KEY, "new-model")
second = asr_mod.OpenAICompatASRBackend()
assert second._base_url == "http://new:2/v1"
assert second._base_url == "https://new.example/v1"
assert second._model == "new-model"
@@ -320,6 +350,17 @@ def test_probe_rejects_schemeless_url_before_any_network(asr_mod, monkeypatch):
assert out == {**out, "ok": False, "status": "invalid_url"}
def test_probe_rejects_non_loopback_http_before_any_network(asr_mod, monkeypatch):
import httpx
def _boom(**kw): # pragma: no cover — must never be constructed
raise AssertionError("network client constructed for an insecure URL")
monkeypatch.setattr(httpx, "Client", _boom)
out = asr_mod.probe_openai_compat_server(base_url="http://asr.example/v1")
assert out == {**out, "ok": False, "status": "invalid_url"}
def test_probe_ok_reports_latency_and_model_found(asr_mod, ss, monkeypatch):
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://localhost:8080/v1/")
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_MODEL_KEY, "qwen3-asr")
@@ -335,6 +376,7 @@ def test_probe_ok_reports_latency_and_model_found(asr_mod, ss, monkeypatch):
assert out["model_found"] is True
assert isinstance(out["latency_ms"], float)
assert captured["url"] == "http://localhost:8080/v1/models" # trailing / trimmed
assert captured["client_kwargs"]["follow_redirects"] is False
# No key configured → the probe must not invent an Authorization header.
assert "Authorization" not in captured["headers"]