Files

T

Palash Debnath 5cab8e0149 feat: rename the product to VoiceStudio (previously OmniVoice-Studio)

Renames what users see. The app, the installers, the window title, the
docs and all 21 locales now say VoiceStudio, with "(previously
OmniVoice-Studio)" noted near the title of each doc surface so people
recognise it.

Deliberately NOT renamed, because renaming any of them silently breaks
an existing install — there is no legacy-path fallback anywhere in this
codebase:

  - bundle identifier com.debpalash.omnivoice-studio (MSI UpgradeCode,
    macOS TCC grants, managed venv, WebView localStorage, the
    single-instance lock)
  - data directories OmniVoice / .omnivoice and omnivoice.db
  - the ~150 OMNIVOICE_* environment variables
  - the X-OmniVoice-* HTTP headers (a wire protocol)
  - the published Docker image paths
  - the OmniVoice ENGINE, which is a model name and not this product

tests/test_identity_paths_survive_the_rename.py pins every one of those
so a future well-meaning sweep cannot orphan a user's library.

Linux .deb users install a new package name and should apt remove
omnivoice-studio; that note is in the changelog.

2026-08-07 01:30:58 +05:30

1.5 KiB

Raw Permalink Blame History

Evaluation

Evaluate VoiceStudio models with standard TTS metrics: WER (intelligibility), SIM-o (speaker similarity), and UTMOS (naturalness).

Supported Test Sets

Test Set	Languages	WER Module	Metrics
LibriSpeech-PC	English	HuBERT WER	WER + Speaker Sim + MOS
Seed-TTS (en)	English	Whisper WER	WER + MOS
Seed-TTS (zh)	Chinese	Paraformer WER	WER + MOS
FLEURS	102 languages	Omnilingual-ASR WER	WER (per-language + macro-avg)
MiniMax Multilingual	24 languages	Whisper + Paraformer	WER + MOS

Prerequisites

pip install omnivoice[eval]
# or
uv sync --extra eval

Quick Start

cd examples
bash run_eval.sh
# run_eval.sh will
# (1) download all required test sets and test models;
# (2) inference and evaluation for each test set.

Metrics Explained

WER (Word Error Rate)

Measures how intelligible the generated speech is by transcribing it with an ASR model and comparing to the reference text. Lower is better. Note that some languages actually use CER (Character Error Rate).

LibriSpeech-PC: HuBERT-based ASR
Seed-TTS: Whisper (en) or Paraformer (zh)
MiniMax: Whisper for non-Chinese, Paraformer for Chinese
FLEURS: Omnilingual-ASR multilingual model

Speaker Similarity

Cosine similarity between speaker embeddings (ECAPA-TDNN + WavLM) of the reference and generated audio. Higher is better.

UTMOS (Predicted MOS)

Neural network that predicts Mean Opinion Score from audio. Higher is better.

1.5 KiB Raw Permalink Blame History