Files

T

Palash Debnath 3cae853440 feat(catalogue): one page, one axis — setup summary over per-family engines and weights

The Model Catalogue put the same decision on two axes: an Engines pane with
TTS/ASR/LLM tabs and a Models pane with TTS/ASR/Dictation/Diarisation
sections, dictation shown in both, plus storage stats, the HF token and the
voice-preview toggle parked on the model list. Settings → Voice still carried
Engines and Models entries that only pointed back here.

Now the page reads top-down: a SetupSummary (speech, transcription,
dictation, language model — engine, device, one status word, Change), the
engine list for one family, and that family's downloadable weights under it
(TTS under TTS; offline ASR, streaming dictation and diarisation under ASR;
nothing for LLM, whose engines bring their own). One storage line points at
Settings → Storage.

- ModelStoreTab takes a `family` and scopes sections and the recommendation
  preset to it (scopeReco); stats strip, HF-token toolbar and previews
  panel removed from it.
- Settings: Engines/Models categories and CataloguePointer removed; models
  directory → Storage, HF mirror → Network (both restart-flagged), voice
  previews → Storage. "Manage models" in disk usage opens the catalogue.
- Store: openCatalogue takes a family (pane key tolerated, ignored);
  pendingCatalogueTab gone.
- Engine matrix title is now the locale's plain "Engines".
- i18n: catalogue.* summary keys in all 21 locales; pane/pointer keys dropped.
- Docs: "Model Catalogue → Engines" is "Model Catalogue"; "→ Models" is
  "→ Downloaded weights".

2026-09-10 06:46:09 -07:00

2.5 KiB

Raw Blame History

VoiceStudio — MLX Whisper Engine

MLX Whisper runs Whisper on the Apple Silicon GPU via MLX. It exists because CTranslate2 (whisperx / faster-whisper) has no Metal build — on a Mac those engines transcribe on the CPU no matter what GPU is present. Measured on an M2 with whisper-large-v3, one 30 s dub chunk: 90.4 s on WhisperX (CPU) vs 20.5 s on MLX (GPU) — which is why auto-detect picks MLX Whisper on every Apple Silicon machine (#1127).

Selecting it

Nothing to do on Apple Silicon — auto-detect prefers it there.
Or explicitly: Model Catalogue, ASR tab → Use, or OMNIVOICE_ASR_BACKEND=mlx-whisper.

Best at

Dubbing on a Mac — it layers the same wav2vec2 forced alignment WhisperX uses on top of the GPU transcription, so word timing (±10–30 ms) and therefore lip-sync accuracy are unchanged. Same model, same alignment, ~4x the speed.
Dictation/capture — the capture path automatically swaps in mlx-community/whisper-large-v3-turbo (~5x faster than large-v3) unless a sherpa dictation model or parakeet-mlx is preferred.

Platform support

Apple Silicon only. A shared platform gate refuses Linux, Windows, and Intel Macs before any package import, so a stray mlx-whisper wheel on the wrong platform never reports itself available (#390). All other platforms use the CUDA/CPU engines instead.

Model selection

ASR_MODEL — default mlx-community/whisper-large-v3-mlx. Any MLX-format Whisper repo works. Weights download on first load — see downloading-models.
OMNIVOICE_ALIGN_DEVICE — force the wav2vec2 aligner's device. The aligner runs on MPS when it can and falls back to CPU; languages without a bundled aligner (~20 major languages have one) keep Whisper's native word timestamps.

Quirks

Audio is decoded through VoiceStudio's validated ffmpeg rather than the bare ffmpeg PATH lookup mlx-whisper would do on its own — a clean from-source install with no system ffmpeg works fine (#479).
The model is warmed into unified memory in the background, so the first transcribe after startup doesn't pay the load cost.
In a packaged app, a native MLX library that fails to load is reported as "unavailable" (with fallback to another engine) rather than crashing the engine list.

Speed comparisons across engines live in performance.

2.5 KiB Raw Blame History Unescape Escape