Palash Debnath
3008f89919
The engine matrix (five columns, three-line rows, every chip on every row) becomes a shadcn table with three columns — Engine · Runs on · Status — and one primary action per row (Use / Install). Everything else lives in a detail panel for the selected row: GPU compatibility chips, isolation, hints and reasons, health and self-test probes, one-click install progress, setup snippet, disk usage, docs, license, the curated-model picker, and now the engine's downloadable WEIGHTS. Weights belong to their engine: every models.yaml entry names the backend ids that load it (`engines:`), the detail panel lists and installs them (EngineWeights, on the model store's install/cancel/remove flow via the extracted useModelDownloads hook), and the sherpa-onnx engine shows its dictation-model picker there. The page's "Downloaded weights" list and recommendation card are gone; only weights no engine owns (speaker diarisation) remain in a small "Other weights" list. A backend test pins the mapping: every entry has an `engines` list and every id is a real backend. - useEngineInventory: the matrix's state machines extracted verbatim (shared/local fetch, residency, health/self-test cooldowns, install poller with overlap guard + epoch, disk-usage generations, license). - Row status phrases: GPU active / CPU fallback / CPU / Available / Needs setup / Installing… / failed; routing "unavailable" never reads Ready. Group captions keep "Ready to use" / "Add more engines". - Engine titles read "Engines" (each locale's own word); backend "Model Catalogue → Engines/Models" messages and docs updated to the new structure. - Dead matrix CSS (phone-tier grid) removed; scopeReco and RecoBanner gone.
2.5 KiB
VoiceStudio — Parakeet TDT v3 (MLX) Engine
NVIDIA's Parakeet TDT v3 on the Apple Silicon GPU, via the small pure-Python
parakeet-mlx package. It gives Macs the Parakeet tier CUDA/CPU users get
through NeMo or sherpa-onnx: 25 European languages, word timestamps from
the TDT decoder itself (no wav2vec2 alignment pass needed), ~1.2 GB download,
~2 GB unified memory, dictation-grade speed on the GPU.
Unlike nemo-parakeet it needs no nemo_toolkit (whose
transformers pin conflicts with the app's) — it is installed by default on
Apple Silicon source installs since 0.3.22.
Selecting it
- Model Catalogue, ASR tab → Use on the Parakeet TDT v3
(MLX) row, or
OMNIVOICE_ASR_BACKEND=parakeet-mlx. - Dictation prefers it automatically: once the model weights are installed (Model Catalogue (ASR tab → the engine's Weights) — the auto-pick never triggers a download), live dictation/capture uses it whenever your system language is one of the 25 covered European languages. Other languages keep the multilingual Whisper engine, so dictation coverage never regresses.
Best at
- Live dictation on a Mac — TDT decoding is fast enough for the capture path, at Parakeet's better-than-Whisper English WER.
- European-language transcription with word timestamps at a fraction of whisper-large-v3's memory and compute.
For languages outside the 25 (CJK, Arabic, ...), use mlx-whisper instead.
Platform support
Apple Silicon only — the same shared MLX platform gate as mlx-whisper refuses Linux, Windows, and Intel Macs before any import (#390). It runs on the unified-memory GPU; there is no CPU tier.
Model selection
ASR_MODEL_PARAKEET_MLX — default mlx-community/parakeet-tdt-0.6b-v3.
Weights download on first load — see
downloading-models.
Quirks
- Long files are processed in 120 s chunks internally to bound unified-memory use; short dictation buffers and dub chunks are unaffected.
- Parakeet v3 auto-detects among its 25 languages but doesn't expose the pick, so the reported language is the one you requested (or none) — it is never hardcoded to English.
- Word timestamps are merged from the decoder's subword tokens — good for subtitles and dictation; for lip-sync-critical dubbing the wav2vec2-aligned engines (mlx-whisper, whisperx) remain the accuracy tier.
Speed comparisons across engines live in performance.