New Settings → System → LLM Skills area: every LLM-powered capability
(Cinematic & Autofit translation, speech-rate slot fitting, glossary
auto-extract, direction parsing, dictation cleanup) becomes a "skill" the
user can toggle or route to a specific provider (local Ollama/LM Studio vs
a remote key) instead of everything riding the one global active provider.
Backend:
- services/llm_skills.py — skill registry + settings_store persistence
(llm_skill.<id>.enabled / .provider), resolution precedence
override > active > none, resolve_skill_client() (OpenAI-compat client
bound to the effective provider; None when disabled/unconfigured) and
skill_backend() (OffBackend when disabled — the exact no-LLM object every
caller already degrades on).
- All five consumption points wired through the registry; a disabled skill
degrades exactly like "no LLM configured" today (Fast translation
fallback, refinement pass-through, heuristic direction parse, no-llm slot
fit, 503 on glossary auto-extract). No new degradation modes; defaults
(enabled + no override) keep existing setups byte-identical.
- OpenAICompatBackend gains an optional bound provider (None = active, the
historical behavior).
- GET /api/settings/llm-skills + PUT /api/settings/llm-skills/{skill_id}
(404 unknown skill/provider); route snapshot updated.
Frontend:
- LLMSkillsPanel (Sparkles, next to LLM Providers): one row per skill —
i18n name/description, enable toggle, provider Select ("Use active
provider" + configured providers, local ones tagged), ready /
needs-setup badge linking to LLM Providers. All strings via t()
(settings.llmskills_*).
Tests: 30 backend (precedence, per-consumption-point disabled semantics,
endpoint round-trips, validation) + 4 panel render/PUT tests. Docs:
translation-engines.md gains an LLM Skills section.
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
P0 — Refinement blocked every dictation final with no timeout. With refinement
auto:true and a slow/dead LLM endpoint, maybe_refine ran unbounded and blocked
the final send in all three capture_ws handlers (~51s measured; the pill hung
"Transcribing…" until the widget's 15s fallback fired). Fix the class: a hard,
env-tunable budget (OMNIVOICE_REFINE_TIMEOUT_S, default 4s) via a new
maybe_refine_async — a slow/dead endpoint now falls back to the unrefined (but
polished) text within the budget and can NEVER delay the final beyond it. The
LLM HTTP call is bounded to the same budget so the orphaned worker unwinds
instead of holding a connection for the client's full 45s. Refinement is now
also fully best-effort in the legacy handler (it can't turn a good final into
an error frame).
P1 — REST /transcribe lacked polish parity. capture.py never applied
polish_text, so REST returned raw "…test" while the WS returned "…test."
Apply text_polish.polish_text to `text` and `refined_text` (segments stay raw),
so the widget POST fallback and MCP/CLI callers match the live socket.
P1 — The #888 "instant first dictation" preload was a no-op. The preload called
warmup() only `if hasattr`, but SherpaDictationBackend had none, and the WS
handlers built a FRESH backend per session so a warm singleton wasn't reused.
Add SherpaDictationBackend.warmup() (builds the recognizer) and share one warm
recognizer per model id across sessions (get_sherpa_dictation_backend, same
invalidation + a shared lock as the capture singleton); each session keeps its
own decode stream. First dictation no longer pays the 1.3–2.5s load.
P1 — llm_ready is a lie (feeds the P0). It only means "an endpoint is
configured", so a placeholder key reads as ready. The P0 timeout makes a dead
endpoint harmless; add last_refine_status so RefinementPanel flags a
configured-but-failing LLM and links to LLM Providers → Test.
Regression tests (fail-before/pass-after): slow-LLM WS final arrives < budget;
maybe_refine_async hard timeout + status; REST polish parity + refined_text
polish; warmup builds the recognizer and a second session reuses it; the panel
honesty note. Backend refinement/capture_ws/capture/sherpa suites, CJK + route
inventory gates, full vitest (733), lint (0 errors) and format all green.
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Phase 2 of Spec 3, on top of Wave 1.1's deterministic collapse. Prompt
design ported from voicebox (MIT): 'text filter, not an assistant' base
instruction + three toggleable sections (smart_cleanup, self_correction,
preserve_technical) + 7 few-shot examples passed as STRUCTURED chat turns
(small local models echo inline examples). Runs through the user's own
Ollama/LM Studio/OpenAI-compat endpoint via llm_backend — new additive
chat_messages() on the adapter; chat() now delegates to it.
Pass-through is the contract: with no LLM configured (backend 'off'),
on any error/timeout, or on an empty reply, the raw transcript stands —
identical default behavior on every platform. Refinement runs off-thread
on FINALS only; the WS final dict gains optional refined_text and the
dictation pill pastes refined_text ?? text (raw kept in history).
Settings: GET/PUT /api/settings/dictation-refinement (loopback-gated,
persisted in the settings table) + a Capture-tab panel with the master
switch + per-flag toggles and a 'no LLM configured' hint.
15 new unit tests: prompt sections per flag, structured few-shot message
shape, and the full maybe_refine pass-through matrix (off backend,
disabled config, LLM failure, empty reply, empty input).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Deterministic pre-pass ported from voicebox (MIT, attribution header):
word-level (token repeated >=6x, punctuation-normalized) + character-level
(2-60-char unit repeated >=6x, catches multi-word and no-space-script
loops). Rhetorical repeats below 6 survive; no LLM involved; identical on
every platform. Applied to the FINAL text in /ws/transcribe and POST
/transcribe — segments keep raw recognition so timings stay truthful.
Phase 1 of Spec 3 (docs/competitive-analysis.md); the optional local-LLM
refinement pass (phase 2) lands with parity program Wave 2.1 in the same
module.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
