The shared voice model's unload emptied the allocator caches and *then*
dropped the reference. That frees nothing: the weights are still reachable
when gc.collect() runs, empty_cache() only returns blocks the allocator
already considered free, and the reference drops a moment later into a cache
nothing will flush again. The unload logs success, the engine leaves the
registry, and nvidia-smi does not move.
Six modules open-coded the same two lines. Exactly one had them inverted --
OmniVoiceBackend.unload, which is the path the engine-registry idle sweep
reaches, which is the sweep a headless worker node runs. So every unload a
user could trigger from the UI worked, and the one that runs unattended on a
machine lending its GPU held 3.6 GB indefinitely. Found on hardware: the
sweep fired on schedule, logged "Released 1 idle engine(s)", and VRAM stayed
flat at 3656 MiB for the next two minutes.
Replace all six with model_manager.unload_shared_model(), which clears the
reference, drops the clone-prompt side cache, then frees -- in that order,
in one place. Two callers gain the side-cache drop they were missing
(/system/flush-memory and the shutdown path), which is the same defect one
step down: an unload that kept the encoded reference tensors belonging to the
model it had just released.
A source guard asserts nothing outside model_manager assigns the shared
reference, so the next caller cannot reintroduce the ordering. It caught the
sixth site while being written.
Also give the AudioSeal watermark models the bargain every other model in the
app already makes: they loaded on the first embed and stayed resident for the
life of the process. CPU-resident, so this is system RAM rather than VRAM,
and the machines that notice are the ones running batches.
The error text on a failing unload changes with the ordering. "Could not be
unloaded, retry after the current generation finishes" was accurate when the
cache flush ran first and aborted before the release; now the release has
already happened and only the flush can fail, so it says that instead of
sending the user to repeat work that is done.
Renames what users see. The app, the installers, the window title, the
docs and all 21 locales now say VoiceStudio, with "(previously
OmniVoice-Studio)" noted near the title of each doc surface so people
recognise it.
Deliberately NOT renamed, because renaming any of them silently breaks
an existing install — there is no legacy-path fallback anywhere in this
codebase:
- bundle identifier com.debpalash.omnivoice-studio (MSI UpgradeCode,
macOS TCC grants, managed venv, WebView localStorage, the
single-instance lock)
- data directories OmniVoice / .omnivoice and omnivoice.db
- the ~150 OMNIVOICE_* environment variables
- the X-OmniVoice-* HTTP headers (a wire protocol)
- the published Docker image paths
- the OmniVoice ENGINE, which is a model name and not this product
tests/test_identity_paths_survive_the_rename.py pins every one of those
so a future well-meaning sweep cannot orphan a user's library.
Linux .deb users install a new package name and should apt remove
omnivoice-studio; that note is in the changelog.
Measured on a 16 GB M2: a generate on omnivoice (~2.8 GB core) followed by a
generate on mlx-audio left BOTH resident (footprint 3.9 → 4.3 GB) because the
OmniVoice core lives in model_manager.model while every other engine lives in
engines._ENGINE_INSTANCES — two caches that never coordinated, and the latter
was never unloaded. That accumulation is a direct contributor to the memory
pressure behind the "Can't reach the local backend" OOM deaths.
- services/engine_memory.py: evict_other_tts_engines(keep_id) unloads every
OTHER resident TTS engine before the incoming one loads — spans both stores
(the OmniVoice core under its async lock, and the instance cache). No-op when
nothing else is resident, so steady-state single-engine use pays nothing; only
a real switch evicts. Default on; OMNIVOICE_SINGLE_ENGINE_RESIDENT=0 to keep
several warm. Wired into the /generate path right after the engine resolves.
- TTSBackend.unload() (the ABC default) now actually frees the held model: it
clears _MODEL_ATTRS (_model/_tts) and empties the device cache. Every
in-process engine but OmniVoice previously inherited a NO-OP unload(), so an
engine switch dropped the instance ref but left its model for GC with the GPU
cache un-emptied. One change fixes all of them and is future-proof.
- FasterWhisperBackend.unload() cleared self._asr — an attribute it never
assigns — so its model in self._model was never freed. Fixed.
Live-verified: omnivoice → mlx-audio now DROPS footprint 2300 → 1541 MB (core
evicted) instead of climbing to 4305 with both resident. 7 new unit tests
(eviction spans both stores / keeps the active engine / no-op when disabled /
a failing unload doesn't abort the sweep / ABC unload frees + is idempotent),
order-independent. Backend suite green.
Refs the 16 GB OOM class (#1076#1092#1093#1101)
Co-authored-by: mergetest <nizam4103@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
velixio