Measured on a 16 GB M2: a generate on omnivoice (~2.8 GB core) followed by a
generate on mlx-audio left BOTH resident (footprint 3.9 → 4.3 GB) because the
OmniVoice core lives in model_manager.model while every other engine lives in
engines._ENGINE_INSTANCES — two caches that never coordinated, and the latter
was never unloaded. That accumulation is a direct contributor to the memory
pressure behind the "Can't reach the local backend" OOM deaths.
- services/engine_memory.py: evict_other_tts_engines(keep_id) unloads every
OTHER resident TTS engine before the incoming one loads — spans both stores
(the OmniVoice core under its async lock, and the instance cache). No-op when
nothing else is resident, so steady-state single-engine use pays nothing; only
a real switch evicts. Default on; OMNIVOICE_SINGLE_ENGINE_RESIDENT=0 to keep
several warm. Wired into the /generate path right after the engine resolves.
- TTSBackend.unload() (the ABC default) now actually frees the held model: it
clears _MODEL_ATTRS (_model/_tts) and empties the device cache. Every
in-process engine but OmniVoice previously inherited a NO-OP unload(), so an
engine switch dropped the instance ref but left its model for GC with the GPU
cache un-emptied. One change fixes all of them and is future-proof.
- FasterWhisperBackend.unload() cleared self._asr — an attribute it never
assigns — so its model in self._model was never freed. Fixed.
Live-verified: omnivoice → mlx-audio now DROPS footprint 2300 → 1541 MB (core
evicted) instead of climbing to 4305 with both resident. 7 new unit tests
(eviction spans both stores / keeps the active engine / no-op when disabled /
a failing unload doesn't abort the sweep / ABC unload frees + is idempotent),
order-independent. Backend suite green.
Refs the 16 GB OOM class (#1076#1092#1093#1101)
Co-authored-by: mergetest <nizam4103@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
