Review follow-up. The budget matched model names by substring, so a custom
repo whose name contains "turbo", "small" or "base" (or a word such as
"database") got a reduced budget and could be admitted to CUDA without
enough memory. Reduced budgets now apply only to the exact OpenAI
checkpoint ids, .en variants included. Any other repository, fine-tunes
included, keeps the conservative 5.0 GB, as the engine doc says.
The CUDA preflight demanded 5.0 GB of free VRAM for every model, sized for
full large-v3. The default model is large-v3-turbo, which the pipeline
loads in fp16: about 1.6 GB of weights, not the 3.2 GiB fp32 figure in the
old comment. So a 6 GB card with nothing else resident reported 5.0 GB
free and was sent to CPU every time, although CUDA ran the same audio in
37 s against minutes on CPU.
The budget is now fp16 weights + 1.5 GB workspace (batch 16) + 0.5 GB
headroom, per model and capped at the old 5.0 GB. That is 3.6 GB for turbo
and 5.0 GB for full large-v3 and any unrecognised model. Both CPU-fallback
warnings name the model and the OMNIVOICE_ASR_VRAM_PREFLIGHT=0 opt-out.
The engine doc lists the budgets.
Fixes#2041
Palash Debnath