fix(asr): repair CTranslate2's exec-stack request instead of only routing around it (#692)
ctranslate2 <=4.4.0 marks its native library's stack RWE; kernels that refuse the request fail the dlopen, taking whisperx, faster-whisper and Argos translation down. main learned to detect and fall back; this also fixes the library: core.execstack clears the one ELF bit in place on first probe (Linux-only, memoized, never raises), so the engines load normally on hardened kernels. Argos's probe moves to argostranslate.translate so it cannot advertise an engine whose every request 500s, and the dub route returns one actionable 400. Also: pytorch-whisper survives a CUDA OOM at transcribe time by stepping the batch down (16->4->1, 8->2->1 with word timestamps) and finishing on CPU rather than dropping the chunk.
This commit is contained in:
sean park
@@ -790,6 +790,30 @@ async def dub_translate(req: TranslateRequest):
|
||||
f"switch the Engine dropdown to another provider."
|
||||
)
|
||||
return JSONResponse(status_code=400, content={"error": friendly})
|
||||
# The package imports without its native dep; the *translator*
|
||||
# needs CTranslate2, whose library is rejected outright by kernels
|
||||
# that refuse an executable stack (#692). Repair it (a one-bit ELF
|
||||
# patch), and if that is impossible say so in one actionable 400
|
||||
# instead of the opaque 500 every segment used to produce.
|
||||
try:
|
||||
from core.execstack import ensure_ctranslate2_loadable
|
||||
|
||||
ct2_ok, ct2_detail = ensure_ctranslate2_loadable()
|
||||
except Exception as e: # noqa: BLE001 — repair must not block translation
|
||||
logger.debug("exec-stack repair unavailable (%s) — continuing", e)
|
||||
ct2_ok, ct2_detail = True, "repair probe unavailable"
|
||||
try:
|
||||
import argostranslate.translate # noqa: F401
|
||||
except Exception as e: # noqa: BLE001 — OSError here, not ImportError
|
||||
friendly = (
|
||||
f"The '{provider}' engine's CTranslate2 runtime could not be "
|
||||
f"loaded in this backend ({type(e).__name__}: {e})."
|
||||
+ ("" if ct2_ok else f" {ct2_detail}")
|
||||
+ " Switch the Engine dropdown to NLLB (local) or an online "
|
||||
"provider, or reinstall the backend, then retry."
|
||||
)
|
||||
return JSONResponse(status_code=400, content={"error": friendly})
|
||||
|
||||
target_codes = list(dict.fromkeys(
|
||||
seg.target_lang if seg.target_lang else req.target_lang
|
||||
for seg in req.segments
|
||||
|
||||
@@ -0,0 +1,247 @@
|
||||
"""Repair wheel-shipped shared libraries that request an executable stack.
|
||||
|
||||
Why this exists
|
||||
---------------
|
||||
CTranslate2 wheels up to and including 4.4.0 ship
|
||||
``ctranslate2.libs/libctranslate2-*.so`` with ``PT_GNU_STACK`` marked
|
||||
``RWE`` — a request for an executable stack. Linux kernels that refuse to
|
||||
grant it (hardened kernels, and mainline from 6.x onwards) fail the
|
||||
``dlopen`` outright::
|
||||
|
||||
ImportError: libctranslate2-d3638643.so.4.4.0: cannot enable executable
|
||||
stack as shared object requires: Invalid argument
|
||||
|
||||
Everything that links CTranslate2 dies with it: the **whisperx** and
|
||||
**faster-whisper** ASR engines (#692) *and* Argos translation, which is the
|
||||
default dub translation engine (``argostranslate.translate`` imports
|
||||
``ctranslate2``). #692 taught the ASR selector to fall through to another
|
||||
engine; it never fixed the library, so Linux users on Python 3.11 lost both
|
||||
engines. The pin is upstream and not ours to lift: whisperx 3.4.5 — the last
|
||||
release that supports Python 3.11, which is what ``.python-version``, CI and
|
||||
the installers use — requires ``ctranslate2<4.5.0``, and 4.5.0 is the first
|
||||
release whose ``.so`` drops the exec-stack request.
|
||||
|
||||
The flag is a single bit in the ELF program header, so we clear it in place
|
||||
rather than shipping a patched wheel or asking users for ``patchelf`` (which
|
||||
is not installed on a typical desktop). Inspection is a ~100-byte read with
|
||||
no imports, so :func:`ensure_ctranslate2_loadable` is cheap enough to call
|
||||
from an availability probe: it only rewrites a file when that file would
|
||||
otherwise refuse to load.
|
||||
|
||||
Everything here is a no-op off Linux (macOS/Windows have no such rejection)
|
||||
and never raises — a repair that cannot happen returns a reason, and the
|
||||
caller degrades exactly as it did before.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import glob
|
||||
import logging
|
||||
import os
|
||||
import struct
|
||||
import sys
|
||||
|
||||
logger = logging.getLogger("omnivoice.execstack")
|
||||
|
||||
#: ELF segment type for the stack-permission marker, and its executable bit.
|
||||
_PT_GNU_STACK = 0x6474E551
|
||||
_PF_X = 0x1
|
||||
|
||||
#: Memoized :func:`ensure_ctranslate2_loadable` result — the repair is
|
||||
#: process-wide and idempotent, so probe it once per backend process.
|
||||
_CT2_CHECKED: tuple[bool, str] | None = None
|
||||
|
||||
|
||||
def _elf_header(fh) -> tuple[str, int, int, int, bool] | None:
|
||||
"""Return ``(endian_prefix, e_phoff, e_phentsize, e_phnum, is_64)`` or None.
|
||||
|
||||
None means "not an ELF file we understand" — which is a normal answer
|
||||
(a ``.so`` stub, a text file, a Mach-O), never an error.
|
||||
"""
|
||||
fh.seek(0)
|
||||
ident = fh.read(16)
|
||||
if len(ident) < 16 or ident[:4] != b"\x7fELF":
|
||||
return None
|
||||
if ident[4] not in (1, 2) or ident[5] not in (1, 2):
|
||||
return None
|
||||
is_64 = ident[4] == 2
|
||||
endian = "<" if ident[5] == 1 else ">"
|
||||
# e_phoff / e_phentsize / e_phnum live at class-dependent offsets.
|
||||
if is_64:
|
||||
fh.seek(0x20)
|
||||
(e_phoff,) = struct.unpack(endian + "Q", fh.read(8))
|
||||
fh.seek(0x36)
|
||||
e_phentsize, e_phnum = struct.unpack(endian + "HH", fh.read(4))
|
||||
else:
|
||||
fh.seek(0x1C)
|
||||
(e_phoff,) = struct.unpack(endian + "I", fh.read(4))
|
||||
fh.seek(0x2A)
|
||||
e_phentsize, e_phnum = struct.unpack(endian + "HH", fh.read(4))
|
||||
if not e_phoff or not e_phentsize or not e_phnum:
|
||||
return None
|
||||
# p_flags sits at a different offset per class (ELF64 puts it right after
|
||||
# p_type; ELF32 puts it last), so the caller needs the class too.
|
||||
return endian, e_phoff, e_phentsize, e_phnum, is_64
|
||||
|
||||
|
||||
def _gnu_stack_flags_offset(fh) -> tuple[int, int, str] | None:
|
||||
"""Locate the ``PT_GNU_STACK`` ``p_flags`` field.
|
||||
|
||||
Returns ``(file_offset, flags_value, endian_prefix)``, or None when the
|
||||
file is not an ELF or carries no such segment.
|
||||
"""
|
||||
parsed = _elf_header(fh)
|
||||
if parsed is None:
|
||||
return None
|
||||
endian, e_phoff, e_phentsize, e_phnum, is_64 = parsed
|
||||
flags_rel = 4 if is_64 else 24 # p_flags offset inside the program header
|
||||
for i in range(e_phnum):
|
||||
base = e_phoff + i * e_phentsize
|
||||
fh.seek(base)
|
||||
raw = fh.read(e_phentsize)
|
||||
if len(raw) < flags_rel + 4:
|
||||
continue
|
||||
(p_type,) = struct.unpack_from(endian + "I", raw, 0)
|
||||
if p_type != _PT_GNU_STACK:
|
||||
continue
|
||||
(p_flags,) = struct.unpack_from(endian + "I", raw, flags_rel)
|
||||
return base + flags_rel, p_flags, endian
|
||||
return None
|
||||
|
||||
|
||||
def has_execstack(path: str) -> bool | None:
|
||||
"""True when ``path`` requests an executable stack.
|
||||
|
||||
None when the question does not apply: unreadable, not an ELF, or no
|
||||
``PT_GNU_STACK`` segment.
|
||||
"""
|
||||
try:
|
||||
with open(path, "rb") as fh:
|
||||
found = _gnu_stack_flags_offset(fh)
|
||||
except OSError:
|
||||
return None
|
||||
if found is None:
|
||||
return None
|
||||
_offset, flags, _endian = found
|
||||
return bool(flags & _PF_X)
|
||||
|
||||
|
||||
def clear_execstack(path: str) -> tuple[bool, str]:
|
||||
"""Clear the executable-stack request on ``path``.
|
||||
|
||||
Returns ``(changed, detail)``. ``changed`` is False both when there was
|
||||
nothing to do and when the write was refused (a read-only bundle, for
|
||||
instance) — ``detail`` says which.
|
||||
"""
|
||||
try:
|
||||
with open(path, "rb") as fh:
|
||||
found = _gnu_stack_flags_offset(fh)
|
||||
except OSError as e:
|
||||
return False, f"unreadable ({e.__class__.__name__})"
|
||||
if found is None:
|
||||
return False, "no PT_GNU_STACK segment"
|
||||
offset, flags, endian = found
|
||||
if not flags & _PF_X:
|
||||
return False, "already non-executable"
|
||||
try:
|
||||
with open(path, "r+b") as fh:
|
||||
fh.seek(offset)
|
||||
fh.write(struct.pack(endian + "I", flags & ~_PF_X))
|
||||
fh.flush()
|
||||
os.fsync(fh.fileno())
|
||||
except OSError as e:
|
||||
return False, f"not writable ({e.__class__.__name__}: {e})"
|
||||
return True, "cleared PT_GNU_STACK executable bit"
|
||||
|
||||
|
||||
def ctranslate2_library_paths() -> list[str]:
|
||||
"""Native libraries shipped with the installed ``ctranslate2`` wheel.
|
||||
|
||||
Found without importing ``ctranslate2`` — importing it is the very thing
|
||||
that fails when the exec-stack bit is set.
|
||||
"""
|
||||
import importlib.util
|
||||
|
||||
roots: list[str] = []
|
||||
try:
|
||||
spec = importlib.util.find_spec("ctranslate2")
|
||||
except (ImportError, ValueError): # pragma: no cover — defensive
|
||||
spec = None
|
||||
locations = list(getattr(spec, "submodule_search_locations", None) or []) if spec else []
|
||||
for pkg_dir in locations:
|
||||
roots.append(pkg_dir)
|
||||
roots.append(os.path.join(os.path.dirname(pkg_dir), "ctranslate2.libs"))
|
||||
# Frozen builds flatten the wheel into the bundle directory.
|
||||
meipass = getattr(sys, "_MEIPASS", None)
|
||||
if meipass:
|
||||
roots.append(meipass)
|
||||
roots.append(os.path.join(meipass, "ctranslate2.libs"))
|
||||
out: list[str] = []
|
||||
for root in roots:
|
||||
if not os.path.isdir(root):
|
||||
continue
|
||||
for pattern in ("libctranslate2*.so*", "libctranslate2*.dylib"):
|
||||
out.extend(sorted(glob.glob(os.path.join(root, pattern))))
|
||||
# Dedupe, preserving order.
|
||||
return list(dict.fromkeys(out))
|
||||
|
||||
|
||||
def ensure_ctranslate2_loadable() -> tuple[bool, str]:
|
||||
"""Make ``import ctranslate2`` possible on kernels that refuse exec stacks.
|
||||
|
||||
Returns ``(ok, detail)`` where ``ok`` is False only when a library needs
|
||||
the repair and could not get it — the caller should then report its
|
||||
engine unavailable with ``detail`` as the reason. Memoized: the repair is
|
||||
idempotent and process-wide.
|
||||
"""
|
||||
global _CT2_CHECKED
|
||||
if _CT2_CHECKED is not None:
|
||||
return _CT2_CHECKED
|
||||
|
||||
result: tuple[bool, str]
|
||||
if sys.platform != "linux":
|
||||
# Only Linux rejects an exec-stack request at dlopen time.
|
||||
result = (True, "not applicable off Linux")
|
||||
else:
|
||||
libs = ctranslate2_library_paths()
|
||||
if not libs:
|
||||
result = (True, "no ctranslate2 library found")
|
||||
else:
|
||||
repaired: list[str] = []
|
||||
blocked: list[str] = []
|
||||
for lib in libs:
|
||||
if has_execstack(lib) is not True:
|
||||
continue
|
||||
changed, detail = clear_execstack(lib)
|
||||
if changed:
|
||||
repaired.append(os.path.basename(lib))
|
||||
logger.warning(
|
||||
"Repaired %s: %s — its executable-stack request is "
|
||||
"rejected by this kernel, which broke whisperx, "
|
||||
"faster-whisper and Argos translation (#692)",
|
||||
os.path.basename(lib), detail,
|
||||
)
|
||||
else:
|
||||
blocked.append(f"{lib} ({detail})")
|
||||
if blocked:
|
||||
result = (
|
||||
False,
|
||||
"ctranslate2's native library requests an executable stack, "
|
||||
"which this kernel refuses, and it could not be patched: "
|
||||
+ "; ".join(blocked)
|
||||
+ ". Reinstall the backend on Python 3.12+ (which resolves "
|
||||
"ctranslate2 4.8+, without the exec-stack request), or run "
|
||||
"`patchelf --clear-execstack <library>` once.",
|
||||
)
|
||||
elif repaired:
|
||||
result = (True, "repaired " + ", ".join(repaired))
|
||||
else:
|
||||
result = (True, "no exec-stack request")
|
||||
|
||||
_CT2_CHECKED = result
|
||||
return result
|
||||
|
||||
|
||||
def reset_ctranslate2_cache() -> None:
|
||||
"""Forget the memoized probe result (tests, and after a reinstall)."""
|
||||
global _CT2_CHECKED
|
||||
_CT2_CHECKED = None
|
||||
@@ -286,6 +286,26 @@ def _ctranslate2_cudnn_ok() -> tuple[bool, str]:
|
||||
return True, "ready"
|
||||
|
||||
|
||||
def _ctranslate2_execstack_ok() -> tuple[bool, str]:
|
||||
"""Make CTranslate2 importable on kernels that refuse an executable stack.
|
||||
|
||||
ctranslate2 ≤4.4.0 — the version whisperx 3.4.5 pins, and 3.4.5 is the
|
||||
newest release that supports the Python 3.11 we ship — marks its native
|
||||
library's stack ``RWE``. Kernels that refuse the request fail the dlopen
|
||||
with "cannot enable executable stack", killing whisperx, faster-whisper
|
||||
*and* Argos translation (#692). :mod:`core.execstack` clears that one bit
|
||||
in place, so call this BEFORE importing either engine; it is memoized and
|
||||
only writes when the library would otherwise refuse to load.
|
||||
"""
|
||||
try:
|
||||
from core.execstack import ensure_ctranslate2_loadable
|
||||
|
||||
return ensure_ctranslate2_loadable()
|
||||
except Exception as e: # noqa: BLE001 — a broken repair must not block ASR
|
||||
logger.debug("exec-stack repair unavailable (%s) — continuing", e)
|
||||
return True, "repair probe unavailable"
|
||||
|
||||
|
||||
def _decode_audio_16k_mono(audio_path: str):
|
||||
"""Decode `audio_path` to a 16 kHz mono float32 waveform using VoiceStudio's
|
||||
*validated* ffmpeg, instead of whisperx.load_audio's bare ``"ffmpeg"`` PATH
|
||||
@@ -657,6 +677,9 @@ class WhisperXBackend(ASRBackend):
|
||||
|
||||
@classmethod
|
||||
def is_available(cls) -> tuple[bool, str]:
|
||||
ct2_ok, ct2_detail = _ctranslate2_execstack_ok()
|
||||
if not ct2_ok:
|
||||
return False, f"whisperx cannot load CTranslate2: {ct2_detail}"
|
||||
try:
|
||||
import whisperx # noqa: F401
|
||||
except ImportError as e:
|
||||
@@ -684,6 +707,14 @@ class WhisperXBackend(ASRBackend):
|
||||
# → speechbrain, or a stray k2_fsa redirect import aborts ASR on Windows
|
||||
# (#630/#611/#647). No-op on macOS/Linux and when speechbrain is absent.
|
||||
_harden_speechbrain_lazy_imports()
|
||||
# #692: repair CTranslate2's exec-stack request before the import that
|
||||
# would be rejected by it. Memoized, so this is free after the probe.
|
||||
ct2_ok, ct2_detail = _ctranslate2_execstack_ok()
|
||||
if not ct2_ok:
|
||||
# ImportError (not RuntimeError): this IS a native-import failure,
|
||||
# and the sentinel lets load_active_asr_backend degrade to the next
|
||||
# engine instead of failing ASR wholesale (#1185).
|
||||
raise ImportError(f"whisperx cannot load CTranslate2: {ct2_detail}")
|
||||
import whisperx
|
||||
# #723: re-check the CUDA pick against *currently free* VRAM — the TTS
|
||||
# model may have claimed the card since __init__. A too-big load dies
|
||||
@@ -1020,6 +1051,9 @@ class FasterWhisperBackend(ASRBackend):
|
||||
|
||||
@classmethod
|
||||
def is_available(cls) -> tuple[bool, str]:
|
||||
ct2_ok, ct2_detail = _ctranslate2_execstack_ok()
|
||||
if not ct2_ok:
|
||||
return False, f"faster-whisper cannot load CTranslate2: {ct2_detail}"
|
||||
try:
|
||||
import faster_whisper # noqa: F401
|
||||
except ImportError as e:
|
||||
@@ -1034,6 +1068,9 @@ class FasterWhisperBackend(ASRBackend):
|
||||
def _ensure_model(self):
|
||||
if self._model is not None:
|
||||
return
|
||||
ct2_ok, ct2_detail = _ctranslate2_execstack_ok() # #692, see WhisperX
|
||||
if not ct2_ok:
|
||||
raise ImportError(f"faster-whisper cannot load CTranslate2: {ct2_detail}")
|
||||
from faster_whisper import WhisperModel
|
||||
# Device / compute-type auto-pick:
|
||||
# - CUDA present → GPU fp16
|
||||
@@ -1445,9 +1482,48 @@ class PyTorchWhisperBackend(ASRBackend):
|
||||
f"Underlying: {e}"
|
||||
) from e
|
||||
|
||||
#: Batch sizes to try on CUDA, largest first. The VRAM preflight only sizes
|
||||
#: the *weights*; generation adds an encoder/decoder workspace that scales
|
||||
#: with the batch, and `return_timestamps="word"` keeps every layer's
|
||||
#: cross-attention for the whole batch — gigabytes at batch 16. A card with
|
||||
#: room for the model can therefore still OOM at the first transcribe, which
|
||||
#: used to lose that chunk entirely (the dub retried the same batch size and
|
||||
#: gave up, leaving a hole in the transcript). Step down, then use CPU.
|
||||
_CUDA_BATCH_LADDER = (16, 4, 1)
|
||||
_CUDA_BATCH_LADDER_WORD_TS = (8, 2, 1)
|
||||
|
||||
@staticmethod
|
||||
def _is_oom(exc: BaseException) -> bool:
|
||||
try:
|
||||
import torch
|
||||
|
||||
if isinstance(exc, torch.cuda.OutOfMemoryError):
|
||||
return True
|
||||
except Exception: # noqa: BLE001 — classification must not raise
|
||||
pass
|
||||
return "out of memory" in str(exc).lower()
|
||||
|
||||
def _rebuild_on_cpu(self) -> None:
|
||||
"""Drop the CUDA pipeline and rebuild it on CPU (slower, same model)."""
|
||||
self._pipe = None
|
||||
try:
|
||||
import torch
|
||||
|
||||
torch.cuda.empty_cache()
|
||||
except Exception: # noqa: BLE001 — cache clear is best-effort
|
||||
pass
|
||||
import torch
|
||||
from transformers import pipeline as hf_pipeline
|
||||
|
||||
self._pipe = hf_pipeline(
|
||||
"automatic-speech-recognition",
|
||||
model=self._model_name(),
|
||||
dtype=torch.float32,
|
||||
device="cpu",
|
||||
)
|
||||
|
||||
def transcribe(self, audio_path: str, *, word_timestamps: bool = True) -> dict:
|
||||
import soundfile as sf
|
||||
import torch
|
||||
self._ensure_pipe()
|
||||
# #2039: libsndfile cannot open MP4/M4A (AAC), which /transcribe and
|
||||
# the MCP tool both accept. Those decode through the validated ffmpeg
|
||||
@@ -1460,15 +1536,67 @@ class PyTorchWhisperBackend(ASRBackend):
|
||||
audio_np, sr = _decode_audio_16k_mono(audio_path), 16000
|
||||
if audio_np.ndim > 1:
|
||||
audio_np = audio_np.mean(axis=1)
|
||||
bs = 16 if torch.cuda.is_available() else 2
|
||||
result = self._pipe(
|
||||
{"array": audio_np, "sampling_rate": sr},
|
||||
return_timestamps="word" if word_timestamps else True,
|
||||
chunk_length_s=15,
|
||||
batch_size=bs,
|
||||
)
|
||||
|
||||
def _run(batch_size: int):
|
||||
return self._pipe(
|
||||
{"array": audio_np, "sampling_rate": sr},
|
||||
return_timestamps="word" if word_timestamps else True,
|
||||
chunk_length_s=15,
|
||||
batch_size=batch_size,
|
||||
)
|
||||
|
||||
if self._on_cuda():
|
||||
ladder = (
|
||||
self._CUDA_BATCH_LADDER_WORD_TS if word_timestamps
|
||||
else self._CUDA_BATCH_LADDER
|
||||
)
|
||||
for i, bs in enumerate(ladder):
|
||||
try:
|
||||
result = _run(bs)
|
||||
break
|
||||
except Exception as e: # noqa: BLE001 — only OOM is retryable
|
||||
if not self._is_oom(e):
|
||||
raise
|
||||
try:
|
||||
import torch
|
||||
|
||||
torch.cuda.empty_cache()
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
if i + 1 < len(ladder):
|
||||
logger.warning(
|
||||
"PyTorch Whisper CUDA OOM at batch_size=%d — "
|
||||
"retrying at %d. Free VRAM (Flush models, close "
|
||||
"other GPU apps) for full-speed ASR.",
|
||||
bs, ladder[i + 1],
|
||||
)
|
||||
continue
|
||||
# Smallest batch still OOMs: finish on CPU rather than
|
||||
# return an empty chunk the caller cannot distinguish
|
||||
# from silence.
|
||||
logger.warning(
|
||||
"PyTorch Whisper CUDA OOM even at batch_size=1 — "
|
||||
"transcribing on CPU (slower, same model). Detail: %s", e,
|
||||
)
|
||||
self._rebuild_on_cpu()
|
||||
result = _run(2)
|
||||
else:
|
||||
result = _run(2)
|
||||
return result if isinstance(result, dict) else {"chunks": [], "raw": result}
|
||||
|
||||
def _on_cuda(self) -> bool:
|
||||
"""Whether the built pipeline actually sits on a CUDA device.
|
||||
|
||||
`torch.cuda.is_available()` is the wrong question: `_pick_device()` may
|
||||
have chosen CPU on a CUDA host (low free VRAM), and a CPU pipeline must
|
||||
not be handed a CUDA-sized batch.
|
||||
"""
|
||||
try:
|
||||
device = getattr(self._pipe, "device", None)
|
||||
return "cuda" in str(device).lower()
|
||||
except Exception: # noqa: BLE001
|
||||
return False
|
||||
|
||||
|
||||
# ── NeMo Parakeet TDT (NVIDIA — Open ASR Leaderboard SOTA, 25 langs) ────────
|
||||
|
||||
|
||||
@@ -39,7 +39,12 @@ REGISTRY: dict[str, dict] = {
|
||||
"id": "argos",
|
||||
"display_name": "Argos (Local, Fast)",
|
||||
"pip_package": "argostranslate",
|
||||
"probe_module": "argostranslate",
|
||||
# `argostranslate.translate`, not the bare package: the translator runs
|
||||
# on CTranslate2, and the bare package imports fine on a host whose
|
||||
# kernel rejects CTranslate2's native library (#692) — so a shallow
|
||||
# probe advertised Argos as ready and every translate 500'd. Probe the
|
||||
# module that actually pulls the native dep (same lesson as #1185).
|
||||
"probe_module": "argostranslate.translate",
|
||||
"category": "offline",
|
||||
"needs_key": False,
|
||||
"builtin": True,
|
||||
@@ -123,7 +128,19 @@ def is_frozen() -> bool:
|
||||
def _probe(entry: dict) -> tuple[bool, str | None]:
|
||||
mod = entry.get("probe_module")
|
||||
if not mod:
|
||||
return True, None
|
||||
return True, None
|
||||
if mod.startswith("argostranslate"):
|
||||
# Repair CTranslate2's exec-stack request before the import that would
|
||||
# be rejected by it (#692) — otherwise Argos, the default offline
|
||||
# engine, is unusable on kernels that refuse an executable stack.
|
||||
try:
|
||||
from core.execstack import ensure_ctranslate2_loadable
|
||||
|
||||
ok, detail = ensure_ctranslate2_loadable()
|
||||
if not ok:
|
||||
return False, detail
|
||||
except Exception as e: # noqa: BLE001 — a broken repair must not hide the engine
|
||||
logger.debug("exec-stack repair unavailable (%s) — probing anyway", e)
|
||||
try:
|
||||
importlib.import_module(mod)
|
||||
if entry.get("id") == "nllb":
|
||||
@@ -141,6 +158,11 @@ def _probe(entry: dict) -> tuple[bool, str | None]:
|
||||
return True, None
|
||||
except ImportError as e:
|
||||
return False, f"import {mod!r} failed: {e}"
|
||||
except Exception as e: # noqa: BLE001
|
||||
# A native library that refuses to load raises OSError, not ImportError
|
||||
# (#692). An availability probe must report "unusable here", never take
|
||||
# the engine list down with it.
|
||||
return False, f"import {mod!r} failed ({type(e).__name__}): {e}"
|
||||
|
||||
|
||||
def install_command(engine: "str | dict | None") -> str | None:
|
||||
|
||||
@@ -0,0 +1,178 @@
|
||||
"""Regression tests for #692 — the CTranslate2 exec-stack rejection is now
|
||||
*repaired*, not merely routed around.
|
||||
|
||||
ctranslate2 ≤4.4.0 (what whisperx 3.4.5 pins, and 3.4.5 is the last release
|
||||
supporting the Python 3.11 we ship) marks its native library's stack as
|
||||
executable. Kernels that refuse the request fail the dlopen outright, which
|
||||
took out both CTranslate2 ASR engines *and* Argos — the default offline dub
|
||||
translation engine, whose bare `argostranslate` import succeeds without the
|
||||
native dep, so the engine advertised itself as ready and every translate
|
||||
request 500'd with an opaque ImportError.
|
||||
|
||||
`core.execstack` clears the one ELF bit that causes it. These tests pin the
|
||||
patcher on synthetic ELFs (no ctranslate2 needed), the memoization, and the
|
||||
wiring into every consumer probe.
|
||||
"""
|
||||
import struct
|
||||
|
||||
import pytest
|
||||
|
||||
from core import execstack
|
||||
|
||||
_PT_GNU_STACK = 0x6474E551
|
||||
_PT_LOAD = 1
|
||||
|
||||
|
||||
def _elf(path, *, bits=64, endian="<", flags=0x7, phdr_type=_PT_GNU_STACK):
|
||||
"""Write a minimal ELF whose second program header is `phdr_type`.
|
||||
|
||||
Only the fields the patcher reads are meaningful — a real linker would emit
|
||||
far more, but the point is to prove the offsets are computed correctly for
|
||||
both ELF classes, and to fail loudly if they ever drift.
|
||||
"""
|
||||
is_64 = bits == 64
|
||||
phentsize = 56 if is_64 else 32
|
||||
phoff = 64 if is_64 else 52
|
||||
ident = b"\x7fELF" + bytes([2 if is_64 else 1, 1 if endian == "<" else 2, 1]) + b"\0" * 9
|
||||
header = bytearray(phoff)
|
||||
header[: len(ident)] = ident
|
||||
if is_64:
|
||||
struct.pack_into(endian + "Q", header, 0x20, phoff)
|
||||
struct.pack_into(endian + "HH", header, 0x36, phentsize, 2)
|
||||
else:
|
||||
struct.pack_into(endian + "I", header, 0x1C, phoff)
|
||||
struct.pack_into(endian + "HH", header, 0x2A, phentsize, 2)
|
||||
|
||||
def _phdr(p_type, p_flags):
|
||||
raw = bytearray(phentsize)
|
||||
struct.pack_into(endian + "I", raw, 0, p_type)
|
||||
struct.pack_into(endian + "I", raw, 4 if is_64 else 24, p_flags)
|
||||
return bytes(raw)
|
||||
|
||||
path.write_bytes(bytes(header) + _phdr(_PT_LOAD, 0x5) + _phdr(phdr_type, flags))
|
||||
return str(path)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("bits", [64, 32])
|
||||
@pytest.mark.parametrize("endian", ["<", ">"])
|
||||
def test_clear_execstack_flips_only_the_x_bit(tmp_path, bits, endian):
|
||||
lib = _elf(tmp_path / "libfake.so", bits=bits, endian=endian, flags=0x7)
|
||||
assert execstack.has_execstack(lib) is True
|
||||
|
||||
changed, detail = execstack.clear_execstack(lib)
|
||||
assert changed is True and "cleared" in detail
|
||||
assert execstack.has_execstack(lib) is False
|
||||
|
||||
# Read + write permissions survive; only PF_X is gone.
|
||||
with open(lib, "rb") as fh:
|
||||
offset, flags, _ = execstack._gnu_stack_flags_offset(fh)
|
||||
assert flags == 0x6
|
||||
|
||||
# Idempotent: a second pass is a no-op, so a restart never rewrites.
|
||||
assert execstack.clear_execstack(lib) == (False, "already non-executable")
|
||||
|
||||
|
||||
def test_non_executable_stack_is_left_alone(tmp_path):
|
||||
lib = _elf(tmp_path / "libok.so", flags=0x6)
|
||||
assert execstack.has_execstack(lib) is False
|
||||
before = (tmp_path / "libok.so").read_bytes()
|
||||
assert execstack.clear_execstack(lib) == (False, "already non-executable")
|
||||
assert (tmp_path / "libok.so").read_bytes() == before
|
||||
|
||||
|
||||
def test_elf_without_gnu_stack_segment(tmp_path):
|
||||
lib = _elf(tmp_path / "libnostack.so", phdr_type=_PT_LOAD, flags=0x7)
|
||||
assert execstack.has_execstack(lib) is None
|
||||
assert execstack.clear_execstack(lib) == (False, "no PT_GNU_STACK segment")
|
||||
|
||||
|
||||
def test_non_elf_and_missing_files_are_not_errors(tmp_path):
|
||||
text = tmp_path / "notelf.so"
|
||||
text.write_bytes(b"#!/bin/sh\necho hi\n")
|
||||
assert execstack.has_execstack(str(text)) is None
|
||||
assert execstack.clear_execstack(str(text))[0] is False
|
||||
|
||||
missing = str(tmp_path / "nope" / "libghost.so")
|
||||
assert execstack.has_execstack(missing) is None
|
||||
changed, detail = execstack.clear_execstack(missing)
|
||||
assert changed is False and "unreadable" in detail
|
||||
|
||||
|
||||
def test_ensure_is_memoized_and_reports_unrepairable(tmp_path, monkeypatch):
|
||||
lib = _elf(tmp_path / "libctranslate2-test.so.4.4.0", flags=0x7)
|
||||
monkeypatch.setattr(execstack.sys, "platform", "linux")
|
||||
monkeypatch.setattr(execstack, "ctranslate2_library_paths", lambda: [lib])
|
||||
monkeypatch.setattr(
|
||||
execstack, "clear_execstack", lambda p: (False, "not writable (PermissionError)")
|
||||
)
|
||||
execstack.reset_ctranslate2_cache()
|
||||
ok, detail = execstack.ensure_ctranslate2_loadable()
|
||||
assert ok is False
|
||||
# Actionable: names the library, why, and both ways out.
|
||||
assert "executable stack" in detail and "patchelf" in detail and "3.12" in detail
|
||||
|
||||
# Memoized — a second call does not re-probe (the paths lookup would raise).
|
||||
monkeypatch.setattr(
|
||||
execstack, "ctranslate2_library_paths", lambda: pytest.fail("re-probed")
|
||||
)
|
||||
assert execstack.ensure_ctranslate2_loadable() == (ok, detail)
|
||||
execstack.reset_ctranslate2_cache()
|
||||
|
||||
|
||||
def test_ensure_repairs_then_reports_ok(tmp_path, monkeypatch):
|
||||
lib = _elf(tmp_path / "libctranslate2-test.so.4.4.0", flags=0x7)
|
||||
monkeypatch.setattr(execstack.sys, "platform", "linux")
|
||||
monkeypatch.setattr(execstack, "ctranslate2_library_paths", lambda: [lib])
|
||||
execstack.reset_ctranslate2_cache()
|
||||
ok, detail = execstack.ensure_ctranslate2_loadable()
|
||||
assert ok is True and "repaired" in detail
|
||||
assert execstack.has_execstack(lib) is False
|
||||
execstack.reset_ctranslate2_cache()
|
||||
|
||||
|
||||
def test_ensure_is_a_noop_off_linux(tmp_path, monkeypatch):
|
||||
"""macOS/Windows never reject an exec-stack request — don't touch signed
|
||||
bundles looking for a problem that cannot exist there."""
|
||||
monkeypatch.setattr(execstack.sys, "platform", "darwin")
|
||||
monkeypatch.setattr(
|
||||
execstack, "ctranslate2_library_paths", lambda: pytest.fail("probed off Linux")
|
||||
)
|
||||
execstack.reset_ctranslate2_cache()
|
||||
ok, detail = execstack.ensure_ctranslate2_loadable()
|
||||
assert ok is True and "off Linux" in detail
|
||||
execstack.reset_ctranslate2_cache()
|
||||
|
||||
|
||||
# ── Wiring: every consumer of the native lib must consult the repair ─────────
|
||||
|
||||
|
||||
def test_asr_probes_report_unavailable_when_repair_impossible(monkeypatch):
|
||||
from services import asr_backend as ab
|
||||
|
||||
monkeypatch.setattr(
|
||||
ab, "_ctranslate2_execstack_ok", lambda: (False, "kernel refuses it")
|
||||
)
|
||||
okx, msgx = ab.WhisperXBackend.is_available()
|
||||
okf, msgf = ab.FasterWhisperBackend.is_available()
|
||||
assert okx is False and "CTranslate2" in msgx and "kernel refuses it" in msgx
|
||||
assert okf is False and "CTranslate2" in msgf and "kernel refuses it" in msgf
|
||||
|
||||
|
||||
def test_argos_probe_module_pulls_the_native_dep():
|
||||
"""`argostranslate` alone imports fine with a broken CTranslate2 — the
|
||||
registry must probe the module that actually loads it, or the Engine
|
||||
selector advertises an engine whose every request fails."""
|
||||
from services.translation_engines import REGISTRY
|
||||
|
||||
assert REGISTRY["argos"]["probe_module"] == "argostranslate.translate"
|
||||
|
||||
|
||||
def test_engine_probe_survives_a_native_load_failure(monkeypatch):
|
||||
from services import translation_engines as te
|
||||
|
||||
def boom(name):
|
||||
raise OSError("libctranslate2-x.so: cannot enable executable stack")
|
||||
|
||||
monkeypatch.setattr(te.importlib, "import_module", boom)
|
||||
ok, detail = te._probe({"probe_module": "deep_translator"})
|
||||
assert ok is False and "OSError" in detail
|
||||
@@ -0,0 +1,102 @@
|
||||
"""The pytorch-whisper fallback must survive a CUDA OOM instead of losing a chunk.
|
||||
|
||||
The VRAM preflight sizes the *weights*; generation adds a workspace that scales
|
||||
with the batch, and word timestamps keep every layer's cross-attention for the
|
||||
whole batch. So a card with room for the model still OOMs at the first
|
||||
transcribe — and the dub path merely retried the identical call, gave up, and
|
||||
emitted nothing for that chunk: a silent hole in the transcript, indistinguishable
|
||||
from silence. Step the batch down, then finish on CPU.
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from services.asr_backend import PyTorchWhisperBackend
|
||||
|
||||
_OOM = RuntimeError("CUDA out of memory. Tried to allocate 2.00 GiB")
|
||||
|
||||
|
||||
class _FakePipe:
|
||||
"""Stands in for a transformers ASR pipeline on CUDA."""
|
||||
|
||||
def __init__(self, oom_below_batch):
|
||||
self.device = "cuda:0"
|
||||
self.oom_below_batch = oom_below_batch
|
||||
self.calls = []
|
||||
|
||||
def __call__(self, _audio, *, return_timestamps, chunk_length_s, batch_size):
|
||||
self.calls.append(batch_size)
|
||||
if batch_size > self.oom_below_batch:
|
||||
raise _OOM
|
||||
return {"text": "ok", "chunks": [{"text": "ok", "timestamp": (0.0, 1.0)}]}
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def audio(tmp_path):
|
||||
import numpy as np
|
||||
import soundfile as sf
|
||||
|
||||
path = tmp_path / "a.wav"
|
||||
sf.write(path, np.zeros(16000, dtype="float32"), 16000)
|
||||
return str(path)
|
||||
|
||||
|
||||
def test_oom_steps_down_the_batch_and_still_returns_text(audio):
|
||||
pipe = _FakePipe(oom_below_batch=2)
|
||||
backend = PyTorchWhisperBackend(asr_pipe=pipe)
|
||||
out = backend.transcribe(audio, word_timestamps=True)
|
||||
assert out["text"] == "ok"
|
||||
# Tried the word-timestamp ladder in order, stopping at the first that fits.
|
||||
assert pipe.calls == [8, 2]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"word_timestamps,expected", [(True, [8, 2, 1]), (False, [16, 4, 1])]
|
||||
)
|
||||
def test_batch_ladder_is_smaller_when_word_timestamps_are_requested(
|
||||
audio, monkeypatch, word_timestamps, expected
|
||||
):
|
||||
"""Word timestamps retain per-layer cross-attention for the whole batch, so
|
||||
the ladder must start lower than for plain transcription."""
|
||||
pipe = _FakePipe(oom_below_batch=0)
|
||||
backend = PyTorchWhisperBackend(asr_pipe=pipe)
|
||||
sentinel = RuntimeError("cpu rebuild reached")
|
||||
monkeypatch.setattr(
|
||||
backend, "_rebuild_on_cpu", lambda: (_ for _ in ()).throw(sentinel)
|
||||
)
|
||||
with pytest.raises(RuntimeError, match="cpu rebuild reached"):
|
||||
backend.transcribe(audio, word_timestamps=word_timestamps)
|
||||
assert pipe.calls == expected
|
||||
|
||||
|
||||
def test_exhausted_ladder_falls_back_to_cpu(audio, monkeypatch):
|
||||
pipe = _FakePipe(oom_below_batch=0)
|
||||
backend = PyTorchWhisperBackend(asr_pipe=pipe)
|
||||
|
||||
cpu = _FakePipe(oom_below_batch=99)
|
||||
cpu.device = "cpu"
|
||||
|
||||
def _rebuild():
|
||||
backend._pipe = cpu
|
||||
|
||||
monkeypatch.setattr(backend, "_rebuild_on_cpu", _rebuild)
|
||||
out = backend.transcribe(audio, word_timestamps=True)
|
||||
assert out["text"] == "ok"
|
||||
assert pipe.calls == [8, 2, 1] and cpu.calls == [2]
|
||||
|
||||
|
||||
def test_non_oom_errors_are_not_retried(audio):
|
||||
class _Boom(_FakePipe):
|
||||
def __call__(self, *a, **kw):
|
||||
self.calls.append(kw["batch_size"])
|
||||
raise ValueError("bad audio")
|
||||
|
||||
pipe = _Boom(oom_below_batch=99)
|
||||
with pytest.raises(ValueError):
|
||||
PyTorchWhisperBackend(asr_pipe=pipe).transcribe(audio)
|
||||
assert pipe.calls == [8] # one attempt, no ladder
|
||||
|
||||
|
||||
def test_cpu_pipeline_keeps_the_small_batch(audio):
|
||||
pipe = _FakePipe(oom_below_batch=99)
|
||||
pipe.device = "cpu"
|
||||
PyTorchWhisperBackend(asr_pipe=pipe).transcribe(audio)
|
||||
assert pipe.calls == [2]
|
||||
@@ -234,3 +234,11 @@ panels instead so neither editor becomes unusably small.
|
||||
from-source checkout.
|
||||
- **Installed it but still "needs install"** — restart the backend so Python
|
||||
picks up the newly-installed module.
|
||||
- **"The 'argos' engine's CTranslate2 runtime could not be loaded…"** — Argos
|
||||
translates on CTranslate2, and on Linux kernels that refuse an executable
|
||||
stack the CTranslate2 library shipped with Python 3.11 installs (4.4.0) is
|
||||
rejected outright. VoiceStudio repairs that library in place on first use; if
|
||||
it cannot (read-only install), the message names the fix — reinstall the
|
||||
backend on Python 3.12+, or run `patchelf --clear-execstack` on the library
|
||||
once — and NLLB stays available in the meantime
|
||||
([#692](https://github.com/debpalash/VoiceStudio/issues/692)).
|
||||
|
||||
@@ -55,10 +55,12 @@ transcription.
|
||||
process, so the engine checks up front and reports itself unavailable
|
||||
instead ([#1371](https://github.com/debpalash/VoiceStudio/issues/1371)).
|
||||
pytorch-whisper covers that case on torch's bundled cuDNN 9.
|
||||
- On some hardened Linux kernels the CTranslate2 native library is rejected
|
||||
with "cannot enable executable stack" (an OSError, not an ImportError) —
|
||||
reported as unavailable rather than crashing engine selection
|
||||
([#692](https://github.com/debpalash/VoiceStudio/issues/692)).
|
||||
- On Linux kernels that refuse an executable stack, the CTranslate2 native
|
||||
library (4.4.0 and older) is rejected with "cannot enable executable stack"
|
||||
(an OSError, not an ImportError). VoiceStudio clears that ELF flag in place
|
||||
on first probe so the engine loads; if the file cannot be written it reports
|
||||
itself unavailable with the repair command rather than crashing engine
|
||||
selection ([#692](https://github.com/debpalash/VoiceStudio/issues/692)).
|
||||
- CTranslate2's GPU teardown can rarely segfault the process at unload. If
|
||||
you hit that, switch to the crash-isolated variant —
|
||||
[faster-whisper-isolated](faster-whisper-isolated.md)
|
||||
|
||||
@@ -61,6 +61,12 @@ A 6 GB card with nothing else loaded runs the default model on the GPU
|
||||
([#2041](https://github.com/debpalash/VoiceStudio/issues/2041)). Disable
|
||||
the check with `OMNIVOICE_ASR_VRAM_PREFLIGHT=0`.
|
||||
|
||||
The preflight sizes the weights; the generation workspace grows with the
|
||||
batch on top. If a transcribe still hits a CUDA out-of-memory, the engine
|
||||
steps the batch down (16 → 4 → 1, or 8 → 2 → 1 with word timestamps) and
|
||||
finishes on the CPU rather than dropping the chunk — no silent holes in a
|
||||
dub transcript.
|
||||
|
||||
## Quirks
|
||||
|
||||
- If the pipeline fails to import (`AutoFeatureExtractor` errors), the cause
|
||||
|
||||
@@ -62,8 +62,12 @@ Two more fallback chains run at load time:
|
||||
process fast-fails with no traceback, so the engine is reported unavailable
|
||||
up front and selection falls through to pytorch-whisper, which uses torch's
|
||||
own cuDNN 9 ([#1371](https://github.com/debpalash/VoiceStudio/issues/1371)).
|
||||
- On some hardened Linux kernels CTranslate2's native library is rejected with
|
||||
"cannot enable executable stack" — reported as unavailable, not a crash
|
||||
- On Linux kernels that refuse an executable stack, CTranslate2's native
|
||||
library (4.4.0 and older — what whisperx 3.4.5 pins on Python 3.11) is
|
||||
rejected with "cannot enable executable stack". VoiceStudio now clears that
|
||||
one ELF flag in place on first probe and the engine loads normally; if the
|
||||
library cannot be written (a read-only bundle), the engine reports itself
|
||||
unavailable with the repair command instead of crashing
|
||||
([#692](https://github.com/debpalash/VoiceStudio/issues/692)).
|
||||
- A partially-installed environment (interrupted sync, antivirus quarantine)
|
||||
can break WhisperX's deep import chain (whisperx → pyannote →
|
||||
|
||||
Reference in New Issue
Block a user