* fix(windows): gate torch.compile on Triton availability (#129, closes #65) plan-02. torch.compile(mode="reduce-overhead") needs Triton at runtime; Triton has no Windows wheel, so the old `device=="cuda"`-only guard in model_manager.py failed on Windows+CUDA and surfaced as a confusing "OOM" (#65). Inference-time, hard to diagnose. - engine_env.should_torch_compile(device): requires CUDA + find_spec("triton") + the existing perf.torch_compile_disabled setting being off; logs the skip reason at INFO and falls back to eager. - model_manager.py call site uses it instead of the bare cuda check. - smoke-test.sh INST-02: import torch + ctranslate2 + whisperx (full ASR path) so a missing transitive dep fails the build instead of crashing mid- transcription (#116). Runs in the CI smoke-matrix on Win/macOS/Linux. setuptools>=75.0 (fix-sequence step 1) already pinned (#58). Linux/CUDA+Triton behaviour unchanged. Tests (TDD): tests/test_torch_compile_gate.py (4). Closes #65; addresses #129/#116. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(windows): also gate subprocess torch.compile on Triton (Greptile #138) Greptile flagged that the in-process gate left a parallel gap: engine subprocesses honour TORCH_COMPILE_DISABLE, but build_engine_env() only set it on the user's Performance toggle — so a Triton-absent host (Windows, or macOS) still exposed subprocess engines to the same crash this PR fixes in-process. - build_engine_env(): set TORCH_COMPILE_DISABLE=1 when the user disabled compile OR Triton is unavailable (find_spec), cross-platform — mirrors should_torch_compile(). Drops the Windows-only scoping (and the now-unused `import sys`). - Refreshed the stale module docstring. - 3 new tests cover the subprocess gate (triton-missing, triton-present, user-opt-out). 7/7 pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * revert(engine_env): keep subprocess TORCH_COMPILE_DISABLE user-driven Reverts the build_engine_env() broadening from the previous commit. Auto- disabling subprocess torch.compile on Triton-absence conflicts with a deliberate, tested contract (test_perf_settings: Windows + flag-off ⇒ no injection; non-Windows ⇒ never inject) — the subprocess var is intentionally under the user's explicit control. The #65 fix is the in-process should_torch_compile() gate (unchanged here), which IS automatic and fully tested. Pushing back on the subprocess auto-gate as a separate, deliberate contract change rather than forcing it through by rewriting established tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
38 lines
1.4 KiB
Python
38 lines
1.4 KiB
Python
"""plan-02 (#129/#65) — torch.compile must be gated on Triton availability.
|
|
|
|
`torch.compile(mode="reduce-overhead")` needs Triton at runtime; Triton has no
|
|
Windows build, so on Windows+CUDA the compile path failed and surfaced as a
|
|
confusing "OOM". The gate skips compile (→ eager) when Triton is absent or the
|
|
user disabled it. Tests force find_spec / the setting and assert the decision.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import importlib.util
|
|
|
|
from services import engine_env
|
|
|
|
|
|
def test_skips_when_device_not_cuda():
|
|
assert engine_env.should_torch_compile("cpu") is False
|
|
assert engine_env.should_torch_compile("mps") is False
|
|
|
|
|
|
def test_skips_when_triton_missing(monkeypatch):
|
|
monkeypatch.setattr(
|
|
importlib.util, "find_spec",
|
|
lambda name: None if name == "triton" else object(),
|
|
)
|
|
assert engine_env.should_torch_compile("cuda") is False
|
|
|
|
|
|
def test_enabled_when_triton_present_and_not_disabled(monkeypatch):
|
|
monkeypatch.setattr(importlib.util, "find_spec", lambda name: object())
|
|
monkeypatch.setattr("services.settings_store.get_text", lambda key, default="0": "0")
|
|
assert engine_env.should_torch_compile("cuda") is True
|
|
|
|
|
|
def test_skips_when_disabled_in_settings(monkeypatch):
|
|
monkeypatch.setattr(importlib.util, "find_spec", lambda name: object())
|
|
monkeypatch.setattr("services.settings_store.get_text", lambda key, default="0": "1")
|
|
assert engine_env.should_torch_compile("cuda") is False
|