Commit Graph
1 Commits
Author SHA1 Message Date
Palash DebnathandClaude Opus 4.8 e61665fe34 feat(routing): host device probe + routing resolver (#21 PR 1/5) (#430)
* feat(routing): canonical host device probe + routing resolver (#21 PR 1/5)

Foundational, backend-only slice of the GPU compatibility matrix (#21).
No API or UI change — wiring lands in PRs 3–5.

- `core/device_caps.py`: single source of truth for host accelerator
  capability. `detect_host_caps()` distinguishes ROCm from CUDA (unlike
  the gguf hardware_probe), never raises, makes no network call, stays
  kernel-free on cold start, and caches per process. Enumerates the full
  degradation contract (torch-unimportable→probe_ok=False, CUDA-init
  raises, device_count==0, multi-GPU, mem_get_info failure, arch
  mismatch, MPS, XPU, DirectML). Plus shared `mlx_supported()` gate
  (#390 groundwork) — exact-string platform check, no regex.
- `services/engine_routing.py`: pure `resolve_routing(gpu_compat, caps)`
  → `{effective_device, routing_status, routing_reason}`; deterministic
  and byte-identical across OSes. Rules for accelerated / cpu_fallback
  (the no-silent-fallback signal) / cpu_only / unavailable, incl. the
  ROCm-not-in-set, DirectML-neutral, and XPU edges.
- `get_best_device()` delegates its family decision to the probe so the
  loader and probe can never disagree; keeps the ROCm HSA env override
  and DirectML device-string return (probe reads, loader writes). String
  contract unchanged.
- 39 unit tests (probe / resolver / mlx gate / reason-scrub contract);
  no new regex (CodeQL-clean), English-only (CJK guard green).

The gguf hardware_probe rebase is a deliberate follow-up: it has its own
torch-mocked suite and a VRAM-driven quant table unaffected by the family
rename, so it stays out of this zero-risk slice.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(routing): address review — full available_families + empty-except comments

CodeRabbit / CodeQL review on PR 1:
- `available_families` no longer drops secondary accelerators on hybrid hosts
  (e.g. NVIDIA + Intel-iGPU-via-IPEX). The probe now detects every accelerator
  independently and picks `family` by priority at the end, instead of
  short-circuiting after the first hit. Routing is unaffected (it keys off
  `family`), but the field is now honest. + hybrid-host test.
- Annotated every `except: pass` in device_caps with an explanatory comment
  (CodeQL py/empty-except).
- Removed the unused `_MIN_NVIDIA_DRIVER` constant — the driver-version check
  stays in wizard preflight (no subprocess on the probe path); documented why.
- `get_best_device()` now checks MPS before DirectML, mirroring the probe's
  family-priority order so loader and probe never disagree.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 23:03:22 +05:30