From d7d4aedd6bdb3ca3bfff24e4ea1ed0f76afd88f0 Mon Sep 17 00:00:00 2001 From: Eve <139727413+netrunnereve@users.noreply.github.com> Date: Mon, 16 Mar 2026 22:05:04 +0000 Subject: [PATCH] Updated Feature matrix (markdown) --- Feature-matrix.md | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/Feature-matrix.md b/Feature-matrix.md index e6dc424..54bfedc 100644 --- a/Feature-matrix.md +++ b/Feature-matrix.md @@ -1,19 +1,17 @@ -| | **CPU (AVX/AVX2)** | **CPU (ARM NEON)** | **Metal** | **CUDA** | **ROCm** | **SYCL** | **Vulkan** | **Kompute** | -|:-----------------------:|:--------------:|:------------------:|:---------:|:----------:|:----------------:|:----------:|:----------:|:-----------:| -| **K-quants** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ 🐢⁵ | 🚫 | -| **I-quants** | ✅ 🐢⁴ | ✅ 🐢⁴ | ✅ 🐢⁴ | ✅ | ✅ | Partial¹ | ✅ 🐢⁴ | 🚫 | -| **Parallel Multi-GPU⁶** | N/A | N/A | N/A | ✅ | ✅ | Sequential only | Sequential only | ❓ | -| **K cache quants** | ✅ | ✅ | ✅ | ✅ | ✅ | ❓ | ✅ | 🚫 | -| **MoE architecture** | ✅ | ✅ | ✅ | ✅ | ✅ | ❓ | ✅ | 🚫 | -| **Flash Attention** | ✅ | ✅ | ✅ | ✅ | ✅ | ❓ | ✅ | 🚫 | +| | **CPU (AVX/AVX2)** | **CPU (ARM NEON)** | **Metal** | **CUDA** | **ROCm** | **SYCL** | **Vulkan** | +|:-----------------------:|:--------------:|:------------------:|:---------:|:----------:|:----------------:|:----------:|:----------:| +| **K-quants** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ 🐢⁵ | +| **I-quants** | ✅ 🐢⁴ | ✅ 🐢⁴ | ✅ 🐢⁴ | ✅ | ✅ | Partial¹ | ✅ 🐢⁴ | +| **Parallel Multi-GPU⁶** | N/A | N/A | N/A | ✅ | ✅ | Sequential only | Sequential only | +| **K cache quants** | ✅ | ✅ | ✅ | ✅ | ✅ | ❓ | ✅ | +| **MoE architecture** | ✅ | ✅ | ✅ | ✅ | ✅ | ❓ | ✅ | +| **Flash Attention** | ✅ | ✅ | ✅ | ✅ | ✅ | ❓ | ✅ | * ✅: feature works * 🚫: feature does not work * ❓: unknown, please contribute if you can test it yourself * 🐢: feature is slow * ¹: IQ3_S and IQ1_S, see #5886 -* ²: Only with `-ngl 0` -* ³: Inference is 50% slower * ⁴: Slower than K-quants of comparable size * ⁵: Generally the CUDA or ROCM backends are faster, though there are cases where Vulkan has faster text generation. See #10879 for benchmarks. * ⁶: By default, all GPU backends can utilize multiple devices by running them sequentially. The CUDA code (which is also used for ROCm via HIP) also has code for running GPUs in parallel via `--split-mode row`. However, this is optimized relatively poorly and is only faster if the interconnect speed is fast vs. the speed of a single GPU. \ No newline at end of file