Eve 3216efef2e AVX BF16 and single scale quant optimizations (llama/10212) ...

* use 128 bit loads (i've tried 256->128 to death and its slower)

* double accumulator

* avx bf16 vec dot

* +3% q4_0 inference

* +7% tg +5% pp compared to master

* slower f16c version, kep for reference

* 256b version, also slow. i tried :)

* revert f16

* faster with madd

* split to functions

* Q8_0 and IQ4_NL, 5-7% faster

* fix potential overflow (performance reduced)

* 16 bit add for q4_0 only

* merge

2024-11-20 21:00:08 +02:00

..

ggml-amx

ggml : build backends as libraries (llama/10256)

2024-11-20 21:00:08 +02:00

ggml-blas

ggml : build backends as libraries (llama/10256)

2024-11-20 21:00:08 +02:00

ggml-cann

ggml : build backends as libraries (llama/10256)

2024-11-20 21:00:08 +02:00

ggml-cpu

AVX BF16 and single scale quant optimizations (llama/10212)

2024-11-20 21:00:08 +02:00

ggml-cuda

ggml : build backends as libraries (llama/10256)

2024-11-20 21:00:08 +02:00

ggml-hip

ggml : build backends as libraries (llama/10256)

2024-11-20 21:00:08 +02:00

ggml-kompute

ggml : build backends as libraries (llama/10256)

2024-11-20 21:00:08 +02:00

ggml-metal

ggml : build backends as libraries (llama/10256)

2024-11-20 21:00:08 +02:00

ggml-musa/ggml

ggml : build backends as libraries (llama/10256)

2024-11-20 21:00:08 +02:00

ggml-rpc

ggml : build backends as libraries (llama/10256)

2024-11-20 21:00:08 +02:00

ggml-sycl

sycl: Use syclcompat::dp4a (llama/10267)

2024-11-20 21:00:08 +02:00

ggml-vulkan

ggml : build backends as libraries (llama/10256)

2024-11-20 21:00:08 +02:00

kompute-shaders

whisper : reorganize source code + improve CMake (#2256)

2024-06-26 19:34:09 +03:00

vulkan-shaders

vulkan: Optimize contiguous copies (llama/10254)

2024-11-15 15:21:04 +02:00

CMakeLists.txt

cmake : fix ppc64 check (#0)

2024-11-15 15:21:04 +02:00

ggml-aarch64.c

ggml : build backends as libraries (llama/10256)

2024-11-20 21:00:08 +02:00

ggml-aarch64.h

ggml : build backends as libraries (llama/10256)

2024-11-20 21:00:08 +02:00

ggml-alloc.c

ggml : move more prints to the ggml log system (llama/9839)

2024-11-01 10:19:05 +02:00

ggml-amx.cpp

llama : refactor model loader with backend registry (llama/10026)

2024-11-15 15:21:04 +02:00

ggml-backend-impl.h

llama : refactor model loader with backend registry (llama/10026)

2024-11-15 15:21:04 +02:00

ggml-backend-reg.cpp

ggml : build backends as libraries (llama/10256)

2024-11-20 21:00:08 +02:00

ggml-backend.cpp

ggml : tmp workaround for whisper.cpp (skip) (#2565)

2024-11-16 20:21:24 +02:00

ggml-blas.cpp

llama : refactor model loader with backend registry (llama/10026)

2024-11-15 15:21:04 +02:00

ggml-cann.cpp

CANN: adjust backend registry refactor. (llama/10158)

2024-11-15 15:21:04 +02:00

ggml-common.h

ggml-quants : ternary packing for TriLMs and BitNet b1.58 (llama/8151)

2024-09-24 19:45:08 +03:00

ggml-cpu-impl.h

ggml : add ggml-cpu-impl.h (skip) (#0)

2024-09-24 19:45:08 +03:00

ggml-cpu.c

fix q4_0_8_8 format for corrupted tokens issue (llama/10198)

2024-11-15 15:21:04 +02:00

ggml-cuda.cu

metal : optimize FA kernels (llama/10171)

2024-11-15 15:21:04 +02:00

ggml-impl.h

ggml : build backends as libraries (llama/10256)

2024-11-20 21:00:08 +02:00

ggml-kompute.cpp

kompute: add mul_mat_q4_k shader (llama/10097)

2024-11-15 15:21:04 +02:00

ggml-metal.m

metal : fix build and some more comments (llama/10229)

2024-11-15 15:21:04 +02:00

ggml-metal.metal

metal : more precise Q*K in FA vec kernel (llama/10247)

2024-11-15 15:21:04 +02:00

ggml-quants.c

ggml : build backends as libraries (llama/10256)

2024-11-20 21:00:08 +02:00

ggml-quants.h

ggml : build backends as libraries (llama/10256)

2024-11-20 21:00:08 +02:00

ggml-rpc.cpp

ggml : move CPU backend to a separate file (llama/10144)

2024-11-15 15:21:04 +02:00

ggml-sycl.cpp

sycl : Fixes to broken builds and test-backend-ops (llama/10257)

2024-11-15 15:21:04 +02:00

ggml-threading.cpp

ggml : build backends as libraries (llama/10256)

2024-11-20 21:00:08 +02:00

ggml-threading.h

ggml : build backends as libraries (llama/10256)

2024-11-20 21:00:08 +02:00

ggml-vulkan.cpp

vulkan: Optimize contiguous copies (llama/10254)

2024-11-15 15:21:04 +02:00

ggml.c

ggml : build backends as libraries (llama/10256)

2024-11-20 21:00:08 +02:00

sgemm.cpp

whisper : reorganize source code + improve CMake (#2256)

2024-06-26 19:34:09 +03:00

sgemm.h

whisper : reorganize source code + improve CMake (#2256)

2024-06-26 19:34:09 +03:00