mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2026-09-28 08:57:40 -05:00
bri-prismandGeorgi Gerganov
e351231c4f
* metal: FWHT kernels for block widths above 512 The Metal FWHT covers widths 64 to 512, one row per simdgroup with N/32 values per lane. Wider blocks need more registers per lane than that layout allows. kernel_fwht_tg runs one row per threadgroup with 256 threads, so each thread keeps N/256 values. Butterflies below the simdgroup width still shuffle, those up to the threadgroup width go through threadgroup memory, and the rest stay in registers. Same butterfly and sign convention as the simdgroup kernel. Widths 64 to 512 keep the simdgroup kernel. 1024 through 8192 use the new one, for both F32 and F16 sources. The wide kernels allocate float[N] of threadgroup memory, 32 KB at 8192, so the size check takes the device limit and reports those widths as unsupported where they would not fit. Without that a device with less threadgroup memory would accept the op and then abort on a nil pipeline. test-backend-ops on M5 Pro: MUL_MAT_HADAMARD 26/26, MUL_MAT 1265/1265. * cont : add TODOs --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>