mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2026-09-25 07:27:30 -05:00
bri-prism
9a9f939b80
* metal: add F16 input to the FWHT The Metal FWHT kernel accepts F32 input only. This change makes the source type a template parameter, so the kernel reads an F16 source directly instead of requiring a converted copy. The F32 instantiations are unchanged. The pipeline name now carries the source type, and supports_op accepts an F16 src1 for the Hadamard hint at the four sizes the kernels cover. Every other F16 src1 path still goes through ggml_metal_supports_mul_mat_op. These are the test cases mentioned in #27779. test-backend-ops on M5 Pro: MUL_MAT_HADAMARD 16/16, MUL_MAT 1265/1265. * metal: ask the same FWHT question in supports_op and the dispatch supports_op admitted an F16 src1 on the type, the hint and the width alone, but the dispatch also requires src1 and dst to be contiguous and the same shape. A Hadamard hinted MUL_MAT that passed the first and failed the second reached the generic path, which has no F32 src0 by F16 src1 kernel, and aborted on a nil pipeline: kernel not found in any metal library: base = 'kernel_mul_mv_f32_f16_4' ggml_metal_encoder_set_pipeline: nil Metal pipeline ggml_metal_use_fwht now holds the whole condition and both callers use it, so they cannot drift apart again. The added test case has src1 and dst of different shapes, which aborted before this change and is declined by the Metal backend after it. * metal: branchless butterfly select in the FWHT simdgroup kernel Review suggestion. Replaces the ternary in the shuffle stages with val2 - val + 2*((lane & i) == 0)*val, which is the same value without the select. Measured on M5 Pro, interleaved A/B, five rounds, first discarded, on a Hadamard matmul with block 512 and 65536 rows so the kernel rather than the launch dominates: 1324.6 us before, 1285.0 us after, a 3.0% gain, and faster in every round. At the shapes already in the perf suite the op runs 1.6 to 3.9 us against a 1.6 us launch floor, so the difference is not visible there. FOR_UNROLL on the same loops was also measured and made no difference, the delta changing sign between rounds, so it is not included. * metal: move the FWHT dispatch predicates to ggml-metal-common Review feedback. ggml_metal_use_fwht and ggml_metal_fwht_supported_size were static inline in ggml-metal-device.h. They now follow the ggml_metal_op_mul_mat_use_mm pattern: declared in ggml-metal-common.h and implemented in ggml-metal-common.cpp, which is already the home for helpers shared between supports_op and the op dispatch. The predicate is named ggml_metal_op_mul_mat_use_fwht to sit alongside the _use_mm pair it parallels. This also fixes the macos-latest-arm64 build. The header needed ggml-impl.h for ggml_get_op_params_i32, but ggml-metal-device.h is reached from tools/tuning through ggml-metal-tuning.h, and that target does not have ggml/src on its include path. ggml-metal-common.cpp already includes ggml-impl.h, so the accessor is used normally there and the header goes back to needing nothing extra. * metal: keep the FWHT size check internal and group the dispatch helpers Applies the patch from the review. ggml_metal_fwht_supported_size becomes static in ggml-metal-common.cpp since nothing outside it needs the size list, which also drops stdint.h from the header again, and ggml_metal_op_mul_mat_use_fwht joins the existing _use_mm declarations under their shared comment instead of carrying its own block. * tests: drop the mismatched-shape Hadamard case I added a case with m != k to cover an abort, but the hint is a promise that src0 is a Hadamard matrix, so src0 is square and dst has the same shape as src1. Every other case in the suite holds to that. The case was not a valid op, and on CPU it compared the FWHT against a real matmul of a non-square src0, which cannot agree. The supports_op and dispatch conditions still come from one predicate, which is what keeps them from disagreeing on contiguity.