mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2026-09-26 16:07:32 -05:00
70c4e1582e
* vulkan: add int8 coopmat quantized matmul shader * apply scales inline * use scalar sums * probe and directly access coopmat values instead of going through shmem * add q8_0 support * add BK_STEP to shader, default to 2 * use larger workgroups * double buffering * preload scales * coopmat load first, then wmma * use float for scales * add faster RDNA int->float conversion * workgroup scheduling for cache proximity * clean up * use wave32 * restructure for vgpr use * skip computation for inactive tiles * only force subgroup size 32 on AMD RDNA * use BK_STEP 4 * fix compilation * move quant-specific prefetch function out of main file * add q4_1, q5_0, q5_1 support * restructure mmq cm1 functions * enable mul_mat_id support * fix segfault * fix mul_mat_id bug * support iq4_nl and mxfp4 * remove elem row/col fast path, invalid for RDNA4 * use shmem arrays for LUTs * use 4-byte loads where possible * add q3_k, q4_k, q5_k, q6_k and nvfp4 support * fix l warptile * improve performance * improve performance * improvements * dedup b scales * merge shmem arrays * undo uint8_t, gate to RDNA3/4 * add RDNA4 architecture, use for hardcoded coopmat elem thread access, set BK_STEP back to 4 * improve offset application * clean up * fix iq4_nl and nvfp4 performance * rdna4 tuning * use BK_STEP 2 on MUL_MAT_ID * adapt to upstream changes * fix shmem support function, clean up comments * fix warptile logic Co-authored-by: Piotr Wilkin (ilintar) <piotr.wilkin@syndatis.com> * vulkan: add IQ4_XS support to the coopmat1 integer matmul shader (#28440) Adds IQ4_XS to mul_mmq_cm1: dedicated block_a_load/block_a_to_shmem that expand both nibbles of each packed32 word through cm1_kvalues, LOAD_VEC_A 8 and an IQ4_XS-sized a_panel_bytes estimate for the L2-friendly scheduling. Assisted-by: OpenAI Codex Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> * avoid compiling f16 acc shader variants --------- Co-authored-by: Piotr Wilkin (ilintar) <piotr.wilkin@syndatis.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>