upstream-archive/whisper.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/whisper.cpp.git synced 2026-07-23 11:10:57 -05:00
Code Issues Packages Projects Releases Wiki Activity

cuda: align snake fusion matcher with the other backends (llama/25460)

Browse Source 
* cuda: fix snake fusion type predicate, a and inv_b are F32

The matcher required a->type == x->type while launch_snake reads both
as const float *, matching the CPU and Metal contract where a and inv_b
stay F32. F16/BF16 chains never fused and fell back to the naive path,
and a hypothetical all F16 chain would have read F16 bits as float.
Aligns the predicate and the comment with ggml-cpu.c

* cuda: reject snake fusion on non-contiguous operands

The kernel reads x[idx] and a[c] / inv_b[c] linearly, so a
non-contiguous view passing the matcher would silently read wrong data.
Mirror the contiguity guard already present in the CPU, Vulkan and
Metal matchers.
This commit is contained in:
Pascal
2026-07-09 10:00:06 +02:00
committed by Georgi Gerganov
parent def36a5831
commit 1b0b078a98
1 changed files with 9 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
ggml/src/ggml-cuda/ggml-cuda.cu
View File

@@ -3165,18 +3165,21 @@ static int ggml_cuda_try_fuse(ggml_backend_cuda_context * cuda_ctx, ggml_cgraph
(a->ne[2] == 1 && a->ne[3] == 1);
const bool shape_ok = ggml_are_same_shape(a, inv_b) && a->ne[0] == 1 && a->ne[1] == x->ne[1];
// x must be in the supported whitelist and every operand / intermediate
// result must share x's type, since launch_snake casts a / inv_b as
// float and templates the kernel on a single T. Mixed precision chains
// fall back to the naive path.
// x is in the supported whitelist and every chain intermediate shares
// x's type. launch_snake reads a and inv_b as const float *, so they
// stay F32.
const ggml_tensor * sin1 = cgraph->nodes[i + 1];
const bool types_ok = (x->type == GGML_TYPE_F32 || x->type == GGML_TYPE_F16 || x->type == GGML_TYPE_BF16) &&
(a->type == x->type) && (inv_b->type == x->type) &&
(a->type == GGML_TYPE_F32) && (inv_b->type == GGML_TYPE_F32) &&
(mul0->type == x->type) && (sin1->type == x->type) &&
(sqr->type == x->type) && (mul1->type == x->type) &&
(add->type == x->type);
if (types_ok && shape_ok && dim_ok && x_in_add == x) {
// kernel reads x[idx] and a[c] / inv_b[c] linearly, so every operand is contiguous
const bool contig_ok = ggml_is_contiguous(x) && ggml_is_contiguous(add) &&
ggml_is_contiguous(a) && ggml_is_contiguous(inv_b);
if (types_ok && shape_ok && dim_ok && contig_ok && x_in_add == x) {
ggml_cuda_op_snake_fused(*cuda_ctx, x, a, inv_b, add);
return 4;
}