model : re-enable -sm tensor for qwen4exp (#28569)

#27941 disabled -sm tensor for qwen4exp because test-llama-archs asserted on the
Meta device once the fixture carried a PLE layer:
GGML_ASSERT(ggml_backend_buffer_is_meta(tensor->buffer)) at ggml-backend-meta.cpp:476.

With host-resident embeddings the PLE gather is a CPU node and hc_init (the REPEAT
that fans the embedding out to the hc streams) was first reached through layer 0's
PLE path, after that gather. ggml_backend_sched_split_graph pass 2 expands a device
assignment upwards only until it meets a CPU node, so the REPEAT stayed on the CPU
and the later reshape of hc_init inside the meta split viewed a host-resident node.

Expanding hc_init right after it is built puts the REPEAT directly before the first
device node, where pass 2 assigns it; the embedding reshape stays in the CPU split
and is copied in as a split input, as in deepseek4.
This commit is contained in:
Kevin Hopper
2026-10-01 08:16:49 +03:00
committed by GitHub
parent 0c1e57098b
commit 10f340d1a2
2 changed files with 2 additions and 1 deletions
-1
View File
@@ -1171,7 +1171,6 @@ bool llm_arch_supports_sm_tensor(const llm_arch & arch) {
case LLM_ARCH_KIMI_K3:
case LLM_ARCH_GLM5_NEXT:
case LLM_ARCH_QWEN3TTS:
case LLM_ARCH_QWEN4EXP: // TODO: fix test-llama-archs
return false;
default:
return true;
+2
View File
@@ -393,6 +393,8 @@ llama_model_qwen4exp::graph::graph(const llama_model & model, const llm_graph_pa
ggml_reshape_3d(ctx0, inpL, n_embd, 1, n_tokens),
n_embd, hc, n_tokens, 1);
cb(res_hc, "hc_init", -1);
// make sure hc_init is in the same graph split as the first layer (-sm tensor)
ggml_build_forward_expand(gf, res_hc);
for (int il = 0; il < n_layer; ++il) {
res->t_layer_inp[il] = res_hc;