mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2026-09-25 15:37:25 -05:00
wendadawen
828fdf282e
* model : add DFlash layer-input taps for HunyuanVL DFlash speculative decoding needs the target graph to expose the residual stream entering each layer (res->t_layer_inp[il]) - the draft model reads those tensors to build its cross-context. Qwen3 and the other DFlash-capable targets register them, but the Hunyuan graphs do not, so serving a DFlash draft against a HunyuanOCR target aborts during the first graph build: GGML_ASSERT(t_layer_inp[il] != nullptr && "layer input tensor is null") Register the tensor at the top of the layer loop, mirroring qwen3. The layer input is the residual stream entering layer il, i.e. the output of layer il-1, which is what the draft's target_layers metadata refers to (the converter writes target_layer_ids+1). hunyuan-dense.cpp reuses this graph, so it is covered as well; hunyuan-moe has a separate graph and is untouched. The vector is only read when a speculative implementation enables those layer ids, so there is no behaviour change without a draft model. Tested with tencent/HunyuanOCR 1.5 and its DFlash draft: image requests now run, draft acceptance is ~0.5 and the OCR output is byte-identical to the non-speculative run. Co-authored-by: wendadawen <wendadawen@qq.com> * convert : fix DFlash draft conversion against HunYuan targets Converting a DFlash draft with a HunYuan target failed in two ways. 1. DFlashModel.set_vocab() reuses the target class' vocab handling by calling it unbound with the draft instance, but HunYuanModel.set_vocab() called self._fix_special_tokens(), a method that only exists on HunYuanModel, so the conversion always aborted with AttributeError: 'DFlashModel' object has no attribute '_fix_special_tokens' Make the vocab helpers module-level functions taking the model explicitly, so they do not depend on the instance being a HunYuanModel. They have no other callers, so the two id lookups are folded into _fix_special_tokens(). 2. The delegated call runs with self.dir_model pointed at the target but keeps the draft's self.hparams, so config lookups inside the target's vocab code (the pad_token_id < 0 guard, eod_token_id) read the draft's config instead of the target's. That aborts on targets with pad_token_id = -1 (e.g. the HunyuanOCR v1.0 checkpoint) and otherwise writes special token ids that disagree with the target. Add _vocab_hparams(): it returns the target's config (with text_config merged to the root, as TextModel does) when the model is a draft converted with --target-model-dir, and the model's own hparams otherwise, so a normal conversion is unaffected. Tested: converting tencent/HunyuanOCR/dflash succeeds with both the 1.5 and the v1.0 target; converting the base model without --target-model-dir produces a byte-identical GGUF to before. Co-authored-by: wendadawen <wendadawen@qq.com> * convert : fix DFlash draft vocab against HunYuan targets Switch hparams to the target config for the duration of the borrowed set_vocab(), matching the existing dir_model swap, instead of teaching HunYuanModel::set_vocab about draft models. * convert : fix HunYuan special token ids for DFlash drafts * convert : use load_hparams for HunYuan special token ids