upstream-archive/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-10-01 18:37:28 -05:00
Code Issues Packages Projects Releases Wiki Activity

convert : write Gemma embedding scale for DFlash drafts (#29802)

Browse Source 
* convert : write Gemma embedding scale for DFlash drafts

A DFlash draft shares the target's token embeddings. Gemma scales them by sqrt(hidden_size) in the forward pass, and the draft config does not state that scale, so the converted draft read unscaled embeddings.

Take the scale from the target config when the draft config has none.

Assisted-by: Claude

* convert : check with get_model_architecture for gemma models
This commit is contained in:
Yu Chengye
2026-10-01 16:34:32 +02:00
committed by GitHub
parent 42d958167a
commit 2b36825cbc
1 changed files with 6 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
conversion/qwen.py
+6
View File
@@ -729,6 +729,12 @@ class DFlashModel(Qwen3Model):
embedding_scale = dflash_config.get(
"input_embedding_scale", self.hparams.get("input_embedding_scale")
)
if embedding_scale is None and self.target_model_dir is not None:
# the draft shares the target's token embeddings, and Gemma scales them by sqrt(hidden_size) in the forward pass
target_hparams = ModelBase.load_hparams(self.target_model_dir, False)
if get_model_architecture(target_hparams, ModelType.TEXT).startswith("Gemma"):
target_hparams = {**target_hparams, **target_hparams.get("text_config", {})}
embedding_scale = target_hparams["hidden_size"] ** 0.5
if embedding_scale is not None:
self.gguf_writer.add_embedding_scale(float(embedding_scale))