mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2026-07-23 11:10:55 -05:00
e6dd0e29a6
* args: overhaul mmap/mlock/dio into single arg Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * docs: update docs with llama-gen-docs Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * chore: satisfy code quality Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * args: make the `+` sign an actual modifier now Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * chore: general code clean up + comments Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * arg: fix deprecated flags support Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * arg: quick sanity check Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * bench: sync llama-bench argument parsing Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * fix: bugfix variable behaviour + llama-bench lm column size Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * arg: inverse commands should do the opposite instead of doing nothing Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * bench: fix incorrect dash Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * bench: fix missing modifiers for deprecated flags Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * llama: switch back to thread_local Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * arg: switch back to single enum Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * docs: update arg docs Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * chore: fix missing `mlock` from llama_load_mode_from_str + cleanup llama-bench Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * llama: fix mlock not activating Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * arg: add deprecation warning when old and new flags are combined Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * arg: cont add comment for todo in the future Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * docs: sync with upstream Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * docs: re-sync with upstream again Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> --------- Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
Diffusion Text Generation
This directory contains implementations for Diffusion LLMs (DLLMs)
More Info:
Parameters
The diffusion CLI supports various parameters to control the generation process:
Core Diffusion Parameters
--diffusion-steps: Number of diffusion steps (default: 256)--diffusion-algorithm: Algorithm for token selection0: DIFFUSION_ALGORITHM_ORIGIN - Token will be generated in a purely random order from https://arxiv.org/abs/2107.03006.1: DIFFUSION_ALGORITHM_ENTROPY_BASED - Entropy-based selection2: DIFFUSION_ALGORITHM_MARGIN_BASED - Margin-based selection3: DIFFUSION_ALGORITHM_RANDOM - Random selection4: DIFFUSION_ALGORITHM_CONFIDENCE_BASED - Confidence-based selection (default)- More documentation here https://github.com/DreamLM/Dream
--diffusion-visual: Enable live visualization during generation
Scheduling Parameters
Choose one of the following scheduling methods:
Timestep-based scheduling:
--diffusion-eps: Epsilon value for timestep scheduling (e.g., 0.001)
Block-based scheduling:
--diffusion-block-length: Block size for block-based scheduling (e.g., 32)
Sampling Parameters
--temp: Temperature for sampling (0.0 = greedy/deterministic, higher = more random)--top-k: Top-k filtering for sampling--top-p: Top-p (nucleus) filtering for sampling--seed: Random seed for reproducibility
Model Parameters
-m: Path to the GGUF model file-p: Input prompt text-ub: Maximum sequence length (ubatch size)-c: Context size-b: Batch size
Examples
Dream architecture:
llama-diffusion-cli -m dream7b.gguf -p "write code to train MNIST in pytorch" -ub 512 --diffusion-eps 0.001 --diffusion-algorithm 3 --diffusion-steps 256 --diffusion-visual
LLaDA architecture:
llama-diffusion-cli -m llada-8b.gguf -p "write code to train MNIST in pytorch" -ub 512 --diffusion-block-length 32 --diffusion-steps 256 --diffusion-visual
RND1 architecture:
llama-diffusion-cli -m RND1-Base-0910.gguf -p "write code to train MNIST in pytorch" -ub 512 --diffusion-algorithm 1 --diffusion-steps 256 --diffusion-visual --temp 0.5 --diffusion-eps 0.001