upstream-archive/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-08-06 01:50:50 -05:00
Code Issues Packages Projects Releases Wiki Activity

Add initial version for top-p sampling

Browse Source 
As we only support static graphs for the time and we don't know the size
of the output of top-p, we have to do value-scaling same as for min-p
operator.

Further improvements can be applied to the unit-test (i.e. check for
equivalence of top_p happening on backend with top_p happening on cpu)
and also by constructing candidates and sorting those as opposed to
reversing the sort of the logits (this would be arange +
get_rows instead of argsort + get_rows)
This commit is contained in:
Oliver Simons
2025-11-28 15:08:20 +01:00
parent 117e2079a9
commit 333da805fe
4 changed files with 185 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
common/sampling.cpp
View File

@@ -169,6 +169,7 @@ static bool common_sampler_type_has_backend_support(enum common_sampler_type typ
case COMMON_SAMPLER_TYPE_TOP_K:
case COMMON_SAMPLER_TYPE_TEMPERATURE:
case COMMON_SAMPLER_TYPE_MIN_P:
case COMMON_SAMPLER_TYPE_TOP_P:
return true;
default:
return false;
@@ -382,6 +383,9 @@ struct common_sampler * common_sampler_init(const struct llama_model * model, st
case COMMON_SAMPLER_TYPE_MIN_P:
llama_sampler_chain_add(result->chain_backend, llama_sampler_backend_init_min_p(params.min_p));
break;
case COMMON_SAMPLER_TYPE_TOP_P:
llama_sampler_chain_add(result->chain_backend, llama_sampler_backend_init_top_p(params.top_p));
break;
default:
GGML_ASSERT(false && "unsupported backend sampler");
}