upstream-archive/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-08-04 17:10:48 -05:00
Code Issues Packages Projects Releases Wiki Activity
Files
4e732e0a6c49d41e5aaf3e460f9685a63529adb5
llama.cpp/tools/server/server-context.cpp
Aman Gupta 4e732e0a6c llama: allow partial seq_rm for GDN models for speculative decoding 
Currently speculative checkpoint needs to restart from a checkpoint
after some draft tokens are not accepted, this leads to some wastage in
running the target again. This PR adds the ability to rollback upto
`draft_max` by storing the GDN intermediates.
2026-05-14 11:32:23 +08:00

182 KiB
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink