upstream-archive/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-08-06 10:00:48 -05:00
Code Issues Packages Projects Releases Wiki Activity
Files
b22ff4b7b43b6d0d91636f85692ff216cb7cb607
llama.cpp/conversion/base.py
Kashif Rasul afcda09d15 vocab : fix HybridDNA tokenizer (#23466) 
* vocab : mark hybriddna k-mers to avoid BPE token collisions

* improved loop

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
2026-05-22 11:17:31 +02:00

125 KiB
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink