llama.cpp

upstream-archive/llama.cpp

Fork 0

mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-09-30 09:57:38 -05:00

Commit Graph

Author	SHA1	Message	Date
Georgi Gerganov	eb34620aec	Add tokenizer test + revert to C++11 (#355 ) * Add test-tokenizer-0 to do a few tokenizations - feel free to expand * Added option to convert-pth-to-ggml.py script to dump just the vocabulary * Added ./models/ggml-vocab.bin containing just LLaMA vocab data (used for tests) * Added utility to load vocabulary file from previous point (temporary implementation) * Avoid using std::string_view and drop back to C++11 (hope I didn't break something) * Rename gpt_vocab -> llama_vocab * All CMake binaries go into ./bin/ now	2023-03-21 17:29:41 +02:00
Radoslav Gerganov	60f819a2b1	Add section to README on how to run the project on Android (#130 )	2023-03-14 15:30:08 +02:00
Georgi Gerganov	319cdb3e1f	Final touches	2023-03-10 21:50:46 +02:00

Author

SHA1

Message

Date

Georgi Gerganov

eb34620aec

Add tokenizer test + revert to C++11 (#355 )

* Add test-tokenizer-0 to do a few tokenizations - feel free to expand
* Added option to convert-pth-to-ggml.py script to dump just the vocabulary
* Added ./models/ggml-vocab.bin containing just LLaMA vocab data (used for tests)
* Added utility to load vocabulary file from previous point (temporary implementation)
* Avoid using std::string_view and drop back to C++11 (hope I didn't break something)
* Rename gpt_vocab -> llama_vocab
* All CMake binaries go into ./bin/ now

2023-03-21 17:29:41 +02:00

Radoslav Gerganov

60f819a2b1

Add section to README on how to run the project on Android (#130 )

2023-03-14 15:30:08 +02:00

Georgi Gerganov

319cdb3e1f

Final touches

2023-03-10 21:50:46 +02:00