Jesse Gross bef41f710a tokenizer, ml: remove dead code ...

Several pieces outlived the code that used them. The root tokenizer
package implemented the GGUF-side vocabularies for the Go engine and the
safetensors-to-GGUF converter; nothing has imported it since the
converter went. ml/backend.go held the Go engine's Backend, Context and
Tensor interfaces, with fs.Config existing only to be returned from them,
and a single CUDA template instance under ml/backend/ggml survived the
engine removal along with the gitattributes entries for that tree and the
CI change-filter globs for it and for the long-gone llama/llama.cpp. From
the image generation engine, an integration test group that no test
registers, its build tag, and the StepBar progress widget remained.
DeviceInfo.IsBetter has no caller at all.

All of it goes. Tidying the module file drops the regexp2 dependency and
leaves protobuf as an indirect requirement. The llama3.2 tokenizer
fixtures stay: the MLX runner's tokenizer uses them for its GGML parity
test.

2026-09-16 14:06:08 -07:00

..

device_test.go

llama: enable FA on CUDA CC 6.x GPUs (#16994)

2026-07-02 17:11:39 -07:00

device.go

tokenizer, ml: remove dead code

2026-09-16 14:06:08 -07:00

path_test.go

runner: Remove CGO engines, use llama-server exclusively for GGML models (#16031)

2026-05-29 13:35:47 -07:00

path.go

runner: Remove CGO engines, use llama-server exclusively for GGML models (#16031)

2026-05-29 13:35:47 -07:00