textgen/docs/What Works.md at fe0b385dae88e32dacb2218cc3af7db2f37a736b

mirror of https://github.com/oobabooga/textgen.git synced 2026-09-21 13:38:03 -05:00

Files

T

oobabooga 2f08dce7b0 Remove ExLlamaV2 backend

- archived upstream: https://github.com/turboderp-org/exllamav2/commit/7dc12af3a81f34ac3f27cd7602ed539b638933ca
- replaced by ExLlamaV3, which has much better quantization accuracy

2026-03-05 14:02:13 -08:00

797 B

Raw Blame History

What Works

Loader	Loading LoRAs	Training LoRAs	Multimodal	Perplexity evaluation
llama.cpp	❌	❌	✅*	❌
Transformers	✅	✅	✅**	✅
ExLlamav3_HF	❌	❌	❌	✅
ExLlamav3	❌	❌	✅	❌
TensorRT-LLM	❌	❌	❌	❌

❌ = not supported

✅ = supported

* Via the mmproj parameter (multimodal projector file).

** Via the send_pictures extension.

797 B Raw Blame History

What Works

797 B

Raw Blame History