Zach Winter 555881ebc8 ui: reduce per-token render cost when streaming (#26053) ...

* performance harness - the empirical root

Assisted-by: Claude Opus 4.8

* 210.36ms -> 2.67ms per streamed token

Assisted-by: Claude Opus 4.8

* 11.58ms -> 0.62ms per streamed token

Assisted-by: Claude Opus 4.8

* 22.02ms -> 3.33ms per streamed token

Assisted-by: Claude Opus 4.8

* 3.07ms -> 1.36ms per streamed token at 40 messages

Assisted-by: Claude Opus 4.8

---------

Co-authored-by: Zach Winter <dmtommy@icloud.com>

2026-07-24 22:09:46 +02:00

..

batched-bench

cmake : add install() for impl libraries + fix apple builds (#23511)

2026-05-22 11:46:26 +03:00

cli

args: refactor mlock/mmap/directio into load-mode (#20834)

2026-07-23 20:32:56 +08:00

completion

args: refactor mlock/mmap/directio into load-mode (#20834)

2026-07-23 20:32:56 +08:00

cvector-generator

libs : rename libcommon -> libllama-common (#21936)

2026-04-17 11:11:46 +03:00

export-lora

docs: fix export-lora --lora-scaled syntax [no release] (#24703)

2026-06-18 16:46:17 +02:00

fit-params

cmake : add install() for impl libraries + fix apple builds (#23511)

2026-05-22 11:46:26 +03:00

gguf-split

libs : rename libcommon -> libllama-common (#21936)

2026-04-17 11:11:46 +03:00

imatrix

Move duplicated imatrix code into single common imatrix-loader.cpp (#22445)

2026-06-04 17:45:40 +02:00

llama-bench

args: refactor mlock/mmap/directio into load-mode (#20834)

2026-07-23 20:32:56 +08:00

mtmd

mtmd: use RAII for setting and resetting non-causal attention (#25723)

2026-07-22 18:10:03 +02:00

parser

chat: harden caps check (#24973)

2026-06-25 02:49:22 +02:00

perplexity

perplexity : fix format specifier in LOG_ERR (#23788)

2026-05-28 10:34:58 +03:00

quantize

Add Q2_0 quantization: type definition and CPU backend (#24448)

2026-07-07 12:05:47 -07:00

results

libs : rename libcommon -> libllama-common (#21936)

2026-04-17 11:11:46 +03:00

rpc

binaries : Improve rpc-server and export-graph-ops names. (#25045)

2026-06-27 10:31:29 +03:00

server

server: support "reasoning_effort": "none" in OAI API (#26045)

2026-07-24 19:19:10 +02:00

tokenize

tokenize : drop --stdin mutual-exclusion check (#25672)

2026-07-15 18:41:51 +02:00

tts

logs : reduce (#23021)

2026-05-14 13:05:52 +03:00

ui

ui: reduce per-token render cost when streaming (#26053)

2026-07-24 22:09:46 +02:00

CMakeLists.txt

cmake: skip cvector-generator and export-lora when CPU backend is disabled (#24053)

2026-06-04 13:13:19 +03:00