 leejet
|
59c23bce0d
|
fix: use tokenizer-specific pre-tokenization rules (#1975)
|
2026-09-15 02:37:52 +08:00 |
|
|
leejet
|
f9ddc0f388
|
refactor: require external Gemma 2 and GPT-OSS tokenizers (#1974)
|
2026-09-15 01:25:13 +08:00 |
|
|
leejet
|
4964abdfc5
|
feat: support external Hugging Face tokenizer JSON files (#1973)
|
2026-09-15 00:27:39 +08:00 |
|
|
Maphist0
|
7f986a9d73
|
feat: add SenseNova U1.5 support (#1935)
|
2026-09-12 00:39:59 +08:00 |
|
|
leejet
|
dbb611264e
|
feat: add verbose logging and log-level selection (#1941)
|
2026-09-06 23:14:03 +08:00 |
|
|
vmobilis
|
9cfe2af8f9
|
feat: display number of tokens for SD models (#1831)
|
2026-07-29 22:21:54 +08:00 |
|
|
leejet
|
7f0e728b7d
|
fix: normalize CLIP prompts before special-token splitting (#1670)
|
2026-06-17 00:33:00 +08:00 |
|
|
leejet
|
f3fd359b58
|
refactor: reorganize src model layout (#1615)
|
2026-06-07 03:21:12 +08:00 |
|
|
leejet
|
0982807139
|
feat: add PiD support (#1585)
|
2026-05-31 22:38:39 +08:00 |
|
|
leejet
|
d3b2cb047e
|
fix: split tokens before normalization (#1582)
|
2026-05-30 18:38:46 +08:00 |
|
|
leejet
|
92dc7268fc
|
feat: add microsoft lens support (#1560)
|
2026-05-27 01:04:17 +08:00 |
|
|
leejet
|
baf7eda1e4
|
refactor: minify vocab files (#1509)
|
2026-05-17 23:06:58 +08:00 |
|
|
leejet
|
67dda3f897
|
feat: add ltx2.3 support (#1463)
* add GemmaTokenizer
* add basic ltx2.3 support
* change vocab file encoding
* fix ci
* fix ubuntu build
* add temporal tiling support
* add ltx audio support
* update ggml submodule url
* fix generate_video
* add i2v support
* minify bundled Gemma tokenizer vocab sources
* pass video fps into temporal rope embeddings
* fix av_ca_timestep_scale_multiplier
* add LTX2Scheduler support
* update docs
* fix ci
|
2026-05-17 16:46:20 +08:00 |
|
|
leejet
|
0665a7f8bf
|
feat: add hidream o1 image support (#1485)
|
2026-05-15 00:40:21 +08:00 |
|
|
akleine
|
970c4a3312
|
chore: replace some NULL with nullptr + use "%zu" for printing some size_t data (#1457)
|
2026-04-27 22:42:57 +08:00 |
|
|
leejet
|
c41c5ded7a
|
feat: add left padding support to tokenizers (#1424)
|
2026-04-15 23:17:47 +08:00 |
|
|
leejet
|
9ac7b672c2
|
refactor: introduce shared tokenizer abstraction and split implementations (#1423)
|
2026-04-15 22:44:39 +08:00 |
|