qdrant

mirror of https://github.com/qdrant/qdrant.git synced 2026-08-06 01:50:57 -05:00

Files

Andrey Vasnetsov 5d2ec88276 Second thoughts on CJK tokenizer (#2260 )

* charabia tokenizer for CJK, init

* add charabia to .proto

* generate grpc docs

* rename charabia to multilingual

* disable charabia default features

* ignore stopwords and separators in tokenizer

* fix rebase conflict

* fix test

* fix test

* Bump `charabia` crate version...

...and enable `charabia` features that does not require any additional dependencies by default

* Expose optional `charabia` features in the `segment` crate

* Expose optional `charabia` features in the `qdrant` crate

* fix codespell

* more regression tests

* more regression tests

* make language-specific tests feature-flagged

---------

Co-authored-by: Anatolii Smolianinov <zarkonesmall@gmail.com>
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>

2023-07-19 14:59:40 +02:00

grpc

Second thoughts on CJK tokenizer (#2260 )

2023-07-19 14:59:40 +02:00

imgs

[WIP] Limit segment size #135 (#195 )

2022-01-18 15:49:35 +01:00

redoc

Second thoughts on CJK tokenizer (#2260 )

2023-07-19 14:59:40 +02:00

roadmap

docs: update roadmap with quantization (#1961 )