Commit Graph

3618 Commits

Author SHA1 Message Date
Ivan Pleshkov
cd2f334271 horizontal sum and test 2022-01-04 16:30:36 +03:00
Andrey Vasnetsov
3b2b1bbf93 use seeded rng for stochastic test 2022-01-04 11:25:24 +01:00
Ivan Pleshkov
70e376bf5e add avx2 implementation 2022-01-04 00:32:13 +03:00
Ivan Pleshkov
297f54141d remove blas 2022-01-03 22:16:27 +03:00
Anton Kaliaev
cb0a1b6bbc use map_or instead of map & unwrap_or (#172)
https://rust-lang.github.io/rust-clippy/master/index.html#map_unwrap_or
2022-01-03 17:28:56 +01:00
Anton Kaliaev
298685102c add missing commas (#173) 2022-01-03 17:28:46 +01:00
Anton Kaliaev
57fa65072f use copied instead of cloned (#174)
* use copied instead of cloned

https://rust-lang.github.io/rust-clippy/master/index.html#cloned_instead_of_copied

* use copied instead of cloned
2022-01-03 17:28:36 +01:00
Anton Kaliaev
e196ce025b fix clippy warnings (#175)
* fix clippy warnings

- doc links
- explicit deref
- if instead of match for single bool condition
- combine similar match branches

* revert removal of transmute

* return Some when vec is not empty
2022-01-03 17:28:21 +01:00
Daniil
1ad529c315 Allow to include vector into search result (#176)
* feat(#50): include vector into search result

allow to specify 'with_vector' parameter in search api to get search results vector data

* test(#50): fix tests

* chore(#50): apply cargo fmt

* chore(#50): update api docs

run tools/generate_openapi_models.sh

Co-authored-by: Daniil Sunyaev <dasforrum@gmail.com>
2022-01-03 17:12:01 +01:00
Andrey Vasnetsov
f4a0511385 Add OpenAPI version selector (#171)
* add OpenAPI version selector

* fix typo
2022-01-03 11:34:09 +01:00
Ivan Pleshkov
bcaa160ad8 Remove dyn Metric from vector storage and use generics (#163)
* Remove dyn Metric from vector storage and use generics

* upd benchmark code

* fix benchmark usage

* fmt

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2021-12-31 09:38:33 +01:00
Arnaud Gourlay
b3fbd0f1c8 rocksdb 0.17.0 (#149) 2021-12-15 09:53:36 +01:00
Arnaud Gourlay
0f5e761ee0 tokio 1.14.0 (#148) 2021-12-15 09:53:08 +01:00
Arnaud Gourlay
072454ee68 Fix benches compilation for TestRawScorerProducer::new (#142) 2021-12-13 17:16:15 +01:00
Arnaud Gourlay
98e99e1f6f Improve error reporting in collection_loader (#143) 2021-12-13 17:15:56 +01:00
Andrey Vasnetsov
3cc7e5f7aa refactor: replace parking_lot with tokio mutex for WAL (#140)
* refactor: replace parking_lot with tokio mutex for WAL

* cargo fmt
2021-12-13 11:44:51 +01:00
Andrey Vasnetsov
d7148126ce Split collection update API into several endpoints (#126)
* split storage operation structures #32

* cargo fmt #32

* split collection update api into several endpoints #32

* cargo fmt #32

* fix tonic-related code with new structures

* upd alias structures

* use ytt teplate engine for OpenAPI Endpoint schema generation
2021-12-13 09:38:33 +01:00
Anton V
eb786ab64b Multithread optimizer (#134)
* run optimizers on tokio thread pool for cpu-bound tasks

* [WIP] move check condition to another thread

* [WIP] optimizer iter live not long enough

* [WIP] change Box to Arc in optimizers vector

* add blocking handles management

* cargo fmt apply

* Update lib/collection/src/update_handler.rs

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>

* [WIP] optimizer iter live not long enough

* [WIP] change Box to Arc in optimizers vector

* add blocking handles management

* fix code review issues

* use CollectionConfig available cpu value

* apply updated fmt

* [WIP] move count of optimization threads to OptimizersConfig

* optimization options

* fix formatting

* fix proto for optimizer config

* fmt

* Update config/config.yaml

related task: https://github.com/qdrant/qdrant/issues/30

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2021-12-09 18:18:20 +01:00
Andrey Vasnetsov
617b97d3f7 add comments for segment entitites (#136)
* add comments for segment entitites

* fmt

* cargo fmt
2021-12-09 11:06:25 +01:00
Andrey Vasnetsov
5e551cb810 fix new version of clippy 2021-12-05 19:24:04 +01:00
Andrey Vasnetsov
97b2270485 Rustdoc and README for internal entities and processes (#123)
* extend comments for strorage crate

* update comments and readme for collection crate

* apply cargo fmt

* fix tests

* apply fmt
2021-11-29 09:39:22 +01:00
Andrey Vasnetsov
8f4acc41f0 replace incompatible hashlib (#132) 2021-11-27 16:11:01 +01:00
Anton V
c922e16d2b Upgrade rust version (#127)
* - update rust version in Dockerfile
- use rust edition 2021

* - update rust edition for libs

* remove *.lock from gitignore
2021-11-24 11:26:19 +03:00
Andrey Vasnetsov
77bc6899ec fix refactoring (#124) 2021-11-21 19:32:18 +01:00
Marcin Puc
c1a790a7af Add more refactorings (#122) 2021-11-12 19:16:00 +01:00
Marcin Puc
fdcc4fa674 Fix ndarray imports in segment benches (#121) 2021-11-11 10:42:27 +01:00
Marcin Puc
c603f0075e Add various refactorings (#118) 2021-11-10 21:32:25 +01:00
Andrey Vasnetsov
29830f953d default payload return is null + update openAPI 2021-10-24 18:47:58 +02:00
Andrey Vasnetsov
bf3d8c2575 data consistency fixes and updates (#112)
* update segment version after completed update only

* more stable updates: check pre-existing points on update, fail recovery, WAL proper ack. check_unprocessed_points WIP

* switch to async channel

* perform update operations in a separate thread (#111)

* perform update operations in a separate thread

* ordered sending update signal

* locate a segment merging versioning bug

* rename id_mapper -> id_tracker

* per-record versioning

* clippy fixes

* cargo fmt

* rm limit of open files

* fail recovery test

* cargo fmt

* wait for worker stops befor dropping the runtime
2021-10-24 18:10:39 +02:00
Andrey Vasnetsov
ab58a41a11 fix seeded rng in caused blinking test 2021-10-12 22:05:16 +02:00
HaiCheViet
f55e5aa7b7 Features/filter payload (#104)
* update more test

* update fmt

* reduce non usecode and update docker version

* update commend code

* update name filter

* renames and minor fixes

* fix linter

Co-authored-by: hai che <haiche@jobhop.com>
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Co-authored-by: Andrey Vasnetsov <vasnetsov93@gmail.com>
2021-10-12 11:07:36 +02:00
Andrey Vasnetsov
370d16d878 use seeded random number generator in search graph tests 2021-10-11 20:41:04 +02:00
Andrey Vasnetsov
a7c865c24c use custom openblas fork wit implemented dynamic arch flag (#106)
* use custom openblas fork wit implemented dynamic arch flag

* add comment
2021-10-06 19:20:57 +03:00
Andrey Vasnetsov
cca7839e18 fix alias operation deadlock (#103) (#105)
* fix alias operation deadlock (#103)

* cargo fmt (#103)
2021-10-06 18:50:51 +03:00
Andrey Vasnetsov
42e8576dac raise 404 error instead of 400 in case if collection not found #99 2021-09-25 14:46:04 +02:00
Konstantin
20d454e8e0 Exposed update_colletions operation over grpc (#89) 2021-09-05 14:17:40 +02:00
Andrey Vasnetsov
dd5694103f Fix point retrieve from copy-on-write proxy segment (#94) 2021-08-29 23:32:07 +02:00
Andrey Vasnetsov
6cab7f2a0b dynamic arch (#79)
* dynamic arch

* fix fmt
2021-08-29 23:31:55 +02:00
Andrey Vasnetsov
083455bbbe handle limit=0 corner case in scroll api #90 2021-08-22 22:18:56 +02:00
Alexander Galibey
2cbe1d4f6b Decouple searcher and updater from collection (#93) 2021-08-22 22:11:00 +02:00
Andrey Vasnetsov
53ed637137 skip moved points (double) processing for segments updater in apply_points_to_appendable 2021-08-18 00:37:19 +02:00
Andrey Vasnetsov
446d0c29f7 Deadlock fix (#91)
* refactor: segment managers -> collection managers

* fix segments holder deadlock

* apply cargo fmt

* fix cargo clippy

* replace sequential segment locking with multiple try_lock attempts to prevent deadlocks
2021-08-15 23:26:01 +02:00
Alexander Galibey
f3e8194310 Remove AtomicRefCell wrapper for condition checker (#84) 2021-08-03 10:35:55 +02:00
Andrey Vasnetsov
0bd0a1da42 fix new clippy suggestions 2021-07-30 23:49:02 +02:00
Daniil Naumetc
dd304495f5 Small cosmetics (#66)
* some small cosmetics

* fixed "expect" linting

* fix formation
2021-07-15 15:25:44 +02:00
Andrey Vasnetsov
2bfcac726e disable debug test 2021-07-11 21:54:49 +02:00
Andrey Vasnetsov
d5d8d2c407 fix serde deserialisation issue in WAL 2021-07-11 21:22:22 +02:00
Konstantin G
f70be9bb35 Own search runtime out of the async scope 2021-07-05 23:07:45 +01:00
Andrey Vasnetsov
910c3b9f8a Revert "Revert "Actix update (#55)" (#56)" (#57)
This reverts commit 53ddce350e.
2021-07-05 23:38:00 +02:00
Andrey Vasnetsov
53ddce350e Revert "Actix update (#55)" (#56)
This reverts commit 12e25089cb.
2021-07-05 00:43:23 +02:00