upstream-archive/qdrant
1
0
Fork 0
mirror of https://github.com/qdrant/qdrant.git synced 2026-08-06 10:00:58 -05:00
Code Issues Packages Projects Releases Wiki Activity
631 Commits 344 Branches 118 Tags
ce240decacfa115fefb01d2f6a3d8ccf46feeda8
Commit Graph

138 Commits

Author SHA1 Message Date
Arnaud Gourlay
96413485b6 the gRPC bootstrap timeout needs to be higher than DEFAULT_META_OP_WAIT (#678) 2022-06-13 09:18:50 +02:00
Arnaud Gourlay
1d25b22f5f add more error context for consenus init (#677) 2022-06-10 14:15:37 +02:00
Andrey Vasnetsov
fc2f579ec6 smarter default number of shards (#668) 
* smarter default number of shards

* upd openapi
 2022-06-07 16:59:10 +02:00
Andrey Vasnetsov
8e52c255ab add consensus configuration + lower log level + fix flickering test 2022-06-06 21:44:03 +02:00
Egor Ivkov
9ddaa1c749 Add leader_id and role to cluster API (#660) 2022-06-03 22:37:50 +02:00
Egor Ivkov
88d14732ea Wait for leader to be established (#658) 
* Wait for leader to be established

* use condwar instead of channel (#659)

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-06-03 15:58:07 +03:00
Andrey Vasnetsov
1b458780eb On disk payload storage (#634) 
* implement on-disk payload storage

* fmt + clippy

* config param for on-disk payload storage

* upd openapi definitions

* add integration test with on-disk payload

* fix clippy

* review fixes

* fmt
 2022-06-01 17:23:34 +02:00
Egor Ivkov
4b83464061 Skip consensus init on restart (#638) 
* Skip consensus init on restart

* Added test

* CI
 2022-06-01 11:38:23 +03:00
Andrey Vasnetsov
c286bde562 Cluster api (#631) 
* cluster status API

* add cluster api to OpenAPI specification
 2022-05-27 14:52:51 +02:00
Arnaud Gourlay
63a1552cb1 [sharding] Materialize remote shards (#599) 
* materialize remote shards at collection creation and load

* code review naming

* use dedicated integration test

* introduce ChannelService

* introduce CreateCollectionDistributed

* fix test name

* create missing remote shards while applying the SnapshotData
 2022-05-24 15:42:04 +02:00
Andrey Vasnetsov
5a4c67ce2c fix cros headers - allow all requests (#619) 2022-05-23 15:55:35 +02:00
Andrey Vasnetsov
036c186e0d Better error reporting in enums (#587) 
* remove conflicting deprecated apis

* implement custom JsonSchema for the PointInsertOperations

* fmt

* clippy

* revert unnesessare changes
 2022-05-23 13:28:17 +02:00
Andrey Vasnetsov
e3562493c2 add optional CORS headers (#611) 2022-05-19 17:56:17 +02:00
Egor Ivkov
0cf8c070f8 Log only h2 errors (#608) 2022-05-19 12:35:21 +03:00
Egor Ivkov
27440937bc Add multipeer deployment test (#590) 
* Add multipeer deployment test

And corresponding fixes for snapshot and wal logic to make test green

* Add CI for multipeer test

* Increase number of peers to 5

* Fix pytest installation

* Fix python version

* Install requests lib
 2022-05-18 15:22:38 +03:00
Arnaud Gourlay
0a33e00647 Pool gRPC transport channels (#584) 
* Pool gRPC transport channels

* code review: do not use an extra layer of Arc

* code review: better naming
 2022-05-13 14:44:06 +02:00
Egor Ivkov
75f424a5ef Raft configuration updates - add peer (#578) 
* Raft configuration updates - add peer

* Review: clarify send error

* Review: move `first_peer` into `ConsensusEnabled`

* Fix consensus test
 2022-05-11 17:10:18 +03:00
Andrey Vasnetsov
7ca14af967 default values for debug and info configuration params (#577) 2022-05-10 17:00:31 +02:00
Andrey Vasnetsov
5800319edb refactor distances + add score_threshold + fix negative euclid distance (#569) 
* refactor distances + add score_threshold + fix negative euclid distance

* generate docs

* fix clippy

* Update lib/segment/src/spaces/tools.rs

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
 2022-05-09 17:32:48 +02:00
Arnaud Gourlay
c7396525f9 Switch consensus to a runtime configuration (#565) 
* switch consensus to a runtime configuration

* propose_sender is always present

* better toggle switch with ConsensusEnabled
 2022-05-06 17:56:00 +02:00
Arnaud Gourlay
841ba05850 [Sharding] Introduce cluster configuration (#547) 
* cluster configuration

* default cluster configuration to not pollute config.yaml

* code review suggestions

* please clippy
 2022-05-04 20:22:45 +02:00
Arnaud Gourlay
1f72ce6b1f return a JSON body in case of JSON decoding error (#556) 2022-05-04 20:11:27 +02:00
Egor Ivkov
f3fc921e51 Derive this peer uri if not supplied (#546) 2022-05-03 16:17:06 +03:00
Egor Ivkov
40c4c1ec2c Allows bootstrapping peer on addition (#529) 
* Allows bootstrapping peer on addition

* Review: warn on address replacement

* Review: more detailed comments
 2022-05-02 18:02:35 +03:00
Egor Ivkov
d4bb6e4f49 Send Raft messages with gRPC client (#509) 
* Send Raft messages with gRPC client

* Review: more specific thread names
 2022-04-27 13:01:41 +03:00
Egor Ivkov
fd1b8a457c Adds this_peer_id generation as part of raft state (#503) 
Also some raft state refactoring
 2022-04-25 12:14:01 +03:00
Andrey Vasnetsov
61e6a1872c ensure lib/api version is same as main package (#504) 2022-04-25 10:10:09 +02:00
Egor Ivkov
77b7e666be Add gRPC RaftService (#490) 
* Add gRPC `RaftService`

* Review: Back-pressure with bounded channel
 2022-04-20 14:23:26 +03:00
Egor Ivkov
ca62c734e4 Save unapplied committed entries indexes (#477) 2022-04-14 11:05:20 +03:00
Arnaud Gourlay
ca82a80bd3 Internal gRPC API for get collection info (#476) 2022-04-13 19:45:34 +02:00
Arnaud Gourlay
62d4273ed2 [Sharding] internal API for shard-aware points retrieval (#475) 2022-04-13 14:10:15 +02:00
Arnaud Gourlay
367afa3866 [sharding] Add internal gRPC API for search recommend & scroll (#463) 
* Add internal gRPC API for search recommend & scroll

* introduce BadShardSelection which translates into a client side error

* introduce Collection target_shards to decrease code duplication

* recommend uses a shard_selection aware retrieve
 2022-04-13 13:16:53 +02:00
Arnaud Gourlay
74a1eb8294 p2p gRPC API is now internal (#464) 2022-04-12 11:53:02 +02:00
Arnaud Gourlay
b77482702a add internal API for CollectionUpdateOperations (#445) 2022-04-11 11:27:44 +02:00
Egor Ivkov
ee4cd768ea Allows specifying collection meta operation commit timeout in API (#444) 
* Allows specifying collection meta operation commit timeout in API

Also restructures a part of gRPC collection API

* Review: link to dev docs

* Review: Timeout error description

Also committed generated protobuf code that somehow was left out

* Review: seconds in API and minor proto fixes
 2022-04-08 14:42:38 +03:00
Egor Ivkov
aeea31dfa4 Wait for operation commit (#436) 
* Wait for operation commit

* Review: Option with timeout, return execution err to API

* Review: Add comment about error forwarding in ???
 2022-04-06 18:00:44 +03:00
Arnaud Gourlay
ded49e505c [Sharding] Shard aware updates for internal API (#427) 
* Shard aware updates for internal API

* code-review: remove general purpose collection update API

* code review: add clarification comment
 2022-04-04 11:25:43 +02:00
Egor Ivkov
d3e0d7dcec Integrate consensus into execution pipeline (#410) 
* Integrate consensus into execution pipeline

* Review: Remove unnecessary config
 2022-03-31 17:54:55 +03:00
Arnaud Gourlay
967e265900 [sharding] Extract grpc feature into sub-crate (#412) 
* extract grpc feature into sub-crate

* move build.rs to api crate

* split proto files between services and objects

* update docs
 2022-03-30 10:12:02 +02:00
Andrey Vasnetsov
8dcb900113 fix intergation test: freeze python deps on working versions 2022-03-28 23:56:08 +02:00
Arnaud Gourlay
a0e52e84aa alias gRPC endpoint (#408) 2022-03-24 16:21:01 +01:00
Egor Ivkov
2343078ea1 Collection meta as global state (#394) 
* Collection meta as global state

* Improve collection wal entry error handling
 2022-03-22 18:27:03 +03:00
Arnaud Gourlay
de741b9c84 add untracked generated file google.protobuf.rs (#396) 2022-03-21 11:57:35 +01:00
Gabriel Velo
f69a7b740f json as payload (#306) 
add json as payload
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-03-21 07:09:10 -03:00
Egor Ivkov
dcfb44de6f Consensus with Raft feature (#385) 
Consensus will be used to synchronize global state which is:
- Collection meta
- Shard to Peer mapping
 2022-03-17 15:15:06 +03:00
Egor Ivkov
a5bd04f446 Remove unused cli.rs (#383) 2022-03-14 18:47:47 +03:00
Egor Ivkov
960937b58b Sharding related collection API changes (#381) 
* NonZeroU32 shard number

* Inconsistent Failure option

* Generate docs and apis
 2022-03-11 19:26:09 +03:00
Egor Ivkov
2bb52abc71 Multiple shards in collection (#360) 
* Multiple shards in collection

* Collection and Shard async drop

* Generate APIs for shard_number param
 2022-03-07 16:22:47 +03:00
Andrey Vasnetsov
5cf9a8c21b [gRPC] recommendation method (#364) 
* grpc: recommend points

* grpc:recommend docs
 2022-03-03 20:39:07 +01:00
Arnaud Gourlay
c928217a8c enable gRPC via a runtime config (#352) 
* enable gRPC via a runtime config

* code review

* code review: rename rest_port to http_port

* Update config/config.yaml

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>

* disable grpc by default

* enable gPRC in integration test

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-03-03 19:23:31 +01:00