upstream-archive/qdrant
1
0
Fork 0
mirror of https://github.com/qdrant/qdrant.git synced 2026-08-07 02:20:55 -05:00
Code Issues Packages Projects Releases Wiki Activity
5,465 Commits 346 Branches 118 Tags
94de0e2d2dcf997bf26bec34dad407d8de9a9abb
Commit Graph

3618 Commits

Author SHA1 Message Date
Andrey Vasnetsov
7c254a0de0 preserve fields order in OpenAPI schema (#655) 2022-06-02 19:12:30 +02:00
Andrey Vasnetsov
2601c017de Smarter defaults (#637) 
* auto segments number

* auto segments number

* replace vector number limits with vector size limits

* fmt
 2022-06-01 18:09:38 +02:00
Ivan Pleshkov
708a5b211a fix handwrited comment in qdrant.rs (#653) 2022-06-01 17:45:14 +02:00
Andrey Vasnetsov
1b458780eb On disk payload storage (#634) 
* implement on-disk payload storage

* fmt + clippy

* config param for on-disk payload storage

* upd openapi definitions

* add integration test with on-disk payload

* fix clippy

* review fixes

* fmt
 2022-06-01 17:23:34 +02:00
Andrey Vasnetsov
c45078686f refactor histogram: remove cell (#651) 2022-06-01 14:50:22 +04:00
Egor Ivkov
4b83464061 Skip consensus init on restart (#638) 
* Skip consensus init on restart

* Added test

* CI
 2022-06-01 11:38:23 +03:00
Andrey Vasnetsov
25e1f2fc92 Remove method for histogram (#647) 
* add remove method for histogram

* fmt + clippy
 2022-05-31 18:35:21 +02:00
Andrey Vasnetsov
54a66844c5 add version to the segment (#640) 
* add version to the segment

* add collection version saving

* Update lib/collection/src/lib.rs

Co-authored-by: Egor Ivkov <e.o.ivkov@gmail.com>

* Update lib/segment/src/segment.rs

Co-authored-by: Egor Ivkov <e.o.ivkov@gmail.com>

* remove self

* add comment

Co-authored-by: Egor Ivkov <e.o.ivkov@gmail.com>
 2022-05-31 14:16:49 +02:00
Ivan Pleshkov
8968a9f437 Integration of improved histogram (#643) 
* integration of improved histogram

* use stat tools for estimation
 2022-05-31 14:27:15 +04:00
Andrey Vasnetsov
3d505f5ae5 gRPC int payload (#564) 
* use custom message for json representation in gRPC

* fmt
 2022-05-30 19:10:37 +02:00
Andrey Vasnetsov
a94933c4ea New improved hisogram (#636) 
* fixes for histogram

* fmt

* remove obsolete function

* remove obsolete check

* fix bugs

Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>
 2022-05-30 18:59:01 +02:00
Arnaud Gourlay
135fec9c6c [sharding] Persist shard information (#628) 
* persist shard information

* create default local shard config file for existing shards

* minor review fix

* fmt + clippy

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-05-30 13:38:12 +02:00
Andrey Vasnetsov
c286bde562 Cluster api (#631) 
* cluster status API

* add cluster api to OpenAPI specification
 2022-05-27 14:52:51 +02:00
Ivan Pleshkov
2581a2e3f2 arm unused const warning (#633) 2022-05-27 16:34:00 +04:00
dependabot[bot]
ab975486d0 Bump uuid from 1.0.0 to 1.1.0 (#630) 
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.0.0 to 1.1.0.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/1.0.0...1.1.0)

---
updated-dependencies:
- dependency-name: uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-05-25 20:43:36 +02:00
Gabriel Velo
493b7381b7 [real-time index] Extend PayloadFieldIndex trait with load, update and flush (#625) 
[real-time index] Extend PayloadFieldIndex trait with load, update and flush methods
 2022-05-25 11:25:40 -03:00
Gabriel Velo
6852ecb4c9 [real-time index] add OnDIsk persistence layer for geo index (#620) 
[real-time index] add OnDIsk persistence layer for geo index
 2022-05-24 11:11:30 -03:00
Arnaud Gourlay
63a1552cb1 [sharding] Materialize remote shards (#599) 
* materialize remote shards at collection creation and load

* code review naming

* use dedicated integration test

* introduce ChannelService

* introduce CreateCollectionDistributed

* fix test name

* create missing remote shards while applying the SnapshotData
 2022-05-24 15:42:04 +02:00
Andrey Vasnetsov
036c186e0d Better error reporting in enums (#587) 
* remove conflicting deprecated apis

* implement custom JsonSchema for the PointInsertOperations

* fmt

* clippy

* revert unnesessare changes
 2022-05-23 13:28:17 +02:00
Ivan Pleshkov
519caaaa08 [real-time index] OnDisk persistence layer for keyword index (#604) 2022-05-20 19:26:58 +04:00
Egor Ivkov
08b8de8692 Fix deadlock on collection creation from snapshot (#615) 2022-05-20 18:05:43 +03:00
Egor Ivkov
8f3f7ca336 Remove serialization wrappers (#612) 2022-05-20 14:03:24 +03:00
Gabriel Velo
8fd4a94934 [real-time index] add write to and load from rocksdb instance. (#601) 
* [real-time index] add write to and load from rocksdb instance.

* [real-time index] add flush method

* [real-time index] fix error handling in load()

* [real-time index] add TODO tag to remember to remove result unwrap when we add error propagation to ValueIndexer trait.
 2022-05-20 12:54:29 +04:00
Egor Ivkov
27440937bc Add multipeer deployment test (#590) 
* Add multipeer deployment test

And corresponding fixes for snapshot and wal logic to make test green

* Add CI for multipeer test

* Increase number of peers to 5

* Fix pytest installation

* Fix python version

* Install requests lib
 2022-05-18 15:22:38 +03:00
dependabot[bot]
9543e8c733 Bump schemars from 0.8.9 to 0.8.10 (#602) 
Bumps [schemars](https://github.com/GREsau/schemars) from 0.8.9 to 0.8.10.
- [Release notes](https://github.com/GREsau/schemars/releases)
- [Changelog](https://github.com/GREsau/schemars/blob/master/CHANGELOG.md)
- [Commits](https://github.com/GREsau/schemars/compare/v0.8.9...v0.8.10)

---
updated-dependencies:
- dependency-name: schemars
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-05-17 21:59:05 +02:00
dependabot[bot]
6ec5ff0596 Bump uuid from 0.8.2 to 1.0.0 (#499) 
* Bump uuid from 0.8.2 to 1.0.0

Bumps [uuid](https://github.com/uuid-rs/uuid) from 0.8.2 to 1.0.0.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/0.8.2...1.0.0)

---
updated-dependencies:
- dependency-name: uuid
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* schemars 0.8.9 with uuid1 support

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
 2022-05-17 10:27:31 +02:00
Andrey Vasnetsov
21dfda5d32 Parallel segments creation (#598) 
* use threads to create segments in parallel

* fmt

* try join all threads before reporting error

* fmt
 2022-05-16 18:29:15 +02:00
Gabriel Velo
3dcf6f6842 [real-time index] add In-mem btree numeric index (#588) 2022-05-16 10:20:00 -03:00
Ivan Pleshkov
0e9adbda44 async create_dir_all (#596) 2022-05-16 14:08:12 +04:00
Ivan Pleshkov
494e64bfc8 async create collection path (#597) 2022-05-16 14:08:01 +04:00
Ivan Pleshkov
effb79733d Unite rocksdb of segment (#585) 
unite rocksdb for segment
 2022-05-13 17:06:07 +04:00
Arnaud Gourlay
0a33e00647 Pool gRPC transport channels (#584) 
* Pool gRPC transport channels

* code review: do not use an extra layer of Arc

* code review: better naming
 2022-05-13 14:44:06 +02:00
Egor Ivkov
75f424a5ef Raft configuration updates - add peer (#578) 
* Raft configuration updates - add peer

* Review: clarify send error

* Review: move `first_peer` into `ConsensusEnabled`

* Fix consensus test
 2022-05-11 17:10:18 +03:00
Ivan Pleshkov
a402b0ad84 Disable wal for rocksdb (#576) 
rocksdb disable wal
 2022-05-11 14:22:09 +04:00
dependabot[bot]
e277f1c121 Bump pprof from 0.8.0 to 0.9.0 (#579) 
Bumps [pprof](https://github.com/tikv/pprof-rs) from 0.8.0 to 0.9.0.
- [Release notes](https://github.com/tikv/pprof-rs/releases)
- [Changelog](https://github.com/tikv/pprof-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tikv/pprof-rs/commits)

---
updated-dependencies:
- dependency-name: pprof
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-05-10 20:50:20 +02:00
Ivan Pleshkov
e5f83bf479 rocksdb limited logs and common settings (#571) 2022-05-10 10:20:28 +04:00
Gabriel Velo
59ddb81330 add key encoding for numeric index. (#570) 
add key encoding for numeric index.
 2022-05-09 14:43:57 -03:00
Andrey Vasnetsov
5800319edb refactor distances + add score_threshold + fix negative euclid distance (#569) 
* refactor distances + add score_threshold + fix negative euclid distance

* generate docs

* fix clippy

* Update lib/segment/src/spaces/tools.rs

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
 2022-05-09 17:32:48 +02:00
Arnaud Gourlay
c7396525f9 Switch consensus to a runtime configuration (#565) 
* switch consensus to a runtime configuration

* propose_sender is always present

* better toggle switch with ConsensusEnabled
 2022-05-06 17:56:00 +02:00
dependabot[bot]
cfcff554fb Bump tonic from 0.7.1 to 0.7.2 (#563) 
Bumps [tonic](https://github.com/hyperium/tonic) from 0.7.1 to 0.7.2.
- [Release notes](https://github.com/hyperium/tonic/releases)
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/tonic/compare/v0.7.1...v0.7.2)

---
updated-dependencies:
- dependency-name: tonic
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-05-05 21:12:12 +02:00
dependabot[bot]
71b3c922d1 Bump tonic-build from 0.7.1 to 0.7.2 (#562) 
Bumps [tonic-build](https://github.com/hyperium/tonic) from 0.7.1 to 0.7.2.
- [Release notes](https://github.com/hyperium/tonic/releases)
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/tonic/compare/v0.7.1...v0.7.2)

---
updated-dependencies:
- dependency-name: tonic-build
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-05-05 20:55:05 +02:00
Andrey Vasnetsov
30940c3989 wip: histogram PoC + some chore (#533) 2022-05-05 14:32:00 +02:00
Arnaud Gourlay
2ff03c20fd fix typo in openapi spec (#558) 2022-05-05 07:39:57 +02:00
Arnaud Gourlay
841ba05850 [Sharding] Introduce cluster configuration (#547) 
* cluster configuration

* default cluster configuration to not pollute config.yaml

* code review suggestions

* please clippy
 2022-05-04 20:22:45 +02:00
dependabot[bot]
e2f309f878 Bump num-traits from 0.2.14 to 0.2.15 (#548) 
Bumps [num-traits](https://github.com/rust-num/num-traits) from 0.2.14 to 0.2.15.
- [Release notes](https://github.com/rust-num/num-traits/releases)
- [Changelog](https://github.com/rust-num/num-traits/blob/master/RELEASES.md)
- [Commits](https://github.com/rust-num/num-traits/compare/num-traits-0.2.14...num-traits-0.2.15)

---
updated-dependencies:
- dependency-name: num-traits
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-05-04 08:48:18 +02:00
Egor Ivkov
f3fc921e51 Derive this peer uri if not supplied (#546) 2022-05-03 16:17:06 +03:00
dependabot[bot]
ff8bf5ddf7 Bump ordered-float from 2.10.0 to 3.0.0 (#539) 
Bumps [ordered-float](https://github.com/reem/rust-ordered-float) from 2.10.0 to 3.0.0.
- [Release notes](https://github.com/reem/rust-ordered-float/releases)
- [Commits](https://github.com/reem/rust-ordered-float/compare/v2.10.0...v3.0.0)

---
updated-dependencies:
- dependency-name: ordered-float
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-05-03 11:16:38 +02:00
Egor Ivkov
40c4c1ec2c Allows bootstrapping peer on addition (#529) 
* Allows bootstrapping peer on addition

* Review: warn on address replacement

* Review: more detailed comments
 2022-05-02 18:02:35 +03:00
Arnaud Gourlay
794384f301 document flaky test and add more info in case of failure (#535) 2022-05-02 15:23:56 +02:00
Arnaud Gourlay
6f939648d1 [sharding] Remote shard operations via gRPC (#505) 
* Remote shard operations via gRPC

* wire up properly the peer id address lookup

* remove unreachable_code by negating config. predicate

* TryFrom i32 for CollectionStatus

* split conversion code to improve readability

* collect HashMap results

* optimize imports

* extract request building code for readability

* ip_to_address uses Uri

* ip_to_address is encapsulated in the RemoteShard
 2022-04-29 19:40:55 +02:00