Commit Graph

37 Commits

Author SHA1 Message Date
dependabot[bot]
c21855551f Bump tokio from 1.18.2 to 1.19.2 (#664)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.18.2 to 1.19.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/commits)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-06 21:43:35 +02:00
Gabriel Velo
c15981092a [WIP] [real-time index] Implement payloadstorage for structpayloadindex (#642)
* [real-time index] Extend FieldIndex enum and StructPayloadIndex with method from PayloadStorage

* [real-time index] add missing remove_point methods

* [real-time index] add new index to FieldIndex enum

* fix compile

* are you happy fmt

* merge load and remove

* fix test generics

* decrement points count

* remove from histogram

* simplify histogram usage

* [real-time index] remove old tests and fix clippy warnings

* histogram: method to derive range by size (#657)

* [real-time index] add histogram based payload_blocks implementation.

* payload blocks

* fmt

* clippy

* [real-time index] refactor Segment to use PayloadIndex instead of PayloadStorage.

* fix tests

* fmt

* clippy

* rename indexes

* remove redundent params

* add struct payload deletion test + fix delete payload in map index

* remove payload threshold

Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2022-06-06 17:14:20 +02:00
Andrey Vasnetsov
f92535aa39 update qdrant.rs 2022-06-02 19:15:35 +02:00
Andrey Vasnetsov
7c254a0de0 preserve fields order in OpenAPI schema (#655) 2022-06-02 19:12:30 +02:00
Andrey Vasnetsov
2601c017de Smarter defaults (#637)
* auto segments number

* auto segments number

* replace vector number limits with vector size limits

* fmt
2022-06-01 18:09:38 +02:00
Ivan Pleshkov
708a5b211a fix handwrited comment in qdrant.rs (#653) 2022-06-01 17:45:14 +02:00
Andrey Vasnetsov
1b458780eb On disk payload storage (#634)
* implement on-disk payload storage

* fmt + clippy

* config param for on-disk payload storage

* upd openapi definitions

* add integration test with on-disk payload

* fix clippy

* review fixes

* fmt
2022-06-01 17:23:34 +02:00
Andrey Vasnetsov
3d505f5ae5 gRPC int payload (#564)
* use custom message for json representation in gRPC

* fmt
2022-05-30 19:10:37 +02:00
dependabot[bot]
ab975486d0 Bump uuid from 1.0.0 to 1.1.0 (#630)
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.0.0 to 1.1.0.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/1.0.0...1.1.0)

---
updated-dependencies:
- dependency-name: uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-25 20:43:36 +02:00
dependabot[bot]
9543e8c733 Bump schemars from 0.8.9 to 0.8.10 (#602)
Bumps [schemars](https://github.com/GREsau/schemars) from 0.8.9 to 0.8.10.
- [Release notes](https://github.com/GREsau/schemars/releases)
- [Changelog](https://github.com/GREsau/schemars/blob/master/CHANGELOG.md)
- [Commits](https://github.com/GREsau/schemars/compare/v0.8.9...v0.8.10)

---
updated-dependencies:
- dependency-name: schemars
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-17 21:59:05 +02:00
dependabot[bot]
6ec5ff0596 Bump uuid from 0.8.2 to 1.0.0 (#499)
* Bump uuid from 0.8.2 to 1.0.0

Bumps [uuid](https://github.com/uuid-rs/uuid) from 0.8.2 to 1.0.0.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/0.8.2...1.0.0)

---
updated-dependencies:
- dependency-name: uuid
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* schemars 0.8.9 with uuid1 support

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
2022-05-17 10:27:31 +02:00
Arnaud Gourlay
0a33e00647 Pool gRPC transport channels (#584)
* Pool gRPC transport channels

* code review: do not use an extra layer of Arc

* code review: better naming
2022-05-13 14:44:06 +02:00
Andrey Vasnetsov
5800319edb refactor distances + add score_threshold + fix negative euclid distance (#569)
* refactor distances + add score_threshold + fix negative euclid distance

* generate docs

* fix clippy

* Update lib/segment/src/spaces/tools.rs

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
2022-05-09 17:32:48 +02:00
dependabot[bot]
cfcff554fb Bump tonic from 0.7.1 to 0.7.2 (#563)
Bumps [tonic](https://github.com/hyperium/tonic) from 0.7.1 to 0.7.2.
- [Release notes](https://github.com/hyperium/tonic/releases)
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/tonic/compare/v0.7.1...v0.7.2)

---
updated-dependencies:
- dependency-name: tonic
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-05 21:12:12 +02:00
dependabot[bot]
71b3c922d1 Bump tonic-build from 0.7.1 to 0.7.2 (#562)
Bumps [tonic-build](https://github.com/hyperium/tonic) from 0.7.1 to 0.7.2.
- [Release notes](https://github.com/hyperium/tonic/releases)
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/tonic/compare/v0.7.1...v0.7.2)

---
updated-dependencies:
- dependency-name: tonic-build
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-05 20:55:05 +02:00
Egor Ivkov
f3fc921e51 Derive this peer uri if not supplied (#546) 2022-05-03 16:17:06 +03:00
Egor Ivkov
40c4c1ec2c Allows bootstrapping peer on addition (#529)
* Allows bootstrapping peer on addition

* Review: warn on address replacement

* Review: more detailed comments
2022-05-02 18:02:35 +03:00
Arnaud Gourlay
6f939648d1 [sharding] Remote shard operations via gRPC (#505)
* Remote shard operations via gRPC

* wire up properly the peer id address lookup

* remove unreachable_code by negating config. predicate

* TryFrom i32 for CollectionStatus

* split conversion code to improve readability

* collect HashMap results

* optimize imports

* extract request building code for readability

* ip_to_address uses Uri

* ip_to_address is encapsulated in the RemoteShard
2022-04-29 19:40:55 +02:00
dependabot[bot]
681d3d33e6 Bump tonic-build from 0.7.0 to 0.7.1 (#520)
Bumps [tonic-build](https://github.com/hyperium/tonic) from 0.7.0 to 0.7.1.
- [Release notes](https://github.com/hyperium/tonic/releases)
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/tonic/compare/v0.7.0...v0.7.1)

---
updated-dependencies:
- dependency-name: tonic-build
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-28 21:05:03 +02:00
Egor Ivkov
d4bb6e4f49 Send Raft messages with gRPC client (#509)
* Send Raft messages with gRPC client

* Review: more specific thread names
2022-04-27 13:01:41 +03:00
Andrey Vasnetsov
61e6a1872c ensure lib/api version is same as main package (#504) 2022-04-25 10:10:09 +02:00
Egor Ivkov
77b7e666be Add gRPC RaftService (#490)
* Add gRPC `RaftService`

* Review: Back-pressure with bounded channel
2022-04-20 14:23:26 +03:00
Egor Ivkov
9885628f0a Remove java related proto3 options (#485)
As the project does not use java code generation
2022-04-14 18:39:52 +03:00
dependabot[bot]
bd8ad015c8 Bump prost-types from 0.10.0 to 0.10.1 (#479)
* Bump prost-types from 0.10.0 to 0.10.1

Bumps [prost-types](https://github.com/tokio-rs/prost) from 0.10.0 to 0.10.1.
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](https://github.com/tokio-rs/prost/compare/v0.10.0...v0.10.1)

---
updated-dependencies:
- dependency-name: prost-types
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* prost 0.10.1

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
2022-04-14 11:17:02 +02:00
Arnaud Gourlay
ca82a80bd3 Internal gRPC API for get collection info (#476) 2022-04-13 19:45:34 +02:00
Arnaud Gourlay
62d4273ed2 [Sharding] internal API for shard-aware points retrieval (#475) 2022-04-13 14:10:15 +02:00
Arnaud Gourlay
367afa3866 [sharding] Add internal gRPC API for search recommend & scroll (#463)
* Add internal gRPC API for search recommend & scroll

* introduce BadShardSelection which translates into a client side error

* introduce Collection target_shards to decrease code duplication

* recommend uses a shard_selection aware retrieve
2022-04-13 13:16:53 +02:00
Arnaud Gourlay
b77482702a add internal API for CollectionUpdateOperations (#445) 2022-04-11 11:27:44 +02:00
Egor Ivkov
ee4cd768ea Allows specifying collection meta operation commit timeout in API (#444)
* Allows specifying collection meta operation commit timeout in API

Also restructures a part of gRPC collection API

* Review: link to dev docs

* Review: Timeout error description

Also committed generated protobuf code that somehow was left out

* Review: seconds in API and minor proto fixes
2022-04-08 14:42:38 +03:00
Arnaud Gourlay
36ad5ffa80 do not format output of tonic as it is now using prettyplease (#451) 2022-04-08 10:41:52 +02:00
dependabot[bot]
5403fc13a2 Bump tonic from 0.7.0 to 0.7.1 (#441)
Bumps [tonic](https://github.com/hyperium/tonic) from 0.7.0 to 0.7.1.
- [Release notes](https://github.com/hyperium/tonic/releases)
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/tonic/compare/v0.7.0...v0.7.1)

---
updated-dependencies:
- dependency-name: tonic
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-07 10:15:53 +02:00
Andrey Vasnetsov
ce21abf033 Values count condition (#439)
* add values_count condition

* fmt

* fix tests and clippy

* fmt
2022-04-05 14:31:53 +02:00
Arnaud Gourlay
ded49e505c [Sharding] Shard aware updates for internal API (#427)
* Shard aware updates for internal API

* code-review: remove general purpose collection update API

* code review: add clarification comment
2022-04-04 11:25:43 +02:00
Andrey Vasnetsov
b07428f620 Is empty condition (#423)
* is-empty condition

* fmt

* better assert

* fmt
2022-04-03 16:55:51 +02:00
Andrey Vasnetsov
adc1f4ad97 Bool filter (#421)
* bool match condition

* use generic values for match requests

* fmt

* upd grpc interface

* upd grpc docs
2022-04-03 16:08:34 +02:00
dependabot[bot]
c2d4a9927f Bump tonic from 0.6.2 to 0.7.0 (#432)
* Bump tonic from 0.6.2 to 0.7.0

Bumps [tonic](https://github.com/hyperium/tonic) from 0.6.2 to 0.7.0.
- [Release notes](https://github.com/hyperium/tonic/releases)
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/tonic/compare/v0.6.2...v0.7.0)

---
updated-dependencies:
- dependency-name: tonic
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* tonic-build 0.7.0

* prost 0.10.0

* prost 0.7.0 for storage because of raft dependency

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
2022-04-01 22:50:35 +02:00
Arnaud Gourlay
967e265900 [sharding] Extract grpc feature into sub-crate (#412)
* extract grpc feature into sub-crate

* move build.rs to api crate

* split proto files between services and objects

* update docs
2022-03-30 10:12:02 +02:00