Commit Graph

126 Commits

Author SHA1 Message Date
dependabot[bot]
d6e6d75ff1 Bump schemars from 0.8.11 to 0.8.12 (#1505)
Bumps [schemars](https://github.com/GREsau/schemars) from 0.8.11 to 0.8.12.
- [Release notes](https://github.com/GREsau/schemars/releases)
- [Changelog](https://github.com/GREsau/schemars/blob/master/CHANGELOG.md)
- [Commits](https://github.com/GREsau/schemars/compare/v0.8.11...v0.8.12)

---
updated-dependencies:
- dependency-name: schemars
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-28 09:51:03 +01:00
dependabot[bot]
5acc496462 Bump prost from 0.11.7 to 0.11.8 (#1504)
Bumps [prost](https://github.com/tokio-rs/prost) from 0.11.7 to 0.11.8.
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](https://github.com/tokio-rs/prost/commits/v0.11.8)

---
updated-dependencies:
- dependency-name: prost
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-28 09:49:26 +01:00
dependabot[bot]
6840df3de1 Bump prost from 0.11.6 to 0.11.7 (#1495)
Bumps [prost](https://github.com/tokio-rs/prost) from 0.11.6 to 0.11.7.
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](https://github.com/tokio-rs/prost/commits)

---
updated-dependencies:
- dependency-name: prost
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-23 21:18:10 +01:00
Luis Cossío
c15fec2a2a feat: add "any" to "match" statements (#1466)
* feat: add "any" to "match" statements

Adds support for using "any" to match on a list of different possible values

* enable proper counting in cardinality estimation

* fmt

* upd OpenAPI

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2023-02-15 11:11:20 +01:00
Andrey Vasnetsov
99eaabc31a remove deprecate point ids param in gprc (#1305)
* remove deprecate point ids param in gprc

* upd docs
2023-02-06 13:37:18 +01:00
Roman Titov
a50e7bdcd4 Add read_consistency parameter to the APIs (#1371) (#1407)
* WIP: Add `read_consistency` parameter to the APIs

* WIP: Add `read_consistency` parameter to the APIs

TODO:
- Add documentation

* `cargo fmt`

* Add gRPC documentation

* Add OpenAPI documentation

* Cleanup

* fixup! Add OpenAPI documentation

* fixup! Add gRPC documentation

Who would have known there's `generate_grpc_docs.sh`!? 🥲🙈🤦‍♀️

* generate openapi

* Fix `read_consistency` query parameter deserialization

* Further improve `read_consistency` query parameter deserialization

* `cargo clippy`

* Fix `Payload` comparison during read operation result resolving

* Fix grammar

* rename `read_consistency` -> `consistency` and add integration test

* use majority for test

* fix tests

* Fix tests

* fixup! Fix tests

Apply the same fix to `ScoredPoint`

* Remove an `unwrap`

* fixup! Fix tests

Gotta love those negative conditions, or how a missed `!` can ruin your day... 🤦‍♀️

* Make internal API calls strictly "local-shard only"

* Implement a few basic traits for `ResolverRecord`

* fixup! Implement a few basic traits for `ResolverRecord`

* Revert "Make internal API calls strictly "local-shard only""

This reverts commit 25378e61ff.

* Fix `Record::payload` and `ScoredPoint::payload` serialization

* Revert "Fix `Record::payload` and `ScoredPoint::payload` serialization"

This reverts commit b566bea49b.

* Fix `Record::payload` and `ScoredPoint::payload` visibility

* fixup! Fix `Record::payload` and `ScoredPoint::payload` visibility

Remove `todo!()`

* refactoring

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2023-02-03 01:12:28 +01:00
dependabot[bot]
7abb9754b3 Bump uuid from 1.2.2 to 1.3.0 (#1420)
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.2.2 to 1.3.0.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/1.2.2...1.3.0)

---
updated-dependencies:
- dependency-name: uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-01 19:59:41 +01:00
dependabot[bot]
efc51edbb8 Bump tokio from 1.24.2 to 1.25.0 (#1413)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.24.2 to 1.25.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/commits/tokio-1.25.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-30 19:43:24 +01:00
Arnaud Gourlay
2d9e015905 Forward write request according to write consistency (#1405)
* Forward write request according to write consistency

* use gRPC internal update API by making shard_id optional

* improve and test

* mark leader peer as failed in case of service error during forward

* forward write ordering param
2023-01-30 15:02:21 +01:00
Arnaud Gourlay
4c0f849749 Clippy rust 1.67 (#1406)
* inline format! args

* inline format! args

* explicit lifetime could be elided

* fmt
2023-01-26 17:48:52 +01:00
Andrey Vasnetsov
f356455b07 add write consistency param to rest and grpc api (#1397)
* add write consistency param to rest and grpc api

* fix grpcurl + refactor grpc for better extencibility

* upd grpc docs

* fix conversion
2023-01-25 11:49:01 +01:00
Andrey Vasnetsov
61d970ef90 Init collection from (#1364)
* WIP

* WIP: add shards selection

* fmt

* WIP: collection init from other collection

* fmt

* fix bugs

* add collection write lock + use collection manager runtime instead of search runtime

* fmt

* integration test

* better names ?

* payload indexes transfer
2023-01-22 01:16:55 +01:00
Arnaud Gourlay
86b8ff6b7f Delete snapshots API (#1377)
* Delete snapshots API

* remove unecessary plurals
2023-01-20 19:28:56 +01:00
Arnaud Gourlay
6f284da22e List aliases API (rest & gRPC) (#1363)
* List aliases API (rest & gRPC)

* rework api to avoid conflicts
2023-01-19 07:59:02 +01:00
dependabot[bot]
dbbee103a0 Bump tokio from 1.24.1 to 1.24.2 (#1367)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.24.1 to 1.24.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/commits)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-18 21:08:33 +01:00
dependabot[bot]
0c7347ba0b Bump prost-types from 0.11.5 to 0.11.6 (#1348)
Bumps [prost-types](https://github.com/tokio-rs/prost) from 0.11.5 to 0.11.6.
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](https://github.com/tokio-rs/prost/compare/v0.11.5...v0.11.6)

---
updated-dependencies:
- dependency-name: prost-types
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-13 20:17:13 +01:00
dependabot[bot]
9b2cf7c5a1 Bump prost from 0.11.5 to 0.11.6 (#1336)
Bumps [prost](https://github.com/tokio-rs/prost) from 0.11.5 to 0.11.6.
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](https://github.com/tokio-rs/prost/compare/v0.11.5...v0.11.6)

---
updated-dependencies:
- dependency-name: prost
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-11 20:00:36 +01:00
dependabot[bot]
19a965d5a5 Bump tokio from 1.23.1 to 1.24.1 (#1323)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.23.1 to 1.24.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.23.1...tokio-1.24.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-09 09:47:55 +01:00
Roman Titov
2ed11d544b Add configuration option to select "memory advice" used for memmapped storage
- Add a simple abstraction/wrapper over `memmap2::Advice` and a global switch
  to select "memory advice" used for memmapped storage to the `segment` crate
- Add `mmap_advice` configuration option to the `storage::types::StorageConfig`
- Implement `search-points` benchmark
2023-01-06 21:02:57 +10:00
dependabot[bot]
0bc5851f49 Bump tokio from 1.23.0 to 1.23.1 (#1317)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.23.0 to 1.23.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.23.0...tokio-1.23.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-05 19:52:56 +01:00
Andrey Vasnetsov
926a058c73 Fixing missed leader inconsistency (#1286)
* WIP: introduce local state

* WIP: sync local state with consensus in idle

* fmt

* rm extensive debug

* update triple replication test

* update triple replication test

* rm debug logs

* fix check for established leader + only sync local if no proposals

* remove unused file

* test compatible with python 3.8

* test compatible with python 3.8

* longer wait for consensus

* longer wait for consensus

* extra sleep in test

* remove extra sleep

* Fixing missed leader inconsistency - transfers (#1298)

* explicit request timeout

* explicit request timeout

* explicit request timeout

* explicit request timeout

* explicit request timeout

* prevent double handelling of the transfer termination

* kill the process

* log on inconsistency

* log on inconsistency

* log on inconsistency

* debug

* revert debug in test

* forward updates to partial shards, abort transactions on dead node report

* fmt

* disable retry of transfer, if the transfer was cancelled + allow predictable ports in test

* fix import
2022-12-23 00:45:04 +01:00
Andrey Vasnetsov
a2c5de586b Recommendation with other collection (#1283)
* WIP: move recommendation API out of collection

* recomendations with external source of vectors

* fix test

* fmt

* fixes + test

* review fixes
2022-12-22 09:29:36 +01:00
dependabot[bot]
32208e700b Bump prost-types from 0.11.2 to 0.11.5 (#1300)
Bumps [prost-types](https://github.com/tokio-rs/prost) from 0.11.2 to 0.11.5.
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](https://github.com/tokio-rs/prost/compare/v0.11.2...v0.11.5)

---
updated-dependencies:
- dependency-name: prost-types
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-21 20:30:51 +01:00
dependabot[bot]
e57f87a7ba Bump prost from 0.11.3 to 0.11.5 (#1295)
Bumps [prost](https://github.com/tokio-rs/prost) from 0.11.3 to 0.11.5.
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](https://github.com/tokio-rs/prost/compare/v0.11.3...v0.11.5)

---
updated-dependencies:
- dependency-name: prost
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-20 19:34:42 +01:00
Arnaud Gourlay
584cdfa801 update all dependencies Cargo.lock (#1293) 2022-12-20 10:30:49 +01:00
Arnaud Gourlay
e3c6bb8a53 update tonic 0.8 & prost 0.11 (#1285)
* update tonic 0.8 & prost 0.11

* install protoc via package manager
2022-12-19 15:30:33 +01:00
Andrey Vasnetsov
d05b2def10 allow set different M param for default HNSW and payload-related links (#1257)
* allow set different M param for default HNSW and payload-related links

* review fixes
2022-12-06 13:53:42 +01:00
dependabot[bot]
c1caf271e5 Bump tokio from 1.22.0 to 1.23.0 (#1267)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.22.0 to 1.23.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.22.0...tokio-1.23.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-06 13:24:31 +01:00
Andrey Vasnetsov
04434f02b6 Update points by filter (#1249)
* allow to set remove payload by selector

* fmt
2022-12-01 09:16:07 +01:00
Andrey Vasnetsov
9f36ac60ee Full payload update (#1245)
* implement api to fully overwrite payload of the point

* fmt
2022-11-28 13:46:58 +01:00
Ivan Pleshkov
ac9538d5af Hnsw links memmap (#1211)
* add on disk key

* remove obsolete graph initialization

* remove obsolete max level

* update openapi

* graph links trait

* use mmap option

* same format for ram and mmap

* fix segment unit tests

* are you happy fmt

* are you happy clippy

* fix ci and add mmap test

* review fixes

* remove unused try-from

* fix version compatibility

* avoid loading from disk during conversion

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2022-11-25 11:47:52 +01:00
dependabot[bot]
18a67686df Bump tokio from 1.21.2 to 1.22.0 (#1234)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.21.2 to 1.22.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.21.2...tokio-1.22.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-21 20:28:57 +01:00
Arnaud Gourlay
6266d9bcdc enable compression for gRPC (#1186)
* enable compression for gRPC

* add benchmarks

* fix grpc benchmark

* discardResponseBodies to reduce memory usage

* improve & split benchmarks

* do not use gzip for inter node communication as benchmarks are not conclusive
2022-11-18 13:34:49 +01:00
Arnaud Gourlay
98fc5b2fa4 0.11.1 cherry picks (#1193)
* Complete feature road-map

* docs: add erare-humanum as a contributor for code (#1181)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* update rust version and fix clippy complains (#1190)

* handle collections transition from single-node to cluster (#1189)

* Update to v0.11.1 (#1191)

* update to v0.11.1

* also default version

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>

* consensus_state to consensus_state

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
2022-11-07 11:31:12 +01:00
Arnaud Gourlay
286ca9f5e4 0.11.0 cherry picks (#1174)
* implement concern-factor, forbig disabling the last node, improve tes… (#1168)

* implement concern-factor, forbig disabling the last node, improve test stability

* upd docs

* improve test_recover_dead_node

* rename to write_consistency_factor

* Fix bug: applied index higher than commit on restart (#1172)

* create collection with default status = dead + await for activation on create_collecton_op_submit

* fmt

* fix unit tests

Co-authored-by: Egor Ivkov <e.o.ivkov@gmail.com>

* upd integration tests

* fix transaction handling

* upd protobuf

* upd to v0.11.0

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Co-authored-by: Egor Ivkov <e.o.ivkov@gmail.com>
2022-10-27 10:16:21 +02:00
Andrey Vasnetsov
21754ef2ef report indexed payload points in info api + other fixes (#1164)
* report indexed payload points in info api + other fixes

* rollback debug changes

* clippy

* clippy
2022-10-24 09:43:02 +02:00
Ivan Pleshkov
44626bc80a exact search parameter (#1150)
* exact search parameter

* exact search test

* decrease attempts count because of test speed

* minor review fixes

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2022-10-20 15:39:21 +04:00
Andrey Vasnetsov
cf29b67640 v0.10.5 2022-10-18 10:49:43 +02:00
Andrey Vasnetsov
411ddabe49 v0.10.4 2022-10-13 09:14:11 +02:00
Andrey Vasnetsov
68a46a0f2e v0.10.3 2022-10-13 09:14:11 +02:00
dependabot[bot]
d0789f3a90 Bump uuid from 1.1.2 to 1.2.1 (#1111)
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.1.2 to 1.2.1.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/1.1.2...1.2.1)

---
updated-dependencies:
- dependency-name: uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-11 08:25:24 +02:00
dependabot[bot]
814b0ebcb4 Bump schemars from 0.8.10 to 0.8.11 (#1089)
Bumps [schemars](https://github.com/GREsau/schemars) from 0.8.10 to 0.8.11.
- [Release notes](https://github.com/GREsau/schemars/releases)
- [Changelog](https://github.com/GREsau/schemars/blob/master/CHANGELOG.md)
- [Commits](https://github.com/GREsau/schemars/compare/v0.8.10...v0.8.11)

---
updated-dependencies:
- dependency-name: schemars
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-04 11:17:22 +02:00
Andrey Vasnetsov
935f069d53 upd version to v0.10.2 2022-09-30 11:08:16 +02:00
dependabot[bot]
fd9c39f1ab Bump tokio from 1.21.1 to 1.21.2 (#1067)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.21.1 to 1.21.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.21.1...tokio-1.21.2)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-28 11:45:26 +02:00
Egor Ivkov
9a3fad44b2 Revert "Revert replication changes (#1055)" (#1056)
This reverts commit a2ed0cf5db.
2022-09-23 16:46:06 +03:00
Egor Ivkov
a2ed0cf5db Revert replication changes (#1055)
* Revert "Replication set `Change` enum"

This reverts commit 28e8487542.

* Revert changes (#1053)

* Revert "Generate API - replication follow up (#1043)"

This reverts commit 183ea9e5e3.

* Revert "Replication related API changes (#1037)"

This reverts commit d690b700ff.

* Revert "Persist replica state (#1035)"

This reverts commit 31c3ae482d.

* Revert "[replication] Account for replicas in shard distribution (#1009)"

This reverts commit e332d482b0.

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2022-09-23 16:33:07 +03:00
Egor Ivkov
183ea9e5e3 Generate API - replication follow up (#1043)
Also removes `JsonSchema` from structs not exposed in API
2022-09-22 14:53:00 +03:00
Egor Ivkov
d690b700ff Replication related API changes (#1037)
* Replication related API changes

* minor proto file fix
2022-09-21 18:49:47 +03:00
Andrey Vasnetsov
f918a14819 upd to v0.10.1 (#1031)
* upd to v0.10.1

* upd lock
2022-09-19 15:31:35 +02:00
Andrey Vasnetsov
dc07b01e1f remove deprecated fields from API (#1030)
* remove depricated fields from API

* fmt

* upd openapi and integration tests

* fix grpc test

* regenerate storage reference data

* improve docs

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
2022-09-19 13:51:03 +02:00