Commit Graph

3332 Commits

Author SHA1 Message Date
Jojii
b94d02ad30 Add error handling if vector is missing for point (#5211)
* add error handling for missing vectors

* add integration test

* add call without specifying 'using' parameter
2024-10-10 12:48:34 +02:00
Arnaud Gourlay
29df805470 Fix typo in RocksDB wrapper type (#5207) 2024-10-09 14:18:32 +02:00
Kumar Shivendu
f7b6148242 Dont consider resharding shards as unhealthy for /readyz (#5204)
* Dont consider resharding shards as unhealthy for /readyz

* clippy
2024-10-09 11:14:24 +02:00
Andrey Vasnetsov
e0d507dac7 Inference interface in REST and gRPC (#5165)
* include document & image objects into grpc API

* introduce image and object to rest api

* minor refactoring

* rename Vector -> VectorInternal

* decompose vector data structures

* add schema

* fmt

* grpc docs

* fix conversion

* fix clippy

* fix another conversion

* rename VectorInput -> VectorInputInternal

* replace grpc TryFrom with async functions

* fmt

* replace rest TryFrom with async functions

* add image and object into query rest

* separate inference related conversions

* move json-related conversions into a separate file

* move vector-related transformations into a separate file

* move more vector related-conversions into dedicated module
2024-10-09 10:15:46 +02:00
Tim Visée
b22e177584 Report CPU endianness in telemetry output (#5180)
* Report CPU endianness in telemetry output

* Make endianness optional, don't report if anonymizing telemetry

* Update OpenAPI definition

* Switch back to single compile time endianness check

* Update OpenAPI specification

* Remove cpu-endian dependency

* De-anonymize CPU endianness

---------

Co-authored-by: xzfc <xzfcpw@gmail.com>
2024-10-08 14:50:40 +02:00
Tim Visée
082274bd68 Add wait parameter to cluster metadata API (#5174)
* Add blocking implementation of update cluster metadata

* Expose wait parameter to cluster metadata API

* Refactor shard transfer dispatcher into toc dispatcher

* Reformat
2024-10-08 13:03:59 +02:00
dependabot[bot]
e4fb392558 build(deps): bump indexmap from 2.5.0 to 2.6.0 (#5192)
Bumps [indexmap](https://github.com/indexmap-rs/indexmap) from 2.5.0 to 2.6.0.
- [Changelog](https://github.com/indexmap-rs/indexmap/blob/master/RELEASES.md)
- [Commits](https://github.com/indexmap-rs/indexmap/compare/2.5.0...2.6.0)

---
updated-dependencies:
- dependency-name: indexmap
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 10:30:56 +02:00
dependabot[bot]
81853ecce4 build(deps): bump rustls from 0.23.13 to 0.23.14 (#5199)
Bumps [rustls](https://github.com/rustls/rustls) from 0.23.13 to 0.23.14.
- [Release notes](https://github.com/rustls/rustls/releases)
- [Changelog](https://github.com/rustls/rustls/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rustls/rustls/compare/v/0.23.13...v/0.23.14)

---
updated-dependencies:
- dependency-name: rustls
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 09:36:50 +02:00
dependabot[bot]
e0b5968677 build(deps): bump sysinfo from 0.31.4 to 0.32.0 (#5197)
Bumps [sysinfo](https://github.com/GuillaumeGomez/sysinfo) from 0.31.4 to 0.32.0.
- [Changelog](https://github.com/GuillaumeGomez/sysinfo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/GuillaumeGomez/sysinfo/compare/v0.31.4...v0.32.0)

---
updated-dependencies:
- dependency-name: sysinfo
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 09:31:32 +02:00
dependabot[bot]
42eecc3ac4 build(deps): bump procfs from 0.16.0 to 0.17.0 (#5193)
Bumps [procfs](https://github.com/eminence/procfs) from 0.16.0 to 0.17.0.
- [Release notes](https://github.com/eminence/procfs/releases)
- [Commits](https://github.com/eminence/procfs/compare/v0.16.0...v0.17.0)

---
updated-dependencies:
- dependency-name: procfs
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 09:30:07 +02:00
dependabot[bot]
aaab1e5421 build(deps): bump futures from 0.3.30 to 0.3.31 (#5194)
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.30 to 0.3.31.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.30...0.3.31)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 09:28:19 +02:00
dependabot[bot]
fa97ae5bd3 build(deps): bump ph from 0.8.3 to 0.8.4 (#5198)
Bumps [ph](https://github.com/beling/bsuccinct-rs) from 0.8.3 to 0.8.4.
- [Commits](https://github.com/beling/bsuccinct-rs/commits)

---
updated-dependencies:
- dependency-name: ph
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 08:53:34 +02:00
dependabot[bot]
d4a65bd56a build(deps): bump clap from 4.5.18 to 4.5.19 (#5196)
Bumps [clap](https://github.com/clap-rs/clap) from 4.5.18 to 4.5.19.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.18...clap_complete-v4.5.19)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 08:52:16 +02:00
dependabot[bot]
92f5bc6901 build(deps): bump cc from 1.1.21 to 1.1.28 (#5191)
Bumps [cc](https://github.com/rust-lang/cc-rs) from 1.1.21 to 1.1.28.
- [Release notes](https://github.com/rust-lang/cc-rs/releases)
- [Changelog](https://github.com/rust-lang/cc-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/cc-rs/compare/cc-v1.1.21...cc-v1.1.28)

---
updated-dependencies:
- dependency-name: cc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 08:51:21 +02:00
Andrey Vasnetsov
68179bf697 make operationId consistent with grpc (#5201) 2024-10-07 22:02:22 +02:00
Andrey Vasnetsov
f5cc150dca enable mmap vector storage by default (#5188) 2024-10-07 17:09:52 +02:00
Ivan Pleshkov
080352cc56 mmap geo index api (#5163)
* define mmap geo index

fix compilation

deleted flags

load new mmap

geo index tests

fix tests

fix build after rebase

add files list

* refactor get_stored_sub_regions output type

* review remanings

* mmap geo index api

* fix after rebase
2024-10-07 14:35:17 +02:00
xzfc
10ce515ecf shell.nix: bump to rust 1.81, misc changes (#5185)
* shell.nix: add a comment

* tools/nix/update.py: let it run outside of the nix-shell

* shell.nix: bump to rust 1.81
2024-10-07 10:30:20 +00:00
Ivan Pleshkov
e00bd70f9c Mmap geo index (#4841)
* define mmap geo index

fix compilation

deleted flags

load new mmap

geo index tests

fix tests

fix build after rebase

add files list

* are you happy fmt

* refactor get_stored_sub_regions output type

* repr(C) for geohash

* Replace manual flattening with flat map iterator

* review remanings

* remove panics

* add files for snapshot

* rename Dynamic into InMemory

* are you happy fmt

* review renames

* Use copied rather than cloned

* minor review refactoring + comments

---------

Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: generall <andrey@vasnetsov.com>
2024-10-07 12:08:58 +02:00
xzfc
90b5aadcde Drop all-contributors (#5186) 2024-10-07 11:40:28 +02:00
Andrey Vasnetsov
e76132b6e7 enable sparse snapshots (#5183) 2024-10-06 17:50:54 +02:00
Jojii
b210c3d09e Improve docs for new_unchecked_verification_pass (#5181) 2024-10-04 23:27:35 +02:00
xzfc
761069bc0a Restore SnapshotFormat::Streamable snapshots (#5179) 2024-10-04 16:39:56 +00:00
xzfc
98c4e605ae Support SnapshotFormat::Streamable, add REST endpoint (#5178) 2024-10-04 16:36:19 +00:00
xzfc
bc2104604c Introduce SnapshotFormat (#5175) 2024-10-04 16:09:30 +00:00
Roman Titov
a794983050 Instantly self-elect Raft leader, when cluster contains single peer (#5173)
* When starting first peer of a new cluster, tick a few times to establish leader instantly

* Instantly self-elect itself as Raft leader, when cluster contains single peer
2024-10-04 17:00:49 +02:00
xzfc
7932286709 Update tar_ext::BuilderExt (#5112)
* tar_ext::BuilderExt: generalize and add drop workaround

- Generalize: instead of taking `std::fs::File`, take
  `impl Write + Seek`.
- Add a workaround for drop: `tar::Builder<W>::drop` might panic if the
  underlying writer panics when using in async context.

* Drop unnecessary write_fn usage

* tar_ext::BuilderExt: support both seekable and streaming variants

* tar_ext::BuilderExt: support both owned/borrowed variants
2024-10-04 14:10:09 +00:00
Tim Visée
d578c428a5 When removing a shard, remove its configuration file first (#5176) 2024-10-04 12:10:28 +02:00
Tim Visée
e2dfa6d78e Fix panic on start if resharding shard is partially created (#5169)
* Reuse shard path if it already exists

* Recreate rather than reuse shard directory
2024-10-02 11:22:17 +02:00
Luis Cossío
045779233a Facets: Support for bool payload (#5100)
* enable faceting with boolean index

* test and fix

* also test uuid
2024-10-01 09:17:50 -03:00
dependabot[bot]
4bd0ac8909 build(deps): bump rustls-pemfile from 2.1.3 to 2.2.0 (#5156)
Bumps [rustls-pemfile](https://github.com/rustls/pemfile) from 2.1.3 to 2.2.0.
- [Release notes](https://github.com/rustls/pemfile/releases)
- [Commits](https://github.com/rustls/pemfile/compare/v/2.1.3...v/2.2.0)

---
updated-dependencies:
- dependency-name: rustls-pemfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-01 10:50:03 +02:00
dependabot[bot]
86de9fadc2 build(deps): bump flate2 from 1.0.33 to 1.0.34 (#5157)
Bumps [flate2](https://github.com/rust-lang/flate2-rs) from 1.0.33 to 1.0.34.
- [Release notes](https://github.com/rust-lang/flate2-rs/releases)
- [Changelog](https://github.com/rust-lang/flate2-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/flate2-rs/compare/1.0.33...1.0.34)

---
updated-dependencies:
- dependency-name: flate2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-01 10:42:16 +02:00
dependabot[bot]
3b2110fe6c build(deps): bump io-uring from 0.6.4 to 0.7.0 (#5158)
Bumps [io-uring](https://github.com/tokio-rs/io-uring) from 0.6.4 to 0.7.0.
- [Commits](https://github.com/tokio-rs/io-uring/commits)

---
updated-dependencies:
- dependency-name: io-uring
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-01 10:15:34 +02:00
dependabot[bot]
a5cebd8eef build(deps): bump rstest from 0.22.0 to 0.23.0 (#5159)
Bumps [rstest](https://github.com/la10736/rstest) from 0.22.0 to 0.23.0.
- [Release notes](https://github.com/la10736/rstest/releases)
- [Changelog](https://github.com/la10736/rstest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/la10736/rstest/compare/v0.22.0...v0.23.0)

---
updated-dependencies:
- dependency-name: rstest
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-01 10:13:09 +02:00
dependabot[bot]
4d1e925b0e build(deps): bump tempfile from 3.12.0 to 3.13.0 (#5162)
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.12.0 to 3.13.0.
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.12.0...v3.13.0)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-01 10:09:47 +02:00
dependabot[bot]
7e230c9131 build(deps): bump reqwest from 0.12.7 to 0.12.8 (#5160)
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.12.7 to 0.12.8.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.12.7...v0.12.8)

---
updated-dependencies:
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-01 10:08:09 +02:00
dependabot[bot]
fa55b92ade build(deps): bump async-trait from 0.1.82 to 0.1.83 (#5155)
Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.82 to 0.1.83.
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.82...0.1.83)

---
updated-dependencies:
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-01 09:20:58 +02:00
dependabot[bot]
52db80a292 build(deps): bump ordered-float from 4.2.2 to 4.3.0 (#5154)
Bumps [ordered-float](https://github.com/reem/rust-ordered-float) from 4.2.2 to 4.3.0.
- [Release notes](https://github.com/reem/rust-ordered-float/releases)
- [Commits](https://github.com/reem/rust-ordered-float/compare/v4.2.2...v4.3.0)

---
updated-dependencies:
- dependency-name: ordered-float
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-01 09:19:46 +02:00
Jojii
a638378df2 remove obsolete timeout (#5142) 2024-09-26 12:59:17 +02:00
Jojii
8980483416 [Strict mode] Merge StrictMode config types (#5137)
* merge StrictModeConfig and StrictModeConfigDiff into one type

* review remarks
2024-09-26 12:36:39 +02:00
Jojii
29556cebc2 [Strict mode] Unit tests (#5124)
* add tests for strict mode

* add good example for strict mode test

* Add succes case to test_request_exact

---------

Co-authored-by: timvisee <tim@visee.me>
2024-09-26 10:27:05 +02:00
Jojii
3595310706 [Strict mode] migrate toc_new -> toc (#5120)
* Replace all remaining calls of toc() with toc_new

* mirgate toc_new -> toc
2024-09-25 15:15:58 +02:00
Jojii
791d064bde [Strict mode] default config (#5037)
* add default config for strict mode

* add default values as example to config/config.yaml

* improve implementation

* update grpc docs
2024-09-25 14:22:36 +02:00
Jojii
a864d4c92e [Strict mode] GRPC verification (#5003)
* first grpc implementations

* make usage of 'scroll' more clear

* introduce 'CheckedTocProvider' trait

* implement more grpc types

* more grpc implementations

* rename StrictModeCheckedProvider to StrictModeCheckedTocProvider

* don't use deprecated feature for UncheckedTocProvider
2024-09-25 13:22:37 +02:00
Jojii
fddb20ba1a [Strict mode] Finish REST requests (#4990)
* add strict mode to more requests

* add timeout check from request query

* unify timeout checking

* adjust functions not requiring strict mode

* add missing checks

* add Prefetch check for query api

* review remarks
2024-09-25 12:42:30 +02:00
Jojii
792bbb7523 Trigger optimizers when uploading snapshots (#5140)
* Trigger optimizers when uploading snapshots

* Trigger optimizers through new method when optimizer config is updated

* review remarks

* Add comment on why ignoring channel send errors is fine

* Remove obsolete return value omitting

---------

Co-authored-by: timvisee <tim@visee.me>
2024-09-24 17:52:03 +02:00
Arnaud Gourlay
f3a2807904 Add unit test for set_payload bug (#5132) 2024-09-24 10:38:45 +02:00
Arnaud Gourlay
d527ab9cd0 Promote atomicwrites to workspace (#5135) 2024-09-24 09:44:12 +02:00
dependabot[bot]
8a0a6e9278 build(deps): bump cc from 1.1.19 to 1.1.21 (#5129)
Bumps [cc](https://github.com/rust-lang/cc-rs) from 1.1.19 to 1.1.21.
- [Release notes](https://github.com/rust-lang/cc-rs/releases)
- [Changelog](https://github.com/rust-lang/cc-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/cc-rs/compare/cc-v1.1.19...cc-v1.1.21)

---
updated-dependencies:
- dependency-name: cc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-24 07:42:24 +02:00
dependabot[bot]
f9d72a0922 build(deps): bump bytes from 1.7.1 to 1.7.2 (#5128)
Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.7.1 to 1.7.2.
- [Release notes](https://github.com/tokio-rs/bytes/releases)
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/bytes/compare/v1.7.1...v1.7.2)

---
updated-dependencies:
- dependency-name: bytes
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-23 21:43:34 +02:00