upstream-archive/qdrant
1
0
Fork 0
mirror of https://github.com/qdrant/qdrant.git synced 2026-08-07 02:20:55 -05:00
Code Issues Packages Projects Releases Wiki Activity
5,465 Commits 346 Branches 118 Tags
94de0e2d2dcf997bf26bec34dad407d8de9a9abb
Commit Graph

3618 Commits

Author SHA1 Message Date
Andrey Vasnetsov
0e42672525 fix snapshot recovery - fix proper deactivation messages (#1238) 2022-11-23 13:47:14 +01:00
Arnaud Gourlay
359c0a7200 Fix panic on substraction overflow (#1236) 
* Fix panic on substraction overflow

qdrant::startup] Panic occurred in file lib/collection/src/collection_manager/holders/proxy_segment.rs at line 494: attempt to subtract with overflow

* fix assignment
 2022-11-22 13:39:01 +01:00
dependabot[bot]
18a67686df Bump tokio from 1.21.2 to 1.22.0 (#1234) 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.21.2 to 1.22.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.21.2...tokio-1.22.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-21 20:28:57 +01:00
dependabot[bot]
24f3c90140 Bump rayon from 1.5.3 to 1.6.0 (#1233) 
Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.5.3 to 1.6.0.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/compare/v1.5.3...rayon-core-v1.6.0)

---
updated-dependencies:
- dependency-name: rayon
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-21 20:28:20 +01:00
Andrey Vasnetsov
803ad06abe Extend telemetry (#1230) 
* WIP: extend telemetry api for faster cluster manager

* fmt

* clippy fix
 2022-11-21 10:26:54 +01:00
Andrey Vasnetsov
5cf0653cff update collection write consistency param on raft-snapshot apply (#1227) 2022-11-18 13:36:09 +01:00
Andrey Vasnetsov
8cc7f67f1b spawn tokio task to prevent blocking execution thread (#1226) 2022-11-18 13:35:57 +01:00
Andrey Vasnetsov
4a9081308b do not crash the service if some collection failed (#1223) 2022-11-18 13:35:46 +01:00
Andrey Vasnetsov
d340d3fa4e disable exact counting in collection cluster info api (#1221) 2022-11-18 13:35:35 +01:00
Arnaud Gourlay
6266d9bcdc enable compression for gRPC (#1186) 
* enable compression for gRPC

* add benchmarks

* fix grpc benchmark

* discardResponseBodies to reduce memory usage

* improve & split benchmarks

* do not use gzip for inter node communication as benchmarks are not conclusive
 2022-11-18 13:34:49 +01:00
Arnaud Gourlay
c5558ab22b Probabilistic search sampling for better limit (#1199) 
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-11-16 20:53:58 +01:00
Andrey Vasnetsov
772326f848 Recover distributed collection (#1214) 
* WIP: shards recovering API: download snapshot

* fmt

* snapshot recovery API

* fmt

* snapshot recovery integration test + fixes

* review fixes

* review fixes 2

* Update lib/storage/src/content_manager/snapshots/recover.rs

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
 2022-11-16 10:55:23 +01:00
Andrey Vasnetsov
aebadb7380 save params of collection after update (#1212) 2022-11-16 10:55:17 +01:00
dependabot[bot]
96a99935e9 Bump memmap2 from 0.5.7 to 0.5.8 (#1210) 
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.5.7 to 0.5.8.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases)
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.5.7...v0.5.8)

---
updated-dependencies:
- dependency-name: memmap2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-09 21:51:07 +01:00
Andrey Vasnetsov
dad331fc26 fix colletion params update (#1208) 2022-11-09 13:10:54 +01:00
Andrey Vasnetsov
cef16c90ef wait for state deactivation on replica update failure (#1200) 
* wait for state deactivation on replica update failure

* review fixes
 2022-11-09 12:58:22 +01:00
dependabot[bot]
8504cf502a Bump num_cpus from 1.13.1 to 1.14.0 (#1203) 2022-11-09 08:31:47 +00:00
dependabot[bot]
50a6c10c1d Bump ordered-float from 3.3.0 to 3.4.0 (#1204) 
Bumps [ordered-float](https://github.com/reem/rust-ordered-float) from 3.3.0 to 3.4.0.
- [Release notes](https://github.com/reem/rust-ordered-float/releases)
- [Commits](https://github.com/reem/rust-ordered-float/compare/v3.3.0...v3.4.0)

---
updated-dependencies:
- dependency-name: ordered-float
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-09 08:42:10 +01:00
dependabot[bot]
aa3139b748 Bump indicatif from 0.17.1 to 0.17.2 (#1202) 
Bumps [indicatif](https://github.com/console-rs/indicatif) from 0.17.1 to 0.17.2.
- [Release notes](https://github.com/console-rs/indicatif/releases)
- [Commits](https://github.com/console-rs/indicatif/compare/0.17.1...0.17.2)

---
updated-dependencies:
- dependency-name: indicatif
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-09 08:41:44 +01:00
dependabot[bot]
03e550b05d Bump env_logger from 0.9.1 to 0.9.3 (#1201) 
Bumps [env_logger](https://github.com/env-logger-rs/env_logger) from 0.9.1 to 0.9.3.
- [Release notes](https://github.com/env-logger-rs/env_logger/releases)
- [Changelog](https://github.com/env-logger-rs/env_logger/blob/main/CHANGELOG.md)
- [Commits](https://github.com/env-logger-rs/env_logger/compare/v0.9.1...v0.9.3)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-09 08:41:25 +01:00
Andrey Vasnetsov
7e4424ab57 Cosine dist zero vectors (#1198) 
* skip pre-processing of zero-length vector for cosine distance

* fmt
 2022-11-08 17:36:20 +01:00
Ivan Pleshkov
d555a244c2 Hnsw links storage (#1171) 
* hnsw compact links storage

* dump links memsize

* are you happy fmt

* fix build

* calculate capacity and add shrink

* fmt

* remove garbage

* fix unit tests

* default values for graph layers

* remove obsolete fields

* add comments

* more comments

* use one flattened links structure

* refactor and add unit tests

* move merge into graph builder

* fix messages in test_hnsw_graph_properties
 2022-11-08 13:42:50 +04:00
dependabot[bot]
558d12b255 Bump pprof from 0.10.1 to 0.11.0 (#1188) 
Bumps [pprof](https://github.com/tikv/pprof-rs) from 0.10.1 to 0.11.0.
- [Release notes](https://github.com/tikv/pprof-rs/releases)
- [Changelog](https://github.com/tikv/pprof-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tikv/pprof-rs/commits)

---
updated-dependencies:
- dependency-name: pprof
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-07 14:41:44 +01:00
Arnaud Gourlay
98fc5b2fa4 0.11.1 cherry picks (#1193) 
* Complete feature road-map

* docs: add erare-humanum as a contributor for code (#1181)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* update rust version and fix clippy complains (#1190)

* handle collections transition from single-node to cluster (#1189)

* Update to v0.11.1 (#1191)

* update to v0.11.1

* also default version

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>

* consensus_state to consensus_state

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
 2022-11-07 11:31:12 +01:00
Andrey Vasnetsov
383100ebb6 remove suggesting changes in replications on replication factor change (#1177) 2022-10-31 09:57:22 +01:00
Ivan Pleshkov
017df52528 Use less ram for id tracker (#1176) 
* use less ram for id tracker

* are you happy clippy

* use vec for internals

* use versions for internal ids

* keys test

* Use less ram for id tracker fixes (#1182)

* WIP: internal_to_version

* fmt

* fix unit tests

* add comment

Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-10-31 10:44:00 +04:00
Andrey Vasnetsov
a0b6fa81b7 Consensus q n a (#1169) 
* Expand comments and minor refactoring for raft state

* fmt

* add comments to consensus.rs
 2022-10-30 20:34:09 +01:00
erare-humanum
93a4499859 Update: unmaintained crate memmap -> memmap2 (#559) (#1160) 
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-10-30 11:50:27 +01:00
Arnaud Gourlay
286ca9f5e4 0.11.0 cherry picks (#1174) 
* implement concern-factor, forbig disabling the last node, improve tes… (#1168)

* implement concern-factor, forbig disabling the last node, improve test stability

* upd docs

* improve test_recover_dead_node

* rename to write_consistency_factor

* Fix bug: applied index higher than commit on restart (#1172)

* create collection with default status = dead + await for activation on create_collecton_op_submit

* fmt

* fix unit tests

Co-authored-by: Egor Ivkov <e.o.ivkov@gmail.com>

* upd integration tests

* fix transaction handling

* upd protobuf

* upd to v0.11.0

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Co-authored-by: Egor Ivkov <e.o.ivkov@gmail.com>
 2022-10-27 10:16:21 +02:00
Andrey Vasnetsov
071d1948b0 update and improve openapi definitions 2022-10-24 16:04:31 +02:00
Andrey Vasnetsov
ab19576c5d Force sync vector storage offset (#1110) 
* simplify vector storage `update` semantic, always assume inplace vector replacement

* fmt
 2022-10-24 12:34:26 +02:00
Andrey Vasnetsov
54e0857708 ignore points with missing expernal ids instead of panic (#1109) 2022-10-24 12:33:43 +02:00
Arnaud Gourlay
1ff09516a0 limit set to zero returns an empty result (#1083) 2022-10-24 12:32:46 +02:00
Andrey Vasnetsov
21754ef2ef report indexed payload points in info api + other fixes (#1164) 
* report indexed payload points in info api + other fixes

* rollback debug changes

* clippy

* clippy
 2022-10-24 09:43:02 +02:00
Ivan Pleshkov
516dcd7020 Telemetry level of detail (#1049) 
* telemetry level of detail

* rename duration aggregator

* are you happy fmt

* move total searches sum

* separate levels

* optional bucket size

* search telemetry improvements

* separate web telemetry into methods

* tonic telemetry methods

* merge optimizations

* are you happy fmt

* better rounding

* qdrant configs on level 1

* provide collection params

* add peers count

* collection points count

* update openapi

* use pattern in actix telemetry

* are you happy fmt

* merge dev

* are you happy fmt

* fix merge conflicts

* update openapi

* fix build

* are you happy fmt

* add exact searches statistics

* process replica set

* update openapi

* fix wrong name

* fix naming

* fix unwrap

* review

* fmt

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-10-23 00:48:55 +02:00
Egor Ivkov
a96f92cbfe Record message send failures and stop consensus on service error (#1152) 
* Record message send failures and stop consensus on service error

* Generate openapi

* Review fixes: log first error and remove on success
 2022-10-22 23:23:37 +02:00
dependabot[bot]
740c8c412b Bump futures from 0.3.24 to 0.3.25 (#1153) 
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.24 to 0.3.25.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.24...0.3.25)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-21 08:17:53 +02:00
Ivan Pleshkov
44626bc80a exact search parameter (#1150) 
* exact search parameter

* exact search test

* decrease attempts count because of test speed

* minor review fixes

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-10-20 15:39:21 +04:00
Arnaud Gourlay
faa4008938 Enable replication in consensus test (#1140) 2022-10-20 11:36:22 +02:00
Andrey Vasnetsov
75563b20b4 Consensus fault tollerance improvements (#1149) 
* consensus fault tollerance improvements

* fmt
 2022-10-20 11:22:22 +02:00
Andrey Vasnetsov
bf48066145 Full reassign payload on upsert (#1148) 
* set_full_payload on upsert

* fmt

* fix clippy
 2022-10-20 10:26:33 +02:00
Andrey Vasnetsov
701865e528 fmt (#1145) 2022-10-20 09:05:59 +02:00
dependabot[bot]
2bd7a36006 Bump ordered-float from 3.2.0 to 3.3.0 (#1147) 
Bumps [ordered-float](https://github.com/reem/rust-ordered-float) from 3.2.0 to 3.3.0.
- [Release notes](https://github.com/reem/rust-ordered-float/releases)
- [Commits](https://github.com/reem/rust-ordered-float/compare/v3.2.0...v3.3.0)

---
updated-dependencies:
- dependency-name: ordered-float
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-20 08:56:18 +02:00
dependabot[bot]
7d9ca67ea9 Bump async-trait from 0.1.57 to 0.1.58 (#1146) 
Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.57 to 0.1.58.
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.57...0.1.58)

---
updated-dependencies:
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-20 08:55:54 +02:00
Andrey Vasnetsov
123ae1607d Await for collection created for sure (#1144) 
* run plunger operation after user request to ensure that the majority of the consensus actually applied previous operation

* fmt

* fix test

* ensure all operations are unique in consensus queue

* add extra timeout for CI tests

* fix timout param in delete peer + do not explicilty await on peer removal

* fmt
 2022-10-19 17:59:16 +02:00
Egor Ivkov
d3849d712b Allow peer force removal (#1139) 
* Allow peer force removal

* Remove shards
 2022-10-19 17:01:13 +02:00
Andrey Vasnetsov
2b4ab27506 Write lock for user requests (#1143) 
* lock update operations only

* fmt

* fix

* lock openapi integration test
 2022-10-19 12:57:52 +02:00
Andrey Vasnetsov
7f8c088711 Replication transfer fixes (#1142) 
* fix validation and task assignment during replica transfer

* fmt

* upd openapi

* extend integration test
 2022-10-19 11:38:26 +02:00
Ivan Pleshkov
f50f1f4c71 #1119 disable updating feature (#1126) 
* disable updating feature

* customize error message

* update namings

* Update src/actix/api/service_api.rs

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>

* fix build

* update api

* update api and allow deletion

* update openapi

* replace Bad Request with Forbidden

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-10-18 12:58:38 +02:00
Andrey Vasnetsov
cf29b67640 v0.10.5 2022-10-18 10:49:43 +02:00