upstream-archive/qdrant
1
0
Fork 0
mirror of https://github.com/qdrant/qdrant.git synced 2026-09-29 17:37:58 -05:00
Code Issues Packages Projects Releases Wiki Activity
1,077 Commits 437 Branches 119 Tags
consensus-bootstrap-fail
Commit Graph
372 Commits
Author SHA1 Message Date
dependabot[bot] fc24951d1a Bump arc-swap from 1.5.1 to 1.6.0 (#1310) 
Bumps [arc-swap](https://github.com/vorner/arc-swap) from 1.5.1 to 1.6.0.
- [Release notes](https://github.com/vorner/arc-swap/releases)
- [Changelog](https://github.com/vorner/arc-swap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vorner/arc-swap/compare/v1.5.1...v1.6.0)

---
updated-dependencies:
- dependency-name: arc-swap
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-01-04 10:12:34 +01:00
Andrey Vasnetsov bd05a69a5f Id mapper inconsistency (#1302) 
* always flush wal

* always flush wal fix

* always flush wal fmt

* flush wal during background flush

* async wal flush

* use id-tracker internal id for next-id instead of vector storage

* add flush order and recovery comment

fix merge bug

* longer timeout in test
 2022-12-25 22:36:31 +01:00
Andrey Vasnetsov 431fcbb4ca better validation for transfer initiation 2022-12-23 21:16:13 +01:00
Andrey Vasnetsov 926a058c73 Fixing missed leader inconsistency (#1286) 
* WIP: introduce local state

* WIP: sync local state with consensus in idle

* fmt

* rm extensive debug

* update triple replication test

* update triple replication test

* rm debug logs

* fix check for established leader + only sync local if no proposals

* remove unused file

* test compatible with python 3.8

* test compatible with python 3.8

* longer wait for consensus

* longer wait for consensus

* extra sleep in test

* remove extra sleep

* Fixing missed leader inconsistency - transfers (#1298)

* explicit request timeout

* explicit request timeout

* explicit request timeout

* explicit request timeout

* explicit request timeout

* prevent double handelling of the transfer termination

* kill the process

* log on inconsistency

* log on inconsistency

* log on inconsistency

* debug

* revert debug in test

* forward updates to partial shards, abort transactions on dead node report

* fmt

* disable retry of transfer, if the transfer was cancelled + allow predictable ports in test

* fix import
 2022-12-23 00:45:04 +01:00
Andrey Vasnetsov a2c5de586b Recommendation with other collection (#1283) 
* WIP: move recommendation API out of collection

* recomendations with external source of vectors

* fix test

* fmt

* fixes + test

* review fixes
 2022-12-22 09:29:36 +01:00
dependabot[bot] 16a4d54885 Bump num_cpus from 1.14.0 to 1.15.0 (#1296) 
Bumps [num_cpus](https://github.com/seanmonstar/num_cpus) from 1.14.0 to 1.15.0.
- [Release notes](https://github.com/seanmonstar/num_cpus/releases)
- [Changelog](https://github.com/seanmonstar/num_cpus/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/num_cpus/compare/v1.14.0...v1.15.0)

---
updated-dependencies:
- dependency-name: num_cpus
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-20 19:35:10 +01:00
Arnaud Gourlay e3c6bb8a53 update tonic 0.8 & prost 0.11 (#1285) 
* update tonic 0.8 & prost 0.11

* install protoc via package manager
 2022-12-19 15:30:33 +01:00
dependabot[bot] 1f90e12eec Bump semver from 1.0.14 to 1.0.16 (#1292) 
Bumps [semver](https://github.com/dtolnay/semver) from 1.0.14 to 1.0.16.
- [Release notes](https://github.com/dtolnay/semver/releases)
- [Commits](https://github.com/dtolnay/semver/compare/1.0.14...1.0.16)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-19 14:52:56 +01:00
dependabot[bot] b76f671ac4 Bump async-trait from 0.1.59 to 0.1.60 (#1291) 
Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.59 to 0.1.60.
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.59...0.1.60)

---
updated-dependencies:
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-19 14:16:12 +01:00
Arnaud Gourlay b6c37d72f2 Explicitly flush consensus WAL after (#1282) 
* panic on wal flush

* use latest wal

* try different flushing after truncate scheme

* pull latest wal branch rev

* use latest wal with fix
 2022-12-19 13:21:00 +01:00
Arnaud Gourlay d77debf122 Clippy for Rust 1.66 (#1284) 2022-12-16 10:53:51 +01:00
Andrey Vasnetsov aeba32ba3e Shard build refactoring (#1280) 
* refactor some code of shard creation and optimizers

* Shard build refactoring debug (#1279)

* add debig code

* debug code

* debug code

* debug code

* debug code

* debug code

* debug code

* debug code

* debug code

* debug code

* debug code

* refactor stop-checking during the optimization

* remove debug logs

* improve delete-renaming schema

* fmt

* move collection file removing to the async task

* rename check_optimization_stopped into more general check_process_stopped
 2022-12-14 10:11:46 +01:00
Andrey Vasnetsov 69d9afe0e0 fix 404 error code propagation (#1281) 2022-12-14 10:11:28 +01:00
Andrey Vasnetsov d05b2def10 allow set different M param for default HNSW and payload-related links (#1257) 
* allow set different M param for default HNSW and payload-related links

* review fixes
 2022-12-06 13:53:42 +01:00
Andrey Vasnetsov 49c3e37061 add parameter that defines a source of truth for snapshot recovery (#1262) 
* add parameter that defines a source of truth for snapshot recovery

* review fixes
 2022-12-06 13:53:03 +01:00
dependabot[bot] c1caf271e5 Bump tokio from 1.22.0 to 1.23.0 (#1267) 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.22.0 to 1.23.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.22.0...tokio-1.23.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-06 13:24:31 +01:00
Andrey Vasnetsov 4523ff5746 Peer id in snapshot name (#1260) 
* Add peer id into the snapshot name to distinguish snapshots from different ndoes

* fmt
 2022-12-06 12:31:58 +01:00
Andrey Vasnetsov 0925d02667 include backtrace for service errors (#1256) 2022-12-06 12:31:38 +01:00
Andrey Vasnetsov a23adcdd82 Missed points on update fix (#1255) 
* add context to point-not-found error

* fmt

* add backtrace

* use upgradable lock

* unique points in insert operation

* remove debug code
 2022-12-06 12:31:24 +01:00
Arnaud Gourlay 9ef2115655 add collection name to shard centric logs (#1254) 2022-12-02 14:09:41 +01:00
Arnaud Gourlay 0cc6903a6d Update wal to revision 90471ad7 (#1253) 2022-12-01 23:05:16 +01:00
Andrey Vasnetsov 04434f02b6 Update points by filter (#1249) 
* allow to set remove payload by selector

* fmt
 2022-12-01 09:16:07 +01:00
dependabot[bot] 4e4ccc83e6 Bump async-trait from 0.1.58 to 0.1.59 (#1250) 
Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.58 to 0.1.59.
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.58...0.1.59)

---
updated-dependencies:
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-29 20:30:01 +01:00
Arnaud Gourlay 8e82d1fb66 Bump wal to latest revision 94edd02 (#1246) 2022-11-29 12:03:51 +01:00
Andrey Vasnetsov 9f36ac60ee Full payload update (#1245) 
* implement api to fully overwrite payload of the point

* fmt
 2022-11-28 13:46:58 +01:00
Andrey Vasnetsov b1da0a5be4 update openapi - telemetry + max_indexing_threads 2022-11-25 13:27:26 +01:00
Ivan PleshkovandAndrey Vasnetsov ac9538d5af Hnsw links memmap (#1211) 
* add on disk key

* remove obsolete graph initialization

* remove obsolete max level

* update openapi

* graph links trait

* use mmap option

* same format for ram and mmap

* fix segment unit tests

* are you happy fmt

* are you happy clippy

* fix ci and add mmap test

* review fixes

* remove unused try-from

* fix version compatibility

* avoid loading from disk during conversion

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-11-25 11:47:52 +01:00
dependabot[bot] a802b0ede4 Bump env_logger from 0.9.3 to 0.10.0 (#1243) 
Bumps [env_logger](https://github.com/rust-cli/env_logger) from 0.9.3 to 0.10.0.
- [Release notes](https://github.com/rust-cli/env_logger/releases)
- [Changelog](https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-cli/env_logger/compare/v0.9.3...v0.10.0)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-25 08:56:36 +01:00
Andrey Vasnetsov 38444d2cc7 Fix consensus freezes (#1241) 
* do not await for replica deactivation if there is a logical error

* use async mutex in raft service + integration test for deadlock

* fmt
 2022-11-24 15:36:03 +01:00
Arnaud Gourlay 52254755a0 Improve thread naming (#1240) 
* improve local shard thread naming

* setup proper runtime for toc
 2022-11-24 13:38:45 +01:00
Arnaud Gourlay 359c0a7200 Fix panic on substraction overflow (#1236) 
* Fix panic on substraction overflow

qdrant::startup] Panic occurred in file lib/collection/src/collection_manager/holders/proxy_segment.rs at line 494: attempt to subtract with overflow

* fix assignment
 2022-11-22 13:39:01 +01:00
dependabot[bot] 18a67686df Bump tokio from 1.21.2 to 1.22.0 (#1234) 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.21.2 to 1.22.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.21.2...tokio-1.22.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-21 20:28:57 +01:00
Andrey Vasnetsov 803ad06abe Extend telemetry (#1230) 
* WIP: extend telemetry api for faster cluster manager

* fmt

* clippy fix
 2022-11-21 10:26:54 +01:00
Andrey Vasnetsov 5cf0653cff update collection write consistency param on raft-snapshot apply (#1227) 2022-11-18 13:36:09 +01:00
Andrey Vasnetsov 8cc7f67f1b spawn tokio task to prevent blocking execution thread (#1226) 2022-11-18 13:35:57 +01:00
Andrey Vasnetsov 4a9081308b do not crash the service if some collection failed (#1223) 2022-11-18 13:35:46 +01:00
Andrey Vasnetsov d340d3fa4e disable exact counting in collection cluster info api (#1221) 2022-11-18 13:35:35 +01:00
Arnaud GourlayandAndrey Vasnetsov c5558ab22b Probabilistic search sampling for better limit (#1199) 
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-11-16 20:53:58 +01:00
Andrey VasnetsovandArnaud Gourlay 772326f848 Recover distributed collection (#1214) 
* WIP: shards recovering API: download snapshot

* fmt

* snapshot recovery API

* fmt

* snapshot recovery integration test + fixes

* review fixes

* review fixes 2

* Update lib/storage/src/content_manager/snapshots/recover.rs

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
 2022-11-16 10:55:23 +01:00
Andrey Vasnetsov aebadb7380 save params of collection after update (#1212) 2022-11-16 10:55:17 +01:00
Andrey Vasnetsov dad331fc26 fix colletion params update (#1208) 2022-11-09 13:10:54 +01:00
Andrey Vasnetsov cef16c90ef wait for state deactivation on replica update failure (#1200) 
* wait for state deactivation on replica update failure

* review fixes
 2022-11-09 12:58:22 +01:00
dependabot[bot] 8504cf502a Bump num_cpus from 1.13.1 to 1.14.0 (#1203) 2022-11-09 08:31:47 +00:00
dependabot[bot] 50a6c10c1d Bump ordered-float from 3.3.0 to 3.4.0 (#1204) 
Bumps [ordered-float](https://github.com/reem/rust-ordered-float) from 3.3.0 to 3.4.0.
- [Release notes](https://github.com/reem/rust-ordered-float/releases)
- [Commits](https://github.com/reem/rust-ordered-float/compare/v3.3.0...v3.4.0)

---
updated-dependencies:
- dependency-name: ordered-float
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-09 08:42:10 +01:00
dependabot[bot] aa3139b748 Bump indicatif from 0.17.1 to 0.17.2 (#1202) 
Bumps [indicatif](https://github.com/console-rs/indicatif) from 0.17.1 to 0.17.2.
- [Release notes](https://github.com/console-rs/indicatif/releases)
- [Commits](https://github.com/console-rs/indicatif/compare/0.17.1...0.17.2)

---
updated-dependencies:
- dependency-name: indicatif
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-09 08:41:44 +01:00
dependabot[bot] 03e550b05d Bump env_logger from 0.9.1 to 0.9.3 (#1201) 
Bumps [env_logger](https://github.com/env-logger-rs/env_logger) from 0.9.1 to 0.9.3.
- [Release notes](https://github.com/env-logger-rs/env_logger/releases)
- [Changelog](https://github.com/env-logger-rs/env_logger/blob/main/CHANGELOG.md)
- [Commits](https://github.com/env-logger-rs/env_logger/compare/v0.9.1...v0.9.3)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-09 08:41:25 +01:00
dependabot[bot] 558d12b255 Bump pprof from 0.10.1 to 0.11.0 (#1188) 
Bumps [pprof](https://github.com/tikv/pprof-rs) from 0.10.1 to 0.11.0.
- [Release notes](https://github.com/tikv/pprof-rs/releases)
- [Changelog](https://github.com/tikv/pprof-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tikv/pprof-rs/commits)

---
updated-dependencies:
- dependency-name: pprof
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-07 14:41:44 +01:00
Arnaud Gourlayallcontributors[bot]Andrey Vasnetsov
 		98fc5b2fa4 0.11.1 cherry picks (#1193) 
* Complete feature road-map

* docs: add erare-humanum as a contributor for code (#1181)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* update rust version and fix clippy complains (#1190)

* handle collections transition from single-node to cluster (#1189)

* Update to v0.11.1 (#1191)

* update to v0.11.1

* also default version

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>

* consensus_state to consensus_state

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
 2022-11-07 11:31:12 +01:00
Andrey Vasnetsov 383100ebb6 remove suggesting changes in replications on replication factor change (#1177) 2022-10-31 09:57:22 +01:00
Arnaud GourlayEgor IvkovAndrey Vasnetsov
 		286ca9f5e4 0.11.0 cherry picks (#1174) 
* implement concern-factor, forbig disabling the last node, improve tes… (#1168)

* implement concern-factor, forbig disabling the last node, improve test stability

* upd docs

* improve test_recover_dead_node

* rename to write_consistency_factor

* Fix bug: applied index higher than commit on restart (#1172)

* create collection with default status = dead + await for activation on create_collecton_op_submit

* fmt

* fix unit tests

Co-authored-by: Egor Ivkov <e.o.ivkov@gmail.com>

* upd integration tests

* fix transaction handling

* upd protobuf

* upd to v0.11.0

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Co-authored-by: Egor Ivkov <e.o.ivkov@gmail.com>
 2022-10-27 10:16:21 +02:00