upstream-archive/qdrant
1
0
Fork 0
mirror of https://github.com/qdrant/qdrant.git synced 2026-10-03 11:27:37 -05:00
Code Issues Packages Projects Releases Wiki Activity
1,009 Commits 453 Branches 119 Tags
d340d3fa4e7fa471db241ca5b9b8545b48c28721
Commit Graph
610 Commits
Author SHA1 Message Date
Andrey Vasnetsov d340d3fa4e disable exact counting in collection cluster info api (#1221) 2022-11-18 13:35:35 +01:00
Arnaud Gourlay 6266d9bcdc enable compression for gRPC (#1186) 
* enable compression for gRPC

* add benchmarks

* fix grpc benchmark

* discardResponseBodies to reduce memory usage

* improve & split benchmarks

* do not use gzip for inter node communication as benchmarks are not conclusive
 2022-11-18 13:34:49 +01:00
Arnaud GourlayandAndrey Vasnetsov c5558ab22b Probabilistic search sampling for better limit (#1199) 
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-11-16 20:53:58 +01:00
Andrey VasnetsovandArnaud Gourlay 772326f848 Recover distributed collection (#1214) 
* WIP: shards recovering API: download snapshot

* fmt

* snapshot recovery API

* fmt

* snapshot recovery integration test + fixes

* review fixes

* review fixes 2

* Update lib/storage/src/content_manager/snapshots/recover.rs

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
 2022-11-16 10:55:23 +01:00
Andrey Vasnetsov aebadb7380 save params of collection after update (#1212) 2022-11-16 10:55:17 +01:00
dependabot[bot] 96a99935e9 Bump memmap2 from 0.5.7 to 0.5.8 (#1210) 
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.5.7 to 0.5.8.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases)
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.5.7...v0.5.8)

---
updated-dependencies:
- dependency-name: memmap2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-09 21:51:07 +01:00
Andrey Vasnetsov dad331fc26 fix colletion params update (#1208) 2022-11-09 13:10:54 +01:00
Andrey Vasnetsov cef16c90ef wait for state deactivation on replica update failure (#1200) 
* wait for state deactivation on replica update failure

* review fixes
 2022-11-09 12:58:22 +01:00
dependabot[bot] 8504cf502a Bump num_cpus from 1.13.1 to 1.14.0 (#1203) 2022-11-09 08:31:47 +00:00
dependabot[bot] 50a6c10c1d Bump ordered-float from 3.3.0 to 3.4.0 (#1204) 
Bumps [ordered-float](https://github.com/reem/rust-ordered-float) from 3.3.0 to 3.4.0.
- [Release notes](https://github.com/reem/rust-ordered-float/releases)
- [Commits](https://github.com/reem/rust-ordered-float/compare/v3.3.0...v3.4.0)

---
updated-dependencies:
- dependency-name: ordered-float
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-09 08:42:10 +01:00
dependabot[bot] aa3139b748 Bump indicatif from 0.17.1 to 0.17.2 (#1202) 
Bumps [indicatif](https://github.com/console-rs/indicatif) from 0.17.1 to 0.17.2.
- [Release notes](https://github.com/console-rs/indicatif/releases)
- [Commits](https://github.com/console-rs/indicatif/compare/0.17.1...0.17.2)

---
updated-dependencies:
- dependency-name: indicatif
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-09 08:41:44 +01:00
dependabot[bot] 03e550b05d Bump env_logger from 0.9.1 to 0.9.3 (#1201) 
Bumps [env_logger](https://github.com/env-logger-rs/env_logger) from 0.9.1 to 0.9.3.
- [Release notes](https://github.com/env-logger-rs/env_logger/releases)
- [Changelog](https://github.com/env-logger-rs/env_logger/blob/main/CHANGELOG.md)
- [Commits](https://github.com/env-logger-rs/env_logger/compare/v0.9.1...v0.9.3)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-09 08:41:25 +01:00
Andrey Vasnetsov 7e4424ab57 Cosine dist zero vectors (#1198) 
* skip pre-processing of zero-length vector for cosine distance

* fmt
 2022-11-08 17:36:20 +01:00
Ivan Pleshkov d555a244c2 Hnsw links storage (#1171) 
* hnsw compact links storage

* dump links memsize

* are you happy fmt

* fix build

* calculate capacity and add shrink

* fmt

* remove garbage

* fix unit tests

* default values for graph layers

* remove obsolete fields

* add comments

* more comments

* use one flattened links structure

* refactor and add unit tests

* move merge into graph builder

* fix messages in test_hnsw_graph_properties
 2022-11-08 13:42:50 +04:00
dependabot[bot] 558d12b255 Bump pprof from 0.10.1 to 0.11.0 (#1188) 
Bumps [pprof](https://github.com/tikv/pprof-rs) from 0.10.1 to 0.11.0.
- [Release notes](https://github.com/tikv/pprof-rs/releases)
- [Changelog](https://github.com/tikv/pprof-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tikv/pprof-rs/commits)

---
updated-dependencies:
- dependency-name: pprof
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-07 14:41:44 +01:00
Arnaud Gourlayallcontributors[bot]Andrey Vasnetsov
 		98fc5b2fa4 0.11.1 cherry picks (#1193) 
* Complete feature road-map

* docs: add erare-humanum as a contributor for code (#1181)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* update rust version and fix clippy complains (#1190)

* handle collections transition from single-node to cluster (#1189)

* Update to v0.11.1 (#1191)

* update to v0.11.1

* also default version

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>

* consensus_state to consensus_state

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
 2022-11-07 11:31:12 +01:00
Andrey Vasnetsov 383100ebb6 remove suggesting changes in replications on replication factor change (#1177) 2022-10-31 09:57:22 +01:00
Ivan PleshkovandAndrey Vasnetsov 017df52528 Use less ram for id tracker (#1176) 
* use less ram for id tracker

* are you happy clippy

* use vec for internals

* use versions for internal ids

* keys test

* Use less ram for id tracker fixes (#1182)

* WIP: internal_to_version

* fmt

* fix unit tests

* add comment

Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-10-31 10:44:00 +04:00
Andrey Vasnetsov a0b6fa81b7 Consensus q n a (#1169) 
* Expand comments and minor refactoring for raft state

* fmt

* add comments to consensus.rs
 2022-10-30 20:34:09 +01:00
erare-humanumandAndrey Vasnetsov 93a4499859 Update: unmaintained crate memmap -> memmap2 (#559) (#1160) 
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-10-30 11:50:27 +01:00
Arnaud GourlayEgor IvkovAndrey Vasnetsov
 		286ca9f5e4 0.11.0 cherry picks (#1174) 
* implement concern-factor, forbig disabling the last node, improve tes… (#1168)

* implement concern-factor, forbig disabling the last node, improve test stability

* upd docs

* improve test_recover_dead_node

* rename to write_consistency_factor

* Fix bug: applied index higher than commit on restart (#1172)

* create collection with default status = dead + await for activation on create_collecton_op_submit

* fmt

* fix unit tests

Co-authored-by: Egor Ivkov <e.o.ivkov@gmail.com>

* upd integration tests

* fix transaction handling

* upd protobuf

* upd to v0.11.0

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Co-authored-by: Egor Ivkov <e.o.ivkov@gmail.com>
 2022-10-27 10:16:21 +02:00
Andrey Vasnetsov 071d1948b0 update and improve openapi definitions 2022-10-24 16:04:31 +02:00
Andrey Vasnetsov ab19576c5d Force sync vector storage offset (#1110) 
* simplify vector storage `update` semantic, always assume inplace vector replacement

* fmt
 2022-10-24 12:34:26 +02:00
Andrey Vasnetsov 54e0857708 ignore points with missing expernal ids instead of panic (#1109) 2022-10-24 12:33:43 +02:00
Arnaud Gourlay 1ff09516a0 limit set to zero returns an empty result (#1083) 2022-10-24 12:32:46 +02:00
Andrey Vasnetsov 21754ef2ef report indexed payload points in info api + other fixes (#1164) 
* report indexed payload points in info api + other fixes

* rollback debug changes

* clippy

* clippy
 2022-10-24 09:43:02 +02:00
Ivan PleshkovandAndrey Vasnetsov 516dcd7020 Telemetry level of detail (#1049) 
* telemetry level of detail

* rename duration aggregator

* are you happy fmt

* move total searches sum

* separate levels

* optional bucket size

* search telemetry improvements

* separate web telemetry into methods

* tonic telemetry methods

* merge optimizations

* are you happy fmt

* better rounding

* qdrant configs on level 1

* provide collection params

* add peers count

* collection points count

* update openapi

* use pattern in actix telemetry

* are you happy fmt

* merge dev

* are you happy fmt

* fix merge conflicts

* update openapi

* fix build

* are you happy fmt

* add exact searches statistics

* process replica set

* update openapi

* fix wrong name

* fix naming

* fix unwrap

* review

* fmt

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-10-23 00:48:55 +02:00
Egor Ivkov a96f92cbfe Record message send failures and stop consensus on service error (#1152) 
* Record message send failures and stop consensus on service error

* Generate openapi

* Review fixes: log first error and remove on success
 2022-10-22 23:23:37 +02:00
dependabot[bot] 740c8c412b Bump futures from 0.3.24 to 0.3.25 (#1153) 
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.24 to 0.3.25.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.24...0.3.25)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-21 08:17:53 +02:00
Ivan PleshkovandAndrey Vasnetsov 44626bc80a exact search parameter (#1150) 
* exact search parameter

* exact search test

* decrease attempts count because of test speed

* minor review fixes

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-10-20 15:39:21 +04:00
Arnaud Gourlay faa4008938 Enable replication in consensus test (#1140) 2022-10-20 11:36:22 +02:00
Andrey Vasnetsov 75563b20b4 Consensus fault tollerance improvements (#1149) 
* consensus fault tollerance improvements

* fmt
 2022-10-20 11:22:22 +02:00
Andrey Vasnetsov bf48066145 Full reassign payload on upsert (#1148) 
* set_full_payload on upsert

* fmt

* fix clippy
 2022-10-20 10:26:33 +02:00
Andrey Vasnetsov 701865e528 fmt (#1145) 2022-10-20 09:05:59 +02:00
dependabot[bot] 2bd7a36006 Bump ordered-float from 3.2.0 to 3.3.0 (#1147) 
Bumps [ordered-float](https://github.com/reem/rust-ordered-float) from 3.2.0 to 3.3.0.
- [Release notes](https://github.com/reem/rust-ordered-float/releases)
- [Commits](https://github.com/reem/rust-ordered-float/compare/v3.2.0...v3.3.0)

---
updated-dependencies:
- dependency-name: ordered-float
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-20 08:56:18 +02:00
dependabot[bot] 7d9ca67ea9 Bump async-trait from 0.1.57 to 0.1.58 (#1146) 
Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.57 to 0.1.58.
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.57...0.1.58)

---
updated-dependencies:
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-20 08:55:54 +02:00
Andrey Vasnetsov 123ae1607d Await for collection created for sure (#1144) 
* run plunger operation after user request to ensure that the majority of the consensus actually applied previous operation

* fmt

* fix test

* ensure all operations are unique in consensus queue

* add extra timeout for CI tests

* fix timout param in delete peer + do not explicilty await on peer removal

* fmt
 2022-10-19 17:59:16 +02:00
Egor Ivkov d3849d712b Allow peer force removal (#1139) 
* Allow peer force removal

* Remove shards
 2022-10-19 17:01:13 +02:00
Andrey Vasnetsov 2b4ab27506 Write lock for user requests (#1143) 
* lock update operations only

* fmt

* fix

* lock openapi integration test
 2022-10-19 12:57:52 +02:00
Andrey Vasnetsov 7f8c088711 Replication transfer fixes (#1142) 
* fix validation and task assignment during replica transfer

* fmt

* upd openapi

* extend integration test
 2022-10-19 11:38:26 +02:00
Ivan PleshkovArnaud GourlayAndrey Vasnetsov
 		f50f1f4c71 #1119 disable updating feature (#1126) 
* disable updating feature

* customize error message

* update namings

* Update src/actix/api/service_api.rs

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>

* fix build

* update api

* update api and allow deletion

* update openapi

* replace Bad Request with Forbidden

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-10-18 12:58:38 +02:00
Andrey Vasnetsov cf29b67640 v0.10.5 2022-10-18 10:49:43 +02:00
Andrey Vasnetsov ef7b4ac2a7 propagate payload index to temporary segment during optimization (#1133) 2022-10-18 10:49:37 +02:00
Arnaud GourlayandAndrey Vasnetsov 9324be1d60 [Replication] Add replicas (#1085) 
* Add replicas to ReplicaSet

* unproxify shard & miscs

* no exlusive write locking while adding replicas

* on_optimizer_config_update on Shard with async. recursion

* no exlusive write locking while removing replicas

* shortcut replica set propagation #1101

* remove unused field

* make RequestShardTransfer callback sync.

* promote local & remote to replica state

* fixes for replica sync api (#1123)

* code review

* fix replica set update - fail only if all failed

* Add replica redesign (#1131)

* refactor shard/mod.rs

* wip

* fmt

* it compiles

* temporary disable replica placemeant change on replication factor

* fmt

* finish todos

* small refactoring

* remove change::add

* replica-set -> shard-replica-set

* fmt

* upd openapi

* fix finish transfer logic

* fix existing integration tests

* shard transfer validation

* fmt

* review fixes

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-10-17 20:12:10 +02:00
Andrey Vasnetsov 411ddabe49 v0.10.4 2022-10-13 09:14:11 +02:00
Andrey Vasnetsov 35ae83f3fc validate geo coordinate before adding it to the payload index (#1125) 
* validate geo coordinate before adding it to the payload index

* revert changes in api test
 2022-10-13 09:14:11 +02:00
Andrey Vasnetsov 68a46a0f2e v0.10.3 2022-10-13 09:14:11 +02:00
Andrey Vasnetsov b275d452a8 shard replica manipulation API (#1121) 2022-10-12 15:19:43 +02:00
Andrey Vasnetsov 62cc0880b8 Replica set snapshot (#1118) 
* replication set loading + snapshoting

* test + fixes

* fix typo

* fmt

* remove duplicated
 2022-10-12 12:19:08 +02:00
Egor Ivkov da039fd533 Telemetry data for replica set shard (#1115) 2022-10-11 16:31:02 +03:00