upstream-archive/qdrant
1
0
Fork 0
mirror of https://github.com/qdrant/qdrant.git synced 2026-08-07 02:20:55 -05:00
Code Issues Packages Projects Releases Wiki Activity
5,465 Commits 346 Branches 118 Tags
94de0e2d2dcf997bf26bec34dad407d8de9a9abb
Commit Graph

3618 Commits

Author SHA1 Message Date
Egor Ivkov
4a6b185dad Retry bootstrap if term is 0 (#776) 2022-07-05 10:21:02 +02:00
Andrey Vasnetsov
098bc4c751 Count api (#777) 
* count api

* test for approx counting

* fmt + clippy

* unit test

* add warning
 2022-07-05 09:33:38 +02:00
dependabot[bot]
ebb5b3da3c Bump geo from 0.22.0 to 0.22.1 (#779) 
Bumps [geo](https://github.com/georust/geo) from 0.22.0 to 0.22.1.
- [Release notes](https://github.com/georust/geo/releases)
- [Changelog](https://github.com/georust/geo/blob/main/CHANGES.md)
- [Commits](https://github.com/georust/geo/compare/geo-0.22.0...geo-0.22.1)

---
updated-dependencies:
- dependency-name: geo
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-07-04 21:58:11 +02:00
Arnaud Gourlay
b23d3c7243 gRPC API for snapshot (#772) 2022-07-04 19:14:40 +02:00
Egor Ivkov
1753811e6a Property tests for Storage trait impl (#760) 
* Property tests for Storage trait impl

* Review fix: add comments to `CollectionContainer`
 2022-07-04 15:38:09 +03:00
Andrey Vasnetsov
f612460f88 merge optimizer should try to take all possible segments, not just 3 (#775) 
* merge optimizer should try to take all possible segments, not just 3

* prevent creating optimizing tasks if it reaches max available threads
 2022-07-04 14:35:15 +04:00
Andrey Vasnetsov
e983b07a15 Parallel hnsw building (#773) 
* parallel hnsw building

* improve hnsw payload blocks condition

* update indexing optimizer condition

* fmt
 2022-07-03 01:14:05 +02:00
Andrey Vasnetsov
fac87018c4 Snapshoting API (#764) 
* wip: rest api for snapshots

* fmt

* fix tests

* fmt

* wip: collection snapshoting and recovery

* fmt

* remote shard snapshots

* remote shard test

* fmt

* extend proxy snapshot test + fix double read lock

* fmt + clippy

* openapi schema

* Update openapi/openapi-snapshots.ytt.yaml

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>

* Update openapi/openapi-snapshots.ytt.yaml

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>

* Update src/main.rs

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>

* Update src/main.rs

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>

* reduce writes on snapshots location

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
 2022-07-01 13:21:57 +02:00
Arnaud Gourlay
cb82b7be01 Fix clippy 1.62 (#771) 
* clippy let_unit_value

error: this let-binding has unit value
   --> lib/segment/src/vector_storage/simple_vector_storage.rs:397:9
    |
397 | /         let _top_idx = match closest.get(0) {
398 | |             Some(scored_point) => {
399 | |                 assert_ne!(scored_point.idx, 2);
400 | |                 assert_eq!(&raw_res1[scored_point.idx as usize], scored_point);
...   |
404 | |             }
405 | |         };
    | |__________^
    |
    = note: `-D clippy::let-unit-value` implied by `-D warnings`
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#let_unit_value
help: omit the `let` binding
    |
397 ~         match closest.get(0) {
398 +             Some(scored_point) => {
399 +                 assert_ne!(scored_point.idx, 2);
400 +                 assert_eq!(&raw_res1[scored_point.idx as usize], scored_point);
401 +             }
402 +             None => {

* clippy format_push_string

error: `format!(..)` appended to existing `String`
   --> src/main.rs:224:21
    |
224 |                     error.push_str(&format!("Deadlock #{}\n", i));
    |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: `-D clippy::format-push-string` implied by `-D warnings`
    = help: consider using `write!` to avoid the extra allocation
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#format_push_string

error: `format!(..)` appended to existing `String`
   --> src/main.rs:226:25
    |
226 | /                         error.push_str(&format!(
227 | |                             "Thread Id {:#?}\n{:#?}\n",
228 | |                             t.thread_id(),
229 | |                             t.backtrace()
230 | |                         ));
    | |__________________________^
    |
    = help: consider using `write!` to avoid the extra allocation
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#format_push_string

error: could not compile `qdrant` due to 2 previous errors
warning: build failed, waiting for other jobs to finish...
error: could not compile `qdrant` due to 2 previous errors
 2022-07-01 01:59:49 +02:00
Arnaud Gourlay
db9e5a3abb SegmentHolder snapshot all segments (#759) 
* SegmentHolder snapshot all segments

* snapshot proxy segment

* release segment resources

* use write_version in points deletion
 2022-06-30 13:50:15 +02:00
Andrey Vasnetsov
9abb73a431 v0.8.3 (#755) 
* v0.8.3

* update rest schema
 2022-06-28 11:51:29 +02:00
Arnaud Gourlay
101cf221c1 internal point upserts support PointsBatch (#756) 2022-06-28 11:27:36 +02:00
dependabot[bot]
4abc9d1689 Bump pprof from 0.9.1 to 0.10.0 (#754) 
Bumps [pprof](https://github.com/tikv/pprof-rs) from 0.9.1 to 0.10.0.
- [Release notes](https://github.com/tikv/pprof-rs/releases)
- [Changelog](https://github.com/tikv/pprof-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tikv/pprof-rs/compare/v0.9.1...v0.10.0)

---
updated-dependencies:
- dependency-name: pprof
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-06-28 09:43:39 +02:00
Andrey Vasnetsov
92c05bb56f set max number of open files for rocksdb 2022-06-28 02:22:08 +02:00
Andrey Vasnetsov
8ac3eef4c7 fix db flush ordering to prevent persisting versions for un-persisted points 2022-06-28 00:38:10 +02:00
Arnaud Gourlay
9129a3d6eb introduce segment-level snapshots (#751) 2022-06-27 18:58:39 +02:00
Andrey Vasnetsov
850e937c2a Storage points tracking refactoring (#750) 
* segment refactoring

* rm points iterator

* fmt
 2022-06-27 15:17:09 +02:00
Andrey Vasnetsov
83c21450c3 Pagination (#743) 
* offset param for search and recomendation

* test and fix for pagination

* clippy + fmt

* upd schemas

* fix grpc requests in test
 2022-06-27 13:22:02 +02:00
dependabot[bot]
b7a2e6e001 Bump geo from 0.21.0 to 0.22.0 (#742) 
Bumps [geo](https://github.com/georust/geo) from 0.21.0 to 0.22.0.
- [Release notes](https://github.com/georust/geo/releases)
- [Changelog](https://github.com/georust/geo/blob/main/CHANGES.md)
- [Commits](https://github.com/georust/geo/compare/geo-0.21.0...geo-0.22.0)

---
updated-dependencies:
- dependency-name: geo
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-06-24 21:12:53 +02:00
Andrey Vasnetsov
c6405efb1f v0.8.2 (#740) 2022-06-24 13:14:48 +02:00
Arnaud Gourlay
83115ef586 hot fix to ensure bin compat. of RaftState (#739) 2022-06-24 10:34:52 +02:00
Egor Ivkov
e12b91e10c Promote learner (#735) 
* Promote learner

Also fixes ERROR voters not found

* Review fixes

* Review: Find learner first
 2022-06-24 11:27:58 +03:00
Arnaud Gourlay
97a5a68e48 Init. Raft config with last applied entry (#737) 2022-06-24 10:12:13 +02:00
Andrey Vasnetsov
10b468620c Storage bug fixes (#736) 
* fix: dont use shared tombstone set to estimate number of removed vectors

* fix: unlock collection before removing temp segments and attempt removal with timeout

* clippy & fmt

* review fixes

* fmt

* fmt
 2022-06-23 20:19:29 +02:00
Egor Ivkov
17a6b25e29 Add new peers as learners (#710) 
* Add new peers as learners

* Fix Storage impl

* test added

* Test fixes
 2022-06-22 19:50:27 +03:00
Andrey Vasnetsov
4691521daa Fix possible double read deadlock (#725) 2022-06-22 15:00:49 +02:00
Arnaud Gourlay
025b8b10de use latest raft-rs (#722) 2022-06-22 09:39:30 +02:00
Arnaud Gourlay
fa5a501e25 Remove last sleep consensus test (#720) 
* remove sleep by polling peer addition

* cleanup
 2022-06-21 17:59:59 +02:00
Egor Ivkov
d52dff2aa1 Fix entry reapplication on restart (#715) 
* Fix entry reapplication on restart

* Review fixes

* Rename `UnappliedEntries` to `EntryApplyProgressQueue`

* Remove `snapshot_was_applied`
 2022-06-21 13:28:44 +03:00
dependabot[bot]
5157983e2f Bump tower from 0.4.12 to 0.4.13 (#717) 
Bumps [tower](https://github.com/tower-rs/tower) from 0.4.12 to 0.4.13.
- [Release notes](https://github.com/tower-rs/tower/releases)
- [Commits](https://github.com/tower-rs/tower/compare/tower-0.4.12...tower-0.4.13)

---
updated-dependencies:
- dependency-name: tower
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-06-20 22:12:26 +02:00
Arnaud Gourlay
48f56d88a0 there are no readers for on_consensus_op - it can be a simple Mutex (#714) 2022-06-20 14:07:49 +02:00
Arnaud Gourlay
ac294bf926 remove unecessary async fn on cluster status (#713) 2022-06-20 12:41:29 +02:00
Arnaud Gourlay
7cb7a9d7a8 Fix deadlock in cluster status. (#712) 
* fix deadlock - do not reacquire read lock in the same thread

* integration tests use deadlock detector
 2022-06-17 22:31:34 +02:00
Andrey Vasnetsov
9871cace14 upd version to 0.8.1 (#708) 2022-06-16 15:30:46 +02:00
Andrey Vasnetsov
40245c5e30 Cifs dir delete (#705) 
* consume objects before removing the data

* fmt

* rm debug code
 2022-06-16 12:38:43 +02:00
Egor Ivkov
e9cb75bf90 Fix unapplied entries len (#703) 2022-06-15 19:29:40 +03:00
Egor Ivkov
533fd208b6 Move out consensus logic from ToC (#675) 2022-06-15 16:18:26 +03:00
Andrey Vasnetsov
c82f11956b Better logging (#681) 
* disable info level for non-essential logging

* welcome banner

* fmt + clippy

* return debug log level in dev mode
 2022-06-14 10:31:02 +02:00
dependabot[bot]
e3c58b47fa Bump geo from 0.20.1 to 0.21.0 (#683) 
Bumps [geo](https://github.com/georust/geo) from 0.20.1 to 0.21.0.
- [Release notes](https://github.com/georust/geo/releases)
- [Changelog](https://github.com/georust/geo/blob/main/CHANGES.md)
- [Commits](https://github.com/georust/geo/compare/geo-0.20.1...geo-0.21.0)

---
updated-dependencies:
- dependency-name: geo
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-06-14 08:40:12 +02:00
Andrey Vasnetsov
63b868233e release v0.8.0 (#667) 
* release v0.8.0

* upd roadmap
 2022-06-08 12:47:10 +02:00
Andrey Vasnetsov
d5ed8050b7 bug fix: prevent division by zero if empty payload 2022-06-08 02:39:58 +02:00
Andrey Vasnetsov
0edbc9d1e5 bug fix: move local shard config save to collection level 2022-06-08 00:42:48 +02:00
Andrey Vasnetsov
fc2f579ec6 smarter default number of shards (#668) 
* smarter default number of shards

* upd openapi
 2022-06-07 16:59:10 +02:00
Andrey Vasnetsov
a264a8262a fix typos 2022-06-07 15:16:23 +02:00
Andrey Vasnetsov
8e52c255ab add consensus configuration + lower log level + fix flickering test 2022-06-06 21:44:03 +02:00
dependabot[bot]
c21855551f Bump tokio from 1.18.2 to 1.19.2 (#664) 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.18.2 to 1.19.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/commits)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-06-06 21:43:35 +02:00
Gabriel Velo
c15981092a [WIP] [real-time index] Implement payloadstorage for structpayloadindex (#642) 
* [real-time index] Extend FieldIndex enum and StructPayloadIndex with method from PayloadStorage

* [real-time index] add missing remove_point methods

* [real-time index] add new index to FieldIndex enum

* fix compile

* are you happy fmt

* merge load and remove

* fix test generics

* decrement points count

* remove from histogram

* simplify histogram usage

* [real-time index] remove old tests and fix clippy warnings

* histogram: method to derive range by size (#657)

* [real-time index] add histogram based payload_blocks implementation.

* payload blocks

* fmt

* clippy

* [real-time index] refactor Segment to use PayloadIndex instead of PayloadStorage.

* fix tests

* fmt

* clippy

* rename indexes

* remove redundent params

* add struct payload deletion test + fix delete payload in map index

* remove payload threshold

Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-06-06 17:14:20 +02:00
Egor Ivkov
9ddaa1c749 Add leader_id and role to cluster API (#660) 2022-06-03 22:37:50 +02:00
dependabot[bot]
af97733467 Bump async-trait from 0.1.53 to 0.1.56 (#663) 
Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.53 to 0.1.56.
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.53...0.1.56)

---
updated-dependencies:
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-06-03 22:34:22 +02:00
Andrey Vasnetsov
f92535aa39 update qdrant.rs 2022-06-02 19:15:35 +02:00