upstream-archive/qdrant
1
0
Fork 0
mirror of https://github.com/qdrant/qdrant.git synced 2026-10-02 10:57:53 -05:00
Code Issues Packages Projects Releases Wiki Activity
1,198 Commits 453 Branches 119 Tags
2aff9fc8a3a93c1ead02bdda7bb6bca4e09efe48
Commit Graph
746 Commits
Author SHA1 Message Date
Ivan Pleshkov 2aff9fc8a3 vector index as enum 2023-03-08 21:02:59 +00:00
Ivan Pleshkov 41d5ecd63b separate payload and vector index bases 2023-03-08 20:00:08 +00:00
Andrey Vasnetsov 167fb518da Fix recover snapshot peer (#1534) 
* allow to recover distributed snapshot on local intance + allow recover non-existing collections

* async snapshot recovery

* fix tests

* be polite earlier
 2023-03-07 17:03:01 +01:00
Andrey Vasnetsov 4f74c74998 Fix connections flood bug (#1522) 
* fixed excessive requests during search

* fmt
 2023-03-04 21:49:43 +01:00
Ivan PleshkovandAndrey Vasnetsov 5174388ab4 integrate quantized data to storages (#1311) 
* integrate quantized data to storages

* revert gitignore

* are you happy clippy

* quantize in optimizer

* provide flag

* fix segfault

* skip quantization flag, update scores

* use quantization flag

* are you happy fmt

* use quantization flag

* quantized search test

* are you happy fmt

* refactor test, refactor scorer choosing

* are you happy fmt

* run quantization on segment builder

* decrease testing parameters

* simplify segment

* update version

* remove use_quantization flag

* provide quantization config

* quantization version up

* euclid dist

* add euclid test

* saveload

* fix initialization bugs

* quantization lib version up

* fix arm build

* refactor scorer selecting

* quant lib version up

* are you happy fmt

* are you happy fmt

* are you happy clippy

* add save/load test for simple storage

* add comments

* quantiles

* quantization mmap

* remove f32

* mmap test

* fix mmap slice

* fix mmap test

* use chunks for quantization storage

* fix build

* are you happy fmt

* update quantization library

* update quantization lib

* update quantization lib

* integrate api changes

* are you happy fmt

* change quantization api

* additional checks in tests

* update quantization version

* fix unit tests

* add quantization to storage config

* use quantization for all cardinality search cases

* Integrate quantization suggestions 2 (#1520)

* review api

* wip: refactor quantization integrations

* wip: refactor quantization integrations

* wip: fmt

* include quantization into snapshot

* fmt

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2023-03-03 17:46:17 +01:00
dependabot[bot] 2f21b05c4c Bump tokio from 1.25.0 to 1.26.0 (#1517) 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.25.0 to 1.26.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.25.0...tokio-1.26.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-03 08:18:04 +01:00
Arnaud Gourlay d13e38dcff Wait for all shards to be active in when testing collection recovery (#1508) 2023-03-01 14:29:30 +01:00
Ivan Pleshkov 83efed1fcf fix windows warning unused parallel_graph_build (#1513) 2023-03-01 11:10:22 +01:00
dependabot[bot] b5ded897a2 Bump atomicwrites from 0.3.1 to 0.4.0 (#1511) 
Bumps [atomicwrites](https://github.com/untitaker/rust-atomicwrites) from 0.3.1 to 0.4.0.
- [Release notes](https://github.com/untitaker/rust-atomicwrites/releases)
- [Commits](https://github.com/untitaker/rust-atomicwrites/compare/0.3.1...0.4.0)

---
updated-dependencies:
- dependency-name: atomicwrites
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-01 08:35:21 +01:00
dependabot[bot] 830c0bf8d5 Bump prost-types from 0.11.6 to 0.11.8 (#1509) 
Bumps [prost-types](https://github.com/tokio-rs/prost) from 0.11.6 to 0.11.8.
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](https://github.com/tokio-rs/prost/compare/v0.11.6...v0.11.8)

---
updated-dependencies:
- dependency-name: prost-types
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-01 07:17:03 +01:00
Andrey Vasnetsov 791f6c33c1 allow to use optimizer_config rest api, to avoid confusion with param name in collection_info api (#1507) 2023-02-28 10:20:32 +01:00
dependabot[bot] d6e6d75ff1 Bump schemars from 0.8.11 to 0.8.12 (#1505) 
Bumps [schemars](https://github.com/GREsau/schemars) from 0.8.11 to 0.8.12.
- [Release notes](https://github.com/GREsau/schemars/releases)
- [Changelog](https://github.com/GREsau/schemars/blob/master/CHANGELOG.md)
- [Commits](https://github.com/GREsau/schemars/compare/v0.8.11...v0.8.12)

---
updated-dependencies:
- dependency-name: schemars
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-28 09:51:03 +01:00
dependabot[bot] b419a5929b Bump tempfile from 3.3.0 to 3.4.0 (#1506) 
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.3.0 to 3.4.0.
- [Release notes](https://github.com/Stebalien/tempfile/releases)
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/NEWS)
- [Commits](https://github.com/Stebalien/tempfile/commits)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-28 09:50:24 +01:00
dependabot[bot] 5acc496462 Bump prost from 0.11.7 to 0.11.8 (#1504) 
Bumps [prost](https://github.com/tokio-rs/prost) from 0.11.7 to 0.11.8.
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](https://github.com/tokio-rs/prost/commits/v0.11.8)

---
updated-dependencies:
- dependency-name: prost
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-28 09:49:26 +01:00
Arnaud Gourlay ea485a9029 Fix condition check for values_count (#1502) 2023-02-28 09:47:28 +01:00
Andrey Vasnetsov 65953b85d4 allow null in snapshot priority (#1493) 2023-02-24 00:44:05 +01:00
dependabot[bot] 6840df3de1 Bump prost from 0.11.6 to 0.11.7 (#1495) 
Bumps [prost](https://github.com/tokio-rs/prost) from 0.11.6 to 0.11.7.
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](https://github.com/tokio-rs/prost/commits)

---
updated-dependencies:
- dependency-name: prost
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-23 21:18:10 +01:00
Arnaud GourlayandAndrey Vasnetsov 2770a24766 Nested payload filters (#1487) 
* Nested payload filters

* close ToDo + add parsing of multuiple array values

* fmt

* improve testing nested arrays

* fix NumericIndex to accumulate points_to_values mapping

* revert numberic index + strict array field access

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2023-02-23 15:57:12 +01:00
Roman Titov 77f2acd11e Fix collection recovery during Raft snapshot application (#1475) 
- Fix remote shards state recovery during Raft snapshot application
- Fix local shards data recovery during Raft snapshot application
- Refactor `test_collection_recovery` test
 2023-02-23 23:13:22 +10:00
Andrey Vasnetsov 5849adca91 add listener shard state (#1482) 
* add listener shard state

* add to config

* add test
 2023-02-23 14:11:59 +01:00
dependabot[bot] a2ff17dc7f Bump memmap2 from 0.5.9 to 0.5.10 (#1491) 
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.5.9 to 0.5.10.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases)
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.5.9...v0.5.10)

---
updated-dependencies:
- dependency-name: memmap2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-23 10:56:20 +01:00
dependabot[bot] 349bdfe40f Bump memmap2 from 0.5.8 to 0.5.9 (#1479) 
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.5.8 to 0.5.9.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases)
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.5.8...v0.5.9)

---
updated-dependencies:
- dependency-name: memmap2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-20 09:48:15 +01:00
Arnaud Gourlay 3803af502d Access payload key through nested Array (#1465) 
* Access payload key through nested Array

* support removal through array index

* propagate type changes

* fmt

* avoid allocating Vec of one element
 2023-02-17 17:41:28 +01:00
Luis CossíoandAndrey Vasnetsov c15fec2a2a feat: add "any" to "match" statements (#1466) 
* feat: add "any" to "match" statements

Adds support for using "any" to match on a list of different possible values

* enable proper counting in cardinality estimation

* fmt

* upd OpenAPI

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2023-02-15 11:11:20 +01:00
Roman Titov cf04b1a92b Add /cluster/recover API endpoint (#1451) 
- Add required internal `raft::raw_node::RawNode::report_snapshot` calls
- Add methods to propagate `raft::raw_node::RawNode::request_snapshot` call to the consensus thread
- Add `/cluster/recover` API endpooint
- Add `test_collection_recovery.py` integration test (disabled on CI; documents current, incorrect behavior)
 2023-02-15 04:19:11 +10:00
dependabot[bot] 97aaeb100f Bump rocksdb from 0.20.0 to 0.20.1 (#1464) 
Bumps [rocksdb](https://github.com/rust-rocksdb/rust-rocksdb) from 0.20.0 to 0.20.1.
- [Release notes](https://github.com/rust-rocksdb/rust-rocksdb/releases)
- [Changelog](https://github.com/rust-rocksdb/rust-rocksdb/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-rocksdb/rust-rocksdb/compare/v0.20.0...v0.20.1)

---
updated-dependencies:
- dependency-name: rocksdb
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-10 20:30:27 +01:00
dependabot[bot]andIvan Pleshkov 1204682000 Bump rocksdb from 0.19.0 to 0.20.0 (#1461) 
* Bump rocksdb from 0.19.0 to 0.20.0

Bumps [rocksdb](https://github.com/rust-rocksdb/rust-rocksdb) from 0.19.0 to 0.20.0.
- [Release notes](https://github.com/rust-rocksdb/rust-rocksdb/releases)
- [Changelog](https://github.com/rust-rocksdb/rust-rocksdb/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-rocksdb/rust-rocksdb/compare/v0.19.0...v0.20.0)

---
updated-dependencies:
- dependency-name: rocksdb
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix api changes

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>
 2023-02-10 10:22:34 +01:00
Andrey Vasnetsov 153d484489 fix file hamdlers release order (#1462) 2023-02-10 08:44:16 +01:00
Andrey Vasnetsov ac251358b9 Fix flacky windows test (#1448) 
* use async sleep

* fmt

* disablee test_parallel_graph_build on CI on windows untill there is a way to reproduce
 2023-02-09 20:48:00 +01:00
Roman Titov 1be2f67a2e Reimplement ExtendedPointId deserialization (#1453) 
Reimplement `ExtendedPointId` deserialization, to produce better error message when deserialization fails
 2023-02-10 02:36:36 +10:00
Andrey Vasnetsov 26551c64ef rollback custom deserialization for point id 2023-02-08 14:13:57 +01:00
Arnaud Gourlay 8ab4b1f0d4 disable test_task_stop on Windows (#1443) 2023-02-07 17:16:03 +01:00
dependabot[bot] f0f01a7e0f Bump proptest from 1.0.0 to 1.1.0 (#1438) 
Bumps [proptest](https://github.com/proptest-rs/proptest) from 1.0.0 to 1.1.0.
- [Release notes](https://github.com/proptest-rs/proptest/releases)
- [Changelog](https://github.com/proptest-rs/proptest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/proptest-rs/proptest/compare/1.0.0...v1.1.0)

---
updated-dependencies:
- dependency-name: proptest
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-06 19:44:38 +01:00
dependabot[bot] 6704ea5ad7 Bump anyhow from 1.0.68 to 1.0.69 (#1437) 
Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.68 to 1.0.69.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.68...1.0.69)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-06 19:43:14 +01:00
Andrey Vasnetsov 2254284455 add extra timeout for windows (#1436) 
* add extra timeout for windows

* fix
 2023-02-06 19:17:42 +01:00
Andrey Vasnetsov f9f03d5ccd fix test_task_stop for windows 2023-02-06 15:07:35 +01:00
Andrey Vasnetsov 39cc731611 Windows support (#1432) 
* windows-support

* fmt

* install protoc with action

* upd WAL revision
 2023-02-06 14:00:23 +01:00
Andrey Vasnetsov 99eaabc31a remove deprecate point ids param in gprc (#1305) 
* remove deprecate point ids param in gprc

* upd docs
 2023-02-06 13:37:18 +01:00
dependabot[bot] 9f6fccece0 Bump fs_extra from 1.2.0 to 1.3.0 (#1428) 
Bumps [fs_extra](https://github.com/webdesus/fs_extra) from 1.2.0 to 1.3.0.
- [Release notes](https://github.com/webdesus/fs_extra/releases)
- [Commits](https://github.com/webdesus/fs_extra/commits)

---
updated-dependencies:
- dependency-name: fs_extra
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-03 19:51:10 +01:00
Roman TitovandAndrey Vasnetsov a50e7bdcd4 Add read_consistency parameter to the APIs (#1371) (#1407) 
* WIP: Add `read_consistency` parameter to the APIs

* WIP: Add `read_consistency` parameter to the APIs

TODO:
- Add documentation

* `cargo fmt`

* Add gRPC documentation

* Add OpenAPI documentation

* Cleanup

* fixup! Add OpenAPI documentation

* fixup! Add gRPC documentation

Who would have known there's `generate_grpc_docs.sh`!? 🥲🙈🤦‍♀️

* generate openapi

* Fix `read_consistency` query parameter deserialization

* Further improve `read_consistency` query parameter deserialization

* `cargo clippy`

* Fix `Payload` comparison during read operation result resolving

* Fix grammar

* rename `read_consistency` -> `consistency` and add integration test

* use majority for test

* fix tests

* Fix tests

* fixup! Fix tests

Apply the same fix to `ScoredPoint`

* Remove an `unwrap`

* fixup! Fix tests

Gotta love those negative conditions, or how a missed `!` can ruin your day... 🤦‍♀️

* Make internal API calls strictly "local-shard only"

* Implement a few basic traits for `ResolverRecord`

* fixup! Implement a few basic traits for `ResolverRecord`

* Revert "Make internal API calls strictly "local-shard only""

This reverts commit 25378e61ff.

* Fix `Record::payload` and `ScoredPoint::payload` serialization

* Revert "Fix `Record::payload` and `ScoredPoint::payload` serialization"

This reverts commit b566bea49b.

* Fix `Record::payload` and `ScoredPoint::payload` visibility

* fixup! Fix `Record::payload` and `ScoredPoint::payload` visibility

Remove `todo!()`

* refactoring

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2023-02-03 01:12:28 +01:00
Andrey Vasnetsov e3066b96fc allow to overwrite max number of cpus with env variable (#1425) 2023-02-02 14:44:03 +01:00
dependabot[bot] 7abb9754b3 Bump uuid from 1.2.2 to 1.3.0 (#1420) 
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.2.2 to 1.3.0.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/1.2.2...1.3.0)

---
updated-dependencies:
- dependency-name: uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-01 19:59:41 +01:00
dependabot[bot] 04a5f253ee Bump async-trait from 0.1.63 to 0.1.64 (#1417) 
Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.63 to 0.1.64.
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.63...0.1.64)

---
updated-dependencies:
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-01-31 20:00:01 +01:00
Arnaud Gourlay 68b34b1550 Implement Deserialize for ExtendedPointId to improve error reporting (#1416) 
* Implement Deserialize for ExtendedPointId to improve error reporting

* better error message
 2023-01-31 19:57:39 +01:00
Arnaud Gourlay ad9558d208 Enable ordering of update requests on replica leader (#1410) 
* Enable ordering of update requests on replica leader

* add time for replication
 2023-01-31 15:06:56 +01:00
dependabot[bot] 69fef4d743 Bump futures from 0.3.25 to 0.3.26 (#1412) 
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.25 to 0.3.26.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.25...0.3.26)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-01-30 20:16:12 +01:00
dependabot[bot] efc51edbb8 Bump tokio from 1.24.2 to 1.25.0 (#1413) 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.24.2 to 1.25.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/commits/tokio-1.25.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-01-30 19:43:24 +01:00
Arnaud Gourlay 90a2c26267 Fix dev branch after merge issue (#1409) 2023-01-30 17:10:00 +01:00
Arnaud Gourlay 2d9e015905 Forward write request according to write consistency (#1405) 
* Forward write request according to write consistency

* use gRPC internal update API by making shard_id optional

* improve and test

* mark leader peer as failed in case of service error during forward

* forward write ordering param
 2023-01-30 15:02:21 +01:00
Andrey Vasnetsov 9826e0338c add init state to replica set (#1404) 
* add init state to replica set

* update openapi

* disallow activating shard from non-initialize state

* fmt

* remove useless operation confirm on collection creation

* clippy fixes

* fix test
 2023-01-30 14:58:34 +01:00