upstream-archive/qdrant
1
0
Fork 0
mirror of https://github.com/qdrant/qdrant.git synced 2026-08-06 10:00:58 -05:00
Code Issues Packages Projects Releases Wiki Activity
5,465 Commits 344 Branches 118 Tags
94de0e2d2dcf997bf26bec34dad407d8de9a9abb
Commit Graph

1780 Commits

Author SHA1 Message Date
Arnaud Gourlay
1435ea2069 Remove unused json-patch dependency (#1331) 2023-01-09 19:55:01 +01:00
Roman Titov
2ed11d544b Add configuration option to select "memory advice" used for memmapped storage 
- Add a simple abstraction/wrapper over `memmap2::Advice` and a global switch
  to select "memory advice" used for memmapped storage to the `segment` crate
- Add `mmap_advice` configuration option to the `storage::types::StorageConfig`
- Implement `search-points` benchmark
 2023-01-06 21:02:57 +10:00
dependabot[bot]
49bc997e52 Bump atomic_refcell from 0.1.8 to 0.1.9 (#1313) 
Bumps [atomic_refcell](https://github.com/bholley/atomic_refcell) from 0.1.8 to 0.1.9.
- [Release notes](https://github.com/bholley/atomic_refcell/releases)
- [Commits](https://github.com/bholley/atomic_refcell/commits)

---
updated-dependencies:
- dependency-name: atomic_refcell
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-01-04 20:56:39 +01:00
Andrey Vasnetsov
bd05a69a5f Id mapper inconsistency (#1302) 
* always flush wal

* always flush wal fix

* always flush wal fmt

* flush wal during background flush

* async wal flush

* use id-tracker internal id for next-id instead of vector storage

* add flush order and recovery comment

fix merge bug

* longer timeout in test
 2022-12-25 22:36:31 +01:00
Andrey Vasnetsov
431fcbb4ca better validation for transfer initiation 2022-12-23 21:16:13 +01:00
dependabot[bot]
16a4d54885 Bump num_cpus from 1.14.0 to 1.15.0 (#1296) 
Bumps [num_cpus](https://github.com/seanmonstar/num_cpus) from 1.14.0 to 1.15.0.
- [Release notes](https://github.com/seanmonstar/num_cpus/releases)
- [Changelog](https://github.com/seanmonstar/num_cpus/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/num_cpus/compare/v1.14.0...v1.15.0)

---
updated-dependencies:
- dependency-name: num_cpus
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-20 19:35:10 +01:00
dependabot[bot]
1f90e12eec Bump semver from 1.0.14 to 1.0.16 (#1292) 
Bumps [semver](https://github.com/dtolnay/semver) from 1.0.14 to 1.0.16.
- [Release notes](https://github.com/dtolnay/semver/releases)
- [Commits](https://github.com/dtolnay/semver/compare/1.0.14...1.0.16)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-19 14:52:56 +01:00
Arnaud Gourlay
d77debf122 Clippy for Rust 1.66 (#1284) 2022-12-16 10:53:51 +01:00
Ivan Pleshkov
02071314bd remove obsolete internal scorer (#1271) 2022-12-14 14:47:22 +04:00
Andrey Vasnetsov
aeba32ba3e Shard build refactoring (#1280) 
* refactor some code of shard creation and optimizers

* Shard build refactoring debug (#1279)

* add debig code

* debug code

* debug code

* debug code

* debug code

* debug code

* debug code

* debug code

* debug code

* debug code

* debug code

* refactor stop-checking during the optimization

* remove debug logs

* improve delete-renaming schema

* fmt

* move collection file removing to the async task

* rename check_optimization_stopped into more general check_process_stopped
 2022-12-14 10:11:46 +01:00
dependabot[bot]
fdf531955d Bump rayon from 1.6.0 to 1.6.1 (#1277) 
Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.6.0 to 1.6.1.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/compare/rayon-core-v1.6.0...rayon-core-v1.6.1)

---
updated-dependencies:
- dependency-name: rayon
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-14 09:04:07 +01:00
dependabot[bot]
760b9e8710 Bump geo from 0.23.0 to 0.23.1 (#1270) 
Bumps [geo](https://github.com/georust/geo) from 0.23.0 to 0.23.1.
- [Release notes](https://github.com/georust/geo/releases)
- [Changelog](https://github.com/georust/geo/blob/main/CHANGES.md)
- [Commits](https://github.com/georust/geo/compare/geo-0.23.0...geo-0.23.1)

---
updated-dependencies:
- dependency-name: geo
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-14 08:42:18 +01:00
dependabot[bot]
c9c9cbd300 Bump json-patch from 0.2.7 to 0.3.0 (#1276) 
Bumps [json-patch](https://github.com/idubrov/json-patch) from 0.2.7 to 0.3.0.
- [Release notes](https://github.com/idubrov/json-patch/releases)
- [Changelog](https://github.com/idubrov/json-patch/blob/main/CHANGELOG.md)
- [Commits](https://github.com/idubrov/json-patch/compare/v0.2.7...v0.3.0)

---
updated-dependencies:
- dependency-name: json-patch
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-14 08:41:54 +01:00
Andrey Vasnetsov
d05b2def10 allow set different M param for default HNSW and payload-related links (#1257) 
* allow set different M param for default HNSW and payload-related links

* review fixes
 2022-12-06 13:53:42 +01:00
dependabot[bot]
614aa69c43 Bump json-patch from 0.2.6 to 0.2.7 (#1268) 
Bumps [json-patch](https://github.com/idubrov/json-patch) from 0.2.6 to 0.2.7.
- [Release notes](https://github.com/idubrov/json-patch/releases)
- [Commits](https://github.com/idubrov/json-patch/compare/v0.2.6...v0.2.7)

---
updated-dependencies:
- dependency-name: json-patch
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-06 13:52:33 +01:00
Andrey Vasnetsov
0925d02667 include backtrace for service errors (#1256) 2022-12-06 12:31:38 +01:00
Andrey Vasnetsov
a23adcdd82 Missed points on update fix (#1255) 
* add context to point-not-found error

* fmt

* add backtrace

* use upgradable lock

* unique points in insert operation

* remove debug code
 2022-12-06 12:31:24 +01:00
Ivan Pleshkov
ac9538d5af Hnsw links memmap (#1211) 
* add on disk key

* remove obsolete graph initialization

* remove obsolete max level

* update openapi

* graph links trait

* use mmap option

* same format for ram and mmap

* fix segment unit tests

* are you happy fmt

* are you happy clippy

* fix ci and add mmap test

* review fixes

* remove unused try-from

* fix version compatibility

* avoid loading from disk during conversion

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-11-25 11:47:52 +01:00
dependabot[bot]
24f3c90140 Bump rayon from 1.5.3 to 1.6.0 (#1233) 
Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.5.3 to 1.6.0.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/compare/v1.5.3...rayon-core-v1.6.0)

---
updated-dependencies:
- dependency-name: rayon
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-21 20:28:20 +01:00
Arnaud Gourlay
c5558ab22b Probabilistic search sampling for better limit (#1199) 
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-11-16 20:53:58 +01:00
dependabot[bot]
96a99935e9 Bump memmap2 from 0.5.7 to 0.5.8 (#1210) 
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.5.7 to 0.5.8.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases)
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.5.7...v0.5.8)

---
updated-dependencies:
- dependency-name: memmap2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-09 21:51:07 +01:00
dependabot[bot]
8504cf502a Bump num_cpus from 1.13.1 to 1.14.0 (#1203) 2022-11-09 08:31:47 +00:00
dependabot[bot]
50a6c10c1d Bump ordered-float from 3.3.0 to 3.4.0 (#1204) 
Bumps [ordered-float](https://github.com/reem/rust-ordered-float) from 3.3.0 to 3.4.0.
- [Release notes](https://github.com/reem/rust-ordered-float/releases)
- [Commits](https://github.com/reem/rust-ordered-float/compare/v3.3.0...v3.4.0)

---
updated-dependencies:
- dependency-name: ordered-float
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-09 08:42:10 +01:00
Andrey Vasnetsov
7e4424ab57 Cosine dist zero vectors (#1198) 
* skip pre-processing of zero-length vector for cosine distance

* fmt
 2022-11-08 17:36:20 +01:00
Ivan Pleshkov
d555a244c2 Hnsw links storage (#1171) 
* hnsw compact links storage

* dump links memsize

* are you happy fmt

* fix build

* calculate capacity and add shrink

* fmt

* remove garbage

* fix unit tests

* default values for graph layers

* remove obsolete fields

* add comments

* more comments

* use one flattened links structure

* refactor and add unit tests

* move merge into graph builder

* fix messages in test_hnsw_graph_properties
 2022-11-08 13:42:50 +04:00
dependabot[bot]
558d12b255 Bump pprof from 0.10.1 to 0.11.0 (#1188) 
Bumps [pprof](https://github.com/tikv/pprof-rs) from 0.10.1 to 0.11.0.
- [Release notes](https://github.com/tikv/pprof-rs/releases)
- [Changelog](https://github.com/tikv/pprof-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tikv/pprof-rs/commits)

---
updated-dependencies:
- dependency-name: pprof
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-07 14:41:44 +01:00
Arnaud Gourlay
98fc5b2fa4 0.11.1 cherry picks (#1193) 
* Complete feature road-map

* docs: add erare-humanum as a contributor for code (#1181)

* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* update rust version and fix clippy complains (#1190)

* handle collections transition from single-node to cluster (#1189)

* Update to v0.11.1 (#1191)

* update to v0.11.1

* also default version

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>

* consensus_state to consensus_state

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
 2022-11-07 11:31:12 +01:00
Ivan Pleshkov
017df52528 Use less ram for id tracker (#1176) 
* use less ram for id tracker

* are you happy clippy

* use vec for internals

* use versions for internal ids

* keys test

* Use less ram for id tracker fixes (#1182)

* WIP: internal_to_version

* fmt

* fix unit tests

* add comment

Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-10-31 10:44:00 +04:00
erare-humanum
93a4499859 Update: unmaintained crate memmap -> memmap2 (#559) (#1160) 
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-10-30 11:50:27 +01:00
Andrey Vasnetsov
071d1948b0 update and improve openapi definitions 2022-10-24 16:04:31 +02:00
Andrey Vasnetsov
ab19576c5d Force sync vector storage offset (#1110) 
* simplify vector storage `update` semantic, always assume inplace vector replacement

* fmt
 2022-10-24 12:34:26 +02:00
Andrey Vasnetsov
54e0857708 ignore points with missing expernal ids instead of panic (#1109) 2022-10-24 12:33:43 +02:00
Arnaud Gourlay
1ff09516a0 limit set to zero returns an empty result (#1083) 2022-10-24 12:32:46 +02:00
Andrey Vasnetsov
21754ef2ef report indexed payload points in info api + other fixes (#1164) 
* report indexed payload points in info api + other fixes

* rollback debug changes

* clippy

* clippy
 2022-10-24 09:43:02 +02:00
Ivan Pleshkov
516dcd7020 Telemetry level of detail (#1049) 
* telemetry level of detail

* rename duration aggregator

* are you happy fmt

* move total searches sum

* separate levels

* optional bucket size

* search telemetry improvements

* separate web telemetry into methods

* tonic telemetry methods

* merge optimizations

* are you happy fmt

* better rounding

* qdrant configs on level 1

* provide collection params

* add peers count

* collection points count

* update openapi

* use pattern in actix telemetry

* are you happy fmt

* merge dev

* are you happy fmt

* fix merge conflicts

* update openapi

* fix build

* are you happy fmt

* add exact searches statistics

* process replica set

* update openapi

* fix wrong name

* fix naming

* fix unwrap

* review

* fmt

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-10-23 00:48:55 +02:00
Ivan Pleshkov
44626bc80a exact search parameter (#1150) 
* exact search parameter

* exact search test

* decrease attempts count because of test speed

* minor review fixes

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-10-20 15:39:21 +04:00
Andrey Vasnetsov
bf48066145 Full reassign payload on upsert (#1148) 
* set_full_payload on upsert

* fmt

* fix clippy
 2022-10-20 10:26:33 +02:00
dependabot[bot]
2bd7a36006 Bump ordered-float from 3.2.0 to 3.3.0 (#1147) 
Bumps [ordered-float](https://github.com/reem/rust-ordered-float) from 3.2.0 to 3.3.0.
- [Release notes](https://github.com/reem/rust-ordered-float/releases)
- [Commits](https://github.com/reem/rust-ordered-float/compare/v3.2.0...v3.3.0)

---
updated-dependencies:
- dependency-name: ordered-float
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-20 08:56:18 +02:00
Andrey Vasnetsov
ef7b4ac2a7 propagate payload index to temporary segment during optimization (#1133) 2022-10-18 10:49:37 +02:00
Arnaud Gourlay
9324be1d60 [Replication] Add replicas (#1085) 
* Add replicas to ReplicaSet

* unproxify shard & miscs

* no exlusive write locking while adding replicas

* on_optimizer_config_update on Shard with async. recursion

* no exlusive write locking while removing replicas

* shortcut replica set propagation #1101

* remove unused field

* make RequestShardTransfer callback sync.

* promote local & remote to replica state

* fixes for replica sync api (#1123)

* code review

* fix replica set update - fail only if all failed

* Add replica redesign (#1131)

* refactor shard/mod.rs

* wip

* fmt

* it compiles

* temporary disable replica placemeant change on replication factor

* fmt

* finish todos

* small refactoring

* remove change::add

* replica-set -> shard-replica-set

* fmt

* upd openapi

* fix finish transfer logic

* fix existing integration tests

* shard transfer validation

* fmt

* review fixes

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-10-17 20:12:10 +02:00
Andrey Vasnetsov
35ae83f3fc validate geo coordinate before adding it to the payload index (#1125) 
* validate geo coordinate before adding it to the payload index

* revert changes in api test
 2022-10-13 09:14:11 +02:00
dependabot[bot]
d0789f3a90 Bump uuid from 1.1.2 to 1.2.1 (#1111) 
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.1.2 to 1.2.1.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/1.1.2...1.2.1)

---
updated-dependencies:
- dependency-name: uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-11 08:25:24 +02:00
dependabot[bot]
414ba89e9b Bump ordered-float from 3.1.0 to 3.2.0 (#1090) 
Bumps [ordered-float](https://github.com/reem/rust-ordered-float) from 3.1.0 to 3.2.0.
- [Release notes](https://github.com/reem/rust-ordered-float/releases)
- [Commits](https://github.com/reem/rust-ordered-float/compare/v3.1.0...v3.2.0)

---
updated-dependencies:
- dependency-name: ordered-float
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-04 11:17:59 +02:00
dependabot[bot]
814b0ebcb4 Bump schemars from 0.8.10 to 0.8.11 (#1089) 
Bumps [schemars](https://github.com/GREsau/schemars) from 0.8.10 to 0.8.11.
- [Release notes](https://github.com/GREsau/schemars/releases)
- [Changelog](https://github.com/GREsau/schemars/blob/master/CHANGELOG.md)
- [Commits](https://github.com/GREsau/schemars/compare/v0.8.10...v0.8.11)

---
updated-dependencies:
- dependency-name: schemars
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-10-04 11:17:22 +02:00
Andrey Vasnetsov
c333e95019 fix the filter-based points remove operation in proxy segment (#1050) 
* fix the filter-based points remove operation in proxy segment

* propagate primary clause in proxy segment
 2022-09-24 01:04:51 +02:00
dependabot[bot]
3bfd81d6c6 Bump ordered-float from 3.0.0 to 3.1.0 (#1039) 
Bumps [ordered-float](https://github.com/reem/rust-ordered-float) from 3.0.0 to 3.1.0.
- [Release notes](https://github.com/reem/rust-ordered-float/releases)
- [Commits](https://github.com/reem/rust-ordered-float/compare/v3.0.0...v3.1.0)

---
updated-dependencies:
- dependency-name: ordered-float
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-09-21 10:45:46 +02:00
Andrey Vasnetsov
03d08236a8 better struct for fts (#1026) 2022-09-16 14:07:50 +02:00
dependabot[bot]
519c4db868 Bump semver from 1.0.13 to 1.0.14 (#1024) 
Bumps [semver](https://github.com/dtolnay/semver) from 1.0.13 to 1.0.14.
- [Release notes](https://github.com/dtolnay/semver/releases)
- [Commits](https://github.com/dtolnay/semver/compare/1.0.13...1.0.14)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-09-15 21:56:32 +02:00
Andrey Vasnetsov
19ac5b4750 change service error into field error (#1023) 2022-09-15 15:24:00 +02:00
Ivan Pleshkov
8c6507987b Integrate tiny vec to named vectors (#1007) 
* integration

* are you happy fmt

* are you happy clippy

* implement partial eq

* is that it?

* small renaming

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-09-15 14:45:48 +04:00