upstream-archive/qdrant
1
0
Fork 0
mirror of https://github.com/qdrant/qdrant.git synced 2026-08-06 18:10:58 -05:00
Code Issues Packages Projects Releases Wiki Activity
5,465 Commits 346 Branches 118 Tags
94de0e2d2dcf997bf26bec34dad407d8de9a9abb
Commit Graph

3618 Commits

Author SHA1 Message Date
Arnaud Gourlay
2a81fc9142 Remove sled for alias mappings (#402) 
* improve alias tests

* introduce new alias persistence and remove Sled

* introduce FileStorageError

* use new error in AliasMapping

* aliases are kept in memory and saved to disk when modified

* better naming

* make alias renaming atomic

* fmt

* use correct alias and docs

* code review: cleaner error conversion
 2022-03-23 14:21:04 +01:00
Egor Ivkov
2343078ea1 Collection meta as global state (#394) 
* Collection meta as global state

* Improve collection wal entry error handling
 2022-03-22 18:27:03 +03:00
Gabriel Velo
f69a7b740f json as payload (#306) 
add json as payload
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-03-21 07:09:10 -03:00
Arnaud Gourlay
34048c6fed index geo payload (#366) 
* index geo payload

* fix group_points_per_region

* code review: use entry API

* geohash into box for payload_blocks

* add failing test for investigation

* workaround for when exp < min

* test with 2 geo values per point

* fix rounding error in cardinality estimation

* fmt

* code review: decompose using Iterator of slices

* code review: handle points with severak geo payloads

* refactor

* fix review comments

* review fixes

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-03-20 13:10:08 +01:00
Arnaud Gourlay
c25fd83e2e [sharding] Introduce local & remote shards (#384) 
* Shard is now LocalShard

* introduce draft RemoteShard

* wire things via polymorphism via the enum

* split and move files into shard module

* setup trait polymorphism

* use trait implementation

* code review: add search & retrieve to shard operations

* code review: remove segments from ShardOperations trait as it does not make sense for RemoteShard
 2022-03-18 10:18:53 +01:00
Egor Ivkov
dcfb44de6f Consensus with Raft feature (#385) 
Consensus will be used to synchronize global state which is:
- Collection meta
- Shard to Peer mapping
 2022-03-17 15:15:06 +03:00
Ivan Pleshkov
2b46cb8801 Merge branch 'master' into remove-blas 2022-03-16 13:16:27 +04:00
Ivan Pleshkov
eb6885b75a are you happy clippy 2022-03-16 13:15:14 +04:00
Ivan Pleshkov
b479c8c29f revome unsafe functions from crate api 2022-03-16 12:58:20 +04:00
Ivan Pleshkov
b6bf803c3a fix build 2022-03-16 12:26:32 +04:00
Ivan Pleshkov
3df7c36c88 remove avx512 2022-03-16 12:25:02 +04:00
Ivan Pleshkov
37f32ee28a euclid metric neon speedup 2022-03-16 12:21:03 +04:00
Ivan Pleshkov
5483146018 arm neon speed 2022-03-15 13:28:57 +04:00
Ivan Pleshkov
0d1a02c90e fmt 2022-03-15 09:01:46 +00:00
Ivan Pleshkov
cf53667614 speedup for avx and sse 2022-03-15 08:59:02 +00:00
Egor Ivkov
960937b58b Sharding related collection API changes (#381) 
* NonZeroU32 shard number

* Inconsistent Failure option

* Generate docs and apis
 2022-03-11 19:26:09 +03:00
Ivan Pleshkov
4b930eda0a fix warnings on arm 2022-03-11 13:39:48 +04:00
Ivan Pleshkov
4634478f3e add stable build support 2022-03-11 11:45:41 +03:00
Egor Ivkov
cb36ca8e90 Fixes shard drop on collection creation (#380) 
* Fixes shard drop on collection creation

* Concurrent waiting for the shard drop

* Handle create_dir_all error

* Decrease n segments per shard
 2022-03-10 19:16:56 +03:00
Ivan Pleshkov
a91e77b824 fix segment benches 2022-03-09 21:40:16 +00:00
Ivan Pleshkov
2418642557 are you happy fmt 2022-03-09 15:50:21 +00:00
Ivan Pleshkov
2c4fd0a205 Merge branch 'master' into remove-blas 2022-03-09 15:48:25 +00:00
Ivan Pleshkov
063d0abe83 use fma instructions for arm 2022-03-09 12:41:21 +04:00
dependabot[bot]
bb8dba39bb Bump pprof from 0.6.2 to 0.7.0 (#369) 
* Bump pprof from 0.6.2 to 0.7.0

Bumps [pprof](https://github.com/tikv/pprof-rs) from 0.6.2 to 0.7.0.
- [Release notes](https://github.com/tikv/pprof-rs/releases)
- [Changelog](https://github.com/tikv/pprof-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tikv/pprof-rs/commits)

---
updated-dependencies:
- dependency-name: pprof
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* use new feature name

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
 2022-03-09 09:33:26 +01:00
dependabot[bot]
61af7efe15 Bump hashring from 0.2.1 to 0.3.0 (#368) 
Bumps [hashring](https://github.com/jeromefroe/hashring-rs) from 0.2.1 to 0.3.0.
- [Release notes](https://github.com/jeromefroe/hashring-rs/releases)
- [Changelog](https://github.com/jeromefroe/hashring-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jeromefroe/hashring-rs/compare/0.2.1...0.3.0)

---
updated-dependencies:
- dependency-name: hashring
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-03-09 09:04:32 +01:00
Ivan Pleshkov
4a24c275d9 use avx+fma instead of avx2 2022-03-08 08:33:14 +00:00
Ivan Pleshkov
92bfa1e162 simd refactor 2022-03-08 07:57:52 +00:00
Egor Ivkov
8a2c17f0d4 Migrate legacy collection format (#367) 2022-03-07 20:20:43 +03:00
Egor Ivkov
2bb52abc71 Multiple shards in collection (#360) 
* Multiple shards in collection

* Collection and Shard async drop

* Generate APIs for shard_number param
 2022-03-07 16:22:47 +03:00
Ivan Pleshkov
4e5b06833c avx512f test 2022-03-06 21:11:19 +04:00
Andrey Vasnetsov
c6441f06c1 [gPRC] docs + refactoring (#363) 
* wip: doc generator script

* fix enum default value + CollectionInfo upd + collections comments

* refactor grpc + generate docs

* disable clippy for auto-generated files

* Update src/tonic/proto/points.proto

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
 2022-03-03 15:30:09 +01:00
Ivan Pleshkov
7f00a1d7e0 add neon aarch64 support 2022-03-02 23:42:16 +04:00
Arnaud Gourlay
903b4ffe34 fix duplicate bench group (#359) 2022-03-02 19:40:17 +01:00
Andrey Vasnetsov
c3cfc93574 fix PayloadSelector enum for better openapi compatibility (#358) 2022-03-02 15:01:27 +01:00
Andrey Vasnetsov
bdc3194b13 allow to return vectors and payload with recommendation request #353 (#357) 
* allow to return vectors and payload with recommendation request #353

* fmt

* unify `with_vector` comment line

* upd openapi
 2022-03-02 10:42:58 +01:00
Egor Ivkov
59160eff9f Add consistent hashing library (#348) 2022-02-28 20:37:06 +03:00
Arnaud Gourlay
d79eb01304 Turn geo query into set of geohashes (#315) 
* wip: geo helpers

* draft impl

* code review

* clippy

* please clippy

* code review: use haversine distance from geo lib

* filter properly geohashes for the circle

* code review: better approximate match for circles by being less strict

* make sure to return at least the precision where the area is fully covered

* add test for distance smaller than highest precision

* random query test

* fmt

* improve area computation and document south pole issue

* wip: sphere_neighbor

* fix max latitude

* fmt

* protect from longitude overflow & increase reliability

* more testing

* refactor

* fix clippy

* cleanup tests

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2022-02-28 17:55:52 +01:00
Arnaud Gourlay
c59c35d44d gRPC retrieve methods (#343) 
* gRPC retrieve methods

* code review: remove duplicate do_get_points

* code review: import for readability

* code review: reinstate fullpath by convention

* code review: experiment changing shape of with_vector flag

* code review: update openapi spec
 2022-02-28 15:13:08 +01:00
Egor Ivkov
b008e3dede Select shard for operation (#340) 2022-02-28 11:38:43 +03:00
Arnaud Gourlay
747d069eb3 fix new clippy lints in 1.59 (#341) 2022-02-24 20:59:23 +01:00
Arnaud Gourlay
55b3116be5 update tokio 1.17 (#339) 2022-02-24 10:53:47 +01:00
Andrey Vasnetsov
7f16f2fe09 better logic for proxy segment flushing #317 (#336) 2022-02-23 15:00:42 +01:00
Egor Ivkov
b07aa392a6 Periodic flushing (#332) 
* Periodic flushing

* Save flush error
 2022-02-23 16:52:33 +03:00
dependabot[bot]
9654596953 Bump geo from 0.18.0 to 0.19.0 (#334) 
Bumps [geo](https://github.com/georust/geo) from 0.18.0 to 0.19.0.
- [Release notes](https://github.com/georust/geo/releases)
- [Changelog](https://github.com/georust/geo/blob/master/CHANGES.md)
- [Commits](https://github.com/georust/geo/compare/geo-0.18.0...geo-0.19.0)

---
updated-dependencies:
- dependency-name: geo
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-02-21 20:14:26 +01:00
Egor Ivkov
3b62670a1a Introduce Shard abstraction (#318) 
* Introduce Shard abstraction

* Optimizer thread name improved and avg_vectors moved
 2022-02-18 17:51:44 +03:00
Andrey Vasnetsov
e45379e438 Better optimizer error reporting + small bug fixes (#316) 
* optimizer error reporting, decouple data removing, optimizator fix

* fmt

* fmt + clippy

* update openapi
 2022-02-16 09:59:11 +01:00
Andrey Vasnetsov
0b9463a674 reproduce with integration test and fix #309 
* reproduce with integration test

* create new update channel each time we spin up a new update worker
 2022-02-15 10:21:46 +01:00
dependabot[bot]
bfbf37e0df Bump rand from 0.8.4 to 0.8.5 (#311) 
Bumps [rand](https://github.com/rust-random/rand) from 0.8.4 to 0.8.5.
- [Release notes](https://github.com/rust-random/rand/releases)
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-random/rand/compare/0.8.4...0.8.5)

---
updated-dependencies:
- dependency-name: rand
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-02-15 07:47:23 +01:00
Andrey Vasnetsov
58ce28a933 [gPRC] search method (#300) 
* wip: grpc search + refactor payload selector

* grpc: search api + test

* fmt

* fix clippy

* update openapi schema (backward compatible)
 2022-02-14 12:49:24 +01:00
Andrey Vasnetsov
02a50212e5 Refactor proto & rest (#302) 
* reorder points.proto

* simplify grpc + replace match with enum (backward compatible)

* fmt

* remove try_match

* upd openapi schema

* fix grpc test

* fix grpc readme
 2022-02-14 09:58:23 +01:00