Commit Graph

1780 Commits

Author SHA1 Message Date
Ivan Pleshkov
b479c8c29f revome unsafe functions from crate api 2022-03-16 12:58:20 +04:00
Ivan Pleshkov
b6bf803c3a fix build 2022-03-16 12:26:32 +04:00
Ivan Pleshkov
3df7c36c88 remove avx512 2022-03-16 12:25:02 +04:00
Ivan Pleshkov
37f32ee28a euclid metric neon speedup 2022-03-16 12:21:03 +04:00
Ivan Pleshkov
5483146018 arm neon speed 2022-03-15 13:28:57 +04:00
Ivan Pleshkov
0d1a02c90e fmt 2022-03-15 09:01:46 +00:00
Ivan Pleshkov
cf53667614 speedup for avx and sse 2022-03-15 08:59:02 +00:00
Ivan Pleshkov
4b930eda0a fix warnings on arm 2022-03-11 13:39:48 +04:00
Ivan Pleshkov
4634478f3e add stable build support 2022-03-11 11:45:41 +03:00
Ivan Pleshkov
a91e77b824 fix segment benches 2022-03-09 21:40:16 +00:00
Ivan Pleshkov
2418642557 are you happy fmt 2022-03-09 15:50:21 +00:00
Ivan Pleshkov
2c4fd0a205 Merge branch 'master' into remove-blas 2022-03-09 15:48:25 +00:00
Ivan Pleshkov
063d0abe83 use fma instructions for arm 2022-03-09 12:41:21 +04:00
dependabot[bot]
bb8dba39bb Bump pprof from 0.6.2 to 0.7.0 (#369)
* Bump pprof from 0.6.2 to 0.7.0

Bumps [pprof](https://github.com/tikv/pprof-rs) from 0.6.2 to 0.7.0.
- [Release notes](https://github.com/tikv/pprof-rs/releases)
- [Changelog](https://github.com/tikv/pprof-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tikv/pprof-rs/commits)

---
updated-dependencies:
- dependency-name: pprof
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* use new feature name

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
2022-03-09 09:33:26 +01:00
Ivan Pleshkov
4a24c275d9 use avx+fma instead of avx2 2022-03-08 08:33:14 +00:00
Ivan Pleshkov
92bfa1e162 simd refactor 2022-03-08 07:57:52 +00:00
Ivan Pleshkov
4e5b06833c avx512f test 2022-03-06 21:11:19 +04:00
Ivan Pleshkov
7f00a1d7e0 add neon aarch64 support 2022-03-02 23:42:16 +04:00
Arnaud Gourlay
903b4ffe34 fix duplicate bench group (#359) 2022-03-02 19:40:17 +01:00
Andrey Vasnetsov
c3cfc93574 fix PayloadSelector enum for better openapi compatibility (#358) 2022-03-02 15:01:27 +01:00
Andrey Vasnetsov
bdc3194b13 allow to return vectors and payload with recommendation request #353 (#357)
* allow to return vectors and payload with recommendation request #353

* fmt

* unify `with_vector` comment line

* upd openapi
2022-03-02 10:42:58 +01:00
Arnaud Gourlay
d79eb01304 Turn geo query into set of geohashes (#315)
* wip: geo helpers

* draft impl

* code review

* clippy

* please clippy

* code review: use haversine distance from geo lib

* filter properly geohashes for the circle

* code review: better approximate match for circles by being less strict

* make sure to return at least the precision where the area is fully covered

* add test for distance smaller than highest precision

* random query test

* fmt

* improve area computation and document south pole issue

* wip: sphere_neighbor

* fix max latitude

* fmt

* protect from longitude overflow & increase reliability

* more testing

* refactor

* fix clippy

* cleanup tests

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2022-02-28 17:55:52 +01:00
Arnaud Gourlay
747d069eb3 fix new clippy lints in 1.59 (#341) 2022-02-24 20:59:23 +01:00
dependabot[bot]
9654596953 Bump geo from 0.18.0 to 0.19.0 (#334)
Bumps [geo](https://github.com/georust/geo) from 0.18.0 to 0.19.0.
- [Release notes](https://github.com/georust/geo/releases)
- [Changelog](https://github.com/georust/geo/blob/master/CHANGES.md)
- [Commits](https://github.com/georust/geo/compare/geo-0.18.0...geo-0.19.0)

---
updated-dependencies:
- dependency-name: geo
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-21 20:14:26 +01:00
Andrey Vasnetsov
e45379e438 Better optimizer error reporting + small bug fixes (#316)
* optimizer error reporting, decouple data removing, optimizator fix

* fmt

* fmt + clippy

* update openapi
2022-02-16 09:59:11 +01:00
Andrey Vasnetsov
58ce28a933 [gPRC] search method (#300)
* wip: grpc search + refactor payload selector

* grpc: search api + test

* fmt

* fix clippy

* update openapi schema (backward compatible)
2022-02-14 12:49:24 +01:00
Andrey Vasnetsov
02a50212e5 Refactor proto & rest (#302)
* reorder points.proto

* simplify grpc + replace match with enum (backward compatible)

* fmt

* remove try_match

* upd openapi schema

* fix grpc test

* fix grpc readme
2022-02-14 09:58:23 +01:00
Andrey Vasnetsov
7f4ff39201 geo coordinates range validation (#295) 2022-02-10 19:34:07 +01:00
Gabriel Velo
4483ea0d60 fix: #101 Payload type consistency is not enforced. 2022-02-09 11:46:01 -03:00
dependabot[bot]
2e1ced6601 Bump rocksdb from 0.17.0 to 0.18.0 (#292)
Bumps [rocksdb](https://github.com/rust-rocksdb/rust-rocksdb) from 0.17.0 to 0.18.0.
- [Release notes](https://github.com/rust-rocksdb/rust-rocksdb/releases)
- [Changelog](https://github.com/rust-rocksdb/rust-rocksdb/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-rocksdb/rust-rocksdb/compare/v0.17.0...v0.18.0)

---
updated-dependencies:
- dependency-name: rocksdb
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-09 07:42:17 +01:00
Arnaud Gourlay
3d2b2ce8c2 new gRPC points update API #242 (#264) 2022-02-03 17:36:13 +01:00
dependabot[bot]
b0471acb31 Bump parking_lot from 0.11.2 to 0.12.0 (#271)
Bumps [parking_lot](https://github.com/Amanieu/parking_lot) from 0.11.2 to 0.12.0.
- [Release notes](https://github.com/Amanieu/parking_lot/releases)
- [Changelog](https://github.com/Amanieu/parking_lot/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Amanieu/parking_lot/compare/0.11.2...0.12.0)

---
updated-dependencies:
- dependency-name: parking_lot
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-31 21:46:21 +01:00
Andrey Vasnetsov
65787f7f55 UUID as point id (#265)
* wip: u64 -> u128 + serialization tests

* breaking: use more flexible structure for saving point ids

* replace u64 external id type with enum

* update openapi definitions for uuid + fix retrieve point api + bash script tests
2022-01-31 13:18:07 +01:00
Tim Eggert
559e7a8055 Delete Points By Filter API #39 (#250)
* Delete Points By Filter API #39

* make delete_by_filter part of existing delete query + fix merge issues #39

* apply fmt

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2022-01-25 16:22:18 +01:00
Arnaud Gourlay
4c73d1ba8f name threads for observability (#247) 2022-01-24 13:39:12 +01:00
dependabot[bot]
879062538e Bump atomicwrites from 0.2.5 to 0.3.1 (#236)
Bumps [atomicwrites](https://github.com/untitaker/rust-atomicwrites) from 0.2.5 to 0.3.1.
- [Release notes](https://github.com/untitaker/rust-atomicwrites/releases)
- [Commits](https://github.com/untitaker/rust-atomicwrites/compare/0.2.5...0.3.1)

---
updated-dependencies:
- dependency-name: atomicwrites
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-20 21:11:04 +01:00
dependabot[bot]
31781a7420 Bump schemars from 0.8.3 to 0.8.8 (#233)
Bumps [schemars](https://github.com/GREsau/schemars) from 0.8.3 to 0.8.8.
- [Release notes](https://github.com/GREsau/schemars/releases)
- [Changelog](https://github.com/GREsau/schemars/blob/master/CHANGELOG.md)
- [Commits](https://github.com/GREsau/schemars/compare/v0.8.3...v0.8.8)

---
updated-dependencies:
- dependency-name: schemars
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-20 19:55:15 +01:00
dependabot[bot]
7b9998cad7 Bump rmp-serde from 0.14.4 to 1.0.0 (#216)
Bumps [rmp-serde](https://github.com/3Hren/msgpack-rust) from 0.14.4 to 1.0.0.
- [Release notes](https://github.com/3Hren/msgpack-rust/releases)
- [Commits](https://github.com/3Hren/msgpack-rust/compare/rmp-serde/v0.14.4...rmp-serde/v1.0.0)

---
updated-dependencies:
- dependency-name: rmp-serde
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-20 10:24:47 +01:00
dependabot[bot]
71d0e814bd Bump pprof from 0.4.3 to 0.6.1 (#223)
Bumps [pprof](https://github.com/tikv/pprof-rs) from 0.4.3 to 0.6.1.
- [Release notes](https://github.com/tikv/pprof-rs/releases)
- [Changelog](https://github.com/tikv/pprof-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tikv/pprof-rs/commits)

---
updated-dependencies:
- dependency-name: pprof
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-20 09:08:29 +01:00
dependabot[bot]
ade7e3b24d Bump geo from 0.17.1 to 0.18.0 (#224)
Bumps [geo](https://github.com/georust/geo) from 0.17.1 to 0.18.0.
- [Release notes](https://github.com/georust/geo/releases)
- [Changelog](https://github.com/georust/geo/blob/master/CHANGES.md)
- [Commits](https://github.com/georust/geo/commits/geo-0.18.0)

---
updated-dependencies:
- dependency-name: geo
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-20 08:24:25 +01:00
dependabot[bot]
98db2555f7 Bump atomic_refcell from 0.1.7 to 0.1.8 (#226)
Bumps [atomic_refcell](https://github.com/bholley/atomic_refcell) from 0.1.7 to 0.1.8.
- [Release notes](https://github.com/bholley/atomic_refcell/releases)
- [Commits](https://github.com/bholley/atomic_refcell/commits)

---
updated-dependencies:
- dependency-name: atomic_refcell
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-19 21:46:48 +01:00
dependabot[bot]
144ce688b5 Bump serde_cbor from 0.11.1 to 0.11.2 (#219)
Bumps [serde_cbor](https://github.com/pyfisch/cbor) from 0.11.1 to 0.11.2.
- [Release notes](https://github.com/pyfisch/cbor/releases)
- [Commits](https://github.com/pyfisch/cbor/compare/v0.11.1...v0.11.2)

---
updated-dependencies:
- dependency-name: serde_cbor
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-19 13:18:28 +01:00
dependabot[bot]
aeb069cd38 Bump ordered-float from 1.1.1 to 2.10.0 (#212)
Bumps [ordered-float](https://github.com/reem/rust-ordered-float) from 1.1.1 to 2.10.0.
- [Release notes](https://github.com/reem/rust-ordered-float/releases)
- [Commits](https://github.com/reem/rust-ordered-float/compare/v1.1.1...v2.10.0)

---
updated-dependencies:
- dependency-name: ordered-float
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-19 12:11:03 +01:00
Arnaud Gourlay
fff3f81b5b remove unused dependencies (#209) 2022-01-19 11:31:48 +01:00
Andrey Vasnetsov
898b692f39 [WIP] Limit segment size #135 (#195)
* add parameters to optimizer config

* benchmark search speed in different segment sizes

* use constructor for FilteredScorer

* * Implement benchmarks for HNSW index search with different number of
  stored points
* Fix minor issue in HNSW graph edge assignment
* Update profiler with call-graph report generation
* Add profiling guide
* Add HNSW graph statistics test function (debug inly)

* limit resulting segment size in merge optimizer

* fix clippy

* stop the music

* fix clippy once again

* fmt once again
2022-01-18 15:49:35 +01:00
Andrey Vasnetsov
0f91c9a5e2 [WIP] Force optimization stop #31 (#161)
* implement checking stop-flag in the optimization routine

* wip: optimization cancel test

* force optimization stop during the construction of vector index

* fix clippy
2022-01-18 15:06:42 +01:00
Arnaud Gourlay
24bd88e8bd hnsw entry points - replace filter+next by find 2022-01-18 09:47:04 +01:00
Prokudin Alexander
ee461ce0a6 Extend clippy to workspace and fix some warnings (#199)
* Fix clippy in linting workflow

* Add toolchain override flag

* Add components to toolchain installation explicitly

* Add --workspace flag to clippy to check all packages

* Remove unnecessary clones

* remove redundant .clone() calls

* fix wrong arguments order in tests (typo)

* Fix vec! macro usage in test

* Correct redundant assert! usages

* Provide a quick fix for 'unused' test function lint

* fix unsound Send + Sync

* fix clippy complains

* fmt

* fix clippy

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2022-01-17 23:33:26 +01:00
Ivan Pleshkov
1ee72b59bc are you happy fmt? 2022-01-05 01:48:50 +03:00
Ivan Pleshkov
a013860031 add sse support 2022-01-05 01:17:49 +03:00