Commit Graph

56 Commits

Author SHA1 Message Date
Egor Ivkov
ee4cd768ea Allows specifying collection meta operation commit timeout in API (#444)
* Allows specifying collection meta operation commit timeout in API

Also restructures a part of gRPC collection API

* Review: link to dev docs

* Review: Timeout error description

Also committed generated protobuf code that somehow was left out

* Review: seconds in API and minor proto fixes
2022-04-08 14:42:38 +03:00
Andrey Vasnetsov
b07428f620 Is empty condition (#423)
* is-empty condition

* fmt

* better assert

* fmt
2022-04-03 16:55:51 +02:00
Andrey Vasnetsov
adc1f4ad97 Bool filter (#421)
* bool match condition

* use generic values for match requests

* fmt

* upd grpc interface

* upd grpc docs
2022-04-03 16:08:34 +02:00
Andrey Vasnetsov
6fc8d0affe upd openAPI schema names (#416) 2022-03-29 14:12:52 +02:00
Andrey Vasnetsov
8dcb900113 fix intergation test: freeze python deps on working versions 2022-03-28 23:56:08 +02:00
Arnaud Gourlay
2a81fc9142 Remove sled for alias mappings (#402)
* improve alias tests

* introduce new alias persistence and remove Sled

* introduce FileStorageError

* use new error in AliasMapping

* aliases are kept in memory and saved to disk when modified

* better naming

* make alias renaming atomic

* fmt

* use correct alias and docs

* code review: cleaner error conversion
2022-03-23 14:21:04 +01:00
Arnaud Gourlay
cfed400240 gitignore __pycache__/ (#401) 2022-03-21 17:39:40 +01:00
Gabriel Velo
f69a7b740f json as payload (#306)
add json as payload
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2022-03-21 07:09:10 -03:00
Andrey Vasnetsov
20c1e0f20d upd naming in api (#361) 2022-03-03 13:27:51 +01:00
Andrey Vasnetsov
c3cfc93574 fix PayloadSelector enum for better openapi compatibility (#358) 2022-03-02 15:01:27 +01:00
Andrey Vasnetsov
bdc3194b13 allow to return vectors and payload with recommendation request #353 (#357)
* allow to return vectors and payload with recommendation request #353

* fmt

* unify `with_vector` comment line

* upd openapi
2022-03-02 10:42:58 +01:00
Andrey Vasnetsov
0b9463a674 reproduce with integration test and fix #309
* reproduce with integration test

* create new update channel each time we spin up a new update worker
2022-02-15 10:21:46 +01:00
Andrey Vasnetsov
f5feff8080 Validated integration tests - #232
* wip: re-implement http tests using openapi validation library

* collection delete api

* payload operations test

* payload delete operations test

* payload index creation and removing test

* uuid operations test

* schema consistency test

* add tests into pipeline

* chmod +x

* faster local runs with docker cache

* upd instruction
2022-02-11 16:43:47 +01:00
Arnaud Gourlay
80652c5624 clean openapi setup (#288) 2022-02-07 07:59:09 +01:00
Andrey Vasnetsov
4a5cf094a4 fix in create index api: post -> put (#272) 2022-02-01 16:09:38 +01:00
Andrey Vasnetsov
efc0e4643d add missiong wait for collection index operation (#270) 2022-01-31 17:30:15 +01:00
Andrey Vasnetsov
65787f7f55 UUID as point id (#265)
* wip: u64 -> u128 + serialization tests

* breaking: use more flexible structure for saving point ids

* replace u64 external id type with enum

* update openapi definitions for uuid + fix retrieve point api + bash script tests
2022-01-31 13:18:07 +01:00
Andrey Vasnetsov
a2f35044c9 enable with_vector flag in api #50 (#267)
* enable `with_vector` flag in api #50

* fix readme
2022-01-31 11:31:16 +01:00
Tim Eggert
559e7a8055 Delete Points By Filter API #39 (#250)
* Delete Points By Filter API #39

* make delete_by_filter part of existing delete query + fix merge issues #39

* apply fmt

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2022-01-25 16:22:18 +01:00
Andrey Vasnetsov
9440143af0 rename point update api structure #208 (#251) 2022-01-24 19:32:14 +01:00
Andrey Vasnetsov
d51a70fa93 add openapi validation during generation #208 (#248)
* add openapi validation during generation #208

* fix: POST -> PUT in point update api implementation and docs #208

* fix: openapi structure exposure

* fix: api usage in stress test
2022-01-24 17:33:57 +01:00
Andrey Vasnetsov
02f53f2ef0 fix: openapi specs validation #208 (#246)
* fix: openapi specs validation #208

* fix: upd api descriptions
2022-01-24 11:46:14 +01:00
Arnaud Gourlay
97cb5091bc Split Points API #208 (#221)
Split Points API #208
2022-01-24 07:54:47 +01:00
Andrey Vasnetsov
898b692f39 [WIP] Limit segment size #135 (#195)
* add parameters to optimizer config

* benchmark search speed in different segment sizes

* use constructor for FilteredScorer

* * Implement benchmarks for HNSW index search with different number of
  stored points
* Fix minor issue in HNSW graph edge assignment
* Update profiler with call-graph report generation
* Add profiling guide
* Add HNSW graph statistics test function (debug inly)

* limit resulting segment size in merge optimizer

* fix clippy

* stop the music

* fix clippy once again

* fmt once again
2022-01-18 15:49:35 +01:00
Andrey Vasnetsov
38ee907af6 minor style fixes 2022-01-04 01:02:50 +01:00
Daniil
1ad529c315 Allow to include vector into search result (#176)
* feat(#50): include vector into search result

allow to specify 'with_vector' parameter in search api to get search results vector data

* test(#50): fix tests

* chore(#50): apply cargo fmt

* chore(#50): update api docs

run tools/generate_openapi_models.sh

Co-authored-by: Daniil Sunyaev <dasforrum@gmail.com>
2022-01-03 17:12:01 +01:00
Andrey Vasnetsov
f4a0511385 Add OpenAPI version selector (#171)
* add OpenAPI version selector

* fix typo
2022-01-03 11:34:09 +01:00
Andrey Vasnetsov
d7148126ce Split collection update API into several endpoints (#126)
* split storage operation structures #32

* cargo fmt #32

* split collection update api into several endpoints #32

* cargo fmt #32

* fix tonic-related code with new structures

* upd alias structures

* use ytt teplate engine for OpenAPI Endpoint schema generation
2021-12-13 09:38:33 +01:00
Andrey Vasnetsov
b73b641403 Update api descriptions 37 (#125)
* upd API description in OpenAPI #37

* upd openapi
2021-11-28 12:45:55 +01:00
Andrey Vasnetsov
e7e13162c2 add openblas patch to the root package 2021-10-27 23:02:41 +02:00
Andrey Vasnetsov
be1474d3ed upd version to 0.4.1 2021-10-25 23:03:59 +02:00
Andrey Vasnetsov
29830f953d default payload return is null + update openAPI 2021-10-24 18:47:58 +02:00
Andrey Vasnetsov
3a7485aa8d update OpenAPI schema 2021-10-24 18:21:17 +02:00
Andrey Vasnetsov
b4b39f8f75 v0.3.6 2021-08-30 00:12:59 +02:00
Andrey Vasnetsov
d5d8d2c407 fix serde deserialisation issue in WAL 2021-07-11 21:22:22 +02:00
trean
d30e6fa8ee Implementation of points scroll API #38 (#40)
* WIP: filtered points iterator #38

* add paginated filtered point request function #38

* add scroll api + openapi definitions #38

* fix openapi #38
2021-06-20 15:30:12 +02:00
Andrey Vasnetsov
cfc5beeac7 add payload schema to collection info + indexing fixes 2021-06-13 22:31:09 +02:00
Andrey Vasnetsov
3616631300 Filtrable hnsw (#26)
* raw points scorer

* raw point scorer for memmap storage

* search interface prepare

* graph binary saving + store PointOffsetId as u32

* WIP: entry points

* connect new link method

* update libs + search layer method + visited list + search context + update rust

* implement Euclid metric + always use MinHeap for priority queue

* small refactor

* search for 0 level entry

* update visited pool to be lock free and thread safe

* use ef_construct from graph layer struct + limit visited links to M

* add metric pre-processing before on vector upsert

* old hnsw heuristic

* save hnsw graph for export

* search method + tests

* small fixes

* add benchmark and profiler

* build time optimizations

* use SeaHash

* remove unsed benchmark

* merge hnsw graph function

* WIP:HNSW index build function

* HNSW build_index with additional indexing

* refactor fixtures

* graph save and load test

* test and fixes for filterable HNSW

* enable hnsw index for query planning

* fix cardinality estimation tests + remove query planner as class

* small refactor

* store full copy of collection settings with collection + allow partial override on creation #16

* API for updating collection parameters #16

* refactor: move collection error -> types

* report collection status in info API #17

* update OpenAPI Schema
2021-05-30 17:14:42 +02:00
Andrey Vasnetsov
dba7c9e558 fix filter query cardinality estimation 2021-05-04 20:12:51 +02:00
Andrey Vasnetsov
904d23c7cc fix openapi definitions 2021-04-06 14:13:41 +02:00
Andrey Vasnetsov
bb94656568 fix openapi 2021-04-06 14:13:41 +02:00
Andrey Vasnetsov
5231787ecd update OpenAPI definitions 2021-04-06 14:13:41 +02:00
Andrey Vasnetsov
ff2b1bfe60 Describe wait param in OpenAPI 2021-02-09 17:11:16 +01:00
Andrey Vasnetsov
44485277d0 hot-fix documentation 2021-01-25 00:13:37 +01:00
Andrey Vasnetsov
ae8e911941 recommender API documentation 2021-01-24 23:41:04 +01:00
Andrey Vasnetsov
62768a8193 recommender API 2021-01-24 23:38:59 +01:00
Andrey Vasnetsov
5c2afdc5b9 geo-radius + update wiki 2021-01-24 17:16:06 +01:00
Andrey Vasnetsov
09fea5a087 remove incorrect usage of "allOf" from API specification 2020-11-29 00:15:15 +01:00
Andrey Vasnetsov
423005b53a fix openapi schema 2020-11-28 23:34:29 +01:00
Andrey Vasnetsov
d0bdabeb2d fix openapi status enum 2020-11-24 20:32:09 +01:00