Andrey Vasnetsov
99eaabc31a
remove deprecate point ids param in gprc ( #1305 )
...
* remove deprecate point ids param in gprc
* upd docs
2023-02-06 13:37:18 +01:00
Roman Titov
a50e7bdcd4
Add read_consistency parameter to the APIs ( #1371 ) ( #1407 )
...
* WIP: Add `read_consistency` parameter to the APIs
* WIP: Add `read_consistency` parameter to the APIs
TODO:
- Add documentation
* `cargo fmt`
* Add gRPC documentation
* Add OpenAPI documentation
* Cleanup
* fixup! Add OpenAPI documentation
* fixup! Add gRPC documentation
Who would have known there's `generate_grpc_docs.sh`!? 🥲 🙈 🤦♀️
* generate openapi
* Fix `read_consistency` query parameter deserialization
* Further improve `read_consistency` query parameter deserialization
* `cargo clippy`
* Fix `Payload` comparison during read operation result resolving
* Fix grammar
* rename `read_consistency` -> `consistency` and add integration test
* use majority for test
* fix tests
* Fix tests
* fixup! Fix tests
Apply the same fix to `ScoredPoint`
* Remove an `unwrap`
* fixup! Fix tests
Gotta love those negative conditions, or how a missed `!` can ruin your day... 🤦♀️
* Make internal API calls strictly "local-shard only"
* Implement a few basic traits for `ResolverRecord`
* fixup! Implement a few basic traits for `ResolverRecord`
* Revert "Make internal API calls strictly "local-shard only""
This reverts commit 25378e61ffb566bea49bandrey@vasnetsov.com >
2023-02-03 01:12:28 +01:00
Andrey Vasnetsov
9826e0338c
add init state to replica set ( #1404 )
...
* add init state to replica set
* update openapi
* disallow activating shard from non-initialize state
* fmt
* remove useless operation confirm on collection creation
* clippy fixes
* fix test
2023-01-30 14:58:34 +01:00
Andrey Vasnetsov
f356455b07
add write consistency param to rest and grpc api ( #1397 )
...
* add write consistency param to rest and grpc api
* fix grpcurl + refactor grpc for better extencibility
* upd grpc docs
* fix conversion
2023-01-25 11:49:01 +01:00
Andrey Vasnetsov
61d970ef90
Init collection from ( #1364 )
...
* WIP
* WIP: add shards selection
* fmt
* WIP: collection init from other collection
* fmt
* fix bugs
* add collection write lock + use collection manager runtime instead of search runtime
* fmt
* integration test
* better names ?
* payload indexes transfer
2023-01-22 01:16:55 +01:00
Arnaud Gourlay
86b8ff6b7f
Delete snapshots API ( #1377 )
...
* Delete snapshots API
* remove unecessary plurals
2023-01-20 19:28:56 +01:00
Arnaud Gourlay
6f284da22e
List aliases API (rest & gRPC) ( #1363 )
...
* List aliases API (rest & gRPC)
* rework api to avoid conflicts
2023-01-19 07:59:02 +01:00
Andrey Vasnetsov
a2c5de586b
Recommendation with other collection ( #1283 )
...
* WIP: move recommendation API out of collection
* recomendations with external source of vectors
* fix test
* fmt
* fixes + test
* review fixes
2022-12-22 09:29:36 +01:00
Andrey Vasnetsov
d05b2def10
allow set different M param for default HNSW and payload-related links ( #1257 )
...
* allow set different M param for default HNSW and payload-related links
* review fixes
2022-12-06 13:53:42 +01:00
Andrey Vasnetsov
49c3e37061
add parameter that defines a source of truth for snapshot recovery ( #1262 )
...
* add parameter that defines a source of truth for snapshot recovery
* review fixes
2022-12-06 13:53:03 +01:00
Andrey Vasnetsov
04434f02b6
Update points by filter ( #1249 )
...
* allow to set remove payload by selector
* fmt
2022-12-01 09:16:07 +01:00
Andrey Vasnetsov
9f36ac60ee
Full payload update ( #1245 )
...
* implement api to fully overwrite payload of the point
* fmt
2022-11-28 13:46:58 +01:00
Andrey Vasnetsov
b1da0a5be4
update openapi - telemetry + max_indexing_threads
2022-11-25 13:27:26 +01:00
Ivan Pleshkov
ac9538d5af
Hnsw links memmap ( #1211 )
...
* add on disk key
* remove obsolete graph initialization
* remove obsolete max level
* update openapi
* graph links trait
* use mmap option
* same format for ram and mmap
* fix segment unit tests
* are you happy fmt
* are you happy clippy
* fix ci and add mmap test
* review fixes
* remove unused try-from
* fix version compatibility
* avoid loading from disk during conversion
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
2022-11-25 11:47:52 +01:00
Andrey Vasnetsov
772326f848
Recover distributed collection ( #1214 )
...
* WIP: shards recovering API: download snapshot
* fmt
* snapshot recovery API
* fmt
* snapshot recovery integration test + fixes
* review fixes
* review fixes 2
* Update lib/storage/src/content_manager/snapshots/recover.rs
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com >
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com >
2022-11-16 10:55:23 +01:00
Arnaud Gourlay
98fc5b2fa4
0.11.1 cherry picks ( #1193 )
...
* Complete feature road-map
* docs: add erare-humanum as a contributor for code (#1181 )
* docs: update README.md [skip ci]
* docs: update .all-contributorsrc [skip ci]
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
* update rust version and fix clippy complains (#1190 )
* handle collections transition from single-node to cluster (#1189 )
* Update to v0.11.1 (#1191 )
* update to v0.11.1
* also default version
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
* consensus_state to consensus_state
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
2022-11-07 11:31:12 +01:00
Arnaud Gourlay
286ca9f5e4
0.11.0 cherry picks ( #1174 )
...
* implement concern-factor, forbig disabling the last node, improve tes… (#1168 )
* implement concern-factor, forbig disabling the last node, improve test stability
* upd docs
* improve test_recover_dead_node
* rename to write_consistency_factor
* Fix bug: applied index higher than commit on restart (#1172 )
* create collection with default status = dead + await for activation on create_collecton_op_submit
* fmt
* fix unit tests
Co-authored-by: Egor Ivkov <e.o.ivkov@gmail.com >
* upd integration tests
* fix transaction handling
* upd protobuf
* upd to v0.11.0
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
Co-authored-by: Egor Ivkov <e.o.ivkov@gmail.com >
2022-10-27 10:16:21 +02:00
Andrey Vasnetsov
071d1948b0
update and improve openapi definitions
2022-10-24 16:04:31 +02:00
Andrey Vasnetsov
21754ef2ef
report indexed payload points in info api + other fixes ( #1164 )
...
* report indexed payload points in info api + other fixes
* rollback debug changes
* clippy
* clippy
2022-10-24 09:43:02 +02:00
Ivan Pleshkov
516dcd7020
Telemetry level of detail ( #1049 )
...
* telemetry level of detail
* rename duration aggregator
* are you happy fmt
* move total searches sum
* separate levels
* optional bucket size
* search telemetry improvements
* separate web telemetry into methods
* tonic telemetry methods
* merge optimizations
* are you happy fmt
* better rounding
* qdrant configs on level 1
* provide collection params
* add peers count
* collection points count
* update openapi
* use pattern in actix telemetry
* are you happy fmt
* merge dev
* are you happy fmt
* fix merge conflicts
* update openapi
* fix build
* are you happy fmt
* add exact searches statistics
* process replica set
* update openapi
* fix wrong name
* fix naming
* fix unwrap
* review
* fmt
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
2022-10-23 00:48:55 +02:00
Egor Ivkov
a96f92cbfe
Record message send failures and stop consensus on service error ( #1152 )
...
* Record message send failures and stop consensus on service error
* Generate openapi
* Review fixes: log first error and remove on success
2022-10-22 23:23:37 +02:00
Ivan Pleshkov
44626bc80a
exact search parameter ( #1150 )
...
* exact search parameter
* exact search test
* decrease attempts count because of test speed
* minor review fixes
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
2022-10-20 15:39:21 +04:00
Egor Ivkov
d3849d712b
Allow peer force removal ( #1139 )
...
* Allow peer force removal
* Remove shards
2022-10-19 17:01:13 +02:00
Andrey Vasnetsov
7f8c088711
Replication transfer fixes ( #1142 )
...
* fix validation and task assignment during replica transfer
* fmt
* upd openapi
* extend integration test
2022-10-19 11:38:26 +02:00
Ivan Pleshkov
f50f1f4c71
#1119 disable updating feature ( #1126 )
...
* disable updating feature
* customize error message
* update namings
* Update src/actix/api/service_api.rs
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com >
* fix build
* update api
* update api and allow deletion
* update openapi
* replace Bad Request with Forbidden
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com >
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
2022-10-18 12:58:38 +02:00
Andrey Vasnetsov
cf29b67640
v0.10.5
2022-10-18 10:49:43 +02:00
Arnaud Gourlay
9324be1d60
[Replication] Add replicas ( #1085 )
...
* Add replicas to ReplicaSet
* unproxify shard & miscs
* no exlusive write locking while adding replicas
* on_optimizer_config_update on Shard with async. recursion
* no exlusive write locking while removing replicas
* shortcut replica set propagation #1101
* remove unused field
* make RequestShardTransfer callback sync.
* promote local & remote to replica state
* fixes for replica sync api (#1123 )
* code review
* fix replica set update - fail only if all failed
* Add replica redesign (#1131 )
* refactor shard/mod.rs
* wip
* fmt
* it compiles
* temporary disable replica placemeant change on replication factor
* fmt
* finish todos
* small refactoring
* remove change::add
* replica-set -> shard-replica-set
* fmt
* upd openapi
* fix finish transfer logic
* fix existing integration tests
* shard transfer validation
* fmt
* review fixes
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
2022-10-17 20:12:10 +02:00
Andrey Vasnetsov
411ddabe49
v0.10.4
2022-10-13 09:14:11 +02:00
Andrey Vasnetsov
68a46a0f2e
v0.10.3
2022-10-13 09:14:11 +02:00
Andrey Vasnetsov
b275d452a8
shard replica manipulation API ( #1121 )
2022-10-12 15:19:43 +02:00
Egor Ivkov
ca01b8ac8e
Fix: Do not rely on empty message for is_leader_established flag ( #1103 )
...
* Fix: Do not rely on empty message for `is_leader_established` flag
* Record last consensus update
* Update OpenAPI
2022-10-05 18:10:33 +03:00
Andrey Vasnetsov
935f069d53
upd version to v0.10.2
2022-09-30 11:08:16 +02:00
Ivan Pleshkov
8eedcf3735
add empty security to openapi ( #1079 )
2022-09-30 10:02:40 +02:00
Egor Ivkov
9a3fad44b2
Revert "Revert replication changes ( #1055 )" ( #1056 )
...
This reverts commit a2ed0cf5db
2022-09-23 16:46:06 +03:00
Egor Ivkov
a2ed0cf5db
Revert replication changes ( #1055 )
...
* Revert "Replication set `Change` enum"
This reverts commit 28e8487542#1053 )
* Revert "Generate API - replication follow up (#1043 )"
This reverts commit 183ea9e5e3#1037 )"
This reverts commit d690b700ff#1035 )"
This reverts commit 31c3ae482d#1009 )"
This reverts commit e332d482b0andrey@vasnetsov.com >
2022-09-23 16:33:07 +03:00
Egor Ivkov
183ea9e5e3
Generate API - replication follow up ( #1043 )
...
Also removes `JsonSchema` from structs not exposed in API
2022-09-22 14:53:00 +03:00
Andrey Vasnetsov
f918a14819
upd to v0.10.1 ( #1031 )
...
* upd to v0.10.1
* upd lock
2022-09-19 15:31:35 +02:00
Andrey Vasnetsov
dc07b01e1f
remove deprecated fields from API ( #1030 )
...
* remove depricated fields from API
* fmt
* upd openapi and integration tests
* fix grpc test
* regenerate storage reference data
* improve docs
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com >
2022-09-19 13:51:03 +02:00
Andrey Vasnetsov
a39a811567
V0.10.0 ( #1029 )
...
* Revert "Rollback v0.10.0 changes before release"
This reverts commit 18b75d9069
2022-09-19 10:05:57 +02:00
Andrey Vasnetsov
03d08236a8
better struct for fts ( #1026 )
2022-09-16 14:07:50 +02:00
Ivan Pleshkov
4d3b4b132d
More multivector api updates ( #1025 )
...
* rest multivector api updates
* update openapi
* update grpc doc
2022-09-16 12:25:42 +04:00
Andrey Vasnetsov
94883e7062
allow empty with_vector
2022-09-16 01:37:34 +02:00
Andrey Vasnetsov
278b3def9b
use structure for better openapi generator compatibility ( #1020 )
2022-09-15 10:51:46 +02:00
Arnaud Gourlay
4c949283a9
update gRPC docs ( #1014 )
2022-09-14 11:49:36 +02:00
Ivan Pleshkov
f6b2186193
[WIP] Many named vectors per point ( #958 )
...
* many named vectors per point (segment-level)
* operation result for dim function
* beautifulized vector name
* fix naming bug
* segment version migration
* fmt
* add segment tests
* are you happy clippy
* fix build
* [WIP] many named vectors per point (collection-level) (#975 )
* config and search
* fix placeholders for proxy segment move
* remove VectorType from collection
* are you happy fmt
* vectors in grps messages
* create collections with vectors
* segment holder fixes
* are you happy fmt
* remove default vector name placeholders
* are you happy fmt
* are you happy clippy
* fix build
* fix web api
* are you happy clippy
* are you happy fmt
* record vector&vectors
* openapi update
* fix openapi integration tests
* segment builder fix todo
* vector names for update from segment
* remove unwrap
* backward compatibility
* upd openapi
* backward compatible PointStruct
* upd openapi
* fix record back-comp
* fmt
* vector configuration backward compatibility
* fix vetor storage size estimation
* fmt
* multi-vec segment test + index test
* fmt
* api integration tests
* [WIP] Named vectors struct (#1002 )
* move to separate file
* named vectors as struct
* use cow
* fix build
* keys iterator
* avoid copy in PointStruct -> get_vectors
* avoid another copy
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
2022-09-11 20:59:23 +02:00
Andrey Vasnetsov
c3bc5b3873
rename grpc fields + re-generate docs ( #980 )
2022-09-01 18:59:47 +02:00
Andrey Vasnetsov
b9eee55a9f
Full text search ( #963 )
...
* allow additional params for payload field index
* fmt
* wip: full text index building
* fmt
* text search request
* text search request
* full text index persitance and loading
* fmt
* enable fts index in mapping
* clippy
* fix tests + add integration test
* review fixes: extend payload index test
* revert incedental change
2022-09-01 12:50:12 +02:00
Andrey Vasnetsov
e8216e9400
do not fail if cannot get file creation time ( #973 )
2022-08-30 23:33:56 +02:00
Andre Zayarni
086216616a
Updated ETA
2022-08-22 16:37:43 +02:00
Arnaud Gourlay
03c10fc310
Batch recommendation API ( #954 )
2022-08-22 09:41:22 +02:00
