Andrey Vasnetsov
a2c5de586b
Recommendation with other collection ( #1283 )
...
* WIP: move recommendation API out of collection
* recomendations with external source of vectors
* fix test
* fmt
* fixes + test
* review fixes
2022-12-22 09:29:36 +01:00
Andrey Vasnetsov
d05b2def10
allow set different M param for default HNSW and payload-related links ( #1257 )
...
* allow set different M param for default HNSW and payload-related links
* review fixes
2022-12-06 13:53:42 +01:00
Andrey Vasnetsov
49c3e37061
add parameter that defines a source of truth for snapshot recovery ( #1262 )
...
* add parameter that defines a source of truth for snapshot recovery
* review fixes
2022-12-06 13:53:03 +01:00
Andrey Vasnetsov
04434f02b6
Update points by filter ( #1249 )
...
* allow to set remove payload by selector
* fmt
2022-12-01 09:16:07 +01:00
Andrey Vasnetsov
9f36ac60ee
Full payload update ( #1245 )
...
* implement api to fully overwrite payload of the point
* fmt
2022-11-28 13:46:58 +01:00
Andrey Vasnetsov
b1da0a5be4
update openapi - telemetry + max_indexing_threads
2022-11-25 13:27:26 +01:00
Ivan Pleshkov and Andrey Vasnetsov
ac9538d5af
Hnsw links memmap ( #1211 )
...
* add on disk key
* remove obsolete graph initialization
* remove obsolete max level
* update openapi
* graph links trait
* use mmap option
* same format for ram and mmap
* fix segment unit tests
* are you happy fmt
* are you happy clippy
* fix ci and add mmap test
* review fixes
* remove unused try-from
* fix version compatibility
* avoid loading from disk during conversion
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
2022-11-25 11:47:52 +01:00
Andrey Vasnetsov and Arnaud Gourlay
772326f848
Recover distributed collection ( #1214 )
...
* WIP: shards recovering API: download snapshot
* fmt
* snapshot recovery API
* fmt
* snapshot recovery integration test + fixes
* review fixes
* review fixes 2
* Update lib/storage/src/content_manager/snapshots/recover.rs
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com >
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com >
2022-11-16 10:55:23 +01:00
98fc5b2fa4
0.11.1 cherry picks ( #1193 )
...
* Complete feature road-map
* docs: add erare-humanum as a contributor for code (#1181 )
* docs: update README.md [skip ci]
* docs: update .all-contributorsrc [skip ci]
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
* update rust version and fix clippy complains (#1190 )
* handle collections transition from single-node to cluster (#1189 )
* Update to v0.11.1 (#1191 )
* update to v0.11.1
* also default version
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
* consensus_state to consensus_state
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
2022-11-07 11:31:12 +01:00
286ca9f5e4
0.11.0 cherry picks ( #1174 )
...
* implement concern-factor, forbig disabling the last node, improve tes… (#1168 )
* implement concern-factor, forbig disabling the last node, improve test stability
* upd docs
* improve test_recover_dead_node
* rename to write_consistency_factor
* Fix bug: applied index higher than commit on restart (#1172 )
* create collection with default status = dead + await for activation on create_collecton_op_submit
* fmt
* fix unit tests
Co-authored-by: Egor Ivkov <e.o.ivkov@gmail.com >
* upd integration tests
* fix transaction handling
* upd protobuf
* upd to v0.11.0
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
Co-authored-by: Egor Ivkov <e.o.ivkov@gmail.com >
2022-10-27 10:16:21 +02:00
Andrey Vasnetsov
071d1948b0
update and improve openapi definitions
2022-10-24 16:04:31 +02:00
Andrey Vasnetsov
21754ef2ef
report indexed payload points in info api + other fixes ( #1164 )
...
* report indexed payload points in info api + other fixes
* rollback debug changes
* clippy
* clippy
2022-10-24 09:43:02 +02:00
Ivan Pleshkov and Andrey Vasnetsov
516dcd7020
Telemetry level of detail ( #1049 )
...
* telemetry level of detail
* rename duration aggregator
* are you happy fmt
* move total searches sum
* separate levels
* optional bucket size
* search telemetry improvements
* separate web telemetry into methods
* tonic telemetry methods
* merge optimizations
* are you happy fmt
* better rounding
* qdrant configs on level 1
* provide collection params
* add peers count
* collection points count
* update openapi
* use pattern in actix telemetry
* are you happy fmt
* merge dev
* are you happy fmt
* fix merge conflicts
* update openapi
* fix build
* are you happy fmt
* add exact searches statistics
* process replica set
* update openapi
* fix wrong name
* fix naming
* fix unwrap
* review
* fmt
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
2022-10-23 00:48:55 +02:00
Egor Ivkov
a96f92cbfe
Record message send failures and stop consensus on service error ( #1152 )
...
* Record message send failures and stop consensus on service error
* Generate openapi
* Review fixes: log first error and remove on success
2022-10-22 23:23:37 +02:00
Ivan Pleshkov and Andrey Vasnetsov
44626bc80a
exact search parameter ( #1150 )
...
* exact search parameter
* exact search test
* decrease attempts count because of test speed
* minor review fixes
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
2022-10-20 15:39:21 +04:00
Egor Ivkov
d3849d712b
Allow peer force removal ( #1139 )
...
* Allow peer force removal
* Remove shards
2022-10-19 17:01:13 +02:00
Andrey Vasnetsov
7f8c088711
Replication transfer fixes ( #1142 )
...
* fix validation and task assignment during replica transfer
* fmt
* upd openapi
* extend integration test
2022-10-19 11:38:26 +02:00
f50f1f4c71
#1119 disable updating feature ( #1126 )
...
* disable updating feature
* customize error message
* update namings
* Update src/actix/api/service_api.rs
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com >
* fix build
* update api
* update api and allow deletion
* update openapi
* replace Bad Request with Forbidden
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com >
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
2022-10-18 12:58:38 +02:00
Andrey Vasnetsov
cf29b67640
v0.10.5
2022-10-18 10:49:43 +02:00
Arnaud Gourlay and Andrey Vasnetsov
9324be1d60
[Replication] Add replicas ( #1085 )
...
* Add replicas to ReplicaSet
* unproxify shard & miscs
* no exlusive write locking while adding replicas
* on_optimizer_config_update on Shard with async. recursion
* no exlusive write locking while removing replicas
* shortcut replica set propagation #1101
* remove unused field
* make RequestShardTransfer callback sync.
* promote local & remote to replica state
* fixes for replica sync api (#1123 )
* code review
* fix replica set update - fail only if all failed
* Add replica redesign (#1131 )
* refactor shard/mod.rs
* wip
* fmt
* it compiles
* temporary disable replica placemeant change on replication factor
* fmt
* finish todos
* small refactoring
* remove change::add
* replica-set -> shard-replica-set
* fmt
* upd openapi
* fix finish transfer logic
* fix existing integration tests
* shard transfer validation
* fmt
* review fixes
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
2022-10-17 20:12:10 +02:00
Andrey Vasnetsov
411ddabe49
v0.10.4
2022-10-13 09:14:11 +02:00
Andrey Vasnetsov
68a46a0f2e
v0.10.3
2022-10-13 09:14:11 +02:00
Andrey Vasnetsov
b275d452a8
shard replica manipulation API ( #1121 )
2022-10-12 15:19:43 +02:00
Egor Ivkov
ca01b8ac8e
Fix: Do not rely on empty message for is_leader_established flag ( #1103 )
...
* Fix: Do not rely on empty message for `is_leader_established` flag
* Record last consensus update
* Update OpenAPI
2022-10-05 18:10:33 +03:00
Andrey Vasnetsov
935f069d53
upd version to v0.10.2
2022-09-30 11:08:16 +02:00
Ivan Pleshkov
8eedcf3735
add empty security to openapi ( #1079 )
2022-09-30 10:02:40 +02:00
Egor Ivkov
9a3fad44b2
Revert "Revert replication changes ( #1055 )" ( #1056 )
...
This reverts commit a2ed0cf5db
2022-09-23 16:46:06 +03:00
Egor Ivkov and Andrey Vasnetsov
a2ed0cf5db
Revert replication changes ( #1055 )
...
* Revert "Replication set `Change` enum"
This reverts commit 28e8487542#1053 )
* Revert "Generate API - replication follow up (#1043 )"
This reverts commit 183ea9e5e3#1037 )"
This reverts commit d690b700ff#1035 )"
This reverts commit 31c3ae482d#1009 )"
This reverts commit e332d482b0andrey@vasnetsov.com >
2022-09-23 16:33:07 +03:00
Egor Ivkov
183ea9e5e3
Generate API - replication follow up ( #1043 )
...
Also removes `JsonSchema` from structs not exposed in API
2022-09-22 14:53:00 +03:00
Andrey Vasnetsov
f918a14819
upd to v0.10.1 ( #1031 )
...
* upd to v0.10.1
* upd lock
2022-09-19 15:31:35 +02:00
Andrey Vasnetsov and Arnaud Gourlay
dc07b01e1f
remove deprecated fields from API ( #1030 )
...
* remove depricated fields from API
* fmt
* upd openapi and integration tests
* fix grpc test
* regenerate storage reference data
* improve docs
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com >
2022-09-19 13:51:03 +02:00
Andrey Vasnetsov
a39a811567
V0.10.0 ( #1029 )
...
* Revert "Rollback v0.10.0 changes before release"
This reverts commit 18b75d9069
2022-09-19 10:05:57 +02:00
Andrey Vasnetsov
03d08236a8
better struct for fts ( #1026 )
2022-09-16 14:07:50 +02:00
Ivan Pleshkov
4d3b4b132d
More multivector api updates ( #1025 )
...
* rest multivector api updates
* update openapi
* update grpc doc
2022-09-16 12:25:42 +04:00
Andrey Vasnetsov
94883e7062
allow empty with_vector
2022-09-16 01:37:34 +02:00
Andrey Vasnetsov
278b3def9b
use structure for better openapi generator compatibility ( #1020 )
2022-09-15 10:51:46 +02:00
Arnaud Gourlay
4c949283a9
update gRPC docs ( #1014 )
2022-09-14 11:49:36 +02:00
Ivan Pleshkov and Andrey Vasnetsov
f6b2186193
[WIP] Many named vectors per point ( #958 )
...
* many named vectors per point (segment-level)
* operation result for dim function
* beautifulized vector name
* fix naming bug
* segment version migration
* fmt
* add segment tests
* are you happy clippy
* fix build
* [WIP] many named vectors per point (collection-level) (#975 )
* config and search
* fix placeholders for proxy segment move
* remove VectorType from collection
* are you happy fmt
* vectors in grps messages
* create collections with vectors
* segment holder fixes
* are you happy fmt
* remove default vector name placeholders
* are you happy fmt
* are you happy clippy
* fix build
* fix web api
* are you happy clippy
* are you happy fmt
* record vector&vectors
* openapi update
* fix openapi integration tests
* segment builder fix todo
* vector names for update from segment
* remove unwrap
* backward compatibility
* upd openapi
* backward compatible PointStruct
* upd openapi
* fix record back-comp
* fmt
* vector configuration backward compatibility
* fix vetor storage size estimation
* fmt
* multi-vec segment test + index test
* fmt
* api integration tests
* [WIP] Named vectors struct (#1002 )
* move to separate file
* named vectors as struct
* use cow
* fix build
* keys iterator
* avoid copy in PointStruct -> get_vectors
* avoid another copy
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
2022-09-11 20:59:23 +02:00
Andrey Vasnetsov
c3bc5b3873
rename grpc fields + re-generate docs ( #980 )
2022-09-01 18:59:47 +02:00
Andrey Vasnetsov
b9eee55a9f
Full text search ( #963 )
...
* allow additional params for payload field index
* fmt
* wip: full text index building
* fmt
* text search request
* text search request
* full text index persitance and loading
* fmt
* enable fts index in mapping
* clippy
* fix tests + add integration test
* review fixes: extend payload index test
* revert incedental change
2022-09-01 12:50:12 +02:00
Andrey Vasnetsov
e8216e9400
do not fail if cannot get file creation time ( #973 )
2022-08-30 23:33:56 +02:00
Andre Zayarni
086216616a
Updated ETA
2022-08-22 16:37:43 +02:00
Arnaud Gourlay
03c10fc310
Batch recommendation API ( #954 )
2022-08-22 09:41:22 +02:00
Andrey Vasnetsov and Arnaud Gourlay
a2acca0345
Segment batch search ( #813 )
...
* batch search benchmark
* collect filter iterator in indexed search
* fmt
* fix
* fix
* fmt
* use new tempfile create
* auto batching
* Clippy fixes
* REST, gRPC and internal APIs
* fix bugs & less duplication
* two steps payload retrieval mechanism & fix duplication
* add proxy_segment implementation & tests
* add gRPC docs
* remove unused code (#950 )
* only filter ids within a batch
* add more equivalence tests
* add integration test search vs batch
* assert more search options in tests
* cleanup assertions
* fix offset panic
* rename search batch API
* openapi spec
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com >
2022-08-18 14:48:17 +02:00
Andrey Vasnetsov
993e94eaef
V0.9.1 ( #939 )
...
* minor fix + upd version to 0.9.1
* fmt
* fix clippy
* fix test
2022-08-10 21:15:40 +02:00
Arnaud Gourlay
935e9a21aa
update v0.9.0 ( #919 )
...
* update v0.9.0
* update gRPC docs
* update abort transfer
2022-08-08 11:13:54 +02:00
Andrey Vasnetsov
abbd3bcbf1
Fix transfer abort on restart ( #921 )
...
* await for consensus leader before submitting transfer abort + api
* fmt
2022-08-06 14:53:35 +02:00
Andrey Vasnetsov and Arnaud Gourlay
60c7131cc6
Indexed vectors in collection info ( #879 )
...
* include number of indexed points into the info api
* fmt
* await for a certain level of indexing progressing before promoting shard
* log before waiting
* await for index fixes
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com >
2022-08-03 16:19:19 +02:00
Egor Ivkov
0a634453e3
Peer removal API ( #894 )
...
* Remove peer api
* Keep the same cluster status api for compatibility
2022-08-02 15:56:41 +03:00
Ivan Pleshkov
76d9584dda
openapi for optimizators telemetry ( #893 )
2022-08-02 15:04:19 +04:00
Andrey Vasnetsov