Andrey Vasnetsov
30a432690e
add random fail for test
2023-01-05 12:32:10 +01:00
Andrey Vasnetsov
926a058c73
Fixing missed leader inconsistency ( #1286 )
...
* WIP: introduce local state
* WIP: sync local state with consensus in idle
* fmt
* rm extensive debug
* update triple replication test
* update triple replication test
* rm debug logs
* fix check for established leader + only sync local if no proposals
* remove unused file
* test compatible with python 3.8
* test compatible with python 3.8
* longer wait for consensus
* longer wait for consensus
* extra sleep in test
* remove extra sleep
* Fixing missed leader inconsistency - transfers (#1298 )
* explicit request timeout
* explicit request timeout
* explicit request timeout
* explicit request timeout
* explicit request timeout
* prevent double handelling of the transfer termination
* kill the process
* log on inconsistency
* log on inconsistency
* log on inconsistency
* debug
* revert debug in test
* forward updates to partial shards, abort transactions on dead node report
* fmt
* disable retry of transfer, if the transfer was cancelled + allow predictable ports in test
* fix import
2022-12-23 00:45:04 +01:00
Andrey Vasnetsov
a2c5de586b
Recommendation with other collection ( #1283 )
...
* WIP: move recommendation API out of collection
* recomendations with external source of vectors
* fix test
* fmt
* fixes + test
* review fixes
2022-12-22 09:29:36 +01:00
dependabot[bot]
32208e700b
Bump prost-types from 0.11.2 to 0.11.5 ( #1300 )
...
Bumps [prost-types](https://github.com/tokio-rs/prost ) from 0.11.2 to 0.11.5.
- [Release notes](https://github.com/tokio-rs/prost/releases )
- [Commits](https://github.com/tokio-rs/prost/compare/v0.11.2...v0.11.5 )
---
updated-dependencies:
- dependency-name: prost-types
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-21 20:30:51 +01:00
dependabot[bot]
e57f87a7ba
Bump prost from 0.11.3 to 0.11.5 ( #1295 )
...
Bumps [prost](https://github.com/tokio-rs/prost ) from 0.11.3 to 0.11.5.
- [Release notes](https://github.com/tokio-rs/prost/releases )
- [Commits](https://github.com/tokio-rs/prost/compare/v0.11.3...v0.11.5 )
---
updated-dependencies:
- dependency-name: prost
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-20 19:34:42 +01:00
Arnaud Gourlay
584cdfa801
update all dependencies Cargo.lock ( #1293 )
2022-12-20 10:30:49 +01:00
Arnaud Gourlay
e3c6bb8a53
update tonic 0.8 & prost 0.11 ( #1285 )
...
* update tonic 0.8 & prost 0.11
* install protoc via package manager
2022-12-19 15:30:33 +01:00
Andrey Vasnetsov
d05b2def10
allow set different M param for default HNSW and payload-related links ( #1257 )
...
* allow set different M param for default HNSW and payload-related links
* review fixes
2022-12-06 13:53:42 +01:00
dependabot[bot]
c1caf271e5
Bump tokio from 1.22.0 to 1.23.0 ( #1267 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.22.0 to 1.23.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.22.0...tokio-1.23.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-06 13:24:31 +01:00
Andrey Vasnetsov
04434f02b6
Update points by filter ( #1249 )
...
* allow to set remove payload by selector
* fmt
2022-12-01 09:16:07 +01:00
Andrey Vasnetsov
9f36ac60ee
Full payload update ( #1245 )
...
* implement api to fully overwrite payload of the point
* fmt
2022-11-28 13:46:58 +01:00
Ivan Pleshkov and Andrey Vasnetsov
ac9538d5af
Hnsw links memmap ( #1211 )
...
* add on disk key
* remove obsolete graph initialization
* remove obsolete max level
* update openapi
* graph links trait
* use mmap option
* same format for ram and mmap
* fix segment unit tests
* are you happy fmt
* are you happy clippy
* fix ci and add mmap test
* review fixes
* remove unused try-from
* fix version compatibility
* avoid loading from disk during conversion
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
2022-11-25 11:47:52 +01:00
dependabot[bot]
18a67686df
Bump tokio from 1.21.2 to 1.22.0 ( #1234 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.21.2 to 1.22.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.21.2...tokio-1.22.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-21 20:28:57 +01:00
Arnaud Gourlay
6266d9bcdc
enable compression for gRPC ( #1186 )
...
* enable compression for gRPC
* add benchmarks
* fix grpc benchmark
* discardResponseBodies to reduce memory usage
* improve & split benchmarks
* do not use gzip for inter node communication as benchmarks are not conclusive
2022-11-18 13:34:49 +01:00
98fc5b2fa4
0.11.1 cherry picks ( #1193 )
...
* Complete feature road-map
* docs: add erare-humanum as a contributor for code (#1181 )
* docs: update README.md [skip ci]
* docs: update .all-contributorsrc [skip ci]
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
* update rust version and fix clippy complains (#1190 )
* handle collections transition from single-node to cluster (#1189 )
* Update to v0.11.1 (#1191 )
* update to v0.11.1
* also default version
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
* consensus_state to consensus_state
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
2022-11-07 11:31:12 +01:00
286ca9f5e4
0.11.0 cherry picks ( #1174 )
...
* implement concern-factor, forbig disabling the last node, improve tes… (#1168 )
* implement concern-factor, forbig disabling the last node, improve test stability
* upd docs
* improve test_recover_dead_node
* rename to write_consistency_factor
* Fix bug: applied index higher than commit on restart (#1172 )
* create collection with default status = dead + await for activation on create_collecton_op_submit
* fmt
* fix unit tests
Co-authored-by: Egor Ivkov <e.o.ivkov@gmail.com >
* upd integration tests
* fix transaction handling
* upd protobuf
* upd to v0.11.0
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
Co-authored-by: Egor Ivkov <e.o.ivkov@gmail.com >
2022-10-27 10:16:21 +02:00
Andrey Vasnetsov
21754ef2ef
report indexed payload points in info api + other fixes ( #1164 )
...
* report indexed payload points in info api + other fixes
* rollback debug changes
* clippy
* clippy
2022-10-24 09:43:02 +02:00
Ivan Pleshkov and Andrey Vasnetsov
44626bc80a
exact search parameter ( #1150 )
...
* exact search parameter
* exact search test
* decrease attempts count because of test speed
* minor review fixes
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
2022-10-20 15:39:21 +04:00
Andrey Vasnetsov
cf29b67640
v0.10.5
2022-10-18 10:49:43 +02:00
Andrey Vasnetsov
411ddabe49
v0.10.4
2022-10-13 09:14:11 +02:00
Andrey Vasnetsov
68a46a0f2e
v0.10.3
2022-10-13 09:14:11 +02:00
dependabot[bot]
d0789f3a90
Bump uuid from 1.1.2 to 1.2.1 ( #1111 )
...
Bumps [uuid](https://github.com/uuid-rs/uuid ) from 1.1.2 to 1.2.1.
- [Release notes](https://github.com/uuid-rs/uuid/releases )
- [Commits](https://github.com/uuid-rs/uuid/compare/1.1.2...1.2.1 )
---
updated-dependencies:
- dependency-name: uuid
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-11 08:25:24 +02:00
dependabot[bot]
814b0ebcb4
Bump schemars from 0.8.10 to 0.8.11 ( #1089 )
...
Bumps [schemars](https://github.com/GREsau/schemars ) from 0.8.10 to 0.8.11.
- [Release notes](https://github.com/GREsau/schemars/releases )
- [Changelog](https://github.com/GREsau/schemars/blob/master/CHANGELOG.md )
- [Commits](https://github.com/GREsau/schemars/compare/v0.8.10...v0.8.11 )
---
updated-dependencies:
- dependency-name: schemars
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-04 11:17:22 +02:00
Andrey Vasnetsov
935f069d53
upd version to v0.10.2
2022-09-30 11:08:16 +02:00
dependabot[bot]
fd9c39f1ab
Bump tokio from 1.21.1 to 1.21.2 ( #1067 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.21.1 to 1.21.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.21.1...tokio-1.21.2 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-28 11:45:26 +02:00
Egor Ivkov
9a3fad44b2
Revert "Revert replication changes ( #1055 )" ( #1056 )
...
This reverts commit a2ed0cf5db
2022-09-23 16:46:06 +03:00
Egor Ivkov and Andrey Vasnetsov
a2ed0cf5db
Revert replication changes ( #1055 )
...
* Revert "Replication set `Change` enum"
This reverts commit 28e8487542#1053 )
* Revert "Generate API - replication follow up (#1043 )"
This reverts commit 183ea9e5e3#1037 )"
This reverts commit d690b700ff#1035 )"
This reverts commit 31c3ae482d#1009 )"
This reverts commit e332d482b0andrey@vasnetsov.com >
2022-09-23 16:33:07 +03:00
Egor Ivkov
183ea9e5e3
Generate API - replication follow up ( #1043 )
...
Also removes `JsonSchema` from structs not exposed in API
2022-09-22 14:53:00 +03:00
Egor Ivkov
d690b700ff
Replication related API changes ( #1037 )
...
* Replication related API changes
* minor proto file fix
2022-09-21 18:49:47 +03:00
Andrey Vasnetsov
f918a14819
upd to v0.10.1 ( #1031 )
...
* upd to v0.10.1
* upd lock
2022-09-19 15:31:35 +02:00
Andrey Vasnetsov and Arnaud Gourlay
dc07b01e1f
remove deprecated fields from API ( #1030 )
...
* remove depricated fields from API
* fmt
* upd openapi and integration tests
* fix grpc test
* regenerate storage reference data
* improve docs
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com >
2022-09-19 13:51:03 +02:00
Andrey Vasnetsov
a39a811567
V0.10.0 ( #1029 )
...
* Revert "Rollback v0.10.0 changes before release"
This reverts commit 18b75d9069
2022-09-19 10:05:57 +02:00
Andrey Vasnetsov
03d08236a8
better struct for fts ( #1026 )
2022-09-16 14:07:50 +02:00
dependabot[bot]
04351c9637
Bump tokio from 1.21.0 to 1.21.1 ( #1010 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.21.0 to 1.21.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.21.0...tokio-1.21.1 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-14 09:17:04 +02:00
Ivan Pleshkov and Andrey Vasnetsov
f6b2186193
[WIP] Many named vectors per point ( #958 )
...
* many named vectors per point (segment-level)
* operation result for dim function
* beautifulized vector name
* fix naming bug
* segment version migration
* fmt
* add segment tests
* are you happy clippy
* fix build
* [WIP] many named vectors per point (collection-level) (#975 )
* config and search
* fix placeholders for proxy segment move
* remove VectorType from collection
* are you happy fmt
* vectors in grps messages
* create collections with vectors
* segment holder fixes
* are you happy fmt
* remove default vector name placeholders
* are you happy fmt
* are you happy clippy
* fix build
* fix web api
* are you happy clippy
* are you happy fmt
* record vector&vectors
* openapi update
* fix openapi integration tests
* segment builder fix todo
* vector names for update from segment
* remove unwrap
* backward compatibility
* upd openapi
* backward compatible PointStruct
* upd openapi
* fix record back-comp
* fmt
* vector configuration backward compatibility
* fix vetor storage size estimation
* fmt
* multi-vec segment test + index test
* fmt
* api integration tests
* [WIP] Named vectors struct (#1002 )
* move to separate file
* named vectors as struct
* use cow
* fix build
* keys iterator
* avoid copy in PointStruct -> get_vectors
* avoid another copy
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
2022-09-11 20:59:23 +02:00
24ac939b4a
Sync Points API ( #985 )
...
* shard sync operation
* fnt
* tests
* fix test
* Update lib/collection/src/collection_manager/segments_updater.rs
Co-authored-by: Egor Ivkov <e.o.ivkov@gmail.com >
* Update lib/collection/src/collection_manager/segments_updater.rs
Co-authored-by: Egor Ivkov <e.o.ivkov@gmail.com >
* match payload after vector only
* Update lib/collection/src/collection_manager/segments_updater.rs
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com >
Co-authored-by: Egor Ivkov <e.o.ivkov@gmail.com >
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com >
2022-09-05 15:09:39 +02:00
dependabot[bot]
a261d4f25e
Bump tokio from 1.20.1 to 1.21.0 ( #983 )
2022-09-05 09:31:21 +00:00
Andrey Vasnetsov
c3bc5b3873
rename grpc fields + re-generate docs ( #980 )
2022-09-01 18:59:47 +02:00
Andrey Vasnetsov
b9eee55a9f
Full text search ( #963 )
...
* allow additional params for payload field index
* fmt
* wip: full text index building
* fmt
* text search request
* text search request
* full text index persitance and loading
* fmt
* enable fts index in mapping
* clippy
* fix tests + add integration test
* review fixes: extend payload index test
* revert incedental change
2022-09-01 12:50:12 +02:00
Arnaud Gourlay
03c10fc310
Batch recommendation API ( #954 )
2022-08-22 09:41:22 +02:00
Andrey Vasnetsov and Arnaud Gourlay
a2acca0345
Segment batch search ( #813 )
...
* batch search benchmark
* collect filter iterator in indexed search
* fmt
* fix
* fix
* fmt
* use new tempfile create
* auto batching
* Clippy fixes
* REST, gRPC and internal APIs
* fix bugs & less duplication
* two steps payload retrieval mechanism & fix duplication
* add proxy_segment implementation & tests
* add gRPC docs
* remove unused code (#950 )
* only filter ids within a batch
* add more equivalence tests
* add integration test search vs batch
* assert more search options in tests
* cleanup assertions
* fix offset panic
* rename search batch API
* openapi spec
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com >
2022-08-18 14:48:17 +02:00
Andrey Vasnetsov
993e94eaef
V0.9.1 ( #939 )
...
* minor fix + upd version to 0.9.1
* fmt
* fix clippy
* fix test
2022-08-10 21:15:40 +02:00
Egor Ivkov
a4e760af34
AddPeer in one consensus operation ( #923 )
...
* AddPeer in one consensus operation
2022-08-09 17:00:38 +03:00
Arnaud Gourlay
935e9a21aa
update v0.9.0 ( #919 )
...
* update v0.9.0
* update gRPC docs
* update abort transfer
2022-08-08 11:13:54 +02:00
Andrey Vasnetsov
00e965fc96
Configuration change on peer removal ( #914 )
...
* propose configuration change along with removal of the peer
* fmt
2022-08-04 14:47:49 +02:00
Andrey Vasnetsov and Arnaud Gourlay
60c7131cc6
Indexed vectors in collection info ( #879 )
...
* include number of indexed points into the info api
* fmt
* await for a certain level of indexing progressing before promoting shard
* log before waiting
* await for index fixes
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com >
2022-08-03 16:19:19 +02:00
Andrey Vasnetsov
826165cfcf
fix enum conversion in gRPC ( #898 )
2022-08-03 09:48:30 +02:00
Andrey Vasnetsov
bae041ecb4
upd version to 0.8.6
2022-07-26 19:28:53 +02:00
dependabot[bot]
79c8e0ef8b
Bump tokio from 1.20.0 to 1.20.1 ( #856 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.20.0 to 1.20.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.20.0...tokio-1.20.1 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-26 08:58:16 +02:00
Andrey Vasnetsov
314f139183
upd v0.8.5 ( #852 )
2022-07-25 20:05:07 +02:00
Andrey Vasnetsov