Commit Graph

203 Commits

Author SHA1 Message Date
Kaya Gökalp
6c0f32f7d0 ci: move from actions-rs to dtolnay/rust-toolchain (#1995) 2023-05-31 10:00:32 +02:00
Roman Titov
abffa27ff0 Refactor Dockerfile (#1594)
* Refactor Dockerfile

- fix cross-compilation
- improve caching

TODO:
- check if `lld` is used/works for linkage (and enable if not used, or remove if doesn't work)

* Remove `aarch64` linker config from `.cargo/config.toml` (seems to be unnecessary)

* Expose `LINKER` argument and enable `lld` linker

* Add `mold` linker support

* Document Dockerfile

* Add closing ` in the Dockerfile comments

Co-authored-by: Tim Visée <tim+github@visee.me>

---------

Co-authored-by: Tim Visée <tim+github@visee.me>
2023-05-26 16:01:43 +02:00
Andrey Vasnetsov
5f0e3585db move linker to docker (#1863)
* move linker to docker

* move mold install to builder

* use release profile by default

* fix mold download

* add build-args to CI builds

* add missing buil args
2023-05-09 11:57:31 +02:00
Jesse
34055f0ddf Custom build profile for CI docker builds (#1859)
* Use custom build profile for docker builds

* trigger ci

* change source code to trigger ci with rebuild one more time

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2023-05-08 20:39:59 +02:00
Jesse
b67c655ddb Add caching of docker layers in CI (#1856)
* Add caching of docker layers in CI

Build required docker images for CI in a workflow step using buildkit's
gha cache type. This will populate the local layer cache from github
actions' cache. Builds in subsequent CI steps will be nearly instant,
because all layers can be reused.

* add minor change to see if build time is any faster

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2023-05-08 17:02:29 +02:00
Roman Titov
16b7fb7688 Improve handling out-of-RAM errors during Qdrant startup (#1777)
* WIP: Start working on out-of-RAM errors handling [skip ci]

* Implement basic handling of out-of-RAM errors during Qdrant startup

* Try to fix CI fail by allowing both V1 and V2 cgroups

* Try to fix CI fail by improving cgroups handling

* Fix cgroups path detection/handling (+ some minor stylistic changes)

* fixup! Fix cgroups path detection/handling (+ some minor stylistic changes)

* Add test

* Enable low RAM test

* fixup! Add test

* free memory checks

* rm unused function

* Oom fallback script (#1809)

* add recover mode in qdrant + script for handelling OOM

* fix clippy

* reformat entrypoint.sh

* fix test

* add logging to test

* fix test

* fix test

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2023-05-02 20:54:13 +02:00
Andrey Vasnetsov
62f202914d enable ci test on all branches (#1810)
* enable ci test on all branches

* rollback push trigger

* Also run workflows on branches having / in name

---------

Co-authored-by: timvisee <tim@visee.me>
2023-05-01 13:01:19 +02:00
Shubham Mishra
051dbb29e5 Added binary release workflow (#1767) 2023-04-25 12:13:48 +02:00
Tim Visée
e985ff31b6 General improvements (#1687)
* Change file mode of some .sh files to make them executable

* Remove whitespace on empty lines from YAML configurations

* Fix unused import warning on Windows
2023-04-10 20:03:56 +02:00
Tim Visée
d74bfa088c Add CI job to enforce generated OpenAPI and gRPC file consistency (#1667)
* Add test script to ensure OpenAPI files are consistent with sources

* Add CI job to test OpenAPI file consistency

* Add CI task to test gRPC file consistency

* Tweak consistency scripts a bit, touch temp file to trigger gRPC rebuild

* Don't test .gitignored files

* Mention updating the OpenAPI specification is enforced by CI

* Update CI job configuration

* Also check consistency of gRPC docs

* Rename temporary files to have a .diff prefix

* Add docs to consistency checking scripts
2023-04-10 14:38:53 +02:00
stencillogic
21f1d90cea Add tls support (#1581)
* add TLS support

* update github workflow

* fix formatting

* fix tests

* fix clippy warning

* fix PR remarks
2023-03-20 20:59:49 +01:00
Andrey Vasnetsov
f2fcf8e3e1 repo-token for protoc 2023-02-07 18:40:31 +01:00
Andrey Vasnetsov
39cc731611 Windows support (#1432)
* windows-support

* fmt

* install protoc with action

* upd WAL revision
2023-02-06 14:00:23 +01:00
Arnaud Gourlay
9f48cb8a71 Save consensus logs in case of CI failure (#1335)
* try actions/upload-artifact

* dedicated folder for consensus test logs based on pytest name

* improve naming

* add new transient test folder
2023-01-12 14:52:19 +01:00
Arnaud Gourlay
e3c6bb8a53 update tonic 0.8 & prost 0.11 (#1285)
* update tonic 0.8 & prost 0.11

* install protoc via package manager
2022-12-19 15:30:33 +01:00
dependabot[bot]
74b681f6b6 Bump Swatinem/rust-cache from 1 to 2 (#1232)
Bumps [Swatinem/rust-cache](https://github.com/Swatinem/rust-cache) from 1 to 2.
- [Release notes](https://github.com/Swatinem/rust-cache/releases)
- [Changelog](https://github.com/Swatinem/rust-cache/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Swatinem/rust-cache/compare/v1...v2)

---
updated-dependencies:
- dependency-name: Swatinem/rust-cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-21 21:01:59 +01:00
Andrey Vasnetsov
c9ad035f98 enable CI on dev (#1057)
* enable CI on dev

* update dependabot
2022-09-23 16:02:39 +02:00
Andrey Vasnetsov
01807d5cf3 Multiplatform build fix 2 (#1000)
* upd dockerfile to multi-arch build

* debug actions

* fail faster on install

* add buildplatform

* enable CI

* dedicated push for github registry

* default platform

* fix default target arch

* revert debug ci changes
2022-09-12 13:26:40 +02:00
Egor Ivkov
026bd040b0 Add import formatting rules (#820)
* Add import formatting rules

* Review fix: update rusty hook
2022-07-15 15:42:25 +03:00
Arnaud Gourlay
7cb7a9d7a8 Fix deadlock in cluster status. (#712)
* fix deadlock - do not reacquire read lock in the same thread

* integration tests use deadlock detector
2022-06-17 22:31:34 +02:00
dependabot[bot]
ce240decac Bump actions/setup-python from 3 to 4 (#687)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 3 to 4.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-14 08:57:18 +02:00
Egor Ivkov
4b83464061 Skip consensus init on restart (#638)
* Skip consensus init on restart

* Added test

* CI
2022-06-01 11:38:23 +03:00
Egor Ivkov
08b8de8692 Fix deadlock on collection creation from snapshot (#615) 2022-05-20 18:05:43 +03:00
Egor Ivkov
55ccf1fff5 Run consensus in docker-compose (#609)
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2022-05-18 19:28:17 +03:00
Egor Ivkov
27440937bc Add multipeer deployment test (#590)
* Add multipeer deployment test

And corresponding fixes for snapshot and wal logic to make test green

* Add CI for multipeer test

* Increase number of peers to 5

* Fix pytest installation

* Fix python version

* Install requests lib
2022-05-18 15:22:38 +03:00
Arnaud Gourlay
c7396525f9 Switch consensus to a runtime configuration (#565)
* switch consensus to a runtime configuration

* propose_sender is always present

* better toggle switch with ConsensusEnabled
2022-05-06 17:56:00 +02:00
Egor Ivkov
d0d1f615f2 Fixes test_alias_operation (#497) 2022-04-20 18:04:51 +03:00
Arnaud Gourlay
c09034f6e8 Install protoc to avoid having to recompile it (#452)
* Install protoc to avoid having to recompile it

* cleanup bash script call sites

* move script out of worflows

* cleaner script
2022-04-11 08:37:09 +02:00
Egor Ivkov
aeea31dfa4 Wait for operation commit (#436)
* Wait for operation commit

* Review: Option with timeout, return execution err to API

* Review: Add comment about error forwarding in ???
2022-04-06 18:00:44 +03:00
Egor Ivkov
d3e0d7dcec Integrate consensus into execution pipeline (#410)
* Integrate consensus into execution pipeline

* Review: Remove unnecessary config
2022-03-31 17:54:55 +03:00
Arnaud Gourlay
aab61623c9 remove uncessary Blas dependencies (#397) 2022-03-21 16:00:38 +01:00
dependabot[bot]
7275577102 Bump actions/checkout from 2 to 3 (#390)
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-18 12:01:30 +01:00
Ivan Pleshkov
2c4fd0a205 Merge branch 'master' into remove-blas 2022-03-09 15:48:25 +00:00
Arnaud Gourlay
c928217a8c enable gRPC via a runtime config (#352)
* enable gRPC via a runtime config

* code review

* code review: rename rest_port to http_port

* Update config/config.yaml

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>

* disable grpc by default

* enable gPRC in integration test

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2022-03-03 19:23:31 +01:00
Arnaud Gourlay
f902cbbfaf dedicated cache for Clippy build (#337) 2022-02-23 10:58:51 +01:00
Arnaud Gourlay
5a78384e88 Integration tests use dedicated key cache to avoid false sharing (#335)
* build outside of bash script to use cache

* add cache key to dissociate gRPC builds
2022-02-22 12:18:07 +01:00
Arnaud Gourlay
5a9f16ebfd setup github action cache (#320) 2022-02-18 15:09:53 +01:00
Arnaud Gourlay
2ced2b43f8 validate storage compatibility - (#286) (#289) 2022-02-08 20:22:24 +01:00
Arnaud Gourlay
5a5da0a16d use existing integration tests as CI (#279) 2022-02-03 19:22:39 +01:00
Prokudin Alexander
ee461ce0a6 Extend clippy to workspace and fix some warnings (#199)
* Fix clippy in linting workflow

* Add toolchain override flag

* Add components to toolchain installation explicitly

* Add --workspace flag to clippy to check all packages

* Remove unnecessary clones

* remove redundant .clone() calls

* fix wrong arguments order in tests (typo)

* Fix vec! macro usage in test

* Correct redundant assert! usages

* Provide a quick fix for 'unused' test function lint

* fix unsound Send + Sync

* fix clippy complains

* fmt

* fix clippy

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2022-01-17 23:33:26 +01:00
Ivan Pleshkov
d0cbfe4cb7 remove blas dependency 2022-01-04 17:51:53 +03:00
Andrey Vasnetsov
a695f61f9f update test requirements 2021-12-26 20:30:42 +01:00
Andrey Vasnetsov
c607ec41bd Push images to docker-hub (#102) 2021-10-04 17:57:46 +03:00
Konstantin
93e0fb5c2c [CLIPPY] Fix the last portion of rules and enable CI check (#53)
* [CLIPPY] Fixed the warning for references of the user defined types

* [CLIPPY] Fix module naming issue

* [CLIPPY] Fix the last set of warnings and enable clippy check during CI

* Moved cargo fmt and cargo clippy into it's own action
2021-07-03 13:12:21 +02:00
Konstantin
a667747369 Applied and enforced rust fmt code formatting tool (#48)
* Apply cargo fmt command

* Enabled cargo fmt on build
2021-06-29 10:17:50 +02:00
Andrey Vasnetsov
84ee5dd67b Allow manual trigger docker build 2021-06-20 19:29:05 +02:00
Andrey Vasnetsov
02d3cbe615 Update docker-image.yml 2021-04-06 20:07:43 +02:00
Andrey Vasnetsov
9233bd9b84 Create docker-image.yml 2021-04-06 20:06:51 +02:00
Andrey Vasnetsov
20ad71507e fix test script 2021-01-24 17:19:19 +01:00
Andrey Vasnetsov
ed7992196b add dependencies installation before tests 2021-01-05 14:23:36 +01:00