Commit Graph
57 Commits
Author SHA1 Message Date
Yaroslav Halchenko 6624c95afd codespell: workflow, config, typos fixed (#2248)
* Add github action to codespell master on push and PRs

* Add rudimentary codespell config

* some skips

* fix some ambigous typos

* [DATALAD RUNCMD] run codespell throughout

=== Do not change lines below ===
{
 "chain": [],
 "cmd": "codespell -w",
 "exit": 0,
 "extra_inputs": [],
 "inputs": [],
 "outputs": [],
 "pwd": "."
}
^^^ Do not change lines above ^^^

* Add dev branch as target for the workflow
2023-07-12 16:30:42 +02:00
Tim Visée 65876f7e8b Also run CI tests on macOS (#2227)
* Also test on macOS

* Remove trailing spaces from GitHub Workflow configurations
2023-07-10 11:36:06 +02:00
Tim Visée 36ef2d718a Fix macOS binary release job (#2228)
* Attempt to fix macOS binary release job

* Trigger release binary jobs on release publish, not on create
2023-07-10 11:35:03 +02:00
Arnaud Gourlay 3f3a4ca5bb Cancel sooner consensus test and upload logs (#2102) 2023-06-19 14:16:22 +02:00
Kaya Gökalp 6c0f32f7d0 ci: move from actions-rs to dtolnay/rust-toolchain (#1995) 2023-05-31 10:00:32 +02:00
Roman TitovandTim Visée abffa27ff0 Refactor Dockerfile (#1594)
* Refactor Dockerfile

- fix cross-compilation
- improve caching

TODO:
- check if `lld` is used/works for linkage (and enable if not used, or remove if doesn't work)

* Remove `aarch64` linker config from `.cargo/config.toml` (seems to be unnecessary)

* Expose `LINKER` argument and enable `lld` linker

* Add `mold` linker support

* Document Dockerfile

* Add closing ` in the Dockerfile comments

Co-authored-by: Tim Visée <tim+github@visee.me>

---------

Co-authored-by: Tim Visée <tim+github@visee.me>
2023-05-26 16:01:43 +02:00
Andrey Vasnetsov 5f0e3585db move linker to docker (#1863)
* move linker to docker

* move mold install to builder

* use release profile by default

* fix mold download

* add build-args to CI builds

* add missing buil args
2023-05-09 11:57:31 +02:00
JesseandAndrey Vasnetsov 34055f0ddf Custom build profile for CI docker builds (#1859)
* Use custom build profile for docker builds

* trigger ci

* change source code to trigger ci with rebuild one more time

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2023-05-08 20:39:59 +02:00
JesseandAndrey Vasnetsov b67c655ddb Add caching of docker layers in CI (#1856)
* Add caching of docker layers in CI

Build required docker images for CI in a workflow step using buildkit's
gha cache type. This will populate the local layer cache from github
actions' cache. Builds in subsequent CI steps will be nearly instant,
because all layers can be reused.

* add minor change to see if build time is any faster

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2023-05-08 17:02:29 +02:00
Roman TitovandAndrey Vasnetsov 16b7fb7688 Improve handling out-of-RAM errors during Qdrant startup (#1777)
* WIP: Start working on out-of-RAM errors handling [skip ci]

* Implement basic handling of out-of-RAM errors during Qdrant startup

* Try to fix CI fail by allowing both V1 and V2 cgroups

* Try to fix CI fail by improving cgroups handling

* Fix cgroups path detection/handling (+ some minor stylistic changes)

* fixup! Fix cgroups path detection/handling (+ some minor stylistic changes)

* Add test

* Enable low RAM test

* fixup! Add test

* free memory checks

* rm unused function

* Oom fallback script (#1809)

* add recover mode in qdrant + script for handelling OOM

* fix clippy

* reformat entrypoint.sh

* fix test

* add logging to test

* fix test

* fix test

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2023-05-02 20:54:13 +02:00
Andrey Vasnetsovandtimvisee 62f202914d enable ci test on all branches (#1810)
* enable ci test on all branches

* rollback push trigger

* Also run workflows on branches having / in name

---------

Co-authored-by: timvisee <tim@visee.me>
2023-05-01 13:01:19 +02:00
Shubham Mishra 051dbb29e5 Added binary release workflow (#1767) 2023-04-25 12:13:48 +02:00
Tim Visée e985ff31b6 General improvements (#1687)
* Change file mode of some .sh files to make them executable

* Remove whitespace on empty lines from YAML configurations

* Fix unused import warning on Windows
2023-04-10 20:03:56 +02:00
Tim Visée d74bfa088c Add CI job to enforce generated OpenAPI and gRPC file consistency (#1667)
* Add test script to ensure OpenAPI files are consistent with sources

* Add CI job to test OpenAPI file consistency

* Add CI task to test gRPC file consistency

* Tweak consistency scripts a bit, touch temp file to trigger gRPC rebuild

* Don't test .gitignored files

* Mention updating the OpenAPI specification is enforced by CI

* Update CI job configuration

* Also check consistency of gRPC docs

* Rename temporary files to have a .diff prefix

* Add docs to consistency checking scripts
2023-04-10 14:38:53 +02:00
stencillogic 21f1d90cea Add tls support (#1581)
* add TLS support

* update github workflow

* fix formatting

* fix tests

* fix clippy warning

* fix PR remarks
2023-03-20 20:59:49 +01:00
Andrey Vasnetsov f2fcf8e3e1 repo-token for protoc 2023-02-07 18:40:31 +01:00
Andrey Vasnetsov 39cc731611 Windows support (#1432)
* windows-support

* fmt

* install protoc with action

* upd WAL revision
2023-02-06 14:00:23 +01:00
Arnaud Gourlay 9f48cb8a71 Save consensus logs in case of CI failure (#1335)
* try actions/upload-artifact

* dedicated folder for consensus test logs based on pytest name

* improve naming

* add new transient test folder
2023-01-12 14:52:19 +01:00
Arnaud Gourlay e3c6bb8a53 update tonic 0.8 & prost 0.11 (#1285)
* update tonic 0.8 & prost 0.11

* install protoc via package manager
2022-12-19 15:30:33 +01:00
dependabot[bot] 74b681f6b6 Bump Swatinem/rust-cache from 1 to 2 (#1232)
Bumps [Swatinem/rust-cache](https://github.com/Swatinem/rust-cache) from 1 to 2.
- [Release notes](https://github.com/Swatinem/rust-cache/releases)
- [Changelog](https://github.com/Swatinem/rust-cache/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Swatinem/rust-cache/compare/v1...v2)

---
updated-dependencies:
- dependency-name: Swatinem/rust-cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-21 21:01:59 +01:00
Andrey Vasnetsov c9ad035f98 enable CI on dev (#1057)
* enable CI on dev

* update dependabot
2022-09-23 16:02:39 +02:00
Andrey Vasnetsov 01807d5cf3 Multiplatform build fix 2 (#1000)
* upd dockerfile to multi-arch build

* debug actions

* fail faster on install

* add buildplatform

* enable CI

* dedicated push for github registry

* default platform

* fix default target arch

* revert debug ci changes
2022-09-12 13:26:40 +02:00
Egor Ivkov 026bd040b0 Add import formatting rules (#820)
* Add import formatting rules

* Review fix: update rusty hook
2022-07-15 15:42:25 +03:00
Arnaud Gourlay 7cb7a9d7a8 Fix deadlock in cluster status. (#712)
* fix deadlock - do not reacquire read lock in the same thread

* integration tests use deadlock detector
2022-06-17 22:31:34 +02:00
dependabot[bot] ce240decac Bump actions/setup-python from 3 to 4 (#687)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 3 to 4.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-14 08:57:18 +02:00
Egor Ivkov 4b83464061 Skip consensus init on restart (#638)
* Skip consensus init on restart

* Added test

* CI
2022-06-01 11:38:23 +03:00
Egor Ivkov 08b8de8692 Fix deadlock on collection creation from snapshot (#615) 2022-05-20 18:05:43 +03:00
Egor IvkovandAndrey Vasnetsov 55ccf1fff5 Run consensus in docker-compose (#609)
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2022-05-18 19:28:17 +03:00
Egor Ivkov 27440937bc Add multipeer deployment test (#590)
* Add multipeer deployment test

And corresponding fixes for snapshot and wal logic to make test green

* Add CI for multipeer test

* Increase number of peers to 5

* Fix pytest installation

* Fix python version

* Install requests lib
2022-05-18 15:22:38 +03:00
Arnaud Gourlay c7396525f9 Switch consensus to a runtime configuration (#565)
* switch consensus to a runtime configuration

* propose_sender is always present

* better toggle switch with ConsensusEnabled
2022-05-06 17:56:00 +02:00
Egor Ivkov d0d1f615f2 Fixes test_alias_operation (#497) 2022-04-20 18:04:51 +03:00
Arnaud Gourlay c09034f6e8 Install protoc to avoid having to recompile it (#452)
* Install protoc to avoid having to recompile it

* cleanup bash script call sites

* move script out of worflows

* cleaner script
2022-04-11 08:37:09 +02:00
Egor Ivkov aeea31dfa4 Wait for operation commit (#436)
* Wait for operation commit

* Review: Option with timeout, return execution err to API

* Review: Add comment about error forwarding in ???
2022-04-06 18:00:44 +03:00
Egor Ivkov d3e0d7dcec Integrate consensus into execution pipeline (#410)
* Integrate consensus into execution pipeline

* Review: Remove unnecessary config
2022-03-31 17:54:55 +03:00
Arnaud Gourlay aab61623c9 remove uncessary Blas dependencies (#397) 2022-03-21 16:00:38 +01:00
dependabot[bot] 7275577102 Bump actions/checkout from 2 to 3 (#390)
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-18 12:01:30 +01:00
Ivan Pleshkov 2c4fd0a205 Merge branch 'master' into remove-blas 2022-03-09 15:48:25 +00:00
Arnaud GourlayandAndrey Vasnetsov c928217a8c enable gRPC via a runtime config (#352)
* enable gRPC via a runtime config

* code review

* code review: rename rest_port to http_port

* Update config/config.yaml

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>

* disable grpc by default

* enable gPRC in integration test

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2022-03-03 19:23:31 +01:00
Arnaud Gourlay f902cbbfaf dedicated cache for Clippy build (#337) 2022-02-23 10:58:51 +01:00
Arnaud Gourlay 5a78384e88 Integration tests use dedicated key cache to avoid false sharing (#335)
* build outside of bash script to use cache

* add cache key to dissociate gRPC builds
2022-02-22 12:18:07 +01:00
Arnaud Gourlay 5a9f16ebfd setup github action cache (#320) 2022-02-18 15:09:53 +01:00
Arnaud Gourlay 2ced2b43f8 validate storage compatibility - (#286) (#289) 2022-02-08 20:22:24 +01:00
Arnaud Gourlay 5a5da0a16d use existing integration tests as CI (#279) 2022-02-03 19:22:39 +01:00
Prokudin AlexanderandAndrey Vasnetsov ee461ce0a6 Extend clippy to workspace and fix some warnings (#199)
* Fix clippy in linting workflow

* Add toolchain override flag

* Add components to toolchain installation explicitly

* Add --workspace flag to clippy to check all packages

* Remove unnecessary clones

* remove redundant .clone() calls

* fix wrong arguments order in tests (typo)

* Fix vec! macro usage in test

* Correct redundant assert! usages

* Provide a quick fix for 'unused' test function lint

* fix unsound Send + Sync

* fix clippy complains

* fmt

* fix clippy

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2022-01-17 23:33:26 +01:00
Ivan Pleshkov d0cbfe4cb7 remove blas dependency 2022-01-04 17:51:53 +03:00
Andrey Vasnetsov a695f61f9f update test requirements 2021-12-26 20:30:42 +01:00
Andrey Vasnetsov c607ec41bd Push images to docker-hub (#102) 2021-10-04 17:57:46 +03:00
Konstantin 93e0fb5c2c [CLIPPY] Fix the last portion of rules and enable CI check (#53)
* [CLIPPY] Fixed the warning for references of the user defined types

* [CLIPPY] Fix module naming issue

* [CLIPPY] Fix the last set of warnings and enable clippy check during CI

* Moved cargo fmt and cargo clippy into it's own action
2021-07-03 13:12:21 +02:00
Konstantin a667747369 Applied and enforced rust fmt code formatting tool (#48)
* Apply cargo fmt command

* Enabled cargo fmt on build
2021-06-29 10:17:50 +02:00
Andrey Vasnetsov 84ee5dd67b Allow manual trigger docker build 2021-06-20 19:29:05 +02:00