* feat: Print commit id in dev container build workflow
* fix: Keep .git while building Qdrant binary in docker
* fix: Remove redundant printing of git commit id
* fix: Copy only git files first
* fix: Docker build should have commit id if present
* refactor: Use fewer lines of code
* feat: Expose git commit id in the health check endpoint
* fix: CI errors
* test: Add test for health check api
* feat: Add / endpoint to openapi schema
* Make git commit hash optional
* ci: Enable debugging setup-protoc action
* Install later protobuf compiler through GitHub Action
* Disable debug mode for setup-protoc job
* refactor: Use commit instead of commit_id
* fix: Use commit instead of commit_id gRPC docs
* test: Update ping API test
* refactor: Rename ping api to root api
---------
Co-authored-by: timvisee <tim@visee.me>
* ci: Preserve images from the dev branch
* fix: Trigger build for dev containers on dev branch merge
* refactor: Rename jobs and remove unused code
* recover dockerhub credentials from the dev build CI
* fix typo
* add workflow dispatch
* switch to github token [skip ci]
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* Read-only API keys
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
Correct placement of OpenAPI security
Place regex dep with actix/tonic
* Read-only API keys
* Replace with pytests
* API Key tests run on the same job
* Drop allow dead-code
* Rename setting key
* Containerized tests
* No special config files
* DRY
* refactor: re-use can_write method
* refactor: replace static by constants
* refactor: get PID from `$!`
* refactor: use explicit brackets on boolean condition
* style: fix identation
* small fixes + account for new APIs
* specify security in openapi
* small fix + chmod for .sh testfile
* add best-efford check for api consistency
---------
Co-authored-by: Amr Hassan <amr.hassan@gmail.com>
Co-authored-by: generall <andrey@vasnetsov.com>
* WIP: Add proper HTTPS support for remote snapshot downloads to snapshot recover APIs
* Implement HTTPS client configuration
* fixup! Implement HTTPS client configuration
Move HTTPS client configuration from `actix/certificate_helpers.rs` to
`common/http_client.rs`
* Initialize and propagate HTTPS client to shard snapshot API
* Use reqwest client reference where possible
* Add integration test
* Simplify HTTP(S) client initialization
* fixup! Simplify HTTP(S) client initialization
* Fix lifetime conflicts after rebase on dev
* Add comment to elaborate on PEM concatenation
* fixup! Add integration test
Add `test_tls_snapshot_shard_transfer.sh` run to the existing TLS test job
instead of creating a new one
---------
Co-authored-by: timvisee <tim@visee.me>
* improve automatic issue creation
- restrict to only create automatic issues on PRs of authors of the org, or on dev/main
- modify link to failed job, currently this only applies to the issue reporting job, because it's hard to link them to the actual flaky failure run, but it already improves it.
- reduce retries to only 2. Having 3 tries in total feels more than enough.
* debug author association
* ignore issue updating failure
* cleanup
* ci: use nextest for running rust tests
- enable retries with exponential backoff
- adjust test_building_cancellation to run sequentially
* run whole suite in partitions
* nextest.toml: remove profile override for specific test, lower amount of retries
* reuse builds in rust tests
* fixup rust.yml
* ditch partitioned run
* [empty commit] trigger ci
* automate flaky test reporting
* remove step name
* rename to Rust tests
* rebase + checkout only the issue template
* specify saving junit report in nextest.toml
* fix xq command, add postprocess for macos, update download and upload actions
* Build an unprivileged container image
This adds building an additional container image with the `-unprivileged` suffix. The resulting container is not running with a root user. Otherwise it is the same as the standard qdrant container image.
Having an image that is running as nonRoot is necessary in a lot of enterprise environments due to security and compliance restrictions.
* Update to use only one Dockerfile with buildargs and one action job to reuse the cached layers from the privileged image
* Also build latest-unprivileged and address feedback
* add OCI labels on docker image
* set version in build script
* Update .github/workflows/docker-image.yml
Co-authored-by: Tim Visée <tim+github@visee.me>
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
* Add tests for create collection after dropping
* Decrease test collection creation after dropping to 100 iterations
* Upload 50 points on every peer
* Upload more points, 20 calls to every peer
* Set shard number and replication factor
---------
Co-authored-by: timvisee <tim@visee.me>
* Add github action to codespell master on push and PRs
* Add rudimentary codespell config
* some skips
* fix some ambigous typos
* [DATALAD RUNCMD] run codespell throughout
=== Do not change lines below ===
{
"chain": [],
"cmd": "codespell -w",
"exit": 0,
"extra_inputs": [],
"inputs": [],
"outputs": [],
"pwd": "."
}
^^^ Do not change lines above ^^^
* Add dev branch as target for the workflow
* Refactor Dockerfile
- fix cross-compilation
- improve caching
TODO:
- check if `lld` is used/works for linkage (and enable if not used, or remove if doesn't work)
* Remove `aarch64` linker config from `.cargo/config.toml` (seems to be unnecessary)
* Expose `LINKER` argument and enable `lld` linker
* Add `mold` linker support
* Document Dockerfile
* Add closing ` in the Dockerfile comments
Co-authored-by: Tim Visée <tim+github@visee.me>
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
* Use custom build profile for docker builds
* trigger ci
* change source code to trigger ci with rebuild one more time
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* Add caching of docker layers in CI
Build required docker images for CI in a workflow step using buildkit's
gha cache type. This will populate the local layer cache from github
actions' cache. Builds in subsequent CI steps will be nearly instant,
because all layers can be reused.
* add minor change to see if build time is any faster
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* WIP: Start working on out-of-RAM errors handling [skip ci]
* Implement basic handling of out-of-RAM errors during Qdrant startup
* Try to fix CI fail by allowing both V1 and V2 cgroups
* Try to fix CI fail by improving cgroups handling
* Fix cgroups path detection/handling (+ some minor stylistic changes)
* fixup! Fix cgroups path detection/handling (+ some minor stylistic changes)
* Add test
* Enable low RAM test
* fixup! Add test
* free memory checks
* rm unused function
* Oom fallback script (#1809)
* add recover mode in qdrant + script for handelling OOM
* fix clippy
* reformat entrypoint.sh
* fix test
* add logging to test
* fix test
* fix test
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* enable ci test on all branches
* rollback push trigger
* Also run workflows on branches having / in name
---------
Co-authored-by: timvisee <tim@visee.me>
