The recover-remote-concurrent step can hang indefinitely with S3 storage;
cap CI steps at 15 minutes and curl requests at 120 seconds so stuck runs
do not consume the full 6-hour job timeout.
Co-authored-by: Cursor <cursoragent@cursor.com>
* ci: fix rust-cache running before checkout in shard-snapshot job
The `test-shard-snapshot-api-s3-minio` job ran `Swatinem/rust-cache`
before `actions/checkout`, so there was no `Cargo.lock`/source present
when the cache action ran. As a result the action no-opped and the job
never actually restored or saved a Rust build cache. Move the cache
step to run after checkout.
Co-authored-by: Cursor <cursoragent@cursor.com>
* ci: remove dead rust-cache steps from docker-only jobs
These jobs only build Docker images via `docker buildx`/docker build
and never invoke cargo directly, so the `Swatinem/rust-cache` step
never populates or restores anything useful:
- integration-tests.yml: `test-consistency` (docker buildx + shell
consistency checks only)
- docker-image.yml: `build` and `build-gpu` (docker buildx only)
Drop the unused cache steps.
Co-authored-by: Cursor <cursoragent@cursor.com>
* ci: drop rust-cache from coverage and release workflows
These workflows run on rare triggers (coverage: nightly schedule;
release-artifacts: on release publish), so by the time they run again
their cache is almost always already evicted from the 10 GB Actions
cache budget. They mostly just consume cache space that would otherwise
be useful to the frequently-run PR builds, so remove the rust-cache
steps entirely:
- coverage.yml: unit-coverage and integration-coverage
- release-artifacts.yml: build-windows-binaries
Co-authored-by: Cursor <cursoragent@cursor.com>
* ci: drop explicit clippy cache key in rust-lint
The lint job has no compatible cache peer to share with: clippy
produces clippy-driver metadata that cargo fingerprints separately
from normal build/test artifacts, and its `--all-features` invocation
pulls in the GPU dependency crates, so it matches neither the
rust-tests nor the gpu group. With nothing to share, the explicit
`key: clippy` is equivalent to the default per-job key, so drop it for
consistency with the rest of the workflows.
Co-authored-by: Cursor <cursoragent@cursor.com>
* ci: dedicated rust-cache keys with single writer per group
Give related jobs explicit cache keys and let only the broadest build
in each group save the cache (`save-if: "false"` on the others), so a
narrower build can't overwrite it with fewer compiled dependencies
first.
- `rust-tests` (rust.yml): explicit `shared-key: rust-tests`. It builds
`--workspace --tests`, which pulls in dev-dependency features that the
integration-tests builds (plain `cargo build`) do not, so it is kept
as its own dependency cache rather than merged with integration-tests.
- `integration-tests`: writer = integration-tests; readers =
integration-tests-consensus, test-shard-snapshot-api-s3-minio.
- `edge`: writer = edge-test (clippy + examples build); reader =
edge-rust-check. Both build the amalgamated qdrant-edge `examples`
package.
rust-gpu keeps its own default cache (the `gpu` feature adds the
ash/gpu-allocator/shaderc dependency crates).
Co-authored-by: Cursor <cursoragent@cursor.com>
* Reuse `integration-tests` build cache for `io-bridge-object-store-tests` workflow
* Cleanup Docker cache in GHA workflows
* ci: only save rust-cache on the integration branch
Under the 10 GB Actions cache budget there's no practical benefit to
saving feature-branch build caches: they'd be evicted before being
reused. Scope every rust-cache *writer* to save only on `dev` (the
branch PRs target and restore from) via `save-if`:
- rust.yml, rust-lint.yml, edge-test.yml, integration-tests.yml
Reader jobs already have `save-if: "false"` and are unchanged.
Also drop rust-cache from rust-gpu.yml entirely: it only runs on
`master` pushes (releases, a few times a month at most), so any cache
would always be evicted between runs.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
* Potential fix for code scanning alert no. 7: Workflow does not contain permissions
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* Potential fix for code scanning alert no. 9: Workflow does not contain permissions
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* Potential fix for code scanning alert no. 10: Workflow does not contain permissions
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* Potential fix for code scanning alert no. 19: Workflow does not contain permissions
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* Potential fix for code scanning alert no. 20: Workflow does not contain permissions
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* Set permissions in GitHub workflow jobs
---------
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* ci: centralize protoc setup in a composite action
Add .github/actions/setup-protoc that pins both the arduino/setup-protoc
SHA and the protoc version (34.1) in one place, so future bumps no longer
require touching every workflow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* ci: replace arduino/setup-protoc with direct release-zip download
The arduino action queries the GitHub API on every invocation even when
the version is fully pinned, which is unnecessary work. Inline a small
shell step that downloads the protoc release zip directly using
RUNNER_OS/RUNNER_ARCH for asset selection. Supports Linux, macOS, and
Windows (x64 + arm64 where protobuf publishes assets).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* ci: parallelize consensus tests with pytest-xdist
Enable pytest-xdist for consensus_tests to run tests across multiple
workers in parallel, significantly reducing CI wall time (~20min → ~5-7min).
Changes:
- Add `-n auto --dist=loadfile` to the consensus test pytest invocation
- Remove hardcoded port_seed from tests that don't need fixed ports for
restart/rejoin (test_order_by, test_consensus_compaction,
test_named_vector_crud, test_listener_node)
- Give test_cluster_rejoin its own PORT_SEED=15000 to avoid port
conflicts with auth tests (PORT_SEED=10000)
- Derive restart ports from killed PeerProcess objects instead of
hardcoded arithmetic where possible
- Add xdist_group("auth") marker to auth test files to ensure they
run on the same worker (they share PORT_SEED=10000)
Made-with: Cursor
* fix: remove remaining hardcoded port_seed=20000 causing parallel test conflicts
8 test files were using port_seed=20000 as a positional argument to
start_cluster(), which was missed in the initial change. When running
in parallel with pytest-xdist, multiple workers would try to bind to
the same port range (20000-20x02), causing port conflicts and cascading
test failures.
Also remove port_seed=23000 from test_snapshot_recovery_kill.py since
it doesn't need fixed ports for restart.
Made-with: Cursor
* fix: use saved port for restart in test_two_follower_nodes_down
The test was restarting killed peers on hardcoded ports (20200/20100)
that previously matched port_seed=20000. After switching to random
ports, the restart ports no longer match the original peer ports,
causing raft state URI mismatches and peer startup failures.
Save the p2p_port from the killed PeerProcess and reuse it for restart.
Made-with: Cursor
* Reuse p2p ports when restarting killed peers in consensus tests
When a peer is killed and restarted with random ports, it gets a new
consensus URI. The cluster needs a Raft operation to update this URI,
which under CPU contention from parallel test workers can exceed the
30-second timeout. Fix by capturing each peer's p2p_port before killing
and reusing it on restart, so the URI stays the same and no consensus
update is needed.
Made-with: Cursor
* A few improvements for parallel runs (#8731)
* fix: make auth tests' PORT_SEED per-worker to avoid port collisions
* ci: improve failure visibility for parallel consensus tests
* Three small changes to make hangs, interleaved output, and coverage runs behave predictably under pytest-xdist
* fix: two test bugs surfaced by parallel runs and revert drop PR_SET_PDEATHSIG helper
* fix: wait for count convergence in test_triple_replication
* fix: clean leaked peer processes at test start
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
Co-authored-by: tellet-q <166374656+tellet-q@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Skip broken tests
* Add rocksdb dropper
This is a temporary tool. It will be removed later in this PR.
* [automated] Drop rocksdb
This commit is made by running tools/rocksdb/drop.sh
* Touch-up after ast-grep
The previous automated commit removed items, but not their comments.
Also, some blocks are left with only one item.
Also, ast-grep can't handle macros like `vec![]`.
This commit completes the job.
* Remove RocksDB dropper
* Remove mentions of rocksdb feature in CI
* Fix clippy warnings
These `FIXME` comments added previously in this PR by ast-grep by
"peeling" cfg_attr like this:
-#[cfg_attr(not(feature = "rocksdb"), expect(...))]
+#[expect(...)] // FIXME(rocksdb): ...
This commit removes these allow/expect attributes and fixes clippy
lints.
* Remove leftover rocksdb-related code
Removed:
- Cargo.toml: rocksdb cargo feature flag and dependencies.
- code: rocksdb-related items that was not under feature flag.
- flags.rs: rocksdb-related qdrant feature flags.
Disabled:
- Some benchmarks because these do not compile now.
* Print warning if rocksdb leftovers found in snapshots
* Remove Clone from tokenizer
* Regenerate openapi.json
* Move pyproject.toml to root
* Migrate pyproject.toml from Poetry to uv
* Update GH workflows
* Update test script, doc and nix to use uv
* Use latest uv
* Fix uv.lock
* Cleanup shell.nix
* Cleanup
- Explicit `uv sync` is not required, `uv run` will install deps
automatically.
- We don't provide a python package, so the `[build-system]` section
is not needed.
* Fix UV_VERSION inconsistency
---------
Co-authored-by: tellet-q <elena.dubrovina@qdrant.com>
Co-authored-by: xzfc <xzfcpw@gmail.com>
* Disable rocksdb compile time feature by default
* Also disable RocksDB feature in segment crate
* Enable RocksDB feature in all CI builds
* Remove extra job for testing non-RocksDB build, it's the default now
* Keep RocksDB structures in generated OpenAPI schema
* Fix obsolete --workspace flag breaking builds with explicit features
* Also build including RocksDB in e2e tests on CI
* Storage compat. tests all supported versions
* v1.15.0 creates problems
* less parallelism to not blow the disk
* Add xdist group (#7568)
* Add session fixture and xdist group
* Increase number of workers back to 4
* Revert caching
* Remove folders with docker_client
* Fix --dist=loadgroup
* Disable v1.15.0
---------
Co-authored-by: tellet-q <166374656+tellet-q@users.noreply.github.com>
* Bump poetry version to avoid potential compat. issues
* Poetry does not run bash script anymore
* fix path
* use new --project flag
* migrate also coverage for good measure
* Introduce coverage reports for integration tests
* Install cargo-llvm-cov
* Use multiline script
* Explicityl setup COVERAGE env var
* fix integration tests and log generated data
* fix ls path
* upload lcov file to GH artifacts
* integration profraw dynamic filename
* Fix llvm profile filename template
* Use interrupt instead of kill and merge consensus test results into same file
* Drop upload artifact stage
* install llvm-cov
* upload as artifact and export coverage files
* try simplifying workflow
* Migrate coverage generation to existing dedicated gh workflow
* trigger on coverage related branches
* Build only if qdrant binary with cov doesnt exist
* Use valid yaml
* include mode in profraw filename
* split coverage workflow into parallel jobs
* add poetry version to env
* log poetry version to install
* clean up integration test workflow
* Simplify comments
* Fix shard snapshot operations for s3
* tests/shard-snapshot-api.sh: Improve error reporting
* Drop SnapshotStorageManager::get_shard_snapshot_path
* Fix s3 test on CI
