Commit Graph

265 Commits

Author SHA1 Message Date
dependabot[bot]
cf0bf378ac build(deps): bump astral-sh/setup-uv from 8.3.0 to 8.3.2 (#9820)
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.3.0 to 8.3.2.
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](d31148d669...11f9893b08)

---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
  dependency-version: 8.3.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-13 16:46:34 -04:00
Andrey Vasnetsov
bb0e5d7544 Fix flaky disk exhaustion in edge-test CI job (#9807)
The publish/ directory is a separate Cargo workspace, so it does not
inherit the root workspace's [profile.dev] debug = "line-tables-only"
override. Example binaries were built with full debug info, each
statically linking qdrant-edge at ~700 MB per binary. Linking several
of them concurrently ran the runner out of disk, surfacing as
"mold: failed to write to an output file. Disk full?" + SIGBUS.

Mirror the debug info override in the publish workspace and free
~20 GB of unused preinstalled software on Linux runners as headroom.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 11:12:07 +02:00
Andrey Vasnetsov
0a1d0e0b0e ci: build single wheel on dev push, test all stable Pythons on release (#9751)
Building the full qdrant-edge wheel matrix (15 build jobs + 8 test jobs
across Linux/macOS/Windows, glibc/musl, CPython/PyPy) on every push to
dev is too expensive. Dev pushes now build only the most popular flavor:
x86_64 manylinux_2_17 CPython. Thanks to abi3-py310 that single wheel
covers every CPython >= 3.10, and it is tested on the latest stable
Python (3.14) on ubuntu-latest.

The full platform matrix still runs on manual workflow_dispatch
releases, and their test matrix is extended from {3.10, pypy3.11} to
all stable CPythons (3.10-3.14) plus PyPy on all four OS runners.

Since macOS/Windows builds are skipped on push, edge-py-test and
merge-artifacts switch to !cancelled()-based conditions so they are not
skip-propagated.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 13:46:49 +02:00
dependabot[bot]
e1f00cc8d3 build(deps): bump astral-sh/setup-uv from 8.2.0 to 8.3.0 (#9701)
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.2.0 to 8.3.0.
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](fac544c07d...d31148d669)

---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
  dependency-version: 8.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-06 15:42:35 -04:00
dependabot[bot]
439fe4ff02 build(deps): bump docker/setup-buildx-action from 4.1.0 to 4.2.0 (#9700)
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 4.1.0 to 4.2.0.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](d7f5e7f509...bb05f3f551)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-version: 4.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-06 15:41:35 -04:00
dependabot[bot]
4f4cf383d7 build(deps): bump docker/build-push-action from 7.2.0 to 7.3.0 (#9699)
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 7.2.0 to 7.3.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](f9f3042f7e...53b7df96c9)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-version: 7.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-06 15:41:01 -04:00
qdrant-cloud-bot
27b29b76e7 Add timeouts to shard snapshot S3 CI tests to fail fast on hangs. (#9693)
The recover-remote-concurrent step can hang indefinitely with S3 storage;
cap CI steps at 15 minutes and curl requests at 120 seconds so stuck runs
do not consume the full 6-hour job timeout.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-06 17:21:28 +02:00
Andrey Vasnetsov
fa03671ede ci: apply SemVer version bump on macOS and Windows too (#9649)
The previous fix only updated the Linux block because its old_string
matched a longer comment that the macOS/Windows blocks never had, so
those two jobs kept the original PEP 440 `.dev<N>` syntax without the
patch bump. Result: latest CI on `dev` (run 28509950366) still failed
with `unexpected character '.' after patch version number` on both
macOS and Windows, showing `version = "0.7.2.dev55"`.

Bring both blocks in line with Linux: SemVer `-dev<N>` suffix and the
patch-component bump. Same awk logic on all three, so maturin sees a
Cargo-valid version on every runner.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-07-01 12:37:19 +02:00
Andrey Vasnetsov
8525429fe8 ci: use SemVer syntax for dev version in Cargo.toml (#9648)
Cargo rejects PEP 440 `.dev<N>` syntax because it isn't valid SemVer, so
build the SemVer form `-dev<N>` instead. maturin translates the pre-release
identifier back to PEP 440 `.dev<N>` when constructing the wheel filename
and metadata, so the published version is unchanged from the intent.

Fixes the "unexpected character '.' after patch version number" error
seen on the first push of this branch.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-07-01 12:09:24 +02:00
Andrey Vasnetsov
1873b12534 ci: publish qdrant-edge-py dev wheels to GCS on merge to dev (#9647)
* ci: publish qdrant-edge-py pre-releases on merge to dev

Trigger the edge-py release workflow on push to `dev` and rewrite the
Cargo.toml package version to `X.Y.Z.dev<run_number>` before building.

PEP 440 treats the `.devN` suffix as a pre-release, so `pip install
qdrant-edge-py` continues to install the latest stable version while
users who want to try dev builds can pin explicitly:

    pip install qdrant-edge-py==0.7.2.dev137
    pip install --pre qdrant-edge-py

Manual `workflow_dispatch` runs are unchanged and still use the exact
Cargo.toml version.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: publish dev wheels to GCS instead of PyPI

On push to `dev`, upload the built wheels to gs://qdrant-debug/edge-py-dev/
(same bucket as the debug binaries, different sub-folder) and regenerate a
pip-compatible HTML index of every wheel currently in the bucket.

Users install a specific dev version with:

    pip install --find-links \
      https://storage.googleapis.com/qdrant-debug/edge-py-dev/qdrant-edge-py.html \
      qdrant-edge-py==0.7.2.dev137

The PyPI publish job is now gated back to `inputs.publish`, so manual
workflow_dispatch runs still target PyPI for stable releases.

Reuses the existing GCS_ACCESS_KEY_ID / GCS_SECRET_ACCESS_KEY secrets and
the AWS CLI + S3-compatible XML API pattern established by debug-tools.yml.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: keep only latest dev version in GCS bucket

After uploading the freshly built wheels, delete every wheel in the
bucket that doesn't belong to the current version. Order matters:
upload happens first so there's never a window with zero wheels
available in the index.

Current version is extracted from the wheel filename (PEP 427
guarantees version is the second `-`-separated field, and PEP 440
versions can't contain `-`).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: auto-bump patch component for dev wheel versions

Extend the version-patch step to bump the patch component of the SemVer
version in `Cargo.toml` before appending `.dev<run_number>`. For example
`0.7.2` in Cargo.toml becomes `0.7.3.dev137` in the built wheel.

This keeps every dev build strictly above the latest published stable in
PEP 440 ordering, so consumers can install "the current dev" without
knowing the run number:

    pip install --find-links <index-url> qdrant-edge-py --pre

Without the bump, `0.7.2.dev137` would sort below stable `0.7.2` and
`--pre` would silently prefer the stable release.

Manual `workflow_dispatch` runs are unchanged and still publish the
exact Cargo.toml version to PyPI.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-07-01 11:42:44 +02:00
dependabot[bot]
be43475295 build(deps): bump mozilla-actions/sccache-action (#9615)
Bumps [mozilla-actions/sccache-action](https://github.com/mozilla-actions/sccache-action) from 1583d6b38d7be47f593cb472781bbb21cab4321e to 9e7fa8a12102821edf02ca5dbea1acd0f89a2696.
- [Release notes](https://github.com/mozilla-actions/sccache-action/releases)
- [Commits](1583d6b38d...9e7fa8a121)

---
updated-dependencies:
- dependency-name: mozilla-actions/sccache-action
  dependency-version: 9e7fa8a12102821edf02ca5dbea1acd0f89a2696
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-30 14:13:41 +02:00
dependabot[bot]
9260223bb0 build(deps): bump actions/setup-python from 6.2.0 to 6.3.0 (#9614)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 6.2.0 to 6.3.0.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](a309ff8b42...ece7cb06ca)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-version: 6.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-29 21:41:37 +02:00
qdrant-cloud-bot
3a2b4cc702 ci: build edge debug tools with explicit --bin flags (#9603)
After `--package qdrant`, cargo scopes `--bin` to the qdrant crate only.
Pass `--package edge-shard-query --bin edge-shard-query` so the edge tool
is actually produced for collection/upload.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-28 13:18:06 +02:00
qdrant-cloud-bot
12bc1cdc54 ci: pass --package qdrant when building debug tools (#9602)
Without an explicit qdrant package, cargo applies --features service_debug
to every -p target including edge-shard-query, which does not define it.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-28 12:51:23 +02:00
qdrant-cloud-bot
dec75ab694 ci: scope service_debug feature to qdrant package only (#9601)
edge-shard-query does not define service_debug; use qdrant/service_debug
when building qdrant bins and edge tools in one cargo invocation.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-28 11:12:36 +02:00
qdrant-cloud-bot
c529f5b23d ci: speed up debug-tools workflow build (#9599)
Use the perf profile instead of release, merge qdrant and edge tool builds
into one cargo invocation, and add sccache so shared deps compile once and
rustc outputs are cached across runs.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-28 08:21:26 +02:00
Andrey Vasnetsov
33771fb6c1 Generalize edge S3 scroll tool into edge-shard-query (#9598)
Turn the single-purpose `edge-s3-scroll` tool into a sub-command-based
`edge-shard-query` that can run different read requests against a
ReadOnlyEdgeShard opened over object storage.

- Add `scroll` and `search` sub-commands; shared connection/cache args
  live at the top level.
- Accept arbitrary payload filters as JSON via `--filter` (curl
  `--data` style: literal JSON, `@file`, or `@-` for stdin), parsed
  straight into the filter DSL. Keep `--filter-key`/`--filter-value`
  as a shortcut.
- `search` accepts a query vector (JSON array, comma list, or
  `@file`/`@-`), named vector, offset, score threshold, and HNSW
  params.
- Rename the package/binary/dir (`s3_scroll` -> `shard_query`) since
  the tool is no longer scroll-only nor S3-only.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 23:58:20 +02:00
Andrey Vasnetsov
734b2a3f41 ci: build edge-s3-scroll among the debug tools (#9572)
Add the edge-s3-scroll tool (lib/edge/tools/s3_scroll) to the debug
tools build/upload. It's a standalone workspace crate without the
qdrant `service_debug` feature, so it's built in a separate `cargo build
-p edge-s3-scroll` invocation and then collected/uploaded alongside the
rest.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 15:19:05 +02:00
Arnaud Gourlay
25fded969a ci: add nightly model_testing workflow (#9563)
* ci: add nightly model_testing workflow

Run the model_testing binary nightly on dev with two sequential passes
(async-scorer and without io_uring), reusing a single built binary.

Note: a temporary pull_request trigger is included to test-drive the
workflow on the PR; it is to be removed before merge.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci: fix checkout ref for pull_request event

github.ref_name resolves to <pr>/merge on pull_request, which checkout
treats as a branch and fails to fetch. Use the default ref outside the
nightly schedule.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci: drop mold/clang linker from model_testing workflow

free-disk-space (large-packages) removes the llvm/clang packages, so the
clang linker used for mold no longer exists on the runner. Build time is
dominated by the long model-testing runs, so the default linker is fine.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci: random seed, constants, both-run, backtrace, dispatch inputs, failure issue

- Random seed per run (logged for repro), shared by both passes
- OP_NUM/SHARD_COUNT/RESTART_PROBABILITY/ID_POOL as env constants
- no-io_uring run uses if: !cancelled() so both passes always run
- RUST_BACKTRACE=1 for debuggable panics
- workflow_dispatch inputs to override op_num/seed
- rust-cache saves on the nightly (schedule) run, not the never-matching dev ref
- open/update a GitHub issue on scheduled failures

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci: run model_testing with 2 shards by default

Exercises the cross-shard reload/WAL-replay path (the 2-shard-only revert
class) instead of single-shard only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci: bump model_testing OP_NUM to 200000

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci: remove temporary pull_request trigger from model_testing workflow

Test-drive validated on the PR; the nightly now runs only on schedule and
workflow_dispatch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 13:39:59 +02:00
Andrey Vasnetsov
e706f772ee ci: build debug tools as static musl binaries (#9569)
Build the debug tools for x86_64-unknown-linux-musl so the resulting
binaries are statically linked and run on any Linux host regardless of
distro / libc. Mirrors the musl setup used in release-artifacts.yml
(apt deps + taiki-e/setup-cross-toolchain-action) and drops the
gnu-specific mold linker hack, which the cross toolchain replaces.

Cache is keyed per target so the musl artifacts persist across runs
without colliding with gnu caches.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 13:14:50 +02:00
Andrey Vasnetsov
6f9225a7c9 ci: fix GCS multipart upload (disable default AWS CLI checksums) (#9568)
Recent AWS CLI versions add default CRC32 integrity checksums on
uploads, which GCS's S3-compatible UploadPart rejects with
SignatureDoesNotMatch. This only affected the larger debug binaries that
go through multipart upload. Opt out via AWS_REQUEST_CHECKSUM_CALCULATION
/ AWS_RESPONSE_CHECKSUM_VALIDATION=when_required.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 13:07:01 +02:00
Andrey Vasnetsov
453b2f632f ci: also build debug tools on push to dev (#9567)
Re-add the push-to-dev trigger so debug tools are rebuilt automatically
on every merge to dev, while keeping manual dispatch with a branch
input. The branch used for checkout and the object prefix falls back to
the pushed branch when no input is given.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 10:32:18 +02:00
Andrey Vasnetsov
41372d0848 ci: build debug tools and upload to GCS (#9565)
* ci: build debug tools and publish to GHCR via ORAS

Adds a `Build debug tools` workflow that compiles the `service_debug`
helper binaries (wal_inspector, wal_pop, segment_inspector,
schema_generator, model_testing) and pushes them to GHCR as an OCI
artifact using ORAS.

These are debug-only tools built between releases and are intentionally
kept out of the release artifacts. Publishing them as an OCI artifact
gives a versioned, browsable, registry-backed location alongside our
Docker images instead of burying them in per-run CI artifacts.

Pull with:
  oras pull ghcr.io/qdrant/qdrant/debug-tools:dev

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci: upload debug tools to S3 instead of GHCR

Switch the debug-tools workflow from publishing an OCI artifact via ORAS
to uploading the `service_debug` binaries to S3 with public-read, giving
plain HTTPS download links and no client tooling beyond curl.

Binaries are uploaded under a moving `<branch>/` prefix (latest) and an
immutable `<branch>-<sha>/` prefix (history); direct links are written
to the job summary.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci: target GCS (S3-compatible) instead of AWS S3

Point the debug-tools upload at GCS via its S3-compatible XML API using
AWS-style HMAC interoperability keys. Drops per-object ACLs (GCS buckets
use uniform access; public read is granted via IAM) and uses GCS
path-style public URLs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci: set debug-tools bucket to qdrant-debug

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci: run debug-tools manually with a branch input

Drop the push trigger so the workflow only runs on explicit dispatch
from the UI, and add a `branch` input selecting which branch to build.
The chosen branch (and its actual HEAD commit) form the object prefix.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 10:07:33 +02:00
dependabot[bot]
c68b89297e build(deps): bump actions/checkout from 6.0.3 to 7.0.0 (#9538)
Bumps [actions/checkout](https://github.com/actions/checkout) from 6.0.3 to 7.0.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](df4cb1c069...9c091bb21b)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 7.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-23 09:53:04 +02:00
qdrant-cloud-bot
9efc061764 ci: fix and consolidate Rust build caches (#9516)
* ci: fix rust-cache running before checkout in shard-snapshot job

The `test-shard-snapshot-api-s3-minio` job ran `Swatinem/rust-cache`
before `actions/checkout`, so there was no `Cargo.lock`/source present
when the cache action ran. As a result the action no-opped and the job
never actually restored or saved a Rust build cache. Move the cache
step to run after checkout.

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci: remove dead rust-cache steps from docker-only jobs

These jobs only build Docker images via `docker buildx`/docker build
and never invoke cargo directly, so the `Swatinem/rust-cache` step
never populates or restores anything useful:

- integration-tests.yml: `test-consistency` (docker buildx + shell
  consistency checks only)
- docker-image.yml: `build` and `build-gpu` (docker buildx only)

Drop the unused cache steps.

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci: drop rust-cache from coverage and release workflows

These workflows run on rare triggers (coverage: nightly schedule;
release-artifacts: on release publish), so by the time they run again
their cache is almost always already evicted from the 10 GB Actions
cache budget. They mostly just consume cache space that would otherwise
be useful to the frequently-run PR builds, so remove the rust-cache
steps entirely:

- coverage.yml: unit-coverage and integration-coverage
- release-artifacts.yml: build-windows-binaries

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci: drop explicit clippy cache key in rust-lint

The lint job has no compatible cache peer to share with: clippy
produces clippy-driver metadata that cargo fingerprints separately
from normal build/test artifacts, and its `--all-features` invocation
pulls in the GPU dependency crates, so it matches neither the
rust-tests nor the gpu group. With nothing to share, the explicit
`key: clippy` is equivalent to the default per-job key, so drop it for
consistency with the rest of the workflows.

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci: dedicated rust-cache keys with single writer per group

Give related jobs explicit cache keys and let only the broadest build
in each group save the cache (`save-if: "false"` on the others), so a
narrower build can't overwrite it with fewer compiled dependencies
first.

- `rust-tests` (rust.yml): explicit `shared-key: rust-tests`. It builds
  `--workspace --tests`, which pulls in dev-dependency features that the
  integration-tests builds (plain `cargo build`) do not, so it is kept
  as its own dependency cache rather than merged with integration-tests.
- `integration-tests`: writer = integration-tests; readers =
  integration-tests-consensus, test-shard-snapshot-api-s3-minio.
- `edge`: writer = edge-test (clippy + examples build); reader =
  edge-rust-check. Both build the amalgamated qdrant-edge `examples`
  package.

rust-gpu keeps its own default cache (the `gpu` feature adds the
ash/gpu-allocator/shaderc dependency crates).

Co-authored-by: Cursor <cursoragent@cursor.com>

* Reuse `integration-tests` build cache for `io-bridge-object-store-tests` workflow

* Cleanup Docker cache in GHA workflows

* ci: only save rust-cache on the integration branch

Under the 10 GB Actions cache budget there's no practical benefit to
saving feature-branch build caches: they'd be evicted before being
reused. Scope every rust-cache *writer* to save only on `dev` (the
branch PRs target and restore from) via `save-if`:

- rust.yml, rust-lint.yml, edge-test.yml, integration-tests.yml

Reader jobs already have `save-if: "false"` and are unchanged.

Also drop rust-cache from rust-gpu.yml entirely: it only runs on
`master` pushes (releases, a few times a month at most), so any cache
would always be evicted between runs.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
2026-06-20 13:44:45 +02:00
Tim Visée
9acece1e2c Set GitHub workflow permissions explicitly (#9432)
* Potential fix for code scanning alert no. 7: Workflow does not contain permissions

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for code scanning alert no. 9: Workflow does not contain permissions

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for code scanning alert no. 10: Workflow does not contain permissions

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for code scanning alert no. 19: Workflow does not contain permissions

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for code scanning alert no. 20: Workflow does not contain permissions

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Set permissions in GitHub workflow jobs

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-06-11 16:02:12 +02:00
dependabot[bot]
a6b580b245 build(deps): bump codecov/codecov-action from 6.0.1 to 7.0.0 (#9382)
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 6.0.1 to 7.0.0.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](e79a6962e0...fb8b3582c8)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-version: 7.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-09 11:24:08 +02:00
dependabot[bot]
4779818d4b build(deps): bump astral-sh/setup-uv from 8.1.0 to 8.2.0 (#9383)
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.1.0 to 8.2.0.
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](08807647e7...fac544c07d)

---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
  dependency-version: 8.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-09 11:23:35 +02:00
dependabot[bot]
f816cff01b build(deps): bump actions/checkout from 6.0.2 to 6.0.3 (#9273)
Bumps [actions/checkout](https://github.com/actions/checkout) from 6.0.2 to 6.0.3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](de0fac2e45...df4cb1c069)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 6.0.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-03 10:59:40 +02:00
tellet-q
2b667dfa4c ci: bump test_many_collections timeout to avoid flaky CI timeouts (#9253) 2026-06-01 15:54:07 +02:00
tellet-q
d559671c68 Revert setting vm.max_map_count to higher value and increase pytest timeout for e2e tests (#9191) 2026-05-27 11:15:06 +02:00
Daniel Boros
5e109bbecc feat: sync->async bridge (#9093)
* feat: add io_bridge

* fix: cached dispatcher

* feat: add open_with_handle

* fix: naming

* fix: wording

* fix: rebase io_bridge onto split BorrowedReadPipeline/OwnedReadPipeline

* fix: linter

* feat: add S3 backend

* chore: remove io_bridge

* fix: linter

* fix: linter

* fix: read handle

* chore: simplify S3Source

* fix: s3 test

* feat: support multi runtime

* fix: clippy errors

* fix: review comments

* feat: add io design

* feat: add S3 backend

* chore: fix docs

* fix: dev changes

* chore: add some docs

* chore: remove explicit type

* feat: add new methods

* [WIP] review refactor

* fmt

* fix: bytes alignment

* fix: linter

* feat: remove Bytes

* fix: ci/cd

* fix: tests

* fix: tests

* refactor: simplify io_bridge pipeline to Handle-based dispatch

Replace the BridgeRuntime worker thread + request channel + boxed
BridgeRequest with direct tokio Handle usage:

- BridgeRuntime is now just an Arc<Runtime>; schedule() spawns the read
  future via Handle::spawn instead of routing it through a dispatcher
  thread. Removes BridgeRequest and the now-unreachable S3RuntimeShutDown
  error variant.
- Guard against a panicking read task hanging wait() forever: the spawned
  task catches unwinds and converts them into a TaskPanicked error reply,
  so every scheduled slot is always answered.
- Encapsulate slot bookkeeping in PendingSlots, exposing only the needed
  operations instead of a public map + counter.
- Split the grown pipeline.rs into a pipeline/ module (slots / inner /
  borrowed / owned), de-duplicating the shared read-future construction.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor: move pipeline buffer ownership into the read future

Instead of the pipeline owning the destination Vec<T> in a slot map and
the future writing through a SendBytePtr raw pointer, let the future
allocate the buffer itself and return it through BridgeResponse. The
buffer crosses the worker-thread boundary as a normal move via the reply
channel, wrapped in a SendableVec<T> newtype that asserts Send for
T: bytemuck::Pod only.

This removes the entire unsafe SendBytePtr apparatus from the pipeline:
no raw pointer, no unsafe fn, no per-call-site unsafe blocks, no
heap-stability invariants. The only remaining unsafe in the crate is one
bounded `unsafe impl<T: Pod> Send for SendableVec<T>` with a trivially
true invariant (Pod types are plain bytes).

PendingSlots collapses back to PendingSlots<U>: slots no longer carry
buffers. AlignedBufWriter::from_raw_bytes (used only by the SendBytePtr
path) and its test are removed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor: drop SendableVec wrapper now that Item: Send

UniversalRead's element type is now bound to `Item` (`Pod + Send`),
so the io_bridge pipeline no longer needs a hand-rolled `Send` wrapper
around `Vec<T>` to ship buffers through the reply channel. Replace
`SendableVec<T>` with `Vec<T>` end-to-end and tighten the local impls
to `T: Item`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: implement UniversalRead::reopen for BlobFile

BlobFile has no cached file metadata or mapping — `len()` queries the
object store fresh on each call — so reopen is a no-op, matching the
io_uring impl.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: add new fs impl for Blob

* fix: is_in_ram_or_mmap for S3

---------

Co-authored-by: generall <andrey@vasnetsov.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 17:28:49 +02:00
dependabot[bot]
6ec3fe65dd build(deps): bump taiki-e/setup-cross-toolchain-action (#9164)
Bumps [taiki-e/setup-cross-toolchain-action](https://github.com/taiki-e/setup-cross-toolchain-action) from 1.40.1 to 1.41.0.
- [Release notes](https://github.com/taiki-e/setup-cross-toolchain-action/releases)
- [Changelog](https://github.com/taiki-e/setup-cross-toolchain-action/blob/main/CHANGELOG.md)
- [Commits](129361238c...3d9770ce98)

---
updated-dependencies:
- dependency-name: taiki-e/setup-cross-toolchain-action
  dependency-version: 1.41.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-26 10:55:06 +02:00
dependabot[bot]
214fff64dc build(deps): bump docker/build-push-action from 7.1.0 to 7.2.0 (#9163)
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 7.1.0 to 7.2.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](bcafcacb16...f9f3042f7e)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-version: 7.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-26 10:54:31 +02:00
dependabot[bot]
12d5a98cde build(deps): bump docker/setup-buildx-action from 4.0.0 to 4.1.0 (#9162)
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 4.0.0 to 4.1.0.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](4d04d5d948...d7f5e7f509)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-version: 4.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-26 10:53:49 +02:00
tellet-q
76bb521e7e Run Long Integration Tests against dev on schedule (#9100) 2026-05-20 09:59:38 +02:00
tellet-q
b57a01814f Fix long running e2e test (#9090)
* store container logs

* increase vm.max_map_count
2026-05-19 15:01:15 +02:00
dependabot[bot]
973e782ffd build(deps): bump codecov/codecov-action from 6.0.0 to 6.0.1 (#9081)
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 6.0.0 to 6.0.1.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](57e3a136b7...e79a6962e0)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-version: 6.0.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-19 06:44:20 +02:00
dependabot[bot]
d983f5544c build(deps): bump sigstore/cosign-installer from 4.1.1 to 4.1.2 (#9001)
Bumps [sigstore/cosign-installer](https://github.com/sigstore/cosign-installer) from 4.1.1 to 4.1.2.
- [Release notes](https://github.com/sigstore/cosign-installer/releases)
- [Commits](cad07c2e89...6f9f177880)

---
updated-dependencies:
- dependency-name: sigstore/cosign-installer
  dependency-version: 4.1.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-12 08:20:30 +02:00
dependabot[bot]
5ee548a1a9 build(deps): bump taiki-e/setup-cross-toolchain-action (#8898)
Bumps [taiki-e/setup-cross-toolchain-action](https://github.com/taiki-e/setup-cross-toolchain-action) from 1.39.2 to 1.40.1.
- [Release notes](https://github.com/taiki-e/setup-cross-toolchain-action/releases)
- [Changelog](https://github.com/taiki-e/setup-cross-toolchain-action/blob/main/CHANGELOG.md)
- [Commits](d62f33b587...129361238c)

---
updated-dependencies:
- dependency-name: taiki-e/setup-cross-toolchain-action
  dependency-version: 1.40.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-05 07:47:28 +02:00
dependabot[bot]
d1b0013c56 build(deps): bump rui314/setup-mold (#8809)
Bumps [rui314/setup-mold](https://github.com/rui314/setup-mold) from 725a8794d15fc7563f59595bd9556495c0564878 to 9c9c13bf4c3f1adef0cc596abc155580bcb04444.
- [Commits](725a8794d1...9c9c13bf4c)

---
updated-dependencies:
- dependency-name: rui314/setup-mold
  dependency-version: 9c9c13bf4c3f1adef0cc596abc155580bcb04444
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-28 07:48:11 +02:00
Andrey Vasnetsov
f2dca149ca ci: centralize just setup in a composite action (#8807)
* ci: pin just version to 1.50.0 in edge workflows

Without an explicit just-version, the setup-just action resolves "latest"
on every run, making CI implicitly depend on whatever was released most
recently. Pin to 1.50.0 so just upgrades are an intentional change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: centralize just setup in a composite action

Add .github/actions/setup-just wrapping extractions/setup-just@v4.0.0 with
just-version pinned to 1.50.0, so the version is declared once instead of
duplicated across every edge workflow.

Also replaces the xzfc/setup-just fork in edge-rust-release: the reason
for the fork (missing just-version pinning support in extractions/
setup-just) was resolved in v4.0.0, so the workaround is no longer needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 00:33:35 +02:00
Andrey Vasnetsov
070ca541fd ci: centralize protoc setup in a composite action (#8805)
* ci: centralize protoc setup in a composite action

Add .github/actions/setup-protoc that pins both the arduino/setup-protoc
SHA and the protoc version (34.1) in one place, so future bumps no longer
require touching every workflow.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: replace arduino/setup-protoc with direct release-zip download

The arduino action queries the GitHub API on every invocation even when
the version is fully pinned, which is unnecessary work. Inline a small
shell step that downloads the protoc release zip directly using
RUNNER_OS/RUNNER_ARCH for asset selection. Supports Linux, macOS, and
Windows (x64 + arm64 where protobuf publishes assets).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 23:11:40 +02:00
tellet-q
461e6f63e9 ci: improve docker build caches for e2e and consensus jobs (#8786) 2026-04-27 12:12:06 +02:00
qdrant-cloud-bot
f689dcc8cc ci: parallelize consensus tests with pytest-xdist (#8717)
* ci: parallelize consensus tests with pytest-xdist

Enable pytest-xdist for consensus_tests to run tests across multiple
workers in parallel, significantly reducing CI wall time (~20min → ~5-7min).

Changes:
- Add `-n auto --dist=loadfile` to the consensus test pytest invocation
- Remove hardcoded port_seed from tests that don't need fixed ports for
  restart/rejoin (test_order_by, test_consensus_compaction,
  test_named_vector_crud, test_listener_node)
- Give test_cluster_rejoin its own PORT_SEED=15000 to avoid port
  conflicts with auth tests (PORT_SEED=10000)
- Derive restart ports from killed PeerProcess objects instead of
  hardcoded arithmetic where possible
- Add xdist_group("auth") marker to auth test files to ensure they
  run on the same worker (they share PORT_SEED=10000)

Made-with: Cursor

* fix: remove remaining hardcoded port_seed=20000 causing parallel test conflicts

8 test files were using port_seed=20000 as a positional argument to
start_cluster(), which was missed in the initial change. When running
in parallel with pytest-xdist, multiple workers would try to bind to
the same port range (20000-20x02), causing port conflicts and cascading
test failures.

Also remove port_seed=23000 from test_snapshot_recovery_kill.py since
it doesn't need fixed ports for restart.

Made-with: Cursor

* fix: use saved port for restart in test_two_follower_nodes_down

The test was restarting killed peers on hardcoded ports (20200/20100)
that previously matched port_seed=20000. After switching to random
ports, the restart ports no longer match the original peer ports,
causing raft state URI mismatches and peer startup failures.

Save the p2p_port from the killed PeerProcess and reuse it for restart.

Made-with: Cursor

* Reuse p2p ports when restarting killed peers in consensus tests

When a peer is killed and restarted with random ports, it gets a new
consensus URI. The cluster needs a Raft operation to update this URI,
which under CPU contention from parallel test workers can exceed the
30-second timeout. Fix by capturing each peer's p2p_port before killing
and reusing it on restart, so the URI stays the same and no consensus
update is needed.

Made-with: Cursor

* A few improvements for parallel runs (#8731)

* fix: make auth tests' PORT_SEED per-worker to avoid port collisions
* ci: improve failure visibility for parallel consensus tests
* Three small changes to make hangs, interleaved output, and coverage runs behave predictably under pytest-xdist
* fix: two test bugs surfaced by parallel runs and revert drop PR_SET_PDEATHSIG helper
* fix: wait for count convergence in test_triple_replication
* fix: clean leaked peer processes at test start

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Cursor Agent <agent@cursor.com>
Co-authored-by: tellet-q <166374656+tellet-q@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 08:07:27 +02:00
dependabot[bot]
3ea99599ae build(deps): bump taiki-e/upload-rust-binary-action (#8738)
Bumps [taiki-e/upload-rust-binary-action](https://github.com/taiki-e/upload-rust-binary-action) from 1.30.0 to 1.30.2.
- [Release notes](https://github.com/taiki-e/upload-rust-binary-action/releases)
- [Changelog](https://github.com/taiki-e/upload-rust-binary-action/blob/main/CHANGELOG.md)
- [Commits](10c1cf6a3d...f0d45ae91e)

---
updated-dependencies:
- dependency-name: taiki-e/upload-rust-binary-action
  dependency-version: 1.30.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-21 08:19:17 +02:00
dependabot[bot]
6f08bffcda build(deps): bump astral-sh/setup-uv from 8.0.0 to 8.1.0 (#8739)
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.0.0 to 8.1.0.
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](cec208311d...08807647e7)

---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
  dependency-version: 8.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-20 22:05:59 -04:00
Arnaud Gourlay
40d6b429a2 Revert "Disable CI for edge examples due to yanked core2" (#8684)
* Revert "Disable CI for edge examples due to yanked core2"

This reverts commit dd7a2bd4c5.

* cargo update --package include-flate
2026-04-15 15:19:11 +02:00
Arnaud Gourlay
f6ea7b0b36 Disable CI for edge examples due to yanked core2 (#8682) 2026-04-15 11:07:48 +02:00
dependabot[bot]
a5e0464267 build(deps): bump docker/build-push-action from 7.0.0 to 7.1.0 (#8660)
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 7.0.0 to 7.1.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](d08e5c354a...bcafcacb16)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-version: 7.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-14 09:01:17 +02:00