mirror of
https://github.com/qdrant/qdrant.git
synced 2026-09-21 13:37:46 -05:00
Andrey VasnetsovandClaude Opus 5
e97bd7d1e5
* fix(uio): don't fetch a zero-length object on a populating async open
The `PreferBackground`/`Blocking` arm of the disk cache's `open_async` built
its prefetch range by hand as `0..len`. For a zero-length object that is
`0..0`, which `object_store` rejects client-side with
`InvalidGetRange::Inconsistent` ("Range started at 0 and ended at 0") rather
than answering with an empty body.
The failure is not contained to the file: a single empty object anywhere
under a segment prefix fails the whole segment open, and a read-only
follower then logs "skipping unloadable segment" and serves the shard
without it — silently returning results computed over a subset of the data.
It is also deterministic, so the segment stays dropped on every retry.
The sync path already handles this correctly via `schedule_whole`
(`read_from_into_byte_buffer` disambiguates the unsatisfiable-range error
with a `len` call and yields an empty buffer); the async arm was the only
place assembling the range itself. Create the local mirror first and skip
the fetch entirely when there is nothing to read.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* ci: stop using the minio/mc image, which no longer exists
`minio/mc` has been withdrawn from Docker Hub — pulling it now fails with
"repository does not exist or may require 'docker login'". The readiness
loop ran it 60 times with output redirected to /dev/null, so the pull
error was invisible and the job failed as "rustfs did not become ready in
60s", pointing at the wrong component. Bucket creation used the same
image and would have failed next.
Use the AWS CLI that ships with the runner image instead: no third-party
container to pull for either step. The readiness probe stays an
authenticated call (`s3api list-buckets`), so it still waits out
credential setup and not just the port opening, and it now reports the
final failure instead of swallowing it.
Also pin the rustfs service image by digest. That was not the cause here
— rc.5 and rc.6 both work — but this workflow already pins its actions by
SHA, and a service tracking `latest` is how a green suite turns red with
no change to the repo.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
83 lines
2.8 KiB
YAML
83 lines
2.8 KiB
YAML
name: io_bridge_object_store integration
|
|
|
|
on:
|
|
push:
|
|
branches: [master, dev]
|
|
paths:
|
|
- "lib/common/io_bridge_object_store/**"
|
|
- "lib/common/common/src/universal_io/**"
|
|
- ".github/workflows/io-bridge-object-store-tests.yml"
|
|
pull_request:
|
|
paths:
|
|
- "lib/common/io_bridge_object_store/**"
|
|
- "lib/common/common/src/universal_io/**"
|
|
- ".github/workflows/io-bridge-object-store-tests.yml"
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
env:
|
|
CARGO_TERM_COLOR: always
|
|
|
|
jobs:
|
|
integration:
|
|
name: rustfs-backed integration
|
|
runs-on: ubuntu-latest
|
|
env:
|
|
AWS_ACCESS_KEY_ID: ci-test-user
|
|
AWS_SECRET_ACCESS_KEY: ci-test-secret-not-for-prod
|
|
AWS_DEFAULT_REGION: us-east-1
|
|
services:
|
|
rustfs:
|
|
# Pinned by digest so a moving upstream tag cannot change what CI tests against.
|
|
image: rustfs/rustfs:v1.0.0-rc.5@sha256:b7014e0ce2bc703c1316b3ef760e29dfae61fe4a50d1a66fa89638e0f8ea211f
|
|
ports:
|
|
- 9000:9000
|
|
env:
|
|
RUSTFS_ACCESS_KEY: ci-test-user
|
|
RUSTFS_SECRET_KEY: ci-test-secret-not-for-prod
|
|
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
|
|
|
|
# The AWS CLI ships with the runner image, so readiness and bucket setup pull
|
|
# no third-party container. An authenticated call, so this also waits out
|
|
# credential setup rather than just the port opening.
|
|
- name: Wait for rustfs to be ready
|
|
run: |
|
|
for _ in $(seq 1 60); do
|
|
if aws --endpoint-url http://localhost:9000 s3api list-buckets >/dev/null 2>&1; then
|
|
echo "rustfs is up"
|
|
exit 0
|
|
fi
|
|
sleep 1
|
|
done
|
|
echo "rustfs did not become ready in 60s; last attempt was:" >&2
|
|
aws --endpoint-url http://localhost:9000 s3api list-buckets
|
|
exit 1
|
|
|
|
- name: Create test bucket
|
|
run: aws --endpoint-url http://localhost:9000 s3api create-bucket --bucket test-bucket
|
|
|
|
- name: Install Rust toolchain
|
|
uses: dtolnay/rust-toolchain@631a55b12751854ce901bb631d5902ceb48146f7 # stable
|
|
|
|
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6
|
|
with:
|
|
# Dedicated cache for this job would mostly sit idle and get evicted between runs.
|
|
# Instead, we reuse the integration-tests cache, which overlaps enough to speed this build up.
|
|
shared-key: integration-tests
|
|
save-if: "false"
|
|
|
|
- name: Install Protoc
|
|
uses: ./.github/actions/setup-protoc
|
|
|
|
- name: Run integration tests
|
|
env:
|
|
S3_INTEGRATION_TEST: "1"
|
|
RUSTFS_ENDPOINT: "http://localhost:9000"
|
|
RUSTFS_ACCESS_KEY: "ci-test-user"
|
|
RUSTFS_SECRET_KEY: "ci-test-secret-not-for-prod"
|
|
RUSTFS_BUCKET: "test-bucket"
|
|
run: cargo test -p io_bridge_object_store -- --ignored
|