upstream-archive/qdrant

Fork 0

mirror of https://github.com/qdrant/qdrant.git synced 2026-09-25 07:27:41 -05:00

Files

T

History

c218fa67c0 Configurable key prefix for object storage snapshots (#10715 )

* Configurable key prefix for object storage snapshots

Snapshot object keys were always derived from the local snapshots path,
so every deployment sharing a bucket wrote under the same `snapshots/`
root. Each cloud config block gains an optional `prefix`, and objects
become `<prefix>/snapshots/...`.

The prefix is applied by wrapping the client in `PrefixStore`, so the
snapshot operations and the names returned by the API are unchanged.
Leading, trailing and repeated slashes are dropped, and an empty prefix
is a no-op.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* Test that snapshot operations cannot escape the storage prefix

Hostile targets are handed to the cloud manager directly, past
`validate_snapshot_name`: parent references, absolute paths, encoded
slashes, backslashes and empty paths. Writes must stay under the prefix,
and objects planted outside the prefix must be invisible to list,
download, stream and delete.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* Normalize empty prefix components

Refactor prefix handling to remove empty components.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Tim Visée <tim+github@visee.me>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

2026-09-22 12:48:26 +02:00

test_data

test: cover TurboQuant, turbo4 datatype and keyword prefix in compat data (#10165 )

2026-08-11 11:21:57 +02:00

tls_cluster

Ignore cacert for internal requests if not configured (#8099 )

2026-02-12 10:59:38 +01:00

__init__.py

Refactor bash tests: low-disk (#6845 )

2025-07-14 12:42:06 +02:00

client_utils.py

[ai] test: wait for cluster readiness in flaky e2e test (#9012 )

2026-05-12 11:36:11 +02:00

conftest.py

tests: use local cache when building image for e2e tests (#8822 )

2026-09-18 11:52:27 +02:00

models.py

Remove trailing whitespaces (#7435 )

2025-10-21 14:05:57 +02:00

README.md

tests: use local cache when building image for e2e tests (#8822 )

2026-09-18 11:52:27 +02:00

test_bugs.py

Bump client timeout in flaky snapshots-consensus e2e test (#9808 )

2026-07-12 21:14:04 +02:00

test_continuous_snapshots.py

Remove trailing whitespaces (#7435 )

2025-10-21 14:05:57 +02:00

test_data_compatibility.py

test: cover TurboQuant, turbo4 datatype and keyword prefix in compat data (#10165 )

2026-08-11 11:21:57 +02:00

test_immutable_index.py

Remove trailing whitespaces (#7435 )

2025-10-21 14:05:57 +02:00

test_low_disk.py

Remove trailing whitespaces (#7435 )

2025-10-21 14:05:57 +02:00

test_low_ram.py

Remove RocksDB references (#10561 )

2026-09-09 17:40:46 +02:00

test_many_collections.py

ci: bump test_many_collections timeout to avoid flaky CI timeouts (#9253 )

2026-06-01 15:54:07 +02:00

test_quota_disk.py

Global quota API (#10035 )

2026-08-03 11:20:34 +02:00

test_snapshots_recovery.py

Configurable key prefix for object storage snapshots (#10715 )

2026-09-22 12:48:26 +02:00

test_strict_mode_memory.py

Enable the single_file_mmap_vector_storage flag by default (#9332 )

2026-06-18 16:24:03 +02:00

test_tls.py

fix: reduce flakiness of gRPC TLS e2e test (#9086 )

2026-05-19 09:55:30 +02:00

utils.py

[ai] use readiness probe instead of a time.sleep (#8884 )

2026-05-04 11:34:36 +02:00

README.md

TLS test

Regenerate certificates

run PROJECT_ROOT/tests/e2e_tests/test_data/gen.sh
replace old certificates in PROJECT_ROOT/tests/e2e_tests/test_data/cert with the newly generated ones

Data compatibility test

In order to detect quickly breakage of storage compatibility, we check that the current code understands the storage format from the latest stable release.

To not burden the git repository with tracking the large binary files, we are pushing storage archives to our GCP bucket.

As features are added, the storage format may change. This means updating the reference storage data and snapshot. This is done by running the gen_storage_compat_data.sh script.

Regenerate storage data

Follow those steps to recreate the reference storage data and snapshot.

run PROJECT_ROOT/tests/e2e_tests/test_data/compatibility/gen_storage_compat_data.sh
make sure to pick the right version when asked for which system generated the files
push the new archives to the GCP bucket (ask for the credentials if you don't have them)

Local buildx cache

When no pre-built qdrant/qdrant:e2e-tests exists locally, the qdrant_image fixture builds one via a dedicated qdrant-e2e-builder (docker-container driver) with a persistent cache at ~/.cache/qdrant-e2e-buildx.

First run is cold; subsequent runs reuse the cargo chef cook layer while Cargo.lock is unchanged.
Cache is shared across checkouts and worktrees.
The builder container is kept between runs. Remove it with docker buildx rm qdrant-e2e-builder.
The cache directory is never pruned and grows with every Cargo.lock change. Reset it with rm -rf ~/.cache/qdrant-e2e-buildx.
CI pre-builds qdrant/qdrant:e2e-tests in the workflow, so this path is skipped there.