tools/coverage.sh was split into tools/unit-test-coverage.sh and
tools/integration-test-coverage.sh in #6414, but DEVELOPMENT.md still
pointed at the old path and the new script's own usage header kept the
old name. Updated both to the unit-test script.
* Add optional dial9 Tokio telemetry behind a `dial9` feature
Integrate dial9 so storage runtimes can emit production-friendly Tokio
traces. Recording is off unless the crate is built with `--features dial9`
and DIAL9_ENABLED=true is set at runtime; with the feature off, runtime
construction is byte-for-byte unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7X6MkjY3P7wpP2MfHdTDY
* Enable dial9 CPU and schedule profiling
Turn on cpu-profiling and sched events behind the same `dial9` feature,
add the DIAL9_CPU_* / DIAL9_SCHEDULE_* env knobs, and document the frame
pointer rustflags the stack unwinder needs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7X6MkjY3P7wpP2MfHdTDY
* Harden dial9 env parsing and the writer-failure path
- Reset Cargo.lock to the branch point and re-resolve, so the diff is
additive instead of re-resolving unrelated packages. This drops the
heck 0.5.0 -> 0.4.1 downgrade, which sat in the default build graph and
would have changed proto codegen identifier casing. The remaining
non-additive entry, toml_parser 1.0.9 -> 1.1.3, is forced by
proc-macro-crate via dial9-trace-format-derive.
- Parse DIAL9_* booleans the way dial9 does, accepting 1/y/yes/on and
0/n/no/off and warning on anything else. `str::parse::<bool>` took only
exact lowercase true/false, so DIAL9_CPU_PROFILE_ENABLED=0 silently left
99 Hz sampling on and DIAL9_ENABLED=1 silently left recording off.
- Require the numeric knobs to be positive. A zero disk budget made dial9
evict everything and stop recording within seconds while the log still
reported telemetry enabled.
- Treat a set-but-empty DIAL9_TRACE_DIR as unset. It skipped the /tmp
fallback and wrote up to the full budget into the working directory,
which is /qdrant next to storage/ in the official image.
- Return a disabled guard as soon as the trace writer fails, before
with_cpu_profiling and with_sched_events run. Those start their profilers
eagerly, opening a perf event per thread and installing a process-global
signal handler that build() would then discard.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7X6MkjY3P7wpP2MfHdTDY
* Correct the dial9 docs and give them their own section
- `--cfg tokio_unstable` is required for any task data at all, not merely
for fuller coverage: dial9's poll, spawn and terminate hooks are all
`#[cfg(tokio_unstable)]`, and nothing in the repo sets the flag. Without
it there is no task timeline and DIAL9_TASK_TRACKING_ENABLED does nothing.
- Document `-C debuginfo=2`. `[profile.perf]` inherits `release` and sets no
`debug` key, so the documented build symbolized off the ELF symtab with
inlined callees collapsed and no file or line, unlike `[profile.bench]`
which sets `debug = true` for this reason.
- Move the dial9 material out from between the feature list and the prose
that belongs to it. Those paragraphs describe `tracing` instrumentation
and read as dial9's when the example is wedged in front of them, which
points readers at `#[tracing::instrument]` for a tool that records Tokio
runtime events and no tracing spans.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7X6MkjY3P7wpP2MfHdTDY
* Use cfg_select!
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: timvisee <tim@visee.me>
* Move pyproject.toml to root
* Migrate pyproject.toml from Poetry to uv
* Update GH workflows
* Update test script, doc and nix to use uv
* Use latest uv
* Fix uv.lock
* Cleanup shell.nix
* Cleanup
- Explicit `uv sync` is not required, `uv run` will install deps
automatically.
- We don't provide a python package, so the `[build-system]` section
is not needed.
* Fix UV_VERSION inconsistency
---------
Co-authored-by: tellet-q <elena.dubrovina@qdrant.com>
Co-authored-by: xzfc <xzfcpw@gmail.com>
* Add docs about running coverage tests locally
* Add link to CI coverage report
* higlight that we only cover unit tests
* Add CI coverage report image
* Add more steps
* Remove OOM instructions
* Use poetry for openapi tests
* Use poetry for consensus tests
* Use poetry for gen_storage_compat_data/populate_db.py
* Make consensus_tests scripts executable
* Use poetry for test-consensus-compose
* Cleanup
* Don't call poetry run in scripts
* Add optional `tracing` crate dependency to all `lib/*` sub-crates
* Add optional alternative "tracing-logger"...
...that works a bit better with logs produced by the `tracing` crate
* Make `tracing-logger` reuse `log_level` config parameter (and `QDRANT__LOG_LEVEL` env-var)...
...instead of default `RUST_LOG` env-var only
* Replace `env_logger` with `tracing` (#2381)
* Replace `env_logger` with `tracing`/`tracing-subscriber`/`tracing-log`
TODO:
- Implement slog drain that would *properly* convert slog records into tracing events
(`slog-stdlog` is fine for now, though)
* fixup! Replace `env_logger` with `tracing`/`tracing-subscriber`/`tracing-log`
Fix `consensus` test
* fixup! fixup! Replace `env_logger` with `tracing`/`tracing-subscriber`/`tracing-log`
Forget to `unwrap` the result 🙈
* fixup! fixup! Replace `env_logger` with `tracing`/`tracing-subscriber`/`tracing-log`
`tracing_subscriber::fmt::init` initializes `tracing_log::LogTracer`
automatically with default `tracing-subscriber` features enabled 💁♀️
* Update `DEVELOPMENT.md` documentation regarding `tracing`
* Update default log level for development to make it less noisy
---------
Co-authored-by: timvisee <tim@visee.me>
* Implement "intelligent" default log-level filtering
---------
Co-authored-by: timvisee <tim@visee.me>
* Add github action to codespell master on push and PRs
* Add rudimentary codespell config
* some skips
* fix some ambigous typos
* [DATALAD RUNCMD] run codespell throughout
=== Do not change lines below ===
{
"chain": [],
"cmd": "codespell -w",
"exit": 0,
"extra_inputs": [],
"inputs": [],
"outputs": [],
"pwd": "."
}
^^^ Do not change lines above ^^^
* Add dev branch as target for the workflow
* Add optional `tracing` support
TODO:
- add documentation
* Add explicit compile-time check for `--cfg tokio_unstable` rust flags...
...if `console-subscriber` feature is enabled
* Add basic documentation
* fixup! Add basic documentation
Add links, reworded few phrases and document `tracing` feature
* fixup! fixup! Add basic documentation
- Fix `tokio-tracing` link
* fixup! Add basic documentation
Too much code-blocks...
* Move tracing setup into a separate function
* fixup! Move tracing setup into a separate function
- move tracing setup to a separate file
- fix error handling
* fixup! Move tracing setup into a separate function
Add missing file 🙈
* Add test script to ensure OpenAPI files are consistent with sources
* Add CI job to test OpenAPI file consistency
* Add CI task to test gRPC file consistency
* Tweak consistency scripts a bit, touch temp file to trigger gRPC rebuild
* Don't test .gitignored files
* Mention updating the OpenAPI specification is enforced by CI
* Update CI job configuration
* Also check consistency of gRPC docs
* Rename temporary files to have a .diff prefix
* Add docs to consistency checking scripts
* wip: re-implement http tests using openapi validation library
* collection delete api
* payload operations test
* payload delete operations test
* payload index creation and removing test
* uuid operations test
* schema consistency test
* add tests into pipeline
* chmod +x
* faster local runs with docker cache
* upd instruction
* add parameters to optimizer config
* benchmark search speed in different segment sizes
* use constructor for FilteredScorer
* * Implement benchmarks for HNSW index search with different number of
stored points
* Fix minor issue in HNSW graph edge assignment
* Update profiler with call-graph report generation
* Add profiling guide
* Add HNSW graph statistics test function (debug inly)
* limit resulting segment size in merge optimizer
* fix clippy
* stop the music
* fix clippy once again
* fmt once again
陈志谦