* Add REST API for reading audit logs across the cluster
Introduces a new `GET /audit/logs` endpoint that retrieves audit log
entries from all peers in the cluster. The API supports filtering by
time range (time_from/time_to), dynamic key=value field filters, and
a built-in limit parameter to prevent reading too many logs at once.
- Add `audit_reader` module in storage crate for efficient file-based
log retrieval, selecting only files whose date range overlaps the
query window
- Add `GetAuditLog` internal gRPC RPC for cross-peer log retrieval
- Add `GET /audit/logs` REST endpoint restricted to management access
- Aggregate and sort results from all peers by timestamp (newest first)
Made-with: Cursor
* [AI] Update filter_files_by_time_range make it aware of the log rotation granulatity (either hourly or daily)
* [AI] For AuditLogParams, try to use DateTime types natively into serde, instead of manual parsing
* [AI] Refactor API into separate file, propagate timeout, use spawn-blocking
* [AI] introduce cancellation token
* [AI] move timestamp to constant
* small manual fixes
* review fixes part 1
* review: switch to POST instead of GET
* [AI] review: sorting update
* [AI] use strict typing
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* Add deferred point count to UpdateQueueInfo
* Hide if prevent_unoptimized is false
* Clippy
* Openapi
* Iterate over appendable segments only
* Only calculate deferred point count if prevent_unoptimized is true
* Rebase fixes
* Codespell
* Move `DenseVector`/`MultiDenseVector` from `api` to `segment`
* Move `OrderByInterface` from `api` to `segment`
Reason: it's used in `edge` which shouldn't depend on `api`.
* Make `shard` -> `api` dependency optional
* Remove `api` from the amalgamation
* Don't install protoc in edge Actions
* Fixups of amalgamator
Fix issues that break `qdrant-edge` build process:
- `use … as segment;` - this causes `ast-grep` rules to replace wrong
paths. So, rename to avoid collisions.
- `#[macro_use]` and `extern crate` required be in the top-level
`lib.rs`.
- `format!("…", crate::something::…)` - `ast-grep` can't fix paths
inside macros. Fixed by moving `crate::something::…` out of the macro.
* Add lib/edge/publish workspace and amalgamation script
* Move `lib/edge/examples` into `lib/edge/publish/` workspace
And fix them to use the generated `qdrant-edge` crate.
* Add github workflow
* Cleanup `qdrant-edge` public API
Removes empty modules. Checked by `cargo doc`.
* [manual] make untagged enum for consistent API for FeedbackStrategy
* add #[validate(nested)]
---------
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
* add rest and grpc interfaces
Also handle inference for this query
* follow refactor from base branch
* renaming from Ms Cooper
* get started on validation testing
* more validations
* test equivalence with query when less than 2 feedback elements
* rename feedback query to relevance_feedback query
* fix extraction of context pairs
* rename feedback vector to example
* make coderabbit happier
* upd test
---------
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
* weighted rrf implementation
* test
* fmt
* fix edge
* validate number of sources and number of weights
* do not partial match
* upd schema
* review fixes
* update formula
* remove calcualtions from tests
* update comment, because AI have OCD
* fmt
* Fix RST_STREAM by draining http2 channels
* remove controversial duration time for now
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* Update queue info into CollectionInfo
* basic test
* fix
* cleaner
* Update lib/collection/src/shards/local_shard/mod.rs
Co-authored-by: xzfc <5121426+xzfc@users.noreply.github.com>
* fix the fix
* rename to op_num
* skip_serializing_if just in case
* first step
* test queue length with staging feature
* gate integration test based on binary feature
---------
Co-authored-by: xzfc <5121426+xzfc@users.noreply.github.com>
* introduce update_mode parameter for upsert operation to control if we want to insert, update, or upsert
* add test
* upd dockstring
* require resharding once all peers have updated version
* use service error
* fix clippy again
* wait for same version before resharding in tests
* make some existing fields optional
particularly, RemoteShardTelemetry's `searches` and `updates`, and
LocalShardTelemetry's `optimizations`
* Include shards information in cluster telemetry
* upd openapi
* nits
* only include partial snapshot info if non-empty
* fixes
* feat: Add enable_hnsw option for payload field indexes
Add optional enable_hnsw parameter to all payload index types to control
whether additional HNSW graph links are built for each indexed field.
- Add enable_hnsw field to all 8 payload index param types
- Update gRPC proto definitions and conversions
- Update OpenAPI schema
- Modify HNSW graph builder to respect enable_hnsw flag
- Add enable_hnsw() helper methods to PayloadSchemaParams and PayloadFieldSchema
- Update all tests to include new field (default: None)
When enable_hnsw is true and payload_M > 0, additional HNSW links will
be built for the payload field. Default value is true for backward compatibility.
* Fix Some format problems
* fix: address comment problem
---------
Co-authored-by: EC2 Default User <ec2-user@ip-10-78-171-148.ec2.internal>
* wrap details in another struct, less Options
* include version in peers info
* update openapi
* include app telemetry in internal service
* move debug assertion after check
* Add a test for recovery after kill during Partial
* Address AI review
* simplify test
* Merge pull request #7829
* introduce a new state
* switch to ManualRecovery for user-initiated snapshot operations
* fmt
* fix test and regen api
* test fix: instead of force recovery on update failed snapshots now re…
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* move QdrantInternalService to its own file
* implement internal telemetry endpoint
cleaner diff on src/main.rs
upd response in proto
* use try_from conversion
* rename request/response in proto
* Add ReadActive replica state
* Add consensus function to switch from Partial to ReadActive and confirm
* On stream records to a different shard ID, switch to ReadActive first
* Update OpenAPI spec
* In stream records peer sync, require all peers to be up-to-date
* Review fixes
* Rename ReadActive to ActiveRead
* update openapi spec
* explicit state checking functions
* typo
* update fallback routing with usage of a new state
---------
Co-authored-by: KShivendu <kshivendu1@gmail.com>
Co-authored-by: generall <andrey@vasnetsov.com>
