26 Commits

Author SHA1 Message Date
Andrey Vasnetsov
b9cc0778f5 add tracing ID into audit logging (#8402)
* add tracing ID into audit logging

* format and refactor

* clippy

* review fixes

* fmt
2026-03-17 00:00:06 +01:00
Andrey Vasnetsov
1ae2070e1c audit logging (#8071)
* initial implementation

* internal logging type

* wrap more places into auth function

* [manual refactor] use &Auth in .toc fucntion - minimise direct .access()

* [manual refactor] fmt

* [manual refactor] remove unannotated .access in creation of snapshots

* [manual refactor] remove unannotated .access in cluster telemetry

* [manual refactor] remove unannotated .access

* [manual refactor] do not log /metrics api access

* [manual refactor] do not run the service if audit logging failed to init

* [manual refactor] make auditable operation names for point and cluster updates

* [manual refactor] remove excess cloning of auth object

* [manual refactor] fmt

* [AI] instead of manual writing into the file, use tracing_appender crate

* [AI] simplify configuration to match internal crate

* fmt

* [manual refactor] cover staging options

* [AI] refactor x-forwarded-for handelling

* [manual refactor] use consts for x-forwarded-for + recover handelling for RFC case in tonic

* Update src/tonic/forwarded.rs

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* [manual] review fix: use FORWARDED in actix

* [manual] review fix: do not log strict mode checks

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-02-09 19:16:20 +01:00
Jojii
290f9e2675 Always load InferenceService (#7050)
* Always load InferenceService

* Add BM25 openapi test

* Add warning to development.yaml config (#7051)
2025-08-13 16:59:42 +02:00
Tim Visée
9a445a71fe Add all feature flag, report enabled flags on start (#6228)
* Add magic all flag, if set all features will be enabled

* Report enabled feature flags on start, if any is enabled

* Remove pub visibility from all, add comment it's only applied on init

* Don't check all flag in is_empty
2025-03-24 14:11:28 +01:00
Tim Visée
4f8735462d Add feature flag to skip RocksDB in immutable payload indices (#6226) 2025-03-22 11:01:24 +01:00
Tim Visée
6aa24ee51e Fix snapshot restore with new shard key format (#6202)
* Use new shard key storage wrapper type when recovering snapshot

* Fix compiler warning
2025-03-19 17:57:53 +01:00
Tim Visée
45d2fd9cf7 Mutable ID tracker integration (#6174)
* Rename mutable ID tracker mappings file

* Make new ID tracker the default, add new variant for RocksDB tracker

* In mutable ID tracker, only list files if they exist on disk

* Don't use the new mutable ID tracker yet

* Feature flag usage of new mutable ID tracker

* Simplify RocksDB check a bit

* Rename both the mutable ID tracker files
2025-03-19 13:37:54 +01:00
Tim Visée
75eb0b38af Add global feature flags (#6194)
* Add global feature flags structure, load on start

* Feature flag use of new shard key format

* Enable feature flag for new shard key format by default in dev builds

* Prevent unwrapping everywhere; Single place for initialization (#6198)

* Disable shard key feature flag by default for now, it causes problems

---------

Co-authored-by: Jojii <15957865+JojiiOfficial@users.noreply.github.com>
2025-03-19 11:15:40 +01:00
Roman Titov
189fb810e7 Add resharding_enabled option to ClusterConfig (#5770)
* Add `resharding_enabled` option to `ClusterConfig`

If `resharding_enabled` is *not* set, `StartResharding` API returns an error

* Enable `resharding_enabled` option in development config

* fixup! Add `resharding_enabled` option to `ClusterConfig`

Fix tests

* fixup! Enable `resharding_enabled` option in development config

😅

* fixup! Enable `resharding_enabled` option in development config

🙄🙄🙄
2025-01-15 16:07:06 +01:00
n0x29a
ca6e0f351b Inference Service, billing-id (#5287)
* Inference Service, token

---------

Co-authored-by: n0x29a <n0x29a@example.com>
2024-10-22 12:49:00 +02:00
n0x29a
548d33f8fb Inference Service Integration (#5240)
* Inference Service Integration

* Update src/common/inference/service.rs

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>

* Inference Service Integration

* review fixes

---------

Co-authored-by: n0x29a <n0x29a@example.com>
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2024-10-16 19:37:38 +02:00
Arnaud Gourlay
e527af28d7 Remove unused debug setting (#2339) 2023-08-21 13:41:43 +02:00
Andrey Vasnetsov
967d387a37 Web UI integration (#2009)
* wip: integrate UI web interface

* fmt

* Api key exclusion rule + fixes

* upd welcome url

* rollback api key in config
2023-06-06 21:42:24 +02:00
Roman Titov
5d3f9e167e Improve handling of out-of-disk-space errors during Qdrant startup (#1755)
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2023-04-24 13:02:29 +02:00
Tim Visée
e985ff31b6 General improvements (#1687)
* Change file mode of some .sh files to make them executable

* Remove whitespace on empty lines from YAML configurations

* Fix unused import warning on Windows
2023-04-10 20:03:56 +02:00
Andrey Vasnetsov
ef7b4ac2a7 propagate payload index to temporary segment during optimization (#1133) 2022-10-18 10:49:37 +02:00
Ivan Pleshkov
f357bd5d9b Allow to flush segment in separate thread (#927)
* allow to flush segment in separate thread

* flush as separate function (#928)

* flush as separate function

* review suggestion

* reduce locks during vector scoring

* fmt

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>

* don't run background flush twice

* Update lib/segment/src/segment.rs

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>

* increase flush interval

* Update lib/segment/src/segment.rs

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>

* are you happy fmt

* test background flush

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
2022-08-15 13:47:52 +04:00
Arnaud Gourlay
841ba05850 [Sharding] Introduce cluster configuration (#547)
* cluster configuration

* default cluster configuration to not pollute config.yaml

* code review suggestions

* please clippy
2022-05-04 20:22:45 +02:00
Arnaud Gourlay
74a1eb8294 p2p gRPC API is now internal (#464) 2022-04-12 11:53:02 +02:00
Arnaud Gourlay
2a81fc9142 Remove sled for alias mappings (#402)
* improve alias tests

* introduce new alias persistence and remove Sled

* introduce FileStorageError

* use new error in AliasMapping

* aliases are kept in memory and saved to disk when modified

* better naming

* make alias renaming atomic

* fmt

* use correct alias and docs

* code review: cleaner error conversion
2022-03-23 14:21:04 +01:00
Arnaud Gourlay
c928217a8c enable gRPC via a runtime config (#352)
* enable gRPC via a runtime config

* code review

* code review: rename rest_port to http_port

* Update config/config.yaml

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>

* disable grpc by default

* enable gPRC in integration test

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2022-03-03 19:23:31 +01:00
Konstantin
9b9991d6bb [GRPC] Introduce GRPC API based on tonic (#76) 2021-08-01 20:49:15 +02:00
Andrey Vasnetsov
0514721912 add debug logging for HNSW build + small refactoring 2021-05-31 14:48:07 +02:00
Andrey Vasnetsov
4ba31ef328 fix WAL indexes + stress test 2020-12-02 00:23:29 +01:00
Andrey Vasnetsov
3206ceb4f1 add actix web server 2020-06-09 15:19:39 +02:00
Andrey Vasnetsov
e4452500a0 init 2020-05-30 23:35:38 +02:00