Commit Graph
12 Commits
Author SHA1 Message Date
Andrey Vasnetsov 6e0ddbafa9 disk cache hygiene (#6323)
* wip: implement explicit populate and clear_cache functions for all components

* fmt

* implement clear and populate for vector storages

* fmt

* implement clear and populate for payload storage

* wip: implement explicit populate and clear_cache functions payload indexes

* implement explicit populate and clear_cache functions payload indexes

* fix clippy on CI

* only compile posix_fadvise on linux

* only compile posix_fadvise on linux

* implement explicit populate and clear_cache functions for quantized vectors

* fmt

* remove post-load prefault

* fix typo

* implement is-on-disk for payload indexes, implement clear on drop for segment, implement clear after segment build

* fmt

* also evict quantized vectors after optimization

* re-use and replace advise_dontneed
2025-04-20 23:54:40 +02:00
Kumar Shivendu f514b29641 refactor: OpenAPI tests (#3606)
* test: Restructure OpenAPI tests

* fix: Remove cache dirs

* fix: Remove cache dirs from helpers

* fix: Path of openapi.json

* fix: Update path of openapi tests everywhere
2024-03-05 17:07:42 +01:00
Andrey Vasnetsov e59280f1bb Revert "fix: Include git commit id while building docker image if possible" (#3468) 2024-03-05 16:52:57 +01:00
Kumar Shivendu de5d120197 fix: Include git commit id while building docker image if possible (#3419)
* feat: Print commit id in dev container build workflow

* fix: Keep .git while building Qdrant binary in docker

* fix: Remove redundant printing of git commit id

* fix: Copy only git files first

* fix: Docker build should have commit id if present

* refactor: Use fewer lines of code
2024-03-05 16:52:51 +01:00
Andrey Vasnetsov 17dfe92e4d test for snapshot operations with different volumes (#2554)
* test for snapshot operations with different volumes

* fix collection snapshot upload + test for shards snapshots

* replace persistance -> move_file

* simplify shard snapsot upload method

* fmt

* chmod + x

* replace network host

* fmt

* bash linter [skip ci]

* fix review
2023-09-04 16:45:08 +02:00
JesseandAndrey Vasnetsov b11729e77b Add caching of docker layers in CI (#1856)
* Add caching of docker layers in CI

Build required docker images for CI in a workflow step using buildkit's
gha cache type. This will populate the local layer cache from github
actions' cache. Builds in subsequent CI steps will be nearly instant,
because all layers can be reused.

* add minor change to see if build time is any faster

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2023-05-17 11:24:00 +02:00
Andrey Vasnetsov e911faf8b3 wait for leader election before submitting the consensus operation (#897) 2022-08-03 10:59:45 +02:00
Andrey Vasnetsov f612460f88 merge optimizer should try to take all possible segments, not just 3 (#775)
* merge optimizer should try to take all possible segments, not just 3

* prevent creating optimizing tasks if it reaches max available threads
2022-07-04 14:35:15 +04:00
Andrey VasnetsovandArnaud Gourlay fac87018c4 Snapshoting API (#764)
* wip: rest api for snapshots

* fmt

* fix tests

* fmt

* wip: collection snapshoting and recovery

* fmt

* remote shard snapshots

* remote shard test

* fmt

* extend proxy snapshot test + fix double read lock

* fmt + clippy

* openapi schema

* Update openapi/openapi-snapshots.ytt.yaml

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>

* Update openapi/openapi-snapshots.ytt.yaml

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>

* Update src/main.rs

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>

* Update src/main.rs

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>

* reduce writes on snapshots location

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
2022-07-01 13:21:57 +02:00
Egor IvkovandAndrey Vasnetsov 55ccf1fff5 Run consensus in docker-compose (#609)
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2022-05-18 19:28:17 +03:00
Andrey Vasnetsov cbb69f0d4a add clang to the builder image 2020-12-13 15:14:07 +01:00
Andrey Vasnetsov d8b0e6c283 dockerfile + examples 2020-10-27 19:57:39 +01:00