* notify optimizers scheduler of the budget change
* Zero permit count when releasing CPU or IO
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
* create REST, CollectionQuery, and ShardQuery structures. Parse all variables and conditions
* move stuff around, rename segment Expression to ParsedExpression
* rename to MultExpression
* add Div in REST, use unique geo_distance tag
* review fixes
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* WIP: compressed versions store
* compressed versions store
* fmt
* model test against vec
* use u32::BITS instead of new const
* clippy 😡
* rename CompressedVersionsStore -> CompressedVersions
* Apply suggestions from code review
* Use ahash
* Assert length of version slice
* Add has function
---------
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
Co-authored-by: Tim Visée <tim+github@visee.me>
Co-authored-by: timvisee <tim@visee.me>
* rename cpu_budget -> resource_budget
* clippy
* add io budget to resources
* fmt
* move budget structures into a separate file
* add extend permit function
* dont extend existing permit
* switch from IO to CPU permit
* do not release resource before aquiring an extension
* fmt
* Review remarks
* Improve resource permit number assertion
* Make resource permit replace_with only acquire extra needed permits
* Remove obsolete drop implementation
* allocate IO budget same as CPU
* review fixes
---------
Co-authored-by: timvisee <tim@visee.me>
* handle rescoring up to local shard
* use heap to keep only best points
* add stopping flag
* handle wrapped segment deletions
* drop ordering, assume always LargeBetter
* use ahash for u32 keys
* micro optimization on update_point_versions
* use Option instead of Vec for error
* clarify comment
* Always delete old point versions when applying point operation
* Add helper struct for updates/deletes, make selection more readable
* Cleanup
* Add debug assertion
* If having multiple point versions, delete first, then apply operations
* Deleted old points with their existing point version, extend test
* Don't delete other points with the same version for now, add test case
* Remove helper struct since updates and deletes are separated again
* Smarter capacity estimate
* apply change to all points with latest version
* Use small vec extra allocations
* Eliminate delete closure, delete internally and pass hardware counter
* Update comment
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* More cases for `bitpacking_links::tests`
* Turn `for_each_packed_link` into iterator
* Test `check_iterator_fold` and `check_exact_size_iterator_len`
* Drop `for_each_packed_link` and use `iterate_packed_links`
* Introduce `LinksIterator` for `segment`
* bump and migrate to rand 0.9.0
also bump rand_distr to 0.5.0 to match it
* Migrate AVX2 and SSE implementations
* Remove unused thread_rng placeholders
* More random migrations
* Migrate GPU tests
* bump seed
---------
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
* support gpu devices without half float
* disable f16 feature
* are you happy clippy
* Extend log warning if f16 is not supported
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
* Finish io measurement for payload storage
* Remove done TODOs
* review remarks
* make signature of `wipe()` consistent
* Remove hardware_counter from tracker.rs and make interfaces consistent
* Add hw_counter to payloads update_storage function from dev
