* include document & image objects into grpc API
* introduce image and object to rest api
* minor refactoring
* rename Vector -> VectorInternal
* decompose vector data structures
* add schema
* fmt
* grpc docs
* fix conversion
* fix clippy
* fix another conversion
* rename VectorInput -> VectorInputInternal
* replace grpc TryFrom with async functions
* fmt
* replace rest TryFrom with async functions
* add image and object into query rest
* separate inference related conversions
* move json-related conversions into a separate file
* move vector-related transformations into a separate file
* move more vector related-conversions into dedicated module
* tar_ext::BuilderExt: generalize and add drop workaround
- Generalize: instead of taking `std::fs::File`, take
`impl Write + Seek`.
- Add a workaround for drop: `tar::Builder<W>::drop` might panic if the
underlying writer panics when using in async context.
* Drop unnecessary write_fn usage
* tar_ext::BuilderExt: support both seekable and streaming variants
* tar_ext::BuilderExt: support both owned/borrowed variants
* add tests for strict mode
* add good example for strict mode test
* Add succes case to test_request_exact
---------
Co-authored-by: timvisee <tim@visee.me>
* first grpc implementations
* make usage of 'scroll' more clear
* introduce 'CheckedTocProvider' trait
* implement more grpc types
* more grpc implementations
* rename StrictModeCheckedProvider to StrictModeCheckedTocProvider
* don't use deprecated feature for UncheckedTocProvider
* Trigger optimizers when uploading snapshots
* Trigger optimizers through new method when optimizer config is updated
* review remarks
* Add comment on why ignoring channel send errors is fine
* Remove obsolete return value omitting
---------
Co-authored-by: timvisee <tim@visee.me>
* Return whether any task was cleaned
* Retrigger optimizers after cleanup timeout if we cleared a handle
* Flatten match used as decision logic on whether to run optimizers
* Add comment to elaborate we have a branch to prevent race condition
* Rename force to ignore_max_handles
* Use swap remove as it is a bit more efficient
* In debug builds on Unix, use sha256sum binary because it is faster
* Add test to ensure sha256sum produces the same hash
* Apply suggestions from code review
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
* Only use sha256sum on Linux platforms
* Remove hash through binary implementation
* Enable release optimizations for sha2 crate in debug builds
---------
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
* Track number of points optimized and expose in telemetry
* refactor
* openapi specs
* remove dbg
* Return num points optimized from optimize() func
* fmt
* fix
* fix type in tests
* Store total points indexed on shard level instead of optimization level
* fmt
* fix test
* trigger ci
* fix openapi schema
* review fixes
* fmt
* improvements and fix test
* review fixes
* use const for indexing optimizer name
* fmt
* return segment id from optimize() func
* review fixes
* fix
* fix
* fik
* minor var name improvement
* Use Option<usize> to return segment id
* Use segment ID type rather than ambiguous usize
* fix test
* avoid intermediate check
* review fixes
* Rename total_indexed_points to total_optimized_points
* Update openapi schema
* optimize() should return number of points in new segment instead of segment id
* add else condition
* take read lock
* fmt
* remove flaky assert
* Count points on new segment without locking
---------
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: generall <andrey@vasnetsov.com>
