* include document & image objects into grpc API
* introduce image and object to rest api
* minor refactoring
* rename Vector -> VectorInternal
* decompose vector data structures
* add schema
* fmt
* grpc docs
* fix conversion
* fix clippy
* fix another conversion
* rename VectorInput -> VectorInputInternal
* replace grpc TryFrom with async functions
* fmt
* replace rest TryFrom with async functions
* add image and object into query rest
* separate inference related conversions
* move json-related conversions into a separate file
* move vector-related transformations into a separate file
* move more vector related-conversions into dedicated module
* Trigger optimizers when uploading snapshots
* Trigger optimizers through new method when optimizer config is updated
* review remarks
* Add comment on why ignoring channel send errors is fine
* Remove obsolete return value omitting
---------
Co-authored-by: timvisee <tim@visee.me>
* Check that peer still exist, when applying finish transfer operation
* Simplify `Collection::finish_shard_transfer` method
* Simplify "peer exists" check
* Handle destination replica before source replica
* Only remove source replica, if destination replica is `Active`
* Tweak destination replica handling to be more like original impl
* rename keyword -> string for internal data structures
* implement MatchAny and expect for filter on UUID mmap index
* implement MatchAny and expect for cardinality estimation on UUID mmap index
* refactor PayloadFieldIndex filter for handling incorrect empty query parsing case
* rename to FacetRequestInternal
* add rest endpoint
* fix correctness by fetching the whole list of values
* fix mmap map index variant
Also removes test for sorted output, for now
* add ytt spec
* fix clippy
* use hashmap inside of local shard
* rename operation to `facet`, add access test
* whitelist endpoint
* change api
* make limit optional
* make mmap_type independent from segment structures
* make bitvec and thiserror workspace dependencies
* move mmap_type into common/memory subcrate
* fmt
* implement facets in ToC
- implement CheckableCollectionOperation for FacetRequest
- implement facet in Collection
* complete facet of internal service + From<StorageError> refactor
- finalize connecting internal service to table of content
- make error_to_status a From implementation, refactor accordingly
* Add and fix missing timeout in the retrieve APIs
* no need for extra timeout handling as single ops are doing it
* don't surface timeout too high in shard transfer infra
* add todos for future work
* update timeout after first query
* Start with fresh state through SaveOnDisk directly
* Remove unused temporary directory parameter
* Add docs, init may persist on disk right away
* Apply review suggestion
* Resume resharding driver on restart if resharding is active
* Update resharding state comment started text
Using since versus started. 'Since' better clarifies that the current
state has been active since that time. While 'started at' could lead to
confusion on whether that time is for starting the whole resharding
operation or just that state.
* Add resharding resumption test during migrating points
* Recover resharding hash ring after loading shards
* Update resharding resume test, interrupt at multiple stages, less points
* Remove unused parameter
* Update collection shard count at resharding shard creation/destruction
* Apply suggestions from code review
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
* Use debug_assert_eq
* Remove trailing comma
---------
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
* universal-query: Grouping REST API
* add API docs
* bump API count
* add jwt validation test
* fix API
* better test
* stay on CoreSearchRequest where possible
* use existing scoring helper
* push broken test to illustrate issue
* track best score per point_id to enable sort on payload
* add example test for query discover with groups
* track ScoredPoint instead of PointId
* add default values for smoother ux
* universal-query: lookup_from implementation
* pushin test demonstrating broken equivalence
* more tests and fixes
* validate fusion not use with 'using' field
* cleanup collection info propagation
* do not clone the query in CollectionQueryResolveRequest
* cleanup
* address last code review
* End resharding and clean up
* Track resharding stage in `ReshardState`...
...and tweak `commit_read`/`commit_write`/`finish` ops to work properly
* Refactor hashring check for migrating and deleting migrated points
* When propagating deletes, only wait for the last batch
---------
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
