* Correctly rebuild hash rings, don't use default with custom sharding
* Correctly initialize hash ring map with custom sharding
Don't load the non key ring by default
* Validate shard key for resharding more aggressively
When using auto sharding, require no shard key. When using custom
sharding, require a shard key.
* Only pass sharding method into shard holder construction
* Remove now obsolete debug assertion, we use same source config
* Correctly reconstruct hash rings, also add shard IDs in auto mode
* Add test, do not break consensus on invalid resharding down operation
* Add tests for resharding down shard count precondition
* Minor test tweaks
* Improve debug assertion
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
---------
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
* simple integration test for tenant promotion
* add cleanup after promotion
* When switching to ReadActive state, set it for the correct peer ID
* fix test
---------
Co-authored-by: timvisee <tim@visee.me>
* Add ReadActive replica state
* Add consensus function to switch from Partial to ReadActive and confirm
* On stream records to a different shard ID, switch to ReadActive first
* Update OpenAPI spec
* In stream records peer sync, require all peers to be up-to-date
* Review fixes
* Rename ReadActive to ActiveRead
* update openapi spec
* explicit state checking functions
* typo
* update fallback routing with usage of a new state
---------
Co-authored-by: KShivendu <kshivendu1@gmail.com>
Co-authored-by: generall <andrey@vasnetsov.com>
* make jwt with payload filter fail
* remove `whole` access requirement
* remove unnecessary mut
* fmt
* adjust test
* fix test
* remove more tests
* fix test
* fix test again
* fix test again
* Replace deprecated PayloadConstraint with JSON Value placeholder
* Fix expect message
---------
Co-authored-by: timvisee <tim@visee.me>
* Add vector count per vector-name to metrics API
* Add to metrics API
* Improve TinyMap::get_or_insert_default and add tests
* Minor improvements
* Update openapi
* Review remarks
* Remove `collection_vectors` since it can be calculated manually
* Respect payload and hashring filters while replicating point
* Merge points instead of replacing for filtered stream records transfer
* cleanup
* Update utils
* Allow filters in transfer
* drop hashring filter since its not required when destination has single shard
* test comment
* Pass actual filter
* use exact=True
* Forward updates that satisfy the filter
* make test long enough for extra points to be upserted
* minor improvements
* fix test
* try with only inserting new points
* Temporarily forward points that match before or after
* Fix failing test
* Only do inserts
* Dont merge points and update old points in test
* Fix the bug in integration test
* fix read_batch_with_hashring fn name and docs
* changes after rebase
* add validation check
* apply suggestions
* Always trim clock tag when forwarding to a different shard ID
* review fixes
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* reorder arguments of scroll_by
* add slow request log for scroll request
* add slow request log for facet request
* add slow request log for count request
* add slow request log for retrieve request
* Remove obsolete clone
* generic implementation of Loggable for Arc
* Refactor scroll by arguments (#7296)
* separate internal scroll operation from the one used by user
* fmt
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
* Allow replicating points between shards
* Add integration test for replicating points between shards
* Drop filter from ForwardProxyShard to for now
* Clean up test
* fix assert failures
* Fix integration test
* Drop reliance on FilteredStreamRecords
* minor improvements
* Currently running optimizer count in metrics
* Clearly state the prerequisites of count_optimizers_running()
* Minor improvements
* improve metric naming
* Re-sort segments on flush, don't use proxy but inner segment state
* Also sort segments when proxying all segments for snapshot
The snapshot operation explicitly flushes segments. We therefore must
satisfy flush ordering there too.
* Add some helpful comments
* Add explicit flush ordering enum for segments
* Minor tweaks
* Fix spelling in comment
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
---------
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
* rename `NamedQuery::new_from_vector` -> `NamedQuery::new
* add feedback query in CollectionQuery. Extract pairs from feedback
* add tests
* fmt
* parameterize num pairs
* rename Linear to Simple
* follow refactor from base branch
