* Measure update operations hardware IO
* Add support for distributed setups
* also measure update_local
* Add consensus tests for HW metrics of update operations
* add test for upserting without waiting
* Disable HW usage reporting when not waiting for update API
* Review remarks
* Fix resharding collecting hw measurements
* Fix metric type
* New struct HardwareData for better accumulation
* Ensure we always apply CPU multiplier
* Apply suggestions from code review
* Update src/actix/api/update_api.rs
Co-authored-by: Tim Visée <tim+github@visee.me>
* Fix assert_with_upper_bound_error threshold calculation.
* Clarifying why we don't measure shard cleanup
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
* Anonymize ShardKey in ReplicaSetTelemetry
* Anonymize vector names in CollectionConfigInternal
* no-op: impl HardwareUsage for HardwareUsage (used in HardwareTelemetry)
* no-op: impl Anonymize for OptimizersStatus (used in OptimizerTelemetry and CollectionsAggregatedTelemetry)
* no-op: impl Anonymize for TrackerStatus in (used in TrackerTelemetry)
* no-op: Anonymize VectorParams in VectorsConfig
* no-op: Copy fields explicitly in OperationDurationStatistics
* no-op: `.clone().map(|x| x.anonymize())` -> `.anonymize()`
* no-op: impl Anonymize for ConsensusThreadStatus (used in PeerInfo and ClusterStatusTelemetry)
* create initial strucutres
* clippy
* start field-query refactoring
* start field-query refactoring (2/N)
* start field-query refactoring (3/N): duplicate is_empty/null condiftions as field condition
* start field-query refactoring (4/N): re-instate is_empty fallback in case new index is not built yet
* filter for is_empty/is_null
* implement add/remove point
* upd schema
* open and create of null-index
* create null-index
* fix test
* Update lib/segment/src/index/query_optimization/condition_converter.rs
Co-authored-by: Tim Visée <tim+github@visee.me>
* unit test for null-index
* more unit tests
* add openapi tests
* fmt
* fix for integartion tests
* rabbit review fix
* make [null] non-empty
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
* Bump Rust edition to 2024
* gen is a reserved keyword now
* Remove ref mut on references
* Mark extern C as unsafe
* Wrap unsafe function bodies in unsafe block
* Geo hash implements Copy, don't reference but pass by value instead
* Replace secluded self import with parent
* Update execute_cluster_read_operation with new match semantics
* Fix lifetime issue
* Replace map_or with is_none_or
* set_var is unsafe now
* Reformat
* create REST, CollectionQuery, and ShardQuery structures. Parse all variables and conditions
* move stuff around, rename segment Expression to ParsedExpression
* rename to MultExpression
* add Div in REST, use unique geo_distance tag
* review fixes
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* bump and migrate to rand 0.9.0
also bump rand_distr to 0.5.0 to match it
* Migrate AVX2 and SSE implementations
* Remove unused thread_rng placeholders
* More random migrations
* Migrate GPU tests
* bump seed
---------
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
* Add limits for filter and conditions
* clippy
* Review remarks + nested condition test
* Fix opnapi specs
* Improve error message by giving info about limits and usage
* Allow max-optimization-thread config to be set to null on update
* add new field. Support both, but prepare for deprecation.
* better openapi
* only introduce change in OptimizersConfigDiff
* move to `api::rest::schema`
* update openapi
* improve test, fix diff to config conversion
* upd grpc docs
* clippy
* remove schemars from common common
---------
Co-authored-by: Gulshan Kumar <kumargu@amazon.com>
* add on_disk option for bool index
* test that all files are covered
* generate openapi and docs
* clippy
* remove `populate` changes
* use `walkdir` crate
* Apply clippy suggestions
---------
Co-authored-by: timvisee <tim@visee.me>
* Optimize JSON-Proto conversion and add unit tests
* mproved memory allocation by eliminating raw loops
* Enhanced readability by reducing nesting and restructuring logic for clarity
* Added tests to assert correctness and reliability of the JSON-Proto conversion process
* Resolve clippy issues
