* Add in-ram-mmap payload storage type
* When RocksDB is not compiled, select in-ram mmap payload storage
* Enable payload storage on-disk check again in config mismatch optimizer
* Add feature flag to build in-ram-mmap payload storage
* Tweak is_on_disk condition
* Bump version estimates in runtime feature flags
* add `"match": { "phrase": ... }` condition
* gen grpc and openapi
* [phrase matching] expose `phrase_matching` flag in rest and grpc (#6620)
* expose setting in rest and grpc
* phrase matching openapi test
* regen openapi
* [phrase matching] Text index fixes (#6730)
* allow rocksdb-based immutable text index
* fix repeated-token phrases
* fmt
* Update OpenAPI spec
---------
Co-authored-by: timvisee <tim@visee.me>
* add repeated word case in openapi test
* [phrase match | strict mode] Allow phrase condition when enabled in index (#6749)
* allow phrase filter when index is present
* prettier error message
* clippppppy
---------
Co-authored-by: timvisee <tim@visee.me>
* bq encodings
* are you happy clippy
* are you happy clippy
* are you happy clippy
* are you happy clippy
* gpu tests
* update models
* are you happy fmt
* move additional bits to the end
* fix tests
* Welford's Algorithm
* review remarks
* are you happy clippy
* remove debug println in test
* coderabit nitpicks
* remove unnecessary clone and partialeq
* Use f64 for Welford's Algorithm
* try fix ci
* revert cargo-nextest
* add debug assertions
* Add stopwords support
* Update codespell configuration and improve stopword assertion message
* Add stopwords schema to OpenAPI definition
* Rename StopwordsOption to StopwordsInterface
* Add stopwords support for multiple languages
* Update codespell configuration to skip all Rust files in stop_words directory
* Normalize stopwords to lowercase during insertion
* Add language aliases for stopwords
* + grpc
* Unspecified Language
* clippy fmt
* openapi
* Refactor to use BTreeSet
* update tokenizer test to include custom stopwords with contractions
* simplified grpc StopwordsInterface
* alias for languages
* use `lowercase` flag for stop word normalizing
* grpc: enum -> string
* consistent naming
* Remove UnspecifiedLanguage and use From for conversions
* fix conversions
* fmt
* use stopwords inside different tokenizer
* use ahash
* fix tests
* fix no-rocks-db feature
* simplify code and fix test
* update test
* fix clippy
* fmt
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* Improve telemetry logic and test
* Parametrize telemetry test
* Consistency hash peeer ID across telemetry
* clean test
* Use Option in segments telemetry
* updat openapi spec
* Avoid test failure on change in order of params
* Add in rest and grpc
* add to QueryEnum
* implement Query trait
* connect to scorer creation
* upd tests
* additional changes
* fmt
* gen openapi and grpc docs
* coderabbit fix
* add changes in async scorer
* test sum_scores in more places, refactor to remove repetition
* use f64 as internal expression score
* allow datetime as expression
* add test cases
* fix schema, generate openapi and grpc docs
* fmt
* clippy
* fix test precision
* graceful handling of too large numbers, more tests
* rename to DatetimeExpression for consistency
* use datetime's timestamp as seconds
* capture variable name when parsing
* homogenize DateTime into Datetime
* use interface distinction between datetime strings and payload keys
* fix rebase
* fix after rebase
* Add decay expressions
* remove code duplication in conversion
* test and fix lambda roundtrip
* Remove debug print in decay midpoint validation
* Improve error messages for decay midpoint and scale validation
* use similar lambda for gauss and exp decays
* avoid abs before squaring
* add comment about decay range
* add debug assertion
* Add payload index filtering IO measurements for some indices
* Add payload index metric to api and telemetry
* Also account for index access overhead
* Review remarks
* Anonymize new HardwareUsage field
* Fix tests
* Measure update operations hardware IO
* Add support for distributed setups
* also measure update_local
* Add consensus tests for HW metrics of update operations
* add test for upserting without waiting
* Disable HW usage reporting when not waiting for update API
* Review remarks
* Fix resharding collecting hw measurements
* Fix metric type
* New struct HardwareData for better accumulation
* Ensure we always apply CPU multiplier
* Apply suggestions from code review
* Update src/actix/api/update_api.rs
Co-authored-by: Tim Visée <tim+github@visee.me>
* Fix assert_with_upper_bound_error threshold calculation.
* Clarifying why we don't measure shard cleanup
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
* create initial strucutres
* clippy
* start field-query refactoring
* start field-query refactoring (2/N)
* start field-query refactoring (3/N): duplicate is_empty/null condiftions as field condition
* start field-query refactoring (4/N): re-instate is_empty fallback in case new index is not built yet
* filter for is_empty/is_null
* implement add/remove point
* upd schema
* open and create of null-index
* create null-index
* fix test
* Update lib/segment/src/index/query_optimization/condition_converter.rs
Co-authored-by: Tim Visée <tim+github@visee.me>
* unit test for null-index
* more unit tests
* add openapi tests
* fmt
* fix for integartion tests
* rabbit review fix
* make [null] non-empty
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
* Add docs about running coverage tests locally
* Add link to CI coverage report
* higlight that we only cover unit tests
* Add CI coverage report image
* Add more steps
* Remove OOM instructions
* Add limits for filter and conditions
* clippy
* Review remarks + nested condition test
* Fix opnapi specs
* Improve error message by giving info about limits and usage
