* internal implementation for filter any
* implement api for match `text_any`
* allow naive implementation of text match any
* dedup after kmerge + renames
* improve `expected_should_estimation`
* congruence test
* openapi test
* setup index in each test
---------
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
* allow custom K parameter for RRF
* generate grpc docs and openapi
* use tagged type approach for parametrized fusions
* use params approach in grpc
* simplify api structure
* upd schema
* nits
* rest: parameterized rrf as query variant
* consistency in doc comments
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* make bm25 parameters compatible with default fastembed params
* bm25 params in openapi schema
* fmt
* improve bm25 config deserialization
* more explicit docstring for disabling english
* Retain more closed segments
* Add back rocksdb
* Update WalOptions/WalConfig across the code
* Use NonZeroUsize
* Expose via APIs
* Update gRPC docs
* Fix stoarge compat test
* recompile openapi.json with rocksdb
* default wal retain closed fn
* update openapi.json
* Use qdrant/wal latest commit and remove from config.yaml
* Deprecate init_from
* Mark memmap_threshold as deprecated
* Mark locks API as deprecated
* Mark RBAC collection access payload filter as deprecated
* Allow deprecations in tests and benchmarks
* expose mmr in rest and grpc
* fix referenced vectors in mmr queries
* rename to MmrInput in grpc for consistency
* mmr is now an optional parameter of nearest query
* fix score_threshold at local shard
* fix conversion of rest -> collection
* fix and refactor planned query
* nits
* rename `lambda` to `diversity` in interface
* handle score_threshold within mmr calculation
* add openapi test
* fix candidate limit at collection level
* candidate_limit -> candidates_limit
* finish rename to candidates_limit
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* add mmr as a ScoringQuery
* add mmr to CollectionQueryRequest
* handle mmr after collecting from shards
* use unwrap_or_else
* include mmr as a vector query
* ordering for MMR is None
* fix score threshold
* fmt + todo comment
* remove unused function
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* add `"match": { "phrase": ... }` condition
* gen grpc and openapi
* [phrase matching] expose `phrase_matching` flag in rest and grpc (#6620)
* expose setting in rest and grpc
* phrase matching openapi test
* regen openapi
* [phrase matching] Text index fixes (#6730)
* allow rocksdb-based immutable text index
* fix repeated-token phrases
* fmt
* Update OpenAPI spec
---------
Co-authored-by: timvisee <tim@visee.me>
* add repeated word case in openapi test
* [phrase match | strict mode] Allow phrase condition when enabled in index (#6749)
* allow phrase filter when index is present
* prettier error message
* clippppppy
---------
Co-authored-by: timvisee <tim@visee.me>
* Add function for validating legacy collection names
* Apply legacy collection name validation everywhere
* Add more validation to gRPC endpoints
* Adjust tests
* bq encodings
* are you happy clippy
* are you happy clippy
* are you happy clippy
* are you happy clippy
* gpu tests
* update models
* are you happy fmt
* move additional bits to the end
* fix tests
* Welford's Algorithm
* review remarks
* are you happy clippy
* remove debug println in test
* coderabit nitpicks
* remove unnecessary clone and partialeq
* Use f64 for Welford's Algorithm
* try fix ci
* revert cargo-nextest
* add debug assertions
* Add stopwords support
* Update codespell configuration and improve stopword assertion message
* Add stopwords schema to OpenAPI definition
* Rename StopwordsOption to StopwordsInterface
* Add stopwords support for multiple languages
* Update codespell configuration to skip all Rust files in stop_words directory
* Normalize stopwords to lowercase during insertion
* Add language aliases for stopwords
* + grpc
* Unspecified Language
* clippy fmt
* openapi
* Refactor to use BTreeSet
* update tokenizer test to include custom stopwords with contractions
* simplified grpc StopwordsInterface
* alias for languages
* use `lowercase` flag for stop word normalizing
* grpc: enum -> string
* consistent naming
* Remove UnspecifiedLanguage and use From for conversions
* fix conversions
* fmt
* use stopwords inside different tokenizer
* use ahash
* fix tests
* fix no-rocks-db feature
* simplify code and fix test
* update test
* fix clippy
* fmt
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* Flag index selector
* Flag take_database_snapshot
* Flag field index builder
* Flag bool index
* Flag full text index
* Flag geo index
* Flag map index
* Feature flag numeric index
* Flag skip_rocksdb
* Flag payload storage
* Flag segment
* Flag builder
* Flag backup
* Make RocksDB crate optional
* Tweak feature flag gate
* Disable default segment crate features from all dependents
* Handle flag in collection crate correctly
* Flag payload storage types in tests
* Temporarily disable on-disk check for sparse vectors
* Flag sparse vector RocksDB in GPU code
* Fix compilation error since recent merge
* Flag payload storage types in new function
* remove todo
* Explicitly implement default to more clearly state difference
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* move problems.rs to collection crate
* change type on datetime_key expression
* extract from expression
* check for unindexed fields in formula
* clippy
* dedup code
* Only implement verification for `CollectionQuery` types (#6319)
* verify CollectionQuery's Prefetch
* remove implementation for REST query
* remove implementation for REST query groups
* add basic openapi test
* Add in rest and grpc
* add to QueryEnum
* implement Query trait
* connect to scorer creation
* upd tests
* additional changes
* fmt
* gen openapi and grpc docs
* coderabbit fix
* add changes in async scorer
* test sum_scores in more places, refactor to remove repetition
* use f64 as internal expression score
* allow datetime as expression
* add test cases
* fix schema, generate openapi and grpc docs
* fmt
* clippy
* fix test precision
* graceful handling of too large numbers, more tests
* rename to DatetimeExpression for consistency
* use datetime's timestamp as seconds
* capture variable name when parsing
* homogenize DateTime into Datetime
* use interface distinction between datetime strings and payload keys
* fix rebase
* fix after rebase
