* Add QuantizationConfigDiff type
* Add quantization config diff to vector parameters
* Prefer vector config over collection config for quantization
* Update OpenAPI specification
* Validate quantization configuration quantile in 0.5-1.0 range
As per https://github.com/qdrant/qdrant/pull/1681
* Add test if check if vector specific quantization config is persisted
* Alias quantization to quantization_config in vector parameters
* Remove quantization config diff, use full vector specific config instead
* Regenerate OpenAPI specification and gRPC docs
* Fix compilation error
* Add error handling to quantization config conversions
* Fix quantization integration test, make HNSW test stricter
* Validate VectorConfig/VectorParams, remove obsolete validation
* Add HNSW config diff to vector parameters
* Validate params in collection config
* Add HNSW config to segment vector data config
* Add VectorsConfig params iterator for more elegant conversions
* Prefer vector HNSW config over collection config for building HNSW index
* Base segment vector param HNSW config on collection config
* General improvements
* Rewrite HNSW ef_construct extract function to also consider vector configs
* Update OpenAPI specification
* Add test to check if vector specific HNSW config is persisted
* review changes
* review changes
* Regenerate gRPC docs
* Fix test on Windows
* Regenerate OpenAPI specification
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* feat: add wait parm for all snapshot API
* update openAPI
* avoid unwrap
* remove all unwarp
* return HTTP 202 when no waiting on snapshot creation
* return 202 when non wait
* update open API
* update param on python test
* update and test api
* optimize the test
* return 202 when non-wait delete
* recover fixes
---------
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* actix validation
* add check for memmap/indexing_threshold
* fix actix json settings
* Validate settings configuration on start, print pretty warnings on fail
* Add more validation rules for settings and nested types
* Move shared validation logic into collection/operations
* Show validation warning in log when loading some internal configs
* Show prettier actix JSON validation errors
* Stubs for pretty handling of query errors, reformat validation errors
* Use crate flatten function, the hard work was already done for us
We don't have to flatten validation errors into their qualified field
names ourselves because there is a utility function for this.
* Configure actix path validator
* Actix endpoints don't require public
* Extend validation to more actix types
* Validate all remaining actix path and query properties
* Rephrase range validation messages to clearly describe they're inclusive
* Validate all query params to respond with pretty deserialize errors
* Nicely format JSON payload deserialize error responses
* Improve error reporting for upsert point batches
* Add basic validation test that checks a path, query and payload value
* Add some validation constraints
* Add simple validation error render test
* Update Cargo.lock
---------
Co-authored-by: timvisee <tim+github@visee.me>
* add minimal working is_null filter
* add is_null condition to grpc api (backward compatible)
* add unit tests is_null and is_empty conditions
* add is_null to points.proto file
* add some failing OpenAPI tests
* fix a failing test due to change in collection data
* refactor MultiValue's check for is_null
* fix is_empty condition not picking up "key":[]
* remove duplicate OpenAPI integration test
* reuse same variable in condition checker tests
* update grpc docs
* fix is_null cardinality estimation to match is_empty
* update openapi specs
* remove unused debug statements
* add new test points to original test_collection
* fix failing tests according to newly added points
* add the `"key":[null]` test_case
* Fix incorrect metrics value for cluster commit
* Rewrite metrics logic, don't use registry, write values directly
* Only report REST timings for requests having HTTP 200 response
* Limit metrics reporting of endpoints to whitelist
The whitelist contains a selection of search, recommend and upsert endpoints.
* Add MetricsParam, remove detail level, keep anonymize
* Request metrics in basic API test
* Specify content type for metrics endpoint
* Add OpenAPI test for metrics endpoint, remove from basic API test
This test probes for some strings that must exist in the output
* Add note that metrics endpoint whitelist must be sorted
* feat: add "any" to "match" statements
Adds support for using "any" to match on a list of different possible values
* enable proper counting in cardinality estimation
* fmt
* upd OpenAPI
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* many named vectors per point (segment-level)
* operation result for dim function
* beautifulized vector name
* fix naming bug
* segment version migration
* fmt
* add segment tests
* are you happy clippy
* fix build
* [WIP] many named vectors per point (collection-level) (#975)
* config and search
* fix placeholders for proxy segment move
* remove VectorType from collection
* are you happy fmt
* vectors in grps messages
* create collections with vectors
* segment holder fixes
* are you happy fmt
* remove default vector name placeholders
* are you happy fmt
* are you happy clippy
* fix build
* fix web api
* are you happy clippy
* are you happy fmt
* record vector&vectors
* openapi update
* fix openapi integration tests
* segment builder fix todo
* vector names for update from segment
* remove unwrap
* backward compatibility
* upd openapi
* backward compatible PointStruct
* upd openapi
* fix record back-comp
* fmt
* vector configuration backward compatibility
* fix vetor storage size estimation
* fmt
* multi-vec segment test + index test
* fmt
* api integration tests
* [WIP] Named vectors struct (#1002)
* move to separate file
* named vectors as struct
* use cow
* fix build
* keys iterator
* avoid copy in PointStruct -> get_vectors
* avoid another copy
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* allow additional params for payload field index
* fmt
* wip: full text index building
* fmt
* text search request
* text search request
* full text index persitance and loading
* fmt
* enable fts index in mapping
* clippy
* fix tests + add integration test
* review fixes: extend payload index test
* revert incedental change
* improve alias tests
* introduce new alias persistence and remove Sled
* introduce FileStorageError
* use new error in AliasMapping
* aliases are kept in memory and saved to disk when modified
* better naming
* make alias renaming atomic
* fmt
* use correct alias and docs
* code review: cleaner error conversion
* wip: re-implement http tests using openapi validation library
* collection delete api
* payload operations test
* payload delete operations test
* payload index creation and removing test
* uuid operations test
* schema consistency test
* add tests into pipeline
* chmod +x
* faster local runs with docker cache
* upd instruction
