* Add QuantizationConfigDiff type
* Add quantization config diff to vector parameters
* Prefer vector config over collection config for quantization
* Update OpenAPI specification
* Validate quantization configuration quantile in 0.5-1.0 range
As per https://github.com/qdrant/qdrant/pull/1681
* Add test if check if vector specific quantization config is persisted
* Alias quantization to quantization_config in vector parameters
* Remove quantization config diff, use full vector specific config instead
* Regenerate OpenAPI specification and gRPC docs
* Fix compilation error
* Add error handling to quantization config conversions
* Fix quantization integration test, make HNSW test stricter
* Change file mode of some .sh files to make them executable
* Remove whitespace on empty lines from YAML configurations
* Fix unused import warning on Windows
* Validate VectorConfig/VectorParams, remove obsolete validation
* Add HNSW config diff to vector parameters
* Validate params in collection config
* Add HNSW config to segment vector data config
* Add VectorsConfig params iterator for more elegant conversions
* Prefer vector HNSW config over collection config for building HNSW index
* Base segment vector param HNSW config on collection config
* General improvements
* Rewrite HNSW ef_construct extract function to also consider vector configs
* Update OpenAPI specification
* Add test to check if vector specific HNSW config is persisted
* review changes
* review changes
* Regenerate gRPC docs
* Fix test on Windows
* Regenerate OpenAPI specification
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* Implement validation on generated gRPC point types
* Validate gRPC point API requests
* Nicely describe validation errors
* Add validation extensions for special types used in gRPC
* Validate gRPC collection API requests
* Validate gRPC Raft and snapshot API requests
* Simplify validation builder, guess derives from fields, fix two fields
* Remove point recommending limit constraint to fix gRPC tests
* Clean up validation logic for gRPC in build.rs
* Validate internal gRPC requests, don't error but show warnings in log
* Optimize gRPC validation extensions, remove unused bits
* Validate gRPC collection creation configuration
* Add custom validators for special gRPC types
* Value is automatically attached to validation error
* Cleanup
* Remove obsolete tests
* Add basic gRPC validation test
* Add some gRPC request payload validation tests
* Set max collection name length to 255 in validation to match Actix
* Verify quantization configuration on collection creation
* Update OpenAPI specification
* Update range in doc to match validation definition
* Update OpenAPI specification
* use common channel pool for healthcheck requests
* add dynamic channel pool
* fmt
* lower parallel connections num
* fmt
* fix clippy
* fix clippy
* better status error message
* smarter channel dropping conditions
* per-channel stats
* refactor pool
* rollback dockerfile
* health-check timeout
* prevent reporting last active peer as dead
* introduce update rate limit
* only rate-limit client requests
* ability to save snapshot without locking wal (#1685)
* ability to save snapshot without locking wal
* fix empty wal saving
* skip waiting on local shard in listener mode even if it is a direct request to listener shard
* snapshot recovery test + better handling of segment versions
* use latest wal
* review changes
Refactor `ForwardProxyShard::update` to separate failure of the local shard from the failure of the remote shard.
TODO:
- Figure a way to return the result of the local shard (regardless of status of remote shard)...
- ...and also handle the failure of remote shard
* feat: add wait parm for all snapshot API
* update openAPI
* avoid unwrap
* remove all unwarp
* return HTTP 202 when no waiting on snapshot creation
* return 202 when non wait
* update open API
* update param on python test
* update and test api
* optimize the test
* return 202 when non-wait delete
* recover fixes
---------
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* actix validation
* add check for memmap/indexing_threshold
* fix actix json settings
* Validate settings configuration on start, print pretty warnings on fail
* Add more validation rules for settings and nested types
* Move shared validation logic into collection/operations
* Show validation warning in log when loading some internal configs
* Show prettier actix JSON validation errors
* Stubs for pretty handling of query errors, reformat validation errors
* Use crate flatten function, the hard work was already done for us
We don't have to flatten validation errors into their qualified field
names ourselves because there is a utility function for this.
* Configure actix path validator
* Actix endpoints don't require public
* Extend validation to more actix types
* Validate all remaining actix path and query properties
* Rephrase range validation messages to clearly describe they're inclusive
* Validate all query params to respond with pretty deserialize errors
* Nicely format JSON payload deserialize error responses
* Improve error reporting for upsert point batches
* Add basic validation test that checks a path, query and payload value
* Add some validation constraints
* Add simple validation error render test
* Update Cargo.lock
---------
Co-authored-by: timvisee <tim+github@visee.me>
* add minimal working is_null filter
* add is_null condition to grpc api (backward compatible)
* add unit tests is_null and is_empty conditions
* add is_null to points.proto file
* add some failing OpenAPI tests
* fix a failing test due to change in collection data
* refactor MultiValue's check for is_null
* fix is_empty condition not picking up "key":[]
* remove duplicate OpenAPI integration test
* reuse same variable in condition checker tests
* update grpc docs
* fix is_null cardinality estimation to match is_empty
* update openapi specs
* remove unused debug statements
* add new test points to original test_collection
* fix failing tests according to newly added points
* add the `"key":[null]` test_case
* Raft-rs 0.7.0
* Remove `request_index` field from `ConsensusOperations::RequestSnapshot` message
---------
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
* remove deleted flags from vector storage
* remove deleted flags from mmap
* new simple vector storage format
* are you happy clippy
* remove id_tracker from raw_scorer
* revert vector storage format changes
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Andrey Vasnetsov