* Add github action to codespell master on push and PRs
* Add rudimentary codespell config
* some skips
* fix some ambigous typos
* [DATALAD RUNCMD] run codespell throughout
=== Do not change lines below ===
{
"chain": [],
"cmd": "codespell -w",
"exit": 0,
"extra_inputs": [],
"inputs": [],
"outputs": [],
"pwd": "."
}
^^^ Do not change lines above ^^^
* Add dev branch as target for the workflow
* rename `Lookup` -> `RetrievedLookup`
* feat: expose lookup in groups
- make `with_vectors` and `with_payload` optional
- include in grpc
- move GroupId to inside of PointGroup message
- include `WithLookup` inside of `GroupRequest`, doesn't make sense to keep them separated
* chore: remove allow(dead_code) from builder
* feat: introduce `WithLookupInterface` to allow specifying only the name of the lookup collection
* fix: WithLookupInterface with collection name only brings all payload and vectors
* fix: update grpc docs
* add tests and set better defaults
* remove `RetrievedLookup` from proto
* short-circuit `fill_search_result_with_payload` when it's not needed
* cargo fmt
* proto: revert putting `GroupId` inside `PointGroup`
* update grpc docs
* Remove WithLookupInterface sugar from grpc
- set with_vectors of lookup default to false
- set with_payload of lookup default to true
* update integration tests
* use singular lookup
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Dependabot couldn't find the original pull request head commit, 7f8b1be2d712f4552c8b4cd2377149c8971b1516.
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* one more test for optional vectors
* remove check
* clippy
* Add test retrieving non existing vector name
---------
Co-authored-by: timvisee <tim@visee.me>
* Flatten points selector in delete vectors REST endpoint
This makes the vector deletion API consistent with other endpoints using
the point selector.
* Update tests for flattened delete vectors point selector
* make api consistent with existing ones
* update openapi
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* nested object filter
* code review
* add support for must_not in nested
* extract functions
* support and test must_not in SimpleConditionChecker
* add index matching unit test (to be continued)
* remove extra clone
* test with should
* WIP: Nested object filter suggestions (#1855)
* switch to bitvec
* fix clippy
* more tests
* fmt
* fix some tests
* add test with text
* support for nested should
* do not rely on indexes for nested queries & fix test
* use index to make index-aware checks in nested payload
* fix value-count tests
* re-fa-cto-ring
* fmt
---------
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* Add integration test for optional named vector update/delete endpoints
* Update optional named vector integration tests for API change
* Fix update vectors endpoint not validating inner vectors
* Fix failing test due to unspecified order
* Add update/delete REST endpoints
* Add update/delete vector REST endpoints to OpenAPI specification
* Fix update vector with no changes not triggering a validation error
* Update OpenAPI specification for updated request type
* feat: use 0 as disable for indexing and memmap
* amend: edit test, update doc comments, remove validation
* fix: use constant instead of literal, reword doc comments
* test: update test for loop and timeout
* fix: remove validation on OptimizerConfigDiff and fix test_validatioin.py
* fix: remove validation from build.rs, remove internal hack from doccomments
* fix: update grpc docs
* revert max_optimization_threads validation in test
* update config comments and minor test change
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* Add QuantizationConfigDiff type
* Add quantization config diff to vector parameters
* Prefer vector config over collection config for quantization
* Update OpenAPI specification
* Validate quantization configuration quantile in 0.5-1.0 range
As per https://github.com/qdrant/qdrant/pull/1681
* Add test if check if vector specific quantization config is persisted
* Alias quantization to quantization_config in vector parameters
* Remove quantization config diff, use full vector specific config instead
* Regenerate OpenAPI specification and gRPC docs
* Fix compilation error
* Add error handling to quantization config conversions
* Fix quantization integration test, make HNSW test stricter
* Validate VectorConfig/VectorParams, remove obsolete validation
* Add HNSW config diff to vector parameters
* Validate params in collection config
* Add HNSW config to segment vector data config
* Add VectorsConfig params iterator for more elegant conversions
* Prefer vector HNSW config over collection config for building HNSW index
* Base segment vector param HNSW config on collection config
* General improvements
* Rewrite HNSW ef_construct extract function to also consider vector configs
* Update OpenAPI specification
* Add test to check if vector specific HNSW config is persisted
* review changes
* review changes
* Regenerate gRPC docs
* Fix test on Windows
* Regenerate OpenAPI specification
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* feat: add wait parm for all snapshot API
* update openAPI
* avoid unwrap
* remove all unwarp
* return HTTP 202 when no waiting on snapshot creation
* return 202 when non wait
* update open API
* update param on python test
* update and test api
* optimize the test
* return 202 when non-wait delete
* recover fixes
---------
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* actix validation
* add check for memmap/indexing_threshold
* fix actix json settings
* Validate settings configuration on start, print pretty warnings on fail
* Add more validation rules for settings and nested types
* Move shared validation logic into collection/operations
* Show validation warning in log when loading some internal configs
* Show prettier actix JSON validation errors
* Stubs for pretty handling of query errors, reformat validation errors
* Use crate flatten function, the hard work was already done for us
We don't have to flatten validation errors into their qualified field
names ourselves because there is a utility function for this.
* Configure actix path validator
* Actix endpoints don't require public
* Extend validation to more actix types
* Validate all remaining actix path and query properties
* Rephrase range validation messages to clearly describe they're inclusive
* Validate all query params to respond with pretty deserialize errors
* Nicely format JSON payload deserialize error responses
* Improve error reporting for upsert point batches
* Add basic validation test that checks a path, query and payload value
* Add some validation constraints
* Add simple validation error render test
* Update Cargo.lock
---------
Co-authored-by: timvisee <tim+github@visee.me>
* add minimal working is_null filter
* add is_null condition to grpc api (backward compatible)
* add unit tests is_null and is_empty conditions
* add is_null to points.proto file
* add some failing OpenAPI tests
* fix a failing test due to change in collection data
* refactor MultiValue's check for is_null
* fix is_empty condition not picking up "key":[]
* remove duplicate OpenAPI integration test
* reuse same variable in condition checker tests
* update grpc docs
* fix is_null cardinality estimation to match is_empty
* update openapi specs
* remove unused debug statements
* add new test points to original test_collection
* fix failing tests according to newly added points
* add the `"key":[null]` test_case
* Fix incorrect metrics value for cluster commit
* Rewrite metrics logic, don't use registry, write values directly
* Only report REST timings for requests having HTTP 200 response
* Limit metrics reporting of endpoints to whitelist
The whitelist contains a selection of search, recommend and upsert endpoints.
* Add MetricsParam, remove detail level, keep anonymize
* Request metrics in basic API test
* Specify content type for metrics endpoint
* Add OpenAPI test for metrics endpoint, remove from basic API test
This test probes for some strings that must exist in the output
* Add note that metrics endpoint whitelist must be sorted
* feat: add "any" to "match" statements
Adds support for using "any" to match on a list of different possible values
* enable proper counting in cardinality estimation
* fmt
* upd OpenAPI
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
David Sertic