* Strict mode allows fullscan for multitenant payload index
* different error for missing payload index in multitenant case
* handle payload_m
* add simple test
* handle hnsw.m not set
* add another simple test
* fix test
* fallback to global HSNW config
* new error status code
* do not block on global HNSW and improver error reporting
* clearer error reporting
* review fixes
* fix test error messages
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* Fix UnindexedField infra to handle lookup&range index requirements
* more generic approach
* better test
* restore
* small unit test for sanity
* review nits
* Do not recommend parametrized index
---------
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
* consider length of multivectors for rate limiter
* add openapi test
* organize imports
* better cost estimation for sparse vectors
* restore groups openapi test
* address @timvisee's review
* Fix consensus test
* Improve telemetry logic and test
* Parametrize telemetry test
* Consistency hash peeer ID across telemetry
* clean test
* Use Option in segments telemetry
* updat openapi spec
* Avoid test failure on change in order of params
* don't propagate offset into prefetches
* I want to see CI with that change
* Move import into platform specific scope
* edit test to make sure offset is not propagated
* fix planned query test
---------
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
Co-authored-by: timvisee <tim@visee.me>
* move problems.rs to collection crate
* change type on datetime_key expression
* extract from expression
* check for unindexed fields in formula
* clippy
* dedup code
* Only implement verification for `CollectionQuery` types (#6319)
* verify CollectionQuery's Prefetch
* remove implementation for REST query
* remove implementation for REST query groups
* add basic openapi test
* create initial strucutres
* clippy
* start field-query refactoring
* start field-query refactoring (2/N)
* start field-query refactoring (3/N): duplicate is_empty/null condiftions as field condition
* start field-query refactoring (4/N): re-instate is_empty fallback in case new index is not built yet
* filter for is_empty/is_null
* implement add/remove point
* upd schema
* open and create of null-index
* create null-index
* fix test
* Update lib/segment/src/index/query_optimization/condition_converter.rs
Co-authored-by: Tim Visée <tim+github@visee.me>
* unit test for null-index
* more unit tests
* add openapi tests
* fmt
* fix for integartion tests
* rabbit review fix
* make [null] non-empty
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
* Retry-After on rate limiter hit for REST API
* fix tests
* Ceil retry after seconds number so clients don't retry too early
* Relax retry after time requirement a bit in case CI machines are slow
* Add rate limit test, test retry after if we replanish within a second
* Report time to wait in rate limit error
---------
Co-authored-by: timvisee <tim@visee.me>
* Add limits for filter and conditions
* clippy
* Review remarks + nested condition test
* Fix opnapi specs
* Improve error message by giving info about limits and usage
* Strict mode config: Max collection size
* api specs
* Add tests + set/update payload check
* Improve function names and add comments
* rename config to separate vectors and payload
* fix tests
* Adjust configs docs
* add benchmark
* improve performance by caching shard info
* add bench for size_info() and fix tests
* Also limit the batch-size for vector updates (#5508)
* Also limit the batch-size for vector updates
* clippy
* add lost commit
* Load cache on collection initialization
* add unit type to parameter name
* fix renaming in test
* clearer error message
* fix test
* review remarks
* remove unused function for now
---------
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
* Limit maximum preallocation on search results aggregator hash set
Prevents an allocation failure and panic if a user specifies a huge
search limit.
* Prevent overflow on fixed length priority queue limit with u64::MAX
* Fix overflow with high limit in search API
* Add two basic tests covering high search limit
Collection name under test is equal to the test module name, without `.py` suffix.
* Helps by debugging/tracing failed tests and find relevant logs lines in qdrant log files
* Opens up a possibility to run tests in parallel, given that there are no data sharing
between test modules
Change details:
* defined module scoped `collection_name` fixture in `conftest.py`
* removed `collection_name` module variable
* each test signature modified to declare the dependency to `collection_name` fixture
* `@pytest.mark.parametrize` migrated to `@pytest-cases.parametrize` in cases when
`collection_name` was used as the value
Although test logic passes the assertions, the test might fail
if during the teardown the created collection is not deleted,
because the client could not reach the database due to networking issue.
In order to make test executions more robust, collection deletes
are retried in case of networking issues.
* Fix search and delete using subslice index, rather than container intex
* Add sanity checks in debug assertions
* test
---------
Co-authored-by: generall <andrey@vasnetsov.com>
If `wait=true` param is not set for delete vector API calls, a test asserting later
that data are gonna can become flaky, because at the assertion time the data could be
still there.
The following files contain such tests at the moment:
* `test_multi_vector_uint8.py`
* `test_multi_vector.py`
* `test_multi_vector_unnamed.py`
* `test_optional_vectors.py`
In order to fix these and prevent the flakiness of future test,
`request_with_validation` helper add `wait=true` param, if it is not set
for delete vector calls.
* `QDRANT_HOST` variable is used by all openapi tests, removed its copies from
`test_multi_vector_uint8.py`, `test_multi_vector_unnamed.py`, `test_query.py`,
`test_shard_snapshot.py`, and `test_snapshot.py`
* Added optional `QDRANT_HOST_HEADERS` env variable to set custom HTTP headers
in order to reach Qdrant instance behind a reverse proxy. The content of the
variable is JSON payload, e.g. `{"host": "qdrant.local"}`
* Adapted `./tests/basic_*.sh` scripts to aware of `QDRANT_HOST_HEADERS` env variable
