* universal-query: Add validations to query input
* update openapi spec
* use raw request to by-pass local validations
* add validation for order_by and factorize
* minor conversion improvement
* use NamedVectors in update_vectors
* remove merge from VectorStruct
* rename Multi -> Named in vector struct
* add multi-dense vectors option into VectorStruct
* generate openapi
* rename VectorStruct -> VectorStructInternal
* add conversion for anonymous multivec in grpc
* renames for BatchVectorStruct
* implement multi-dense for batch
* allow multi-dense in batch upserts
* test and fixes
* include old vector into update function and clean posting lists accordingly
* add integration test
* fix counter
* also remove old vector if the insertion is empty
* clippy
* borrow once
* fix max_next_weight correcton on delete + test
* vector index responsible for updating vector storage
* review fixes
* add debug assert
* Allow multivec search on legacy REST API
* show that it works for gRPC as well
* better error message
* update error assertion
* show validation on REST as well
* remove unecessary test
* fix conversion - dim is not vec count
* fmt
* Use TypedMultiDenseVectorRef everywhere (#4224)
* Use TypedMultiDenseVectorRef everywhere
* remove obsolete test
* fix codespell
* fix build
* test single dense vector expansion on upsert
---------
Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>
* REST API multivector for write and retrieve
* add test showing search is not supported
* use helper
* debug asserts
* better naming
* iterator style
* better naming
* add validation test for checking all inner vectors have the same size
* introduce QueryContext, which accumulates runtime info needed for executing search
* fmt
* propagate query context into segment internals
* [WIP] prepare idf stats for search query context
* Split SparseVector and RemmapedSparseVector to guarantee we will not mix them up on the type level
* implement filling of the query context with IDF statistics
* implement re-weighting of the sparse query with idf
* fmt
* update idf param only if explicitly specified (more consistent with diff param update
* replace idf bool with modifier enum, improve further extensibility
* test and fixes
* Update lib/collection/src/operations/types.rs
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
* review fixes
* fmt
---------
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
* Fix incorrect snapshot security test path
* Explicitly check if snapshot files exist
* Change shard snapshot invalid name error type to not found
That makes it consistent with behavior of our other snapshot APIs.
* Extend snapshot API security tests, also test collection and shard API
* Improve function to get collection snapshot path
* Fix arbitrary path traversal in full snapshot path function
* Add integration test to assert arbitrary path traversal is blocked
* Support set by key in low level.
* Rename key field.
* Format.
* Pass key.
* Format.
* Test.
* Clippy.
* Fix ci lint.
* Check grpc consistency.
* Update openapi.
* Fix empty key test case.
* Support array index.
* Format.
* Add test for non exists key.
* Clippy fix.
* Add idempotence test.
* Update index by updated payload.
* Add ut for utils.
* Add ut for 1 level key.
* Fix ut.
* Support no exits key.
* Fix test result.
* Fix after rebase
* handle wildcart insertion into non-existing array
* avoid double read of payload during update
* fix missing removing data from index in case if set_payload removes indexed field
---------
Co-authored-by: Shylock Hg <shylock@DESKTOP-40I855A>
Co-authored-by: Albert Safin <xzfcpw@gmail.com>
Co-authored-by: generall <andrey@vasnetsov.com>
* fix default opeapi file path
* better capture of process id in integration-tests.sh
* add pycache to gitignore
* add extra case for order_by test
* parse dates without timezone but with decimals correctly
* update payload_indexing test
* add extra test for space-separated date
