Commit Graph

19 Commits

Author SHA1 Message Date
Luis Cossío
f62119c670 universal-query: Basic integration tests (#4409)
Adds basic comparison to current existing endpoints
2024-06-10 19:50:33 -04:00
Arnaud Gourlay
86ca51aa2d Allow basic multivec search on legacy API (#4203)
* Allow multivec search on legacy REST API

* show that it works for gRPC as well

* better error message

* update error assertion

* show validation on REST as well

* remove unecessary test

* fix conversion - dim is not vec count

* fmt

* Use TypedMultiDenseVectorRef everywhere (#4224)

* Use TypedMultiDenseVectorRef everywhere

* remove obsolete test

* fix codespell

* fix build

* test single dense vector expansion on upsert

---------

Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>
2024-05-14 17:54:11 +02:00
AMIR
0ee46a5c1c Simplify REST API nesting (#3323) (#3414)
* Simplify REST api nesting (#3323)

* tiny clarity refactor for test

* simpler generated openapi

* fix tests

* wrap deserializer functions into a struct impl

* fix null case

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
2024-05-13 16:16:35 -04:00
Luis Cossío
71159e5cdd setup for faster openapi snapshot tests (#4219) 2024-05-10 20:26:24 +02:00
Arnaud Gourlay
802c3ebbe5 Test Multivector storage mmap and uint8 (#4197) 2024-05-08 11:57:34 +02:00
Arnaud Gourlay
571143ae87 Simplify MaxSim configuration (#4171)
* Simplify MaxSim configuration

* enable extension of multivectorconfig

* rename multi_vec_config to multivec_config
2024-05-06 14:19:42 +02:00
Arnaud Gourlay
4f797c1bb4 REST API multivector for write and retrieve (#4117)
* REST API multivector for write and retrieve

* add test showing search is not supported

* use helper

* debug asserts

* better naming

* iterator style

* better naming

* add validation test for checking all inner vectors have the same size
2024-05-03 10:55:17 +02:00
Andrey Vasnetsov
c173a9f5e5 Sparse idf dot (#4126)
* introduce QueryContext, which accumulates runtime info needed for executing search

* fmt

* propagate query context into segment internals

* [WIP] prepare idf stats for search query context

* Split SparseVector and RemmapedSparseVector to guarantee we will not mix them up on the type level

* implement filling of the query context with IDF statistics

* implement re-weighting of the sparse query with idf

* fmt

* update idf param only if explicitly specified (more consistent with diff param update

* replace idf bool with modifier enum, improve further extensibility

* test and fixes

* Update lib/collection/src/operations/types.rs

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>

* review fixes

* fmt

---------

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
2024-04-29 14:54:14 +02:00
Andrey Vasnetsov
20eb2e1f2f remove usage of vectors_count (#4052)
* remove usage of vectors_count

* skip serialization if none

* add deprecated mention

* Add colon after deprecation note

---------

Co-authored-by: timvisee <tim@visee.me>
2024-04-18 15:22:01 +02:00
Tim Visée
a18859865c Extend shard snapshot security tests (#3867)
* Fix incorrect snapshot security test path

* Explicitly check if snapshot files exist

* Change shard snapshot invalid name error type to not found

That makes it consistent with behavior of our other snapshot APIs.

* Extend snapshot API security tests, also test collection and shard API
2024-03-19 13:28:34 +01:00
Tim Visée
78f132080b Fix arbitrary path traversal vulnerability in full snapshot REST API (#3856)
* Improve function to get collection snapshot path

* Fix arbitrary path traversal in full snapshot path function

* Add integration test to assert arbitrary path traversal is blocked
2024-03-18 20:51:17 +01:00
xzfc
5be063bbc1 Allow paths starting with numbers (#3788)
* Allow paths starting with numbers

* add test

* fix test

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2024-03-07 14:33:45 +01:00
xzfc
28a65f4786 Fix batch_requests (#3745) 2024-03-04 10:52:55 +00:00
xzfc
8d8f8346d3 Switch to JsonPathV2 (#3709) 2024-02-29 14:55:28 +01:00
Tim Visée
35a59eb14d Fix flaky invalid URI snapshot test (#3704)
* Give file system some time clean up temporary snapshot files

* Be more careful with test_snapshot_operations_non_wait test

* Remove sleep
2024-02-28 12:29:29 +01:00
xzfc
01e8322e5d Test payload index overwrite (#3655) 2024-02-23 10:44:15 +00:00
shylock
eff1fd821e Feat/set payload by key (#3548)
* Support set by key in low level.

* Rename key field.

* Format.

* Pass key.

* Format.

* Test.

* Clippy.

* Fix ci lint.

* Check grpc consistency.

* Update openapi.

* Fix empty key test case.

* Support array index.

* Format.

* Add test for non exists key.

* Clippy fix.

* Add idempotence test.

* Update index by updated payload.

* Add ut for utils.

* Add ut for 1 level key.

* Fix ut.

* Support no exits key.

* Fix test result.

* Fix after rebase

* handle wildcart insertion into non-existing array

* avoid double read of payload during update

* fix missing removing data from index in case if set_payload removes indexed field

---------

Co-authored-by: Shylock Hg <shylock@DESKTOP-40I855A>
Co-authored-by: Albert Safin <xzfcpw@gmail.com>
Co-authored-by: generall <andrey@vasnetsov.com>
2024-02-15 15:15:05 +01:00
Luis Cossío
8614b7ea92 Datetime parser: Parse decimals without timezone (#3622)
* fix default opeapi file path

* better capture of process id in integration-tests.sh

* add pycache to gitignore

* add extra case for order_by test

* parse dates without timezone but with decimals correctly

* update payload_indexing test

* add extra test for space-separated date
2024-02-14 18:35:15 -03:00
Kumar Shivendu
50fc2288d0 refactor: OpenAPI tests (#3606)
* test: Restructure OpenAPI tests

* fix: Remove cache dirs

* fix: Remove cache dirs from helpers

* fix: Path of openapi.json

* fix: Update path of openapi tests everywhere
2024-02-14 22:19:26 +05:45