Arnaud Gourlay
3a2d640262
Sparse vectors sanity integration checks ( #3126 )
2023-12-03 18:58:19 +01:00
Luis Cossío
444f837c75
make unused import lint happy ( #2667 )
2023-09-15 12:02:28 -03:00
Tim Visée
1e4ce145a2
Parameterize integration tests ( #2375 )
...
* Clean up imports
* Clean up whitespace
* Use drop collection function from helpers
* Use parameterized integration tests to test on_disk variations
* Promote on_disk_vectors parameter into dedicated fixture
* Merge basic retrieve test with parameterization
* Flatten functions
* Split validation test functions
* Don't parameterize on_disk states when testing validation
* Parameterize init_from_collection test sequence
* Parameterize on_disk_payload in tests with fixture
* Parameterize new batch update tests
* Fix integration test collection setup distance after incorrect merge
2023-08-09 12:32:27 +02:00
Luis Cossío
e70a80add7
CI: make integration tests 45% faster ( #1902 )
...
* use scope="module" where possible
* refactor test_snapshot.py to use loop and timeout instead of long waits
2023-05-16 15:23:03 +02:00
Luis Cossío
e90a03e00b
Group by key ( #1768 )
...
* test: test must_not is_null
* vcs: ignore vscode files
* feat: group-by initial implementation
* cargo fmt
* refactor: same request behavior on reco and search
* refactor: get rid of RefCell
* refactor-fix: correct hashmap keys, and early stops
* chore: small improvements
* feat: groups aggregator
* fix: pull changes from other files
* cargo fix
* cargo fmt
* docs: edit docstrings
* allow dead code (while the complete feature is beint built)
* chore: restructure
* feat: introduce GroupKey, minor other improvements
* cargo fmt
* chore: specify aggregator visibility
* fix: oops, leaking "private" type
* refactor-fix: restructure and refactor group_by
* cargo fix
* fix: don't panic when there is no group-by field
* remove print statements
* amend: `>=` -> `==`
* perf: remove double clone
* chore: sync aggregator from other branch
* chore: cleanup print statemets
* test: ignore big tests
* cargo fmt
* refactor: add early stop when the groups have been filled, improve code
* chore: sync aggregator, remove print from test
* refactor: consider shard_selection, improve collection_by_name handling
* feat: add bucketing to table of content
* refactor: better errors, improve tests
* test: add integration tests
* feat: add endpoints
* refactor: introduce ScoredPoint wrapper, restructure types
* sync aggregator
* edit internal grouping visibility
* feat: group_by internals
* cargo fmt
* cargo fmt
* refactor: turn inner fn into closure
* test: fix test to support new vector output representation
* feat: wire up grouping with actix
* expose grouped_by field
* fix: change output group format
* feat: wire up openapi
* fix: finish wiring up grouping in actix
* tests: fix test_group.py
* cargo fmt
* refactor: extract constants
* remove Hash from ScoredPoint
* `Option<collection_by_name>` -> `collection_by_name`
* fix: handle better cases on `match_on`
* fix: consider that subsequent calls can bring better results
* cargo fmt
* fix clippy warnings
* cargo fmt
* refactor: move `Group` to `types`, localize `hydrate_from`, remove `Deref` impls
* refactor `add_points`
* refactor: turn `GroupKey` into enum
* refactor: make `HashablePoint` inner struct private
* feat: add grpc layer, make new `PointGroup` type to use as output
* fix: update openapi models
* docs: update grpc docs
* fix merge errors
* refactor: add BaseGroupRequest to make code DRYer, improve doc comments
* cargo fmt
* perf: increase precision; choose best groups by score
* misc: add more integration tests, fix review comments
* cargo fmt
* fix: reimplement interface to flatten search and recommend requests, excluding offset
* cargo fmt
* refactor: move `r#do` impl to `GroupRequest`
* fix: update grpc docs
* perf: sort in reverse order
* fix: use fist value of a Value::Array
* fix: validate group_by to not support bracket notation, fix int. tests
* fix: update grpc validation
* tests: update collection_tests
* refactor: move validation to the api layers
* Oops: reupdate tests
* refactor: let the derives derive (thanks @ffuugoo)
* refactor: use a new GroupId on the output
also increases performance by copying less
* remove hashable set, take ordering into an account, fix mutliple groups values support
* fmt
* refactor group_id + rename per_group -> group_size, fix clippy
* remove GroupKey wrapper
* @agourlay review fixes
* refactor: `group_min_scores` and `group_max_scores` -> `group_best_scores`
* refactor: use set difference on `keys_of_unfilled_best_groups`
* refactor: use set intersection on `len_of_filled_best_groups`
* refactor: turn best_group_keys into iterator
* fix: remove [] syntax limitation
* fix: update openapi.json
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com >
2023-05-15 23:05:20 +02:00
Luis Cossío
7af32ec2b7
feat: [http] alias with_vector <-> with_vectors ( #1697 )
2023-04-11 18:19:40 +02:00
Andrey Vasnetsov
b73335259f
Restore is empty functions as before ( #1657 )
...
* is_empty should match with value=null
* fmt
2023-04-04 12:17:07 +02:00
Ibrahim M. Akrab
f85ae02ab6
add isNull condition for payload filtering ( #1617 )
...
* add minimal working is_null filter
* add is_null condition to grpc api (backward compatible)
* add unit tests is_null and is_empty conditions
* add is_null to points.proto file
* add some failing OpenAPI tests
* fix a failing test due to change in collection data
* refactor MultiValue's check for is_null
* fix is_empty condition not picking up "key":[]
* remove duplicate OpenAPI integration test
* reuse same variable in condition checker tests
* update grpc docs
* fix is_null cardinality estimation to match is_empty
* update openapi specs
* remove unused debug statements
* add new test points to original test_collection
* fix failing tests according to newly added points
* add the `"key":[null]` test_case
2023-04-01 20:46:50 +02:00
Andrey Vasnetsov
83c21450c3
Pagination ( #743 )
...
* offset param for search and recomendation
* test and fix for pagination
* clippy + fmt
* upd schemas
* fix grpc requests in test
2022-06-27 13:22:02 +02:00
Andrey Vasnetsov
1b458780eb
On disk payload storage ( #634 )
...
* implement on-disk payload storage
* fmt + clippy
* config param for on-disk payload storage
* upd openapi definitions
* add integration test with on-disk payload
* fix clippy
* review fixes
* fmt
2022-06-01 17:23:34 +02:00
Andrey Vasnetsov
5800319edb
refactor distances + add score_threshold + fix negative euclid distance ( #569 )
...
* refactor distances + add score_threshold + fix negative euclid distance
* generate docs
* fix clippy
* Update lib/segment/src/spaces/tools.rs
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com >
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com >
2022-05-09 17:32:48 +02:00
Andrey Vasnetsov
6af867e037
fix and extend interation test for scrolling over points ( #494 )
2022-04-20 09:52:59 +02:00
Andrey Vasnetsov
b07428f620
Is empty condition ( #423 )
...
* is-empty condition
* fmt
* better assert
* fmt
2022-04-03 16:55:51 +02:00
Andrey Vasnetsov
adc1f4ad97
Bool filter ( #421 )
...
* bool match condition
* use generic values for match requests
* fmt
* upd grpc interface
* upd grpc docs
2022-04-03 16:08:34 +02:00
Andrey Vasnetsov
20c1e0f20d
upd naming in api ( #361 )
2022-03-03 13:27:51 +01:00
Andrey Vasnetsov
c3cfc93574
fix PayloadSelector enum for better openapi compatibility ( #358 )
2022-03-02 15:01:27 +01:00
Andrey Vasnetsov
bdc3194b13
allow to return vectors and payload with recommendation request #353 ( #357 )
...
* allow to return vectors and payload with recommendation request #353
* fmt
* unify `with_vector` comment line
* upd openapi
2022-03-02 10:42:58 +01:00
Andrey Vasnetsov
f5feff8080
Validated integration tests - #232
...
* wip: re-implement http tests using openapi validation library
* collection delete api
* payload operations test
* payload delete operations test
* payload index creation and removing test
* uuid operations test
* schema consistency test
* add tests into pipeline
* chmod +x
* faster local runs with docker cache
* upd instruction
2022-02-11 16:43:47 +01:00
