* Add progress_tracker.rs
* Pass progress tracker around
* Populate progress tracker with actual data
* Expose progress on `/collections/{name}/optimizations` endpoint
* Remove deprecated vectors count from collection info
* Remove vectors count from shard info
* Update OpenAPI and gRPC spec
* Remove vectors count from example
* Deprecate init_from
* Mark memmap_threshold as deprecated
* Mark locks API as deprecated
* Mark RBAC collection access payload filter as deprecated
* Allow deprecations in tests and benchmarks
* add HwMeasurementAcc and API helpers+types
* add config to API to toggle hardware reporting in API responses
* add Default impl for HwMeasurementAcc
* comments
* review remarks
* include usage into API schema
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* rename to FacetRequestInternal
* add rest endpoint
* fix correctness by fetching the whole list of values
* fix mmap map index variant
Also removes test for sorted output, for now
* add ytt spec
* fix clippy
* use hashmap inside of local shard
* rename operation to `facet`, add access test
* whitelist endpoint
* change api
* make limit optional
* Refactor: Convert app and cluster info metrics from counters to gauges
- Change app_info, app_status_recovery_mode, and cluster_enabled metrics from COUNTER to GAUGE
- Update metric values to use boolean as f64 for consistency and clarity
- Improve metric type accuracy for better representation of state information
* fix: Update docs regarding metrics type
* openapi: Update app_info metric type
* metrics: Update test to check for the right metric type
* example for CollectionsResponse
* points_example for ScrollResult
* example for SnapshotDescription
* fix
* example for AliasDescription
* fix: Style nit: add newline
* fix time and id
* fix order_value
* upd openapi
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* universal-query: Grouping REST API
* add API docs
* bump API count
* add jwt validation test
* fix API
* better test
* stay on CoreSearchRequest where possible
* use existing scoring helper
* push broken test to illustrate issue
* track best score per point_id to enable sort on payload
* add example test for query discover with groups
* track ScoredPoint instead of PointId
* add default values for smoother ux
* fix default opeapi file path
* better capture of process id in integration-tests.sh
* add pycache to gitignore
* add extra case for order_by test
* parse dates without timezone but with decimals correctly
* update payload_indexing test
* add extra test for space-separated date
* Add min_should field in Filter struct
* min_should clause checks whether at least given number (min_count) of conditions are met
* modify test cases due to change in Filter struct (set min_should: None)
* add simple condition check unit test
* docs, cardinality estimation, grpc not implemented yet
* Add min_should field in Filter struct
* min_should clause checks whether at least given number (min_count) of conditions are met
* modify test cases due to change in Filter struct (set min_should: None)
* add simple condition check unit test
* Impl min_should clause in REST API
* perform cardinality estimation by estimating cardinalities of intersection and combining as union
* add openapi spec with docs update
* add integration test
* Impl min_should clause in gRPC
* Cargo fmt & clippy
* Fix minor comments
* add equivalence test between min_should and must
* shortcut at min_count matches
* use `Filter::new_*` whenever possible
* Add missing min_should field
* Fix gRPC field ordering & remove deny_unknown_fields
* Empty commit
---------
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
* support \`start_from\`: DateTime
* add order by datetime test
* generate openapi models and grpc docs
* fixup after rebase
* allow string representation of datetime in grpc
* add TODO
* fix `.start_from()`
* use custom deserialization on datetime
* Extend datetime formats in REST API (#3529)
* Fix linter problem
* Support indexing and condition checking and optimize tests
* Update lib/segment/src/types.rs
Co-authored-by: Tim Visée <tim+github@visee.me>
* Update lib/segment/src/types.rs
Co-authored-by: Tim Visée <tim+github@visee.me>
* Update lib/segment/src/types.rs
Co-authored-by: Tim Visée <tim+github@visee.me>
* Add some more tests for wrong or unsupported datetime format
* Reformat
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
Co-authored-by: timvisee <tim@visee.me>
* merge range and datetime range into one interface
* remove separate datetime_range from FieldCondition
* review fixes, remove code duplication. Thanks @xzfc
* update openapi test
* first PR implementation (#2865)
- fetch offset id
- restructure tests
- only let order_by with numeric
- introduce order_by interface
cargo fmt
update openapi
calculate range to fetch using offset + limit, do some cleanup
enable index validation, fix test
Fix pagination
add e2e tests
make test a little more strict
select numeric index on read_ordered_filtered
add filtering test 🫨
fix filtering on order-by
fix pip requirements
add grpc interface, make read_ordered_filtered fallible
fmt
small optimization of `with_payload` and `with_vector`
refactor common logic of point_ops and local_shard_operations
Make filtering test harder and fix limit for worst case
update openapi
small clarity refactor
avoid extra allocation when sorting with offset
stream from numeric index btree instead of calculating range
use payload to store order-by value, instead of modifying Record interface
various fixes:
- fix ordering at collection level, when merging shard results
- fix offset at segment level, to take into account also value offset
- make rust tests pass
remove unused histogram changes
fix error messages and make has_range_index exhaustive
remove unused From impl
Move OrderBy and Direction to segment::data_types::order_by
Refactor normal scroll_by in local_shard_operations.rs
More cleanup + rename OrderableRead to StreamWithValue
empty commit
optimization for merging results from shards and segments
fix case of multi-valued fields
fix IntegerIndexParams name after rebase
precompute offset key
use extracted `read_by_id_stream`
Expose value_offset to user
- rename offset -> value_offset
- extract offset value fetching logic
* remove offset functionality when using order_by
* include order_by in ForwardProxyShard
* extra nits
* remove histogram changes
* more nits
* self review
* resolve conflicts after rebase, not enable order-by with datetime index schema
* make grpc start_from value extendable
* gen grpc docs
---------
Co-authored-by: kwkr <kawka.maciej.93@gmail.com>
Co-authored-by: generall <andrey@vasnetsov.com>
xzfc