This commit introduces the GeoPolygon struct, representing a polygon defined by a list of coordinates. It includes the check_point method to determine if a given point is inside the polygon. Test cases validate the functionality for different scenarios.
* Issue 1905: Configurable location for the tmp snapshot files
* Apply suggestions from code review
Co-authored-by: Tim Visée <tim+github@visee.me>
* fix code review suggestions
* clippy fix
* Propagate temp path, use configured dir for snapshot creation
* Use real temp dir in snapshot tests
* Mention default temporary snapshot file path in configuration
* Use temp everywhere rather than a mix of temp and tmp
* Use consistent naming for temporary snapshot directories
* Extract logic for temporary storage path into toc method
* Resolve clippy warnings
* Apply suggestions from code review
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
* Before inserting vector update, check for correct dimensions
* Improve vector name and data checking on segment level
* Optimize vector compatibility checking
* Remove obsolete vector dimensionality check
* Add test to ensure all segment functions catch bad input
* Test vector checking functions directly as well
* wip: add rest param
* WIP: Refactor `search_with_graph` to use oversampling
* fixup! WIP: Refactor `search_with_graph` to use oversampling
Fix `search_with_graph` quantization config handling
* fixup! WIP: Refactor `search_with_graph` to use oversampling
Use `truncate` instead of `shrink_to` 🤦♀️
* schrink -> truncate
* protection against wrong oversampling value
* Revert second "fixup! WIP: Refactor `search_with_graph` to use oversampling"
Accidentally commited the wrong file 🤦♀️
* fix max method notation
* rollbacK: max notation
* Add `oversampling` field to `QuantizationSearchParams` gRPC type
* Simplify `oversampled_top` calculation
* `./tools/generate_grpc_docs.sh`
* `./tools/generate_openapi_models.sh`
---------
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
* Correctly count vectors in segment info for normal segment
* Correctly count vectors in segment info for proxy segment
* Simplify available point count method
* Minor improvements
* Add unit test for point and vector counts in segment
* Add unit test for point and vector counts in proxy segment
* Improve vector counting for proxy segment
* async raw scorer
* fmt
* Disable `async_raw_scorer` on non-Linux platforms
* Refactor `async_raw_scorer.rs`
* Conditionally enable `async_raw_scorer` in `segment` crate
* Add `async_scorer` config parameter to the config...
...and enable `async_raw_scorer`, if config parameter is set to `true`
* fixup! Add `async_scorer` config parameter to the config...
Fix tests
* Add basic `async_raw_scorer` test
* Extend `async_raw_scorer` tests to be more extensive
* Async uring vector storage io uring (#2041)
* replace tokio-uring with just low level io-uring
* fnt
* minor fixes
* add sync
* wip: try to use less submissions
* fmt
* fix uring size
* larger buffer
* check for overflow
* submit with re-try
* mmap owns uring context
* large disk parallelism
* rollbacK: large disk parallelism
* fix windows build
* explicitly panic on uring fail
* fix windows build again
* use async scorer in the quantization re-scoring
* refactor
* rename UringReader
* refactor buffers
* fix for windows
* error checking
* fix handing
---------
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
* Skip applied WAL operations on shard load
* fixup! Skip applied WAL operations on shard load
- Skip WAL operation up to the *minimal* persisted version
- Add explanation comment
* fixup! Skip applied WAL operations on shard load
- "beautify" `skip_while` expression
- handle version `0` corner case (and add explanation)
* Update `Segment::handle_version` comment
* Simplify `Segment::handle_version` a bit
* Refactor `check_unprocessed_points`
* fixup! Refactor `check_unprocessed_points`
Optimized `check_unprocessed_points`
* Revert `LocalShard::load_from_wal` and `check_unprocessed_points` changes
* Downgrade failure in `check_unprocessed_points` to a warning
* Persist/track the first un-truncated index in `SerdeWal`...
...and use it as first index for WAL operations
* fixup! Downgrade failure in `check_unprocessed_points` to a warning
Downgrade failure in `check_unprocessed_points` to a warning *during `LocalShard::load_from_wal`* 🙈
* fixup! Persist/track the first un-truncated index in `SerdeWal`...
Words...
* Fix a bug in `SerdeWal::truncated_prefix_entries_num` and add explanation comment
* Fix a bug in `SerdeWal::ack`
* prevent unnesessary savings, use json, never decrease ack index
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* feat: add lookup_ids function
- rename GroupId -> PseudoId
- impl try_from PseudoId to PointIdType
- create lookup_ids function
- add tests
* fix: keep `GroupId` name in the openapi schema
* refactor: make new type for PseudoId
* refactor: make new `Lookup` output for lookup_ids()
* changes from review, thanks @agourlay
* Clone current segment config to deprecated type
* Remove segment level quantization config from segment config
* Also deprecate current VectorDataConfig
* Update old segment migration to work with new refactoring
* Move index into vector data config
* Move vector data config migration logic into segment level
* Remove hnsw_config from vector data config
* Rename collection params to vector data conversions function
* Move storage type into vector data config
* Set appendable flag correctly
* Clean up and reformat
* Make segment on disk flag not optional
* Add appendable flag to segment config to replace storage type
* Remove storage type from segment config
* Deprecate storage type enum
* Use consistent variable naming
* Cleanup
* Add segment config migration for v0.5.0 to current
* Bump segment to 0.6.0
* Remove serde defaults for new storage and vector data config types
These default value configurations are not needed anymore, because these
structs are not used to deserialize old data. All current fields should
always be available in these structs. When new fields are added in new
functions, the serde default annotation must be set again.
* Cleanup
* Update OpenAPI specification
This updates the returned data structure on telemetry endpoints, as a
result of segment configuration refactoring.
* Fix quantization configuration not falling back to collection config
* Fix compiler warning when building in release mode
* Move deprecated type structs into compat module
* Update allow deprecated attributes
* Assign quantization config only in segment optimizer
* Remove unsued parameter
* Add vector storage type enum to vector data config
* Remove appendable and on_disk flags from segment and vector config
* Update OpenAPI specification
* add tests
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
