* wip: generalization trait for queries
* implement generalization for point operations
* fmt
* log priority queue
* wip: SlowRequestsListener
* fmt
* fix clippy
* simplify generalization
* fmt
* implement collection of requests profiles for update API
* implement API for viewing slow requests log
* add collection name to update worker
* add datetime to log
* fmt
* probabilistic counter of unique requests
* rename
* compute hash before converting into json value
* move logable out of generalizable
* fmt
* log query request
* fmt
* some fixes
* move measurement into local shard
* fmt
* upd openapi (not important)
* For enum variants, has discriminant
* Make SearchParams Copy
* Hash 0.0 and -0.0 the same
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Correctly hash enum variants and float values
* Hash through ordered float instead
* Fix priority queue not keeping longest request for hash
* SearchParams implements Copy
* Fix clippy warning
* Add unordered_hash_unique
* skip serialization if none
* Use OrderedFloat for hashing a float
* Use OrderedFloat for hashing a float
* only log updates if they are performed
---------
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Tim Visée <tim+github@visee.me>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: xzfc <xzfcpw@gmail.com>
* [strict-mode] Add max number of payload index count
* improve docs and validation for StrictModeConfig
* fix from coderabbit
* test blocking access to payload indexes
* polish test
* Bump segment versions on *-by-filter operations to acknowledge WAL
* Assert exactly one segment with the new operation version
* Review remarks
* Separate segment version number to not affect partial snapshots
* Adjust unit test
* Remove old bump_segment_version implementations
* Clarify max_persisted_segment_version_overwrite in flush_all
* Move trait implementations - Clippy
* Apply segment bumping to all *-by-filter functions
* Remove AtomicOptionU64
* Update function name and comments
* Remove max condition, start with overwrite value
* Add assertion, can never have zero segments
* Make max_persisted_segment_version_overwrite monotonic with fetch_max
---------
Co-authored-by: timvisee <tim@visee.me>
* make sure that grpc::Vector can always use new format
* specify target version for ToDo
* Update lib/api/src/conversions/vectors.rs
---------
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
* Create and load for an appendable quantization
remove feature flag
more todo
review remarks
review remarks
* fix after rebase
* Enable BQ in appendable segments
* remove const placeholder
* fix after rebase
* fix after rebase
* fix config mismatch optimizer
* fix sq infinity loop
* add comment
* Add VectorStorageEnum::{get_vector_bytes_opt, get_vector_layout}
* Update CompressedWithVectors format to include base vectors
* StorageGraphLinksVectors
* Clarify code
* Create and load for an appendable quantization
remove feature flag
more todo
review remarks
review remarks
* fix after rebase
* Rename is_appendable to supports_appendable
---------
Co-authored-by: timvisee <tim@visee.me>
* Quantization storage builder for chunked mmap
* fix after rebase
* Update lib/segment/src/vector_storage/quantized/quantized_chunked_mmap_storage.rs
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
* Add unit test
* Flush text index deletes before reopening them
* Fix unused warning if compiling without RocksDB
* Add e2e test
---------
Co-authored-by: timvisee <tim@visee.me>
* PrimitiveVectorElement: use zerocopy
* PrimitiveVectorElement: also use zerocopy::FromBytes
* serialize_graph_links: back_index is Vec<PointOffsetType>
* for_each_link_with_vector: loosen lifetimes
* internal implementation for filter any
* implement api for match `text_any`
* allow naive implementation of text match any
* dedup after kmerge + renames
* improve `expected_should_estimation`
* congruence test
* openapi test
* setup index in each test
---------
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
* In proxy segment, don't return deleted point versions
More specifically, don't return a point version from the wrapped segment
if that point is already marked for deletion within the proxy.
* Do get wrapped segment point version if it's newer than soft delete
* Flatten function
* allow custom K parameter for RRF
* generate grpc docs and openapi
* use tagged type approach for parametrized fusions
* use params approach in grpc
* simplify api structure
* upd schema
* nits
* rest: parameterized rrf as query variant
* consistency in doc comments
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
