* Add QuantizationConfigDiff type
* Add quantization config diff to vector parameters
* Prefer vector config over collection config for quantization
* Update OpenAPI specification
* Validate quantization configuration quantile in 0.5-1.0 range
As per https://github.com/qdrant/qdrant/pull/1681
* Add test if check if vector specific quantization config is persisted
* Alias quantization to quantization_config in vector parameters
* Remove quantization config diff, use full vector specific config instead
* Regenerate OpenAPI specification and gRPC docs
* Fix compilation error
* Add error handling to quantization config conversions
* Fix quantization integration test, make HNSW test stricter
* Validate VectorConfig/VectorParams, remove obsolete validation
* Add HNSW config diff to vector parameters
* Validate params in collection config
* Add HNSW config to segment vector data config
* Add VectorsConfig params iterator for more elegant conversions
* Prefer vector HNSW config over collection config for building HNSW index
* Base segment vector param HNSW config on collection config
* General improvements
* Rewrite HNSW ef_construct extract function to also consider vector configs
* Update OpenAPI specification
* Add test to check if vector specific HNSW config is persisted
* review changes
* review changes
* Regenerate gRPC docs
* Fix test on Windows
* Regenerate OpenAPI specification
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* remove deleted flags from vector storage
* remove deleted flags from mmap
* new simple vector storage format
* are you happy clippy
* remove id_tracker from raw_scorer
* revert vector storage format changes
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* integrate quantized data to storages
* revert gitignore
* are you happy clippy
* quantize in optimizer
* provide flag
* fix segfault
* skip quantization flag, update scores
* use quantization flag
* are you happy fmt
* use quantization flag
* quantized search test
* are you happy fmt
* refactor test, refactor scorer choosing
* are you happy fmt
* run quantization on segment builder
* decrease testing parameters
* simplify segment
* update version
* remove use_quantization flag
* provide quantization config
* quantization version up
* euclid dist
* add euclid test
* saveload
* fix initialization bugs
* quantization lib version up
* fix arm build
* refactor scorer selecting
* quant lib version up
* are you happy fmt
* are you happy fmt
* are you happy clippy
* add save/load test for simple storage
* add comments
* quantiles
* quantization mmap
* remove f32
* mmap test
* fix mmap slice
* fix mmap test
* use chunks for quantization storage
* fix build
* are you happy fmt
* update quantization library
* update quantization lib
* update quantization lib
* integrate api changes
* are you happy fmt
* change quantization api
* additional checks in tests
* update quantization version
* fix unit tests
* add quantization to storage config
* use quantization for all cardinality search cases
* Integrate quantization suggestions 2 (#1520)
* review api
* wip: refactor quantization integrations
* wip: refactor quantization integrations
* wip: fmt
* include quantization into snapshot
* fmt
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* add on disk key
* remove obsolete graph initialization
* remove obsolete max level
* update openapi
* graph links trait
* use mmap option
* same format for ram and mmap
* fix segment unit tests
* are you happy fmt
* are you happy clippy
* fix ci and add mmap test
* review fixes
* remove unused try-from
* fix version compatibility
* avoid loading from disk during conversion
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* exact search parameter
* exact search test
* decrease attempts count because of test speed
* minor review fixes
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* many named vectors per point (segment-level)
* operation result for dim function
* beautifulized vector name
* fix naming bug
* segment version migration
* fmt
* add segment tests
* are you happy clippy
* fix build
* [WIP] many named vectors per point (collection-level) (#975)
* config and search
* fix placeholders for proxy segment move
* remove VectorType from collection
* are you happy fmt
* vectors in grps messages
* create collections with vectors
* segment holder fixes
* are you happy fmt
* remove default vector name placeholders
* are you happy fmt
* are you happy clippy
* fix build
* fix web api
* are you happy clippy
* are you happy fmt
* record vector&vectors
* openapi update
* fix openapi integration tests
* segment builder fix todo
* vector names for update from segment
* remove unwrap
* backward compatibility
* upd openapi
* backward compatible PointStruct
* upd openapi
* fix record back-comp
* fmt
* vector configuration backward compatibility
* fix vetor storage size estimation
* fmt
* multi-vec segment test + index test
* fmt
* api integration tests
* [WIP] Named vectors struct (#1002)
* move to separate file
* named vectors as struct
* use cow
* fix build
* keys iterator
* avoid copy in PointStruct -> get_vectors
* avoid another copy
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* allow additional params for payload field index
* fmt
* wip: full text index building
* fmt
* text search request
* text search request
* full text index persitance and loading
* fmt
* enable fts index in mapping
* clippy
* fix tests + add integration test
* review fixes: extend payload index test
* revert incedental change
* [WIP] column oriented filter context
* suggestion
* [WIP] fix lifetimes and add more checkers
* refactor and externd struct filter context
* fmt
* add type alias for the condition checker
* fmt
Co-authored-by: gabriel velo <gabriel.velo@gmail.com>
* index geo payload
* fix group_points_per_region
* code review: use entry API
* geohash into box for payload_blocks
* add failing test for investigation
* workaround for when exp < min
* test with 2 geo values per point
* fix rounding error in cardinality estimation
* fmt
* code review: decompose using Iterator of slices
* code review: handle points with severak geo payloads
* refactor
* fix review comments
* review fixes
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* wip: u64 -> u128 + serialization tests
* breaking: use more flexible structure for saving point ids
* replace u64 external id type with enum
* update openapi definitions for uuid + fix retrieve point api + bash script tests
* implement checking stop-flag in the optimization routine
* wip: optimization cancel test
* force optimization stop during the construction of vector index
* fix clippy
* feat(#50): include vector into search result
allow to specify 'with_vector' parameter in search api to get search results vector data
* test(#50): fix tests
* chore(#50): apply cargo fmt
* chore(#50): update api docs
run tools/generate_openapi_models.sh
Co-authored-by: Daniil Sunyaev <dasforrum@gmail.com>
* update segment version after completed update only
* more stable updates: check pre-existing points on update, fail recovery, WAL proper ack. check_unprocessed_points WIP
* switch to async channel
* perform update operations in a separate thread (#111)
* perform update operations in a separate thread
* ordered sending update signal
* locate a segment merging versioning bug
* rename id_mapper -> id_tracker
* per-record versioning
* clippy fixes
* cargo fmt
* rm limit of open files
* fail recovery test
* cargo fmt
* wait for worker stops befor dropping the runtime
Tim Visée