Commit Graph

15 Commits

Author SHA1 Message Date
xzfc
89d399ef99 Incremental HNSW index building: append-only case (#6325)
* Pass FeatureFlags into VectorIndexBuildArgs

* Incremental HNSW index building: append-only case

* Use debug_assert

* first_few_ids

* Check deleted_point_count

* Drop unused method
2025-04-09 18:56:53 +00:00
Ivan Pleshkov
73259267ae GPU HNSW integration (#5535)
* gpu hnsw

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2024-12-05 00:58:49 +01:00
Ivan Pleshkov
07c25f00f1 fix mmap field index snapshot (#4847) 2024-08-07 15:40:10 +02:00
xzfc
26b0f08455 Tidy up integration tests (#4756) 2024-07-26 21:28:37 +02:00
Luis Cossío
57aee5addf universal-query: Impl of query_internal in collection (#4331)
* move ScoredPointTies to segment, make inner by reference

* `query_internal` implementation

* remove empty utils mod

* use `then_with`

* Improve readability, remove duplicated code

* refactoring suggestions

* don't collect eagerly

* remove unused import

* dont panic on empty transpose input

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2024-05-31 08:56:06 -04:00
Ivan Pleshkov
93ed4ab5c8 Multidense vectors quantization (#4202)
* quantized multivector definition

new multiquantization scorers

encode query

maxsim for quantized vectors

remove obsolete todo

reuse existing scorers

create multivector quantized storage

save load offsets

add test

fix vectors count

tempopery disable test while debugging

fix tests

fix build

less static lifetimes

less static lifetimes

fix build

* fix build after rebase

* add persistence test

* fix codespell

* increase accuracy in tests

* review remarks

* add comment references

* are you happy codespell

* don't use bincode
2024-05-28 12:36:09 +02:00
Ivan Pleshkov
528d759429 byte storage quantization fix and test (#4063)
* byte storage quantization fix and test

* apply quantization_preprocess in quantization scorers

* exact true

* exact true

* calculate sames count

* wrong distance getter

* fix Manhattan distance getter

* less acc check

* fix build

* update acc
2024-04-19 11:04:09 +02:00
Ivan Pleshkov
224e4f600a Byte storage integration into segment (#4049)
* byte storage with quantization

raw scorer integration

config and test

are you happy fmt

fn renamings

cow refactor

use quantization branch

quantization update

* are you happy clippy

* don't use distance in quantized scorers

* fix build

* add fn quantization_preprocess

* apply preprocessing for only cosine float metric

* fix sparse vectors tests

* update openapi

* more complicated integration test

* update openapi comment

* mmap byte storages support

* fix async test

* move .unwrap closer to the actual check of the vector presence

* fmt

* remove distance similarity function

* avoid copying data while working with cow

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2024-04-18 00:42:17 +02:00
Arnaud Gourlay
54d28d2d53 Integrate Multivector at segment level (#3973)
* Integrate Multivector at segment level

* more tests - green with lower accuracy

* decrease expected accuracy to fix test

* cleanup test

* multivector can not be empty

* vary number of vector per multivec point
2024-04-08 12:22:24 +02:00
Ivan Pleshkov
1fbcebcdbe Multivector hnsw test (#3880)
* colbert query scorers (#3713)

* colbert query scorers

fix build after rebase

* review remarks

multivector hnsw test

* fixes after rebase

* fix test

* fix build after rebase

* review remark
2024-03-21 13:58:02 +01:00
Ivan Pleshkov
7d57fb8859 Sparse vectors advanced search api support (#3128)
* sparse vectors query scorer

plain search test without internals

fix sparse_vector_index_plain_search

fix async scorer build

fix unit tests

are you happy fmt

sparse index with full query support

fix benches, use exact flag to force plain search

add alias and comment to exact flag

fix tests

recommendations TODOs fix

add simple test

add todo

refactor index

test vs comparison dense and sparse discovery

are you happy fmt

propogate error from avg negatives

reuse filtered points list

rollback api changes

better discovery test

are you happy fmt

rollback openapi

fix build

fix tests

review remark recommendations

review remarks

review remarks

codespell

integrate plain search

Update lib/collection/src/recommendations.rs

Co-authored-by: Luis Cossío <luis.cossio@qdrant.com>

Update lib/collection/src/recommendations.rs

Co-authored-by: Luis Cossío <luis.cossio@qdrant.com>

Update lib/collection/src/recommendations.rs

Co-authored-by: Luis Cossío <luis.cossio@qdrant.com>

fix comment of plain search

Update lib/collection/src/recommendations.rs

Co-authored-by: Luis Cossío <luis.cossio@qdrant.com>

fix tests

* simplify search query function
2023-12-05 20:32:23 +01:00
Arnaud Gourlay
ec2dc36719 SparseVectorIndex implements VectorIndex (#2900)
* SparseVector implements VectorIndex

* dedicated telemetry

* conflict

* easy code reviews

* simplify tracking indexed points count

* move telemetry conversion to sparse file

* move max_result_count out of inverted index trait

* unify sparse vector fixtures

* simpler conversion

* add todo regarding OOM potential

* reuse check deleted from raw scorer with TODO

* change new to open to handle mmap index
2023-11-03 16:23:05 +01:00
Luis Cossío
3b17deb259 Discovery search handler at hnsw level (#2836)
* add discovery_search_with_graph handler

* add precision test

* rename `DiscoveryPair` -> `ContextPair`

* review fixes

* renames + comments

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2023-10-26 19:36:15 -04:00
Andrey Vasnetsov
a75420ddcc Fix batch request with duplicated filter (#2051)
* fix double usage of iterator

* tests
2023-06-09 00:05:00 +02:00
Arnaud Gourlay
fcf55b8943 merge integration binaries (segment) (#2033) 2023-06-07 08:46:49 +02:00