upstream-archive/qdrant
1
0
Fork 0
mirror of https://github.com/qdrant/qdrant.git synced 2026-08-07 02:20:55 -05:00
Code Issues Packages Projects Releases Wiki Activity
5,465 Commits 346 Branches 118 Tags
94de0e2d2dcf997bf26bec34dad407d8de9a9abb
Commit Graph

1780 Commits

Author SHA1 Message Date
dependabot[bot]
03642f4c2e Bump uuid from 1.4.1 to 1.5.0 (#2993) 
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.4.1 to 1.5.0.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/1.4.1...1.5.0)

---
updated-dependencies:
- dependency-name: uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-11-14 10:04:49 +01:00
dependabot[bot]
629e71c08d Bump memmap2 from 0.7.1 to 0.9.0 (#2999) 
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.7.1 to 0.9.0.
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.7.1...v0.9.0)

---
updated-dependencies:
- dependency-name: memmap2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-11-14 08:59:46 +01:00
dependabot[bot]
7331952054 Bump tempfile from 3.8.0 to 3.8.1 (#2996) 
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.8.0 to 3.8.1.
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stebalien/tempfile/commits)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-11-14 07:19:29 +01:00
Ivan Pleshkov
7f065d4a0a Integrate vector into NamedVectors struct (#2966) 
* Integrate vector into NamedVectors struct

* revert into_default_vector

* remove unnecessary functions

* remove unnecessary functions

* remove unneccesary function

* remove unnecessary match

* remove matches

* revert preprocess signature

* remove unnecessary functions
 2023-11-11 17:29:49 +01:00
Arnaud Gourlay
bc5d0976a1 Sparse vector validation (#2965) 
* Sparse vector validation

* cheaper indices uniqueness validation and tests
 2023-11-10 16:42:10 +01:00
Tim Visée
9394fc1757 Some micro optimizations in reserve/resize usage (#2967) 
* In HNSW graph layer building, infer right capacity on collect

* Resize with empty vector, use constant function rather than clone

* Use resize_with in other places as well
 2023-11-10 09:42:14 +01:00
Andrey Vasnetsov
c95b2448d9 Shard key routing for update requests (#2909) 
* add shard_key into output data structures for points

* fmt

* add shard selector for point update operations

* fix creating index without sharding

* Merge serde attributes

* Code review changes

* review fixes

* upd openapi

---------

Co-authored-by: timvisee <tim@visee.me>
 2023-11-09 15:06:02 +01:00
Arnaud Gourlay
c492ab5c54 Share filter results for plain search in HNSW (#2958) 2023-11-09 13:36:51 +01:00
Tim Visée
dbb10e456d Fix try_set_capacity functions (#2960) 
* Fix `try_set_capacity` functions, add unit tests

* Skip setting capacity if already grown

* Improve chunked vectors insertion function

* Remove if capacity is zero
 2023-11-09 13:35:49 +01:00
Arnaud Gourlay
1ae1f483d4 Benchmark sparse filter plain search (#2947) 
* Sparse filter plain search

* code review
 2023-11-09 09:57:43 +01:00
Luis Cossío
904a7ab306 Discovery API (#2861) 
* create and connect discovery http and grpc interfaces

* add openapi tests

* fix bad rebase

* Add better descriptions

* remove numpy from openapi tests

* fix rebase artifact

* remove already addressed TODO

* add more tests

* 🤡🔫 (cfg batch handler)

* add timeout query param for discover requests

* More gRPC validation

* make fields pydantic_openapi_generator_v3 friendly

* `context_pairs` -> `context` with struct for pairs

* discovery api is only discovery or context,
move struct description to fields

---------

Co-authored-by: timvisee <tim@visee.me>
 2023-11-08 14:22:31 +01:00
Arnaud Gourlay
6d5fa056b9 Sparse vector search benchmarks (#2944) 2023-11-08 13:07:33 +01:00
Arnaud Gourlay
b3aea28426 Extra tests for Sparse Vector Index (#2925) 2023-11-07 15:03:37 +01:00
Tim Visée
7d7cf61f16 Fix divide by zero panic, fall back to 0 (#2933) 
* Fix divide by zero panic, if rhs is zero default to 0

* Add unit test

Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>

---------

Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>
 2023-11-06 14:30:55 +01:00
Arnaud Gourlay
ce0a9c0bb0 Improve sizing in GraohLayersBuilder (#2932) 2023-11-06 11:15:30 +01:00
Andrey Vasnetsov
8b05c4a388 Int indexing (#2927) 
* remove point from index if value was replaced with incompatible

* fmt
 2023-11-06 09:43:06 +01:00
Arnaud Gourlay
ec2dc36719 SparseVectorIndex implements VectorIndex (#2900) 
* SparseVector implements VectorIndex

* dedicated telemetry

* conflict

* easy code reviews

* simplify tracking indexed points count

* move telemetry conversion to sparse file

* move max_result_count out of inverted index trait

* unify sparse vector fixtures

* simpler conversion

* add todo regarding OOM potential

* reuse check deleted from raw scorer with TODO

* change new to open to handle mmap index
 2023-11-03 16:23:05 +01:00
Luis Cossío
80bfd414ba update RawScorer composition diagram (#2916) 2023-11-03 08:52:28 -04:00
Luis Cossío
50f3ef1757 Fix recommend best_score formula (#2904) 
* fix recommend best score formula

* fix integration test

* review fixes

* rebase and fix conflicts

* correct proptest comment
 2023-11-02 13:30:44 -04:00
Arnaud Gourlay
8a05462c6d Raw scorer for sparse vectors (#2915) 
* Raw scorer for sparse vectors

* sparse vector storage trait

* use get sparse vector fn

* remove dedicated types

---------

Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>
 2023-11-02 14:59:18 +01:00
Ivan Pleshkov
9b88a225ac Remove temporary conversion to dense vector (#2906) 2023-11-01 18:46:59 +01:00
Luis Cossío
3a772b7b8c Add DiscoveryRequest and preprocessing (#2867) 
* add `DiscoverRequest` and batch preprocessing logic

* Add description for DiscoverRequest, use serde(default)  on offset

* review fixes
 2023-11-01 11:30:08 -04:00
Ivan Pleshkov
cb917ea0db raw scorer with operation result (#2897) 2023-10-31 23:44:20 +01:00
Luis Cossío
7dd2efa6e0 increase ef to get better precision on tests (#2894) 2023-10-30 22:33:11 +01:00
Ivan Pleshkov
3e2698c5fb Return visited list by drop (#2801) 
* return visited list by drop

* review remarks
 2023-10-30 18:02:52 +01:00
Arnaud Gourlay
97ffee476e Simple sparse vector storage (#2806) 
* Simple sparse vector storage

* better todo
 2023-10-30 11:27:36 +01:00
Luis Cossío
3b17deb259 Discovery search handler at hnsw level (#2836) 
* add discovery_search_with_graph handler

* add precision test

* rename `DiscoveryPair` -> `ContextPair`

* review fixes

* renames + comments

---------

Co-authored-by: generall <andrey@vasnetsov.com>
 2023-10-26 19:36:15 -04:00
Ivan Pleshkov
95d240cb44 mutable geo index (#2768) 2023-10-26 15:50:20 +02:00
Ivan Pleshkov
dd5e2d9d98 vector index update function (#2877) 2023-10-26 15:43:01 +02:00
Ivan Pleshkov
f3cd93379c propagate vector type into QueryVector (#2878) 2023-10-26 15:40:28 +02:00
Andrey Vasnetsov
8b9df28f83 Hnsw parallel build accuracy (#2869) 
* fixes for parallel HNSW graph building

* mft

* fix the bug by introducing ready_list, do not use unready point in parallel building

* fmt

* rm debug code

* clippy fix

* fix test

* fmt

* review fixes

* clippy fix

* fix filtered hnsw build
 2023-10-25 16:12:36 +02:00
Apostolos Matsagkas
327c78678c Fix overeager marking of visited points during search (#2864) 2023-10-25 11:56:18 +02:00
Arnaud Gourlay
87c9e79cb1 Sparse vector in QueryVector (#2854) 
* Sparse vector in QueryVector

* remove experimental sparse raw scorer

* code review

* code review
 2023-10-20 17:07:02 +02:00
Arnaud Gourlay
d135fbb441 Fix cargo test doc (#2856) 
* fix cargo test doc

* run doc test on CI

* try re-ordering tests execution to check if it takes less time
 2023-10-20 16:45:37 +02:00
Luis Cossío
9e70494990 Add discovery context scorer (#2820) 
* add discovery context query

* add tests

* small optimization for loss_by

* fix async raw scorer

* ditch micro-optimization

* remove sigmoid,  rename `iter_all` -> `flat_iter`

* fix tests, add more cases
 2023-10-19 15:29:37 -03:00
Ivan Pleshkov
e610ab202d Use enum vector type in vector storage (#2841) 
* use enum vector type in vector storage

* add to_vec_ref
 2023-10-19 14:49:31 +02:00
Luis Cossío
f54d769b97 Allow selecting custom entry points at segment level (#2834) 
* allow selecting custom entry points at segment level

* empty commit

* fix ci build

* restrict refactor up to search_with_graph
 2023-10-18 09:14:32 -03:00
Arnaud Gourlay
ecaff66f4c Sparse vector index integration (#2835) 
* inverted index as trait

* introduce sparse vector index integration

* sparse vectors search cancellation

* clippy

* expose helper for search sparse

* code review
 2023-10-18 12:03:04 +02:00
Tim Visée
24cbe77d79 Update ordered float to 4.1.1 (#2823) 
* Bump ordered-float to 3.9.2 and 2.10.1

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>

* Bump ordered-float to 4.1.1

---------

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
 2023-10-16 13:28:33 +02:00
Luis Cossío
7207cb442a Add QueryVector::Discovery (#2809) 
* - add DiscoveryQuery to QueryVector
- merge reco_query_scorer and discovery_query_scorer into custom_query_scorer
- reuse reco_query_scorer equivalency tests for discovery scorer

* self review

* empty commit
 2023-10-15 18:52:02 -03:00
Ivan Pleshkov
8e4acce59f dense vector storage (#2817) 2023-10-13 22:30:49 +02:00
Andrey Vasnetsov
fa3aebe660 optimize usage of the match-any filter in case it is empty (#2803) 
* optimize usage of the match-any filter in case it is empty

* fix outdated tests
 2023-10-12 15:45:32 +02:00
Luis Cossío
66fe2026a7 Add discovery query scorer (#2791) 
* add DiscoveryQuery

* refactor and add tests

* add discovery query scorer

* add quantized discovery query scorer
 2023-10-12 09:52:05 -03:00
Ivan Pleshkov
8cdeda26d8 Quantization storage as separate entity (#2797) 
* quantization storage as separate entity

* simplify max threads calculation
 2023-10-12 11:21:34 +02:00
Ivan Pleshkov
baf7e5fc82 fix rescoring for small cardinality case (#2789) 
* fix rescoring for small cardinality case

* remove obsolete comment

* review fix
 2023-10-11 13:57:29 +02:00
Arnaud Gourlay
27be199f5a Allow JsonPath for PayloadSelector (#2756) 
* Allow JsonPath for PayloadSelector

* code review

* mention Entry API in performance note

* pre-process exclude list to avoid incorrect side-effects

* remove support for array paths

* review refactoring

* add test for implicit/explicit array notation

* [skip ci] panic -> unreachable

---------

Co-authored-by: generall <andrey@vasnetsov.com>
 2023-10-06 12:27:30 +02:00
Arnaud Gourlay
8b2373c7f4 Fix Clippy 1.73 for consistent Ord impl in NumericIndex (#2767) 2023-10-05 18:51:45 +02:00
Arnaud Gourlay
3bff12dba6 Make GeoPolygon interiors optional (#2766) 2023-10-05 18:34:39 +02:00
Ivan Pleshkov
d83806703e Stack allocated geohash (#2764) 
* stack allocated geohash

* Update lib/segment/src/index/field_index/geo_hash.rs

Co-authored-by: Tim Visée <tim+github@visee.me>

* Update lib/segment/src/index/field_index/geo_hash.rs

Co-authored-by: Tim Visée <tim+github@visee.me>

---------

Co-authored-by: Tim Visée <tim+github@visee.me>
 2023-10-05 15:54:55 +02:00
Ivan Pleshkov
b916617713 Immutable numeric index (#2666) 
* immutable numeric index

* avoid Vec<u8> encoding

* test inf float

* rename test

* fix nan case for float payload index

* are you happy fmt

* set of numeric pair as separate struct

* fix build

* are you happy clippy

* more tests

* remove from histogram

* review fixes
 2023-10-05 14:12:23 +02:00