upstream-archive/qdrant
1
0
Fork 0
mirror of https://github.com/qdrant/qdrant.git synced 2026-08-07 02:20:55 -05:00
Code Issues Packages Projects Releases Wiki Activity
5,465 Commits 346 Branches 118 Tags
94de0e2d2dcf997bf26bec34dad407d8de9a9abb
Commit Graph

1780 Commits

Author SHA1 Message Date
xzfc
3dcf69dacd GraphLinksMmap: remove Option (#5374) 
* Introduce GraphLayerData

* trait GraphLinks: do not require Default
 2024-11-07 10:54:14 +00:00
Jojii
7b677fb7fa Hardware counting for quantization (#5369) 
* add cpu measurement for quantization

* clippy

* fix tests

* discard hardware counters in benchmarks

* Forwarding hardware counter in distributed setup (#5371)

* make hardware counter available in distributed setup

* clippy

---------

Co-authored-by: generall <andrey@vasnetsov.com>
 2024-11-06 16:23:02 +01:00
Arnaud Gourlay
a5c9ce49aa Do not sort sparse vector before remapping it (#5351) 
* Do not clone sparse vector to remap it

* sort only once

* remove unwrap_or_else change
 2024-11-06 12:51:23 +01:00
dependabot[bot]
d0cbd40b0f build(deps): bump geo from 0.28.0 to 0.29.1 (#5355) 
* build(deps): bump geo from 0.28.0 to 0.29.1

Bumps [geo](https://github.com/georust/geo) from 0.28.0 to 0.29.1.
- [Changelog](https://github.com/georust/geo/blob/main/CHANGES.md)
- [Commits](https://github.com/georust/geo/compare/geo-0.28.0...geo-0.29.1)

---
updated-dependencies:
- dependency-name: geo
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* migrate the new trait system

* fix clippy in tests

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
 2024-11-06 11:21:47 +01:00
Andrey Vasnetsov
f843ab6610 detailed is_on_disk (#5373) 2024-11-06 11:04:30 +01:00
xzfc
978c4bdd99 Fixups for VistedPool (#5365) 2024-11-05 10:52:00 +01:00
Tim Visée
fedebbbf0e Fix immutable map index mishandling point deletions (#5346) 
* Fix search and delete using subslice index, rather than container intex

* Add sanity checks in debug assertions

* test

---------

Co-authored-by: generall <andrey@vasnetsov.com>
 2024-11-01 19:57:22 +01:00
xzfc
59c2dedcaa VisitedList: Vec<usize> -> Vec<u8> (#5336) 
* Do not use VisitedListHandle::count_visits_since()

* VisitedList: use u8 instead of usize

* Test VisitedList
 2024-10-31 18:26:25 +01:00
dependabot[bot]
95219292ae build(deps): bump smol_str from 0.3.1 to 0.3.2 (#5325) 
Bumps [smol_str](https://github.com/rust-analyzer/smol_str) from 0.3.1 to 0.3.2.
- [Changelog](https://github.com/rust-analyzer/smol_str/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-analyzer/smol_str/compare/v0.3.1...v0.3.2)

---
updated-dependencies:
- dependency-name: smol_str
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-10-29 10:54:04 +01:00
Arnaud Gourlay
d06e8d0e89 Distance matrix leverages HasVector condition (#5317) 2024-10-27 10:17:42 +01:00
Andrey Vasnetsov
633d996e61 HasVector filtering condition (#5303) 
* include vector storage into struct vector index

* implement has_vector

* generate schemas

* refactor query filter optimizer so avoid too many function arguments

* test + fix for sparse vectors

* Update lib/segment/src/index/struct_payload_index.rs

Co-authored-by: Jojii <15957865+JojiiOfficial@users.noreply.github.com>

* Update lib/segment/src/index/query_optimization/optimizer.rs

Co-authored-by: Jojii <15957865+JojiiOfficial@users.noreply.github.com>

* fmt

---------

Co-authored-by: Jojii <15957865+JojiiOfficial@users.noreply.github.com>
 2024-10-25 18:47:03 +02:00
Kumar Shivendu
31ce928fcf Track vector storage usage in bytes for each of the segments (#5292) 
* Track disk usage for each of the vectors and segments

* Dont extend vector_data field

* Remove comment and improve var name

* Introduce vectors_size_bytes and rename available_size_in_bytes

* Introduce vectors_size_bytes and rename available_size_in_bytes

* fix linter

* Avoid converting to u64

* calculate vectors_size_bytes for proxy segments

* Fix OpenAPI spec

* fix typo
 2024-10-25 15:09:31 +05:30
Jojii
98ef428878 Hw counter utilization checks (#5288) 
* Enforce usage of hardware counter values

* improve comments

* log a warning in release mode

* some minor improvements

* avoid cloning for hardware counter

* fmt

---------

Co-authored-by: generall <andrey@vasnetsov.com>
 2024-10-25 09:52:18 +02:00
Arnaud Gourlay
fb2b6a116d Allocate less rocksdb options (#5309) 2024-10-24 16:40:56 +02:00
Arnaud Gourlay
e55abe662d [perf] No backtrace on sparse vector index version mismatch (#5305) 2024-10-24 12:31:48 +02:00
Arnaud Gourlay
c1ce1ab6ef [perf] Do not reallocate RocksDB write_options (#5301) 2024-10-23 20:00:03 +02:00
Arnaud Gourlay
6f1ef42907 Promote criterion to workspace (#5291) 2024-10-22 16:29:23 +02:00
Jojii
669cc4f895 Cpu measurements for dense vector + custom sparse scorer (#5261) 
* add measurements for dense vector and custom sparse scorer

* adjust sparse vector scoring cpu measurement

* review suggestions

---------

Co-authored-by: generall <andrey@vasnetsov.com>
 2024-10-22 14:46:17 +02:00
Arnaud Gourlay
90cd35a196 Promote ordered-float to workspace (#5285) 2024-10-22 10:45:38 +02:00
dependabot[bot]
6905cf2b4e build(deps): bump io-uring from 0.7.0 to 0.7.1 (#5273) 
Bumps [io-uring](https://github.com/tokio-rs/io-uring) from 0.7.0 to 0.7.1.
- [Commits](https://github.com/tokio-rs/io-uring/commits)

---
updated-dependencies:
- dependency-name: io-uring
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-10-22 09:04:30 +02:00
dependabot[bot]
1558e7dfcd build(deps): bump ordered-float from 4.3.0 to 4.4.0 (#5277) 
Bumps [ordered-float](https://github.com/reem/rust-ordered-float) from 4.3.0 to 4.4.0.
- [Release notes](https://github.com/reem/rust-ordered-float/releases)
- [Commits](https://github.com/reem/rust-ordered-float/compare/v4.3.0...v4.4.0)

---
updated-dependencies:
- dependency-name: ordered-float
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-10-22 08:52:40 +02:00
Arnaud Gourlay
a822b4bb31 Remove forgotten eprintln (#5262) 2024-10-18 16:31:53 +02:00
Jojii
5696543123 Add CPU measurement for multivectors (#5241) 
* Add CPU measurement for multivectors

* add further test sparse vector search

* make updating vector_query_context consistent

* review fix + take element type into consideration

---------

Co-authored-by: generall <andrey@vasnetsov.com>
 2024-10-18 09:58:02 +02:00
Jojii
73a785f145 Add HardwareCounterCell + CPU measurement for sparse vector search (#5239) 
* Add HardwareCounterCell and counting for sparse plain search

* Add measurement for indexed sparse vector

* add tests for calculations

* move SegmentQueryContent higher in the call stack

* move hardware counters inside the query context

* fix clippy

* Fix applying hardware measurements and add test for this

---------

Co-authored-by: generall <andrey@vasnetsov.com>
 2024-10-17 16:24:13 +02:00
Arnaud Gourlay
4d83c198c0 Do not create errors eagerly (#5254) 2024-10-17 12:28:48 +02:00
Arnaud Gourlay
e85a336b79 Remove unused dependencies (#5247) 2024-10-16 11:23:08 +02:00
Arnaud Gourlay
90d3d57483 [perf] Do not capture backtrace on get_sparse_vector_opt (#5219) 
* Do not capture backtrace on get_sparse_vector_opt

* clean import
 2024-10-15 11:07:01 +02:00
Arnaud Gourlay
3de4e107d4 fix lints for Clippy 1.82 (#5229) 
* fix lints for Clippy 1.82

* regen openapi
 2024-10-14 13:13:58 +02:00
Arnaud Gourlay
2e0f72364f Payload storage can include files in snapshots (#5214) 2024-10-14 09:26:19 +02:00
Andrey Vasnetsov
a74b7807bc handle the case of empty geo hashes (#5222) 2024-10-11 21:05:27 +02:00
Arnaud Gourlay
aa6dbbfb82 Promote iter function to payload storage interface (#5205) 2024-10-10 13:38:48 +02:00
Arnaud Gourlay
29df805470 Fix typo in RocksDB wrapper type (#5207) 2024-10-09 14:18:32 +02:00
Andrey Vasnetsov
e0d507dac7 Inference interface in REST and gRPC (#5165) 
* include document & image objects into grpc API

* introduce image and object to rest api

* minor refactoring

* rename Vector -> VectorInternal

* decompose vector data structures

* add schema

* fmt

* grpc docs

* fix conversion

* fix clippy

* fix another conversion

* rename VectorInput -> VectorInputInternal

* replace grpc TryFrom with async functions

* fmt

* replace rest TryFrom with async functions

* add image and object into query rest

* separate inference related conversions

* move json-related conversions into a separate file

* move vector-related transformations into a separate file

* move more vector related-conversions into dedicated module
 2024-10-09 10:15:46 +02:00
dependabot[bot]
e0b5968677 build(deps): bump sysinfo from 0.31.4 to 0.32.0 (#5197) 
Bumps [sysinfo](https://github.com/GuillaumeGomez/sysinfo) from 0.31.4 to 0.32.0.
- [Changelog](https://github.com/GuillaumeGomez/sysinfo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/GuillaumeGomez/sysinfo/compare/v0.31.4...v0.32.0)

---
updated-dependencies:
- dependency-name: sysinfo
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-10-08 09:31:32 +02:00
dependabot[bot]
42eecc3ac4 build(deps): bump procfs from 0.16.0 to 0.17.0 (#5193) 
Bumps [procfs](https://github.com/eminence/procfs) from 0.16.0 to 0.17.0.
- [Release notes](https://github.com/eminence/procfs/releases)
- [Commits](https://github.com/eminence/procfs/compare/v0.16.0...v0.17.0)

---
updated-dependencies:
- dependency-name: procfs
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-10-08 09:30:07 +02:00
Ivan Pleshkov
080352cc56 mmap geo index api (#5163) 
* define mmap geo index

fix compilation

deleted flags

load new mmap

geo index tests

fix tests

fix build after rebase

add files list

* refactor get_stored_sub_regions output type

* review remanings

* mmap geo index api

* fix after rebase
 2024-10-07 14:35:17 +02:00
Ivan Pleshkov
e00bd70f9c Mmap geo index (#4841) 
* define mmap geo index

fix compilation

deleted flags

load new mmap

geo index tests

fix tests

fix build after rebase

add files list

* are you happy fmt

* refactor get_stored_sub_regions output type

* repr(C) for geohash

* Replace manual flattening with flat map iterator

* review remanings

* remove panics

* add files for snapshot

* rename Dynamic into InMemory

* are you happy fmt

* review renames

* Use copied rather than cloned

* minor review refactoring + comments

---------

Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: generall <andrey@vasnetsov.com>
 2024-10-07 12:08:58 +02:00
xzfc
761069bc0a Restore SnapshotFormat::Streamable snapshots (#5179) 2024-10-04 16:39:56 +00:00
xzfc
98c4e605ae Support SnapshotFormat::Streamable, add REST endpoint (#5178) 2024-10-04 16:36:19 +00:00
xzfc
bc2104604c Introduce SnapshotFormat (#5175) 2024-10-04 16:09:30 +00:00
xzfc
7932286709 Update tar_ext::BuilderExt (#5112) 
* tar_ext::BuilderExt: generalize and add drop workaround

- Generalize: instead of taking `std::fs::File`, take
  `impl Write + Seek`.
- Add a workaround for drop: `tar::Builder<W>::drop` might panic if the
  underlying writer panics when using in async context.

* Drop unnecessary write_fn usage

* tar_ext::BuilderExt: support both seekable and streaming variants

* tar_ext::BuilderExt: support both owned/borrowed variants
 2024-10-04 14:10:09 +00:00
Luis Cossío
045779233a Facets: Support for bool payload (#5100) 
* enable faceting with boolean index

* test and fix

* also test uuid
 2024-10-01 09:17:50 -03:00
dependabot[bot]
3b2110fe6c build(deps): bump io-uring from 0.6.4 to 0.7.0 (#5158) 
Bumps [io-uring](https://github.com/tokio-rs/io-uring) from 0.6.4 to 0.7.0.
- [Commits](https://github.com/tokio-rs/io-uring/commits)

---
updated-dependencies:
- dependency-name: io-uring
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-10-01 10:15:34 +02:00
dependabot[bot]
a5cebd8eef build(deps): bump rstest from 0.22.0 to 0.23.0 (#5159) 
Bumps [rstest](https://github.com/la10736/rstest) from 0.22.0 to 0.23.0.
- [Release notes](https://github.com/la10736/rstest/releases)
- [Changelog](https://github.com/la10736/rstest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/la10736/rstest/compare/v0.22.0...v0.23.0)

---
updated-dependencies:
- dependency-name: rstest
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-10-01 10:13:09 +02:00
dependabot[bot]
52db80a292 build(deps): bump ordered-float from 4.2.2 to 4.3.0 (#5154) 
Bumps [ordered-float](https://github.com/reem/rust-ordered-float) from 4.2.2 to 4.3.0.
- [Release notes](https://github.com/reem/rust-ordered-float/releases)
- [Commits](https://github.com/reem/rust-ordered-float/compare/v4.2.2...v4.3.0)

---
updated-dependencies:
- dependency-name: ordered-float
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-10-01 09:19:46 +02:00
Jojii
8980483416 [Strict mode] Merge StrictMode config types (#5137) 
* merge StrictModeConfig and StrictModeConfigDiff into one type

* review remarks
 2024-09-26 12:36:39 +02:00
Jojii
791d064bde [Strict mode] default config (#5037) 
* add default config for strict mode

* add default values as example to config/config.yaml

* improve implementation

* update grpc docs
 2024-09-25 14:22:36 +02:00
Arnaud Gourlay
d527ab9cd0 Promote atomicwrites to workspace (#5135) 2024-09-24 09:44:12 +02:00
dependabot[bot]
6b73ac1791 build(deps): bump charabia from 0.9.0 to 0.9.1 (#5125) 
Bumps [charabia](https://github.com/meilisearch/charabia) from 0.9.0 to 0.9.1.
- [Release notes](https://github.com/meilisearch/charabia/releases)
- [Commits](https://github.com/meilisearch/charabia/compare/v0.9.0...v0.9.1)

---
updated-dependencies:
- dependency-name: charabia
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-09-23 21:27:12 +02:00
dependabot[bot]
f6246b0e44 build(deps): bump atomicwrites from 0.4.3 to 0.4.4 (#5130) 
Bumps [atomicwrites](https://github.com/untitaker/rust-atomicwrites) from 0.4.3 to 0.4.4.
- [Commits](https://github.com/untitaker/rust-atomicwrites/compare/0.4.3...0.4.4)

---
updated-dependencies:
- dependency-name: atomicwrites
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-09-23 21:24:50 +02:00