upstream-archive/qdrant
1
0
Fork 0
mirror of https://github.com/qdrant/qdrant.git synced 2026-08-06 10:00:58 -05:00
Code Issues Packages Projects Releases Wiki Activity
5,465 Commits 344 Branches 118 Tags
94de0e2d2dcf997bf26bec34dad407d8de9a9abb
Commit Graph

1780 Commits

Author SHA1 Message Date
dependabot[bot]
c8f01d4064 Bump ordered-float from 3.5.0 to 3.6.0 (#1596) 
Bumps [ordered-float](https://github.com/reem/rust-ordered-float) from 3.5.0 to 3.6.0.
- [Release notes](https://github.com/reem/rust-ordered-float/releases)
- [Commits](https://github.com/reem/rust-ordered-float/compare/v3.5.0...v3.6.0)

---
updated-dependencies:
- dependency-name: ordered-float
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-21 21:04:18 +01:00
dependabot[bot]
83896ee5ad Bump ordered-float from 3.4.0 to 3.5.0 (#1590) 
Bumps [ordered-float](https://github.com/reem/rust-ordered-float) from 3.4.0 to 3.5.0.
- [Release notes](https://github.com/reem/rust-ordered-float/releases)
- [Commits](https://github.com/reem/rust-ordered-float/compare/v3.4.0...v3.5.0)

---
updated-dependencies:
- dependency-name: ordered-float
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-21 10:34:39 +01:00
dependabot[bot]
9a89b22d5d Bump walkdir from 2.3.2 to 2.3.3 (#1574) 
Bumps [walkdir](https://github.com/BurntSushi/walkdir) from 2.3.2 to 2.3.3.
- [Release notes](https://github.com/BurntSushi/walkdir/releases)
- [Commits](https://github.com/BurntSushi/walkdir/compare/2.3.2...2.3.3)

---
updated-dependencies:
- dependency-name: walkdir
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-17 07:48:45 +01:00
Ivan Pleshkov
1c6a6cea54 fix remove geo index (#1566) 2023-03-15 15:29:28 +01:00
Ivan Pleshkov
faf1235333 Remove deleted flags from vector storage (#1561) 
* remove deleted flags from vector storage

* remove deleted flags from mmap

* new simple vector storage format

* are you happy clippy

* remove id_tracker from raw_scorer

* revert vector storage format changes

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2023-03-15 14:05:07 +01:00
dependabot[bot]
a7023858b9 Bump semver from 1.0.16 to 1.0.17 (#1560) 
Bumps [semver](https://github.com/dtolnay/semver) from 1.0.16 to 1.0.17.
- [Release notes](https://github.com/dtolnay/semver/releases)
- [Commits](https://github.com/dtolnay/semver/compare/1.0.16...1.0.17)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-15 13:47:00 +01:00
dependabot[bot]
134e044a7e Bump geo from 0.23.1 to 0.24.0 (#1545) 
Bumps [geo](https://github.com/georust/geo) from 0.23.1 to 0.24.0.
- [Release notes](https://github.com/georust/geo/releases)
- [Changelog](https://github.com/georust/geo/blob/main/CHANGES.md)
- [Commits](https://github.com/georust/geo/compare/geo-0.23.1...geo-0.24.0)

---
updated-dependencies:
- dependency-name: geo
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>
 2023-03-15 13:19:21 +01:00
dependabot[bot]
7dc329b8ea Bump rayon from 1.6.1 to 1.7.0 (#1525) 
Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.6.1 to 1.7.0.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/compare/rayon-core-v1.6.1...rayon-core-v1.7.0)

---
updated-dependencies:
- dependency-name: rayon
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>
 2023-03-15 13:15:09 +01:00
Ivan Pleshkov
7fd15ef4af Don't update payload if point was not upserted (#1564) 
* fix values count loading in geoindex

* don't update payload if point was not upserted

* remove debug comments

* post-check of internal id

* revert changes from geo indexing branch
 2023-03-15 13:36:46 +04:00
Ivan Pleshkov
30ef3b6ea8 Fix values count loading in geoindex (#1563) 
* fix values count loading in geoindex

* WIP: fix loading statistics in geo-index

* WIP: fmt

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2023-03-15 12:56:42 +04:00
Andrey Vasnetsov
97c06f0377 Fix histogram for large integer values (#1562) 
* make histogram generic

* fmt
 2023-03-15 09:52:42 +01:00
Andrey Vasnetsov
91991f62bc ensure segment flushing order + extend logging (#1551) 
* ensure segment flushing order + extend logging

* fix review
 2023-03-13 10:46:06 +01:00
Ivan Pleshkov
2be8590968 Enum vector index (#1539) 
* separate payload and vector index bases

* vector index as enum
 2023-03-12 14:14:21 +01:00
Andrey Vasnetsov
501a4b1831 add benchmark for random vector scoring 2023-03-11 23:25:18 +01:00
Ivan Pleshkov
35719c0fab Remove vector scorer trait (#1546) 
* remove vector scorer

* nits

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2023-03-11 20:13:24 +01:00
Tim Visée
68a9c2ad48 Fix clippy warnings (#1542) 2023-03-10 12:57:38 +01:00
Andrey Vasnetsov
c6dd977c5e Quantization re scoring param (#1538) 
* parameter to disable rescoring with quantization

* fix

* fix default

* review fixes

* upd api autogen
 2023-03-09 11:51:29 +01:00
Ivan Pleshkov
84e6cd8290 Vector storages enum (#1533) 
* separate scoring and data containing

* vector storage enum

* fix test build

* are you happy clippy

* review fixes
 2023-03-09 09:43:20 +04:00
Ivan Pleshkov
5174388ab4 integrate quantized data to storages (#1311) 
* integrate quantized data to storages

* revert gitignore

* are you happy clippy

* quantize in optimizer

* provide flag

* fix segfault

* skip quantization flag, update scores

* use quantization flag

* are you happy fmt

* use quantization flag

* quantized search test

* are you happy fmt

* refactor test, refactor scorer choosing

* are you happy fmt

* run quantization on segment builder

* decrease testing parameters

* simplify segment

* update version

* remove use_quantization flag

* provide quantization config

* quantization version up

* euclid dist

* add euclid test

* saveload

* fix initialization bugs

* quantization lib version up

* fix arm build

* refactor scorer selecting

* quant lib version up

* are you happy fmt

* are you happy fmt

* are you happy clippy

* add save/load test for simple storage

* add comments

* quantiles

* quantization mmap

* remove f32

* mmap test

* fix mmap slice

* fix mmap test

* use chunks for quantization storage

* fix build

* are you happy fmt

* update quantization library

* update quantization lib

* update quantization lib

* integrate api changes

* are you happy fmt

* change quantization api

* additional checks in tests

* update quantization version

* fix unit tests

* add quantization to storage config

* use quantization for all cardinality search cases

* Integrate quantization suggestions 2 (#1520)

* review api

* wip: refactor quantization integrations

* wip: refactor quantization integrations

* wip: fmt

* include quantization into snapshot

* fmt

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2023-03-03 17:46:17 +01:00
Ivan Pleshkov
83efed1fcf fix windows warning unused parallel_graph_build (#1513) 2023-03-01 11:10:22 +01:00
dependabot[bot]
b5ded897a2 Bump atomicwrites from 0.3.1 to 0.4.0 (#1511) 
Bumps [atomicwrites](https://github.com/untitaker/rust-atomicwrites) from 0.3.1 to 0.4.0.
- [Release notes](https://github.com/untitaker/rust-atomicwrites/releases)
- [Commits](https://github.com/untitaker/rust-atomicwrites/compare/0.3.1...0.4.0)

---
updated-dependencies:
- dependency-name: atomicwrites
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-01 08:35:21 +01:00
dependabot[bot]
d6e6d75ff1 Bump schemars from 0.8.11 to 0.8.12 (#1505) 
Bumps [schemars](https://github.com/GREsau/schemars) from 0.8.11 to 0.8.12.
- [Release notes](https://github.com/GREsau/schemars/releases)
- [Changelog](https://github.com/GREsau/schemars/blob/master/CHANGELOG.md)
- [Commits](https://github.com/GREsau/schemars/compare/v0.8.11...v0.8.12)

---
updated-dependencies:
- dependency-name: schemars
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-28 09:51:03 +01:00
dependabot[bot]
b419a5929b Bump tempfile from 3.3.0 to 3.4.0 (#1506) 
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.3.0 to 3.4.0.
- [Release notes](https://github.com/Stebalien/tempfile/releases)
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/NEWS)
- [Commits](https://github.com/Stebalien/tempfile/commits)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-28 09:50:24 +01:00
Arnaud Gourlay
ea485a9029 Fix condition check for values_count (#1502) 2023-02-28 09:47:28 +01:00
Arnaud Gourlay
2770a24766 Nested payload filters (#1487) 
* Nested payload filters

* close ToDo + add parsing of multuiple array values

* fmt

* improve testing nested arrays

* fix NumericIndex to accumulate points_to_values mapping

* revert numberic index + strict array field access

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2023-02-23 15:57:12 +01:00
dependabot[bot]
a2ff17dc7f Bump memmap2 from 0.5.9 to 0.5.10 (#1491) 
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.5.9 to 0.5.10.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases)
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.5.9...v0.5.10)

---
updated-dependencies:
- dependency-name: memmap2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-23 10:56:20 +01:00
dependabot[bot]
349bdfe40f Bump memmap2 from 0.5.8 to 0.5.9 (#1479) 
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.5.8 to 0.5.9.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases)
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.5.8...v0.5.9)

---
updated-dependencies:
- dependency-name: memmap2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-20 09:48:15 +01:00
Arnaud Gourlay
3803af502d Access payload key through nested Array (#1465) 
* Access payload key through nested Array

* support removal through array index

* propagate type changes

* fmt

* avoid allocating Vec of one element
 2023-02-17 17:41:28 +01:00
Luis Cossío
c15fec2a2a feat: add "any" to "match" statements (#1466) 
* feat: add "any" to "match" statements

Adds support for using "any" to match on a list of different possible values

* enable proper counting in cardinality estimation

* fmt

* upd OpenAPI

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2023-02-15 11:11:20 +01:00
dependabot[bot]
97aaeb100f Bump rocksdb from 0.20.0 to 0.20.1 (#1464) 
Bumps [rocksdb](https://github.com/rust-rocksdb/rust-rocksdb) from 0.20.0 to 0.20.1.
- [Release notes](https://github.com/rust-rocksdb/rust-rocksdb/releases)
- [Changelog](https://github.com/rust-rocksdb/rust-rocksdb/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-rocksdb/rust-rocksdb/compare/v0.20.0...v0.20.1)

---
updated-dependencies:
- dependency-name: rocksdb
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-10 20:30:27 +01:00
dependabot[bot]
1204682000 Bump rocksdb from 0.19.0 to 0.20.0 (#1461) 
* Bump rocksdb from 0.19.0 to 0.20.0

Bumps [rocksdb](https://github.com/rust-rocksdb/rust-rocksdb) from 0.19.0 to 0.20.0.
- [Release notes](https://github.com/rust-rocksdb/rust-rocksdb/releases)
- [Changelog](https://github.com/rust-rocksdb/rust-rocksdb/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-rocksdb/rust-rocksdb/compare/v0.19.0...v0.20.0)

---
updated-dependencies:
- dependency-name: rocksdb
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix api changes

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>
 2023-02-10 10:22:34 +01:00
Andrey Vasnetsov
ac251358b9 Fix flacky windows test (#1448) 
* use async sleep

* fmt

* disablee test_parallel_graph_build on CI on windows untill there is a way to reproduce
 2023-02-09 20:48:00 +01:00
Roman Titov
1be2f67a2e Reimplement ExtendedPointId deserialization (#1453) 
Reimplement `ExtendedPointId` deserialization, to produce better error message when deserialization fails
 2023-02-10 02:36:36 +10:00
Andrey Vasnetsov
26551c64ef rollback custom deserialization for point id 2023-02-08 14:13:57 +01:00
Andrey Vasnetsov
39cc731611 Windows support (#1432) 
* windows-support

* fmt

* install protoc with action

* upd WAL revision
 2023-02-06 14:00:23 +01:00
dependabot[bot]
9f6fccece0 Bump fs_extra from 1.2.0 to 1.3.0 (#1428) 
Bumps [fs_extra](https://github.com/webdesus/fs_extra) from 1.2.0 to 1.3.0.
- [Release notes](https://github.com/webdesus/fs_extra/releases)
- [Commits](https://github.com/webdesus/fs_extra/commits)

---
updated-dependencies:
- dependency-name: fs_extra
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-03 19:51:10 +01:00
Andrey Vasnetsov
e3066b96fc allow to overwrite max number of cpus with env variable (#1425) 2023-02-02 14:44:03 +01:00
dependabot[bot]
7abb9754b3 Bump uuid from 1.2.2 to 1.3.0 (#1420) 
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.2.2 to 1.3.0.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/1.2.2...1.3.0)

---
updated-dependencies:
- dependency-name: uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-01 19:59:41 +01:00
Arnaud Gourlay
68b34b1550 Implement Deserialize for ExtendedPointId to improve error reporting (#1416) 
* Implement Deserialize for ExtendedPointId to improve error reporting

* better error message
 2023-01-31 19:57:39 +01:00
Arnaud Gourlay
4c0f849749 Clippy rust 1.67 (#1406) 
* inline format! args

* inline format! args

* explicit lifetime could be elided

* fmt
 2023-01-26 17:48:52 +01:00
Roman Titov
6b66d33398 Implemented response resolver (#1370, #1381) 
- Implement `Resolve` trait that "merges" multiple `retrieve`/`scroll_by`/`search` responses
  into a single response, to ensure it's consistent across multiple nodes in the cluster
- Implement tests for the `Resolve` trait implementation
- Add a few additional derives required for the `Resolve` trait implementation

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2023-01-25 05:24:55 +10:00
Arnaud Gourlay
064a1389b9 Name rayon hnsw threads for clarity (#1399) 2023-01-24 19:36:16 +01:00
Andrey Vasnetsov
ca9545366d Fix flacky hnsw (#1383) 
* use fixed seed in random generator

* fmt
 2023-01-23 10:10:46 +01:00
Andrey Vasnetsov
5877039380 Faster snapshot creation (#1353) 
* deduplication after proxy snapshot recovery

* revert changes in apply_points_to_appendable

* clippy fixes

* spawn blocking in snapshot creation
 2023-01-20 23:52:45 +01:00
Andrey Vasnetsov
a8dad69dba enable payload field anonymization (#1368) 2023-01-19 10:23:17 +01:00
Arnaud Gourlay
8c2c4f08eb Remove some blocking IO calls on Tokio threads (#1366) 2023-01-19 07:58:45 +01:00
dependabot[bot]
0f03526a78 Bump geohash from 0.12.0 to 0.13.0 (#1357) 
Bumps [geohash](https://github.com/georust/geohash.rs) from 0.12.0 to 0.13.0.
- [Release notes](https://github.com/georust/geohash.rs/releases)
- [Commits](https://github.com/georust/geohash.rs/compare/v0.12.0...v0.13.0)

---
updated-dependencies:
- dependency-name: geohash
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-01-17 08:47:18 +01:00
Andrey Vasnetsov
0c1833db7f Disable extra segments checks (#1352) 
* optional segment check

* alternative implementation
 2023-01-16 12:05:32 +01:00
Roman Titov
ced0abb4c7 Fix segment snapshotting (#1321) (#1334) 
* WIP: Fix `Segment::take_snapshot`

TODO:
- This commit, probably, breaks snapshotting of segments with memmapped vector storage
- `ProxySegment::take_snapshot` seems to potentially similar bug

* WIP: Fix `Segment::take_snapshot`

- Fix snapshotting of `StructPayloadIndex`
- Fix snapshotting of segments with memmapped vector storage
- Temporarily break `ProxySegment::take_snapshot`

* Fix `ProxySegment::take_snapshot`

* Remove `copy_segment_directory` test

* nitpicking

* clippy fixes

* use OperationError::service_error

* Cleanup `TinyMap` trait bounds and derive `Debug`

* Fix `test_snapshot` test

- Derive `Debug` for `NamedVectors`

* Move utility functions from `segment.rs` to `utils` module

* Contextualize `segment::utils::fs::move_all` a bit more carefully

* Fix a typo

* add backward compatibility with old snapshot formats

* fmt

* add snapshot for compatibility test

* git lfs is a piece of shit

* Nitpicking

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2023-01-13 11:44:42 +01:00
Arnaud Gourlay
72dadee806 Repair segment data while checking consistency (#1314) 
* Repair segment data while checking consistency

* code review

* code review

* code review

* code review drop_by_internal_id

* refactoring

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2023-01-10 20:02:43 +01:00