Commit Graph

160 Commits

Author SHA1 Message Date
tellet-q
bc2d47276b tests: Add more test cases 2024-06-28 16:31:22 +02:00
tellet-q
2a4a6bdb9a tests: Add 2 cases (no query, no prefetch and on filtered prefetch). 2024-06-28 16:31:18 +02:00
tellet-q
5f02bad5a1 Raise ValueError if vector name not found 2024-06-28 15:50:23 +02:00
tellet-q
f2330ed228 Fix make negative test fail 2024-06-28 15:44:10 +02:00
tellet-q
d6211808bc Fix positive flow and add negative tests 2024-06-28 15:42:00 +02:00
tellet-q
2514a4aa32 tests: Add lookup_from test 2024-06-28 14:27:46 +02:00
Arnaud Gourlay
a34262d53c Merge pull request #667 from qdrant/fix-local-maxsim
fix: fix maxsim
2024-06-28 11:59:13 +02:00
George Panchuk
38fe29633b fix: wait for upload collection 2024-06-28 12:35:33 +03:00
George
6400c8316b fix: increase context search limit (#666)
* fix: increase context search limit

* fix: reduce collection size for context test
2024-06-28 10:10:44 +03:00
Arnaud Gourlay
37e167d2e2 add init tests for sparse & multi 2024-06-27 19:59:08 +02:00
Luis Cossío
c3e8ec00e5 Local mode for Query 2024-06-27 19:46:25 +02:00
Andrey Vasnetsov
6ff600c94b rollback fast-embed query method (#658)
* rollback fastembed query method

* un-deprecate query

* fix: regen async, roll back generator tmp workaround

* fix: don't skip fastembed tests

* new: add query batch points interface, fix minor bugs (#660)

* new: add query batch points interface, fix minor bugs

* fix: do not exclude ids from lookup collection from search (#661)

* Update qdrant_client/qdrant_client.py

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>

* fix: regen async

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
2024-06-26 17:13:40 +03:00
George
00da88e947 new: add query interface draft (#655)
* new: add query interface draft

* fix: temporarily update fastembed generator

* review fixes

* generate async client

* generate async client

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2024-06-25 09:21:28 +02:00
George
31aca81b88 new: update prefetch structure, fix start from pydantic v1 bug (#656)
* new: update prefetch structure, fix start from pydantic v1 bug

* fix: fix missing multivec_config replacement in local mode
2024-06-24 12:19:12 +02:00
George
9ea4db1291 new: add conversions pt.1 (#654)
* new: add conversions pt.1

* fix: fix multivec param name

* fix: fix other multivec config param

* fix: fix sparse index config conversion, multivector config typo

* fix: fix empty vectors conversion, fix sparse vector test

* fix: fix legacy sparse vectors usage

* new: add convert query interface

* fix: fix order value conversion

* fix: fix conversion bugs, add tests

* fix: increase coverage

* fix: increase test coverage 2

* fix: fix bugs, increase test coverage

* fix: fix query isinstance check in tests

* minor refactoring

* new: convert flat filter

* fix: add missing min should check

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2024-06-23 15:36:25 +02:00
Andrey Vasnetsov
815a784469 Update openapi v1 10 x (#622)
* update api with conversions

* introduce IDF modifier for sparse vectors in local mode

* generate stuff

* regen async

* Update tests/fixtures/points.py

* fix: address review comments

* upd models

* fix: update grpc overwrite payload conversion, fix order by conversion (#653)

---------

Co-authored-by: George <george.panchuk@qdrant.tech>
2024-06-20 17:55:35 +02:00
George
af4941ec51 fix: fix integer index conversion (#640) 2024-06-07 17:29:33 +02:00
Luis Cossío
597731f0ca allow string RecommendStrategy with prefer_grpc=True (#637) 2024-06-05 19:42:51 -04:00
Andrey Vasnetsov
720746321f V1.9.x (#608)
* update models

* up backward compatibility version

* refactoring: address review comment

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2024-04-22 14:28:02 +02:00
George
96db3864ca new: convert nan and inf in local mode to none (#603)
* tests: update nan payload tests with pydantic 2.7

* fix: update local mode to mimic pydantic>=2.7.0

* fix: fix mypy

* fix: do not convert bytes

* fix: allow nan values in payload in local mode
2024-04-19 15:55:02 +02:00
Andrey Vasnetsov
67103fca36 do not use vectors_count in test and in local mode (#600)
* do not use vectors_count in test and in local mode

* fix: remove indexed vectors count comparison in sparse vector tests

* tests: return tests for full snapshot, increase timeout

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2024-04-18 17:26:33 +02:00
George
16814b4bb9 fix: fix field schema type conversion (#605)
* fix: fix field schema type conversion

* fix: fix names

* fix: fix payload schema type datetime call
2024-04-17 18:36:02 +02:00
Olli-Pekka Heinisuo
d957ca7ba7 Feat: bearer token authentication support (#591)
* bearer token authentication provider support

* add tests and checks, move auth file to separate dir

* fix error message

* remove locks

* rename var

* refactoring: refactor exceptions, fix mypy

* fix: regen async

* tests: extend token tests to check token updates

* new: add warning when auth token provider is used with an insecure connection

* fix: propagate auth token to rest client even with prefer_grpc set

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2024-04-16 15:42:21 +02:00
George
12141f61e6 fix: prevent access to a closed instance methods in local mode (#594)
* fix: prevent access to a closed method instance in local mode

* fix: regen async
2024-04-15 20:00:08 +02:00
Apostolos Matsagkas
fdcbf41ba8 fix: Fix read consistency when performing batch search (#589)
* Fix read consistency when performing batch search

* fix: regen async

* fix: add tests to check read consistency

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2024-04-10 18:11:09 +02:00
George
c8aa21c83d new: add sparse vectors support in discovery and recommend in local mode (#571)
* new: add sparse vectors support in discovery and recommend in local mode

* chore: remove redundant file

* chore: remove another redundant file

* fix: fix type hint for python3.8

* fix: remove redundant types from check

* refactoring: remove redundant union

* refactoring: remove redundant import

* fix: fix type hint

* fix: fix type hint

* refactoring: improve exception description

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>

* refactoring: improve exception description

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>

* refactoring: address review comments

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
2024-04-05 12:13:20 +02:00
George
bd6e755619 fix: fix date object grpc conversion (#580)
* fix: fix date object grpc conversion

* fix: fix date to datetime conversion in filters in local mode

* fix: fix date isinstance check
2024-04-05 11:52:48 +02:00
George
9e56ed62a8 fix: convert date to datetime in local mode to compare with datetime (#576) 2024-04-04 20:33:50 +02:00
George
2e6233800b fix: fix sparse vector conversion (#569) 2024-04-03 11:29:01 +02:00
George
af2f2e92b1 new: check for nan vectors in local mode (#560)
* new: check for nan vectors in local mode

* tests: add tests for search with query containing nans

* refactoring: move sparse nan test to test_sparse_search

* tests: add tests for queries with nans in recommendation api

* tests: add tests for query with nan in discovery

* refactoring: update imports in discovery tests
2024-04-02 17:05:40 +02:00
George
a49b6f679b fix: fix QueryVector isinstance check in local mode (#562)
* fix: fix QueryVector isinstance check in local mode

* fix: update query vector type, fix invalid vector type in search in local mode

* fix: remove get_args_subscribed type hint, restore QueryVector, remove redundant array check
2024-04-02 16:12:21 +02:00
George
483578a1b3 fix: fix update sparse vectors local mode (#559)
* fix: fix update sparse vectors local mode

* fix: validate sparse vector before update

* fix: update update_named_vectors signature
2024-04-02 15:58:48 +02:00
Andrey Vasnetsov
822aefa1c1 WIP: hybrid search with fastembed (#553)
* WIP: hybrid search with fastembed

* hybrid queries with fastembed

* test for hybrid

* fix typo

* new: extend hybrid search tests, fix mypy, small refactoring (#554)

* refactor: align model name parameters in setters, update tests

* fix: fix async

* fix: add a good test, fix sparse vectors in query batch

* refactoring: reduce branching, refactor fastembed tests

---------

Co-authored-by: George <george.panchuk@qdrant.tech>
2024-03-27 13:57:58 +01:00
George
303de4423e Add key set payload (#536)
* fix: fix mypy and pyright

* fix: fix nested key case, add tests

* fix: fix mypy

* fix: fix mypy again

* new: extend jsonpath support, update set and get value by key in local mode

* fix: uncomment tests, fix set payload call, fix corner case

* refactor: split payload_value extractor into several files

* docs: update docstring

* refactor: refactor local mode payload setter (#544)

* refactor: refactor local mode payload setter

* fix: fix mypy

* fix: return docstring

* fix: address review issues

* fix: fix extraction from by invalid keys

* fix: move payload tests to a separate folder to be recognized by pytest

* tests: add nested array filters

* review fixes

* fix: fix async

* fix: set payload by key handle escaped quotes

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2024-03-26 20:54:03 +01:00
George
9221338fcc new: propagate timeout from methods to httpx (#534)
* new: propagate timeout from methods to httpx

* refactor: refactor timeout propagation (#535)

* tests: add timeout test

* refactor: remove redundant kwargs
2024-03-13 21:01:24 +01:00
George
848c63c1fd fix: make datetime range conditions tz aware (#538)
* fix: do not set tz if it is not provided in local mode

* refactor: remove commented out code

* fix: rollback tzinfo, add tzinfo to conditions in local mode

* fix: slightly increase chance of datetime payload generation
2024-03-13 15:51:49 +01:00
Nirant
fc4b3cf9e2 Upgrade FastEmbed Version (#493)
* Update fastembed to v0.2.1

* chore(qdrant_fastembed.py): update DEFAULT_EMBEDDING_MODEL

* fix(fastembed integration): upgrade to latest version

* Prefer black over ruff

* Prefer black over ruff

* Remove hardcoded directory structure from Qdrant Client checks

* new: deprecate current default model, deprecate max token length, update fastembed

* fix: make embedding_model_name method sync

* fix: update poetry lock

* refactor: use list_supported_models() (#501)

* fix: fix fastembed check

* fix: fix fastembed class var assignment

* fix: remove fastembed deprecation from qdrant client (#524)

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
Co-authored-by: Anush <anushshetty90@gmail.com>
2024-03-05 21:09:31 +01:00
George
1438136898 fix: turn on test no fastembed (#522)
* fix: turn on test no fastembed

* fix: ignore input

* Update fastembed tests job name

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
2024-03-04 16:16:13 +01:00
George
462e6cea1c new: add collection exists interface (#518)
* new: add collection exists interface

* tests: fix version comparison

* tests: add collection exists type stub, add async test
2024-03-02 23:03:49 +01:00
George
bc714a5521 new: add min should clause (#519)
* new: add min should clause

* fix: local mode fix payload filter type hint

* tests: extend conversion

* fix: fix param name

* fix: fix grpc structure, update name
2024-03-02 22:46:53 +01:00
Luis Cossío
46c17e170f Add support for datetime ranges (#517) 2024-03-01 20:57:27 -03:00
Armaghan
3ad05b3671 feat: Expose Setting for GRPC Channel-Level Compression at Client Side (#480)
* expose grpc channel-level compression settings in base functions

* expose grpc channel-level compression settings in remote classes

* expose grpc channel-level compression settings in client

* raise TypeError for compression

* added test cases for grcp channel-level compression

* move grpc_compression parameter from client's signature to **kwargs

* use grpc.Compression instead of creating new enum qdrant.grpc.Compression in qdrant/grpc/__init__.py

* refactor grpc_compression type hint

* fix: Compression instead of grpc.Compression in type hint

* tests: move and update tests

* chore: remove magic method

* fix: fix async client generator, update precommit dependencies

* fix: update isort options

* fix: update dev dependencies

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2024-03-01 18:31:16 +01:00
Luis Cossío
09d51e65af Remove deprecated field conversions (#502) 2024-03-01 13:12:53 -03:00
Luis Cossío
332777db2a Local mode of order_by parameter in scroll + datetime support (#491)
* generate rest client (only points_api)

* generate grpc client (points part only)

* add local mode implementation

* update collections_api.py

* add grpc conversions

* fix problematic StartFrom Union order

* add basic congruence test

* route order_by in qdrant clients

* fix local mode `start_from` logic

* add library stubs for dateutil

* generate async client

* generate the rest of the rest client 👻

* test datetime values too

* add int Range and int StartFrom

* update points.proto and points_pb2.py

* fix conversion of range interface

* add conversion fixtures and use better conversion of datetime to timestamp

* remove integer range

* restore `value_by_key()`

* OrderBy grpc to rest conversions

* generate async client

* use `OrderByInterface` instead of only `OrderBy`

* - use equivalent qdrant core datetime parsing,
- use type alias for OrderingValue, instead of custom class,
- better random date generation

* Drop custom datetime parser implementation

* nit fixes

* fix flakiness of the congruence test by subsorting

* better datetime to timestamp conversion

* uncomment conversion fixtures, add Direction

* restore poetry.lock and remove dateutil dep

* poetry lock --no-update

* rename datetime.py to datetime_utils.py

* add StrictInt to StartFrom union

* use more date formats in fixtures

* move conversion test to `test_validate_conversions.py`

* drop `%:z` formats

* add more complex timezones to payload fixtures, fix deserialization

* update datetime parsing test
2024-03-01 09:51:39 -03:00
Arnaud Gourlay
6df4bd076a Fix gRPC conversion for sparse search batch (#484)
* Fix gRPC convertion for sparse search batch

* fix conversion & add test
2024-02-08 18:18:56 +01:00
George
4adbabec7e fix: fix implicit ids in upload collection with paralell > 1 (#460)
* fix: fix implicit ids in upload collection with paralell > 1

* fix: fix type hints

* fix: remove redundant code, simplify type hints

* fix: remove redundant import

* fix: fix batching

* fix: replace generator with list comprehension

* fix: sorry, I was wrong

* tests: update tests

* fix: extend upload records and upload points tests
2024-01-31 16:38:07 +01:00
George
0635f10514 fix: add missing methods to type stub (#454) 2024-01-30 22:58:37 +01:00
George
fb75375e40 fix: convert some types to python jsonable types (#462)
* fix: convert some types to python jsonable types

* fix: fix nan validation

* fix: update overwrite payload and set payload

* tests: update not jsonable payload tests
2024-01-30 22:58:18 +01:00
George
a2adc2d2ad new: deprecate upload records, update tests, prohibit migration of co… (#447)
* new: deprecate upload records, update tests, prohibit migration of collections with custom shards

* Update qdrant_client/qdrant_client.py

* Update qdrant_client/async_qdrant_client.py

* new: replace autogenerated int ids with uuids (#448)

* fix: remove redundant kwargs

* fix: regen async

* fix: remove redundant import

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2024-01-19 20:11:31 +01:00
George
e90379f287 Fix sort sparse vectors (#442)
* fix: fix sparse vector sorting

* fix: fix sparse vector sort
2024-01-19 19:38:37 +01:00