Commit Graph

43 Commits

Author SHA1 Message Date
George Panchuk
698db1d33f new: drop python3.9, replace union and optional with | where possible 2025-11-14 03:20:36 +07:00
George
fbf72f4a31 new: relax test comparison (#1039)
* new: relax test comparison

* rollback: rollback is context search addition

* rollback: return redundant parentheses

* Sparse no overlap distance computation (#1040)

* fix: fix sparse distance computation for recommend, discovery and context queries

* fix: use empty_is_zero

* add zero case to score cmp

* add snapshot saving on failure

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2025-07-15 13:30:23 +02:00
George
a55097ba1c new: add hf token to ci, remove redundant envs, run local inference w… (#951)
* new: add hf token to ci, remove redundant envs, run local inference w/o fastembed

* fix: fix env in ci
2025-04-15 15:06:09 +03:00
Hossam Hagag
5f69b4f28b Unlock numpy ci (#946)
* ci: Unlcok numpy

* fix: Fix type hints

* chore: Remove comment

* fix: Fix type hints
2025-04-15 10:31:48 +03:00
George
5a454a7402 Recommend sum score (#940)
* new: add sum_scores strategy, fix lookup from in query_batch_points, fix euclid between multivectors local mode

* new: recommend average multivectors, fix tests, add new tests for multivectors

* fix: address review comments

* fix: regen async

* tests: uncomment tests

* new: update poetry lock, require pydantic >=2.11 for populate inspection cache

* fix: tmp disable euclidean multivec search tests, fix multivec with nan test

* fix: align euclid and manhattan with core, add tests

* fix: fix type hints

* tests: use atol in comparison

* tests: remove manhattan tests

* tests: reduce number of multivectors

* remove float comparison relaxation

* explain why it is different for euclid and manhattan

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
2025-04-07 11:45:33 +03:00
George
3c0d7d1bdd bump version to 1.13.1-dev, update ci (#883) 2025-01-20 12:40:47 +01:00
George
baa3e264c8 Drop python 3.8 (#848)
* new: drop python 3.8 support, update type hints

* fix: remove 3.8 from ci

* fix: update type hints

* fix: make netlify use python3.10

* fix: try python3.9 for sphinx

* debug: try updating sphinx

* new: bump ffastembed to 0.4.2

* fix: install numpy<2 for mypy

* fix: install numpy via poetry
2024-11-15 15:54:32 +01:00
George
bf14069f65 new: add python3.13 to ci (#849) 2024-11-15 10:17:49 +01:00
George
c152e3c7bf Agnostic local inference (#799)
* new: update previous qdrant version

* new: rollback rest client as types moved to the core

* new: add model_fields_set pydantic compat

* new: add Document to models

* new: add local inference inspectors

* new: add inference functionality to qdrant client

* fix: fix type hints

* fix: fix type hints

* fix: fix type hints

* new: remove _cloud_inference arg from methods

* refactor

* tests: update tests, add path check

* fix: fix type hint

* fix: fix type hint

* fix: resolve plain query

* fix: remove redundant checks

* fix: fix resolve query doc

* tests: add test case for several docs in prefetch

* fix: fix tests

* new: forbid cloud inference with local mode

* fix: fix is_query assignment, fix embed dict

* tests: fix sparse vector values comparison

* fix: regen async

* fix: embed batch

* tests: extend local inference tests

* tests: add update operations and update vectors tests

* tests: remove redundant tests, add plain query points test

* fix: fix type hint

* fix: add missing import

* fix: yet another type hint fix

* tests: add query points groups and query batch points tests

* fix: do not try to embed a vector, comment missed disabled test stmt

* new: add late interaction embeddings without add and query methods

* fix: fix LITE import

* fix: fix problem with retrieving not available path, enable tests

* fix: fix exception message

* refactor: update type hint

* fix: update type hints

* chore: add docstrings to embed methods

* refactor: simplify inspector init

* fix: fix type hint

* new: reuse parser, fix type import

* refactor: rename parser, add docstrings

* fix: fix type hint

* fix: fix type hint

* fix: fix type hint

* refactor: smol refactor, docstrings

* fix: fix inspect embed

* deprecation warning for grpc.PointStruct

* new: populate inspections cache (#823)

* new: populate inspections cache

* fix: fix type hints

* fix: regen async

* fix: fix type hint

* fix: fix typo in filename

* fix: fix points check

* fix: fix type hint
2024-10-29 15:48:23 +01:00
George
1eb9b78f5d new: update dev version (#820)
* new: update dev version

* fix: update qdrant version for backward compatibility in ci
2024-10-25 17:06:02 +02:00
Arnaud Gourlay
44bbded8f2 universal-search: Query Group API and local mode (#690)
* universal-search: Query Group API and local mode

* requires different logic for gRPC error assertion

* you come to me at runtime for a compile time issue

* suddenly throwing a different error

* test more group key types

* extend limit of prefetches during group_by

* rescoring is the issue

* one problem at a time please

* code review

* regen clients

* code review

* add lookup_from to query_points_groups

* test with_lookup

* test and fix gRPC

* drop dedicated conversion

* Update qdrant_client/qdrant_client.py

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>

* regen async

* Distribution-based score fusion in local mode (#703)

* pre-implement dbsf

* add dbsf congruence tests

* mypy lints

* add conversions

* tests: add test for dbsf conversion

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>

* Random sample in local mode (#705)

* pre-implement random sampling

* generate models

* add conversions and tests

* fix mypy lints

* tests: add test for sample random conversion

* use camelcase Sample.Random

* review fixes

* fix mypy

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>

* fix: add type ignore for mypy

* fix: fix type hints for 3.8

* fix: do not run mypy on async client generator in CI, simplify condition

* Grpc comparison in tests (#726)

* add parametrized fixture for using grpc too

* compare grpc and http without running each setup twice

* fix: fix exception types in invalid types test

* fix: remove random seed which led to a erroneous sequence

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2024-08-09 17:03:28 +02:00
George
0fcf4526ed tests: add tests for query points with document (#688) 2024-07-30 13:55:39 +02:00
Andrey Vasnetsov
720746321f V1.9.x (#608)
* update models

* up backward compatibility version

* refactoring: address review comment

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2024-04-22 14:28:02 +02:00
George
1438136898 fix: turn on test no fastembed (#522)
* fix: turn on test no fastembed

* fix: ignore input

* Update fastembed tests job name

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
2024-03-04 16:16:13 +01:00
George
d285bc7115 new: update qdrant version in backward compatibility tests (#513) 2024-03-01 15:46:23 +01:00
Arnaud Gourlay
016032c28f Sparse vectors API and local mode (#378)
* draft

* thread sparse_vectors_config through

* test ingest sparse data

* make async_client consistent for create collection

* regen async

* align client hierarchy

* do not give up

* disable async consistency check for now

* regen

* workaround async client breakage

* generate gRPC

* new healtcheck proto files

* fix gen branch

* fix grpc generation - had to reinstall to downgrade grpcio

* bump test docker image

* wait for update when migrating

* regen REST + update config

* add sparse search congruence

* congruence scroll sparse vectors

* congruence test retrieve sparse

* more sparse tests and waiting on remote

* remove healthcheck service

* misc

* reinstate len check

* fix scoring if no overlap

* sparse distance as numpy float without rounding

* fix payload selector

* fix payload exclude

* dedicated collection name to not collide with openapi engine tests

* fix with_vector, scoring no overlap, closeness assetion

* convert float only once

* regen gRPC to exclude health_check

* code review cleanup

* validate and sort sparse vector before persistence

* regen REST

* test deletes and update with fix

* fix async client

* fix batch size

* fix async client

* don't give up

* remove validated todo

* fix sparse vectors loading test

* fix sparse vectors loading test

* rm empty

* mention sparse vectors in the fake local info

* fix upload collection wait propagation

* fix async. init_from tests

* resync async

* rework test sparse vector generation

* regen clients

* update to dev

* update REST models

* async client

* do not change it yet

* fix migration test

* do not create a sparse configuration by default

* fix backward compatibility by disabling serialization of none fields

* fix type imports

* regen gRPC

* test BatchVectorStruct REST to gRPC conversion

* test gRPC->REST conversion

* sparse grpc tests and conversions fix

* generate async client

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2023-12-02 20:45:49 +01:00
George
6743f28fda new: switch async generator version to 3.10.x (#387)
* new: switch async generator version to 3.10.x

* fix: remove package version update
2023-11-30 12:09:35 +01:00
Andrey Vasnetsov
0f32e578ec update interface and version for fastembed (#340)
* update interface and version for fastembed

* fix types

* fix types

* regen async

* use python 3.11 to check compatibility

* fix docstring

* regen async

* propagate batch size
2023-10-16 18:13:52 +02:00
George
278a68d297 Async qdrant client (#319)
* new: manually implemented async qdrant client

* fix: remove await before sync call

* fix: make upload collection, records and migrate synchronous

* fix: add init method to async client base

* refactoring: remove redundant import

* new: add super().__init__ in qdrant remote, update import in http

* new: mvp async qdrant client generator

* new: fix mypy, update generator script, refactoring

* fix: fix test script

* new: update generator launch script, update async files

* new: refactor async client generator

* refactoring: remove redundant operations, add comments, refactor

* new: add isort, black and autoflake to dev dependencies

* fix: add more checks, fix type hints

* fix: do not check types in async client generator for python3.8

* new: do not type check async_qdrant_fastembed

* fix: fix pyright run

* new: update async client tests

* fix: update versions in CI

* new: update pre-commit python version, update autogenerated files

* new: update generated files, add tests for async generator, update generator script

* fix: exclude generator test from 3.8

* fix: fix python version condition

* fix: add python target version for black

* fix: generate async client only on python 3.9
2023-10-13 22:26:53 +02:00
Andrey Vasnetsov
89843eaad0 python 3.12 support (#326) 2023-10-06 15:55:10 +02:00
Andrey Vasnetsov
8452f2ae72 fix fastembed for pydantic 1.x (#285)
* fix fastembed for pydantic 1.x

* add extras setup in readme

* move fastembed from group to extras

* lock
2023-09-11 15:09:12 +02:00
Andrey Vasnetsov
aed7ecaefb sync API (#280)
* sync API

* fix tests

* backward compatibility version up
2023-09-07 11:29:10 +02:00
generall
395321d8c6 init dev commit - do not cherry-pick 2023-08-21 14:08:08 +02:00
Nirant
536b0eeb22 Add fastembed integration (#210)
* Add skeleton for fastvector registration

* * fix(qdrant_client.py): fix import statement for QdrantClientMixin from fastembed instead of fastvector
* fix(qdrant_client.py): remove unnecessary print statement

* * chore(README.md): add bullet point for extensive test coverage
* feat(README.md): add section for Fast Embeddings + Simpler API
* fix(README.md): fix formatting of code block and update code example for Qdrant Client usage
*

* * fix(qdrant_client.py): remove unused imports and code
* feat(qdrant_client.py): add support for adding and querying documents with fastembed installed
* test(qdrant_client.py): add tests for adding and querying documents with and without fastembed installed

* * feat(qdrant_client.py): add support for fastembed.qdrant_mixin.QdrantAPIExtensions for upsert_docs and search_docs methods

* * fix(qdrant_client.py): remove unused import of SearchParams
* feat(qdrant_client.py): add return type hint to QdrantClient.search_docs method

* * fix(qdrant_client.py): fix import statement for SearchParams class in qdrant_client.py

* * feat(models.py): add QueryResponse model for handling query responses

* * feat(qdrant_client.py): add batch_iterable method to QdrantClient class

* * chore(qdrant_client.py): reformat code for better readability

* * chore(test_qdrant_client.py): add type hinting to fastembed_setup fixture
* test(test_qdrant_client.py): add test for client_close function

* * chore(.gitignore): add pattern for ignoring .tar.gz files

* * refactor(test_qdrant_client.py): remove unused imports and variables
* chore(test_qdrant_client.py): reformat import statements for better readability

* * chore(test_qdrant_client.py): remove unused fixtures and test cases

* * chore(qdrant_client.py): remove unused imports

* Add test skeleton

* * chore(.gitignore): add local_cache/*/* to ignore local cache files

* * fix(qdrant_client.py): change import statement for fastembed to use DefaultEmbedding instead of FlagEmbedding
* refactor(qdrant_client.py): refactor code to remove unnecessary loop

* * fix(qdrant_client.py): import missing models in qdrant_client.py
* feat(qdrant_client.py): add support for search parameters in search method
* refactor(qdrant_client.py): refactor indexing logic to handle embeddings correctly

* * test(test_fast_embed.py): add assertion to verify count of documents in collection
* test(test_fast_embed.py): remove unused code
* test(test_fast_embed.py): add TODO comment for future assertions

* * chore(pyproject.toml): add optional fastembed group to poetry dependencies
* feat(pyproject.toml): add fastembed dependency to fastembed group

* * chore(qdrant_client.py): add check for fastembed installation in QdrantClient constructor

* * fix(test_fast_embed.py): skip test if FastEmbed is not installed
* fix(test_fast_embed.py): add default values for test_no_install parameters
* fix(test_fast_embed.py): skip test if FastEmbed is installed

* Delete unused usage file

* * chore(models.py): remove unused QueryResponse class
* feat(qdrant_client.py): add QueryResponse class

* * refactor(qdrant_client.py): organize imports and fix type hinting for QueryResponse class

* review fixes

* make mypy happy

* review fixes

* fix mypy

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2023-08-18 15:17:46 +02:00
Nirant
394407afb4 Remove MacOS from PR CI (#256)
* * chore(integration-tests.yml): remove MacOS from the matrix of operating systems
* chore(integration-tests.yml): remove setup for docker on MacOS

* enable macos tests on dispatch

* rename workflow

* do not run congruence tests on macos

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2023-08-17 12:55:05 +02:00
Andrey Vasnetsov
31252b7650 V1.4.0 (#238)
* new models conversions

* new models conversions

* Add update collection test

* ignore collection update test

* fix type check

* fix python 3.7

---------

Co-authored-by: timvisee <tim@visee.me>
2023-08-03 11:02:11 +02:00
Andrey Vasnetsov
13b0a9ec63 MacOS CI (#226)
* increase collection timeout

* fix

* more timeouts
2023-07-21 22:13:47 +02:00
Kacper Łukawski
b9a7a15cee [WIP] Enable testing on macos (#199)
* Enable testing on windows and macos

* Add platform to the name

* Add Docker on MacOS

* Temporarily disable tests on Windows

* Explicitly set ports to be opened

* Add problematic case for MacOS

* Increase the timeout in tests
2023-07-07 13:31:32 +02:00
Andrey Vasnetsov
1ec45bd97b V1.3.0 (#194)
* wip: implement release checnges

* lookup tests

* fix mypy
2023-06-23 14:52:04 +02:00
Andrey Vasnetsov
329b4150cf v1.2.0 (#173)
* wip: interface for new APIs

* upd docker version

* remove duplicated comments + extent interface

* start implementing remote client + remove duplicated comments there

* implement remote methods

* quantization converter

* extend coverege

* wip: saving and loading of optional vectors

* update and delete vectors test

* rm unused imports

* nested filters

* fix mypy

* fix pyright

* wip: add search groups and recommend groups (#174)

* wip: add search groups and recommend groups

* fix: fix signature

* fix: fix mypy

* simplify group-by and condition checks

* fix tests

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>

* add match except condition + improve group-by tests

* fix: lock typing exchanges due to broken release

* tests: add maybe and maybe_null checks to group tests

* new: add new methods to type stubs

---------

Co-authored-by: George <george.panchuk@qdrant.tech>
2023-05-24 11:06:10 +02:00
Andrey Vasnetsov
8840f2b5bf V1.1.0 (#134)
* WIP: generate models and conversions

* update quantization param + match any condition + test

* update versions

* consistent naming

* use old grpc types generator

* fix: update type hints

* disable match-any in backward comp test

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2023-03-17 20:06:10 +01:00
George
c57cebf8f3 new: more python versions for type check ci, fix numpy typing for 3.7 (#123)
* new: more python versions for type check ci, fix numpy typing for 3.7

* fix: fix ci python version
2023-02-28 16:53:49 +01:00
Ewald Enzinger
d7668ccf39 Increase Python version limit to <3.12 (#119)
* Increase Python version to <3.12

* Add Python 3.11 to integration tests GH action

* specify numpy versions

* fallback for NDArray type

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2023-02-28 01:00:55 +01:00
George
d35a2c21da Ci type checkers 2 (#104)
* fix: update missing return types and other mypy complaints, add mypy and pyright

* new: add mypy and pyright to ci, update type hints

* fix: add types group in pyproject.toml, add types-protobuf

* fix: fix typo

* fix: fix return types in api
2023-02-09 15:27:39 +04:00
Andrey Vasnetsov
c3389e3c20 update api to v1.0.0 (#101)
* update api

* upd openapi fix bugs

* pylance fix

* fix: add TypeAlias to some common types

* fix: update models import for pylance

* fix: fix Batch test

* fix: update mypy config

* fix: define Payload before Batch

* fix: add missing .result to requests, add assert None for responses (#102)

* pylance fix in generator

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2023-02-08 11:11:25 +01:00
Andrey Vasnetsov
8502755352 update to v0.11.5 (#81)
* update to v0.11.5

* fix backward compatibility

* fix backward compatibility again
2022-12-06 21:41:16 +01:00
Andrey Vasnetsov
6d00eb4117 enable test on multiple OS 2022-09-19 21:21:38 +02:00
Andrey Vasnetsov
34fdf028a9 remove deprecated api fields (#72) 2022-09-19 16:53:04 +02:00
Kacper Łukawski
a7153c28a5 Upgrade typing-extensions to ^4.0.0 (#57)
* Upgrade typing-extensions to ^4.0.0

* Run the integration tests on different python versions

* upd pytest

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2022-08-08 14:11:29 +02:00
Andrey Vasnetsov
0b04f9f3b1 automatic validation scaffold (#38)
* automatic validation scaffold

* grpc -> http for filter

* More conversions

* More conversions

* add all fixtures (#39)

* add rest-to-grpc convertions (#40)

* More conversions

* full coverage for conversions

* fix imports

* add exec permissions

Co-authored-by: M. Yusuf Sarıgöz <yusufsarigoz@gmail.com>
2022-05-22 11:20:55 +02:00
Andrey Vasnetsov
0b499b0fd5 Upd version - 0.7.0 (#26)
* upd version

* enable CI on all PRs

* update client - json as a payload + typed index (#31)

* update client - json as a payload + typed index

* refactor

* upd docker version in tests

* New condition filters (#33)

* upd rest for new filter conditions

* update gprc client

* proper shutdown on upload error

* rm debug print
2022-04-13 09:49:31 +02:00
Andrey Vasnetsov
8187c3bda3 add integration test pipeline for CI (#20)
* add integration test pipeline for CI

* chmod +x

* upd poetry install
2022-02-11 15:07:54 +01:00
Andrey Vasnetsov
68c50603f4 V0.5.x (#18)
* update API for v0.5.x

* update type for IDs

* upd version

* add publication workflow

* upd deps
2022-02-03 12:29:50 +01:00