Commit Graph

38 Commits

Author SHA1 Message Date
George Panchuk
693d1e48a0 new: replace union with | in client classes 2025-12-02 12:37:40 +07:00
George Panchuk
698db1d33f new: drop python3.9, replace union and optional with | where possible 2025-11-14 03:20:36 +07:00
George
ff7f584d33 new: update models, remove init_from and locks (#1100)
* new: update models, remove init_from and locks

* deprecate: remove init from tests

* deprecate: remove lock tests

* new: convert ascii_folding

* fix: fix type stub

* new: convert acorn

* new: convert shard key with fallback

* new: update grpcio and grpcio tools in generator (#1106)

* new: update grpcio and grpcio tools in generator

* fix: bind grpcio and tools versions to 1.62.0 in generator

* Remove deprecated methods (#1103)

* deprecate: remove old api methods

* deprecate: remove type stub for removed methods

* deprecate: remove old api methods from test_qdrant_client

* deprecate: replace search with query points in test_in_memory

* deprecate: replace search methods in fastembed mixin with query points

* deprecate: replace old api methods in test async qdrant client

* deprecate: replace search with query points in test delete points

* deprecate: replace discover and context with query points in test_discovery

* deprecate: replace recommend_groups with query_points_groups in test_group_recommend

* deprecate: replace search_groups in test_group_search

* deprecate: replace recommend with query points in test_recommendation

* deprecate: replace search with query points in test search

* deprecate: replace context and discover with query points in test sparse discovery

* deprecate: replace search with query points in test sparse idf search

* deprecate: replace recommend with query points in test sparse recommend

* deprecate: replace search with query points in test sparse search

* deprecate: replace missing search request with query request in qdrant_fastembed

* deprecate: replace search with query points in test multivector search queries

* deprecate: replace upload records with upload points in test_updates

* deprecate: remove redundant structs (#1104)

* deprecate: remove redundant structs

* fix: do not use removed conversions in local mode

* fix: remove redundant conversions, simplify types.QueryRequest

* deprecate: replace old style grpc vector conversion to a new one (#1105)

* deprecate: replace old style grpc vector conversion to a new one

* fix: ignore union attr in conversion

* review fixes

---------

Co-authored-by: generall <andrey@vasnetsov.com>

---------

Co-authored-by: generall <andrey@vasnetsov.com>

---------

Co-authored-by: generall <andrey@vasnetsov.com>

* new: deprecate add, query, query_batch in fastembed mixin (#1102)

* new: deprecate add, query, query_batch in fastembed mixin

* 1.16 -> 1.17

---------

Co-authored-by: generall <andrey@vasnetsov.com>

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2025-11-11 21:17:08 +07:00
George
200020f19a wip: builtin bm25 support (#1060)
* wip: builtin bm25 support

* remove debug print

* new: add builtin embedder, add server version check for inference

* fix: regen async

* fix: fix fastembed import

* fix: regen async

* tests: add tests for builtin inference

* fix: fix fastembed checks, improve exceptions
2025-09-15 16:18:41 +08:00
George
42856424a6 new: move fastembed imports and constants into a class to support cus… (#994)
* new: move fastembed imports and constants into a class to support custom models and improve exceptions

* new: add is_supported model to FastEmbedMisc

* fix: fix types
2025-05-20 14:46:18 +04:00
George
ce4c51dd3c Default inference batch size (#907)
* fix: fix inference batch size propagation

* fix: propagate batch size to embed worker

* fix: fix type ignore comment

* fix: replace mocking class method with object method

* fix: add missing inference batch size

* fix: fix batch size propagation to worker
2025-02-21 12:47:57 +01:00
George
b4ae703d1b Local inference upload collection and upload records (#862)
* wip: add draft implementation of batch processing

* fix: embed dict and list of docs, remove redundant code

* new: regen async, small refactor

* refactor: add docstrings, rename methods

* Upload points local inference (#881)

* new: separate single and plural model embeddings

* fix: fix lazy embed models

* new: add inference object inspections to upload methods

* wip: local inference upload parallel

* new: add local inference to upload points and upload collection, refactor mixin

* fix: remove redundant code

* redundant import

* tests: check is query for query points batch

* refactor: refactor semi ordered map

* tests: add test for local inference with batches with docs and vectors

* tests: check the order of dict processing

* new: distinguish models by options

* fix: fix typing

* fix: fix types

* new: embed batches with different options

* tests: add tests for batch with different options

* fix: ignore ide incorrect type inspection

* tests: wait for points to be inserted

* fix: set threads to 1 in parallel inference

* new: adjust max internal batch size

* fix: fix type hints

* function to get embeddings size (#892)

* function to get embeddings size

* async client

* keep sync

* new: extend embedding size to support image and late interaction models

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>

* new: add local inference batch size (#894)

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2025-01-29 15:53:24 +01:00
Hossam Hagag
47ff758c56 refactor: Refactored the way of warnings (#864)
* refactor: Refactored the way of warnings

* remove unused imports

* fix: Fix stacklevel in warnings

* regenerated async

* nit

* Updated some warnings with show once

* fix: Fix stack levels

* Updated async

* Update qdrant_client/qdrant_remote.py

Co-authored-by: George <george.panchuk@qdrant.tech>

* Update async client

* Updated warnings

* Updated warnings

* Updated warnings

* fix: fix warning level

* fix: fix warning level in async

* fix: revert append payload condition

---------

Co-authored-by: George <george.panchuk@qdrant.tech>
2025-01-06 16:44:06 +01:00
George
baa3e264c8 Drop python 3.8 (#848)
* new: drop python 3.8 support, update type hints

* fix: remove 3.8 from ci

* fix: update type hints

* fix: make netlify use python3.10

* fix: try python3.9 for sphinx

* debug: try updating sphinx

* new: bump ffastembed to 0.4.2

* fix: install numpy<2 for mypy

* fix: install numpy via poetry
2024-11-15 15:54:32 +01:00
George
8e4279d1d6 fix: fix resolve query type hint (#853) 2024-11-15 14:25:47 +01:00
George
8d6bee2434 new: make model name mandatory in inference structures (#846) 2024-11-12 19:09:44 +01:00
George
e15b21e5fb Alter image structure (#845)
* new: assign image.image Any type in grpc

* new: update image structure, rename embed utils Path to FieldPath
2024-11-11 18:03:26 +01:00
George
d2771afdfb Local inference image support (#836)
* new: add backbone for image support

* new: convert b64 to pil, embed images, add test

* tests: add test file

* refactor: replace 3 different inference object vars with a common one

* fix: fix type hints

* fix: fix type hints

* tests: add tests

* fix: remove redundant imports

* new: propagate image options

* Custom inference object (#837)

* new: add inference object support

* new: add inference object support

* fix: remove redundant import

* refactor: return newline

* fix: fix propagate options test
2024-11-04 08:37:53 +01:00
George
5e0a40e954 Propagate options (#835)
* new: propagate model params in document options

* new: update fastembed version

* fix: fix option propagation, add tests

* fix: update poetry lock

* fix: don't change the set model on embed raw types

* tests: extend propagate options test

* fix: erase cls model storage after each test
2024-10-31 09:56:58 +01:00
George
c152e3c7bf Agnostic local inference (#799)
* new: update previous qdrant version

* new: rollback rest client as types moved to the core

* new: add model_fields_set pydantic compat

* new: add Document to models

* new: add local inference inspectors

* new: add inference functionality to qdrant client

* fix: fix type hints

* fix: fix type hints

* fix: fix type hints

* new: remove _cloud_inference arg from methods

* refactor

* tests: update tests, add path check

* fix: fix type hint

* fix: fix type hint

* fix: resolve plain query

* fix: remove redundant checks

* fix: fix resolve query doc

* tests: add test case for several docs in prefetch

* fix: fix tests

* new: forbid cloud inference with local mode

* fix: fix is_query assignment, fix embed dict

* tests: fix sparse vector values comparison

* fix: regen async

* fix: embed batch

* tests: extend local inference tests

* tests: add update operations and update vectors tests

* tests: remove redundant tests, add plain query points test

* fix: fix type hint

* fix: add missing import

* fix: yet another type hint fix

* tests: add query points groups and query batch points tests

* fix: do not try to embed a vector, comment missed disabled test stmt

* new: add late interaction embeddings without add and query methods

* fix: fix LITE import

* fix: fix problem with retrieving not available path, enable tests

* fix: fix exception message

* refactor: update type hint

* fix: update type hints

* chore: add docstrings to embed methods

* refactor: simplify inspector init

* fix: fix type hint

* new: reuse parser, fix type import

* refactor: rename parser, add docstrings

* fix: fix type hint

* fix: fix type hint

* fix: fix type hint

* refactor: smol refactor, docstrings

* fix: fix inspect embed

* deprecation warning for grpc.PointStruct

* new: populate inspections cache (#823)

* new: populate inspections cache

* fix: fix type hints

* fix: regen async

* fix: fix type hint

* fix: fix typo in filename

* fix: fix points check

* fix: fix type hint
2024-10-29 15:48:23 +01:00
George
8abfd0e6cd fix: fix docstrings to make sphinx happy (#814)
* fix: fix docstrings to make sphinx happy

* debug: downgrade poetry version

* fix: update virtualenv

* debug: remove debug line
2024-10-18 10:55:50 +04:00
George
52ccd4181b fix: add local files only to set sparse model (#802) 2024-10-07 10:53:36 +02:00
Hossam Hagag
d0bda1b3bb chore: Change embed to query_embed in fastembed mixin for sparse models (#797) 2024-10-05 23:29:24 +02:00
George
04845eeba0 new: remove default hybrid search for query_points with fastembed (#735)
* new: remove default hybrid search for query_points with fastembed

* tmp: disable query extension for vector input, etc
2024-08-12 16:14:32 +02:00
Hossam Hagag
f1da81dfe6 Change upload points async (#730)
* made upload_points sync

* updated base_client_generator

* fix: remove excess file

* refactoring: fix timeout type, remove redundant import

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2024-08-09 21:19:45 +02:00
George
d354293a4e new: set idf modifier in fastembed mixin for bm models (#686)
* new: set idf modifier in fastembed mixin for bm models

* fix: update poetry lock

* fix: tmp fix, restrict pyright version

* refactor: update bm model extraction, update fastembed

* fix: fix list of bm models

* fix: fix default bm models list value

* refactoring: remove redundant import
2024-07-30 18:01:47 +02:00
George Panchuk
f77ad3567a fix: regen async 2024-06-27 18:34:41 +03:00
George Panchuk
40e0c62a4f new: update query signature, id type and return type 2024-06-26 17:42:39 +03:00
Andrey Vasnetsov
6ff600c94b rollback fast-embed query method (#658)
* rollback fastembed query method

* un-deprecate query

* fix: regen async, roll back generator tmp workaround

* fix: don't skip fastembed tests

* new: add query batch points interface, fix minor bugs (#660)

* new: add query batch points interface, fix minor bugs

* fix: do not exclude ids from lookup collection from search (#661)

* Update qdrant_client/qdrant_client.py

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>

* fix: regen async

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
2024-06-26 17:13:40 +03:00
George
00da88e947 new: add query interface draft (#655)
* new: add query interface draft

* fix: temporarily update fastembed generator

* review fixes

* generate async client

* generate async client

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2024-06-25 09:21:28 +02:00
Andrey Vasnetsov
815a784469 Update openapi v1 10 x (#622)
* update api with conversions

* introduce IDF modifier for sparse vectors in local mode

* generate stuff

* regen async

* Update tests/fixtures/points.py

* fix: address review comments

* upd models

* fix: update grpc overwrite payload conversion, fix order by conversion (#653)

---------

Co-authored-by: George <george.panchuk@qdrant.tech>
2024-06-20 17:55:35 +02:00
Andrey Vasnetsov
8556cc86d6 Drop black (#623)
* replace black with ruff

* use line length

* remove isort

* regen async

* regen async

* fix ruff version [no-ci]
2024-05-07 20:47:27 +02:00
George
3bdb6f6719 new: add gpu support for fastembed, add fastembed providers (#612)
* new: add gpu support for fastembed, add fastembed providers

* new: update fastembed, add fastembed-gpu, add type alias for onnx providers

* new: update readme

* fix: fix poetry.lock
2024-05-03 21:23:44 +02:00
George
9ef5c0afc4 fix: fix set_parse_model property in async (#583)
* fix: fix set_parse_model property in async

* fix: fix remaining async methods
2024-04-10 18:48:19 +02:00
Yasyf Mohamedali
07404f2e8b Remove async from sparse_embedding_model_name property (#581)
* Remove async from sparse_embedding_model_name property

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2024-04-05 19:19:35 +02:00
Andrey Vasnetsov
822aefa1c1 WIP: hybrid search with fastembed (#553)
* WIP: hybrid search with fastembed

* hybrid queries with fastembed

* test for hybrid

* fix typo

* new: extend hybrid search tests, fix mypy, small refactoring (#554)

* refactor: align model name parameters in setters, update tests

* fix: fix async

* fix: add a good test, fix sparse vectors in query batch

* refactoring: reduce branching, refactor fastembed tests

---------

Co-authored-by: George <george.panchuk@qdrant.tech>
2024-03-27 13:57:58 +01:00
Nirant
fc4b3cf9e2 Upgrade FastEmbed Version (#493)
* Update fastembed to v0.2.1

* chore(qdrant_fastembed.py): update DEFAULT_EMBEDDING_MODEL

* fix(fastembed integration): upgrade to latest version

* Prefer black over ruff

* Prefer black over ruff

* Remove hardcoded directory structure from Qdrant Client checks

* new: deprecate current default model, deprecate max token length, update fastembed

* fix: make embedding_model_name method sync

* fix: update poetry lock

* refactor: use list_supported_models() (#501)

* fix: fix fastembed check

* fix: fix fastembed class var assignment

* fix: remove fastembed deprecation from qdrant client (#524)

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
Co-authored-by: Anush <anushshetty90@gmail.com>
2024-03-05 21:09:31 +01:00
George
a2adc2d2ad new: deprecate upload records, update tests, prohibit migration of co… (#447)
* new: deprecate upload records, update tests, prohibit migration of collections with custom shards

* Update qdrant_client/qdrant_client.py

* Update qdrant_client/async_qdrant_client.py

* new: replace autogenerated int ids with uuids (#448)

* fix: remove redundant kwargs

* fix: regen async

* fix: remove redundant import

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2024-01-19 20:11:31 +01:00
Praveen Palanisamy
70754dbd91 Allow fastembed embedding model params configuration in parity with fastembed APIs (#416)
* Update set_model -> set_model_params;
Allow setting all fastembed.DefaultEmbedding init args for full pass-through

* Add fastembed test for setting custom embedding model params

* Update async client gen to include set_model_params

* Update Async client APIs

* Add explicit Optional since PEP 484 is still in effect

* Regenerate async client APIs

* Revert name change: set_model_params -> set_model

* Add **kwargs to set_model method sig

* Update name set_model_params -> set_model

* Regenerate async client APIs

* Use and pass **kwargs

* Regenerate async client APIs using Py 3.10.x
2024-01-07 18:48:58 +01:00
George
6743f28fda new: switch async generator version to 3.10.x (#387)
* new: switch async generator version to 3.10.x

* fix: remove package version update
2023-11-30 12:09:35 +01:00
generall
ffedbd6b88 regen async 2023-10-16 23:07:47 +02:00
Andrey Vasnetsov
0f32e578ec update interface and version for fastembed (#340)
* update interface and version for fastembed

* fix types

* fix types

* regen async

* use python 3.11 to check compatibility

* fix docstring

* regen async

* propagate batch size
2023-10-16 18:13:52 +02:00
George
278a68d297 Async qdrant client (#319)
* new: manually implemented async qdrant client

* fix: remove await before sync call

* fix: make upload collection, records and migrate synchronous

* fix: add init method to async client base

* refactoring: remove redundant import

* new: add super().__init__ in qdrant remote, update import in http

* new: mvp async qdrant client generator

* new: fix mypy, update generator script, refactoring

* fix: fix test script

* new: update generator launch script, update async files

* new: refactor async client generator

* refactoring: remove redundant operations, add comments, refactor

* new: add isort, black and autoflake to dev dependencies

* fix: add more checks, fix type hints

* fix: do not check types in async client generator for python3.8

* new: do not type check async_qdrant_fastembed

* fix: fix pyright run

* new: update async client tests

* fix: update versions in CI

* new: update pre-commit python version, update autogenerated files

* new: update generated files, add tests for async generator, update generator script

* fix: exclude generator test from 3.8

* fix: fix python version condition

* fix: add python target version for black

* fix: generate async client only on python 3.9
2023-10-13 22:26:53 +02:00