Commit Graph
230 Commits
Author SHA1 Message Date
Andrey Vasnetsov 89843eaad0 python 3.12 support (#326) 2023-10-06 15:55:10 +02:00
Andrey VasnetsovandLuis Cossío bb8c442169 Recommendation api update + local mode (#314)
* WIP: recommendation api update

* implement local new reco + add new tests

* refactor calculate_best_scores, normalize score assertion

* fix raw vectors in new recommend

* fix mypy errors, add new reco on group recommend

* fix more lints from ci

* fix pyright lint

* formattng

* Add descriptions to enums

* no mutable default argument

* edit score comparison precision based on magnitude

* Address review comments

* remove custom message for type ignore

* add coverage test

* improve coverage test a little

* add more fixtures for conversion test

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
2023-10-05 03:38:19 +02:00
Georgeandgenerall 8d5ed2cf70 fix: add init from conversion, support init from in local mode (#317)
* fix: add init from conversion, support init from in local mode

* fix: fix mypy

* more test coverage

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2023-09-29 16:44:50 +02:00
George 8b1295c9f8 fix: fix type hint for get_args_subscribed and batch operation (#301) 2023-09-25 19:36:22 +02:00
George 438534d463 new: add kwargs to QdrantBase (#292) 2023-09-12 20:29:42 +02:00
George eafd92a9a3 fix: explicitly convert np vector to list for previous versions of pydantic (#289) 2023-09-11 19:47:46 +02:00
Andrey Vasnetsov 8452f2ae72 fix fastembed for pydantic 1.x (#285)
* fix fastembed for pydantic 1.x

* add extras setup in readme

* move fastembed from group to extras

* lock
2023-09-11 15:09:12 +02:00
generall 73641f15cc fix README for fastembed 2023-09-07 12:30:19 +02:00
Andrey Vasnetsov aed7ecaefb sync API (#280)
* sync API

* fix tests

* backward compatibility version up
2023-09-07 11:29:10 +02:00
George 2b6a6b724c Congruence values count (#268)
* tests: add some values count congruence tests

* fix: fix incorrect behaviour of value_counts, isempty, isnull in local mode

* fix: update isempty behaviour

* tests: add test cases for isnull and isempty
2023-09-04 18:15:20 +02:00
Andrey Vasnetsov 578db8107a add tests for vector operations (#275)
* add tests for vector operations

* rename vectors->points in vector update API

* fix tests
2023-09-04 18:14:19 +02:00
Nirant 77837b05f2 Fix API Param in README 2023-08-31 09:57:03 +02:00
Nirant a26a07769c Fixes for test, type errors (#266)
* * fix(qdrant_fastembed.py): change method name from encode to embed in QdrantFastembedMixin class
* fix(qdrant_fastembed.py): change method name from encode to embed

* * fix(qdrant_fastembed.py): change DEFAULT_EMBEDDING_MODEL from "sentence-transformers/all-MiniLM-L6-v2" to "BAAI/bge-small-en"

* * fix(qdrant_fastembed.py): add support for different embedding types in QdrantFastembedMixin
* feat(qdrant_fastembed.py): add embed_type parameter to embed method in QdrantFastembedMixin

* * fix(qdrant_fastembed.py): change vector field name from "text-{model_name}" to "fast-{model_name}"

* * fix(qdrant_fastembed.py): add comments to clarify code intent

* * refactor(qdrant_fastembed.py): remove unused import 'Sequence' from typing module

* * test(test_fast_embed.py): refactor test_add_without_query function to simplify parameter handling and remove unnecessary checks and assignments

* * fix(qdrant_fastembed.py): fix type error in NamedVector Query

* * fix(qdrant_fastembed.py): change method name from `embed` to `query_embed` in QdrantFastembedMixin class
* fix(qdrant_fastembed.py): fix query_vectors assignment in `query` method of QdrantFastembedMixin class
2023-08-23 14:50:09 +02:00
generall 6f8c5172d7 sync indexed_only and binary quant params 2023-08-21 23:15:00 +02:00
generall 395321d8c6 init dev commit - do not cherry-pick 2023-08-21 14:08:08 +02:00
generall fd0f52a273 add pull request template 2023-08-21 13:50:10 +02:00
Andrey VasnetsovandGeorge 22f851d1a3 new: add get args for subscripted generics (#260) (#262)
* new: add get args for subscripted generics

* refactoring: remove redundant type (#261)

Co-authored-by: George <george.panchuk@qdrant.tech>
2023-08-21 12:48:41 +02:00
George 897a7ff6c9 Migration tool (#243)
* new: migration tool v1

* new: add docstrings

* fix: fix collection comparison, add tests

* fix: fix mypy complaints

* tests: test multiple collections scenario

* new: add migrate method to client's interface

* new: update migration tool

* new: simplify collisions

* new: update migration logic and tests

* fix: fix mypy complaints

* tests: replace tokenizer type in favour of backward compatibility tests
2023-08-21 11:44:26 +02:00
Andrey Vasnetsov 898de18a11 disable forcing of http2 for cloud connections (#258) 2023-08-21 11:31:48 +02:00
Prashanth Rao 0d8cc12b18 Change top kwarg to limit as per code (#233) 2023-08-21 11:24:15 +02:00
George 75717afcd6 new: replace nested types with get_args where possible (#259) 2023-08-21 11:22:30 +02:00
Nirantandgenerall 536b0eeb22 Add fastembed integration (#210)
* Add skeleton for fastvector registration

* * fix(qdrant_client.py): fix import statement for QdrantClientMixin from fastembed instead of fastvector
* fix(qdrant_client.py): remove unnecessary print statement

* * chore(README.md): add bullet point for extensive test coverage
* feat(README.md): add section for Fast Embeddings + Simpler API
* fix(README.md): fix formatting of code block and update code example for Qdrant Client usage
*

* * fix(qdrant_client.py): remove unused imports and code
* feat(qdrant_client.py): add support for adding and querying documents with fastembed installed
* test(qdrant_client.py): add tests for adding and querying documents with and without fastembed installed

* * feat(qdrant_client.py): add support for fastembed.qdrant_mixin.QdrantAPIExtensions for upsert_docs and search_docs methods

* * fix(qdrant_client.py): remove unused import of SearchParams
* feat(qdrant_client.py): add return type hint to QdrantClient.search_docs method

* * fix(qdrant_client.py): fix import statement for SearchParams class in qdrant_client.py

* * feat(models.py): add QueryResponse model for handling query responses

* * feat(qdrant_client.py): add batch_iterable method to QdrantClient class

* * chore(qdrant_client.py): reformat code for better readability

* * chore(test_qdrant_client.py): add type hinting to fastembed_setup fixture
* test(test_qdrant_client.py): add test for client_close function

* * chore(.gitignore): add pattern for ignoring .tar.gz files

* * refactor(test_qdrant_client.py): remove unused imports and variables
* chore(test_qdrant_client.py): reformat import statements for better readability

* * chore(test_qdrant_client.py): remove unused fixtures and test cases

* * chore(qdrant_client.py): remove unused imports

* Add test skeleton

* * chore(.gitignore): add local_cache/*/* to ignore local cache files

* * fix(qdrant_client.py): change import statement for fastembed to use DefaultEmbedding instead of FlagEmbedding
* refactor(qdrant_client.py): refactor code to remove unnecessary loop

* * fix(qdrant_client.py): import missing models in qdrant_client.py
* feat(qdrant_client.py): add support for search parameters in search method
* refactor(qdrant_client.py): refactor indexing logic to handle embeddings correctly

* * test(test_fast_embed.py): add assertion to verify count of documents in collection
* test(test_fast_embed.py): remove unused code
* test(test_fast_embed.py): add TODO comment for future assertions

* * chore(pyproject.toml): add optional fastembed group to poetry dependencies
* feat(pyproject.toml): add fastembed dependency to fastembed group

* * chore(qdrant_client.py): add check for fastembed installation in QdrantClient constructor

* * fix(test_fast_embed.py): skip test if FastEmbed is not installed
* fix(test_fast_embed.py): add default values for test_no_install parameters
* fix(test_fast_embed.py): skip test if FastEmbed is installed

* Delete unused usage file

* * chore(models.py): remove unused QueryResponse class
* feat(qdrant_client.py): add QueryResponse class

* * refactor(qdrant_client.py): organize imports and fix type hinting for QueryResponse class

* review fixes

* make mypy happy

* review fixes

* fix mypy

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2023-08-18 15:17:46 +02:00
Nirantandgenerall 394407afb4 Remove MacOS from PR CI (#256)
* * chore(integration-tests.yml): remove MacOS from the matrix of operating systems
* chore(integration-tests.yml): remove setup for docker on MacOS

* enable macos tests on dispatch

* rename workflow

* do not run congruence tests on macos

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2023-08-17 12:55:05 +02:00
Kumar Shivendu 9355c24ef1 test: Improve inmemory and local persistence tests (#249) 2023-08-17 12:45:58 +02:00
Tim Visée cddf1c5407 Move collection update integration from congruence to main tests (#240)
* Move collection update integration from congruence to main tests

The integration test for collection update correctness was put in the
wrong place. Local mode does not support this feature. The test as a
whole has been merged into an existing collection update test in our
main testing suite.

* Only test for new collection update parameters in v1.4.0 and above
2023-08-05 21:45:54 +02:00
Andrey Vasnetsovandtimvisee 31252b7650 V1.4.0 (#238)
* new models conversions

* new models conversions

* Add update collection test

* ignore collection update test

* fix type check

* fix python 3.7

---------

Co-authored-by: timvisee <tim@visee.me>
v1.4.0
2023-08-03 11:02:11 +02:00
George 2b16c54915 fix: do not divide by zero (#236)
* fix: do not divide by zero

* fix: python3.7 compatibility
2023-07-28 17:25:14 +03:00
George 84b5374a95 fix: add headers (#235) 2023-07-28 16:25:47 +03:00
George bab30948a6 new: add explicit close method (#227)
* new: add explicit close method

* new: fix async, close local mode, extend interface

* new: update closed flag usage, prevent reinitializing connections
2023-07-27 19:18:31 +03:00
George ba8063ba89 fix: return poetry lock (#232) v1.3.2 2023-07-26 19:10:18 +03:00
George b84aba141e new: bump to v1.3.2 (#230) 2023-07-26 19:04:07 +03:00
Kacper ŁukawskiandGeorge Panchuk c7f221e2a4 Fix cosine normalization (#213)
* Enable vectors comparison in tests (round to 1e-3)

* Normalize vectors if cosine distance is used

* fix: compare scroll results

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2023-07-23 00:06:18 +04:00
generall 6616e1838d fix types 2023-07-22 11:58:37 +02:00
George cc0f3cfa5f new: forbid extra fields in pydantic models (#222) 2023-07-22 00:01:58 +02:00
George 0284069630 new: serialize to json inside pydantic (#224) 2023-07-21 23:52:45 +02:00
generall aacd291ff3 more timeouts 2023-07-21 23:15:25 +02:00
Andrey Vasnetsov 13b0a9ec63 MacOS CI (#226)
* increase collection timeout

* fix

* more timeouts
2023-07-21 22:13:47 +02:00
Arthur Melin 1291087201 Fix incorrect args for grpc.insecure_channel() (#216)
Signed-off-by: Arthur Melin <arthur.melin@wiseone.io>
2023-07-21 10:39:27 +04:00
George ba8251226f fix: fix model fields (#221) 2023-07-20 16:32:06 +04:00
Andrey Vasnetsov 730ed8d239 add missing init_from param (#205) 2023-07-17 10:05:20 +02:00
George ace358992f fix: add args and kwargs to json call pydantic v1 (#215) 2023-07-16 16:08:27 +04:00
Georgeandgenerall 0fffa1abd4 Pydantic v2 (#214)
* new: update autogenerated code

* fix: fix config conversions

* new: add pydantic compat

* new: add types to pydantic_compat

* new: update dependencies

* new: use pydantic compat functions

* new: add pydantic v2 field check

* fix typing

* fix typing

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2023-07-16 01:23:21 +02:00
Luis CossíoandGeorge bc08733e77 Mac: add explicit ports for integration tests (#170)
* fix: add explicit ports for integration tests

* amend: replace hardcoded grpc port

---------

Co-authored-by: George <george.panchuk@qdrant.tech>
2023-07-08 17:09:14 +04:00
Luis CossíoandGeorge Panchuk 0777ea88cb Add gnu-sed instructions for macOS in DEVELOPMENT.md (#181)
* Add gnu-sed instructions for macOS in DEVELOPMENT.md

* new: extend mac os sed instructions

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2023-07-08 16:49:39 +04:00
George ea412c4959 Fix congruence upload (#183)
* fix: fix congruence upload collection list of np.array, fix type hints

* fix: update type hints, remove broken code

* tests: delete collection after test just in case

* fix: return Dict[str, NumpyArray], support it in local mode

* fix: update type
2023-07-08 15:40:36 +04:00
Kacper Łukawski b9a7a15cee [WIP] Enable testing on macos (#199)
* Enable testing on windows and macos

* Add platform to the name

* Add Docker on MacOS

* Temporarily disable tests on Windows

* Explicitly set ports to be opened

* Add problematic case for MacOS

* Increase the timeout in tests
2023-07-07 13:31:32 +02:00
Andrey Vasnetsov 7fa25c8e77 Include license v1.3.1 2023-06-26 14:11:06 +02:00
Andrey Vasnetsov 1ec45bd97b V1.3.0 (#194)
* wip: implement release checnges

* lookup tests

* fix mypy
v1.3.0
2023-06-23 14:52:04 +02:00
George 5fc8bb7f5f fix: fix group search with np array, add tests (#190) 2023-06-19 17:49:00 +02:00
George 76476e406e fix: set empty payload as {} instead of None (#191) 2023-06-19 17:47:41 +02:00