Commit Graph

1780 Commits

Author SHA1 Message Date
Andrey Vasnetsov
c83ddec2cb refactor for PR #15 2021-05-27 00:54:44 +02:00
Joan Fontanals
a633e625f3 allow set full payload from serde_json Values (#15)
* allow set full payload from serde_json Values

* move payloadinterface and variant to segment

* expose direct json string in segment

* refactor extract function

* handle from payloadinterface

* add test on filter nested

* update lib/segment/src/segment.rs

Co-authored-by: Andrey Vasnetsov <vasnetsov93@gmail.com>

* use payloadinterface in collection

Co-authored-by: Andrey Vasnetsov <vasnetsov93@gmail.com>
2021-05-26 23:59:57 +02:00
Andrey Vasnetsov
dba7c9e558 fix filter query cardinality estimation 2021-05-04 20:12:51 +02:00
Andrey Vasnetsov
46ba12a198 update readme + change filter structure 2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
e0636b4920 optimized segment building in a separate directory 2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
398da04aad add indexing optimizer + enable mmap and structure index rebuilding + fix issues 2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
725c33aab2 endpoint option to manage indexes 2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
dcbebac80c empty condition is same as no condition 2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
0a91d3ae06 struct payload index segment test 2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
cd2acd5dac test for cardinality estimation 2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
5dd3c1935e WIP: struct payload index texts 2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
328f445439 implement index filter 2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
c50482d53f WIP: query_points in struct payload index 2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
fe44c4e00e WIP: FieldCondition 2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
7787f07d57 WIP: PayloadFieldIndex for FieldIndex 2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
0dd48e0acb WIP: should, must and must_not estimators 2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
4f798062bd WIP: payload save and load 2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
c3fe443643 WIP 2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
cb9f41a0df implement payload index builder + column index for keywords and integers 2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
e5d7ac7721 extend payload storage interface + start functions for creating struct index 2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
9fc12658da number field index 2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
5a2985c6f7 refactor condition checking for better column storage support 2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
5c2afdc5b9 geo-radius + update wiki 2021-01-24 17:16:06 +01:00
Andrey Vasnetsov
69dc0f3bef fix docker build 2021-01-16 12:11:24 +01:00
Andrey Vasnetsov
8a85c10934 use BLAS for vector dot production 2021-01-04 22:59:22 +01:00
Andrey Vasnetsov
bd7ed06bfe add benchmark for vector similarity speed comparasion 2021-01-03 01:25:41 +01:00
Andrey Vasnetsov
10161f05e5 fix vector upsert on WAL reconstrution + use f32 for vector storage 2020-12-13 20:54:18 +01:00
Andrey Vasnetsov
0909b70c2c replace sled with rocksdb 2020-12-13 00:26:55 +01:00
Andrey Vasnetsov
eab0ac83ba attempt to replace sled for storing vectors 2020-12-08 16:55:09 +01:00
Andrey Vasnetsov
44c6c0a65c attempt better memory usage for in-memory vector storage 2020-12-08 00:26:10 +01:00
Andrey Vasnetsov
320cd579a4 Add description to API models 2020-10-27 00:05:59 +01:00
Andrey Vasnetsov
aa77e69632 OpenAPI definitions generator 2020-10-26 22:49:08 +01:00
Andrey Vasnetsov
3dd550b6b9 OpenAPI 3.0 doc generation 2020-10-26 17:44:31 +01:00
Andrei Vasnetsov
2a2928b09b fix storage issue 2020-10-20 21:51:49 +02:00
Andrei Vasnetsov
bb879a3f80 do not update unappendable collections + update vectors directly in simple segments 2020-10-20 21:21:33 +02:00
Andrei Vasnetsov
40ed0ed98f add search API 2020-10-19 23:55:18 +02:00
Andrey Vasnetsov
e896162884 removing old segments data after optimization 2020-10-04 21:08:41 +02:00
Andrey Vasnetsov
162698ac11 collection loader + fixes and load test 2020-10-04 16:49:28 +02:00
Andrey Vasnetsov
7495fd1d81 storage type selection in optimizer 2020-09-29 13:15:18 +02:00
Andrey Vasnetsov
28e27a9f7e implement basic matcher for memmap storage 2020-09-28 23:37:57 +02:00
Andrey Vasnetsov
6278603415 memmap vector storage 2020-09-28 19:28:57 +02:00
Andrey Vasnetsov
8125611b6f WIP: memmap vector storage 2020-09-22 11:39:48 +02:00
Andrey Vasnetsov
c9a3766fcf simple persistance for vector storage 2020-09-21 21:41:21 +02:00
Andrey Vasnetsov
21242b454a optimize segment merging calls 2020-09-21 19:23:51 +02:00
Andrey Vasnetsov
994e63e150 use enum to be able to get exact segment type in collection 2020-09-21 15:15:48 +02:00
Andrey Vasnetsov
ff0d23dd2e add vector storage commit function 2020-09-20 22:52:47 +02:00
Andrey Vasnetsov
e51d8bfec5 WIP: persistace 2020-09-20 20:59:58 +02:00
Andrey Vasnetsov
45e64debc5 WIP: persistent segment 2020-09-13 23:45:37 +02:00
Andrey Vasnetsov
b135c329c8 WIP: implement table of content 2020-09-13 00:10:40 +02:00
Andrey Vasnetsov
57dcaad499 refactor segment optimizer 2020-08-31 23:23:29 +02:00