Andrey Vasnetsov
c83ddec2cb
refactor for PR #15
2021-05-27 00:54:44 +02:00
Joan Fontanals
a633e625f3
allow set full payload from serde_json Values ( #15 )
...
* allow set full payload from serde_json Values
* move payloadinterface and variant to segment
* expose direct json string in segment
* refactor extract function
* handle from payloadinterface
* add test on filter nested
* update lib/segment/src/segment.rs
Co-authored-by: Andrey Vasnetsov <vasnetsov93@gmail.com >
* use payloadinterface in collection
Co-authored-by: Andrey Vasnetsov <vasnetsov93@gmail.com >
2021-05-26 23:59:57 +02:00
Andrey Vasnetsov
dba7c9e558
fix filter query cardinality estimation
2021-05-04 20:12:51 +02:00
Andrey Vasnetsov
46ba12a198
update readme + change filter structure
2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
e0636b4920
optimized segment building in a separate directory
2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
398da04aad
add indexing optimizer + enable mmap and structure index rebuilding + fix issues
2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
725c33aab2
endpoint option to manage indexes
2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
dcbebac80c
empty condition is same as no condition
2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
0a91d3ae06
struct payload index segment test
2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
cd2acd5dac
test for cardinality estimation
2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
5dd3c1935e
WIP: struct payload index texts
2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
328f445439
implement index filter
2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
c50482d53f
WIP: query_points in struct payload index
2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
fe44c4e00e
WIP: FieldCondition
2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
7787f07d57
WIP: PayloadFieldIndex for FieldIndex
2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
0dd48e0acb
WIP: should, must and must_not estimators
2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
4f798062bd
WIP: payload save and load
2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
c3fe443643
WIP
2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
cb9f41a0df
implement payload index builder + column index for keywords and integers
2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
e5d7ac7721
extend payload storage interface + start functions for creating struct index
2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
9fc12658da
number field index
2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
5a2985c6f7
refactor condition checking for better column storage support
2021-04-06 14:12:46 +02:00
Andrey Vasnetsov
5c2afdc5b9
geo-radius + update wiki
2021-01-24 17:16:06 +01:00
Andrey Vasnetsov
69dc0f3bef
fix docker build
2021-01-16 12:11:24 +01:00
Andrey Vasnetsov
8a85c10934
use BLAS for vector dot production
2021-01-04 22:59:22 +01:00
Andrey Vasnetsov
bd7ed06bfe
add benchmark for vector similarity speed comparasion
2021-01-03 01:25:41 +01:00
Andrey Vasnetsov
10161f05e5
fix vector upsert on WAL reconstrution + use f32 for vector storage
2020-12-13 20:54:18 +01:00
Andrey Vasnetsov
0909b70c2c
replace sled with rocksdb
2020-12-13 00:26:55 +01:00
Andrey Vasnetsov
eab0ac83ba
attempt to replace sled for storing vectors
2020-12-08 16:55:09 +01:00
Andrey Vasnetsov
44c6c0a65c
attempt better memory usage for in-memory vector storage
2020-12-08 00:26:10 +01:00
Andrey Vasnetsov
320cd579a4
Add description to API models
2020-10-27 00:05:59 +01:00
Andrey Vasnetsov
aa77e69632
OpenAPI definitions generator
2020-10-26 22:49:08 +01:00
Andrey Vasnetsov
3dd550b6b9
OpenAPI 3.0 doc generation
2020-10-26 17:44:31 +01:00
Andrei Vasnetsov
2a2928b09b
fix storage issue
2020-10-20 21:51:49 +02:00
Andrei Vasnetsov
bb879a3f80
do not update unappendable collections + update vectors directly in simple segments
2020-10-20 21:21:33 +02:00
Andrei Vasnetsov
40ed0ed98f
add search API
2020-10-19 23:55:18 +02:00
Andrey Vasnetsov
e896162884
removing old segments data after optimization
2020-10-04 21:08:41 +02:00
Andrey Vasnetsov
162698ac11
collection loader + fixes and load test
2020-10-04 16:49:28 +02:00
Andrey Vasnetsov
7495fd1d81
storage type selection in optimizer
2020-09-29 13:15:18 +02:00
Andrey Vasnetsov
28e27a9f7e
implement basic matcher for memmap storage
2020-09-28 23:37:57 +02:00
Andrey Vasnetsov
6278603415
memmap vector storage
2020-09-28 19:28:57 +02:00
Andrey Vasnetsov
8125611b6f
WIP: memmap vector storage
2020-09-22 11:39:48 +02:00
Andrey Vasnetsov
c9a3766fcf
simple persistance for vector storage
2020-09-21 21:41:21 +02:00
Andrey Vasnetsov
21242b454a
optimize segment merging calls
2020-09-21 19:23:51 +02:00
Andrey Vasnetsov
994e63e150
use enum to be able to get exact segment type in collection
2020-09-21 15:15:48 +02:00
Andrey Vasnetsov
ff0d23dd2e
add vector storage commit function
2020-09-20 22:52:47 +02:00
Andrey Vasnetsov
e51d8bfec5
WIP: persistace
2020-09-20 20:59:58 +02:00
Andrey Vasnetsov
45e64debc5
WIP: persistent segment
2020-09-13 23:45:37 +02:00
Andrey Vasnetsov
b135c329c8
WIP: implement table of content
2020-09-13 00:10:40 +02:00
Andrey Vasnetsov
57dcaad499
refactor segment optimizer
2020-08-31 23:23:29 +02:00