* Drop WAL from API
Flatten if statement
another approach
review remarks
switch flag back guard
unit test
* review remarks
* review remarks
* test missing records which are not truncated and not applied
* Drop wal from api: don't use applied seq number (#8030)
* Don't rely on applied seq num
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* remove unused
* Repurpose existing plunger, unset first_skipped in operation branch
* more complicated tests with empty wal
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>
* review remarks
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* update queue dont keep ops in ram showcase
* always load operation from WAL
* revert operations buffering
cound only pending in update worker operations
fix typo
use channel size instead of wal index
* remove result expect
* decrease buffering const
* review remarks
* are you happy codespell
* id tracker persisted mappings offset
* remove obsolete check
* review remarks
* review remarks
* When persisting mappings, truncate file that is larger than we expect
* If persisting mappings fails, truncate file to what we had before
This isn't necessary, but it is nice to clean up partial mappings.
* Fix typos
* Fix typos
* Just truncate the file, we don't have to seek anymore
* Rename length variable to mappings_expected_len
---------
Co-authored-by: timvisee <tim@visee.me>
* Create and load for an appendable quantization
remove feature flag
more todo
review remarks
review remarks
* fix after rebase
* Enable BQ in appendable segments
* remove const placeholder
* fix after rebase
* fix after rebase
* fix config mismatch optimizer
* fix sq infinity loop
* add comment
* Create and load for an appendable quantization
remove feature flag
more todo
review remarks
review remarks
* fix after rebase
* Rename is_appendable to supports_appendable
---------
Co-authored-by: timvisee <tim@visee.me>
* Quantization storage builder for chunked mmap
* fix after rebase
* Update lib/segment/src/vector_storage/quantized/quantized_chunked_mmap_storage.rs
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
* use vector statistics for scalar bq query
* fix minor error
* remove test_binary_scalar_internal test
* do NOT use special file for storing vector stats
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* # This is a combination of 7 commits.
* SameAsStorage default value
* fix coderabbit warnings
* neon for u8 bq
* sse for u8 bq
* fix windows build
* rename function
* add comments
* fmt
* fix arm build
* review remarks
* bq encodings
* are you happy clippy
* are you happy clippy
* are you happy clippy
* are you happy clippy
* gpu tests
* update models
* are you happy fmt
* move additional bits to the end
* fix tests
* Welford's Algorithm
* review remarks
* are you happy clippy
* remove debug println in test
* coderabit nitpicks
* remove unnecessary clone and partialeq
* Use f64 for Welford's Algorithm
* try fix ci
* revert cargo-nextest
* add debug assertions
* support gpu devices without half float
* disable f16 feature
* are you happy clippy
* Extend log warning if f16 is not supported
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
* parent 88f68d95b5
author Ivan Pleshkov <pleshkov.ivan@gmail.com> 1736245999 +0100
committer Ivan Pleshkov <pleshkov.ivan@gmail.com> 1736331073 +0100
gpu test and release workflows
no unprivileged, add sdk
fix ci
include llvm
try to use external sdk
use another vk sdk script
remove ninja
native sdk installation
debug vulkan version
use non-dev packages
debug vkinfo
remove llvm dep
ubuntu process-results
ghcr
* remove running gpu test while pr
* gpu hnsw
add gpu indexing flag
add gpu vector storage time measurement
add chmod flag
build hnsw using gpu
scoring speedup
move gl_GlobalInvocationID to mains
glslc with o flag
fix tests
use gpu atomics instead of multiple runs
enable atomics
add timers
profile search
let shader work while points collection
use generations for visited flags
add gpu init timer
avoid entries download
remove gpu call timers for a while
rename to combined graph builder
move gpu graph builder
restore levels in link access
restore links_map
add gpu start condition
use graph builder links memory
remove mut's
list point ids before processing
download-upload links using methods only
cpu graph builder as separate struct
add cpu threads count
parallel cpu build
single threaded cpu preprocess
try to fix glove accuracy
unsafe Send and Sync for gpu builder
copy graph layers builder instead of moving
remove obsolete mut
fix build
move builders to arc
fix bfb deadlock
move uneccessary clear
run cpu and gpu in separate threads
many vectors storage buffers
debug gpu vector storage
fix vector storage upload bug
add cpu ful graph build
don't use non-ready links for point
try to resolve conflicts while upsert
add indexing unit test for measurements
add test
utilize cpu more efficient
dont clear bhaep
debug gpu runs
try to use vec4 instead of mat4
dump scores count
estimate gpu usage
print usage per run
start using working groups
build shaders for vulkan v1.3
start shaders subgroup
vk instance version 1.3
subgroup vector storage
fix test
nearest heap
nearest heap test
are you happy fmt
add assert to nearest heap test
are you happy fmt
gpu visited flags
start candidates heap
candidates heap push
debug nearest heap
remove subgroupExclusiveMin
fix build
are you happy clippy
remove links dependency from nearest heap
provide input count to nearest test
are you happy fmt
provide ef to shader
are you happy fmt
get subgroup size
move compiled shaders
are you happy fmt
fix workgroups count for vector storage compute test
return true subgroup size
do nearest heap on gpu with partial sorting
remove tmp shader
unite scores and indices
are you happy fmt
gpu nearest sorting
are you happy fmt
fix nearest heap test
links test with subgroups
gpu candidates start
fix candidates sorting
are you happy fmt
fix candidates test on m1 chip
move obsolete and test shaders
start new gpu search context
search context test
fix links uploading
debug hnsw searh on level test
fix gpu hnsw search on level test
move test initialization to separate function
gpu greedy search shader test
test heuristic shader
gpu greedy search test
gpu test heuristic
move greedy search into searcher
start insertion shader
find hnsw patch on gpu
fix gpu patches test
start gpu graph builder
equivalency test
fix layers check
add timers
are you happy fmt
quality test and fix multithreaded bug
debug tests
are you happy fmt
fix levels count and clear links
fix equivalency test
remove obsolete solution
trivial gpu indexing integration
gpu config and skip conflicts for large hnsw
glsl generic vector storage element type
gpu generic vector storage
enable vulkan features for f16 and u8
force_half_precision option
add force f16 test
fix u8 and f16 unit tests
apply generic shaders in hnsw construction
remove conflicts check; start cpu utilization
cpu prebuild
upload first point links to gpu
mark all points in cpu graph as ready
multithreaded cpu and gpu
fix equivalency test
remove cpu links copy
parallel patches clearing
fix multithreaded env
external layers source
more logs to debug cpu+gpu syncronization
clear gpu processed points count
remove some temp logs
fix cpu stop condition bug
new insert vector shader
greedy search returns only ids
download whole layer links
apply new links on gpu directly
update win shader script
parallel links loading
provide memory amount as setting and estimate groups count
start bq
gpu bq test
fix tests
shaders invocations refactor
revert max groups settings
gix build after rebase
fix bug after rebase
upload links to gpu in cpu thread
fix greedy search iteration
atomic checker for conflicts
dump graph changes for debugging
move iterator to separate file
start combined builder refactor
start cpu builder as separate struct
separate gpu and cpu construction
fix tests
fix atomics count bug
upload links to gpu as separate fn
glue cpu and gpu
gpu thread
link points on cpu while gpu is busy
fix gpu bq scoring test
use xor instead of calc it
reallocate with less groups if not enough gpu memory
update compiled shaders
don't use llvm gpu emulator
debug panic if no gpu
remove tmp debug dumping
dockerfile for gpu
customize candidates count
fix gpu storage size
runtime shader compilation
dim as define
nearest heap params as macro
candidates and links capacities as macro
parallel bitonic sort
use bubble sort too
nearest heap as shared
limited shared candidates heap
combine buble and bitonic sorts
one-argumented similarity
cache vector for scoring
dont check candidates overflow
do greedy before search
visited hashtable
Revert "visited hashtable"
This reverts commit 7540d04d3859d02f17908c874927d18076f3eb8e.
use less flushes
bulk sum calculation
Revert "bulk sum calculation"
This reverts commit 6ef78ceb8d6ba7d9bf2077804548df162052c749.
load vector in one read
Revert "load vector in one read"
This reverts commit ea980d0399b90f44638590d40fdbfb1d6ffa9504.
fix build after rebase
fix build after rebase
sq support
simplify bq uploading using constructed quantization
metrics
alignment depends on subgroup size
are you happy fmt
exact flag
use lazy static for device access
remove prints
test reallocation factor
visited flags capacity as a compile time constant
build gpu hnsw for payload blocks
add dynamically sized subgroup size support
fix build
update rust version for nvidia docker
dynamically sized subgroups
start device filter
test device filter
are you happy clippy
devices manager
provide device instead of flag, better locking
partial cpu permit release
free cpus when gpu enabled
use cpu is waiting is disabled
create device only when gpu is enabled
cpu permit release all cases
parallel indices count
provide queue index
small refactor of shader builder
are you happy fmt
quantization params as separate struct
pq refactor
fmt
remove upload_mapped_ptr from gpu buffer
refactor gpu buffer
check buffer range
buffer refactor
refactor pipeline
move shader compiler to instance
refactor context
more refactor
no unwraps in gpu crate
clippy
apply pr changes
are you happy fmt
dont use shaderc in segment
more gpu tests
pq bq tests
provide bq option
fix bq for small dim
bind pq buffers
pq more complex test
cover all storage types by tests
test multi and vectors as iterator
multivectors upload data
multivectors shader
fix multivector tests
refactor shader builder
move storage tests into separate file
move quantization to separate file
refactor gpu vector storage creation
refactor gpu quantization api
dont convert to float when unnecessary
more comments
small upload vectors refactor
fix build after rebase
fix tests after rebase
different vk queue priorities capacity
vectors upload stopper
amd dockerfile
fix build after rebase
parent ec4d42e7f6
author Ivan Pleshkov <pleshkov.ivan@gmail.com> 1692111338 +0200
committer Ivan Pleshkov <pleshkov.ivan@gmail.com> 1732698560 +0100
gpu hnsw
add gpu indexing flag
add gpu vector storage time measurement
add chmod flag
build hnsw using gpu
scoring speedup
move gl_GlobalInvocationID to mains
glslc with o flag
fix tests
use gpu atomics instead of multiple runs
enable atomics
add timers
profile search
let shader work while points collection
use generations for visited flags
add gpu init timer
avoid entries download
remove gpu call timers for a while
rename to combined graph builder
move gpu graph builder
restore levels in link access
restore links_map
add gpu start condition
use graph builder links memory
remove mut's
list point ids before processing
download-upload links using methods only
cpu graph builder as separate struct
add cpu threads count
parallel cpu build
single threaded cpu preprocess
try to fix glove accuracy
unsafe Send and Sync for gpu builder
copy graph layers builder instead of moving
remove obsolete mut
fix build
move builders to arc
fix bfb deadlock
move uneccessary clear
run cpu and gpu in separate threads
many vectors storage buffers
debug gpu vector storage
fix vector storage upload bug
add cpu ful graph build
don't use non-ready links for point
try to resolve conflicts while upsert
add indexing unit test for measurements
add test
utilize cpu more efficient
dont clear bhaep
debug gpu runs
try to use vec4 instead of mat4
dump scores count
estimate gpu usage
print usage per run
start using working groups
build shaders for vulkan v1.3
start shaders subgroup
vk instance version 1.3
subgroup vector storage
fix test
nearest heap
nearest heap test
are you happy fmt
add assert to nearest heap test
are you happy fmt
gpu visited flags
start candidates heap
candidates heap push
debug nearest heap
remove subgroupExclusiveMin
fix build
are you happy clippy
remove links dependency from nearest heap
provide input count to nearest test
are you happy fmt
provide ef to shader
are you happy fmt
get subgroup size
move compiled shaders
are you happy fmt
fix workgroups count for vector storage compute test
return true subgroup size
do nearest heap on gpu with partial sorting
remove tmp shader
unite scores and indices
are you happy fmt
gpu nearest sorting
are you happy fmt
fix nearest heap test
links test with subgroups
gpu candidates start
fix candidates sorting
are you happy fmt
fix candidates test on m1 chip
move obsolete and test shaders
start new gpu search context
search context test
fix links uploading
debug hnsw searh on level test
fix gpu hnsw search on level test
move test initialization to separate function
gpu greedy search shader test
test heuristic shader
gpu greedy search test
gpu test heuristic
move greedy search into searcher
start insertion shader
find hnsw patch on gpu
fix gpu patches test
start gpu graph builder
equivalency test
fix layers check
add timers
are you happy fmt
quality test and fix multithreaded bug
debug tests
are you happy fmt
fix levels count and clear links
fix equivalency test
remove obsolete solution
trivial gpu indexing integration
gpu config and skip conflicts for large hnsw
glsl generic vector storage element type
gpu generic vector storage
enable vulkan features for f16 and u8
force_half_precision option
add force f16 test
fix u8 and f16 unit tests
apply generic shaders in hnsw construction
remove conflicts check; start cpu utilization
cpu prebuild
upload first point links to gpu
mark all points in cpu graph as ready
multithreaded cpu and gpu
fix equivalency test
remove cpu links copy
parallel patches clearing
fix multithreaded env
external layers source
more logs to debug cpu+gpu syncronization
clear gpu processed points count
remove some temp logs
fix cpu stop condition bug
new insert vector shader
greedy search returns only ids
download whole layer links
apply new links on gpu directly
update win shader script
parallel links loading
provide memory amount as setting and estimate groups count
start bq
gpu bq test
fix tests
shaders invocations refactor
revert max groups settings
gix build after rebase
fix bug after rebase
upload links to gpu in cpu thread
fix greedy search iteration
atomic checker for conflicts
dump graph changes for debugging
move iterator to separate file
start combined builder refactor
start cpu builder as separate struct
separate gpu and cpu construction
fix tests
fix atomics count bug
upload links to gpu as separate fn
glue cpu and gpu
gpu thread
link points on cpu while gpu is busy
fix gpu bq scoring test
use xor instead of calc it
reallocate with less groups if not enough gpu memory
update compiled shaders
don't use llvm gpu emulator
debug panic if no gpu
remove tmp debug dumping
dockerfile for gpu
customize candidates count
fix gpu storage size
runtime shader compilation
dim as define
nearest heap params as macro
candidates and links capacities as macro
parallel bitonic sort
use bubble sort too
nearest heap as shared
limited shared candidates heap
combine buble and bitonic sorts
one-argumented similarity
cache vector for scoring
dont check candidates overflow
do greedy before search
visited hashtable
Revert "visited hashtable"
This reverts commit 7540d04d3859d02f17908c874927d18076f3eb8e.
use less flushes
bulk sum calculation
Revert "bulk sum calculation"
This reverts commit 6ef78ceb8d6ba7d9bf2077804548df162052c749.
load vector in one read
Revert "load vector in one read"
This reverts commit ea980d0399b90f44638590d40fdbfb1d6ffa9504.
fix build after rebase
fix build after rebase
sq support
simplify bq uploading using constructed quantization
metrics
alignment depends on subgroup size
are you happy fmt
exact flag
use lazy static for device access
remove prints
test reallocation factor
visited flags capacity as a compile time constant
build gpu hnsw for payload blocks
add dynamically sized subgroup size support
fix build
update rust version for nvidia docker
dynamically sized subgroups
start device filter
test device filter
are you happy clippy
devices manager
provide device instead of flag, better locking
partial cpu permit release
free cpus when gpu enabled
use cpu is waiting is disabled
create device only when gpu is enabled
cpu permit release all cases
parallel indices count
provide queue index
small refactor of shader builder
are you happy fmt
quantization params as separate struct
pq refactor
fmt
remove upload_mapped_ptr from gpu buffer
refactor gpu buffer
check buffer range
buffer refactor
refactor pipeline
move shader compiler to instance
refactor context
more refactor
no unwraps in gpu crate
clippy
apply pr changes
are you happy fmt
dont use shaderc in segment
more gpu tests
pq bq tests
provide bq option
fix bq for small dim
bind pq buffers
pq more complex test
cover all storage types by tests
test multi and vectors as iterator
multivectors upload data
multivectors shader
fix multivector tests
refactor shader builder
move storage tests into separate file
move quantization to separate file
refactor gpu vector storage creation
refactor gpu quantization api
dont convert to float when unnecessary
more comments
small upload vectors refactor
fix build after rebase
fix tests after rebase
different vk queue priorities capacity
experiment with heaps
reverse candidates heap
scan only half of candidates
avoid heap code duplication
call subgroup elect inside bheap
united nearest and candidates heaps
remove cands and nearest heaps shaders
fix tests
start linear implementation
remove nearest and candidates entities
visited flags with id remap
simplify cpu side
move hnsw search shader into tests
simplify
simplify gpu links
test visited flags remap
remove GpuSearchContextGroupAllocation
stopper and allow integrated flag
add timeout comment
refactor batched points
refactor links and device manager
are you happy clippy
fix build after rebase
fix build after rebase
remove temp files
create gpu vectors in segment constructor
clean up
clean up
refactor gpu feature
refactor gpu construction in hnsw.rs
provide gpu device
remove pub
simplify main graph
renames
telemetry and settings refactor
update cargo lock
remove env_logger from deps
update openapi
gpu hnsw unit test
rename test
log device name while indexing
* are you happy clippy
* are you happy clippy
* dockerfile with amd and nvidia gpu support
* add gpu arg to stages
* clear after rebase
* Apply suggestions from code review
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
* review remarks
* fix CI
* cnahge arg to env
---------
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
* gpu hnsw construction
* fix build
* are you happy clippy
* are you happy clippy
* review remarks
* more comments
* external gpu vector storage
* decompose gpu inserter
* are you happy codespell
* are you happy clippy
* parent 33b2c176cb
author Ivan Pleshkov <pleshkov.ivan@gmail.com> 1729676438 +0200
committer Ivan Pleshkov <pleshkov.ivan@gmail.com> 1732012991 +0100
parent 33b2c176cb
author Ivan Pleshkov <pleshkov.ivan@gmail.com> 1729676438 +0200
committer Ivan Pleshkov <pleshkov.ivan@gmail.com> 1732012977 +0100
parent 33b2c176cb
author Ivan Pleshkov <pleshkov.ivan@gmail.com> 1729676438 +0200
committer Ivan Pleshkov <pleshkov.ivan@gmail.com> 1732012901 +0100
parent 33b2c176cb
author Ivan Pleshkov <pleshkov.ivan@gmail.com> 1729676438 +0200
committer Ivan Pleshkov <pleshkov.ivan@gmail.com> 1732012766 +0100
parent 33b2c176cb
author Ivan Pleshkov <pleshkov.ivan@gmail.com> 1729676438 +0200
committer Ivan Pleshkov <pleshkov.ivan@gmail.com> 1732012730 +0100
vulkan api wrapper crate
fmt
gpu is not a workspace member
exclude gpu from workspace
disable tests for gpu crate
ash version up
change collection for pipeline builder
gpu test
gpu is not optional
check device features
remove feature
make all fields private
more comments
are you happy fmt
qdrant team as authors
gpu vectors
move changes from magic branch
move storage updates
are you happy codespell
are you happy clippy
don't make storage_impl public
review remarks
are you happy clippy
are you happy fmt
* Update lib/segment/src/index/hnsw_index/gpu/gpu_vector_storage/tests.rs
Co-authored-by: Tim Visée <tim+github@visee.me>
* review remarks
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
* vulkan api wrapper crate
* fmt
* gpu is not a workspace member
* are you happy codespell
* try to fix CI
* exclude gpu from workspace
* fix build for macos
* disable tests for gpu crate
* fix codespell
* provide gpu feature into gpu trait
* ash version up
* change collection for pipeline builder
* gpu test
* gpu is not optional
* check extensions and layers presense
* check device features
* remove feature
* make all fields private
* more comments
* shader compilation options
* are you happy clippy
* are you happy fmt
* Update lib/gpu/Cargo.toml
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* qdrant team as authors
* merge more changes from magic branch
* are you happy clippy
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* define mmap geo index
fix compilation
deleted flags
load new mmap
geo index tests
fix tests
fix build after rebase
add files list
* refactor get_stored_sub_regions output type
* review remanings
* mmap geo index api
* fix after rebase
* define mmap geo index
fix compilation
deleted flags
load new mmap
geo index tests
fix tests
fix build after rebase
add files list
* are you happy fmt
* refactor get_stored_sub_regions output type
* repr(C) for geohash
* Replace manual flattening with flat map iterator
* review remanings
* remove panics
* add files for snapshot
* rename Dynamic into InMemory
* are you happy fmt
* review renames
* Use copied rather than cloned
* minor review refactoring + comments
---------
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: generall <andrey@vasnetsov.com>
* packed geohash
* refactor
* are you happy codespell
* review remarks
* better iterator
* iterator with only one field
* encode db key as smolstr
* Update geohash_starts_with
---------
Co-authored-by: xzfc <xzfcpw@gmail.com>
* define mmap map index
add point to values
add mmap hash map
are you happy fmt
use new mmap hashmap methods
build index
saturating_sub
are you happy clippy
fix tests build
* integrate facets for mmap index
* mmap tests
* fix ci
* review remarks
* review remarks
* mmap numeric index
dynamic numeric index
remove pub from fields
add mmap enum
remove vec<u8> as number
get_value without self
mmap save load
trait for conversion from P into T
save-load points-to-values
fix unit tests
are you happy clippy
fix ci
apply changes from #4743
NumericIndex: add new_mmap() method
remove mmapvalue trait from immutable
small refactor
implement from_referenced for str
are you happy clippy
deleted flags by idx instead of pairs
* fix build after rebase
* review remarks
* tests macos
* increase deleted count while deletion
* mmap hashmap iterate over keys
* fix tests
* Use associated consts rather than static methods
---------
Co-authored-by: Albert Safin <xzfcpw@gmail.com>
* definitions
* create file
* ranges
* get values
* are you happy clippy
* remove unnecessary function
* are you happy codespell
* review fixes
* are you happy clippy
* fmt
* add padding
* update map public functions as in the dev
* are you happy clippy
* review remarks
* generic structure
* review remarks
* review fixes
* field index values as iterator instead of slice
* Use consistent assertions
* fix wrong values_count update
---------
Co-authored-by: timvisee <tim@visee.me>
* Max size of multivector to fix chunk in storage
* Update lib/common/common/src/validation.rs
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* fix hnsw full scan threshold
* add test
* are you happy clippy
* separate open_vector_storage
* remove public fields from builder
* wip: do not create segment in builder before build
* avoid arc in storage test and low-level loading functions
* WIP: remove internal segment from SegmentBuilder
* fmt
* finalize segment builder fixes
* Revert "are you happy clippy"
This reverts commit c04afa6989.
* Revert "add test"
This reverts commit 8e7ad6207e.
* Revert "fix hnsw full scan threshold"
This reverts commit 8904443fcb.
* remove _daatabse from builder
* fix optimizer test
* fix id tracker versions persistence
* do flush for segment components on build
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* use correct vector storage size
* remove dim from segment entry
* are you happy fmt
* codespell and proportions
* remove obsolete comment
* remove `try_vector_dim`
* are you happy fmt
* remove todo
* revert code of conduct
* check div 0
* Simplify a bit with max iterator
* Update lib/segment/src/index/hnsw_index/hnsw.rs
Co-authored-by: Tim Visée <tim+github@visee.me>
* are you happy fmt
* Update lib/segment/src/index/plain_payload_index.rs
Co-authored-by: Tim Visée <tim+github@visee.me>
* review fix
* set full scan threshold 0 for test
* use u128 also for multivector storages
* fix sparse vector size calculation
* Move size calculation into if-branch
---------
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Tim Visée <tim+github@visee.me>
