mirror of
https://github.com/qdrant/qdrant.git
synced 2026-08-06 18:10:58 -05:00
bce35092ed
* Use resize rather than while-push loop
* Add deleted flags to simple vector storage
* Add deleted flag to memmap vector storage
* Map BitSlice on mmap file for deleted flags
* Use vector specific deletion BitSlice in RawScorer
* Use BitSlice for deleted points, fix check point logic, clarify names
* Extract div_ceil function to shared module
* We can use unchecked set and replace because we just checked the length
* Add deleted count function to vector storage
* Add vector storage point deletion tests
* Keep deleted state in simple vector storage with update_from, add test
* Keep deleted state in memmap vector storage with update_from, add test
* Simplify div_ceil
* Improve deletion handling in update_from in mmap vector storage
* Improve performance, use trickery to get BitSlice view over deleted mmap
* Use BitSlice where possible, construct BitVec more efficiently
* Incorporate vector specific delete flags in quantized raw scorer
* Don't pin MmapMut, it is not required
* With quantization, keep mmap deleted flags in RAM for better performance
* Advice the kernel to prepare deleted flags mmap for faster future access
* Simplify deleted bitslice access, add bound check, remove unused function
* Fix compilation on Windows
* Cleanup
* Rename delete functions to delete_{point,vec} to prevent confusion
* Use then_some rather than match a boolean
* Lock deleted flags in memory only when quantization is available
* Add docs and stabilize issue link to dev_ceil
* Flush deleted mmap when closing segment
This requires us to to wrap the memory map struct in an Arc and Mutex.
Though this may look inefficient, it doesn't have a negative side effect
on deleted flag performance, because the flags are accessed through a
BitSlice that is separate and doesn't use locking.
* Rename some point functions to vec because that makes more sense
* Simplify delete flag fetching option, use deref func instead of asterisk
* Do not calculate slice size manually, use size_of_val
* remove test raw scorer
* use deref in check
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>