mirror of
https://github.com/qdrant/qdrant.git
synced 2026-08-03 00:20:57 -05:00
d48cd31a36
* Minor collection optimizer cleanup * Make optimizers better aware of available vs soft deleted points * Fix incorrect deleted state on proxy segment for double delete * Rename upsert_vector to upsert_point, because we work with points * Refactor point methods for more clear and consistent naming * Replace internal_size in IdTracker with total_point_count * Keep track of vector deletion count on storage creation * Add sparse index optimizer, to optimize indexes with high deletion count * Add minimum vector count threshold to sparse index optimizer * Add sparse index optimizer test * Use consistent naming, write vector in full everywhere * Simplify vacuum optimizer a bit * Merge sparse index optimizer into vacuum optimizer * Improve update_from in segment builder by returning early * More accurately count vectors in segment optimizer * Remove random from vacuum optimizer tests to make them more reliable * Don't expose the total points in segment info, use available points * Process review feedback * Compare available vectors against indexed ones in vacuum optimizer This is much better than using the number of soft-deleted vectors when the segment was created for calculations. Not to mention that value had other problems as well. * Remove create_deleted_vector_count field, update vacuum test parameters * Potentially solve out of bound panic when building index * Review fixes: - Propagate deleted flags into payload hnsw building - Use `total` number of points for building HNSW instead of number of available points - minor refactoring of `hnsw_config` copy -> clone - Better detection of `indexed_points` in HNSW * fix assert condition * Optional named vectors optimizer reveiw 2 (#1794) * review with Ivan * fmt * remove available_vector_count from segment entry * remove total_point_count from segment entry --------- Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com> * rollback changes in deleted count in proxy segment * improve vector threshold detection logic in optimized_segment_builder * style changes * fix propagate deleted points to vectors * Fix typo in method name --------- Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com> Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>