There are a number of perf optimizations that can improve the GPU Vamana index build in cuVS. This issue tracks these optimizations that may be applied with several PRs to limit scope and maintain stability. Perf estimates for each optimization are rough and attempt to be conservative.
Note: speedups are speculative and should not be relied on until optimizations are tested
High Impact / Priority Optimizations
Overall perf gain estimate
- 2-2.5x speedup of
GreedySearch
- 1.5-2.2x speedup of
RobustPrune
Depending on the dataset, the time breakdown can vary between these kernels. On narrow vectors, GreedySearch dominates, while RobustPrune is more costly for wide vectors.
Expected E2E perf gains: 1.6x-2.4x
E2E perf achieved thus far: 1x-1.4x
Additional more challenging / speculative optimizations
There are a number of perf optimizations that can improve the GPU Vamana index build in cuVS. This issue tracks these optimizations that may be applied with several PRs to limit scope and maintain stability. Perf estimates for each optimization are rough and attempt to be conservative.
Note: speedups are speculative and should not be relied on until optimizations are tested
High Impact / Priority Optimizations
GreedySearchandRobustPrunekernels from 32 to 128-256. Block size limits theoretical occupancy to 50%, estimated 1.2x-1.5x kernel perf improvement - got 0% to 38% kernel speedupenqueue_all_neighborsfor improved parallelism. Could significantly reduce sync overheads on method thatGreedySearchspends 30-40% of runtime on. Expected 1.4-1.6x GreedySearch improvement - Optimization unsuccessful, no speedup achievedRobustPrune. Can reduce syncs by 10x during merge loop.RobustPrunespends 20% of time here, so estimate 1.3-1.4x improvement ofRobustPrune. - Achieved 0% to 28% kernel spedupcheck_visiteddata structure ofRobustPrune(hash set or bitset). Currently a linear search, estimate a 1.2x speedup toGreedySearch.Overall perf gain estimate
GreedySearchRobustPruneDepending on the dataset, the time breakdown can vary between these kernels. On narrow vectors,
GreedySearchdominates, whileRobustPruneis more costly for wide vectors.Expected E2E perf gains: 1.6x-2.4x
E2E perf achieved thus far: 1x-1.4x
Additional more challenging / speculative optimizations
RobustPrune. Shared memory is a major occupancy limiter for these kernels, so this may not be feasible.PriorityQueuestructure by replacing them with warp primitives.