CAGRA search: int64_t indices in the API#769
Merged
rapids-bot[bot] merged 17 commits intorapidsai:branch-25.04from Apr 1, 2025
Merged
CAGRA search: int64_t indices in the API#769rapids-bot[bot] merged 17 commits intorapidsai:branch-25.04from
rapids-bot[bot] merged 17 commits intorapidsai:branch-25.04from
Conversation
tfeher
requested changes
Mar 25, 2025
Contributor
tfeher
left a comment
There was a problem hiding this comment.
Thanks Artem, it looks good, just a few documentation issues. It is great that you could implement this without additional kernel instantiations.
cpp/src/neighbors/detail/cagra/search_single_cta_kernel-inl.cuh
Outdated
Show resolved
Hide resolved
Co-authored-by: Tamas Bela Feher <tfeher@nvidia.com>
tfeher
approved these changes
Mar 31, 2025
Contributor
tfeher
left a comment
There was a problem hiding this comment.
Thanks Artem for the update LGTM!
Contributor
|
/merge |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add
int64_tas the type of returned neighbors in cagra::search while keeping the index type unchanged.Down the implementation, the PR introduces the
OutputIndexTtemplate parameter and handles it differently for each implementation:sizeof(OutputIndexT)as a tag in the pointer. Based on this tag, a proper output-writing function is dispatched in the epilogue of the single-cta kernel. By using this technique I avoid instantiating multiple kernels at the negligible runtime cost.Note: we cannot simply instantiate cagra with index type
int64_t, because this will make the graph twice larger and will reduce the search throughput.To test the new search overloads, I turn
AnnCagraTest::testCagrainto a template and add just two instances to the*.cu filesto avoid bloating the number of tests.The new search overloads are used in the ann-bench wrappers, which slightly improves the reported QPS in some cases due to reducing the index conversion overhead.