[Java] New off-heap Dataset support for CAGRA and Bruteforce#902
Merged
rapids-bot[bot] merged 19 commits intorapidsai:branch-25.06from May 29, 2025
Merged
Conversation
Contributor
Author
|
FYI, @ChrisHegarty @narangvivek10. |
cjnolet
added
improvement
java/cuvs-java/src/main/java22/com/nvidia/cuvs/internal/DatasetImpl.java
Outdated
Show resolved
Hide resolved
java/cuvs-java/src/main/java22/com/nvidia/cuvs/internal/DatasetImpl.java
Show resolved
Hide resolved
java/cuvs-java/src/main/java22/com/nvidia/cuvs/internal/DatasetImpl.java
Outdated
Show resolved
Hide resolved
Contributor
|
Also, maybe update the title to |
punAhuja
reviewed
May 27, 2025
java/cuvs-java/src/test/java/com/nvidia/cuvs/BruteForceRandomizedIT.java
Show resolved
Hide resolved
chatman
changed the title
chatman
changed the title
Contributor
Author
|
@narangvivek10 @punAhuja Incorporated your feedback, thanks! Can you approve the PR? |
Member
|
/ok to test 7a75d9a |
Co-authored-by: MithunR <mythrocks@gmail.com>
Contributor
Author
|
Thanks @mythrocks , incorporated your suggestions. I think we need to add a style check for the Java project and standardize tabs vs spaces in a separate PR. This codebase is a mix right now :-( |
Contributor
Author
|
@mythrocks can you please trigger a CI run? |
Contributor
Author
|
@cjnolet Can you please review and merge this? |
mythrocks
approved these changes
May 28, 2025
Member
|
/ok to test 6148cff |
cjnolet
approved these changes
May 28, 2025
Member
|
/merge |
github-project-automation
bot
moved this from In Progress
to Done
in Vector Search, ML, & Data Mining Release Board
mythrocks
pushed a commit
to mythrocks/cuvs
that referenced
this pull request
Jun 3, 2025
…i#902) As reported in rapidsai#698, current `withDataset(float[][] arr)` requires the entire dataset to be copied in heap first, before writing out the MemorySegment for it. Introducing a new `Dataset` (interface and impl) support with a `addVector(float[] vector)` support for adding the vectors into the MemorySegment one by one, without needing to load them all at once. Authors: - Ishan Chattopadhyaya (https://github.com/chatman) - Vivek Narang (https://github.com/narangvivek10) Approvers: - MithunR (https://github.com/mythrocks) - Corey J. Nolet (https://github.com/cjnolet) URL: rapidsai#902
Contributor
Author
|
That's a good idea. Please feel free to rig up a PR. Thanks!
…On Fri, 13 Jun, 2025, 7:28 pm Lorenzo Dematté, ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In
java/cuvs-java/src/main/java22/com/nvidia/cuvs/internal/BruteForceIndexImpl.java
<#902 (comment)>:
> + private float[][] vectors;
+ private Dataset dataset;
So, externally you can provide data either as a float[][] or a Dataset,
but internally it will just be a Dataset.
—
Reply to this email directly, view it on GitHub
<#902 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABDCR5BWXEQDJV6NPM6PY4T3DLKJNAVCNFSM6AAAAAB5HCKD6SVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDSMRUHE2DCMJUGA>
.
You are receiving this because you were assigned.Message ID:
***@***.***>
|
rapids-bot bot
pushed a commit
that referenced
this pull request
Jul 6, 2025
This PR is a follow-up from #902. Still WIP (see self-comments on the changes) but I'd like some early feedback. Authors: - Lorenzo Dematté (https://github.com/ldematte) - MithunR (https://github.com/mythrocks) Approvers: - Chris Hegarty (https://github.com/ChrisHegarty) - MithunR (https://github.com/mythrocks) URL: #1024
punAhuja
pushed a commit
to SearchScale/cuvs
that referenced
this pull request
Jul 15, 2025
This PR is a follow-up from rapidsai#902. Still WIP (see self-comments on the changes) but I'd like some early feedback. Authors: - Lorenzo Dematté (https://github.com/ldematte) - MithunR (https://github.com/mythrocks) Approvers: - Chris Hegarty (https://github.com/ChrisHegarty) - MithunR (https://github.com/mythrocks) URL: rapidsai#1024
punAhuja
pushed a commit
to SearchScale/cuvs
that referenced
this pull request
Jul 16, 2025
This PR is a follow-up from rapidsai#902. Still WIP (see self-comments on the changes) but I'd like some early feedback. Authors: - Lorenzo Dematté (https://github.com/ldematte) - MithunR (https://github.com/mythrocks) Approvers: - Chris Hegarty (https://github.com/ChrisHegarty) - MithunR (https://github.com/mythrocks) URL: rapidsai#1024
rapids-bot bot
pushed a commit
that referenced
this pull request
Jul 26, 2025
In #902 and #1034 we introduced a `Dataset` interface to support on-heap and off-heap ("native") memory seamlessly as inputs for cagra and bruteforce index building. As we expand the functionality of cuvs-java, we realized we have similar needs for outputs (see e.g. #1105 / #1102 or #1104). This PR extends `Dataset` to support being used as an output, wrapping native (off-heap) memory in a convenient and efficient way, and providing common utilities to transform to and from on-heap memory. This work is inspired by the existing raft `mdspan` and `DLTensor` data structures, but tailored to our needs (2d only, just 3 data types, etc.). The PR keeps the current implementation simple and minimal on purpose, but structured in a way that is simple to extend. By itself, the PR is just a refactoring to extend the `Dataset` implementation and reorganize the implementation classes; its real usefulness will be in using it in the PRs mentioned above (in fact, this PR has been extracted from #1105). The implementation class hierarchy is implemented with future extensions in mind: atm we have one `HostMemoryDatasetImpl`, but we are already thinking to have a corresponding `DeviceMemoryDatasetImpl` that will wrap and manage (views) on GPU memory to avoid (in some cases) extra copies of data from GPU memory to CPU memory only to process them or forward them to another algorithm (e.g quantization followed by indexing). Future work will also include add support/refactoring to allocate and manage GPU memory and DLTensors (e.g. working better with/refactoring `prepareTensor`). Authors: - Lorenzo Dematté (https://github.com/ldematte) - MithunR (https://github.com/mythrocks) Approvers: - MithunR (https://github.com/mythrocks) URL: #1111
lowener
pushed a commit
to lowener/cuvs
that referenced
this pull request
Aug 11, 2025
…#1111) In rapidsai#902 and rapidsai#1034 we introduced a `Dataset` interface to support on-heap and off-heap ("native") memory seamlessly as inputs for cagra and bruteforce index building. As we expand the functionality of cuvs-java, we realized we have similar needs for outputs (see e.g. rapidsai#1105 / rapidsai#1102 or rapidsai#1104). This PR extends `Dataset` to support being used as an output, wrapping native (off-heap) memory in a convenient and efficient way, and providing common utilities to transform to and from on-heap memory. This work is inspired by the existing raft `mdspan` and `DLTensor` data structures, but tailored to our needs (2d only, just 3 data types, etc.). The PR keeps the current implementation simple and minimal on purpose, but structured in a way that is simple to extend. By itself, the PR is just a refactoring to extend the `Dataset` implementation and reorganize the implementation classes; its real usefulness will be in using it in the PRs mentioned above (in fact, this PR has been extracted from rapidsai#1105). The implementation class hierarchy is implemented with future extensions in mind: atm we have one `HostMemoryDatasetImpl`, but we are already thinking to have a corresponding `DeviceMemoryDatasetImpl` that will wrap and manage (views) on GPU memory to avoid (in some cases) extra copies of data from GPU memory to CPU memory only to process them or forward them to another algorithm (e.g quantization followed by indexing). Future work will also include add support/refactoring to allocate and manage GPU memory and DLTensors (e.g. working better with/refactoring `prepareTensor`). Authors: - Lorenzo Dematté (https://github.com/ldematte) - MithunR (https://github.com/mythrocks) Approvers: - MithunR (https://github.com/mythrocks) URL: rapidsai#1111
enp1s0
pushed a commit
to enp1s0/cuvs
that referenced
this pull request
Aug 22, 2025
…#1111) In rapidsai#902 and rapidsai#1034 we introduced a `Dataset` interface to support on-heap and off-heap ("native") memory seamlessly as inputs for cagra and bruteforce index building. As we expand the functionality of cuvs-java, we realized we have similar needs for outputs (see e.g. rapidsai#1105 / rapidsai#1102 or rapidsai#1104). This PR extends `Dataset` to support being used as an output, wrapping native (off-heap) memory in a convenient and efficient way, and providing common utilities to transform to and from on-heap memory. This work is inspired by the existing raft `mdspan` and `DLTensor` data structures, but tailored to our needs (2d only, just 3 data types, etc.). The PR keeps the current implementation simple and minimal on purpose, but structured in a way that is simple to extend. By itself, the PR is just a refactoring to extend the `Dataset` implementation and reorganize the implementation classes; its real usefulness will be in using it in the PRs mentioned above (in fact, this PR has been extracted from rapidsai#1105). The implementation class hierarchy is implemented with future extensions in mind: atm we have one `HostMemoryDatasetImpl`, but we are already thinking to have a corresponding `DeviceMemoryDatasetImpl` that will wrap and manage (views) on GPU memory to avoid (in some cases) extra copies of data from GPU memory to CPU memory only to process them or forward them to another algorithm (e.g quantization followed by indexing). Future work will also include add support/refactoring to allocate and manage GPU memory and DLTensors (e.g. working better with/refactoring `prepareTensor`). Authors: - Lorenzo Dematté (https://github.com/ldematte) - MithunR (https://github.com/mythrocks) Approvers: - MithunR (https://github.com/mythrocks) URL: rapidsai#1111
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
As reported in #698, current
withDataset(float[][] arr)requires the entire dataset to be copied in heap first, before writing out the MemorySegment for it.Introducing a new
Dataset(interface and impl) support with aaddVector(float[] vector)support for adding the vectors into the MemorySegment one by one, without needing to load them all at once.