[REVIEW] Update cugraph_dgl to use the new FeatureStore#3143
Merged
rapids-bot[bot] merged 17 commits intorapidsai:branch-23.02from Jan 19, 2023
Merged
[REVIEW] Update cugraph_dgl to use the new FeatureStore#3143rapids-bot[bot] merged 17 commits intorapidsai:branch-23.02from
rapids-bot[bot] merged 17 commits intorapidsai:branch-23.02from
Conversation
Codecov ReportBase: 55.31% // Head: 55.60% // Increases project coverage by
Additional details and impacted files@@ Coverage Diff @@
## branch-23.02 #3143 +/- ##
================================================
+ Coverage 55.31% 55.60% +0.29%
================================================
Files 148 143 -5
Lines 9423 9318 -105
================================================
- Hits 5212 5181 -31
+ Misses 4211 4137 -74 Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
rlratzel
approved these changes
Jan 18, 2023
Contributor
rlratzel
left a comment
There was a problem hiding this comment.
Thanks, this is a large PR which I may have to go over again post-merge, but I did find a few minor things now which shouldn't hold up approval.
Co-authored-by: Rick Ratzel <3039903+rlratzel@users.noreply.github.com>
Member
|
/merge |
nv-rliu
pushed a commit
to nv-rliu/cugraph-gnn
that referenced
this pull request
Jul 15, 2025
This PR updates `cugraph_dgl` to use the new FeatureStore (We see a drastic increase in fetching feature perf, 1900x even with host pytorch Tensors. I expect better perf with UVA tensors which we should be able to do with `rmm's managed memory` . Speedup : - [x] 1900x faster get_feature_data - [x] 20x faster in Heterograph Sampling on obgn-mag ```md | Implimentation | N_GPUS | Earlier Time Taken (s) | PR time taken (s) | SpeedUP | Batch Size | |-------------------|--------|------------------------|-------------------|---------|------------| | DGL Upstream CUDA | 1 | 0.26 | NA | NA | 100_000 | | Cugraph Local | 1 | 2.175 | 0.499 | 5x | 100_000 | | Cugraph Dask | 1 | 113.511 | 5.20 | 21.73x | 100_000 | | Cugraph Dask | 2 | 106.184 | 5.722 | 18x | 100_000 | | Cugraph Dask | 4 | 102.991 | 6.302 | 16.19x | 100_000 | ``` This PR also closes rapidsai/cugraph#3146 and rapidsai/cugraph#3039 and rapidsai/cugraph#3106 - [x] Passes cugraph_dgl unit tests - [x] Add tests for `dgl_uniform_sampler.py` - [x] Verify cugraph_dgl works with `obgn-products` dataloader <s>Verify cugraph_dgl works with `ogbn-papers100M` (To verify large scale scaling of the graphstore) </s> Need bigger machine will do in a followup - [x] Add a dataloader based examples to verify E2E data loading works. To decice: <s>Should we remove all the dead-code related to cugraph_service_store, cugraph_store etc from the cugraph repo ? </s> Benchmarks: PR: ```python3 indices=np.random.randint(0,cugraph_g.num_nodes(),size=20_000) %time output = cugraph_g.get_node_storage(key='feat',ntype='_N').fetch(indices) CPU times: user 132 ms, sys: 263 ms, total: 395 ms Wall time: 14 ms ``` Mainline: ```python3 indices=np.random.randint(0,cugraph_g.num_nodes(),size=20_000) %time output = cugraph_g.get_node_storage(key='feat',ntype='_N').fetch(indices) CPU times: user 132 ms, sys: 263 ms, total: 395 ms Wall time: 27.8 s ``` Authors: - Vibhu Jawa (https://github.com/VibhuJawa) Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: rapidsai/cugraph#3143
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR updates
cugraph_dglto use the new FeatureStore (We see a drastic increase in fetching feature perf, 1900x even with host pytorch Tensors. I expect better perf with UVA tensors which we should be able to do withrmm's managed memory.Speedup :
This PR also closes #3146 and #3039 and #3106
dgl_uniform_sampler.pyobgn-productsdataloaderVerify cugraph_dgl works withogbn-papers100M(To verify large scale scaling of the graphstore)Need bigger machine will do in a followup
To decice:
Should we remove all the dead-code related to cugraph_service_store, cugraph_store etc from the cugraph repo ?Benchmarks:
PR:
Mainline: