Skip to content

Improve memory footprint and performance of graph creation#3542

Merged
rapids-bot[bot] merged 20 commits intorapidsai:branch-23.06from
VibhuJawa:improve_memory_footprint_symmetrsization
May 17, 2023
Merged

Improve memory footprint and performance of graph creation#3542
rapids-bot[bot] merged 20 commits intorapidsai:branch-23.06from
VibhuJawa:improve_memory_footprint_symmetrsization

Conversation

@VibhuJawa
Copy link
Member

@VibhuJawa VibhuJawa commented May 4, 2023

This PR improves the memory footprint in graph creation and resolves story: #3439

The metric we are concerned about is input_to_peak_ratio which is currently 12.1x. Ideally this should be in 2x range.

Metric Meaning: How much memory expands for an input graph of size x. So if you want to analyze a graph of size x, we need 12x GPU ram currently.

Memory overhead ratio reduction:

  • 12x to 6.14x for undirected (Reverse edges added)
  • 5.9x to 3.07x for directed graphs (No reverse edges)

Time Reduction:

  • 4.2x for scale 24
  • 1.67x for scale 23

image

Comparison Table

scale num_input_edges directed renumber input_memory_per_worker PR peak_allocation_across_workers Main: peak_allocation_across_workers Main: input_to_peak_ratio PR input_to_peak_ratio
23 134217728 TRUE TRUE 2.0GB 8.1GB 11.8GB 5.91 4.07
23 134217728 TRUE FALSE 2.0GB 6.1GB 11.8GB 5.91 3.07
23 134217728 FALSE TRUE 2.0GB 12.2GB 23.3GB 11.67 6.08
23 134217728 FALSE FALSE 2.0GB 12.2GB 23.3GB 11.67 6.08
24 268435456 TRUE TRUE 4.0GB 12.3GB 23.8GB 5.94 3.08
24 268435456 TRUE FALSE 4.0GB 12.3GB 23.8GB 5.94 3.08
24 268435456 FALSE TRUE 4.0GB 24.4GB 29.4GB 7.35 6.11
24 268435456 FALSE FALSE 4.0GB 24.4GB 29.4GB 7.35 6.11

Graph Creation:

Scale PR Time Main Time Speed Up
23 2.4 4.1 1.68
23 2.4 4.1 1.68
23 3.6 8.0 2.22
23 3.6 8.0 2.24
24 4.4 9.3 2.12
24 4.4 9.3 2.12
24 7.9 38.7 4.89
24 7.9 37.9 4.81
25 9.6 41.0 4.26
25 9.6 40.8 4.25
image

CC: @rlratzel

@VibhuJawa VibhuJawa added non-breaking Non-breaking change improvement Improvement / enhancement to an existing function labels May 4, 2023
@alexbarghi-nv
Copy link
Member

I think this PR may have some bugs, you may want to run the MG tests before finalizing it

@alexbarghi-nv
Copy link
Member

I ran into issues trying to use this with the latest changes to sampling and edge properties

@VibhuJawa VibhuJawa changed the title Improve memory footprint in graph creation Improve memory footprint and performance of graph creation May 12, 2023
@VibhuJawa VibhuJawa marked this pull request as ready for review May 16, 2023 06:30
@VibhuJawa VibhuJawa requested a review from a team as a code owner May 16, 2023 06:30
@VibhuJawa
Copy link
Member Author

VibhuJawa commented May 16, 2023

I think this PR may have some bugs, you may want to run the MG tests before finalizing it

@alexbarghi-nv , Can you try again. i think i have resolved all the bugs from my side and testing on multiple GPU configurations.

Copy link
Member

@alexbarghi-nv alexbarghi-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@alexbarghi-nv
Copy link
Member

/merge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Improvement / enhancement to an existing function non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants