Skip to content

Feature Quantization [5/5]: Distributed Integration - #737

Merged
jchmura-sc merged 67 commits into
mainfrom
jchmura/feature_quant_integration
Aug 13, 2026
Merged

Feature Quantization [5/5]: Distributed Integration#737
jchmura-sc merged 67 commits into
mainfrom
jchmura/feature_quant_integration

Conversation

@jchmura-sc

@jchmura-sc jchmura-sc commented Aug 4, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Purpose
The purpose of this PR is to integrate feature quantization into our distributed data, loading, partioning, and sampling setup:

  • graph partitioner partitions quantized node feature using node partion book
  • neighbor samplers fetch quantized node features over rpc during subgraph hydration
  • dataloaders dequantize/reconstruct node feature during collate_fn on gpu
  • distributed dataset takes quantization metadata (to pass down to dataloaders) and new quantized node feature store

Context

Updated Changelog.md? NO

Ready for code review?: NO

@jchmura-sc jchmura-sc self-assigned this Aug 4, 2026
This was referenced Aug 4, 2026
Merged
Merged
Merged
Merged
@jchmura-sc

jchmura-sc commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author

/unit_test

@github-actions

github-actions Bot commented Aug 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

GiGL Automation

@ 16:27:16UTC : 🔄 C++ Unit Test started.

@ 16:29:10UTC : ✅ Workflow completed successfully.

@github-actions

github-actions Bot commented Aug 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

GiGL Automation

@ 16:27:19UTC : 🔄 Python Unit Test started.

@ 17:55:38UTC : ❌ Workflow failed.
Please check the logs for more details.

@github-actions

github-actions Bot commented Aug 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

GiGL Automation

@ 16:27:26UTC : 🔄 Scala Unit Test started.

@ 16:39:04UTC : ✅ Workflow completed successfully.

@jchmura-sc

jchmura-sc commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author

/unit_test

@github-actions

github-actions Bot commented Aug 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

GiGL Automation

@ 18:32:29UTC : 🔄 C++ Unit Test started.

@ 18:34:28UTC : ✅ Workflow completed successfully.

@github-actions

github-actions Bot commented Aug 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

GiGL Automation

@ 18:32:30UTC : 🔄 Scala Unit Test started.

@ 18:44:15UTC : ✅ Workflow completed successfully.

@github-actions

github-actions Bot commented Aug 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

GiGL Automation

@ 18:32:36UTC : 🔄 Python Unit Test started.

@ 19:56:40UTC : ✅ Workflow completed successfully.

kmontemayor2-sc
kmontemayor2-sc reviewed Aug 12, 2026
View reviewed changes

@kmontemayor2-sc kmontemayor2-sc left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Jacob! I the robot feedback seems valid but otherwise this LGTM :)

Comment thread gigl/distributed/dataset_factory.py
Comment thread tests/unit/distributed/distributed_neighborloader_test.py Outdated
Comment thread gigl/distributed/utils/neighborloader.py
@jchmura-sc

jchmura-sc commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author

/unit_test

@github-actions

github-actions Bot commented Aug 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

GiGL Automation

@ 20:21:48UTC : 🔄 Scala Unit Test started.

@ 20:33:20UTC : ✅ Workflow completed successfully.

@github-actions

github-actions Bot commented Aug 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

GiGL Automation

@ 20:21:48UTC : 🔄 Python Unit Test started.

@ 21:39:40UTC : ✅ Workflow completed successfully.

@github-actions

github-actions Bot commented Aug 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

GiGL Automation

@ 20:21:48UTC : 🔄 C++ Unit Test started.

@ 20:23:39UTC : ✅ Workflow completed successfully.

kmontemayor2-sc
kmontemayor2-sc approved these changes Aug 12, 2026
View reviewed changes

@kmontemayor2-sc kmontemayor2-sc left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, thanks Jacob :)

mkolodner-sc
mkolodner-sc approved these changes Aug 13, 2026
View reviewed changes

@mkolodner-sc mkolodner-sc left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Jacob!

Comment thread gigl/distributed/utils/neighborloader.py Outdated
@jchmura-sc
jchmura-sc added this pull request to the merge queue Aug 13, 2026
Merged via the queue into main with commit d7cfd87 Aug 13, 2026
7 checks passed
@jchmura-sc
jchmura-sc deleted the jchmura/feature_quant_integration branch August 13, 2026 21:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mkolodner-sc mkolodner-sc mkolodner-sc approved these changes
@kmontemayor2-sc kmontemayor2-sc kmontemayor2-sc approved these changes
@svij-sc svij-sc Awaiting requested review from svij-sc svij-sc is a code owner
@nshah-sc nshah-sc Awaiting requested review from nshah-sc nshah-sc is a code owner
@yliu2-sc yliu2-sc Awaiting requested review from yliu2-sc yliu2-sc is a code owner
@xgao4-sc xgao4-sc Awaiting requested review from xgao4-sc xgao4-sc is a code owner
@zfan3-sc zfan3-sc Awaiting requested review from zfan3-sc zfan3-sc is a code owner

Assignees

@jchmura-sc jchmura-sc

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@jchmura-sc @mkolodner-sc @kmontemayor2-sc