Enable reuse of dummy wgrad tensor - #1651

Merged

ksivaman merged 5 commits into

NVIDIA:mainfrom

vasunvidia:dummy_wgrads

Apr 8, 2025

Commits on Apr 7, 2025

Use dummy wgrads for lower memory consumption
ksivaman
committed
2b071b8
Bug fix to avoid sharing gradients.

vasunvidia
authored and
ksivaman
committed
c014f89
Disable automatic use of batch_p2p_comm for CP2

vasunvidia
authored and
ksivaman
committed
eb8433d
Change weight to origin_weight for LN_LINEAR

vasunvidia
authored and
ksivaman
committed
77b0955
[pre-commit.ci] auto fixes from pre-commit.com hooks

pre-commit-ci[bot]
authored and
ksivaman
committed
e5cacd7