Skip to content

From NVIDIA Megatron-LM for visibility #18

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
RaymondLi0 wants to merge 6,122 commits into
base: multi-query-attention
Choose a base branch
from
Open

From NVIDIA Megatron-LM for visibility #18

RaymondLi0 wants to merge 6,122 commits into from

Conversation

@RaymondLi0
Copy link
Collaborator

@RaymondLi0 RaymondLi0 commented Jan 24, 2023

No description provided.

@RaymondLi0 RaymondLi0 changed the base branch from multi-query-attention to before-merge June 20, 2023 20:12
@RaymondLi0 RaymondLi0 changed the base branch from before-merge to multi-query-attention June 20, 2023 20:12
maanug-nv and others added 28 commits December 18, 2025 22:02
@maanug-nv
Add missing imports in no-triton fallback (#2711)
a6f822c 
Signed-off-by: Maanu Grover <maanug@nvidia.com>
@lmcafee-nvidia
Fixes for #2450. (#2714)
1503c33
@tdene
Add RL parameter to set parallel generation tasks (#2712)
1a2257b
@wdykas
Refit prep 3 (#2708)
30694e0 
Co-authored-by: William Dykas <wdykas@cw-pdx-cs-001-vscode-02.cm.cluster>
Co-authored-by: root <root@gpu-h100-0371.cm.cluster>
Co-authored-by: root <root@gpu-h100-0159.cm.cluster>
@ko3n1g
chore: Add cudagraph codeowners (#2720)
000c2e2
@Phlip79
[docs] Add developer section to docs (#2717)
703bc36
@tdene
Fix UVM argument for RL (#2722)
7f471d7
@Phlip79
[dcos] Update docs title to Megatron Core (#2729)
ddf691d
@HaochenYuan @Phlip79
remove fp16 assert in moe_grouped_gemm & EP (#2495)
4193f3a 
Co-authored-by: Philip Petrakian <pgpetrak@gmail.com>
@jenchen13 @jaredcasper
Improve ModelOpt paths & add more Nemotron/hybrid model support (#2131)
a057662 
Signed-off-by: Jennifer Chen <jennifchen@nvidia.com>
Co-authored-by: Jared Casper <155158+jaredcasper@users.noreply.github.com>
@asolergi-nv @deepakn94
Add options to improve data loader initialization time, especially at…
cfd980b 
… scale (#2445)

Signed-off-by: Antoni-Joan Solergibert <asolergibert@nvidia.com>
Co-authored-by: Deepak Narayanan <2724038+deepakn94@users.noreply.github.com>
@github-actions
Update copy-pr-bot.yaml [skip ci]
1c67e7e
@ko3n1g
ci: Fix copy-pr-bot update (#2736)
8ea3b8d 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@Phlip79
Add oncall to all new PRs (#2734)
5b1ef07
@tomlifu
Hsdp register submesh fix lifuz mirror (#2467)
cc1b0b5 
Signed-off-by: Lifu Zhang <lifuz@login-lyris01.lyris.clusters.nvidia.com>
Co-authored-by: Lifu Zhang <lifuz@login-lyris01.lyris.clusters.nvidia.com>
@shanmugamr1992
Adding stop word support (#2685)
1febe9f 
Co-authored-by: Shanmugam Ramasamy <shanmugamr@cw-dfw-cs-001-login-01.cm.cluster>
@Phlip79
Fix oncall assign (#2737)
a477766
@santhnm2 @kanz-nv
Add support for non-decode CUDA graphs for Mamba models (#2474)
f5d4c3a 
Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>
Co-authored-by: Kan Zhu <kanz@nvidia.com>
@mathemakitten
Update sequence packing case when dummy PackedSeqParams are used (#2743)
0a77122
@youngeunkwon0405
feat: manual registration mode for nccl-ub option when using megatron…
876a046 
…-fsdp (#2661)

Signed-off-by: Youngeun Kwon <youngeunk@nvidia.com>
@github-actions
chore: rotate oncall schedule
ede9ae4
@Phlip79
Update oncall for next few weeks (#2748)
3cf7a63
@nschank
Prep work for migrating to types from ModuleSpec (#2668)
dd7c9f4
@buptzyb
feat(MoE): Refactor cuda_graph_scope (#1920)
2b343d7 
Signed-off-by: Robin Zhang <robinz@nvidia.com>
@github-actions
Update copy-pr-bot.yaml [skip ci]
a2d7d67
@tdene
Fix merge conflict in #1920 (#2781)
11c9680
@chtruong814
ci: Allow disabling external contributors (#2784)
40d590d 
Signed-off-by: Charlie Truong <chtruong@nvidia.com>
@github-actions
chore: rotate oncall schedule
6977db9
ko3n1g and others added 30 commits January 27, 2026 02:12
@ko3n1g
Reapply "Support multimodule communication (#2031)" (#3068)
35e85a6 
This reverts commit 0972f02.
@litianjian @ISEEKYAN
@Phlip79 @sidsingh-nvidia
Add router replay for MoE models (#2101)
db6b895 
Co-authored-by: litianjian <litianjian@bytedance.com>
Co-authored-by: Yan Bai <baiyan1996@icloud.com>
Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>
Co-authored-by: Siddharth Singh <136645615+sidsingh-nvidia@users.noreply.github.com>
@ko3n1g
ci: Disable gpt_dynamic_inference_tp1_pp1_dp8_583m_throughputtest_zmq (…
7031953 
…#3099)

Signed-off-by: oliver könig <okoenig@nvidia.com>
@ko3n1g
ci: Repeat func tests, save logs of unit tests and lessen debug output (
dea21a0 
#3089)

Signed-off-by: oliver könig <okoenig@nvidia.com>
@ko3n1g
ci: Update improvement of step-time (#3104)
dd83fc6 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@ko3n1g
ci: Add GPU health checks (#3100)
2bdf7e1 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@jon-barker
Harden GRPO functional tests (#3065)
d68721b 
Co-authored-by: Jon Barker <jbarker@oci-hsg-cs-001-vscode-01.cm.cluster>
@mathemakitten
Inference functional tests: Write outputs to INFERENCE_OUTPUT_PATH in…
4015ff1 
…stead of TENSORBOARD_PATH (#3061)
@Victarry
Update moe readme. (#2830)
0888a06
@ko3n1g
build: Bump to TE2.12 (#3086)
2b02a28 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@deepakn94
Logging cleanup (only log on rank 0 if possible) (#3036)
6cf285b 
Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>
@Phlip79
Move all bert and t5 tests to nightly (#3106)
65217aa
@Phlip79
Create greptile.json (#3087)
4fb549f
@github-actions
Update copy-pr-bot.yaml [skip ci]
6273d74
@kunlunl
Fix bug of reuse_grad_buf_for_mxfp8_param_ag (#2802)
33224cc 
Signed-off-by: kunlunl <kunlunl@nvidia.com>
@parthmannan
Fix for Hybrid CP (#3091)
fb6a592
@jon-barker
Fix GRPO re-fit functional test (#3113)
f6c8a61
@megnvidia @Phlip79
Minimize README contents (#3020)
991138e 
Signed-off-by: meg miranda <mmiranda@nvidia.com>
Co-authored-by: Philip Petrakian <pgpetrak@gmail.com>
@shjwudp
Add end-to-end tests for M-FSDP and ND-Parallel (#3031)
964c902
@faradawn @maanug-nv
Fix Multimodal Dockerfile (#3006)
38cd9fc 
Co-authored-by: Maanu Grover <109391026+maanug-nv@users.noreply.github.com>
@shjwudp @cspades
[M-FSDP] Fix double buffering not working with activation recompute (#…
b6b49e7 
…2689)

Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>
@maanug-nv
[training migration] Add CheckpointConfig dataclass (#2431)
d4f9347 
Signed-off-by: Maanu Grover <maanug@nvidia.com>
@github-actions
chore: rotate oncall schedule
d5cac80
@maanug-nv
[training migration] Add StragglerDetectionConfig dataclass (#2435)
008926a 
Signed-off-by: Maanu Grover <maanug@nvidia.com>
@tdene
Standardize RL unit tests (#3088)
1453f94
@HaochenYuan @Phlip79 @ko3n1g
Fix for PR-2142 (#3096)
71c49b5 
Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>
Co-authored-by: oliver könig <okoenig@nvidia.com>
@Autumn1998
Use the latest hybrid-ep (#3093)
93ddc24
@dimapihtar
remove retro (#3001)
fc6969f 
Signed-off-by: dimapihtar <dpihtar@gmail.com>
@ko3n1g
ci: Mark test_compatible_with_nd_parallel as flaky (#3122)
c22615e 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@ko3n1g
build: Use merge-commit-sha for container (#3123)
a883e96 
Signed-off-by: oliver könig <okoenig@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.