Skip to content

Fix Llama-3_3-Nemotron-Super-49B-v1 FP8 accuracy threshold configs - #4961

Merged
syuoni merged 3 commits into
NVIDIA:mainfrom
moraxu:fix-nemotron-super-acc-test-fp8
Jun 12, 2025
Merged

Fix Llama-3_3-Nemotron-Super-49B-v1 FP8 accuracy threshold configs#4961
syuoni merged 3 commits into
NVIDIA:mainfrom
moraxu:fix-nemotron-super-acc-test-fp8

Conversation

@moraxu

@moraxu moraxu commented Jun 5, 2025

Copy link
Copy Markdown
Collaborator

Description

Fix Llama-3_3-Nemotron-Super-49B-v1 FP8 accuracy threshold configs

Test Coverage

GitHub Bot Help

/bot [-h] ['run', 'kill', 'skip', 'reuse-pipeline'] ...

Provide a user friendly way for developers to interact with a Jenkins server.

Run /bot [-h|--help] to print this help message.

See details below for each supported subcommand.

Details

run [--disable-fail-fast --skip-test --stage-list "A10-1, xxx" --gpu-type "A30, H100_PCIe" --add-multi-gpu-test --only-multi-gpu-test --disable-multi-gpu-test --post-merge --extra-stage "H100_PCIe-[Post-Merge]-1, xxx"]

Launch build/test pipelines. All previously running jobs will be killed.

--disable-fail-fast (OPTIONAL) : Disable fail fast on build/tests/infra failures.

--skip-test (OPTIONAL) : Skip all test stages, but still run build stages, package stages and sanity check stages. Note: Does NOT update GitHub check status.

--stage-list "A10-1, xxx" (OPTIONAL) : Only run the specified test stages. Examples: "A10-1, xxx". Note: Does NOT update GitHub check status.

--gpu-type "A30, H100_PCIe" (OPTIONAL) : Only run the test stages on the specified GPU types. Examples: "A30, H100_PCIe". Note: Does NOT update GitHub check status.

--only-multi-gpu-test (OPTIONAL) : Only run the multi-GPU tests. Note: Does NOT update GitHub check status.

--disable-multi-gpu-test (OPTIONAL) : Disable the multi-GPU tests. Note: Does NOT update GitHub check status.

--add-multi-gpu-test (OPTIONAL) : Force run the multi-GPU tests. Will also run L0 pre-merge pipeline.

--post-merge (OPTIONAL) : Run the L0 post-merge pipeline instead of the ordinary L0 pre-merge pipeline.

--extra-stage "H100_PCIe-[Post-Merge]-1, xxx" (OPTIONAL) : Run the ordinary L0 pre-merge pipeline and specified test stages. Examples: --extra-stage "H100_PCIe-[Post-Merge]-1, xxx".

kill

kill

Kill all running builds associated with pull request.

skip

skip --comment COMMENT

Skip testing for latest commit on pull request. --comment "Reason for skipping build/test" is required. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

reuse-pipeline

reuse-pipeline

Reuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

@moraxu
moraxu requested review from a team as code owners June 5, 2025 17:14
@moraxu
moraxu requested review from mikeiovine and yizhang-nv June 5, 2025 17:14
@moraxu
moraxu changed the base branch from main to release/0.20 June 5, 2025 17:14
@moraxu
moraxu requested a review from a team as a code owner June 5, 2025 17:14
@moraxu

moraxu commented Jun 5, 2025

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #7784 [ run ] triggered by Bot

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #7784 [ run ] completed with state SUCCESS
/LLM/release-0.20/L0_MergeRequest_PR pipeline #181 completed with status: 'SUCCESS'

Comment thread tests/integration/defs/accuracy/references/gpqa_diamond.yaml Outdated
@moraxu
moraxu force-pushed the fix-nemotron-super-acc-test-fp8 branch from 49607b6 to 6b101b8 Compare June 6, 2025 05:39
@moraxu

moraxu commented Jun 6, 2025

Copy link
Copy Markdown
Collaborator Author

/bot reuse-pipeline

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #7839 [ reuse-pipeline ] triggered by Bot

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #7839 [ reuse-pipeline ] completed with state SUCCESS
Reusing PR_Github #7784 for commit 6b101b8

@moraxu
moraxu force-pushed the fix-nemotron-super-acc-test-fp8 branch from 6b101b8 to c36c202 Compare June 9, 2025 09:08
@moraxu

moraxu commented Jun 9, 2025

Copy link
Copy Markdown
Collaborator Author

/bot reuse-pipeline

@litaotju
litaotju enabled auto-merge (squash) June 9, 2025 09:10
@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #8115 [ reuse-pipeline ] triggered by Bot

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #8115 [ reuse-pipeline ] completed with state SUCCESS
Reusing PR_Github #7784 for commit c36c202

auto-merge was automatically disabled June 9, 2025 16:29

Head branch was pushed to by a user without write access

@moraxu

moraxu commented Jun 9, 2025

Copy link
Copy Markdown
Collaborator Author

/bot reuse-pipeline

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #8145 [ reuse-pipeline ] triggered by Bot

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #8145 [ reuse-pipeline ] completed with state SUCCESS
Reusing PR_Github #7784 for commit a47ca56

@moraxu
moraxu force-pushed the fix-nemotron-super-acc-test-fp8 branch from a47ca56 to 0420970 Compare June 9, 2025 17:47
@moraxu
moraxu changed the base branch from release/0.20 to main June 9, 2025 17:47
@moraxu

moraxu commented Jun 9, 2025

Copy link
Copy Markdown
Collaborator Author

/bot reuse-pipeline

moraxu added 3 commits June 9, 2025 17:51
Signed-off-by: moraxu <mguzek@nvidia.com>
Signed-off-by: moraxu <mguzek@nvidia.com>
Signed-off-by: moraxu <mguzek@nvidia.com>
@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #8151 [ reuse-pipeline ] triggered by Bot

@moraxu
moraxu force-pushed the fix-nemotron-super-acc-test-fp8 branch from 0420970 to 2cee7d0 Compare June 9, 2025 17:54
@moraxu

moraxu commented Jun 9, 2025

Copy link
Copy Markdown
Collaborator Author

/bot reuse-pipeline

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #8151 [ reuse-pipeline ] completed with state SUCCESS
Reusing PR_Github #7784 for commit 0420970

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #8152 [ reuse-pipeline ] triggered by Bot

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #8152 [ reuse-pipeline ] completed with state SUCCESS
Reusing PR_Github #7784 for commit 2cee7d0

@juney-nvidia
juney-nvidia requested review from syuoni and tijyojwad and removed request for mikeiovine and yizhang-nv June 12, 2025 06:18

@syuoni syuoni left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@syuoni
syuoni merged commit 0daa709 into NVIDIA:main Jun 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants