Skip to content

[None][test] Add DeepSeek V4 CI coverage#13988

Merged
lfr-0531 merged 6 commits into
NVIDIA:feat/deepseek_v4from
lfr-0531:user/fanrongl/deepseek-v4-ci-fixes
May 12, 2026
Merged

[None][test] Add DeepSeek V4 CI coverage#13988
lfr-0531 merged 6 commits into
NVIDIA:feat/deepseek_v4from
lfr-0531:user/fanrongl/deepseek-v4-ci-fixes

Conversation

@lfr-0531

Copy link
Copy Markdown
Collaborator

Description

This PR updates DeepSeek V4 CI coverage and L0 multi-GPU triggering behavior:

  • Adds GSM8K smoke evaluation to TestDeepSeekV4Flash.test_auto_dtype and TestDeepSeekV4FlashBase.test_auto_dtype, alongside the existing MMLU smoke checks.
  • Disables pytest-threadleak for the sparse MLA forward test module because DSACacheManager creates background ThreadPoolExecutor threads, matching the existing DSA short-sequence test behavior.
  • Makes L0 merge-request CI default to running multi-GPU tests without changed-file filtering, while preserving the explicit DISABLE_MULTI_GPU_TEST override.

Test Coverage

  • pre-commit run --files jenkins/L0_MergeRequest.groovy tests/integration/defs/accuracy/test_llm_api_pytorch.py tests/unittest/_torch/attention/sparse/test_sparse_mla_forward.py
  • .venv-3.12/bin/python -m py_compile tests/integration/defs/accuracy/test_llm_api_pytorch.py tests/unittest/_torch/attention/sparse/test_sparse_mla_forward.py
  • Built and installed the PR wheel locally before the final test/CI edits. GPU pytest was not run after the user asked to avoid using shared GPUs.

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

  • Test cases are provided for new code paths (see test instructions)

  • Any new dependencies have been scanned for license and vulnerabilities

  • CODEOWNERS updated if ownership changes

  • Documentation updated as needed

  • Update tava architecture diagram if there is a significant design change in PR.

  • The reviewers assigned automatically/manually are appropriate for the PR.

  • Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Tracin and others added 4 commits May 8, 2026 20:06
Signed-off-by: Tracin <10434017+Tracin@users.noreply.github.com>
Signed-off-by: Tracin <10434017+Tracin@users.noreply.github.com>
Signed-off-by: Tracin <10434017+Tracin@users.noreply.github.com>
Add GSM8K smoke coverage to DeepSeek V4 Flash auto-dtype tests, keep sparse MLA DSA tests from failing on known background executor threads, and make L0 default to running multi-GPU tests.

Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
@lfr-0531
lfr-0531 requested a review from a team May 11, 2026 09:10
@lfr-0531
lfr-0531 requested review from a team as code owners May 11, 2026 09:10
@lfr-0531
lfr-0531 requested review from dpitman-nvda and mlefeb01 and removed request for a team May 11, 2026 09:10
@lfr-0531

Copy link
Copy Markdown
Collaborator Author

/bot run --add-multi-gpu-test --disable-fail-fast

@lfr-0531
lfr-0531 requested review from EmmaQiaoCh, Shixiaowei02, Tracin and heyuhhh and removed request for a team, dpitman-nvda and mlefeb01 May 11, 2026 09:15
@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #47715 [ run ] triggered by Bot. Commit: 5a0bcac Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #47715 [ run ] completed with state SUCCESS. Commit: 5a0bcac
/LLM/main/L0_MergeRequest_PR pipeline #37614 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

@lfr-0531

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #47743 [ run ] triggered by Bot. Commit: ea40cf4 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #47743 [ run ] completed with state SUCCESS. Commit: ea40cf4
/LLM/main/L0_MergeRequest_PR pipeline #37638 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

@lfr-0531

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #47814 [ run ] triggered by Bot. Commit: ea40cf4 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #47814 [ run ] completed with state SUCCESS. Commit: ea40cf4
/LLM/main/L0_MergeRequest_PR pipeline #37706 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

@lfr-0531

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #47927 [ run ] triggered by Bot. Commit: 9dcacb0 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #47927 [ run ] completed with state SUCCESS. Commit: 9dcacb0
/LLM/main/L0_MergeRequest_PR pipeline #37774 completed with status: 'SUCCESS'

CI Report

Link to invocation

@lfr-0531
lfr-0531 merged commit d6141d6 into NVIDIA:feat/deepseek_v4 May 12, 2026
5 checks passed
lfr-0531 added a commit that referenced this pull request May 14, 2026
Signed-off-by: Tracin <10434017+Tracin@users.noreply.github.com>
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
Co-authored-by: Tracin <10434017+Tracin@users.noreply.github.com>
Co-authored-by: Fanrong Li <lfr-0531@users.noreply.github.com>
lfr-0531 added a commit to lfr-0531/TensorRT-LLM that referenced this pull request May 29, 2026
Signed-off-by: Tracin <10434017+Tracin@users.noreply.github.com>
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
Co-authored-by: Tracin <10434017+Tracin@users.noreply.github.com>
Co-authored-by: Fanrong Li <lfr-0531@users.noreply.github.com>
(cherry picked from commit 8eccf23)
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
lfr-0531 added a commit to lfr-0531/TensorRT-LLM that referenced this pull request Jun 1, 2026
Signed-off-by: Tracin <10434017+Tracin@users.noreply.github.com>
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
Co-authored-by: Tracin <10434017+Tracin@users.noreply.github.com>
Co-authored-by: Fanrong Li <lfr-0531@users.noreply.github.com>
(cherry picked from commit 8eccf23)
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
lfr-0531 added a commit to lfr-0531/TensorRT-LLM that referenced this pull request Jun 3, 2026
Signed-off-by: Tracin <10434017+Tracin@users.noreply.github.com>
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
Co-authored-by: Tracin <10434017+Tracin@users.noreply.github.com>
Co-authored-by: Fanrong Li <lfr-0531@users.noreply.github.com>
(cherry picked from commit 8eccf23)
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
lfr-0531 added a commit to lfr-0531/TensorRT-LLM that referenced this pull request Jun 7, 2026
Signed-off-by: Tracin <10434017+Tracin@users.noreply.github.com>
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
Co-authored-by: Tracin <10434017+Tracin@users.noreply.github.com>
Co-authored-by: Fanrong Li <lfr-0531@users.noreply.github.com>
(cherry picked from commit 8eccf23)
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
lfr-0531 added a commit to lfr-0531/TensorRT-LLM that referenced this pull request Jun 10, 2026
Signed-off-by: Tracin <10434017+Tracin@users.noreply.github.com>
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
Co-authored-by: Tracin <10434017+Tracin@users.noreply.github.com>
Co-authored-by: Fanrong Li <lfr-0531@users.noreply.github.com>
lfr-0531 added a commit to lfr-0531/TensorRT-LLM that referenced this pull request Jun 11, 2026
Signed-off-by: Tracin <10434017+Tracin@users.noreply.github.com>
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
Co-authored-by: Tracin <10434017+Tracin@users.noreply.github.com>
Co-authored-by: Fanrong Li <lfr-0531@users.noreply.github.com>
lfr-0531 added a commit to lfr-0531/TensorRT-LLM that referenced this pull request Jun 12, 2026
Signed-off-by: Tracin <10434017+Tracin@users.noreply.github.com>
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
Co-authored-by: Tracin <10434017+Tracin@users.noreply.github.com>
Co-authored-by: Fanrong Li <lfr-0531@users.noreply.github.com>
lfr-0531 added a commit to lfr-0531/TensorRT-LLM that referenced this pull request Jun 12, 2026
Signed-off-by: Tracin <10434017+Tracin@users.noreply.github.com>
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
Co-authored-by: Tracin <10434017+Tracin@users.noreply.github.com>
Co-authored-by: Fanrong Li <lfr-0531@users.noreply.github.com>
(cherry picked from commit 8eccf23)
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
lfr-0531 added a commit to lfr-0531/TensorRT-LLM that referenced this pull request Jun 13, 2026
Signed-off-by: Tracin <10434017+Tracin@users.noreply.github.com>
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
Co-authored-by: Tracin <10434017+Tracin@users.noreply.github.com>
Co-authored-by: Fanrong Li <lfr-0531@users.noreply.github.com>
(cherry picked from commit 8eccf23)
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
lfr-0531 added a commit to lfr-0531/TensorRT-LLM that referenced this pull request Jun 16, 2026
Signed-off-by: Tracin <10434017+Tracin@users.noreply.github.com>
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
Co-authored-by: Tracin <10434017+Tracin@users.noreply.github.com>
Co-authored-by: Fanrong Li <lfr-0531@users.noreply.github.com>
(cherry picked from commit 8eccf23)
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
lfr-0531 added a commit to lfr-0531/TensorRT-LLM that referenced this pull request Jun 17, 2026
Signed-off-by: Tracin <10434017+Tracin@users.noreply.github.com>
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
Co-authored-by: Tracin <10434017+Tracin@users.noreply.github.com>
Co-authored-by: Fanrong Li <lfr-0531@users.noreply.github.com>
(cherry picked from commit 8eccf23)
Signed-off-by: Fanrong Li <lfr-0531@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants