Skip to content

[None][test] Waive 6 failed cases for main in QA CI - #14787

Merged
xinhe-nv merged 15 commits into
NVIDIA:mainfrom
tensorrt-cicd:trtllm-ci-report/waive-20260530-163803
Jun 2, 2026
Merged

[None][test] Waive 6 failed cases for main in QA CI#14787
xinhe-nv merged 15 commits into
NVIDIA:mainfrom
tensorrt-cicd:trtllm-ci-report/waive-20260530-163803

Conversation

@tensorrt-cicd

@tensorrt-cicd tensorrt-cicd commented May 30, 2026

Copy link
Copy Markdown
Collaborator

Auto-generated Waive PR

Created by: TensorRT LLM CI Report (requested by qa@nvidia.com)
Target branch: main
Jenkins build: https://prod.blsm.nvidia.com/swqa-tensorrt-qa-test/job/LLM_FUNCTION_CLUSTER_TEST/1482/
Bug(s): 6185150, 6211193, 6223556, 6245098

Waive entries added

accuracy/test_llm_api_pytorch.py::TestQwen3_6_27B::test_fp8 SKIP (https://nvbugs/6185150)
disaggregated/test_disaggregated.py::test_disaggregated_overlap_gen_first[ctx_pp1-TinyLlama-1.1B-Chat-v1.0] SKIP (https://nvbugs/6223556)
test_e2e.py::test_ptp_quickstart_advanced_deepseek_multi_nodes[DeepSeek-V3] SKIP (https://nvbugs/6245098)
test_e2e.py::test_openai_disagg_multi_nodes_completion_service_discovery[etcd] SKIP (https://nvbugs/6211193)
disaggregated/test_disaggregated.py::test_disaggregated_overlap_gen_first[ctx_pp4-TinyLlama-1.1B-Chat-v1.0] SKIP (https://nvbugs/6223556)
accuracy/test_llm_api_pytorch.py::TestQwen3NextInstruct::test_bf16_4gpu[tp4ep4_cudagraph_overlap_adp_off] SKIP (https://nvbugs/6185150)

This PR was auto-generated by TensorRT LLM CI Report. Please review the waive entries before merging.

Summary by CodeRabbit

  • Tests
    • Updated test waiver configurations for internal quality assurance processes.

Note: This release contains no user-visible changes—updates are limited to internal testing infrastructure only.

Bug(s): 6185150, 6211193, 6223556, 6245098
Requested by: qa@nvidia.com
Jenkins build: https://prod.blsm.nvidia.com/swqa-tensorrt-qa-test/job/LLM_FUNCTION_CLUSTER_TEST/1482/

Signed-off-by: tensorrt-cicd <90828364+tensorrt-cicd@users.noreply.github.com>
Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
@xinhe-nv
xinhe-nv marked this pull request as ready for review June 1, 2026 06:55
@xinhe-nv
xinhe-nv enabled auto-merge (squash) June 1, 2026 06:55
@xinhe-nv

xinhe-nv commented Jun 1, 2026

Copy link
Copy Markdown
Collaborator

/bot run --skip-test

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #51327 [ run ] triggered by Bot. Commit: b8f1953 Link to invocation

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/integration/test_lists/waives.txt`:
- Line 118: Add a waiver entry for the DeepSeek-V3 failure in
tests/integration/test_lists/waives.txt for the test named
test_e2e.py::test_ptp_quickstart_advanced_deepseek_multi_nodes[DeepSeek-V3] that
references https://nvbugs/6245098 (or alternatively update the PR summary to
remove the claim that 6245098 is waived); ensure the new line follows the
existing waive list format and includes the exact test identifier and the nvbugs
URL so the waiver is recognized.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: db2d8b90-3a6b-467b-abc2-8a17a079d954

📥 Commits

Reviewing files that changed from the base of the PR and between 35cce74 and b8f1953.

📒 Files selected for processing (1)
  • tests/integration/test_lists/waives.txt

Comment thread tests/integration/test_lists/waives.txt Outdated
@coderabbitai

coderabbitai Bot commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

This PR adds 5 new SKIP waiver entries to the integration test skip list. The entries cover failing test cases: Qwen3NextInstruct BF16 with tensor parallelism and cuGraphs, Qwen3 FP8, two TinyLlama disaggregated overlap generation variants, and OpenAI disaggregated multi-node completion service discovery with etcd.

Changes

Test Skip Waivers

Layer / File(s) Summary
Integration test SKIP waivers
tests/integration/test_lists/waives.txt
Five SKIP entries added for failing tests: Qwen3NextInstruct bf16 4-gpu with tp4ep4_cudagraph_overlap_adp_off, Qwen3_6_27B FP8, TinyLlama disaggregated overlap generation with ctx_pp1 and ctx_pp4, and OpenAI disaggregated multi-node completion service discovery with etcd backend.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Possibly related PRs

Suggested reviewers

  • crazydemo
  • jieli-matrix
  • StanleySun639
  • LarryXFly
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title follows the required format [None][test] and clearly indicates the PR purpose is waiving 6 failed test cases for QA CI, matching the changeset content.
Description check ✅ Passed The PR description is auto-generated and includes the Jenkins build link, bug references, and specific waive entries with links. However, it lacks the recommended sections from the template (Description, Test Coverage, PR Checklist) which are typically expected for manual PRs.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
tests/integration/test_lists/waives.txt (1)

118-118: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

PR summary claims 6 waivers but only 5 entries are present

The PR summary states that 6 test cases are being waived, but this file contains only 5 new SKIP entries. The missing case test_e2e.py::test_ptp_quickstart_advanced_deepseek_multi_nodes[DeepSeek-V3] (https://nvbugs/6245098) referenced in the summary is not present in the waives.txt file. Either add the missing waiver entry or update the PR summary to accurately reflect 5 waivers.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/integration/test_lists/waives.txt` at line 118, PR summary claims 6
waived tests but only 5 SKIP entries exist in the waives list; either add the
missing waiver for the
test_e2e.py::test_ptp_quickstart_advanced_deepseek_multi_nodes[DeepSeek-V3]
(link nvbugs/6245098) to the waives file or change the PR summary to state 5
waivers; update the waives file by adding the exact SKIP line for
test_e2e.py::test_ptp_quickstart_advanced_deepseek_multi_nodes[DeepSeek-V3]
referencing the bug URL, or if that waiver is not needed remove the extra count
in the PR summary so it matches the current entries.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Duplicate comments:
In `@tests/integration/test_lists/waives.txt`:
- Line 118: PR summary claims 6 waived tests but only 5 SKIP entries exist in
the waives list; either add the missing waiver for the
test_e2e.py::test_ptp_quickstart_advanced_deepseek_multi_nodes[DeepSeek-V3]
(link nvbugs/6245098) to the waives file or change the PR summary to state 5
waivers; update the waives file by adding the exact SKIP line for
test_e2e.py::test_ptp_quickstart_advanced_deepseek_multi_nodes[DeepSeek-V3]
referencing the bug URL, or if that waiver is not needed remove the extra count
in the PR summary so it matches the current entries.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a4ed7824-1b18-4ef1-876b-a1b7813d8c65

📥 Commits

Reviewing files that changed from the base of the PR and between 35cce74 and b8f1953.

📒 Files selected for processing (1)
  • tests/integration/test_lists/waives.txt

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #51327 [ run ] completed with state SUCCESS. Commit: b8f1953
/LLM/main/L0_MergeRequest_PR pipeline #40740 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

/bot run --skip-test

@xinhe-nv

xinhe-nv commented Jun 2, 2026

Copy link
Copy Markdown
Collaborator

/bot run --skip-test

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #51465 [ run ] triggered by Bot. Commit: 2a6b9d5 Link to invocation

xinhe-nv added 3 commits June 2, 2026 09:51
Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #51465 [ run ] completed with state SUCCESS. Commit: 2a6b9d5
/LLM/main/L0_MergeRequest_PR pipeline #40871 (Partly Tested) completed with status: 'SUCCESS'
Pipeline passed with automatic retried tests. Check the rerun report for details.

CI Report

Link to invocation

@xinhe-nv

xinhe-nv commented Jun 2, 2026

Copy link
Copy Markdown
Collaborator

/bot reuse-pipeline

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #51510 [ reuse-pipeline ] triggered by Bot. Commit: 6e49f96 Link to invocation

@xinhe-nv

xinhe-nv commented Jun 2, 2026

Copy link
Copy Markdown
Collaborator

/bot reuse-pipeline

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #51515 [ reuse-pipeline ] triggered by Bot. Commit: 6dffdeb Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #51510 [ reuse-pipeline ] completed with state ABORTED. Commit: 6e49f96
Can't reuse PR_Github #51465 (Partly Tested) with status: SUCCESS

Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #51515 [ reuse-pipeline ] completed with state SUCCESS. Commit: 6dffdeb
Reusing PR_Github #51465 (Partly Tested) for commit 6dffdeb

Link to invocation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants