Skip to content

[None][infra] Waive 3 failed cases for main in post-merge - #13789

Merged
jieli-matrix merged 2 commits into
NVIDIA:mainfrom
xinhe-nv:trtllm-ci-report/waive-20260506-015152
May 6, 2026
Merged

[None][infra] Waive 3 failed cases for main in post-merge#13789
jieli-matrix merged 2 commits into
NVIDIA:mainfrom
xinhe-nv:trtllm-ci-report/waive-20260506-015152

Conversation

@xinhe-nv

@xinhe-nv xinhe-nv commented May 6, 2026

Copy link
Copy Markdown
Collaborator

Auto-generated Waive PR

Created by: TensorRT LLM CI Report (requested by qa@nvidia.com)
Target branch: main
Bug(s): 6141803, 6141806, 6143787

Waive entries added

disaggregated/test_disaggregated.py::test_disaggregated_overlap_gen_first[ctx_pp1-TinyLlama-1.1B-Chat-v1.0] SKIP (https://nvbugs/6141806)
accuracy/test_llm_api_pytorch_multimodal.py::TestQwen3VL::test_auto_dtype[forced_chunked_prefill] SKIP (https://nvbugs/6143787)
accuracy/test_llm_api_pytorch.py::TestQwen3_5_4B::test_bf16 SKIP (https://nvbugs/6141803)

This PR was auto-generated by TensorRT LLM CI Report. Please review the waive entries before merging.

Summary by CodeRabbit

  • Chores
    • Updated test infrastructure with test waivers for known issues.

@jieli-matrix
jieli-matrix enabled auto-merge (squash) May 6, 2026 01:52
@jieli-matrix jieli-matrix assigned jieli-matrix and unassigned xinhe-nv May 6, 2026
@coderabbitai

coderabbitai Bot commented May 6, 2026

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

Three test cases were added to the waiver list to skip due to known issues tracked in bug reports: a disaggregated TinyLlama overlap test, a Qwen3 VL multimodal auto-dtype test, and a Qwen3.5 4B bf16 accuracy test.

Changes

Test Waivers

Layer / File(s) Summary
Skip Entries
tests/integration/test_lists/waives.txt
Three SKIP waiver entries added for failing tests with bug references: disaggregated/test_disaggregated.py overlap test (nvbugs/6141806), multimodal Qwen3 VL auto_dtype test (nvbugs/6143787), and Qwen3.5 4B bf16 test (nvbugs/6141803).

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related PRs

  • NVIDIA/TensorRT-LLM#13335: Adds test waivers to the same waives.txt file for failing accuracy tests including a Qwen3.x bf16 test.
  • NVIDIA/TensorRT-LLM#13203: Adds SKIP waivers to waives.txt with overlapping disaggregated TinyLlama-1.1B-Chat-v1.0 test entries.

Suggested reviewers

  • EmmaQiaoCh
  • jieli-matrix
  • StanleySun639
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Description check ❓ Inconclusive The description is auto-generated and includes the referenced bugs, specific waive entries added, and creation details, but it omits required sections like explicit Description, Test Coverage, and PR Checklist. Consider adding explicit Description and Test Coverage sections, and ensure the PR Checklist is completed before merging, as specified in the template.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly specifies that it's adding 3 failed test case waivers for the main branch, which matches the actual changes (3 SKIP entries added).
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/integration/test_lists/waives.txt`:
- Around line 441-443: The three waiver entries appended at the end of
tests/integration/test_lists/waives.txt are out of alphabetical order and must
be sorted: move the "ctx_pp1-TinyLlama-1.1B-Chat-v1.0" entry so it appears
before the existing ctx_pp4 entry (i.e., near the earlier ctx_pp* block), and
insert the two accuracy entries
("accuracy/test_llm_api_pytorch_multimodal.py::TestQwen3VL::test_auto_dtype[forced_chunked_prefill]"
and "accuracy/test_llm_api_pytorch.py::TestQwen3_5_4B::test_bf16") into their
respective accuracy/* section in alphabetical order; easiest fix is to run the
pre-commit hook file-contents-sorter (pre-commit run file-contents-sorter
--files tests/integration/test_lists/waives.txt) and commit the sorted file.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 30ed369f-08f1-4230-9f37-2a412c831566

📥 Commits

Reviewing files that changed from the base of the PR and between a753934 and 085f101.

📒 Files selected for processing (1)
  • tests/integration/test_lists/waives.txt

Comment thread tests/integration/test_lists/waives.txt Outdated
@jieli-matrix

Copy link
Copy Markdown
Collaborator

/bot run --stage-list ""

@jieli-matrix

Copy link
Copy Markdown
Collaborator

/bot run --stage-list ""

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #46899 [ run ] triggered by Bot. Commit: f9d5541 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #46899 [ run ] completed with state SUCCESS. Commit: f9d5541
/LLM/main/L0_MergeRequest_PR pipeline #36908 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

@jieli-matrix

Copy link
Copy Markdown
Collaborator

/bot reuse-pipeline

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #46920 [ reuse-pipeline ] triggered by Bot. Commit: f9d5541 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #46920 [ reuse-pipeline ] completed with state SUCCESS. Commit: f9d5541
Reusing PR_Github #46899 (Partly Tested) for commit f9d5541

Link to invocation

xinhe-nv and others added 2 commits May 6, 2026 14:36
Bug(s): 6141803, 6141806, 6143787
Requested by: qa@nvidia.com

Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
Signed-off-by: Jie Li <lijie@nvidia.com>
@jieli-matrix
jieli-matrix force-pushed the trtllm-ci-report/waive-20260506-015152 branch from 4bde7d3 to f31f730 Compare May 6, 2026 06:37
@jieli-matrix

Copy link
Copy Markdown
Collaborator

/bot reuse-pipeline

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #46933 [ reuse-pipeline ] triggered by Bot. Commit: f31f730 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #46933 [ reuse-pipeline ] completed with state SUCCESS. Commit: f31f730
Reusing PR_Github #46899 (Partly Tested) for commit f31f730

Link to invocation

@jieli-matrix
jieli-matrix merged commit 8b4693f into NVIDIA:main May 6, 2026
6 checks passed
yufeiwu-nv pushed a commit to yufeiwu-nv/TensorRT-LLM that referenced this pull request May 19, 2026
Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
@xinhe-nv
xinhe-nv deleted the trtllm-ci-report/waive-20260506-015152 branch May 27, 2026 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants