[None][test] Waive 2 failed cases for main in QA CI - #14755
[None][test] Waive 2 failed cases for main in QA CI#14755tensorrt-cicd wants to merge 3 commits into
Conversation
Bug(s): 6059036, 6189290 Requested by: qa@nvidia.com Jenkins build: https://prod.blsm.nvidia.com/swqa-tensorrt-qa-test/job/LLM_FUNCTION_TEST/2201/ Signed-off-by: tensorrt-cicd <90828364+tensorrt-cicd@users.noreply.github.com>
Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
|
/bot run --stage-list "" |
📝 WalkthroughWalkthroughThis PR adds a single test waiver entry to the integration test waiver list, skipping the ChangesTest Waiver Entry
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~2 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/integration/test_lists/waives.txt`:
- Line 13: The waiver target "triton_paged-False-1" for
accuracy/test_llm_api_autodeploy.py::TestLlama3_1_8B::test_auto_dtype is
invalid; update the waiver entry in the waives list to match an active
parametrized case or change the test parametrization so the key matches. Replace
the current token in tests/integration/test_lists/waives.txt with one of the
active IDs reported by CI for TestLlama3_1_8B::test_auto_dtype, or adjust the
parametrization in accuracy/test_llm_api_autodeploy.py so that the generated
case IDs include "triton_paged-False-1".
🪄 Autofix (Beta)
❌ Autofix failed (check again to retry)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 735ab283-15e1-413e-9eb2-98f2470c4a5b
📒 Files selected for processing (1)
tests/integration/test_lists/waives.txt
| accuracy/test_llm_api.py::TestMistralNemo12B::test_fp8 SKIP (https://nvbugs/5413197) | ||
| accuracy/test_llm_api_autodeploy.py::TestGemma4MoE::test_bf16 SKIP (https://nvbugs/6158397) | ||
| accuracy/test_llm_api_autodeploy.py::TestGemmaE2B::test_gemma4_e2b_it SKIP (https://nvbugs/6194934) | ||
| accuracy/test_llm_api_autodeploy.py::TestLlama3_1_8B::test_auto_dtype[triton_paged-False-1] SKIP (https://nvbugs/6189290) |
There was a problem hiding this comment.
Fix invalid waiver target; it does not map to any active parametrized test.
validate-test-lists is failing because triton_paged-False-1 is not a valid active ID for accuracy/test_llm_api_autodeploy.py::TestLlama3_1_8B::test_auto_dtype (and the waiver is reported as not in active test lists). This waiver entry currently cannot be applied and blocks merge.
QA coverage assessment: insufficient for this waiver change until the waiver points to an active case.
Actionable follow-up:
- Update
tests/integration/test_lists/waives.txtLine 13 to one of the active IDs reported by CI. - Reconcile the parametrization source in
tests/integration/accuracy/test_llm_api_autodeploy.pyso waiver/test-list keys match exactly.
🧰 Tools
🪛 GitHub Actions: Release Checks / 0_Pre-commit Check.txt
[error] 13-13: pre-commit hook 'validate-test-lists' failed. INVALID PARAMETRIZE ID: accuracy/test_llm_api_autodeploy.py::TestLlama3_1_8B::test_auto_dtype. ID 'triton_paged-False-1' not in valid IDs: ['flashinfer-False-1','flashinfer-False-2','flashinfer-False-4','flashinfer-True-1','flashinfer-True-2','flashinfer-True-4','torch-False-1','torch-False-2'].
[error] 13-13: pre-commit hook 'validate-test-lists' failed. WAIVE NOT IN ACTIVE TEST LISTS: accuracy/test_llm_api_autodeploy.py::TestLlama3_1_8B::test_auto_dtype[triton_paged-False-1].
🪛 GitHub Actions: Release Checks / Pre-commit Check
[error] 13-13: pre-commit hook 'validate-test-lists' failed (exit code 1): INVALID PARAMETRIZE ID: accuracy/test_llm_api_autodeploy.py::TestLlama3_1_8B::test_auto_dtype. ID 'triton_paged-False-1' not in valid IDs: ['flashinfer-False-1','flashinfer-False-2','flashinfer-False-4','flashinfer-True-1','flashinfer-True-2','flashinfer-True-4','torch-False-1','torch-False-2']
[error] 13-13: pre-commit hook 'validate-test-lists' failed (exit code 1): WAIVE NOT IN ACTIVE TEST LISTS for accuracy/test_llm_api_autodeploy.py::TestLlama3_1_8B::test_auto_dtype[triton_paged-False-1]
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/integration/test_lists/waives.txt` at line 13, The waiver target
"triton_paged-False-1" for
accuracy/test_llm_api_autodeploy.py::TestLlama3_1_8B::test_auto_dtype is
invalid; update the waiver entry in the waives list to match an active
parametrized case or change the test parametrization so the key matches. Replace
the current token in tests/integration/test_lists/waives.txt with one of the
active IDs reported by CI for TestLlama3_1_8B::test_auto_dtype, or adjust the
parametrization in accuracy/test_llm_api_autodeploy.py so that the generated
case IDs include "triton_paged-False-1".
|
Note Autofix is a beta feature. Expect some limitations and changes as we gather feedback and continue to improve it. An unexpected error occurred while generating fixes: Not Found - https://docs.github.com/rest/git/refs#get-a-reference |
|
PR_Github #51481 [ run ] triggered by Bot. Commit: |
|
PR_Github #51481 [ run ] completed with state
|
2 participants
Auto-generated Waive PR
Created by: TensorRT LLM CI Report (requested by qa@nvidia.com)
Target branch:
mainJenkins build: https://prod.blsm.nvidia.com/swqa-tensorrt-qa-test/job/LLM_FUNCTION_TEST/2201/
Bug(s): 6059036, 6189290
Waive entries added
This PR was auto-generated by TensorRT LLM CI Report. Please review the waive entries before merging.
Summary by CodeRabbit
Note: This release contains internal testing infrastructure updates with no user-facing changes.