[https://nvbugs/5974335][refactor] Update model path definitions in test_perf.py and clean up waives.txt

[yufeiwu-nv]

Removed outdated model paths and unnecessary entries from MODEL_PATH_DICT in test_perf.py. Updated waives.txt to reflect the removal of tests that are no longer applicable, improving clarity and maintainability.

Signed-off-by: yufeiwu-nv 230315618+yufeiwu-nv@users.noreply.github.com

Summary by CodeRabbit

• Chores
  • Updated performance testing configurations and test waiver management to align with current model support requirements.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR updates performance test infrastructure by removing deprecated Llama v2 model entries, pruning Phi-4 multimodal variants, and adjusting model path mappings in test_perf.py, while synchronizing integration test waivers in waives.txt to skip newly unsupported configurations and enable tests for updated model targets.

Changes

Perf Test Configuration

Layer / File(s)	Summary
Model path dictionary updates tests/integration/defs/perf/test_perf.py	MODEL_PATH_DICT removes Llama v2 entries (llama_v2_7b, llama_v2_13b, llama_v2_70b), prunes Phi-4 multimodal modality-specific variants (image/audio and FP4/FP8 variants), removes deepseek_v3.2_exp_fp4_v2 from tail, and retains GLM-5 FP8 and Kimi K2.5 NVFP4 entries. HF_MODEL_PATH is shortened to end after phi_4_mini_instruct_hf, and LORA_MODEL_PATH initial entries are replaced with llama_v3.1_8b_instruct_fp8.
Integration test waivers tests/integration/test_lists/waives.txt	Test waivers are updated to remove deprecated GLM4Flash and float16-based perf waivers, add deepseek_v3_lite_fp8 with quantization variants (int8_sq_per_tensor, int8_sq_per_token_channel, w4a8_awq), introduce llama_v3.1_nemotron_nano_8b bfloat16 configurations, and add a perf_test_perf_sanity waiver for deepseek-r1 FP4 scenario.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

• NVIDIA/TensorRT-LLM#13641: Both PRs modify tests/integration/test_lists/waives.txt to change which perf/test_perf.py (DeepSeek/waiver) test configurations are skipped/unskipped.
• NVIDIA/TensorRT-LLM#14169: Both PRs modify tests/integration/test_lists/waives.txt to change which integration tests are waived, including perf-related and sanity test cases.
• NVIDIA/TensorRT-LLM#14316: Both PRs modify tests/integration/test_lists/waives.txt by adjusting SKIP waiver entries for perf and perf-sanity test variants.

Suggested reviewers

• tburt-nv
• ZhanruiSunCh
• mzweilz

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The PR description provides a brief explanation of changes but lacks comprehensive documentation of the Description, Test Coverage, and most PR Checklist items specified in the template.	Complete the Description and Test Coverage sections, and verify all PR Checklist items are addressed to match the required template structure.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The PR title accurately describes the main changes: updating model path definitions in test_perf.py and cleaning up waives.txt, which aligns with the actual changeset.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

tests/integration/defs/perf/test_perf.py (1)

1-1: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Update the NVIDIA copyright year for this modified file.

Line 1 still ends at 2025, but this file is modified in 2026 and should be updated accordingly.

Proposed fix

-# SPDX-FileCopyrightText: Copyright (c) 2022-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-FileCopyrightText: Copyright (c) 2022-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.

As per coding guidelines **/*.{cpp,cc,h,hpp,py,cu,cuh}: "Include NVIDIA copyright header on ALL new files; update year on modified files."

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/integration/defs/perf/test_perf.py` at line 1, The SPDX copyright
header in the modified file still lists the year 2025; update the
SPDX-FileCopyrightText header string so the year is 2026 (i.e., change
"2022-2025" to "2022-2026") in test_perf.py to reflect the 2026 modification.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@tests/integration/defs/perf/test_perf.py`:
- Line 1: The SPDX copyright header in the modified file still lists the year
2025; update the SPDX-FileCopyrightText header string so the year is 2026 (i.e.,
change "2022-2025" to "2022-2026") in test_perf.py to reflect the 2026
modification.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 54796924-3e0b-4aeb-aa76-69f39bf963b4

📥 Commits

Reviewing files that changed from the base of the PR and between 57a1b84 and 6c709a9.

📒 Files selected for processing (2)

• tests/integration/defs/perf/test_perf.py
• tests/integration/test_lists/waives.txt

[yufeiwu-nv]

/bot run

[tensorrt-cicd]

PR_Github #50166 [ run ] triggered by Bot. Commit: 145e3d3 Link to invocation

[yufeiwu-nv]

Removed 7 dead bert_attention_plugin does not support SM >= 100 waivers (bart_large_cnn, bert_large, flan_t5_base/large/xl/xxl, mbart_large_50_many_to_one_mmt) that were inadvertently kept during the previous merge — the underlying perf tests no longer exist in any test-db yaml on main, so the original PR removal intent (already applied to the sibling roberta/t5_* entries) is now consistent across the file. Kept the TestGLM4Flash::test_auto_dtype[trtllm-*] waivers since those test parametrizations are still active on b200/h100 and nvbugs/6185480 may not be closed.

/bot run

[yufeiwu-nv]

Removed the 4 perf waivers that this PR originally added — author confirmed those nvbugs (5150255 / 5304388 / 6130334) no longer need to be waived. The PR is now purely the MODEL_PATH_DICT refactor + dead-waiver cleanup.

/bot run

[yufeiwu-nv]

/bot run

[tensorrt-cicd]

PR_Github #50185 [ run ] triggered by Bot. Commit: 5c73391 Link to invocation

[tensorrt-cicd]

PR_Github #50166 [ run ] completed with state ABORTED. Commit: 145e3d3

Link to invocation

[tensorrt-cicd]

PR_Github #50185 [ run ] completed with state SUCCESS. Commit: 5c73391
/LLM/main/L0_MergeRequest_PR pipeline #39727 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[yufeiwu-nv]

/bot help

[github-actions]

GitHub Bot Help

/bot [-h] ['run', 'kill', 'skip', 'reuse-pipeline'] ...

Provide a user friendly way for developers to interact with a Jenkins server.

Run /bot [-h|--help] to print this help message.

See details below for each supported subcommand.

Details

run [--reuse-test (optional)pipeline-id --disable-fail-fast --skip-test --stage-list "A10-PyTorch-1, xxx" --gpu-type "A30, H100_PCIe" --test-backend "pytorch, cpp" --add-multi-gpu-test --only-multi-gpu-test --disable-multi-gpu-test --post-merge --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" --detailed-log --debug(experimental) --high-priority]

Launch build/test pipelines. All previously running jobs will be killed.

--reuse-test (optional)pipeline-id (OPTIONAL) : Allow the new pipeline to reuse build artifacts and skip successful test stages from a specified pipeline or the last pipeline if no pipeline-id is indicated. If the Git commit ID has changed, this option will be always ignored. The DEFAULT behavior of the bot is to reuse build artifacts and successful test results from the last pipeline.

--disable-reuse-test (OPTIONAL) : Explicitly prevent the pipeline from reusing build artifacts and skipping successful test stages from a previous pipeline. Ensure that all builds and tests are run regardless of previous successes.

--disable-fail-fast (OPTIONAL) : Disable fail fast on build/tests/infra failures.

--skip-test (OPTIONAL) : Skip all test stages, but still run build stages, package stages and sanity check stages. Note: Does NOT update GitHub check status.

--stage-list "A10-PyTorch-1, xxx" (OPTIONAL) : Only run the specified test stages. Supports wildcard * for pattern matching (e.g., "*PerfSanity*" matches all stages containing PerfSanity). Examples: "A10-PyTorch-1, xxx", "PerfSanity". Note: Does NOT update GitHub check status.

--gpu-type "A30, H100_PCIe" (OPTIONAL) : Only run the test stages on the specified GPU types. Examples: "A30, H100_PCIe". Note: Does NOT update GitHub check status.

--test-backend "pytorch, cpp" (OPTIONAL) : Skip test stages which don't match the specified backends. Only support [pytorch, cpp, tensorrt, triton]. Examples: "pytorch, cpp" (does not run test stages with tensorrt or triton backend). Note: Does NOT update GitHub pipeline status.

--only-multi-gpu-test (OPTIONAL) : Only run the multi-GPU tests. Note: Does NOT update GitHub check status.

--disable-multi-gpu-test (OPTIONAL) : Disable the multi-GPU tests. Note: Does NOT update GitHub check status.

--add-multi-gpu-test (OPTIONAL) : Force run the multi-GPU tests in addition to running L0 pre-merge pipeline.

--post-merge (OPTIONAL) : Run the L0 post-merge pipeline instead of the ordinary L0 pre-merge pipeline.

--extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" (OPTIONAL) : Run the ordinary L0 pre-merge pipeline and specified test stages. Supports wildcard * for pattern matching. Examples: --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx", --extra-stage "Post-Merge".

--detailed-log (OPTIONAL) : Enable flushing out all logs to the Jenkins console. This will significantly increase the log volume and may slow down the job.

--debug (OPTIONAL) : Experimental feature. Enable access to the CI container for debugging purpose. Note: Specify exactly one stage in the stage-list parameter to access the appropriate container environment. Note: Does NOT update GitHub check status.

--high-priority (OPTIONAL) : Run the pipeline with high priority. This option is restricted to authorized users only and will route the job to a high-priority queue.

kill

kill

Kill all running builds associated with pull request.

skip

skip --comment COMMENT

Skip testing for latest commit on pull request. --comment "Reason for skipping build/test" is required. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

reuse-pipeline

reuse-pipeline

Reuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

[yufeiwu-nv]

/bot skip --comment "only test list modify"

[tensorrt-cicd]

PR_Github #50318 [ skip ] triggered by Bot. Commit: 2109fc8 Link to invocation

[tensorrt-cicd]

PR_Github #50318 [ skip ] completed with state SUCCESS. Commit: 2109fc8
Skipping testing for commit 2109fc8

Link to invocation