[TRTLLM-12062][test] remove obsolete model tests

[xinhe-nv]

Summary by CodeRabbit

• Tests
  • Updated test selection lists across integration test suites to rebalance parameterized test coverage for various hardware configurations and model evaluation scenarios.
  • remove Llama 3.1 70B, Llama 3.1 405B, Llama 3.2 1B, Llama 3.3 Nemotron 49B, Llama 3.1 Nemotron Nano 8B, Llama 2 13B, Mixtral 8x7B, Mistral 7B, Mistral Small 24B, StarCoder2, Phi-4, Bielik 11B tests

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[xinhe-nv]

/bot run --stage-list ""

[coderabbitai]

📝 Walkthrough

Walkthrough

Multiple test list manifests across QA and test-database configurations have been updated to remove test cases for various models including Phi-4, Mistral, Llama, and others. These changes rebalance test coverage selections across hardware configurations while adding limited new test cases to one manifest.

Changes

Cohort / File(s)	Summary
QA Disaggregated & PyTorch Test Lists tests/integration/test_lists/qa/llm_function_core.txt, tests/integration/test_lists/qa/llm_function_l20.txt, tests/integration/test_lists/qa/llm_function_rtx6k.txt	Removed PyTorch LLM API test cases (TestPhi4MM, TestMistral*, TestBielik, TestLlava) and e2e quickstart multimodal variants. llm_function_core.txt also added new disaggregated serving and pytorch API test cases (DeepSeekV3Lite, Gemma3, GPTOSS, Llama3_1, Qwen3Next variants).
QA Spark Test Lists tests/integration/test_lists/qa/llm_spark_core.txt, tests/integration/test_lists/qa/llm_spark_func.yml	Removed multimodal Phi-4 MM (test_ptp_quickstart_multimodal_phi4mm) e2e test cases across all variants, accuracy tests, and selected test_ptp_quickstart_advanced cases for Llama3.1-70B and Nemotron-Super-49B.
QA Triton Integration List tests/integration/test_lists/qa/llm_triton_integration.txt	Removed Mistral v1 7B and Mistral Small 3.1 24B Pixtral test cases covering IFB, Python backend, and ensemble configurations with different cache management strategies.
Test-DB Hardware-Specific Lists tests/integration/test_lists/test-db/l0_a30.yml, tests/integration/test_lists/test-db/l0_dgx_h100.yml, tests/integration/test_lists/test-db/l0_dgx_h200.yml, tests/integration/test_lists/test-db/l0_h100.yml, tests/integration/test_lists/test-db/l0_l40s.yml, tests/integration/test_lists/test-db/l0_rtx_pro_6000.yml	Removed model-specific accuracy and e2e test cases across multiple backends (TRT, PyTorch) including Mistral multimodal, Phi-4, Llama3/Llama3.2, Starcoder2, and Mixtral tests with various quantization modes (fp8, nvfp4, auto_dtype).
Test-DB Single-Entry Removals tests/integration/test_lists/test-db/l0_b200.yml, tests/integration/test_lists/test-db/l0_gb203.yml	Removed individual TRT accuracy test case (TestMixtral8x7B::test_nvfp4_prequantized from B200) and example test case (test_llm_llama_v3_1_1node_single_gpu[llama-3.2-1b-disable_fp8] from GB203).

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Suggested reviewers

• litaotju
• Shixiaowei02
• yuxianq
• mikeiovine
• nvchenghaoz

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The PR description only contains the template and checklist but provides no actual explanation of what was changed, why, or test coverage details.	Add a brief description explaining which obsolete model tests were removed and why, plus a list of affected test files and the rationale for removal.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly describes the main change: removing obsolete model tests from test lists across multiple files.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (1)

tests/integration/test_lists/qa/llm_function_core.txt (1)

861-906: Please confirm these new examples/serve + broad test_e2e entries belong in llm_function_core.txt.

llm_function_core.txt is intended as the primary single-node multi-GPU functional list; these additions may broaden scope beyond core and increase daily/release runtime. If intentional, add a short rationale in PR notes; otherwise consider moving non-core cases to llm_function_full.txt (or a more appropriate QA list).
As per coding guidelines, llm_function_core.txt should remain an intentional curated set for primary single-node multi-GPU functional scenarios.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@tests/integration/test_lists/qa/llm_function_core.txt` around lines 861 -
906, The PR added many examples/serve tests and broad test_e2e entries into
llm_function_core.txt which is meant to be a curated single-node multi-GPU
functional list; either confirm they intentionally expand the core scope by
adding a short rationale to the PR notes or remove/move the non-core entries
(e.g., all examples/serve/test_serve.py and the wide set of test_e2e.py cases
listed) from llm_function_core.txt into llm_function_full.txt (or another
appropriate QA list), and update the PR description to state which tests remain
in core vs moved so reviewers know the change intent.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@tests/integration/test_lists/qa/llm_function_core.txt`:
- Around line 861-906: The PR added many examples/serve tests and broad test_e2e
entries into llm_function_core.txt which is meant to be a curated single-node
multi-GPU functional list; either confirm they intentionally expand the core
scope by adding a short rationale to the PR notes or remove/move the non-core
entries (e.g., all examples/serve/test_serve.py and the wide set of test_e2e.py
cases listed) from llm_function_core.txt into llm_function_full.txt (or another
appropriate QA list), and update the PR description to state which tests remain
in core vs moved so reviewers know the change intent.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: ce0c938f-ae46-45f2-b17e-e08644856bc7

📥 Commits

Reviewing files that changed from the base of the PR and between 1d17dad and 9421db1.

📒 Files selected for processing (14)

• tests/integration/test_lists/qa/llm_function_core.txt
• tests/integration/test_lists/qa/llm_function_l20.txt
• tests/integration/test_lists/qa/llm_function_rtx6k.txt
• tests/integration/test_lists/qa/llm_spark_core.txt
• tests/integration/test_lists/qa/llm_spark_func.yml
• tests/integration/test_lists/qa/llm_triton_integration.txt
• tests/integration/test_lists/test-db/l0_a30.yml
• tests/integration/test_lists/test-db/l0_b200.yml
• tests/integration/test_lists/test-db/l0_dgx_h100.yml
• tests/integration/test_lists/test-db/l0_dgx_h200.yml
• tests/integration/test_lists/test-db/l0_gb203.yml
• tests/integration/test_lists/test-db/l0_h100.yml
• tests/integration/test_lists/test-db/l0_l40s.yml
• tests/integration/test_lists/test-db/l0_rtx_pro_6000.yml

💤 Files with no reviewable changes (13)

• tests/integration/test_lists/test-db/l0_b200.yml
• tests/integration/test_lists/test-db/l0_gb203.yml
• tests/integration/test_lists/qa/llm_spark_core.txt
• tests/integration/test_lists/test-db/l0_l40s.yml
• tests/integration/test_lists/qa/llm_function_l20.txt
• tests/integration/test_lists/test-db/l0_rtx_pro_6000.yml
• tests/integration/test_lists/test-db/l0_a30.yml
• tests/integration/test_lists/test-db/l0_h100.yml
• tests/integration/test_lists/test-db/l0_dgx_h200.yml
• tests/integration/test_lists/test-db/l0_dgx_h100.yml
• tests/integration/test_lists/qa/llm_spark_func.yml
• tests/integration/test_lists/qa/llm_function_rtx6k.txt
• tests/integration/test_lists/qa/llm_triton_integration.txt

[tensorrt-cicd]

PR_Github #44659 [ run ] triggered by Bot. Commit: 9421db1 Link to invocation

[xinhe-nv]

/bot run --stage-list ""

[tensorrt-cicd]

PR_Github #44663 [ run ] triggered by Bot. Commit: d9869f6 Link to invocation

[tensorrt-cicd]

PR_Github #44659 [ run ] completed with state ABORTED. Commit: 9421db1

Link to invocation

[tensorrt-cicd]

PR_Github #44663 [ run ] completed with state FAILURE. Commit: d9869f6
/LLM/main/L0_MergeRequest_PR pipeline #35035 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

[xinhe-nv]

/bot run --stage-list ""

[xinhe-nv]

/bot run --skip-test

[xinhe-nv]

/bot run --skip-test

[tensorrt-cicd]

PR_Github #44892 [ run ] triggered by Bot. Commit: 784429c Link to invocation

[tensorrt-cicd]

PR_Github #44892 [ run ] completed with state SUCCESS. Commit: 784429c
/LLM/main/L0_MergeRequest_PR pipeline #35226 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

[xinhe-nv]

/bot reuse-pipeline

[tensorrt-cicd]

PR_Github #44954 [ reuse-pipeline ] triggered by Bot. Commit: 24ecfd4 Link to invocation

[tensorrt-cicd]

PR_Github #44954 [ reuse-pipeline ] completed with state SUCCESS. Commit: 24ecfd4
Reusing PR_Github #44892 (Partly Tested) for commit 24ecfd4

Link to invocation