Skip to content

Revert "[None][test] Add support for nemotron_3_ultra_550b_nvfp4 model in performance tests and configurations" - #15310

Merged
tburt-nv merged 1 commit into
mainfrom
revert-15166-bug
Jun 12, 2026
Merged

Revert "[None][test] Add support for nemotron_3_ultra_550b_nvfp4 model in performance tests and configurations"#15310
tburt-nv merged 1 commit into
mainfrom
revert-15166-bug

Conversation

@tburt-nv

@tburt-nv tburt-nv commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Reverts #15166

Summary by CodeRabbit

  • Tests

    • Expanded HuggingFace model path mappings with new model variants (Llama v3.1, Phi 4 Mini).
    • Updated model resolution and engine output path handling in performance test framework.
    • Added new performance test case for Llama v3.1 8B Instruct model.
  • Chores

    • Removed Nemotron 3 Ultra 550B model configuration and related test cases.

@tburt-nv
tburt-nv requested review from a team as code owners June 12, 2026 16:48
@tburt-nv
tburt-nv merged commit b03b78f into main Jun 12, 2026
8 of 10 checks passed
@tburt-nv
tburt-nv deleted the revert-15166-bug branch June 12, 2026 16:48
@coderabbitai

coderabbitai Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 8db44068-eb1b-4959-8514-ebee9cddf1dd

📥 Commits

Reviewing files that changed from the base of the PR and between db7161b and 0c2d91b.

📒 Files selected for processing (5)
  • tests/integration/defs/perf/_model_paths.py
  • tests/integration/defs/perf/pytorch_model_config.py
  • tests/integration/defs/perf/test_perf.py
  • tests/integration/test_lists/qa/llm_perf_core.yml
  • tests/integration/test_lists/test-db/l0_perf.yml

📝 Walkthrough

Walkthrough

The PR updates the TensorRT-LLM performance test infrastructure to refactor HuggingFace model path handling. It expands model mappings, reorders test conditions, removes legacy Nemotron-3-Ultra-550B support, and adjusts test matrices to cover new model variants and improve test organization across multiple GPU configurations.

Changes

Performance Test Infrastructure Refactoring

Layer / File(s) Summary
HuggingFace model path mapping expansion
tests/integration/defs/perf/_model_paths.py
The HF_MODEL_PATH dictionary is expanded with new HuggingFace model entries (llama_v3.1_8b_hf, llama_v3.1_8b_instruct_hf, phi_4_mini_instruct_hf) while removing the old nemotron_3_ultra_550b_nvfp4 entry.
Model resolution and engine path refactoring
tests/integration/defs/perf/test_perf.py, tests/integration/defs/perf/pytorch_model_config.py
get_model_dir is rewritten to resolve HuggingFace models through MODEL_PATH_DICT and llm_models_root() with *_hf suffix handling. Build command wiring is updated to use HF_MODEL_PATH lookups. Engine output directory structure is changed to nest under engine_dir/<hf_model_name>/<tp_pp_str>. Legacy model identifiers are removed from gating sets, and the Nemotron-3-Ultra-550B pattern configuration is removed.
Test matrix reorganization and expansion
tests/integration/test_lists/qa/llm_perf_core.yml, tests/integration/test_lists/test-db/l0_perf.yml
Condition blocks are reorganized with new test entries added for nemotron_nano_12b_v2, qwen3.5 variants, llama_v3.3_nemotron_super_49b, and llama_v3.1_nemotron_ultra_253b across multiple GPU configurations. Duplicate test blocks are consolidated, and a new l0_perf test case is added for llama_v3.1_8b_instruct on H100 GPUs.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • NVIDIA/TensorRT-LLM#15166: Modifies the same performance-test infrastructure for nemotron_3_ultra_550b_nvfp4, expanding HF_MODEL_PATH and refactoring model resolution logic in test_perf.py.

Suggested reviewers

  • StanleySun639
  • ruodil
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch revert-15166-bug

Comment @coderabbitai help to get the list of available commands and usage tips.

yufeiwu-nv added a commit to yufeiwu-nv/TensorRT-LLM that referenced this pull request Jun 15, 2026
…#15166)

NVIDIA#15166 was temporarily reverted in NVIDIA#15310. Reapply it here so the
TensorRT perf test removal in l0_perf.yml lands together with the
TensorRT stage/baseline cleanup in this PR, keeping the test list and
Jenkins perf stages consistent.

Signed-off-by: yufeiwu-nv <230315618+yufeiwu-nv@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant