[https://nvbugs/6185234][fix] route load_hf_model_config via AutoConfig

[Hudayday]

Summary

Follow-up to #14293. ModelLoader.load_hf_model_config in
tensorrt_llm/llmapi/llm_utils.py was still calling
transformers.PretrainedConfig.from_pretrained directly, which bypasses
the transformers.models.auto.configuration_auto.CONFIG_MAPPING that
#14293 registers deepseek_v32 / kimi_k2 into. On transformers 5.5.x
the base-class from_pretrained returns a bare PretrainedConfig that
lacks max_position_embeddings. Route through AutoConfig.from_pretrained
so the #14293 mapping applies.

Symptom and CI history

L0_PostMerge stage
GB200-36_GPUs-9_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE8-GPU32-Post-Merge-9,
test perf/test_perf_sanity.py::test_e2e[disagg_upload-gen_only-gb200_deepseek-v32-fp4_8k1k_con1024_ctx1_dep4_gen1_dep 32_eplb256_mtp3_ccb-NIXL]:

GEN_0.0.log:35 at build 2726 still shows:

[TRT-LLM] [W] [llmapi] Failed to load hf model config from
.../DeepSeek-V3.2-Exp-FP4-v2, encountered error:
'PreTrainedConfig' object has no attribute 'max_position_embeddings'

— because PretrainedConfig.from_pretrained doesn't consult CONFIG_MAPPING,
so #14293's registration is moot for this call path. The function returns
None, downstream uses a fallback path that doesn't match the V3.2 config
shape, and on 9-node MPI weight-load workers eventually die mid-load with
PMIX -61 LOST_CONNECTION_TO_PEER.

Mechanism

transformers.PretrainedConfig.from_pretrained (base class) does not
dispatch through CONFIG_MAPPING. AutoConfig.from_pretrained does.
Routing the call through AutoConfig is what _torch/configs/__init__.py
already assumes for the rest of the codebase; this was the only remaining
direct call to the base class in tensorrt_llm/.

Verified standalone on transformers 5.5.3:

• PretrainedConfig.from_pretrained → bare PreTrainedConfig (the bypass)
• AutoConfig.from_pretrained → registered subclass with all attributes
  (the fix)

Test plan

• Extended tests/unittest/_torch/test_custom_config_registration.py
  with test_load_hf_model_config_uses_autoconfig_dispatch that
  asserts ModelLoader.load_hf_model_config returns a
  DeepseekV3Config instance (not bare PretrainedConfig) for
  deepseek_v32 and kimi_k2 model_types.
• Standalone mechanism reproducer run on transformers 5.5.3 inside
  the post-merge container (srun job 2265026): both PretrainedConfig
  bypass and AutoConfig dispatch behavior confirmed.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e6f83335-ddf9-46e3-8bd4-5aaa0a6689ca

📥 Commits

Reviewing files that changed from the base of the PR and between ac0be47 and cf302d3.

📒 Files selected for processing (2)

• tensorrt_llm/llmapi/llm_utils.py
• tests/unittest/_torch/test_custom_config_registration.py

---

📝 Walkthrough

Walkthrough

The PR updates HuggingFace model config loading to use AutoConfig.from_pretrained() instead of PretrainedConfig.from_pretrained(), enabling model-type-specific config dispatch through CONFIG_MAPPING. A regression test validates this behavior for deepseek_v32 and kimi_k2 models.

Changes

AutoConfig dispatch for HuggingFace model configuration

Layer / File(s)	Summary
Use AutoConfig dispatch and validate registration behavior tensorrt_llm/llmapi/llm_utils.py, tests/unittest/_torch/test_custom_config_registration.py	load_hf_model_config now routes through AutoConfig.from_pretrained() to enable CONFIG_MAPPING dispatch for model-type-specific config classes; regression test verifies the behavior works correctly for deepseek_v32 and kimi_k2 models, including preservation of max_position_embeddings.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

• NVIDIA/TensorRT-LLM#14293: Main PR switches load_hf_model_config to use HuggingFace AutoConfig dispatch and adds regression test for deepseek_v32/kimi_k2, directly depending on this PR's CONFIG_MAPPING registration for those same model types.

Suggested reviewers

• liji-nv
• symphonylyh
• VALLIS-NERIA

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title '[https://nvbugs/6185234][fix] route load_hf_model_config via AutoConfig' accurately describes the main change: routing the load_hf_model_config function through AutoConfig instead of PretrainedConfig.
Description check	✅ Passed	The PR description provides clear explanation of the issue, mechanism, symptom, test plan, and CI history. However, it does not include a dedicated 'Description' section header or formal 'Test Coverage' section as specified in the template.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Hudayday]

/bot run --disable-fail-fast --extra-stage "DGX_B200-8_GPUs-PyTorch-1, DGX_B200-8_GPUs-PyTorch-2, DGX_B200-8_GPUs-AutoDeploy-Post-Merge-1, DGX_B200-8_GPUs-PyTorch-PerfSanity-Post-Merge-4, GB200-36_GPUs-9_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE8-GPU32-Post-Merge-9"

[tensorrt-cicd]

PR_Github #49696 [ run ] triggered by Bot. Commit: cf302d3 Link to invocation

[tensorrt-cicd]

PR_Github #49696 [ run ] completed with state SUCCESS. Commit: cf302d3
/LLM/main/L0_MergeRequest_PR pipeline #39302 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[Hudayday]

/bot run --disable-fail-fast --extra-stage "DGX_B200-8_GPUs-PyTorch-1, DGX_B200-8_GPUs-PyTorch-2, DGX_B200-8_GPUs-AutoDeploy-Post-Merge-1, DGX_B200-8_GPUs-PyTorch-PerfSanity-Post-Merge-4, GB200-36_GPUs-9_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE8-GPU32-Post-Merge-9"

[tensorrt-cicd]

PR_Github #49816 [ run ] triggered by Bot. Commit: cf302d3 Link to invocation

[tensorrt-cicd]

PR_Github #49816 [ run ] completed with state SUCCESS. Commit: cf302d3
/LLM/main/L0_MergeRequest_PR pipeline #39401 completed with status: 'SUCCESS'

CI Report

Link to invocation

[Superjomn]

LGTM on the llmapi changes.