[https://nvbugs/6209806][fix] Add an idempotent _relax_llama_validate_architecture() helper that installs a pe

[tensorrt-cicd]

Summary

• Root cause: transformers 5.5.x LlamaConfig.validate_architecture rejects configs where hidden_size % num_attention_heads != 0 even when head_dim is explicitly set, breaking valid models like kanana-1.5-2.1b (hidden_size=1792, num_attention_heads=24, head_dim=128).
• Fix: Add an idempotent _relax_llama_validate_architecture() helper that installs a permissive validator (passes when head_dim is a positive int) and substitutes it in class_validators; call it at the start of load_pretrained_config and load_hf_tokenizer; remove the stale waives.txt entry.
• Automated fix generated by repair-bot

Test plan

• Verify fix on the same GPU type as the original failure
• Check for regressions in related tests

Links

• Bug: https://nvbugs/6209806

Summary by CodeRabbit

Release Notes

• Bug Fixes
  • Fixed loading of Llama models with explicit head_dim configuration settings that previously failed validation checks.

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR adds a compatibility patch for HuggingFace Transformers 5.x LlamaConfig validation that allows models with explicit head_dim to load without failing strict divisibility checks. The patch is applied in config loading and tokenizer initialization paths, and a previously waived test is now enabled.

Changes

Llama Architecture Validation Bypass

Layer / File(s)	Summary
LlamaConfig validation patch definition and config integration tensorrt_llm/_torch/pyexecutor/config_utils.py	_relax_llama_validate_architecture() conditionally patches LlamaConfig.validate_architecture to return early when head_dim is a positive integer, bypassing the hidden_size % num_attention_heads check. The patch also updates __class_validators__ for consistency and is applied at the start of load_pretrained_config() before config materialization.
Tokenizer validation bypass tensorrt_llm/tokenizer/tokenizer.py	load_hf_tokenizer imports and calls _relax_llama_validate_architecture() prior to TransformersTokenizer.from_pretrained() to ensure Transformers 5.x validation is relaxed before tokenizer loading.
Test waiver cleanup tests/integration/test_lists/waives.txt	Removes TestKanana_Instruct::test_auto_dtype from the waives list, confirming the test now passes without requiring a skip.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

• NVIDIA/TensorRT-LLM#13790: Both PRs modify tests/integration/test_lists/waives.txt, with the current PR removing a waived test entry while the related PR manages other test-waiver entries.

Suggested reviewers

• galagam
• nvchenghaoz
• shaharmor98

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the main change: adding an idempotent helper function to relax Llama architecture validation, with proper ticket reference and type tag.
Description check	✅ Passed	The description explains the root cause, the fix, how it integrates into the codebase, and includes test verification and links to the bug tracking the issue.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tensorrt_llm/tokenizer/tokenizer.py`:
- Around line 633-639: The compatibility patch calls
(_ensure_gpt2_bytes_to_unicode_compat and _relax_llama_validate_architecture)
are placed outside the try block in load_hf_tokenizer so import/patch failures
escape instead of being caught; move the two calls (and their import of
_relax_llama_validate_architecture from
tensorrt_llm._torch.pyexecutor.config_utils) inside the existing try: block in
load_hf_tokenizer so any ImportError or runtime error is logged and the function
returns None as intended.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 207680b0-a4cc-4c57-9764-b4cdb1c74c10

📥 Commits

Reviewing files that changed from the base of the PR and between f6ba936 and f3d2b34.

📒 Files selected for processing (3)

• tensorrt_llm/_torch/pyexecutor/config_utils.py
• tensorrt_llm/tokenizer/tokenizer.py
• tests/integration/test_lists/waives.txt

💤 Files with no reviewable changes (1)

• tests/integration/test_lists/waives.txt

[coderabbitai]

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Move the compatibility patch inside the existing try block.

These new lines run before the try: at Line 641, so any import or patching failure now escapes instead of following load_hf_tokenizer()'s current "log and return None" behavior.

💡 Proposed fix

-    _ensure_gpt2_bytes_to_unicode_compat()
-    # AutoTokenizer.from_pretrained internally calls AutoConfig.from_pretrained,
-    # which triggers transformers 5.x LlamaConfig.validate_architecture.
-    # Relax it so models with explicit `head_dim` (e.g. kanana-1.5-2.1b) load.
-    from tensorrt_llm._torch.pyexecutor.config_utils import \
-        _relax_llama_validate_architecture
-    _relax_llama_validate_architecture()
-
     try:
+        _ensure_gpt2_bytes_to_unicode_compat()
+        # AutoTokenizer.from_pretrained internally calls AutoConfig.from_pretrained,
+        # which triggers transformers 5.x LlamaConfig.validate_architecture.
+        # Relax it so models with explicit `head_dim` (e.g. kanana-1.5-2.1b) load.
+        from tensorrt_llm._torch.pyexecutor.config_utils import \
+            _relax_llama_validate_architecture
+        _relax_llama_validate_architecture()
         tokenizer = TransformersTokenizer.from_pretrained(
             model_dir,
             legacy=False,

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tensorrt_llm/tokenizer/tokenizer.py` around lines 633 - 639, The
compatibility patch calls (_ensure_gpt2_bytes_to_unicode_compat and
_relax_llama_validate_architecture) are placed outside the try block in
load_hf_tokenizer so import/patch failures escape instead of being caught; move
the two calls (and their import of _relax_llama_validate_architecture from
tensorrt_llm._torch.pyexecutor.config_utils) inside the existing try: block in
load_hf_tokenizer so any ImportError or runtime error is logged and the function
returns None as intended.

[dongxuy04]

It seems that LlamaConfig will set head_dim if it is None here. So it might always be int and very likely larger than 0. And this seems might disable the self.hidden_size % self.num_attention_heads != 0 check for all cases.