[https://nvbugs/6261164][fix] When spec_config is None, allocate a 1-slot placeholder for…

[tensorrt-cicd]

Summary

• Root cause: SSM/conv backends unconditionally register spec-decoding intermediate state buffers (SpecSSM/SpecCausalConvResourceHandler); with spec decoding off they're never bound and each falls through to a full max_num_state_slots allocation (~650 MiB × 54 layers), OOMing the 44 GiB L40S.
• Fix: When spec_config is None, allocate a 1-slot placeholder for SpecSSM/SpecCausalConv handlers so the cache slot exists without the multi-GB allocation; also removed the now-passing test's waiver.
• Automated fix generated by repair-bot

Test plan

• Verify fix on the same GPU type as the original failure
• Check for regressions in related tests

Links

• Bug: https://nvbugs/6261164

Summary by CodeRabbit

• Performance
  • Reduced memory overhead by optimizing speculative decoding buffer allocation when disabled
• Tests
  • Re-enabled previously skipped test for accuracy validation

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b9d3fe99-75f5-4797-8d21-9f46803aef50

📥 Commits

Reviewing files that changed from the base of the PR and between 846d0f4 and 19ca21d.

📒 Files selected for processing (2)

• tensorrt_llm/_torch/auto_deploy/shim/interface.py
• tests/integration/test_lists/waives.txt

💤 Files with no reviewable changes (1)

• tests/integration/test_lists/waives.txt

---

📝 Walkthrough

Walkthrough

This PR optimizes speculative decoding state buffer allocation in the auto-deploy shim layer. When speculative config is disabled with speculative SSM/Conv handlers, the allocator now uses a 1-slot placeholder instead of full buffer allocation, reducing unnecessary memory overhead. A previously-waived integration test is re-enabled, confirming the fix works.

Changes

Speculative buffer allocation and test enablement

Layer / File(s)	Summary
Speculative buffer allocation optimization tensorrt_llm/_torch/auto_deploy/shim/interface.py	_allocate_unmanaged_state_resource now branches on self._spec_config is None combined with speculative SSM/Conv handler types. When both conditions match, max_num_state_slots is set to 1 as a placeholder; otherwise the existing full allocation and optional Mamba-hybrid padding logic is preserved.
Test waiver removal tests/integration/test_lists/waives.txt	Removes the waiver entry for TestNemotronV2::test_fp8[True], re-enabling this test in CI.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

• NVIDIA/TensorRT-LLM#14787: Modifies tests/integration/test_lists/waives.txt to adjust other integration-test skip/waiver entries.
• NVIDIA/TensorRT-LLM#14783: Also updates tests/integration/test_lists/waives.txt to change CI skip/waiver behavior for test_llm_api_* test cases.
• NVIDIA/TensorRT-LLM#14839: Modifies tests/integration/test_lists/waives.txt to change which integration tests are skipped in CI.

Suggested reviewers

• xinhe-nv
• joyang-nv

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title correctly references the NVBugs ID, specifies the fix type, and accurately summarizes the main change: allocating a 1-slot placeholder when spec_config is None.
Description check	✅ Passed	The description includes a clear root cause analysis, the fix explanation, test verification details, and a bug reference link. It follows the expected PR description structure with summary, test plan, and links sections.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[hnover-nv]

Surely we can do this without allocating any memory. I will reprompt the bot