[TRTLLM-11875][feat] BREAKING: MambaCacheManager based on KVCacheManagerV2 & agentic prefix caching

[VALLIS-NERIA]

Description

This PR adds the KV-cache-manager V2 implementation for hybrid Mamba models and moves Mamba state reuse onto explicit snapshot boundaries.

Key changes:

• Add MambaHybridCacheManagerV2, including recurrent-state allocation, replay, promotion, dynamic-tree capacity, and deterministic stochastic-rounding state.
• Add MambaStateConfig with periodic and fixed-position snapshot policies. The legacy mamba_state_cache_interval remains accepted only as a deprecated config-file alias.
• Reuse Mamba state only at planned snapshot boundaries. Hybrid models that enable block reuse without any snapshot policy now warn and disable block reuse during model-aware config validation.
• Add per-conversation reuse semantics: periodic Mamba snapshots are disabled and only explicit stable conversation boundaries are committed.
• Add V2 Mamba state transfer to Python/NIXL disaggregated serving using explicit slot/layer strides.
• Support one-model MTP and dynamic-tree MTP with V2, including target-tree mask regeneration and accepted-state promotion.
• Handle attention-free hybrid PP stages in disaggregated transfer without zero-head duplication arithmetic.

This is an API/config migration: use kv_cache_config.mamba_state_config.periodic_snapshot_interval instead of kv_cache_config.mamba_state_cache_interval in Python code.

Test Coverage

• Snapshot/config resolution tests, including V2-only fixed offsets and warning-plus-disable behavior when no snapshot policy exists.
• V1/V2 hybrid manager, replay, snapshot planning, state promotion, scheduler, capacity, and per-conversation unit tests.
• V1/V2 Mamba disaggregated extraction and transfer tests, including affine stride layouts and attention-free PP ranks.
• Dynamic-tree metadata, relocation, capacity, and accepted-leaf state-promotion tests.
• Parameterized Qwen3.5 and Nemotron Super V3 MTP accuracy tests are registered in the B200 CI lists for both V1 and V2.

Local validation after merging current main:

• Relevant dynamic-tree/V2 tests: 4 passed.
• Snapshot model-resolution tests: 6 passed.
• Disaggregated peer/rank-info tests: 25 passed.
• Qwen3.5 V2 MTP GSM8K: 94.8446 (threshold 91.327), passed.
• All four new V1/V2 integration node IDs collect successfully.
• Nemotron Super V3 V2 MTP: MMLU 85.38 (threshold 83.483) and GSM8K 92.15 (threshold 88.697), passed.
• Pre-commit checks pass for all changed files.

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions).

• If PR introduces API changes, an appropriate PR label is added, either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities.

• CODEOWNERS is updated if ownership changes.

• Documentation is updated as needed.

• Update the TAVA architecture diagram if there is a significant design change.

• The reviewers assigned automatically or manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, comment /bot help.

[VALLIS-NERIA]

/bot run

[tensorrt-cicd]

PR_Github #60284 [ run ] triggered by Bot. Commit: bfc9660 Link to invocation

[tensorrt-cicd]

PR_Github #60284 [ run ] completed with state FAILURE. Commit: bfc9660
/LLM/main/L0_MergeRequest_PR pipeline #48641 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[VALLIS-NERIA]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #60398 [ run ] triggered by Bot. Commit: 73e76f7 Link to invocation

[tensorrt-cicd]

PR_Github #60398 [ run ] completed with state FAILURE. Commit: 73e76f7
/LLM/main/L0_MergeRequest_PR pipeline #48738 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[tensorrt-cicd]

PR_Github #61755 [ run ] triggered by Bot. Commit: 78be52a Link to invocation

[VALLIS-NERIA]

/bot run

[tensorrt-cicd]

PR_Github #61755 [ run ] completed with state FAILURE. Commit: 78be52a
/LLM/main/L0_MergeRequest_PR pipeline #49957 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[tensorrt-cicd]

PR_Github #61797 [ run ] triggered by Bot. Commit: 78be52a Link to invocation

[VALLIS-NERIA]

/bot run

[tensorrt-cicd]

PR_Github #61797 [ run ] completed with state SUCCESS. Commit: 78be52a
/LLM/main/L0_MergeRequest_PR pipeline #49996 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[tensorrt-cicd]

PR_Github #61809 [ run ] triggered by Bot. Commit: 78be52a Link to invocation

[VALLIS-NERIA]

/bot run

[tensorrt-cicd]

PR_Github #61809 [ run ] completed with state FAILURE. Commit: 78be52a
/LLM/main/L0_MergeRequest_PR pipeline #50009 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[tensorrt-cicd]

PR_Github #61815 [ run ] triggered by Bot. Commit: 78be52a Link to invocation

[VALLIS-NERIA]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #61815 [ run ] completed with state SUCCESS. Commit: 78be52a
/LLM/main/L0_MergeRequest_PR pipeline #50017 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[tensorrt-cicd]

PR_Github #61830 [ run ] triggered by Bot. Commit: 78be52a Link to invocation

[juney-nvidia]

Approved from telemetry and API perspectives

[VALLIS-NERIA]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #61830 [ run ] completed with state SUCCESS. Commit: 78be52a
/LLM/main/L0_MergeRequest_PR pipeline #50026 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[tensorrt-cicd]

PR_Github #61884 [ run ] triggered by Bot. Commit: 78be52a Link to invocation

[VALLIS-NERIA]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #61884 [ run ] completed with state SUCCESS. Commit: 78be52a
/LLM/main/L0_MergeRequest_PR pipeline #50078 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[tensorrt-cicd]

PR_Github #61901 [ run ] triggered by Bot. Commit: 78be52a Link to invocation

[VALLIS-NERIA]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #61901 [ run ] completed with state FAILURE. Commit: 78be52a
/LLM/main/L0_MergeRequest_PR pipeline #50092 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[tensorrt-cicd]

PR_Github #61909 [ run ] triggered by Bot. Commit: 78be52a Link to invocation

[VALLIS-NERIA]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #61909 [ run ] completed with state FAILURE. Commit: 78be52a
/LLM/main/L0_MergeRequest_PR pipeline #50100 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[tensorrt-cicd]

PR_Github #61911 [ run ] triggered by Bot. Commit: 78be52a Link to invocation

[tensorrt-cicd]

PR_Github #61911 [ run ] completed with state FAILURE. Commit: 78be52a
/LLM/main/L0_MergeRequest_PR pipeline #50102 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation