[https://nvbugs/6104831][test] PR #14768 rebased on latest main — perf-sanity verification (DO NOT REVIEW)

[chienchunhung]

Purpose — verification only, NOT for merge

This PR is an experiment to isolate whether the perf-sanity regressions seen on PR #14768 (builds #40858, #41014, #41125 — DeepSeek R1 FP4 V2 on GB200) are attributable to PR #14768's code or to branch staleness.

What this PR contains

The net diff of PR #14768 (A1+A2+A4+A7+A8 disaggregated KV cache transceiver lifetime / RAII / dedup / NIXL keep-alive fixes), applied as a single squashed commit on top of the latest upstream/main HEAD.

• 13 files changed: 707 insertions, 50 deletions
• Identical net diff to PR [https://nvbugs/6104831][fix] Enforce request and buffer index lifecycle integrity #14768 verified file-by-file (same C++ transceiver, BufferIndexHolder, formatter, and unit-test changes)
• The only difference vs PR [https://nvbugs/6104831][fix] Enforce request and buffer index lifecycle integrity #14768: this branch is rebased onto current upstream/main (32 commits ahead of PR [https://nvbugs/6104831][fix] Enforce request and buffer index lifecycle integrity #14768's merge-base), pulling in recent fused-MoE / NVFP4 / speculative-decoding improvements

Hypothesis being tested

PR #14768 fails 5 perf-sanity tests on GB200 against an auto-uploaded baseline:

• aggr_upload-deepseek_r1_fp4_v2_grace_blackwell-r1_fp4_v2_tep4_mtp3_{1k1k,8k1k}
• aggr_upload-deepseek_r1_fp4_v2_grace_blackwell-r1_fp4_v2_tp4_mtp3_{1k1k,8k1k}
• aggr_upload-k25_thinking_fp4_grace_blackwell-k25_thinking_fp4_dep4_8k1k

The PR's diff only modifies disaggregated transceiver code paths (gated by mCacheTransceiver != nullptr / if (self.kv_cache_transceiver):). The failing tests are aggregated-mode (aggr_upload-*) — config files in tests/scripts/perf-sanity/aggregated/ with no cache_transceiver_config field — so PR #14768's modifications are not on the failing tests' execution path. Same tests pass on main postmerge (#2750, #2751, #2754) consistently.

Hypothesis: the perf "regression" reflects baseline drift — main has perf improvements (~632 LOC in fused-MoE / NVFP4 / speculative since PR #14768's merge-base) that the auto-uploaded baseline reflects, but PR #14768's stale branch lacks. Rebasing should resolve the apparent regression.

How this will be verified

/bot run --stage-list "GB200-4_GPUs-PyTorch-PerfSanity-1, GB200-4_GPUs-PyTorch-PerfSanity-2" — runs only the 5 failing tests.

• If green: branch staleness was the cause. Rebase PR [https://nvbugs/6104831][fix] Enforce request and buffer index lifecycle integrity #14768 to fix.
• If still failing: the rebase isn't sufficient. Either a real PR code effect needs further investigation, or the baseline DB needs a refresh.

Will close this PR once the signal is collected.

[chienchunhung]

/bot run --stage-list "GB200-4_GPUs-PyTorch-PerfSanity-1, GB200-4_GPUs-PyTorch-PerfSanity-2"

[chienchunhung]

/bot run --disable-fail-fast --stage-list "GB200-4_GPUs-PyTorch-PerfSanity-1, GB200-4_GPUs-PyTorch-PerfSanity-2"

[tensorrt-cicd]

PR_Github #51879 [ run ] triggered by Bot. Commit: 152f6b3 Link to invocation

[tensorrt-cicd]

PR_Github #51879 [ run ] completed with state SUCCESS. Commit: 152f6b3
/LLM/main/L0_MergeRequest_PR pipeline #41238 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation