[https://nvbugs/5948539][fix] Fix disagg gen-only benchmark

[Tabrizian]

@coderabbitai summary

Description

The benchmark fill loop in py_executor.py blocked the GEN server until all benchmark_req_queues_size requests were fetched before processing any of them. In disagg gen_only mode, the CTX server did not have enough KV cache to send all requests at once — it needed the GEN server to fetch and process requests to free KV cache. This created a deadlock.

Changes:

• Replace the full fill loop with a batched version that fetches tp_size requests per iteration (with ADP) or 1 request per iteration (without ADP), allowing the GEN server to process requests incrementally and free KV cache for the CTX server

Test Coverage

• Reproduced the hang on Lyris with DeepSeek-R1 GEN-DEP32 config (TP32, ISL 8192/OSL 1024, concurrency 1024)
• Verified the fix resolves the hang — benchmark completed successfully in ~737 seconds
• Verified even request distribution across ranks (32 requests per rank)

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[coderabbitai]

📝 Walkthrough

Walkthrough

The PR modifies benchmark handling in the TensorRT-LLM executor by removing a dedicated fill loop and refactoring dummy request padding logic to defer dummy request augmentation in benchmark disaggregation mode until all real requests are fetched.

Changes

Cohort / File(s)	Summary
Executor Benchmark Logic tensorrt_llm/_torch/pyexecutor/py_executor.py	Removed benchmark-specific fill loop from _prepare_and_schedule_batch. Modified _pad_attention_dp_dummy_request to skip dummy request padding in benchmark disaggregation mode until all real benchmark requests are fetched (based on num_fetch_requests and benchmark_req_queues_size) and while not in warmup mode. Updated comments and log messages to reflect this deferral behavior.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description check	✅ Passed	The pull request description clearly explains the issue (deadlock in disagg gen_only mode), the root cause (blocking fill loop), and the solution (batched request fetching). All required sections are present with sufficient detail.
Title check	✅ Passed	The title directly references the bug fix for disagg gen-only benchmark hang, which is the primary change in the pull request that removes the fill loop and adjusts dummy-request padding logic.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Tabrizian]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #38537 [ run ] triggered by Bot. Commit: 459b3e7 Link to invocation

[Tabrizian]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #38540 [ run ] triggered by Bot. Commit: 2c8bb03 Link to invocation

[tensorrt-cicd]

PR_Github #38540 [ run ] completed with state SUCCESS. Commit: 2c8bb03
/LLM/main/L0_MergeRequest_PR pipeline #29886 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

[pcastonguay]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #38590 [ run ] triggered by Bot. Commit: 2c8bb03 Link to invocation

[tensorrt-cicd]

PR_Github #38590 [ run ] completed with state SUCCESS. Commit: 2c8bb03
/LLM/main/L0_MergeRequest_PR pipeline #29926 completed with status: 'SUCCESS'

CI Report

Link to invocation