[https://nvbugs/6245861][fix] Gate the two ID None-checks on `finish_reason in _GEN_PENDING_FINISH_REASONS`… by tensorrt-cicd · Pull Request #14908 · NVIDIA/TensorRT-LLM

tensorrt-cicd · 2026-06-03T13:30:10Z

Summary

Root cause: _verify_ctx_response unconditionally required ctx_request_id/disagg_request_id non-None, but a CTX request that finishes early (finish_reason='stop') never sets up the KV handoff, leaving them None → ValueError → HTTP 500.
Fix: Gate the two ID None-checks on finish_reason in _GEN_PENDING_FINISH_REASONS (shared constant also used by _need_gen); update the broken unit test to use a GEN-pending finish_reason and add positive tests for completed responses.
Automated fix generated by repair-bot

Test plan

Verify fix on the same GPU type as the original failure
Check for regressions in related tests

Links

Bug: https://nvbugs/6245861

Summary by CodeRabbit

Bug Fixes
- Refined disaggregated service validation logic to correctly handle various context completion scenarios, including cases where context completes early before generation handoff.
Tests
- Extended test coverage for response validation with additional edge case verification in disaggregated service operations.

coderabbitai · 2026-06-03T13:33:59Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 7359ebf9-149f-4205-bbdd-7a563bde4e8f

📥 Commits

Reviewing files that changed from the base of the PR and between e5b8094 and 7685d21.

📒 Files selected for processing (2)

tensorrt_llm/serve/openai_disagg_service.py
tests/unittest/disaggregated/test_openai_disagg_service.py

📝 Walkthrough

Walkthrough

The PR refactors CTX finish-reason handling in the OpenAI disaggregated service by centralizing which finish reasons indicate a pending GEN handoff into a module constant, then updates both the GEN decision logic and response validation to use this constant. Response validation now conditionally enforces disaggregated parameter IDs only during pending GEN transitions, allowing early CTX completions to omit these fields.

Changes

Disaggregated Service Finish-Reason Conditional Validation

Layer / File(s)	Summary
Extract finish-reason constant and update conditional logic `tensorrt_llm/serve/openai_disagg_service.py`	Introduces `_GEN_PENDING_FINISH_REASONS` constant to represent pending GEN handoff reasons ("length", "not_finished"), updates `_need_gen` to use this constant instead of an inline list, and refactors `_verify_ctx_response` to conditionally validate `ctx_request_id` and `disagg_request_id` only when finish_reason is a pending GEN reason.
Test coverage for conditional response validation `tests/unittest/disaggregated/test_openai_disagg_service.py`	Adjusts `test_missing_disagg_request_id_includes_ctx_id` to use `finish_reason="length"`, and adds two new test cases verifying that `_verify_ctx_response` accepts responses where `ctx_request_id` or `disagg_request_id` is `None` during early CTX completion scenarios.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically describes the main change: gating ID validation checks on a finish_reason condition, which directly addresses the fix for the reported bug.
Description check	✅ Passed	The PR description includes a clear summary of root cause and fix, test plan confirmation, and relevant bug link, though it lacks explicit sections matching the template structure.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

reasonsolo · 2026-06-04T08:18:37Z

This is exactly how I was planning to fix the bug. 💯

reasonsolo · 2026-06-04T08:18:53Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-04T08:24:31Z

PR_Github #52019 [ run ] triggered by Bot. Commit: 7685d21 Link to invocation

tensorrt-cicd · 2026-06-04T18:27:28Z

PR_Github #52019 [ run ] completed with state SUCCESS. Commit: 7685d21
/LLM/main/L0_MergeRequest_PR pipeline #41360 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

reasonsolo · 2026-06-05T00:41:10Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-05T00:47:37Z

PR_Github #52192 [ run ] triggered by Bot. Commit: 7685d21 Link to invocation

tensorrt-cicd · 2026-06-05T07:27:38Z

PR_Github #52192 [ run ] completed with state SUCCESS. Commit: 7685d21
/LLM/main/L0_MergeRequest_PR pipeline #41512 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

reasonsolo · 2026-06-05T07:39:21Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-05T07:45:30Z

PR_Github #52297 [ run ] triggered by Bot. Commit: 7685d21 Link to invocation

tensorrt-cicd · 2026-06-05T12:14:22Z

PR_Github #52297 [ run ] completed with state FAILURE. Commit: 7685d21
/LLM/main/L0_MergeRequest_PR pipeline #41605 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

reasonsolo · 2026-06-05T12:17:02Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-05T12:22:41Z

PR_Github #52338 [ run ] triggered by Bot. Commit: 7685d21 Link to invocation

tensorrt-cicd · 2026-06-05T16:56:23Z

PR_Github #52338 [ run ] completed with state FAILURE. Commit: 7685d21
/LLM/main/L0_MergeRequest_PR pipeline #41643 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

reasonsolo · 2026-06-08T06:27:27Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-08T06:33:23Z

PR_Github #52678 [ run ] triggered by Bot. Commit: 7685d21 Link to invocation

tensorrt-cicd · 2026-06-08T15:24:31Z

PR_Github #52678 [ run ] completed with state FAILURE. Commit: 7685d21
/LLM/main/L0_MergeRequest_PR pipeline #41946 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

reasonsolo · 2026-06-09T02:39:14Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-09T02:45:06Z

PR_Github #52925 [ run ] triggered by Bot. Commit: 7685d21 Link to invocation

tensorrt-cicd · 2026-06-09T11:30:32Z

PR_Github #52925 [ run ] completed with state FAILURE. Commit: 7685d21
/LLM/main/L0_MergeRequest_PR pipeline #42170 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

reasonsolo · 2026-06-09T11:39:20Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-09T11:45:03Z

PR_Github #53056 [ run ] triggered by Bot. Commit: 7685d21 Link to invocation

tensorrt-cicd · 2026-06-09T18:09:02Z

PR_Github #53056 [ run ] completed with state FAILURE. Commit: 7685d21
/LLM/main/L0_MergeRequest_PR pipeline #42270 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

reasonsolo · 2026-06-10T00:48:43Z

/bot run --disable-fail-fast

reasonsolo · 2026-06-11T05:51:33Z

/bot run --disable-fail-fast

reasonsolo · 2026-06-15T00:15:16Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-15T00:21:21Z

PR_Github #54163 [ run ] triggered by Bot. Commit: 3316698 Link to invocation

reasonsolo · 2026-06-15T08:54:00Z

/bot run --disable-fail-fast

reasonsolo · 2026-06-15T09:45:22Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-15T09:51:02Z

PR_Github #54275 [ run ] triggered by Bot. Commit: 2023d8c Link to invocation

tensorrt-cicd · 2026-06-15T09:55:58Z

PR_Github #54163 [ run ] completed with state ABORTED. Commit: 3316698

Link to invocation

tensorrt-cicd · 2026-06-15T16:41:32Z

PR_Github #54275 [ run ] completed with state SUCCESS. Commit: 2023d8c
/LLM/main/L0_MergeRequest_PR pipeline #43348 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

reasonsolo · 2026-06-16T01:15:04Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-16T01:23:27Z

PR_Github #54394 [ run ] triggered by Bot. Commit: 2023d8c Link to invocation

tensorrt-cicd · 2026-06-16T04:02:28Z

PR_Github #54394 [ run ] completed with state FAILURE. Commit: 2023d8c
/LLM/main/L0_MergeRequest_PR pipeline #43462 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

…ctx_request_id A CTX request that finishes early (finish_reason='stop', e.g. EOS during prefill) never sets up the KV-cache handoff, so ctx_request_id and disagg_request_id stay None. _verify_ctx_response ran before _need_gen and unconditionally required both IDs, turning a completed request into HTTP 500. Gate the ID None-checks on finish_reason in _GEN_PENDING_FINISH_REASONS, mirroring _need_gen which already skips the handoff for such responses. Signed-off-by: tensorrt-cicd <90828364+tensorrt-cicd@users.noreply.github.com>

reasonsolo · 2026-06-16T04:05:17Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-16T04:11:22Z

PR_Github #54454 [ run ] triggered by Bot. Commit: c1ff26d Link to invocation

tensorrt-cicd · 2026-06-16T04:21:54Z

PR_Github #54454 [ run ] completed with state FAILURE. Commit: c1ff26d
/LLM/main/L0_MergeRequest_PR pipeline #43518 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

reasonsolo · 2026-06-16T04:50:15Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-16T04:56:41Z

PR_Github #54463 [ run ] triggered by Bot. Commit: c1ff26d Link to invocation

tensorrt-cicd · 2026-06-16T11:43:51Z

PR_Github #54463 [ run ] completed with state SUCCESS. Commit: c1ff26d
/LLM/main/L0_MergeRequest_PR pipeline #43527 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

reasonsolo · 2026-06-16T12:05:01Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-16T12:11:21Z

PR_Github #54581 [ run ] triggered by Bot. Commit: c1ff26d Link to invocation

tensorrt-cicd · 2026-06-16T14:31:46Z

PR_Github #54581 [ run ] completed with state SUCCESS. Commit: c1ff26d
/LLM/main/L0_MergeRequest_PR pipeline #43624 completed with status: 'SUCCESS'

CI Report

Link to invocation

…reason in _GEN_PENDING_FINISH_REASONS`… (NVIDIA#14908) Signed-off-by: tensorrt-cicd <90828364+tensorrt-cicd@users.noreply.github.com> Signed-off-by: GitLab CI Bot <gitlab-ci@nvidia.com>

tensorrt-cicd requested a review from a team as a code owner June 3, 2026 13:30

tensorrt-cicd requested a review from Superjomn June 3, 2026 13:30

tensorrt-cicd assigned reasonsolo Jun 3, 2026

github-actions Bot assigned tensorrt-cicd Jun 3, 2026

reasonsolo enabled auto-merge (squash) June 4, 2026 08:18

venkywonka mentioned this pull request Jun 4, 2026

[None][test] Waive qwen3_30b_a3b_fp8 pd_disagg mm-encoder test on single-GPU (NIXL setup failure) #14968

Open

reasonsolo requested a review from JunyiXu-nv June 11, 2026 05:52

JunyiXu-nv approved these changes Jun 11, 2026

View reviewed changes

tensorrt-cicd force-pushed the repair-bot-bug6245861 branch from 7685d21 to 3316698 Compare June 12, 2026 23:57

reasonsolo force-pushed the repair-bot-bug6245861 branch from 3316698 to 2023d8c Compare June 15, 2026 08:53

reasonsolo force-pushed the repair-bot-bug6245861 branch from 2023d8c to c1ff26d Compare June 16, 2026 04:05

reasonsolo merged commit f3b718a into NVIDIA:main Jun 16, 2026
7 checks passed

peihu-nv mentioned this pull request Jul 16, 2026

[https://nvbugs/6120535][chore] Unwaive DeepSeek V3.2 disaggregated serving test #16515

Merged

1 task

Uh oh!

Conversation

tensorrt-cicd commented Jun 3, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Links

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jun 3, 2026

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning)

Uh oh!

reasonsolo commented Jun 4, 2026

Uh oh!

reasonsolo commented Jun 4, 2026

Uh oh!

tensorrt-cicd commented Jun 4, 2026

Uh oh!

tensorrt-cicd commented Jun 4, 2026

Uh oh!

reasonsolo commented Jun 5, 2026

Uh oh!

tensorrt-cicd commented Jun 5, 2026

Uh oh!

tensorrt-cicd commented Jun 5, 2026

Uh oh!

reasonsolo commented Jun 5, 2026

Uh oh!

tensorrt-cicd commented Jun 5, 2026

Uh oh!

tensorrt-cicd commented Jun 5, 2026

Uh oh!

reasonsolo commented Jun 5, 2026

Uh oh!

tensorrt-cicd commented Jun 5, 2026

Uh oh!

tensorrt-cicd commented Jun 5, 2026

Uh oh!

reasonsolo commented Jun 8, 2026

Uh oh!

tensorrt-cicd commented Jun 8, 2026

Uh oh!

tensorrt-cicd commented Jun 8, 2026

Uh oh!

reasonsolo commented Jun 9, 2026

Uh oh!

tensorrt-cicd commented Jun 9, 2026

Uh oh!

tensorrt-cicd commented Jun 9, 2026

Uh oh!

reasonsolo commented Jun 9, 2026

Uh oh!

tensorrt-cicd commented Jun 9, 2026

Uh oh!

tensorrt-cicd commented Jun 9, 2026

Uh oh!

reasonsolo commented Jun 10, 2026

Uh oh!

reasonsolo commented Jun 11, 2026

Uh oh!

reasonsolo commented Jun 15, 2026

Uh oh!

tensorrt-cicd commented Jun 15, 2026

Uh oh!

reasonsolo commented Jun 15, 2026

Uh oh!

reasonsolo commented Jun 15, 2026

Uh oh!

tensorrt-cicd commented Jun 15, 2026

Uh oh!

tensorrt-cicd commented Jun 15, 2026

Uh oh!

tensorrt-cicd commented Jun 15, 2026

Uh oh!

reasonsolo commented Jun 16, 2026

Uh oh!

tensorrt-cicd commented Jun 16, 2026

tensorrt-cicd commented Jun 3, 2026 •

edited by coderabbitai Bot

Loading