Skip to content

[None][fix] Cherry-pick PR #14276: Handle unset attention_dp_relax in ADP routers - #14431

Merged
lancelly merged 1 commit into
NVIDIA:feat/deepseek_v4from
lancelly:cp-pr14276-feat-deepseek-v4
May 22, 2026
Merged

[None][fix] Cherry-pick PR #14276: Handle unset attention_dp_relax in ADP routers#14431
lancelly merged 1 commit into
NVIDIA:feat/deepseek_v4from
lancelly:cp-pr14276-feat-deepseek-v4

Conversation

@lancelly

@lancelly lancelly commented May 22, 2026

Copy link
Copy Markdown
Collaborator

Summary

Cherry-pick of upstream main PR #14276 (3de344d384) onto the feat/deepseek_v4 development branch.

Why: feat/deepseek_v4 does not yet have PR #14276 (merged on main 2026-05-19). Without it, the feat/deepseek_v4 line crashes at every Phase-transition point where the request batch contains multiple SchedulingParams objects with attention_dp_relax = None:

File ".../_torch/pyexecutor/scheduler/adp_router.py", line 517, in route_requests
    sorted_requests = sorted(new_requests, key=get_relax_value)
TypeError: '<' not supported between instances of 'NoneType' and 'NoneType'

This was reproduced on an agg DSV4-Pro RWLT validation run (2 nodes × GB300, TP8/TEP8/MTP3) at the Phase 0 → Phase 1 transition (concurrency 1 → 4) — all 8 ranks crashed simultaneously on both DefaultADPRouter.route_requests (line 232) and KVCacheAwareADPRouter.route_requests (line 517), since both routers share the same get_relax_value helper that dereferences scheduling_params.attention_dp_relax which can be None.

The upstream fix in PR #14276 changes the root cause — the default value of SchedulingParams.attention_dp_relax:

-    attention_dp_relax: Optional[bool] = None
+    attention_dp_relax: bool = True

so the router's sorted(...) no longer sees None keys. Treating unset = "relaxed" (True) matches the semantics: an unset request has no specific rank preference and can be routed anywhere.

)

Signed-off-by: peihengh <259410613+peihu-nv@users.noreply.github.com>
(cherry picked from commit 3de344d)
Signed-off-by: Lance Liao <108499334+lancelly@users.noreply.github.com>
@lancelly

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #49805 [ run ] triggered by Bot. Commit: c3539a5 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #49805 [ run ] completed with state SUCCESS. Commit: c3539a5
/LLM/main/L0_MergeRequest_PR pipeline #39392 completed with status: 'SUCCESS'

CI Report

Link to invocation

@lancelly
lancelly merged commit 09e8a11 into NVIDIA:feat/deepseek_v4 May 22, 2026
8 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants