[https://nvbugs/6428057][fix] In _ring_broadcast_sample_state, forward use_host_stop_criteria alongside… - #16130
Open
trtllm-agent wants to merge 2 commits into
Open
[https://nvbugs/6428057][fix] In _ring_broadcast_sample_state, forward use_host_stop_criteria alongside…#16130trtllm-agent wants to merge 2 commits into
_ring_broadcast_sample_state, forward use_host_stop_criteria alongside…#16130trtllm-agent wants to merge 2 commits into
Conversation
Contributor
|
Caution Review failedAn error occurred during the review process. Please try again later. 📝 WalkthroughWalkthroughThe PP ring broadcast of sample state in PyExecutor is updated to propagate an additional ChangesRing broadcast payload update
Estimated code review effort: 1 (Trivial) | ~5 minutes ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
The last PP rank sets state.use_host_stop_criteria=True when it evaluates end-ID/length limits on the host, which suppresses population of finish_reasons_host on the sample state. Prior to this fix, only sample_state.host was ring-broadcasted to non-last PP ranks, so those ranks re-entered the process_draft_tokens branch in update_requests (their state.use_host_stop_criteria stayed at the default False) and tried to index into an empty finish_reasons list, raising IndexError: list index out of range. Forward the flag alongside .host so non-last PP ranks take the same host stop-criteria branch. The receiver accepts both the new 3-tuple form and the legacy 2-tuple form to keep upgrades safe. Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>
trtllm-agent
force-pushed
the
repair-bot-bug6428057
branch
from
July 8, 2026 20:43
40d159e to
8386a90
Compare
Contributor
|
Caution Failed to replace (edit) comment. This is likely due to insufficient permissions or the comment being deleted. Error details |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
use_host_stop_criterialives onSampleStateTorch(not on.host) and is not propagated across PP ranks; non-last-pp ranks default to False and enterprocess_draft_tokens, but the last rank leftfinish_reasons_host=None, so indexing raises IndexError._ring_broadcast_sample_state, forwarduse_host_stop_criteriaalongsidesample_state.hostin a 3-tuple; receiver restores it on the localsample_state(also tolerates the legacy 2-tuple form).Test plan
Links
Summary by CodeRabbit