Skip to content

[nvbug/5337601][fix] Fix disagg + speculative decoding - #5558

Merged
pcastonguay merged 2 commits into
NVIDIA:release/0.21from
Tabrizian:pr-5525
Jul 4, 2025
Merged

[nvbug/5337601][fix] Fix disagg + speculative decoding#5558
pcastonguay merged 2 commits into
NVIDIA:release/0.21from
Tabrizian:pr-5525

Conversation

@Tabrizian

@Tabrizian Tabrizian commented Jun 27, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

[nvbug/5337601][fix] Fix disagg + speculative decoding

@Tabrizian
Tabrizian requested a review from a team as a code owner June 27, 2025 23:09
@Tabrizian
Tabrizian requested a review from lfr-0531 June 27, 2025 23:09
QiJune
QiJune reviewed Jun 28, 2025
View reviewed changes
Comment thread tensorrt_llm/_torch/pyexecutor/py_executor.py
@Tabrizian
Tabrizian changed the base branch from main to release/0.21 June 30, 2025 16:42
@Tabrizian
Tabrizian requested a review from a team as a code owner June 30, 2025 16:42
@Tabrizian
Tabrizian changed the base branch from release/0.21 to main July 1, 2025 17:17
@Tabrizian
Tabrizian changed the base branch from main to release/0.21 July 1, 2025 17:23
@Tabrizian
Tabrizian requested a review from IzzyPutterman July 1, 2025 17:24
@Tabrizian

Tabrizian commented Jul 1, 2025

Copy link
Copy Markdown
Member Author

/bot run

@tensorrt-cicd

tensorrt-cicd commented Jul 1, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #10520 [ run ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented Jul 1, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #10520 [ run ] completed with state FAILURE
/LLM/release-0.21/L0_MergeRequest_PR pipeline #93 completed with status: 'FAILURE'

@Tabrizian
Tabrizian force-pushed the pr-5525 branch 2 times, most recently from d6e4460 to daf47ab Compare July 1, 2025 18:28
@Tabrizian

Tabrizian commented Jul 1, 2025

Copy link
Copy Markdown
Member Author

/bot run

@tensorrt-cicd

tensorrt-cicd commented Jul 1, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #10525 [ run ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented Jul 1, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #10525 [ run ] completed with state FAILURE
/LLM/release-0.21/L0_MergeRequest_PR pipeline #95 completed with status: 'FAILURE'

@Tabrizian

Tabrizian commented Jul 1, 2025

Copy link
Copy Markdown
Member Author

/bot run

@tensorrt-cicd

tensorrt-cicd commented Jul 1, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #10528 [ run ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented Jul 1, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #10528 [ run ] completed with state SUCCESS
/LLM/release-0.21/L0_MergeRequest_PR pipeline #96 completed with status: 'FAILURE'

@Tabrizian

Tabrizian commented Jul 1, 2025

Copy link
Copy Markdown
Member Author

/bot run

@tensorrt-cicd

tensorrt-cicd commented Jul 1, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #10535 [ run ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented Jul 2, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #10694 [ run ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented Jul 2, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #10689 [ run ] completed with state ABORTED

@tensorrt-cicd

tensorrt-cicd commented Jul 2, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #10694 [ run ] completed with state FAILURE
/LLM/release-0.21/L0_MergeRequest_PR pipeline #126 completed with status: 'FAILURE'

@Tabrizian

Tabrizian commented Jul 2, 2025

Copy link
Copy Markdown
Member Author

/bot run --add-multi-gpu-test

@tensorrt-cicd

tensorrt-cicd commented Jul 2, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #10707 [ run ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented Jul 2, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #10707 [ run ] completed with state FAILURE
/LLM/release-0.21/L0_MergeRequest_PR pipeline #129 completed with status: 'FAILURE'

@Tabrizian

Tabrizian commented Jul 2, 2025

Copy link
Copy Markdown
Member Author

/bot run --add-multi-gpu-test

@tensorrt-cicd

tensorrt-cicd commented Jul 2, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #10713 [ run ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented Jul 2, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #10713 [ run ] completed with state SUCCESS
/LLM/release-0.21/L0_MergeRequest_PR pipeline #134 completed with status: 'FAILURE'

mikeiovine and others added 2 commits July 3, 2025 09:05
@mikeiovine @Tabrizian
[nvbug/5337601][fix] Fix disagg + speculative decoding
fc5b559 
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
@Tabrizian
Minor edit
a628337 
Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>

Fix accuracy for disagg + eagle

Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>
@Tabrizian

Tabrizian commented Jul 3, 2025

Copy link
Copy Markdown
Member Author

/bot run --add-multi-gpu-test

@tensorrt-cicd

tensorrt-cicd commented Jul 3, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #10852 [ run ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented Jul 3, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #10852 [ run ] completed with state SUCCESS
/LLM/release-0.21/L0_MergeRequest_PR pipeline #151 completed with status: 'ABORTED'

@Tabrizian

Tabrizian commented Jul 3, 2025

Copy link
Copy Markdown
Member Author

/bot run --add-multi-gpu-test

@tensorrt-cicd

tensorrt-cicd commented Jul 3, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #10869 [ run ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented Jul 4, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #10869 [ run ] completed with state SUCCESS
/LLM/release-0.21/L0_MergeRequest_PR pipeline #152 completed with status: 'SUCCESS'
Pipeline passed with automatic retried tests. Check the rerun report for details.

lfr-0531
lfr-0531 reviewed Jul 4, 2025
View reviewed changes
Comment thread tensorrt_llm/_torch/pyexecutor/py_executor.py
@pcastonguay
pcastonguay merged commit 518915b into NVIDIA:release/0.21 Jul 4, 2025
3 checks passed
@pcastonguay pcastonguay mentioned this pull request Jul 4, 2025
Merged
dc3671 pushed a commit to dc3671/TensorRT-LLM that referenced this pull request Jul 10, 2025
@Tabrizian @mikeiovine @dc3671
[nvbug/5337601][fix] Fix disagg + speculative decoding (NVIDIA#5558)
09a4359 
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>
Co-authored-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
dc3671 pushed a commit to dc3671/TensorRT-LLM that referenced this pull request Jul 10, 2025
@Tabrizian @mikeiovine @dc3671
[nvbug/5337601][fix] Fix disagg + speculative decoding (NVIDIA#5558)
7bdbe12 
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>
Co-authored-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
dc3671 pushed a commit to dc3671/TensorRT-LLM that referenced this pull request Jul 11, 2025
@Tabrizian @mikeiovine @dc3671
[nvbug/5337601][fix] Fix disagg + speculative decoding (NVIDIA#5558)
8684cb0 
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>
Co-authored-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
dc3671 pushed a commit to dc3671/TensorRT-LLM that referenced this pull request Jul 14, 2025
@Tabrizian @mikeiovine @dc3671
[nvbug/5337601][fix] Fix disagg + speculative decoding (NVIDIA#5558)
81a4b33 
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>
Co-authored-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
dc3671 pushed a commit to dc3671/TensorRT-LLM that referenced this pull request Jul 14, 2025
@Tabrizian @mikeiovine @dc3671
[nvbug/5337601][fix] Fix disagg + speculative decoding (NVIDIA#5558)
27c9b04 
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>
Co-authored-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
dc3671 pushed a commit to dc3671/TensorRT-LLM that referenced this pull request Jul 14, 2025
@Tabrizian @mikeiovine @dc3671
[nvbug/5337601][fix] Fix disagg + speculative decoding (NVIDIA#5558)
4b6af20 
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>
Co-authored-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
dc3671 pushed a commit that referenced this pull request Jul 14, 2025
@Tabrizian @mikeiovine @dc3671
[nvbug/5337601][fix] Fix disagg + speculative decoding (#5558)
c8874a7 
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>
Co-authored-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
@jhaotingc jhaotingc mentioned this pull request Jul 22, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@QiJune QiJune QiJune left review comments
@lfr-0531 lfr-0531 lfr-0531 left review comments
@pcastonguay pcastonguay pcastonguay approved these changes
@MartinMarciniszyn MartinMarciniszyn MartinMarciniszyn approved these changes
@IzzyPutterman IzzyPutterman Awaiting requested review from IzzyPutterman
@schetlur-nv schetlur-nv Awaiting requested review from schetlur-nv
+1 more reviewer
@raayandhar raayandhar raayandhar left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@Tabrizian @tensorrt-cicd @MartinMarciniszyn @QiJune @lfr-0531 @pcastonguay @raayandhar @mikeiovine