Skip to content

[https://nvbugs/5289907][fix] Restore per-channel pre-quant - #4545

Merged
litaotju merged 3 commits into
NVIDIA:release/0.20from
Barry-Delaney:user/barry/fix_prequant_release
May 23, 2025
Merged

[https://nvbugs/5289907][fix] Restore per-channel pre-quant#4545
litaotju merged 3 commits into
NVIDIA:release/0.20from
Barry-Delaney:user/barry/fix_prequant_release

Conversation

@Barry-Delaney

@Barry-Delaney Barry-Delaney commented May 21, 2025

Copy link
Copy Markdown
Collaborator

This PR restores the per-expert pre-quant scale kernel into the original per-channel one in MoE modules to fix the broken ModelOpt Mixtral-AWQ support.

@Barry-Delaney Barry-Delaney self-assigned this May 21, 2025
@Barry-Delaney
Barry-Delaney requested a review from a team as a code owner May 21, 2025 15:36
@Barry-Delaney

Barry-Delaney commented May 21, 2025

Copy link
Copy Markdown
Collaborator Author

/bot run

@Barry-Delaney
Barry-Delaney requested a review from Tracin May 21, 2025 15:36
@tensorrt-cicd

tensorrt-cicd commented May 21, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #6041 [ run ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented May 21, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #6041 [ run ] completed with state SUCCESS
/LLM/release-0.20/L0_MergeRequest_PR pipeline #23 completed with status: 'FAILURE'

@Barry-Delaney
Barry-Delaney force-pushed the user/barry/fix_prequant_release branch from 12aac5f to 8dee53e Compare May 22, 2025 02:05
@Barry-Delaney

Barry-Delaney commented May 22, 2025

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

tensorrt-cicd commented May 22, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #6082 [ run ] triggered by Bot

@Barry-Delaney

Barry-Delaney commented May 22, 2025

Copy link
Copy Markdown
Collaborator Author

/bot kill

@tensorrt-cicd

tensorrt-cicd commented May 22, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #6085 [ kill ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented May 22, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #6082 [ run ] completed with state ABORTED

@tensorrt-cicd

tensorrt-cicd commented May 22, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #6085 [ kill ] completed with state SUCCESS
Successfully killed previous jobs for commit 8dee53e

@Barry-Delaney

Barry-Delaney commented May 22, 2025

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

tensorrt-cicd commented May 22, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #6114 [ run ] triggered by Bot

@Barry-Delaney

Barry-Delaney commented May 22, 2025

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

tensorrt-cicd commented May 22, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #6116 [ run ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented May 22, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #6114 [ run ] completed with state ABORTED

@Tracin

Tracin commented May 22, 2025

Copy link
Copy Markdown
Collaborator

LGTM

@tensorrt-cicd

tensorrt-cicd commented May 22, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #6116 [ run ] completed with state SUCCESS
/LLM/release-0.20/L0_MergeRequest_PR pipeline #35 completed with status: 'FAILURE'

@Barry-Delaney
Barry-Delaney force-pushed the user/barry/fix_prequant_release branch from f37b3ca to dec4eb5 Compare May 22, 2025 11:41
@Barry-Delaney

Barry-Delaney commented May 22, 2025

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

tensorrt-cicd commented May 22, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #6145 [ run ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented May 22, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #6145 [ run ] completed with state SUCCESS
/LLM/release-0.20/L0_MergeRequest_PR pipeline #39 completed with status: 'FAILURE'

@Barry-Delaney
Barry-Delaney force-pushed the user/barry/fix_prequant_release branch from dec4eb5 to 039f4be Compare May 22, 2025 17:46
@Barry-Delaney

Barry-Delaney commented May 22, 2025

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

tensorrt-cicd commented May 22, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #6171 [ run ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented May 22, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #6171 [ run ] completed with state SUCCESS
/LLM/release-0.20/L0_MergeRequest_PR pipeline #44 completed with status: 'FAILURE'

@Barry-Delaney
Restore per-channel pre-quant
55a63d3 
Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
@Barry-Delaney
Update TRT test script
176368d 
Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
@Barry-Delaney
Fix pre-commit
a970958 
Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
@Barry-Delaney
Barry-Delaney force-pushed the user/barry/fix_prequant_release branch from 039f4be to a970958 Compare May 23, 2025 01:20
@Barry-Delaney

Barry-Delaney commented May 23, 2025

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

tensorrt-cicd commented May 23, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #6194 [ run ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented May 23, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #6194 [ run ] completed with state SUCCESS
/LLM/release-0.20/L0_MergeRequest_PR pipeline #50 completed with status: 'SUCCESS'

@Barry-Delaney
Barry-Delaney requested a review from litaotju May 23, 2025 04:43
@litaotju
litaotju merged commit 26793e3 into NVIDIA:release/0.20 May 23, 2025
@tongyuantongyu tongyuantongyu mentioned this pull request May 28, 2025
Merged
shaharmor98 pushed a commit to shaharmor98/tekit that referenced this pull request May 28, 2025
@Barry-Delaney @shaharmor98
[https://nvbugs/5289907][fix] Restore per-channel pre-quant (NVIDIA#4545
87c4854 
)

* Restore per-channel pre-quant

Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>

* Update TRT test script

Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>

* Fix pre-commit

Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>

---------

Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
omera-nv pushed a commit to omera-nv/TensorRT-LLM that referenced this pull request Jun 3, 2025
@Barry-Delaney @omera-nv
[https://nvbugs/5289907][fix] Restore per-channel pre-quant (NVIDIA#4545
13c007f 
)

* Restore per-channel pre-quant

Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>

* Update TRT test script

Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>

* Fix pre-commit

Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>

---------

Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@litaotju litaotju litaotju approved these changes
@Tracin Tracin Tracin approved these changes

Assignees

@Barry-Delaney Barry-Delaney

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@Barry-Delaney @tensorrt-cicd @Tracin @litaotju