[https://nvbugs/6445456][fix] Restore inplace ops for functionalization v2 by liji-nv · Pull Request #16410 · NVIDIA/TensorRT-LLM

liji-nv · 2026-07-15T07:45:09Z

Summary by CodeRabbit

Bug Fixes
- Improved handling of in-place operations involving functionalized tensor mutations.
- Restored optional empty outputs correctly during graph processing.
- Preserved tensor aliases and operation arguments more reliably.
Tests
- Added coverage for functionalized in-place operations and tensor cloning.
- Updated integration test waivers for revised DeepSeek V3 Lite test configurations.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

coderabbitai · 2026-07-15T07:50:39Z

📝 Walkthrough

Walkthrough

The copy-removal pass now reconstructs mutation arguments for auto_functionalized_v2 using alias metadata and base indices. New unit tests validate graph rewrites and optional outputs, while DeepSeekV3Lite integration waivers are updated.

Changes

Functionalized mutation handling

Layer / File(s)	Summary
Reconstruct v2 mutation arguments `tensorrt_llm/_torch/compilation/remove_copy_pass.py`	The pass uses v2 alias-write metadata and `_all_bases` indices to rebuild mutation kwargs, with positional fallback handling.
Validate rewritten graphs `tests/unittest/_torch/compilation/test_remove_copy_pass.py`	Tests validate `copy_` rewrites, output-base cloning, removal of `auto_functionalized_v2`, optional `None` preservation, and graph linting.

DeepSeekV3Lite integration waivers

Layer / File(s)	Summary
Update parameterized waivers `tests/integration/test_lists/waives.txt`	Updates bfloat16, nvfp4, and nvfp4_4gpus waiver combinations and associated nvbug references, removing obsolete fp8 block-scale entries.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant FXGraph
  participant CopyRemovalPass
  participant FunctionalizeInner
  participant InplaceOperation
  FXGraph->>CopyRemovalPass: process auto_functionalized_v2 node
  CopyRemovalPass->>FunctionalizeInner: invoke with is_v2=true
  FunctionalizeInner->>FunctionalizeInner: resolve alias writes from _all_bases
  FunctionalizeInner->>InplaceOperation: rebuild mutation kwargs
  CopyRemovalPass->>FXGraph: remove wrapper and rewire clone inputs

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check	⚠️ Warning	The description is still the template and does not explain the issue, solution, or test coverage.	Add a brief issue/solution summary and list the tests that cover the v2 functionalization fix and waiver updates.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title matches the main change and follows the required NVBug/type format.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

mzweilz

Approved for the waive list

liji-nv · 2026-07-15T09:48:12Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-15T09:54:17Z

PR_Github #59431 [ run ] triggered by Bot. Commit: d4bf4eb Link to invocation

tensorrt-cicd · 2026-07-15T18:23:47Z

PR_Github #59431 [ run ] completed with state SUCCESS. Commit: d4bf4eb
/LLM/main/L0_MergeRequest_PR pipeline #47900 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

liji-nv · 2026-07-16T02:27:26Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-16T02:32:50Z

PR_Github #59585 [ run ] triggered by Bot. Commit: 0231186 Link to invocation

liji-nv · 2026-07-16T12:52:35Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-16T12:58:37Z

PR_Github #59709 [ run ] triggered by Bot. Commit: d2de168 Link to invocation

tensorrt-cicd · 2026-07-16T13:04:14Z

PR_Github #59585 [ run ] completed with state ABORTED. Commit: 0231186

Link to invocation

tensorrt-cicd · 2026-07-16T17:19:18Z

PR_Github #59709 [ run ] completed with state SUCCESS. Commit: d2de168
/LLM/main/L0_MergeRequest_PR pipeline #48136 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

tensorrt-cicd · 2026-07-20T08:17:28Z

PR_Github #60323 [ run ] triggered by Bot. Commit: 5ae7af9 Link to invocation

tensorrt-cicd · 2026-07-20T11:56:40Z

PR_Github #60323 [ run ] completed with state SUCCESS. Commit: 5ae7af9
/LLM/main/L0_MergeRequest_PR pipeline #48668 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

liji-nv · 2026-07-21T02:43:18Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-21T02:52:56Z

PR_Github #60545 [ run ] triggered by Bot. Commit: 5ae7af9 Link to invocation

tensorrt-cicd · 2026-07-21T05:13:42Z

PR_Github #60545 [ run ] completed with state SUCCESS. Commit: 5ae7af9
/LLM/main/L0_MergeRequest_PR pipeline #48860 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

liji-nv · 2026-07-21T05:25:41Z

/bot run --disable-fail-fast --add-multi-gpu-test

tensorrt-cicd · 2026-07-21T05:31:26Z

PR_Github #60589 [ run ] triggered by Bot. Commit: 5ae7af9 Link to invocation

tensorrt-cicd · 2026-07-21T07:09:55Z

PR_Github #60589 [ run ] completed with state FAILURE. Commit: 5ae7af9
/LLM/main/L0_MergeRequest_PR pipeline #48900 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

liji-nv · 2026-07-21T12:33:51Z

/bot run --disable-fail-fast --add-multi-gpu-test

tensorrt-cicd · 2026-07-21T12:50:20Z

PR_Github #60693 [ run ] triggered by Bot. Commit: 5ae7af9 Link to invocation

tensorrt-cicd · 2026-07-21T14:03:44Z

PR_Github #60693 [ run ] completed with state FAILURE. Commit: 5ae7af9
/LLM/main/L0_MergeRequest_PR pipeline #48987 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Signed-off-by: Jin Li <59594262+liji-nv@users.noreply.github.com>

liji-nv · 2026-07-22T02:42:05Z

/bot run --disable-fail-fast --add-multi-gpu-test

tensorrt-cicd · 2026-07-22T02:48:52Z

PR_Github #60858 [ run ] triggered by Bot. Commit: 2fc83e0 Link to invocation

tensorrt-cicd · 2026-07-22T07:14:49Z

PR_Github #60858 [ run ] completed with state FAILURE. Commit: 2fc83e0
/LLM/main/L0_MergeRequest_PR pipeline #49132 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

liji-nv · 2026-07-22T07:15:08Z

/bot run --disable-fail-fast --add-multi-gpu-test

liji-nv · 2026-07-22T07:25:23Z

/bot run --disable-fail-fast --add-multi-gpu-test

tensorrt-cicd · 2026-07-22T07:31:21Z

PR_Github #60917 [ run ] triggered by Bot. Commit: 2fc83e0 Link to invocation

tensorrt-cicd · 2026-07-22T12:35:35Z

PR_Github #60917 [ run ] completed with state FAILURE. Commit: 2fc83e0
/LLM/main/L0_MergeRequest_PR pipeline #49185 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

liji-nv · 2026-07-23T03:02:08Z

/bot run --disable-fail-fast --add-multi-gpu-test

tensorrt-cicd · 2026-07-23T03:08:33Z

PR_Github #61172 [ run ] triggered by Bot. Commit: 2fc83e0 Link to invocation

tensorrt-cicd · 2026-07-23T04:51:28Z

PR_Github #61172 [ run ] completed with state SUCCESS. Commit: 2fc83e0
/LLM/main/L0_MergeRequest_PR pipeline #49424 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

liji-nv · 2026-07-23T08:03:26Z

/bot run --disable-fail-fast --add-multi-gpu-test

tensorrt-cicd · 2026-07-23T08:09:19Z

PR_Github #61244 [ run ] triggered by Bot. Commit: 2fc83e0 Link to invocation

tensorrt-cicd · 2026-07-23T08:56:31Z

PR_Github #61244 [ run ] completed with state SUCCESS. Commit: 2fc83e0
/LLM/main/L0_MergeRequest_PR pipeline #49484 completed with status: 'SUCCESS'

CI Report

Link to invocation

…nnahz/dep-1083-port-flashinfer-stable-va-lifecycle-for-native-all-reduce * 'main' of https://github.com/NVIDIA/TensorRT-LLM: (83 commits) [None][feat] Bind SourceIdentity to checkpoint artifacts (NVIDIA#16159) [None][infra] Waive 4 failed cases for main in pre-merge 49550 (NVIDIA#16798) [None][fix] Make FlashInfer sampling op wrappers opaque to Dynamo (NVIDIA#16732) [None][feat] top-k: route decode to CuTe DSL GVR top-k in e2e (NVIDIA#16420) [None][feat] Default GLM-5 to the Python KV-cache transceiver (NVIDIA#16524) [None][chore] Add NVTX ranges to per-iteration ADP sync points in PyExecutor (NVIDIA#16422) [https://nvbugs/6426850][test] Unwaive Qwen3.5 397B NVFP4 ADP4 TRTLLM test (NVIDIA#16348) [https://nvbugs/6445456][fix] Restore inplace ops for functionalization v2 (NVIDIA#16410) [None][infra] Waive 1 failed cases for main in pre-merge 49229 (NVIDIA#16786) [None][fix] Load DeepSeek V4 mixed-precision NVFP4 checkpoints (NVIDIA#16433) [None][feat] ADP conversation router: configurable least-queued placement for new conversations (NVIDIA#16294) [None][infra] Waive 1 failed cases for main in pre-merge 49424 (NVIDIA#16780) [None][infra] Waive 1 failed cases for main in pre-merge 49424 (NVIDIA#16781) [TRTLLMINF-188][infra] Require approval for PerfSanity wildcard runs (NVIDIA#16777) [TRTLLM-13969][feat] Support MiniMax M3 for Disaggregated Serving (NVIDIA#16017) [NVIDIA#11932][fix] Filter CUTLASS MoE GEMM tile configs by device shared memory on SM121 (NVIDIA#12704) [None][chore] Remove attention backend test waivers (NVIDIA#16723) [TRTLLM-14540][perf] Skip fp32 state round-trip in FlashInfer GDN pre… (NVIDIA#16703) [TRTLLM-13694][feat] Add IBDB recipe provenance and refresh configs (NVIDIA#16254) [None][infra] Preview/bump/main (NVIDIA#16758) ... Signed-off-by: Hannah Zhang <hannahz@nvidia.com>

…on v2 (NVIDIA#16410) Signed-off-by: Jin Li <59594262+liji-nv@users.noreply.github.com>

liji-nv requested review from a team as code owners July 15, 2026 07:45

liji-nv requested review from YihuiLu512, asfiyab-nvidia, brnguyen2, mzweilz, schetlur-nv, yiqingy0 and yufeiwu-nv July 15, 2026 07:45

github-actions Bot assigned liji-nv Jul 15, 2026

mzweilz approved these changes Jul 15, 2026

View reviewed changes

yiqingy0 removed their request for review July 16, 2026 02:22

yufeiwu-nv removed their request for review July 16, 2026 06:46

brnguyen2 reviewed Jul 16, 2026

View reviewed changes

Comment thread tensorrt_llm/_torch/compilation/remove_copy_pass.py Outdated

brnguyen2 reviewed Jul 16, 2026

View reviewed changes

Comment thread tensorrt_llm/_torch/compilation/remove_copy_pass.py Outdated

brnguyen2 approved these changes Jul 16, 2026

View reviewed changes

nv-lschneider mentioned this pull request Jul 16, 2026

[https://nvbugs/6020038][feat] Add NCCL-EP v0.1 MoE communication support #15597

Merged

1 task

liji-nv force-pushed the liji/nvbug-6445456-auto-functionalized-v2-main branch from f0ba73f to 078cc82 Compare July 17, 2026 06:01

Merge branch 'main' into liji/nvbug-6445456-auto-functionalized-v2-main

2fc83e0

Signed-off-by: Jin Li <59594262+liji-nv@users.noreply.github.com>

liji-nv merged commit e96c330 into NVIDIA:main Jul 23, 2026
7 checks passed

This was referenced Jul 24, 2026

[https://nvbugs/6473161][chore] unwaive 4 TestLlama3_1 tests #16824

Merged

[https://nvbugs/6473161][fix] Detect v2 at the call site (is_v2 = is_call_function(node… #16624

Closed

yuanjingx87 pushed a commit to yuanjingx87/TensorRT-LLM that referenced this pull request Jul 26, 2026

[https://nvbugs/6445456][fix] Restore inplace ops for functionalizati…

f34e053

…on v2 (NVIDIA#16410) Signed-off-by: Jin Li <59594262+liji-nv@users.noreply.github.com>

Uh oh!

Conversation

liji-nv commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

coderabbitai Bot commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

❌ Failed checks (2 warnings)

Uh oh!

mzweilz left a comment

Choose a reason for hiding this comment

Uh oh!

liji-nv commented Jul 15, 2026

Uh oh!

tensorrt-cicd commented Jul 15, 2026

Uh oh!

tensorrt-cicd commented Jul 15, 2026

Uh oh!

liji-nv commented Jul 16, 2026

Uh oh!

tensorrt-cicd commented Jul 16, 2026

Uh oh!

liji-nv commented Jul 16, 2026

Uh oh!

tensorrt-cicd commented Jul 16, 2026

Uh oh!

tensorrt-cicd commented Jul 16, 2026

Uh oh!

Uh oh!

Uh oh!

tensorrt-cicd commented Jul 16, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

liji-nv commented Jul 21, 2026

Uh oh!

tensorrt-cicd commented Jul 21, 2026

Uh oh!

tensorrt-cicd commented Jul 21, 2026

Uh oh!

liji-nv commented Jul 21, 2026

Uh oh!

tensorrt-cicd commented Jul 21, 2026

Uh oh!

tensorrt-cicd commented Jul 21, 2026

Uh oh!

liji-nv commented Jul 21, 2026

Uh oh!

tensorrt-cicd commented Jul 21, 2026

Uh oh!

tensorrt-cicd commented Jul 21, 2026

Uh oh!

liji-nv commented Jul 22, 2026

Uh oh!

tensorrt-cicd commented Jul 22, 2026

Uh oh!

tensorrt-cicd commented Jul 22, 2026

Uh oh!

liji-nv commented Jul 22, 2026

Uh oh!

liji-nv commented Jul 22, 2026

Uh oh!

tensorrt-cicd commented Jul 22, 2026

Uh oh!

tensorrt-cicd commented Jul 22, 2026

Uh oh!

liji-nv commented Jul 23, 2026

Uh oh!

tensorrt-cicd commented Jul 23, 2026

Uh oh!

liji-nv commented Jul 15, 2026 •

edited

Loading

coderabbitai Bot commented Jul 15, 2026 •

edited

Loading