Skip to content

[AutoDeploy][1/n] Modular InferenceOptimizer - #84

Merged
lucaslie merged 1 commit into
feat/ad-2025-07-07from
ll/base_transformation
Jul 15, 2025
Merged

[AutoDeploy][1/n] Modular InferenceOptimizer#84
lucaslie merged 1 commit into
feat/ad-2025-07-07from
ll/base_transformation

Conversation

@lucaslie

Copy link
Copy Markdown

The interface and initial transforms in the new modular InferenceOptimizer design

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Introduces a modular inference optimizer pipeline by defining a high-level InferenceOptimizer class and migrating existing export and cleanup logic into discrete, configurable transforms.

  • Added a filtered_nodes utility to simplify node filtering by op type.
  • Replaced the legacy export flow in transform.py with the new ModularInferenceOptimizer.
  • Extracted input-constraint cleanup into its own transform and cleaned up the export code accordingly.

Reviewed Changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tensorrt_llm/_torch/auto_deploy/utils/node_utils.py Added OpOrOverload alias and new filtered_nodes utility.
tensorrt_llm/_torch/auto_deploy/transformations/transform.py Switched to ModularInferenceOptimizer, removed legacy export path.
tensorrt_llm/_torch/auto_deploy/transformations/export.py Removed obsolete functions and relocated aliased-param load hook.
tensorrt_llm/_torch/auto_deploy/transform/optimizer.py Introduced InferenceOptimizer entrypoint and transform loop.
tensorrt_llm/_torch/auto_deploy/transform/library/*.py Added new wrapper transforms (export_to_gm, build_model, cleanup_input_constraints).
tensorrt_llm/_torch/auto_deploy/llm_args.py Added transforms config for the new optimizer.
tensorrt_llm/_torch/auto_deploy/config/default.yaml Populated default transform configurations.
Comments suppressed due to low confidence (1)

tensorrt_llm/_torch/auto_deploy/utils/node_utils.py:219

  • [nitpick] Consider adding unit tests for filtered_nodes to validate that it correctly yields nodes matching single and multiple ops, and skips non-matching ones.
def filtered_nodes(

Comment thread tensorrt_llm/_torch/auto_deploy/transformations/export.py
Comment thread tensorrt_llm/_torch/auto_deploy/transformations/transform.py
@lucaslie
lucaslie force-pushed the ll/base_transformation branch from e7505a3 to 62e9dd8 Compare July 14, 2025 18:33
@lucaslie
lucaslie force-pushed the ll/config_deepmerging branch from f0e9c58 to 1bb0090 Compare July 14, 2025 18:35
Comment thread tensorrt_llm/_torch/auto_deploy/transform/interface.py Outdated
@lucaslie
lucaslie force-pushed the ll/config_deepmerging branch from 1bb0090 to f4cd863 Compare July 15, 2025 19:16
Base automatically changed from ll/config_deepmerging to feat/ad-2025-07-07 July 15, 2025 19:25
@lucaslie
lucaslie force-pushed the ll/base_transformation branch from 62e9dd8 to 174147b Compare July 15, 2025 19:27
Comment thread tensorrt_llm/_torch/auto_deploy/transform/interface.py
Comment thread tensorrt_llm/_torch/auto_deploy/transform/interface.py Outdated
Comment thread tensorrt_llm/_torch/auto_deploy/transform/interface.py Outdated
Comment thread tensorrt_llm/_torch/auto_deploy/transform/interface.py Outdated
Comment thread tensorrt_llm/_torch/auto_deploy/transform/interface.py
Comment thread tensorrt_llm/_torch/auto_deploy/transform/library/build_model.py
Comment thread tensorrt_llm/_torch/auto_deploy/transform/interface.py
Comment thread tensorrt_llm/_torch/auto_deploy/transform/optimizer.py Outdated
Comment thread tensorrt_llm/_torch/auto_deploy/transformations/transform.py
Comment thread tensorrt_llm/_torch/auto_deploy/transform/optimizer.py Outdated
Comment thread tensorrt_llm/_torch/auto_deploy/transform/library/build_model.py
Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>
@lucaslie
lucaslie force-pushed the ll/base_transformation branch from 174147b to 909481e Compare July 15, 2025 21:29
@lucaslie
lucaslie enabled auto-merge (squash) July 15, 2025 21:31
@lucaslie
lucaslie disabled auto-merge July 15, 2025 21:31
@lucaslie
lucaslie enabled auto-merge (squash) July 15, 2025 21:32
@lucaslie
lucaslie merged commit 59da6be into feat/ad-2025-07-07 Jul 15, 2025
1 check passed
@lucaslie
lucaslie deleted the ll/base_transformation branch July 15, 2025 21:35
lucaslie added a commit that referenced this pull request Jul 18, 2025
…IA#4367, NVIDIA#4366  (#84)

Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>
lucaslie added a commit that referenced this pull request Jul 21, 2025
…IA#4367, NVIDIA#4366  (#84)

Signed-off-by: Lucas Liebenwein <11156568+lucaslie@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants