Skip to content

[PyTorch] torch.compile for an OperationFuser group holding one operation - #29

Open
pggPL wants to merge 3 commits into
linear_compile_on_mainfrom
ops_fuser_compile
Open

[PyTorch] torch.compile for an OperationFuser group holding one operation#29
pggPL wants to merge 3 commits into
linear_compile_on_mainfrom
ops_fuser_compile

Conversation

@pggPL

@pggPL pggPL commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Description

First step towards torch.compile(fullgraph=True) support for transformer_engine.pytorch.ops.

The approach: keep the pipeline-level _OperationFuserAutogradFunction and let Dynamo trace it as a
higher-order op, with each fusible operation calling its own custom op inside. That keeps the forward
and backward fusion layouts independent (so the backward-only fusions survive), keeps
OperationContext inside the traced scope, and bounds op registration to one entry per op class.

This PR makes that work for an OperationFuser group holding one operation. It converts no real
operation -- the fuser's path is exercised by a test-only operation, so it does not depend on which
operations happen to declare their compute halves. Converting the operations themselves is a follow-up.

Type of change

  • Documentation change (change only to the documentation, either a fix or a new content)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Infra/Build change
  • Code refactoring

Changes

  • register_custom_op now defines an operation's forward and backward as two independent
    two-tier custom ops and hands back both, leaving autograd to the caller. That is what lets a
    pipeline-level autograd.Function decide how the two are wired. The variant that wires autograd
    itself keeps the old behaviour under register_custom_op_with_autograd and is built on the same
    registration -- the pair is the primitive, autograd is what the other one adds. About two thirds of
    the two bodies were the same code before.
  • BasicOperation gains the plumbing an operation needs to opt in: declare two argument containers
    and implement four compute classmethods, and __init_subclass__ registers the custom ops while
    op_forward / op_backward are written once in the base. compile_unsupported_reason lets an
    operation say why it cannot be compiled; it sits on the operation rather than on the args (where
    Linear keeps it) because in ops/ the compile boundary is the fuser group.
  • OperationFuser runs a supported group through its operations' custom ops under compile.
  • Sequential builds its module groups outside the forward pass, since that constructs nn.Modules.

Four side effects had to go

All of them reached outside the higher-order op's scope, and each was found by running the thing:

  • OperationContext objects are created in the forward, but the backward is a separate subgraph, so
    writing to them there mutates an enclosing scope. The backward copies them into its own scope.
  • requires_grad_ on an output -- AOTAutograd's functionalization drops it anyway, and autograd marks
    the outputs of an apply() itself.
  • _do_not_clear on inputs and outputs.
  • warnings.warn from the gate, which is not traceable; the reason is reported from the eager path
    only, so a configuration that is never run eagerly falls back silently.

They are gated on being traced, not on using the custom ops. Under fullgraph=True there is no
leaving the graph, so an unsupported operation does not "fall back": the pipeline is traced either way
and only the choice of implementation changes.

Testing

test_torch_compile.py gains a test-only _ScaleOp and two tests: a single-operation group compiles
and matches eager (output, input gradient, parameter gradient), and an operation without the compute
halves still runs, on its eager implementation.

test_torch_compile.py + test_fusible_ops.py: 1587 passed, 1187 skipped. Lint 10.00/10. RTX Ada.

Gated out and untouched: multi-operation groups, fused operations, grouped operations, userbuffers,
delayed scaling, FP8 block scaling.

Checklist:

  • I have read and followed the contributing guidelines
  • The functionality is complete
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

pggPL added 2 commits August 5, 2026 22:21
register_custom_op now defines an operation's forward and backward as two
independent two-tier custom ops and hands back both, leaving autograd to the
caller. That is what lets a pipeline-level autograd.Function decide how the
two are wired, and so group the forward and backward passes differently --
which is what ops.OperationFuser does.

The variant that wires autograd itself keeps the old behaviour under
register_custom_op_with_autograd, and is now built on the same registration:
the pair is the primitive, autograd is what the other one adds. About two
thirds of the two bodies were the same code before.

BasicOperation gains the plumbing an operation needs to opt in: declare two
argument containers and implement four compute classmethods, and
__init_subclass__ registers the custom ops while op_forward / op_backward are
written once in the base. compile_unsupported_reason lets an operation say why
it cannot be compiled -- it sits here rather than on the args, as Linear has
it, because in ops/ the compile boundary is the fuser group, not the
operation.

Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
A group whose operations declare their compute halves now runs through their
custom ops under torch.compile(fullgraph=True). The pipeline-level
autograd.Function is traced as a higher-order op, which is what will later let
its forward and backward walk different op groupings.

Four side effects reached outside the higher-order op's scope and had to go:

- OperationContext objects are created in the forward, but the backward is a
  separate subgraph, so writing to them there mutates an enclosing scope; the
  backward copies them into its own scope instead;
- requires_grad_ on an output, which AOTAutograd's functionalization drops
  anyway -- autograd marks the outputs of an apply() itself;
- _do_not_clear on inputs and outputs;
- warnings.warn from the gate, which is not traceable, so the reason is
  reported from the eager path only.

They are gated on being traced rather than on using the custom ops. Under
fullgraph there is no leaving the graph, so an unsupported operation does not
fall back: the pipeline is traced either way and only the choice of
implementation changes.

Sequential builds its module groups outside the forward pass, since that
constructs nn.Modules.

Tested with a test-only operation, so the fuser's path does not depend on
which real operations happen to declare their halves.

Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
An operation lists the forward kwargs it takes in fwd_kwarg_names. They are
resolved into its args container like any other config, in the traced Python
where Dynamo guards them, so they reach the custom op through the existing
schema -- a value is guarded, a tensor is lifted into the graph, and a quantized
one crosses as its inner buffers.

An undeclared kwarg still sends the whole group to eager. That is not a schema
limitation, as the old message implied: the kwargs that remain are the grouped
operations' preallocated buffers, which the op writes to, and a custom op may
not mutate a tensor from an enclosing scope.

A kwarg carries no gradient. This matches the eager path, where kwargs never
entered the autograd graph either, and is why only read-only ones are accepted.

The fuser test helper now builds a separate model for the eager and the compiled
pass. Previously both shared one model and the eager pass ran first to produce
the reference, so the compiled pass was always traced on a model whose module
groups, fusions and pre_first_fuser_forward had already run. Those paths are now
traced as well.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant