Skip to content

[PyTorch] Declare fusible ops as custom ops (1/3): framework + Bias - #27

Closed
pggPL wants to merge 3 commits into
linear_compile_rebasedfrom
ops_custom_ops
Closed

[PyTorch] Declare fusible ops as custom ops (1/3): framework + Bias#27
pggPL wants to merge 3 commits into
linear_compile_rebasedfrom
ops_custom_ops

Conversation

@pggPL

@pggPL pggPL commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Description

First step towards torch.compile(fullgraph=True) support for transformer_engine.pytorch.ops.Sequential.

The plan is to keep the pipeline-level _OperationFuserAutogradFunction and let Dynamo trace it as a
higher-order op, with each fusible operation calling its own custom op inside. That keeps the forward
and backward fusion layouts independent (so the backward-only fusions survive), keeps
OperationContext inside the traced scope, and bounds op registration to one entry per op class.

This PR does only the per-op half of that: make every basic/fused operation declarable as a
custom op, with tests. Wiring the fuser up is a separate PR, so nothing in the eager path changes yet.

Type of change

  • Documentation change (change only to the documentation, either a fix or a new content)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Infra/Build change
  • Code refactoring

Changes

  • register_op_halves (dynamo/custom_op.py): registers an op's forward and backward as two
    independent two-tier custom ops and returns callables for both, without registering autograd.
    A caller wiring the halves into its own autograd.Function can then group the forward and backward
    passes differently, which is exactly what OperationFuser does. Reuses the existing
    schema/adapter/TensorSpec machinery; register_custom_op is untouched.
  • Bias converted as the template: split into resolve_fwd_args (reads module config and global
    FP8 state, so it stays in the traced region where Dynamo guards those reads), pure impls that take
    everything as arguments and never touch self, and data-free fakes. op_forward / op_backward
    keep their signatures and drive the same impls, so the eager path is unchanged.
  • tests/pytorch/test_ops_custom_ops.py: the fake-vs-real conformance harness every subsequent op
    will reuse, plus numerics-against-eager and fullgraph=True tests for Bias.

Note on the pass-through-gradient convention

A custom op may not return one of its own inputs. Bias's backward returns grad_output unchanged
whenever there is no grad-input quantizer, which trips that check, and cloning would cost a full-size
copy on the common path. The impl therefore returns None for that slot and the caller substitutes
grad_output. Pass-through grads are common (Identity, Reshape, ConstantScale, Quantize), so
the remaining ops will follow the same convention.

Checklist:

  • I have read and followed the contributing guidelines
  • The functionality is complete
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

pggPL added 3 commits August 5, 2026 16:53
… glue

Registers an op's forward and backward as two independent two-tier custom
ops and returns callables for both, leaving autograd to the caller. This
lets a pipeline-level autograd.Function (which Dynamo traces as a
higher-order op) group the forward and backward passes differently, as
ops.OperationFuser does.

Reuses the existing schema/adapter/TensorSpec machinery; register_custom_op
is untouched.

Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
Splits Bias into config resolution, pure compute and ctx saving, then
registers the compute halves via register_op_halves. resolve_fwd_args reads
module config and global FP8 state, so it stays in the traced region where
Dynamo guards those reads; the impls take everything as arguments and never
touch self.

op_forward/op_backward keep their signatures and drive the same impls, so
the eager path is unchanged.

Adds tests/pytorch/test_ops_custom_ops.py with the fake-vs-real conformance
harness every subsequent op will reuse.

Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
A custom op may not return one of its own inputs, which Bias did whenever
the grad input is grad_output unchanged. Cloning would cost a full-size
copy on the common unquantized path, so the impl returns None for that slot
and the caller substitutes grad_output.

Pass-through grads are common (Identity, Reshape, ConstantScale, Quantize),
so this is the convention those ops will follow too.

Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
@pggPL

pggPL commented Aug 5, 2026

Copy link
Copy Markdown
Owner Author

Superseded: rebased onto linear_compile_on_main and split into #29 and #30.

@pggPL pggPL closed this Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant