[PyTorch] Documentation for op fuser API

[timmoon10]

Description

This PR adds a basic usage guide for the op fuser and includes it in the autogenerated API docs.

It is ready as-is, but if reviews take a while I may expand it with a guide on creating custom fused ops.

Type of change

• Documentation change (change only to the documentation, either a fix or a new content)
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Infra/Build change
• Code refactoring

Changes

• Add basic usage guide for op fuser
• Include TE ops in autogenerated API docs
• Debug TE ops docstrings

Checklist:

• I have read and followed the contributing guidelines
• The functionality is complete
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[greptile-apps]

Greptile Overview

Greptile Summary

This PR adds comprehensive documentation for the operation fuser API, a "bottom-up" alternative to Transformer Engine's monolithic modules that allows users to construct and fuse individual operations flexibly.

Key Changes

New Documentation (251 lines)

• Created docs/examples/op_fuser/op_fuser.rst with detailed usage guide covering:
  • Motivation for the op fuser approach (flexibility vs. monolithic modules)
  • Basic usage with Sequential and FusibleOperation
  • Quantization workflows with FP8/FP4
  • Branching operations (AddExtraInput, MakeExtraOutput)
  • Implementation details (BasicOperation, FusedOperation, OperationFuser)
  • Common misconceptions (not a kernel compiler, not a graph compiler)
• Includes three diagrams showing operation fusion examples

API Documentation Updates

• Added "Operation fuser" section to docs/api/pytorch.rst with 26+ operations:
  • Container classes: Sequential, FusibleOperation
  • Basic operations: Linear, BasicLinear, LayerNorm, RMSNorm, Bias, etc.
  • Activation functions: GELU, SwiGLU, ReLU, GEGLU, etc.
  • Utility operations: Quantize, Reshape, AddExtraInput, MakeExtraOutput
  • Distributed operations: AllGather, AllReduce, ReduceScatter

Docstring Fixes (15 files)

• Fixed RST formatting: single backticks (`) to double backticks (``) for inline code
• Fixed hyperlink formatting: added spaces before closing backticks (e.g., link <url>__ instead of link<url>__)
• Standardized boolean/None literals: `False` → False, `None` → None
• Improved docstring structure and readability

Minor Fixes

• Fixed .gitignore: changed *.DS_Store to .DS_Store (more accurate pattern)

Confidence Score: 5/5

• This PR is completely safe to merge - it contains only documentation improvements with no code logic changes
• Perfect confidence score because: (1) All changes are documentation-only - new RST guide, API reference updates, and docstring formatting fixes, (2) No functional code changes that could introduce bugs, (3) Docstring fixes improve documentation quality and RST rendering, (4) The new op fuser guide is comprehensive and well-structured with clear examples
• No files require special attention - all changes are documentation improvements

Important Files Changed

File Analysis

Filename	Score	Overview
docs/examples/op_fuser/op_fuser.rst	5/5	New comprehensive documentation guide for op fuser API with examples and diagrams
docs/api/pytorch.rst	5/5	Added Operation fuser section with 26 fusible operations to API documentation
.gitignore	5/5	Fixed .DS_Store pattern from *.DS_Store to .DS_Store
transformer_engine/pytorch/ops/basic/activation.py	5/5	Fixed RST hyperlink formatting (added spaces before closing backticks) and improved docstring structure
transformer_engine/pytorch/ops/basic/basic_linear.py	5/5	Fixed docstring formatting: single to double backticks, False/None literal formatting

Sequence Diagram

sequenceDiagram
    participant User
    participant Sequential
    participant OperationFuser
    participant BasicOps
    participant FusedOps
    
    User->>Sequential: forward(input)
    Sequential->>Sequential: _make_module_groups()
    Sequential->>OperationFuser: __call__(input)
    OperationFuser->>OperationFuser: maybe_fuse_ops()
    OperationFuser->>BasicOps: Analyze fusion opportunities
    OperationFuser->>FusedOps: Create fused operations
    FusedOps->>BasicOps: fuser_forward()
    BasicOps-->>FusedOps: output
    FusedOps-->>OperationFuser: output
    OperationFuser-->>Sequential: output
    Sequential-->>User: output

Loading

[greptile-apps]

Additional Comments (1)

1. transformer_engine/pytorch/ops/basic/activation.py, line 387 (link)

   syntax: Extra space before period.

   _{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

_{19 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[timmoon10]

/te-ci core pytorch

[greptile-apps]

_{No files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

_{No files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}