Skip to content

Autotuning for distributed kernels #473

@joydddd

Description

@joydddd

Enable autotuning for distributed kernels launched via torchrun.

Event based benchmarker available at autotuner/benchmarker.py in PR: #393

Enable via

  1. Make sure all torchrun workers benchmark same configs in same order.
  2. Master rank decides the config and communicate that to all processes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions