This repository is no longer supported.

Please start using AWS Neuron reference for NeMo Megatron (https://github.com/aws-neuron/neuronx-nemo-megatron) or Neuron Distributed (https://github.com/aws-neuron/neuronx-distributed) instead.

This Megatron-LM library (source https://github.com/NVIDIA/Megatron-LM) is adapted for use with AWS Trainium via Neuron SDK.