How to calibrate weights of models after apply Accelerating a Hugging Face Llama 3 models with Transformer Engine

[xuanvinh1997]

After reading this tutorial, Accelerating a Hugging Face Llama 2 and Llama 3 models with Transformer Engine, I now understand how to convert HF/Torch layers to TE layers and fuse them. However, the model's output is no longer correct.

My question is: how can I calibrate it without re-training the pretrained model?

Many thanks!