Example of running huggingface model with tensor parallel #20831
Description
Description & Motivation
I understand that there are current example available with Llama 3. https://github.com/Lightning-AI/pytorch-lightning/tree/master/examples/pytorch/tensor_parallel
However, with an in-house implementation of Llama and without the ability of loading a huggingface-format checkpoint (which is the de-facto industry standard), this toy example cannot be used in practice.
Pitch
A working example of tensor parallelism / 2D parallelism with transformers model.
Alternatives
No response
Additional context
No response