How does TransformerEngine interact with TensorRT/Onnx? #238
Unanswered
tylerweitzman
asked this question in
Q&A
Replies: 1 comment
-
|
Hi @tylerweitzman, We are evaluating support for exporting TE models to ONNX and import from TensorRT. Support for FP8 GPT export will be rolled out first. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi,
We are often converting PyTorch models to other engines in order to optimize inference speed. Is it possible to convert a PyTorch model that is using TransformerEngine into TensorRT for faster inference? Or is it a choice either between
nn.Linearwith TensorRT right now?Beta Was this translation helpful? Give feedback.
All reactions