Some questions about usage #6632
Unanswered
fighterhit
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
nvcr.io/nvidia/tritonserver:22.12-py3
torch.save(model.state_dict(), 'model.pt')
. How could I provide services through Triton Inference Server? Is there any recommended usage? I tried to use the torch2trt library, but it was not installed successfully.MessagePack
serialized byte array. TheContent-Type
wasapplication/octet-stream
. Before the data was input into the model, deserialization and image conversion operations were performed. The model output tensor will also be converted into a python list and returned to the client. How could I use Triton Inference Server to meet the above requirements and provide http services?Beta Was this translation helpful? Give feedback.
All reactions