Is there any more detailed docs about djl-serving?

[havatrier]

I'd like to apply djl-serving to serve model inference. Is there any more detailed docs about it, especially about how to define Workflow?

[frankfliu]

@havatrier

We are still working on djl-serving documentation, and Workflow feature is still under development, see this PR for more detail.

For regular model, djl-serving can serve it as tensor in tensor out out of box, you only need a url to the model file:

djl-serving -m "rest_endpoint=file:/models/mymodel/model.pt"

You can use TranslatorFactory to add preprocessing/postprocessing to your model by adding a serving.properties in the model folder, see this example: https://gist.github.com/frankfliu/b96004507e793acb5ca378b5fa17295d
Or you can just pass translator as a parameter in the model url:

djl-serving -m "rest_endpoint=file:/models/mymodel/model.pt?translatorFactory=ai.djl.pytorch.zoo.nlp.qa.PtBertQATranslatorFactory"