DJL Pytorch mkldnn optimisation problem - 'opaque tensors do not have strides

[malcolm-mccarthy]

Hello, I have been trying to use the mkldnn settings suggested in optimisation guidelines. That is I am trying to run the inference with the system environment variable '-Dai.djl.pytorch.use_mkldnn=true'.
When doing so the following error is encountered

'ai.djl.engine.EngineException: opaque tensors do not have strides'

I am using a Torch script generated neural network model file, prepared in the following way.

import torch
from torch.utils import mkldnn as mkldnn_utils
model.eval()
model_mkldnn = mkldnn_utils.to_mkldnn(model)
model_script_mkldnn = torch.jit.trace(model_mkldnn, dummy_tensor)
model_script_mkldnn.save('MKLDNN_model.pt')

The underlying data types are 32 bit float

Running on Windows with gradle project settings

implementation "ai.djl.pytorch:pytorch-engine:0.16.0"
implementation "ai.djl.pytorch:pytorch-jni:1.10.0-0.16.0"
implementation "ai.djl.pytorch:pytorch-native-cpu:1.10.0:win-x86_64"

[frankfliu]

Can you run inference with mkldnn in python?

PyTorch MKLDNN has many limitations, in DJL we made it a global setting, it may not work in some cases.