Trying to get DirectML working in Unity.

[elephantpanda]

I have got ONNX running in Unity by downloading the NuGet packages via Visual studio then copying the files into a Plugins folder in the Assets folder.

This works fine. Only problem is the onnx I'm trying to run is taking about 30 seconds using CPU only. I hope to speed it up using the internal GPU.

I want to use DirectML. I have Intel HD 600 integrated graphics and DirectX 12 which should be compatible. 64bit OS.

The files I have are:
Microsoft.ML.OnnxRuntime.dll (.Net library)
onnxruntime.dll (from the Microsoft.ML.OnnxRuntime.DirectML.1.13.1 NuGet)
DirectML.dll (version 1.9)

I want to make it use Direct ML so I add:

 SessionOptions so = new SessionOptions {  };
      so.AppendExecutionProvider_DML();
     var session = new InferenceSession(filePath, so);

But I am getting the error:

...onnxruntime.dll!00007FFCB72F5183: (caller: 00007FFCB72F45CF) Exception(1) tid(2b30) 80004002 No such interface supported

Any ideas what is going wrong? Like I said, it works find on CPU, but I can't switch it to use Direct ML.

Weirdly I can get a different onnx to create a session. But then when I run it I get this error:

OnnxRuntimeException: [ErrorCode:RuntimeException] Non-zero status code returned while running InstanceNormalization node. Name:'/decoder/mid_block/resnets.0/norm1/InstanceNormalization' Status Message: D:\a_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\MLOperatorAuthorImpl.cpp(1866)\onnxruntime.dll!00007FFF34872A90: (caller: 00007FFF34871F93) Exception(6) tid(42e4) 80070057 The parameter is incorrect.

So does this mean DirectML only works with certain onnx but not others?

Well, I got some onnx working once I built it. (though not in the editor - still not sure why).

Unfortunately for the one I wanted which is a vae_decoder, (the one that took 30 seconds on CPU), it says "not enough memory for resources". So I assume this means that my embedded Intel UHD graphics is not good enough to run this with DirectML.

[RyanUnderhill]

Looks like from your other post that you got it working. Was it loading the wrong onnxruntime.dll?

[elephantpanda]

No, I think it just wasn't finding the right DirectML.dll. I put the newest DirectML.dll in the Unity folder. But I have to mark it as "load on runtime" for it it load in the editor.

Also, the other reason it wasn't working was I didn't have enough RAM.

But now I'm running this on a Shadow PC and it works perfectly..