DML EP is not executing nodes, despite it registering as a provider successfully

[david-macleod]

I am currently experiencing an issue where models are not run using the DML EP, despite using the onnxruntime-directml wheel and the session object returning DML as a registered provider.

import onnxruntime as ort
import torch

model = torch.nn.Sequential(*[torch.nn.Linear(4096, 4096) for _ in range(4)])
x = torch.randn(1, 4096)

torch.onnx.export(torch.jit.script(model), args=x, f="tmp.onnx", input_names=["x"], opset_version=13, example_outputs=x)

options = ort.SessionOptions()
options.log_severity_level = 0
session = ort.InferenceSession("tmp.onnx", options, providers=["DmlExecutionProvider"])
print("EPs", session.get_providers())

for _ in range(10):
    session.run(None, input_feed={"x": x.numpy()})

Which includes the following in the logs

2021-11-05 08:20:54.3613101 [I:onnxruntime:, inference_session.cc:1242 onnxruntime::InferenceSession::Initialize] Initializing session.
2021-11-05 08:20:54.3614344 [I:onnxruntime:, inference_session.cc:1279 onnxruntime::InferenceSession::Initialize] Adding default CPU execution provider.
2021-11-05 08:20:54.3663319 [I:onnxruntime:, reshape_fusion.cc:42 onnxruntime::ReshapeFusion::ApplyImpl] Total fused reshape node count: 0
2021-11-05 08:20:54.3668477 [I:onnxruntime:, reshape_fusion.cc:42 onnxruntime::ReshapeFusion::ApplyImpl] Total fused reshape node count: 0
2021-11-05 08:20:54.3681419 [V:onnxruntime:, inference_session.cc:152 onnxruntime::`anonymous-namespace'::VerifyEachNodeIsAssignedToAnEp] Node placements
2021-11-05 08:20:54.3683129 [V:onnxruntime:, inference_session.cc:154 onnxruntime::`anonymous-namespace'::VerifyEachNodeIsAssignedToAnEp] All nodes have been placed on [CPUExecutionProvider].

but also

EPs ['DmlExecutionProvider', 'CPUExecutionProvider']

Can anyone suggest a fix for this? I am using onnxruntime-directml==1.9.0 and torch==1.10.0+cu113

[wschin]

Could you please try opset_version=10? Thanks. I only see support up to opset 12 in DML EP.

@fdwr , would it be reasonable to add opset 13 support?

[david-macleod]

Thanks for reply! I have since created a full issue from this so will close this now
#9719

[fdwr]

@fdwr , would it be reasonable to add opset 13 support?

@wschin: Indeed, it's in the plan to catch up to 13-15 (obviously not for 1.10 given the timeline, but afterward).