NHWC vs NCHW WebGPU comparison

[grazder]

Hi! I'm trying to understand if there is a difference between NHWC and NCHW layouts , and ultimately choose one format for my application. Does anyone have expertise in this area, has anyone done benchmarks?

[grazder]

@fs-eire

[qjia7]

NHWC is the preferred layout of webgpu. And it's well tested and optimized. But It's also easy to switch the layout since you can specify the layout in webgpu's provider option. Per my understanding, the diff of NHWC and NCHW is mainly for the traditional nn models. If your models are not nn models. There should be no big difference.

[grazder]

Should I convert shapes in my convolutional nn models from NCHW to NHWC format for better performance? Or it is very minor difference? For example I have graph like this:

[fs-eire]

It is a little bit confusing:

• when you look at the model graph, it is usually NCHW. this is because ONNX spec defines in this way. if you want to use the stardard opset onnx.ai, you have no choice but to use NCHW. this is a limitation due to the initial version of ONNX only defines channel first (and the consideration of compatibility in the following versions)
• but when you load the model using ONNX Runtime, there is a runtime "layout transformer" to analyze the model and try to convert it from NCHW to NHWC (if channel last is preferred). If this step is successfully done, the "converted" model in memory will be NHWC, using a microsoft internal opset to represent the channel last operators.

So, when you use netron to visualize the ONNX model, usually you always see channel first operators. At runtime, specifically for WebGPU, we use the EP options "preferredLayout" to control the runtime behavior.

[grazder]

Yeah, I'm trying to understand if layout transforms can affect loading time, inference speed and GPU load? I saw a lot of transpose ops for NHWC during first run as I remembered, but are they performed once?

[fs-eire]

they should only run once, either during initialization (load time) or the first run (warmup time).

[grazder]

great, thanks!