GlobalAveragePoolGrad node does not appear to be assigned an op_name

[georgen117]

I have been adding training for the dnnl endpoint. I have been working on Pooling operators. I ran into issues when working on GlobalAveragePoolGrad. The dnnl endpoint relies on knowing the name of the node to make decisions about that node.

So far all of the nodes we have worked on have been named. For example, MaxPool for the inference op and MaxPoolGrad for the gradient training op.

When trying to add GlobalAveragePoolGrad I ran into an issue the op_name appears to be left empty. Leaving me in a situation that I am unable to properly identify.

Looking at the implementation of the gradient builder (gradient_builder.cc lines 1213-1245) it looks like GlobalAveragePool may still be a work in-progress.

• It is unclear. Is this a bug or intentional?
• Can I expect the op_name value to be assigned the "GlobalAveragePoolGrad" in the future?

[pranav-prakash]

I think not all gradient operators correspond to a single macro-op. In some cases the gradient is implemented via combination of individual ops. E.g. for matmul there's no matmulgrad, it's just lowered to matmul+transpose.

[pranav-prakash]

@georgen117 Although this makes me wonder about how EPs that can perform the macro-op directly are supposed to handle it. Are they expected to recognize and fuse the node sequence? I also haven't checked whether the combination of individual ops is expressed as a subgraph of the original graph or not; if they are, that may give a way of detecting it.

If you figure this out, please do leave a note here!

[SherlockNoMad]

GlobalAveragePool GradientBuilder is indeed an incomplete implementation.

Are you planning to write a op and kernel for GlobalAveragePoolGrad?

[georgen117]

I was planning on writing a GlobalAveragePoolGrad for the DNNL execution provider. I have switched focus to the ops used for NLP models so this is not a high priority item.