NXP backend: Replace pass to fuse activations functions with joint quantization with activation #14816
Conversation
+ Move fused activations to separate QDQ cluster
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14816
Note: Links to docs will display an error until the docs builds have been completed. ❌ 3 New Failures, 1 Cancelled JobAs of commit 6fdef26 with merge base f81e834 ( NEW FAILURES - The following jobs have failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
bot
added
the
CLA Signed
|
@pytorchbot label "module: nxp" "release notes: nxp" |
pytorch-bot
bot
added
module: nxp
|
|
||
|
|
||
| class HardTanhInPlacePattern(QuantizationPattern): | ||
| class HardTanhInPlacePattern(SingleInputBasicPattern): |
There was a problem hiding this comment.
Didn't you forget to remove the get_anchors() method?
| "quantized_decomposed.quantize_per_tensor.default", | ||
| "quantize_per_channel.default", | ||
| "quantized_decomposed.quantize_per_channel.default", | ||
| ] |
There was a problem hiding this comment.
Why not compare the targets directly? It would be more reliable.
|
|
||
| class NeutronTargetNeutronC: | ||
| @staticmethod | ||
| def is_supported_fused_activation(node_: Node) -> bool: |
There was a problem hiding this comment.
Perhaps the suffix _aten should be added to the name. Same for is_fusable_conv_or_linear().
It is not clear that the function is meant for the aten dialect.
| exir_ops.edge.aten.hardtanh.default, | ||
| exir_ops.edge.aten.relu.default, | ||
| exir_ops.edge.aten.sigmoid.default, | ||
| exir_ops.edge.aten.tanh.default, |
There was a problem hiding this comment.
Is this necessary?
I would expect that since the Move*AuxiliaryOperatorIntoSeparateQDQClusterPass now supports these operators, they should be the main nodes within their own QDQ clusters.
| activation_quantizer = self.neutron_quantizer.op_to_quantizer[ | ||
| activation.target | ||
| ] | ||
| activation_quantizer.annotate(gm) |
There was a problem hiding this comment.
Minor:
The code only makes sense if the activation_quantizer only quantizes the output, and not the input. Perhaps a comment explaining that would be useful here. Same for the 3 patterns below.
Summary
This PR replaces optimizations 'fuse_activation_functions.py' by quantization of Conv 2D and Linear ops together with fusable activations - selected activations supported by Neutron (Relu, Relu6, Sigmoid, Tanh). Logic is determined by target specs, now supporting Neutron-C. Tests updated. Relu has now non-shared, standalone quantization.
Test plan
Unit tests provided (test_edge_passes.py, test_quantizer.py).
cc @robert-kalmar @JakeStevens @digantdesai