Skip to content

Commit b53e5b2

Browse files
committed
Update on "[ExecuTorch] XNNPACK: prefer qc over qb when gs == k for non-int4"
* Prefer chanelwise over groupwise when possible for perf and for int8 which doesn't have groupwise support * Fix bug / improve behavior for affine q/dq with gs == k for per_channel * refactor is_per_channel_group state variable * add QuantParams.__str__() TODO - improve affine quant primitives - T237476295 Differential Revision: [D82060758](https://our.internmc.facebook.com/intern/diff/D82060758/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D82060758/)! [ghstack-poisoned]
2 parents 5dbf8c6 + a6d3379 commit b53e5b2

File tree

1 file changed

+2
-7
lines changed

1 file changed

+2
-7
lines changed

backends/xnnpack/operators/quant_params.py

Lines changed: 2 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -96,20 +96,15 @@ def __init__(
9696
assert (
9797
self.per_channel is True
9898
), "Only per channel quantization supports groupwise quantization"
99-
assert (
100-
self.axis == 0,
101-
"Only axis 0 is supported for per channel groupwise quant",
102-
)
10399
assert (
104100
cast(torch.Tensor, scale).ndim == 2
105101
), "Scale must be 2D for per channel groupwise quant"
106102
# Assumed scale shape - [out_channels, in_channels/group_size]
107103
input_channels = cast(torch.Tensor, scale).shape[1] * self.group_size
108104
# 2d weight tensor shape - [out_channels, in_channels]
109105
assert (
110-
tensor.shape[1] == input_channels,
111-
"Invalid input channels for groupwise quant",
112-
)
106+
tensor.shape[1] == input_channels
107+
), "Invalid input channels for groupwise quant"
113108
# Prefer per_channel over per_channel_group when group_size == input_channels for non int4 cases only
114109
# int4 case need more fixes to map qb4w to qc4w. Incorrect scales being passed down to xnnpack.
115110
self.per_channel_group = (

0 commit comments

Comments
 (0)