-
Notifications
You must be signed in to change notification settings - Fork 749
Add XNN U8 op support via quantization #8330
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/8330
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 1fbaf0e with merge base d99970b ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
4897af3 to
c5134f1
Compare
|
@GregoryComer has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
mcr229
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This generally looks right to me.
|
Ah reviewed internally first, please look at the comments there, next time I will remember. |
|
Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as |
|
Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as |
Summary
Support U8 ops in the XNNPACK delegate by treating the input and output tensors as u8 asymmetric-quantized tensors with scale=1 and zero_point=0. This PR adds U8 support for upsample_bilinear2d, cat, slice, and _to_copy (when used to convert u8 to f32). More ops are possible with this method.
Conversion from u8 to f32 is done via transformation into a dequantize op. This is implemented in a new pass - ReplaceU8ConvertWithDqPass. The general U8 to quantized U8 transformation is done in
define_tensorinnode_visitor.py. U8 inputs are created as quantized tensors with the appropriate qparams (scale=1, zp=0).Test plan
I've added op-level u8 tests to each of the new ops, as well as tests for the ReplaceU8ConvertWithDqPass. I've also added an end-to-end test for MobileNetV3 with a wrapper to take U8 inputs, resize and crop, and then convert to f32 and run the model.