-
Notifications
You must be signed in to change notification settings - Fork 204
[5455919] Fix Q/DQ/Cast placement in 'FP32 required' custom ops #554
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[5455919] Fix Q/DQ/Cast placement in 'FP32 required' custom ops #554
Conversation
ajrasane
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Can we add a test case for this?
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #554 +/- ##
==========================================
- Coverage 74.47% 74.43% -0.04%
==========================================
Files 182 182
Lines 18225 18238 +13
==========================================
+ Hits 13573 13576 +3
- Misses 4652 4662 +10 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Signed-off-by: gcunhase <[email protected]>
…m ops in the quantization workflow Signed-off-by: gcunhase <[email protected]>
…custom op Signed-off-by: gcunhase <[email protected]>
Signed-off-by: gcunhase <[email protected]>
Signed-off-by: gcunhase <[email protected]>
Signed-off-by: gcunhase <[email protected]>
7c854c7 to
49a4513
Compare
What does this PR do?
Type of change: Bug fix
Overview: Fix incorrect quantization of custom ops when some input tensors are required to be in INT8 and some in FP32.
Usage
Testing
1. BEVFormer model
$ python -m modelopt.onnx.quantization --onnx_path=/mnt/models/bevformer_tiny_epoch_24_cp2_op13.onnx \ --trt_plugins=$PLUGIN_PATH \ --trt_plugins_precision MultiScaleDeformableAttnTRT:[int8,int32,fp32,int8,int8]:[int8] \ --high_precision_dtype fp162. 5455919 model
Validated model in bug 5455919.
Before your PR is "Ready for review"
Additional Information