Skip to content

Commit 8b77007

Browse files
xieofxiehualxie
andauthored
add op_types_to_quantize to get_qnn_qdq_config (microsoft#23458)
### Description <!-- Describe your changes. --> add op_types_to_quantize to get_qnn_qdq_config so support only quantize part of it. In my understanding, if ops are not quantized, they will use fp16 to run if supported? https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/SupportedOps.html ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> See scenario in issue in Olive microsoft/Olive#1552 --------- Co-authored-by: hualxie <[email protected]>
1 parent 3701e3e commit 8b77007

File tree

1 file changed

+9
-1
lines changed
  • onnxruntime/python/tools/quantization/execution_providers/qnn

1 file changed

+9
-1
lines changed

onnxruntime/python/tools/quantization/execution_providers/qnn/quant_config.py

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,7 @@ def get_qnn_qdq_config(
5454
keep_removable_activations: bool = False,
5555
stride: int | None = None,
5656
calibration_providers: list[str] | None = None,
57+
op_types_to_quantize: list[str] | None = None,
5758
) -> StaticQuantConfig:
5859
"""
5960
Returns a static quantization configuration suitable for running QDQ models on QNN EP.
@@ -120,6 +121,7 @@ def get_qnn_qdq_config(
120121
QuantizeLinear/DequantizeLinear operators from the model.
121122
calibration_providers: Execution providers to run the session during calibration. Default is None which uses
122123
[ "CPUExecutionProvider" ].
124+
op_types_to_quantize: If set to None, all operator types will be quantized except for OP_TYPES_TO_EXCLUDE
123125
124126
Returns:
125127
A StaticQuantConfig object
@@ -164,7 +166,11 @@ def get_qnn_qdq_config(
164166
name_to_initializer,
165167
)
166168

169+
op_types_to_quantize_set = set(op_types_to_quantize) if op_types_to_quantize else None
170+
167171
for node in model.graph.node:
172+
if op_types_to_quantize_set and node.op_type not in op_types_to_quantize_set:
173+
continue
168174
op_types.add(node.op_type)
169175
qnn_compat.process_node(node)
170176

@@ -192,7 +198,9 @@ def get_qnn_qdq_config(
192198
calibrate_method=calibrate_method,
193199
activation_type=activation_type,
194200
weight_type=weight_type,
195-
op_types_to_quantize=list(op_types.difference(OP_TYPES_TO_EXCLUDE)),
201+
op_types_to_quantize=op_types_to_quantize
202+
if op_types_to_quantize
203+
else list(op_types.difference(OP_TYPES_TO_EXCLUDE)),
196204
per_channel=per_channel,
197205
use_external_data_format=(model_has_external_data or model.ByteSize() >= MODEL_SIZE_THRESHOLD),
198206
calibration_providers=calibration_providers,

0 commit comments

Comments
 (0)