Skip to content

Commit 76b1385

Browse files
mcr229facebook-github-bot
authored andcommitted
Use IDs for placeholder/outputs of implicit q/dq nodes (#141)
Summary: Pull Request resolved: #141 In XNNPACK from the partitioner we encounter quantize node that are outputs to the graph. Since these quantize nodes are implicit and not actually quantizing the values, we must give its externalness to the node with which it quantizes. ### Outputs ``` | conv --> q ----> | ----> dq --> | Delegate boundary ``` In this example the output of the delegate is the output of quantized Conv. In XNNPACK, we must designate the output of Conv to be an external value. In this graph, however, the q node is the output. Since q is an implicit node (it does not actually running quantize) We pass its externalness to the conv node. ### Inputs i.e. ``` | q ----> | ----> dq --> conv --> | Delegate boundary ``` In this example q is the placeholder node and is given as input to the first dq. But we are running a quantized conv within the delegate. Since dq is implicit, we pass the externalness of the q place holder to dq, so that the input to Conv is external. Reviewed By: digantdesai Differential Revision: D48667676 fbshipit-source-id: 0be5bf8c3da227463e12eea46d72b76561a34e40
1 parent 80ed456 commit 76b1385

File tree

1 file changed

+25
-13
lines changed

1 file changed

+25
-13
lines changed

backends/xnnpack/operators/node_visitor.py

Lines changed: 25 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -130,21 +130,33 @@ def gen_ids_and_flags(
130130
# This will break if we change the way q/dq are partitioned
131131

132132
# Tensor can still be input if its quantizing node is an input
133-
is_input = (self).is_graph_input(tensor)
133+
if self.is_graph_input(tensor) or (
134+
quant_params.is_input if quant_params else False
135+
):
136+
tensor_input = tensor
137+
if quant_params:
138+
if quant_params.is_input and not self.is_graph_input(tensor):
139+
tensor_input = quant_params.q_input
140+
assert (
141+
tensor_input in self.external_ids.keys()
142+
), f"Tensor {tensor_input}, is_input. ext_ids: {self.external_ids.keys()}"
143+
ext_id = self.external_ids[tensor_input].external_id
144+
xnn_graph.input_ids.append(id_out)
145+
flag = self.external_ids[tensor_input].io_type
134146
# Tensor can still be output if its quantizing node is an output
135-
is_output = self.is_graph_output(tensor)
136-
# handle logic for input/output tensors
137-
if is_input or is_output:
147+
elif self.is_graph_output(tensor) or (
148+
quant_params.is_output if quant_params else False
149+
):
150+
tensor_output = tensor
151+
if quant_params:
152+
if quant_params.is_output and not self.is_graph_output(tensor):
153+
tensor_output = list(tensor.users)[0]
138154
assert (
139-
tensor in self.external_ids.keys()
140-
), f"Tensor {tensor}, is_input: {is_input}, is_output: {is_output}, ext_ids: {self.external_ids.keys()}"
141-
ext_id = self.external_ids[tensor].external_id
142-
if is_input:
143-
xnn_graph.input_ids.append(id_out)
144-
flag = XNN_VALUE_FLAG_EXTERNAL_INPUT
145-
if is_output:
146-
xnn_graph.output_ids.append(id_out)
147-
flag = XNN_VALUE_FLAG_EXTERNAL_OUTPUT
155+
tensor_output in self.external_ids.keys()
156+
), f"Tensor {tensor_output} is_output: ext_ids: {self.external_ids.keys()}"
157+
ext_id = self.external_ids[tensor_output].external_id
158+
xnn_graph.output_ids.append(id_out)
159+
flag = self.external_ids[tensor_output].io_type
148160

149161
return ext_id, id_out, flag
150162

0 commit comments

Comments
 (0)