Merge branch 'master' into implement_isinf

Author: mindest

README.md

@@ -1,14 +1,14 @@
 tf2onnx - convert TensorFlow models to ONNX models.
 ========
 
-![Build Status](https://dev.azure.com/tensorflow-onnx/tensorflow-onnx/_apis/build/status/unit_test?branchName=master)
+[![Build Status](https://dev.azure.com/tensorflow-onnx/tensorflow-onnx/_apis/build/status/unit_test?branchName=master)](https://dev.azure.com/tensorflow-onnx/tensorflow-onnx/_build?definitionId=16&branchName=master)
 
 # Supported ONNX version
-tensorflow-onnx will use the onnx version installed on your system and installs the latest onnx version if none is found.
+tensorflow-onnx will use the ONNX version installed on your system and installs the latest ONNX version if none is found.
 
-By default we use opset 7 for the resulting onnx graph since most runtimes will support opset 7. Opset 7 was introduced in onnx-1.2.
+By default we use opset 7 for the resulting ONNX graph since most runtimes will support opset 7. Opset 7 was introduced in onnx-1.2.
 
-With the release of onnx-1.3 there is now opset 8 - to create an onnx graph for opset 8 use in the command line ```--opset 8```.
+Newer releases of ONNX support higher opsets. For example, to create an ONNX graph for opset 8 use in the command line ```--opset 8```.
 
 # Status
 We support many TensorFlow models. Support for Fully Connected and Convolutional networks is mature. Dynamic LSTM/GRU/Attention networks should work but the code for this is evolving. 
@@ -73,17 +73,16 @@ python -m tf2onnx.convert
     --graphdef SOURCE_GRAPHDEF_PB
     --checkpoint SOURCE_CHECKPOINT
     --saved-model SOURCE_SAVED_MODEL
+    [--output TARGET_ONNX_MODEL]
     [--inputs GRAPH_INPUTS]
     [--outputs GRAPH_OUTPUS]
     [--inputs-as-nchw inputs_provided_as_nchw]
+    [--opset OPSET]
     [--target TARGET]
-    [--output TARGET_ONNX_GRAPH]
-    [--target TARGET]
-    [--continue_on_error]
-    [--verbose]
     [--custom-ops list-of-custom-ops]
-    [--opset OPSET]
     [--fold_const]
+    [--continue_on_error]
+    [--verbose]
 ```
 
 ## Parameters
@@ -100,10 +99,10 @@ the target onnx file path.
 Tensorflow model's input/output names, which can be found with [summarize graph tool](#summarize_graph). Those names typically end on ```:0```, for example ```--inputs input0:0,input1:0```. inputs and outputs are ***not*** needed for models in saved-model format.
 ### --inputs-as-nchw
 By default we preserve the image format of inputs (nchw or nhwc) as given in the TensorFlow model. If your hosts (for example windows) native format nchw and the model is written for nhwc, ```--inputs-as-nchw``` tensorflow-onnx will transpose the input. Doing so is convinient for the application and the converter in many cases can optimize the transpose away. For example ```--inputs input0:0,input1:0 --inputs-as-nchw input0:0``` assumes that images are passed into ```input0:0``` as nchw while the TensorFlow model given uses nhwc.
-### --target 
-Some runtimes need workarounds, for example they don't support all types given in the onnx spec. We'll workaround it in some cases by generating a different graph. Those workarounds are activated with ```--target TARGET```. 
 ### --opset
-by default we uses the newest opset 7 to generate the graph. By specifieing ```--opset``` the user can override the default to generate a graph with the desired opset. For example ```--opset 5``` would create a onnx graph that uses only ops available in opset 5. Because older opsets have in most cases fewer ops, some models might not convert on a older opset.
+By default we use the opset 7 to generate the graph. By specifying ```--opset``` the user can override the default to generate a graph with the desired opset. For example ```--opset 5``` would create a onnx graph that uses only ops available in opset 5. Because older opsets have in most cases fewer ops, some models might not convert on a older opset.
+### --target 
+Some models require special handling to run on some runtimes. In particular, the model may use unsupported data types. Workarounds are activated with ```--target TARGET```. Currently supported values are listed on this [wiki](https://github.com/onnx/tensorflow-onnx/wiki/target). If your model will be run on Windows ML, you should specify the appropriate target value.
 ### --custom-ops
 the runtime may support custom ops that are not defined in onnx. A user can asked the converter to map to custom ops by listing them with the --custom-ops option. Tensorflow ops listed here will be mapped to a custom op with the same name as the tensorflow op but in the onnx domain ai.onnx.converters.tensorflow. For example: ```--custom-ops Print``` will insert a op ```Print``` in the onnx domain ```ai.onnx.converters.tensorflow``` into the graph. We also support a python api for custom ops documented later in this readme. 
 ### --fold_const
@@ -163,7 +162,7 @@ optional arguments:
   --config           yaml config file
   --verbose          verbose output, option is additive
   --opset OPSET      target opset to use
-  --perf csv-file    capture performance numbers or tensorflow and onnx runtime
+  --perf csv-file    capture performance numbers for tensorflow and onnx runtime
   --debug            dump generated graph with shape info
   --fold_const when set, TensorFlow fold_constants transformation will be applied before conversion. This will benefit features including Transpose optimization (e.g. Transpose operations introduced during tf-graph-to-onnx-graph conversion will be removed), and RNN unit conversion (for example LSTM).
 ```

ci_build/azure_pipelines/pretrained_model_test-matrix.yml

@@ -5,7 +5,17 @@ jobs:
   parameters:
     platforms: ['linux', 'windows', 'mac']
     python_versions: ['3.6', '3.5']
-    tf_versions: ['1.13.1', '1.12', '1.11', '1.10', '1.9', '1.8', '1.7', '1.6', '1.5']
+    tf_versions: ['1.12', '1.11', '1.10', '1.9', '1.8', '1.7', '1.6', '1.5']
+    onnx_opsets: ['9', '8', '7']
+    job:
+      steps:
+      - template: 'pretrained_model_test.yml'
+
+- template: 'templates/job_generator.yml'
+  parameters:
+    platforms: ['linux', 'windows', 'mac']
+    python_versions: ['3.7', '3.6', '3.5']
+    tf_versions: ['1.13.1']
     onnx_opsets: ['9', '8', '7']
     job:
       steps:

ci_build/azure_pipelines/pretrained_model_test.yml

@@ -3,7 +3,7 @@
 jobs:
 - template: 'templates/job_generator.yml'
   parameters:
-    python_versions: ['3.6', '3.5']
+    python_versions: ['3.7', '3.6', '3.5']
     tf_versions: ['1.13.1']
     onnx_opsets: ['9', '8', '7']
     job:

ci_build/azure_pipelines/unit_test-matrix.yml

@@ -5,7 +5,16 @@ jobs:
   parameters:
     platforms: ['linux', 'windows', 'mac']
     python_versions: ['3.6', '3.5']
-    tf_versions: ['1.13.1', '1.12', '1.11', '1.10', '1.9', '1.8', '1.7', '1.6', '1.5']
+    tf_versions: ['1.12', '1.11', '1.10', '1.9', '1.8', '1.7', '1.6', '1.5']
+    job:
+      steps:
+      - template: 'unit_test.yml'
+
+- template: 'templates/job_generator.yml'
+  parameters:
+    platforms: ['linux', 'windows', 'mac']
+    python_versions: ['3.7']
+    tf_versions: ['1.13.1']
     job:
       steps:
       - template: 'unit_test.yml'

ci_build/azure_pipelines/unit_test.yml

@@ -3,7 +3,7 @@
 jobs:
 - template: 'templates/job_generator.yml'
   parameters:
-    python_versions: ['3.6', '3.5']
+    python_versions: ['3.7', '3.6', '3.5']
     tf_versions: ['1.13.1']
     job:
       steps:

tests/test_backend.py

@@ -945,6 +945,42 @@ def test_slice(self):
         _ = tf.identity(x_, name=_TFOUTPUT)
         self._run_test_case([_OUTPUT], {_INPUT: x_val})
 
+    @check_opset_min_version(10, "Slice in opset 10 can accept dymaic 'start' and 'ends'")
+    def test_slice_with_non_const(self):
+        x_val = np.array([[1, 2, 3, 4], [5, 6, 7, 8]], dtype=np.float32)
+        t1 = np.array([0, 1], dtype=np.int32)
+        t2 = np.array([2, 2], dtype=np.int32)
+        x0 = tf.placeholder(tf.float32, x_val.shape, name=_TFINPUT)
+        t1_ = tf.placeholder(tf.int32, t1.shape, name=_TFINPUT1)
+        t2_ = tf.placeholder(tf.int32, t2.shape, name=_TFINPUT2)
+        x_ = tf.slice(x0, t1_, t2_)
+        _ = tf.identity(x_, name=_TFOUTPUT)
+        self._run_test_case([_OUTPUT], {_INPUT: x_val, _INPUT1: t1, _INPUT2: t2})
+
+    @check_opset_min_version(10, "Slice in opset 10 can accept dymaic 'start' and 'ends'")
+    def test_slice_with_size_is_negative_one(self):
+        x_val = np.array([[1, 2, 3, 4], [5, 6, 7, 8]], dtype=np.float32)
+        t1 = np.array([0, 1], dtype=np.int32)
+        # input "size" contains -1
+        t2 = np.array([2, -1], dtype=np.int32)
+        x0 = tf.placeholder(tf.float32, x_val.shape, name=_TFINPUT)
+        t1_ = tf.placeholder(tf.int32, t1.shape, name=_TFINPUT1)
+        t2_ = tf.placeholder(tf.int32, t2.shape, name=_TFINPUT2)
+        x_ = tf.slice(x0, t1_, t2_)
+        _ = tf.identity(x_, name=_TFOUTPUT)
+        self._run_test_case([_OUTPUT], {_INPUT: x_val, _INPUT1: t1, _INPUT2: t2})
+
+    @skip_caffe2_backend()
+    def test_slice1(self):
+        # FIXME: only 1 dimension supported by caffe2 and msrt
+        x_val = np.array([[[1, 1, 1], [2, 2, 2]], [[3, 3, 3], [4, 4, 4]], [[5, 5, 5], [6, 6, 6]]], dtype=np.float32)
+        t1 = tf.constant([1, 0, 0], dtype=tf.int32)
+        t2 = tf.constant([1, 1, 3], dtype=tf.int32)
+        x0 = tf.placeholder(tf.float32, x_val.shape, name=_TFINPUT)
+        x_ = tf.slice(x0, t1, t2)
+        _ = tf.identity(x_, name=_TFOUTPUT)
+        self._run_test_case([_OUTPUT], {_INPUT: x_val})
+
     def test_split(self):
         x_val = np.linspace(1.0, 5 * 30.0, 5 * 30).astype(np.float32).reshape(5, 30)
         x0 = tf.placeholder(tf.float32, x_val.shape, name=_TFINPUT)
@@ -1103,17 +1139,6 @@ def test_reducemean(self):
         _ = tf.identity(x_, name=_TFOUTPUT)
         self._run_test_case([_OUTPUT], {_INPUT: x_val})
 
-    @unittest.skip("")
-    def test_slice1(self):
-        # FIXME: only 1 dimension supported by caffe2 and msrt
-        x_val = np.array([[[1, 1, 1], [2, 2, 2]], [[3, 3, 3], [4, 4, 4]], [[5, 5, 5], [6, 6, 6]]], dtype=np.float32)
-        t1 = tf.constant([1, 0, 0], dtype=tf.int32)
-        t2 = tf.constant([1, 1, 3], dtype=tf.int32)
-        x0 = tf.placeholder(tf.float32, x_val.shape, name=_TFINPUT)
-        x_ = tf.slice(x0, t1, t2)
-        _ = tf.identity(x_, name=_TFOUTPUT)
-        self._run_test_case([_OUTPUT], {_INPUT: x_val})
-
     @skip_caffe2_backend()
     @check_onnxruntime_incompatibility("Pow")
     def test_pow_scalar(self):

tf2onnx/__init__.py

@@ -6,8 +6,8 @@
 from __future__ import print_function
 from __future__ import unicode_literals
 
-__all__ = ["utils", "graph_matcher", "graph", "loader", "tfonnx", "shape_inference", "schemas"]
+__all__ = ["utils", "graph_matcher", "graph", "graph_builder", "loader", "tfonnx", "shape_inference", "schemas"]
 
 from .version import version as __version__
 from . import verbose_logging as logging
-from tf2onnx import tfonnx, utils, graph, graph_matcher, shape_inference, schemas  # pylint: disable=wrong-import-order
+from tf2onnx import tfonnx, utils, graph, graph_builder, graph_matcher, shape_inference, schemas  # pylint: disable=wrong-import-order

tf2onnx/graph.py

@@ -20,6 +20,7 @@
 from tf2onnx.utils import port_name, find_opset
 from tf2onnx import optimizer
 from tf2onnx.schemas import get_schema, infer_onnx_shape_dtype
+from tf2onnx import constants
 
 logger = logging.getLogger(__name__)
 
@@ -419,7 +420,8 @@ def make_const(self, name, np_val, skip_conversion=False, raw=True):
         return node
 
     def make_node(self, op_type, inputs, attr=None, output_count=1, outputs=None, skip_conversion=True,
-                  op_name_scope=None, name=None, shapes=None, dtypes=None, domain=None, infer_shape_dtype=True):
+                  op_name_scope=None, name=None, shapes=None, dtypes=None, domain=constants.ONNX_DOMAIN,
+                  infer_shape_dtype=True):
         """Make a new onnx node in the graph"""
         if attr is None:
             attr = {}

tf2onnx/graph_builder.py

@@ -0,0 +1,104 @@
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT license.
+
+"""
+tf2onnx.graph_helper - class to help building graph, such as helping to make complex node
+"""
+
+import numpy as np
+from tf2onnx import utils, logging
+
+
+# pylint: disable=missing-docstring
+
+
+logger = logging.getLogger(__name__)
+
+
+class GraphBuilder(object):
+    """help to build graph"""
+    def __init__(self, graph):
+        self._g = graph
+
+    @property
+    def graph(self):
+        return self._g
+
+    def make_slice(self, kwargs, name=None, shapes=None, dtypes=None):
+        """
+        slice changes its schema at opset 10: it treats some attributes as dynamic input
+        so this function has to process inputs according to graph's opset version
+        to get "inputs" and "attr" to feed "make_node"
+        kwargs: key could be ["data", "starts", "ends", "axes", "steps", "outputs"].
+        """
+        outputs = kwargs.pop("outputs", None)
+
+        if self.graph.opset < 10:
+            # "data" is string
+            # "starts", "ends" and "axes" are attributes, and "axes" is optional.
+            inputs = [kwargs.pop("data")]
+            starts = self.convert_to_attribute(kwargs.pop("starts"))
+            ends = self.convert_to_attribute(kwargs.pop("ends"))
+            axes = self.convert_to_attribute(kwargs.pop("axes", None), is_optional=True)
+            attr = {"starts": starts, "ends": ends, "axes": axes}
+        else:
+            # slice-10 has 3 required inputs "data", "starts", "ends"l
+            # and 2 optional inputs "axes", "steps"
+            # input sequence should be "data", "starts", "ends", "axes", "steps"
+            attr = {}
+            data = self.convert_to_input(kwargs.pop("data"))
+            starts = self.convert_to_input(kwargs.pop("starts"))
+            ends = self.convert_to_input(kwargs.pop("ends"))
+            axes = self.convert_to_input(kwargs.pop("axes", None), is_optional=True)
+            steps = self.convert_to_input(kwargs.pop("steps", None), is_optional=True)
+            inputs = [data, starts, ends, axes, steps]
+
+        # pro-process inputs and attr
+        if kwargs:
+            logger.warning("kwargs contains un-used key")
+
+        new_attr = {}
+        for key, val in attr.items():
+            if val is not None:
+                new_attr[key] = val
+        attr = new_attr
+
+        for ind, val in enumerate(inputs):
+            if val is None:
+                inputs[ind] = ""  # empty string means no connection in ONNX
+        # remove tailing ""
+        while inputs[-1] == "":
+            inputs = inputs[:-1]
+
+        return self.graph.make_node(op_type="Slice", inputs=inputs, attr=attr, name=name,
+                                    outputs=outputs, shapes=shapes, dtypes=dtypes).output[0]
+
+    def convert_to_input(self, tensor, is_optional=False):
+        """in ONNX, input shold come from node, so it must be a string"""
+        if is_optional and tensor is None:
+            return None
+
+        utils.make_sure(tensor is not None, "input is required so it couldn't be None")
+
+        res = tensor
+        if isinstance(tensor, list):
+            res = self.graph.make_const(utils.make_name("const_slice"), np.array(tensor)).output[0]
+
+        utils.make_sure(isinstance(res, str), "input is a dynamic input, so a str is needed")
+
+        return res
+
+    def convert_to_attribute(self, tensor, is_optional=False):
+        if is_optional and tensor is None:
+            return None
+
+        utils.make_sure(tensor is not None, "input is required so it couldn't be None")
+
+        res = tensor
+        if isinstance(tensor, str):
+            const_node = self.graph.get_node_by_output(tensor)
+            res = const_node.get_tensor_value(as_list=True)
+
+        utils.make_sure(isinstance(res, list), "input is an attr, so a list is needed")
+
+        return res

tf2onnx/onnx_opset/nn.py

@@ -15,6 +15,7 @@
 from onnx import onnx_pb
 from onnx.onnx_pb import TensorProto
 from tf2onnx import constants, utils
+from tf2onnx.graph_builder import GraphBuilder
 from tf2onnx.handler import tf_op
 from tf2onnx.onnx_opset import common, controlflow, tensor
 
@@ -463,6 +464,14 @@ def version_7(cls, ctx, node, **kwargs):
 
     @classmethod
     def version_9(cls, ctx, node, **kwargs):
+        cls._convert_since_9(ctx, node, **kwargs)
+
+    @classmethod
+    def version_10(cls, ctx, node, **kwargs):
+        cls._convert_since_9(ctx, node, **kwargs)
+
+    @classmethod
+    def _convert_since_9(cls, ctx, node, **kwargs):
         # float32 out = ResizeBilinear/ResizeNearestNeighbor(T images, int size)
         # https://www.tensorflow.org/api_docs/python/tf/image/resize_nearest_neighbor
         # wants the input to be NHWC - adjust target_shape to this.
@@ -481,8 +490,10 @@ def version_9(cls, ctx, node, **kwargs):
             scales = ctx.make_const(utils.make_name("scales"), scale_val, raw=False)
         else:
             ori_shape = ctx.make_node("Shape", [node.input[0]])
-            ori_shape_hw = ctx.make_node("Slice", ori_shape.output, {"axes": [0], "starts": [1], "ends": [3]})
-            ori_shape_hw_float = ctx.make_node("Cast", ori_shape_hw.output, attr={"to": onnx_pb.TensorProto.FLOAT})
+            attr = {"axes": [0], "starts": [1], "ends": [3]}
+            inputs_map = {"data": ori_shape.output[0], **attr}
+            ori_shape_hw = GraphBuilder(ctx).make_slice(inputs_map)
+            ori_shape_hw_float = ctx.make_node("Cast", [ori_shape_hw], attr={"to": onnx_pb.TensorProto.FLOAT})
 
             target_hw = node.inputs[1]
             target_hw_float = ctx.make_node("Cast", target_hw.output, attr={"to": onnx_pb.TensorProto.FLOAT})
@@ -538,12 +549,13 @@ def version_7(cls, ctx, node, **kwargs):
         new_line = g.make_node(op_type="Concat", inputs=[const_zero_bool.output[0], "line"],
                                attr={"axis": counter_axis},
                                dtypes=[onnx_pb.TensorProto.BOOL])
-        slice_node = g.make_node(op_type="Slice", inputs=[new_line.output[0]],
-                                 attr={"axes": [counter_axis], "starts": [0], "ends": [-1]})
+        attr = {"axes": [counter_axis], "starts": [0], "ends": [-1]}
+        inputs_map = {"data": new_line.output[0], **attr}
+        slice_node = GraphBuilder(g).make_slice(inputs_map)
 
         g.make_node("Identity", ["cond"], outputs=["cond_out"])
         g.make_node("Identity", ["line"], outputs=["res"])
-        g.make_node("Identity", [slice_node.output[0]], outputs=["line_out"])
+        g.make_node("Identity", [slice_node], outputs=["line_out"])
 
         g.add_graph_input("trip", onnx_pb.TensorProto.INT64, [])
         g.add_graph_input("cond", onnx_pb.TensorProto.BOOL, [])