sync from master

Author: pengwa

.gitignore

@@ -1,4 +1,4 @@
-.coverage
+.coverage*
 *.pyc
 .idea
 build

README.md

@@ -33,15 +33,15 @@ onnxruntime (only avaliable on linux):
 
 ```pip install onnxruntime```
 
-For caffe2, follow the instructions here:
+For pytorch/caffe2, follow the instructions here:
 
-```https://caffe2.ai/```
+```https://pytorch.org/```
 
 
-We tested with caffe2 and onnxruntime and unit tests are passing for those.
+We tested with pytorch/caffe2 and onnxruntime and unit tests are passing for those.
 
 ## Supported Tensorflow and Python Versions
-We tested with tensorflow 1.5-1.11 and anaconda **3.5,3.6**.
+We tested with tensorflow 1.5-1.12 and anaconda **3.5,3.6**.
 
 # Installation
 ## From Pypi
@@ -64,13 +64,17 @@ python setup.py bdist_wheel
 
 # Usage
 
-To convert a TensorFlow model, tf2onnx expects a ```frozen TensorFlow graph``` and the user needs to specify inputs and outputs for the graph by passing the input and output
+To convert a TensorFlow model, tf2onnx prefers a ```frozen TensorFlow graph``` and the user needs to specify inputs and outputs for the graph by passing the input and output
 names with ```--inputs INPUTS``` and ```--outputs OUTPUTS```. 
 
 ```
-python -m tf2onnx.convert --input SOURCE_FROZEN_GRAPH_PB
-    --inputs SOURCE_GRAPH_INPUTS
-    --outputs SOURCE_GRAPH_OUTPUS
+python -m tf2onnx.convert 
+    --input SOURCE_GRAPHDEF_PB
+    --graphdef SOURCE_GRAPHDEF_PB
+    --checkpoint SOURCE_CHECKPOINT
+    --saved-model SOURCE_SAVED_MODEL
+    [--inputs GRAPH_INPUTS]
+    [--outputs GRAPH_OUTPUS]
     [--inputs-as-nchw inputs_provided_as_nchw]
     [--target TARGET]
     [--output TARGET_ONNX_GRAPH]
@@ -83,21 +87,26 @@ python -m tf2onnx.convert --input SOURCE_FROZEN_GRAPH_PB
 ```
 
 ## Parameters
-### input
-frozen TensorFlow graph, which can be created with the [freeze graph tool](#freeze_graph).
-### output 
+### --input or --graphdef
+TensorFlow model as graphdef file. If not already frozen we'll try to freeze the model.
+More information about freezing can be found here: [freeze graph tool](#freeze_graph).
+### --checkpoint
+TensorFlow model as checkpoint. We expect the path to the .meta file. tf2onnx will try to freeze the graph.
+### --saved-model
+TensorFlow model as saved_model. We expect the path to the saved_model directory. tf2onnx will try to freeze the graph.
+### --output 
 the target onnx file path.
-### inputs, outputs
-Tensorflow graph's input/output names, which can be found with [summarize graph tool](#summarize_graph). Those names typically end on ```:0```, for example ```--inputs input0:0,input1:0```
-### inputs-as-nchw
+### --inputs, --outputs
+Tensorflow model's input/output names, which can be found with [summarize graph tool](#summarize_graph). Those names typically end on ```:0```, for example ```--inputs input0:0,input1:0```. inputs and outputs are ***not*** needed for models in saved-model format.
+### --inputs-as-nchw
 By default we preserve the image format of inputs (nchw or nhwc) as given in the TensorFlow model. If your hosts (for example windows) native format nchw and the model is written for nhwc, ```--inputs-as-nchw``` tensorflow-onnx will transpose the input. Doing so is convinient for the application and the converter in many cases can optimize the transpose away. For example ```--inputs input0:0,input1:0 --inputs-as-nchw input0:0``` assumes that images are passed into ```input0:0``` as nchw while the TensorFlow model given uses nhwc.
-### target 
+### --target 
 Some runtimes need workarounds, for example they don't support all types given in the onnx spec. We'll workaround it in some cases by generating a different graph. Those workarounds are activated with ```--target TARGET```. 
-### opset
+### --opset
 by default we uses the newest opset 7 to generate the graph. By specifieing ```--opset``` the user can override the default to generate a graph with the desired opset. For example ```--opset 5``` would create a onnx graph that uses only ops available in opset 5. Because older opsets have in most cases fewer ops, some models might not convert on a older opset.
-### custom-ops
+### --custom-ops
 the runtime may support custom ops that are not defined in onnx. A user can asked the converter to map to custom ops by listing them with the --custom-ops option. Tensorflow ops listed here will be mapped to a custom op with the same name as the tensorflow op but in the onnx domain ai.onnx.converters.tensorflow. For example: ```--custom-ops Print``` will insert a op ```Print``` in the onnx domain ```ai.onnx.converters.tensorflow``` into the graph. We also support a python api for custom ops documented later in this readme. 
-### fold_const
+### --fold_const
 when set, TensorFlow fold_constants transformation will be applied before conversion. This will benefit features including Transpose optimization (e.g. Transpose operations introduced during tf-graph-to-onnx-graph conversion will be removed), and RNN unit conversion (for example LSTM). Older TensorFlow version might run into issues with this option depending on the model.
 
 

ci_build/azure_pipelines/onnxruntime_nightly_test.yml

@@ -0,0 +1,15 @@
+# Test against latest onnxruntime nightly package
+
+# TODO: change onnxruntime package name when nightly package is ready
+
+jobs:
+- template: 'templates/job_generator.yml'
+  parameters:
+    tf_versions: ['1.12']
+    onnx_opsets: ['9', '8', '7']
+    onnx_backends:
+      onnxruntime: ['']
+    job:
+      displayName: 'unit_test'
+      steps:
+      - template: 'unit_test.yml'

ci_build/azure_pipelines/templates/setup.yml

@@ -13,12 +13,18 @@ steps:
         pip install $(CI_PIP_ONNX_NAME) $(CI_PIP_ONNX_BACKEND_NAME) numpy --no-deps -U
     fi
 
+    # HACK: before onnxruntime nightly package is ready, install onnxruntime from pypi test
+    if [[ $CI_ONNXRUNTIME_FROM_PYPI_TEST == "true" ]] ;
+    then
+      pip install --index-url https://test.pypi.org/simple/ onnxruntime --force-reinstall
+    fi
+
     python setup.py install
     pip freeze --all
   displayName: 'Setup Environment'
 
 # TODO: remove later
-# Anaconda python 3.6.8 h9f7ef89_1 changed dll lookup logic, numpy failes to load dll on Windows
+# Anaconda python 3.6.8 h9f7ef89_1 changed dll lookup logic, numpy fails to load dll on Windows
 # https://github.com/numpy/numpy/issues/12957
 # https://github.com/ContinuumIO/anaconda-issues/issues/10629
 # Add numpy lib path manually here

tests/backend_test_base.py

@@ -40,6 +40,14 @@ def setUp(self):
             tf.logging.set_verbosity(tf.logging.WARN)
             self.log.setLevel(logging.INFO)
 
+    def tearDown(self):
+        if not self.config.is_debug_mode:
+            utils.delete_directory(self.test_data_directory)
+
+    @property
+    def test_data_directory(self):
+        return os.path.join(self.config.temp_dir, self._testMethodName)
+
     @staticmethod
     def assertAllClose(expected, actual, **kwargs):
         np.testing.assert_allclose(expected, actual, **kwargs)
@@ -72,6 +80,7 @@ def run_onnxruntime(self, model_path, inputs, output_names):
     def _run_backend(self, g, outputs, input_dict):
         model_proto = g.make_model("test")
         model_path = self.save_onnx_model(model_proto, input_dict)
+
         if self.config.backend == "onnxmsrtnext":
             y = self.run_onnxmsrtnext(model_path, input_dict, outputs)
         elif self.config.backend == "onnxruntime":
@@ -84,7 +93,7 @@ def _run_backend(self, g, outputs, input_dict):
 
     def run_test_case(self, feed_dict, input_names_with_port, output_names_with_port, rtol=1e-07, atol=0.,
                       convert_var_to_const=True, constant_fold=True, check_value=True, check_shape=False,
-                      check_dtype=False, process_args=None, onnx_feed_dict=None):
+                      check_dtype=True, process_args=None, onnx_feed_dict=None):
         # optional - passed to process_tf_graph
         if process_args is None:
             process_args = {}
@@ -93,8 +102,6 @@ def run_test_case(self, feed_dict, input_names_with_port, output_names_with_port
             onnx_feed_dict = feed_dict
 
         graph_def = None
-        save_dir = os.path.join(self.config.temp_path, self._testMethodName)
-
         if convert_var_to_const:
             with tf.Session() as sess:
                 variables_lib.global_variables_initializer().run()
@@ -113,20 +120,18 @@ def run_test_case(self, feed_dict, input_names_with_port, output_names_with_port
             expected = sess.run(output_dict, feed_dict=feed_dict)
 
         if self.config.is_debug_mode:
-            if not os.path.exists(save_dir):
-                os.makedirs(save_dir)
-            model_path = os.path.join(save_dir, self._testMethodName + "_original.pb")
-            with open(model_path, "wb") as f:
-                f.write(sess.graph_def.SerializeToString())
+            if not os.path.exists(self.test_data_directory):
+                os.makedirs(self.test_data_directory)
+            model_path = os.path.join(self.test_data_directory, self._testMethodName + "_original.pb")
+            utils.save_protobuf(model_path, sess.graph_def)
             self.log.debug("created file %s", model_path)
 
         graph_def = tf_optimize(input_names_with_port, output_names_with_port,
                                 sess.graph_def, constant_fold)
 
         if self.config.is_debug_mode and constant_fold:
-            model_path = os.path.join(save_dir, self._testMethodName + "_after_tf_optimize.pb")
-            with open(model_path, "wb") as f:
-                f.write(graph_def.SerializeToString())
+            model_path = os.path.join(self.test_data_directory, self._testMethodName + "_after_tf_optimize.pb")
+            utils.save_protobuf(model_path, graph_def)
             self.log.debug("created file  %s", model_path)
 
         tf.reset_default_graph()
@@ -146,9 +151,8 @@ def run_test_case(self, feed_dict, input_names_with_port, output_names_with_port
                 self.assertEqual(expected_val.shape, actual_val.shape)
 
     def save_onnx_model(self, model_proto, feed_dict, postfix=""):
-        save_path = os.path.join(self.config.temp_path, self._testMethodName)
-        target_path = utils.save_onnx_model(save_path, self._testMethodName + postfix, feed_dict, model_proto,
-                                            include_test_data=self.config.is_debug_mode,
+        target_path = utils.save_onnx_model(self.test_data_directory, self._testMethodName + postfix, feed_dict,
+                                            model_proto, include_test_data=self.config.is_debug_mode,
                                             as_text=self.config.is_debug_mode)
 
         self.log.debug("create model file: %s", target_path)

tests/common.py

@@ -6,15 +6,16 @@
 import argparse
 import os
 import sys
-import tempfile
 import unittest
 
 from distutils.version import LooseVersion
+from tf2onnx import utils
 from tf2onnx.tfonnx import DEFAULT_TARGET, POSSIBLE_TARGETS
 
 __all__ = ["TestConfig", "get_test_config", "unittest_main",
-           "check_tf_min_version", "check_opset_min_version", "check_target", "skip_onnxruntime_backend",
-           "skip_caffe2_backend", "check_onnxruntime_incompatibility"]
+           "check_tf_min_version", "skip_tf_versions",
+           "check_opset_min_version", "check_target", "skip_onnxruntime_backend", "skip_caffe2_backend",
+           "check_onnxruntime_incompatibility"]
 
 
 # pylint: disable=missing-docstring
@@ -23,12 +24,12 @@ class TestConfig(object):
     def __init__(self):
         self.platform = sys.platform
         self.tf_version = self._get_tf_version()
-        self.opset = int(os.environ.get("TF2ONNX_TEST_OPSET", 7))
+        self.opset = int(os.environ.get("TF2ONNX_TEST_OPSET", 9))
         self.target = os.environ.get("TF2ONNX_TEST_TARGET", ",".join(DEFAULT_TARGET)).split(',')
         self.backend = os.environ.get("TF2ONNX_TEST_BACKEND", "onnxruntime")
         self.backend_version = self._get_backend_version()
         self.is_debug_mode = False
-        self.temp_path = tempfile.mkdtemp()
+        self.temp_dir = utils.get_temp_directory()
 
     @property
     def is_mac(self):
@@ -67,28 +68,32 @@ def __str__(self):
                             "target={}".format(self.target),
                             "backend={}".format(self.backend),
                             "backend_version={}".format(self.backend_version),
-                            "is_debug_mode={}".format(self.is_debug_mode)])
+                            "is_debug_mode={}".format(self.is_debug_mode),
+                            "temp_dir={}".format(self.temp_dir)])
 
     @staticmethod
     def load():
         config = TestConfig()
         # if not launched by pytest, parse console arguments to override config
         if "pytest" not in sys.argv[0]:
             parser = argparse.ArgumentParser()
-            parser.add_argument('--backend', default=config.backend,
+            parser.add_argument("--backend", default=config.backend,
                                 choices=["caffe2", "onnxmsrtnext", "onnxruntime"],
                                 help="backend to test against")
-            parser.add_argument('--opset', type=int, default=config.opset, help="opset to test against")
+            parser.add_argument("--opset", type=int, default=config.opset, help="opset to test against")
             parser.add_argument("--target", default=",".join(config.target), choices=POSSIBLE_TARGETS,
                                 help="target platform")
             parser.add_argument("--debug", help="output debugging information", action="store_true")
-            parser.add_argument('unittest_args', nargs='*')
+            parser.add_argument("--temp_dir", help="temp dir")
+            parser.add_argument("unittest_args", nargs='*')
 
             args = parser.parse_args()
             config.backend = args.backend
             config.opset = args.opset
             config.target = args.target.split(',')
             config.is_debug_mode = args.debug
+            if args.temp_dir:
+                config.temp_dir = args.temp_dir
 
             # Now set the sys.argv to the unittest_args (leaving sys.argv[0] alone)
             sys.argv[1:] = args.unittest_args

tests/models/regression/checkpoint/checkpoint

@@ -0,0 +1,2 @@
+model_checkpoint_path: "model"
+all_model_checkpoint_paths: "model"

tests/models/regression/checkpoint/model.data-00000-of-00001

@@ -0,0 +1 @@
+�[>�V�?