Merge pull request #354 from onnx/gs/more_tf_formats

support for checkpoint and saved_model format

Author: guschmue

README.md

@@ -33,15 +33,15 @@ onnxruntime (only avaliable on linux):
 
 ```pip install onnxruntime```
 
-For caffe2, follow the instructions here:
+For pytorch/caffe2, follow the instructions here:
 
-```https://caffe2.ai/```
+```https://pytorch.org/```
 
 
-We tested with caffe2 and onnxruntime and unit tests are passing for those.
+We tested with pytorch/caffe2 and onnxruntime and unit tests are passing for those.
 
 ## Supported Tensorflow and Python Versions
-We tested with tensorflow 1.5-1.11 and anaconda **3.5,3.6**.
+We tested with tensorflow 1.5-1.12 and anaconda **3.5,3.6**.
 
 # Installation
 ## From Pypi
@@ -64,13 +64,17 @@ python setup.py bdist_wheel
 
 # Usage
 
-To convert a TensorFlow model, tf2onnx expects a ```frozen TensorFlow graph``` and the user needs to specify inputs and outputs for the graph by passing the input and output
+To convert a TensorFlow model, tf2onnx prefers a ```frozen TensorFlow graph``` and the user needs to specify inputs and outputs for the graph by passing the input and output
 names with ```--inputs INPUTS``` and ```--outputs OUTPUTS```. 
 
 ```
-python -m tf2onnx.convert --input SOURCE_FROZEN_GRAPH_PB
-    --inputs SOURCE_GRAPH_INPUTS
-    --outputs SOURCE_GRAPH_OUTPUS
+python -m tf2onnx.convert 
+    --input SOURCE_GRAPHDEF_PB
+    --graphdef SOURCE_GRAPHDEF_PB
+    --checkpoint SOURCE_CHECKPOINT
+    --saved-model SOURCE_SAVED_MODEL
+    [--inputs GRAPH_INPUTS]
+    [--outputs GRAPH_OUTPUS]
     [--inputs-as-nchw inputs_provided_as_nchw]
     [--target TARGET]
     [--output TARGET_ONNX_GRAPH]
@@ -83,21 +87,26 @@ python -m tf2onnx.convert --input SOURCE_FROZEN_GRAPH_PB
 ```
 
 ## Parameters
-### input
-frozen TensorFlow graph, which can be created with the [freeze graph tool](#freeze_graph).
-### output 
+### --input or --graphdef
+TensorFlow model as graphdef file. If not already frozen we'll try to freeze the model.
+More information about freezing can be found here: [freeze graph tool](#freeze_graph).
+### --checkpoint
+TensorFlow model as checkpoint. We expect the path to the .meta file. tf2onnx will try to freeze the graph.
+### --saved-model
+TensorFlow model as saved_model. We expect the path to the saved_model directory. tf2onnx will try to freeze the graph.
+### --output 
 the target onnx file path.
-### inputs, outputs
-Tensorflow graph's input/output names, which can be found with [summarize graph tool](#summarize_graph). Those names typically end on ```:0```, for example ```--inputs input0:0,input1:0```
-### inputs-as-nchw
+### --inputs, --outputs
+Tensorflow model's input/output names, which can be found with [summarize graph tool](#summarize_graph). Those names typically end on ```:0```, for example ```--inputs input0:0,input1:0```. inputs and outputs are ***not*** needed for models in saved-model format.
+### --inputs-as-nchw
 By default we preserve the image format of inputs (nchw or nhwc) as given in the TensorFlow model. If your hosts (for example windows) native format nchw and the model is written for nhwc, ```--inputs-as-nchw``` tensorflow-onnx will transpose the input. Doing so is convinient for the application and the converter in many cases can optimize the transpose away. For example ```--inputs input0:0,input1:0 --inputs-as-nchw input0:0``` assumes that images are passed into ```input0:0``` as nchw while the TensorFlow model given uses nhwc.
-### target 
+### --target 
 Some runtimes need workarounds, for example they don't support all types given in the onnx spec. We'll workaround it in some cases by generating a different graph. Those workarounds are activated with ```--target TARGET```. 
-### opset
+### --opset
 by default we uses the newest opset 7 to generate the graph. By specifieing ```--opset``` the user can override the default to generate a graph with the desired opset. For example ```--opset 5``` would create a onnx graph that uses only ops available in opset 5. Because older opsets have in most cases fewer ops, some models might not convert on a older opset.
-### custom-ops
+### --custom-ops
 the runtime may support custom ops that are not defined in onnx. A user can asked the converter to map to custom ops by listing them with the --custom-ops option. Tensorflow ops listed here will be mapped to a custom op with the same name as the tensorflow op but in the onnx domain ai.onnx.converters.tensorflow. For example: ```--custom-ops Print``` will insert a op ```Print``` in the onnx domain ```ai.onnx.converters.tensorflow``` into the graph. We also support a python api for custom ops documented later in this readme. 
-### fold_const
+### --fold_const
 when set, TensorFlow fold_constants transformation will be applied before conversion. This will benefit features including Transpose optimization (e.g. Transpose operations introduced during tf-graph-to-onnx-graph conversion will be removed), and RNN unit conversion (for example LSTM). Older TensorFlow version might run into issues with this option depending on the model.
 
 

tests/models/regression/checkpoint/checkpoint

@@ -0,0 +1,2 @@
+model_checkpoint_path: "model"
+all_model_checkpoint_paths: "model"

tests/models/regression/checkpoint/model.data-00000-of-00001

@@ -0,0 +1 @@
+�[>�V�?

tests/models/regression/checkpoint/model.index

@@ -0,0 +1 @@
+�[>�V�?

tests/models/regression/checkpoint/model.meta

@@ -16,19 +16,18 @@
 import traceback
 import zipfile
 
+import PIL.Image
 import numpy as np
 import requests
 import six
 import tensorflow as tf
-from tensorflow.core.framework import graph_pb2
-from tensorflow.python.framework.graph_util import convert_variables_to_constants
 # contrib ops are registered only when the module is imported, the following import statement is needed,
 # otherwise tf runtime error will show up when the tf model is restored from pb file because of un-registered ops.
 import tensorflow.contrib.rnn  # pylint: disable=unused-import
 import yaml
-import PIL.Image
 
 import tf2onnx
+from tf2onnx import loader
 from tf2onnx import utils
 from tf2onnx.graph import GraphUtil
 from tf2onnx.tfonnx import process_tf_graph
@@ -74,23 +73,6 @@ def get_ramp(shape):
 }
 
 
-def freeze_session(sess, keep_var_names=None, output_names=None, clear_devices=True):
-    """Freezes the state of a session into a pruned computation graph."""
-    output_names = [i.replace(":0", "") for i in output_names]
-    graph = sess.graph
-    with graph.as_default():
-        freeze_var_names = list(set(v.op.name for v in tf.global_variables()).difference(keep_var_names or []))
-        output_names = output_names or []
-        output_names += [v.op.name for v in tf.global_variables()]
-        input_graph_def = graph.as_graph_def()
-        if clear_devices:
-            for node in input_graph_def.node:
-                node.device = ""
-        frozen_graph = convert_variables_to_constants(sess, input_graph_def,
-                                                      output_names, freeze_var_names)
-        return frozen_graph
-
-
 class Test(object):
     """Main Test class."""
 
@@ -236,44 +218,14 @@ def run_test(self, name, backend="caffe2", debug=False, onnx_file=None, opset=No
             dir_name = os.path.dirname(self.local)
         print("\tdownloaded", model_path)
 
+        inputs = list(self.input_names.keys())
+        outputs = self.output_names
         if self.model_type in ["checkpoint"]:
-            #
-            # if the input model is a checkpoint, convert it to a frozen model
-            saver = tf.train.import_meta_graph(model_path)
-            with tf.Session() as sess:
-                saver.restore(sess, model_path[:-5])
-                frozen_graph = freeze_session(sess, output_names=self.output_names)
-                tf.train.write_graph(frozen_graph, dir_name, "frozen.pb", as_text=False)
-            model_path = os.path.join(dir_name, "frozen.pb")
+            graph_def, inputs, outputs = loader.from_checkpoint(model_path, inputs, outputs)
         elif self.model_type in ["saved_model"]:
-            try:
-                from tensorflow.contrib.saved_model.python.saved_model import signature_def_utils
-                get_signature_def = lambda meta_graph_def, k: \
-                    signature_def_utils.get_signature_def_by_key(meta_graph_def, k)
-            except ImportError:
-                # TF1.12 changed the api
-                get_signature_def = lambda meta_graph_def, k: meta_graph_def.signature_def[k]
-
-            # saved_model format - convert to checkpoint
-            with tf.Session() as sess:
-                meta_graph_def = tf.saved_model.loader.load(sess, [tf.saved_model.tag_constants.SERVING], model_path)
-                inputs = {}
-                outputs = {}
-                for k in meta_graph_def.signature_def.keys():
-                    inputs_tensor_info = get_signature_def(meta_graph_def, k).inputs
-                    for _, input_tensor in sorted(inputs_tensor_info.items()):
-                        inputs[input_tensor.name] = sess.graph.get_tensor_by_name(input_tensor.name)
-                    outputs_tensor_info = get_signature_def(meta_graph_def, k).outputs
-                    for _, output_tensor in sorted(outputs_tensor_info.items()):
-                        outputs[output_tensor.name] = sess.graph.get_tensor_by_name(output_tensor.name)
-                # freeze uses the node name derived from output:0 so only pass in output:0;
-                # it will provide all outputs of that node.
-                for o in list(outputs.keys()):
-                    if not o.endswith(":0"):
-                        del outputs[o]
-                frozen_graph = freeze_session(sess, output_names=list(outputs.keys()))
-                tf.train.write_graph(frozen_graph, dir_name, "frozen.pb", as_text=False)
-            model_path = os.path.join(dir_name, "frozen.pb")
+            graph_def, inputs, outputs = loader.from_saved_model(model_path, inputs, outputs)
+        else:
+            graph_def, inputs, outputs = loader.from_graphdef(model_path, inputs, outputs)
 
         # create the input data
         inputs = {}
@@ -285,10 +237,6 @@ def run_test(self, name, backend="caffe2", debug=False, onnx_file=None, opset=No
         if self.more_inputs:
             for k, v in self.more_inputs.items():
                 inputs[k] = v
-        tf.reset_default_graph()
-        graph_def = graph_pb2.GraphDef()
-        with open(model_path, "rb") as f:
-            graph_def.ParseFromString(f.read())
 
         graph_def = tf2onnx.tfonnx.tf_optimize(inputs.keys(), self.output_names, graph_def, fold_const)
         shape_override = {}