|
102 | 102 | "id": "yt5HEbsYAbw1" |
103 | 103 | }, |
104 | 104 | "source": [ |
105 | | - "## Defining models and layers in TensorFlow\n", |
| 105 | + "## TensorFlow Modules\n", |
106 | 106 | "\n", |
107 | 107 | "Most models are made of layers. Layers are functions with a known mathematical structure that can be reused and have trainable variables. In TensorFlow, most high-level implementations of layers and models, such as Keras or [Sonnet](https://github.com/deepmind/sonnet), are built on the same foundational class: `tf.Module`.\n", |
108 | 108 | "\n", |
| 109 | + "### Building Modules\n", |
| 110 | + "\n", |
109 | 111 | "Here's an example of a very simple `tf.Module` that operates on a scalar tensor:\n" |
110 | 112 | ] |
111 | 113 | }, |
|
337 | 339 | "id": "JOLVVBT8J_dl" |
338 | 340 | }, |
339 | 341 | "source": [ |
340 | | - "## Saving weights\n", |
| 342 | + "### Saving weights\n", |
341 | 343 | "\n", |
342 | 344 | "You can save a `tf.Module` as both a [checkpoint](./checkpoint.ipynb) and a [SavedModel](./saved_model.ipynb).\n", |
343 | 345 | "\n", |
|
439 | 441 | "id": "pSZebVuWxDXu" |
440 | 442 | }, |
441 | 443 | "source": [ |
442 | | - "## Saving functions\n", |
| 444 | + "### Saving functions\n", |
443 | 445 | "\n", |
444 | 446 | "TensorFlow can run models without the original Python objects, as demonstrated by [TensorFlow Serving](https://tensorflow.org/tfx) and [TensorFlow Lite](https://tensorflow.org/lite), even when you download a trained model from [TensorFlow Hub](https://tensorflow.org/hub).\n", |
445 | 447 | "\n", |
|
699 | 701 | "In this section, you will examine how Keras uses `tf.Module`. A complete user guide to Keras models can be found in the [Keras guide](https://www.tensorflow.org/guide/keras/sequential_model).\n" |
700 | 702 | ] |
701 | 703 | }, |
| 704 | + { |
| 705 | + "metadata": { |
| 706 | + "id": "ds08u3touwe4t" |
| 707 | + }, |
| 708 | + "cell_type": "markdown", |
| 709 | + "source": [ |
| 710 | + "Keras layers and models have a lot more extra features including:\n", |
| 711 | + "\n", |
| 712 | + "* Optional losses\n", |
| 713 | + "* Support for [metrics](https://keras.io/api/layers/base_layer/#add_metric-method)\n", |
| 714 | + "* Built-in support for an optional `training` argument to differentiate between training and inference use\n", |
| 715 | + "* Saving and restoring python objects instead of just black-box functions\n", |
| 716 | + "* `get_config` and `from_config` methods that allow you to accurately store configurations to allow model cloning in Python\n", |
| 717 | + "\n", |
| 718 | + "These features allow for far more complex models through subclassing, such as a custom GAN or a Variational AutoEncoder (VAE) model. Read about them in the [full guide](./keras/custom_layers_and_models.ipynb) to custom layers and models.\n", |
| 719 | + "\n", |
| 720 | + "Keras models also come with extra functionality that makes them easy to train, evaluate, load, save, and even train on multiple machines." |
| 721 | + ] |
| 722 | + }, |
702 | 723 | { |
703 | 724 | "cell_type": "markdown", |
704 | 725 | "metadata": { |
|
874 | 895 | " print(\"Failed:\", e)" |
875 | 896 | ] |
876 | 897 | }, |
877 | | - { |
878 | | - "cell_type": "markdown", |
879 | | - "metadata": { |
880 | | - "id": "YnporXiudF1I" |
881 | | - }, |
882 | | - "source": [ |
883 | | - "Keras layers have a lot more extra features including:\n", |
884 | | - "\n", |
885 | | - "* Optional losses\n", |
886 | | - "* Support for metrics\n", |
887 | | - "* Built-in support for an optional `training` argument to differentiate between training and inference use\n", |
888 | | - "* `get_config` and `from_config` methods that allow you to accurately store configurations to allow model cloning in Python\n", |
889 | | - "\n", |
890 | | - "Read about them in the [full guide](./keras/custom_layers_and_models.ipynb) to custom layers and models." |
891 | | - ] |
892 | | - }, |
893 | 898 | { |
894 | 899 | "cell_type": "markdown", |
895 | 900 | "metadata": { |
|
900 | 905 | "\n", |
901 | 906 | "You can define your model as nested Keras layers.\n", |
902 | 907 | "\n", |
903 | | - "However, Keras also provides a full-featured model class called `tf.keras.Model`. It inherits from `tf.keras.layers.Layer`, so a Keras model can be used, nested, and saved in the same way as Keras layers. Keras models come with extra functionality that makes them easy to train, evaluate, load, save, and even train on multiple machines.\n", |
| 908 | + "However, Keras also provides a full-featured model class called `tf.keras.Model`. It inherits from `tf.keras.layers.Layer`, so a Keras model can be used and nested in the same way as Keras layers. Keras models come with extra functionality that makes them easy to train, evaluate, load, save, and even train on multiple machines.\n", |
904 | 909 | "\n", |
905 | 910 | "You can define the `SequentialModule` from above with nearly identical code, again converting `__call__` to `call()` and changing the parent:" |
906 | 911 | ] |
|
938 | 943 | "source": [ |
939 | 944 | "All the same features are available, including tracking variables and submodules.\n", |
940 | 945 | "\n", |
941 | | - "Note: To emphasize the note above, a raw `tf.Module` nested inside a Keras layer or model will not get its variables collected for training or saving. Instead, nest Keras layers inside of Keras layers." |
| 946 | + "Note: A raw `tf.Module` nested inside a Keras layer or model will not get its variables collected for training or saving. Instead, nest Keras layers inside of Keras layers." |
942 | 947 | ] |
943 | 948 | }, |
944 | 949 | { |
|
1022 | 1027 | "id": "qI9aXLnaHEFF" |
1023 | 1028 | }, |
1024 | 1029 | "source": [ |
1025 | | - "## Saving Keras models\n", |
1026 | | - "\n", |
1027 | | - "Keras models can be checkpointed, and that will look the same as `tf.Module`.\n", |
| 1030 | + "### Saving Keras models\n", |
1028 | 1031 | "\n", |
1029 | | - "Keras models can also be saved with `tf.saved_model.save()`, as they are modules. However, Keras models have convenience methods and other functionality:" |
| 1032 | + "Keras models have their own specialized zip archive saving format, marked by the `.keras` extension. When calling `tf.keras.Model.save`, add a `.keras` extension to the filename. For example:" |
1030 | 1033 | ] |
1031 | 1034 | }, |
1032 | 1035 | { |
|
1037 | 1040 | }, |
1038 | 1041 | "outputs": [], |
1039 | 1042 | "source": [ |
1040 | | - "my_sequential_model.save(\"exname_of_file\")" |
| 1043 | + "my_sequential_model.save(\"exname_of_file.keras\")" |
1041 | 1044 | ] |
1042 | 1045 | }, |
1043 | 1046 | { |
|
1057 | 1060 | }, |
1058 | 1061 | "outputs": [], |
1059 | 1062 | "source": [ |
1060 | | - "reconstructed_model = tf.keras.models.load_model(\"exname_of_file\")" |
| 1063 | + "reconstructed_model = tf.keras.models.load_model(\"exname_of_file.keras\")" |
1061 | 1064 | ] |
1062 | 1065 | }, |
1063 | 1066 | { |
|
1066 | 1069 | "id": "EA7P_MNvpviZ" |
1067 | 1070 | }, |
1068 | 1071 | "source": [ |
1069 | | - "Keras `SavedModels` also save metric, loss, and optimizer states.\n", |
| 1072 | + "Keras zip archives — `.keras` files — also save metric, loss, and optimizer states.\n", |
1070 | 1073 | "\n", |
1071 | 1074 | "This reconstructed model can be used and will produce the same result when called on the same data:" |
1072 | 1075 | ] |
|
1082 | 1085 | "reconstructed_model(tf.constant([[2.0, 2.0, 2.0]]))" |
1083 | 1086 | ] |
1084 | 1087 | }, |
| 1088 | + { |
| 1089 | + "metadata": { |
| 1090 | + "id": "seLIUG2354s" |
| 1091 | + }, |
| 1092 | + "cell_type": "markdown", |
| 1093 | + "source": [ |
| 1094 | + "### Checkpointing Keras models\n", |
| 1095 | + "\n", |
| 1096 | + "Keras models can also be checkpointed, and that will look the same as `tf.Module`." |
| 1097 | + ] |
| 1098 | + }, |
1085 | 1099 | { |
1086 | 1100 | "cell_type": "markdown", |
1087 | 1101 | "metadata": { |
|
0 commit comments