Skip to content

Commit 0747fe6

Browse files
tensorflower-gardenercopybara-github
tensorflower-gardener
authored and
copybara-github
committed
Modified the migration documents to use 'DeterministicRandomTestTool' to check numerical equivalence during TF2 migration.
PiperOrigin-RevId: 413824141
1 parent 07ccab6 commit 0747fe6

File tree

3 files changed

+126
-230
lines changed

3 files changed

+126
-230
lines changed

‎site/en/guide/migrate/migration_debugging.ipynb

Lines changed: 56 additions & 69 deletions
Original file line numberDiff line numberDiff line change
@@ -145,98 +145,61 @@
145145
{
146146
"cell_type": "markdown",
147147
"metadata": {
148-
"id": "usyRSlIRl3r2"
148+
"id": "XKakQBI9-FLb"
149149
},
150150
"source": [
151-
"### Single forward pass validation \n",
152-
"\n",
153-
"Single forward pass validation, including checkpoint loading, is covered in a different [colab](./validate_correctness.ipynb)."
151+
"## Setup"
154152
]
155153
},
156154
{
157155
"cell_type": "code",
158156
"execution_count": null,
159157
"metadata": {
160-
"id": "HVBQbsZeVL_V"
158+
"id": "sopP--i7-LaF"
161159
},
162160
"outputs": [],
163161
"source": [
164-
"import sys\n",
165-
"import unittest\n",
166-
"import numpy as np\n",
167-
"\n",
168-
"import tensorflow as tf\n",
169-
"import tensorflow.compat.v1 as v1"
162+
"!pip uninstall -y -q tensorflow"
163+
]
164+
},
165+
{
166+
"cell_type": "code",
167+
"execution_count": null,
168+
"metadata": {
169+
"id": "i1ghHyXl-Oqd"
170+
},
171+
"outputs": [],
172+
"source": [
173+
"# Install tf-nightly as the DeterministicRandomTestTool is only available in\n",
174+
"# Tensorflow 2.8\n",
175+
"!pip install -q tf-nightly"
170176
]
171177
},
172178
{
173179
"cell_type": "markdown",
174180
"metadata": {
175-
"id": "m5XhfpDky1Kw"
181+
"id": "usyRSlIRl3r2"
176182
},
177183
"source": [
178-
"Define a context manager to control random number generation."
184+
"### Single forward pass validation \n",
185+
"\n",
186+
"Single forward pass validation, including checkpoint loading, is covered in a different [colab](./validate_correctness.ipynb)."
179187
]
180188
},
181189
{
182190
"cell_type": "code",
183191
"execution_count": null,
184192
"metadata": {
185-
"id": "DXEUeSUvkeEV"
193+
"id": "HVBQbsZeVL_V"
186194
},
187195
"outputs": [],
188196
"source": [
189-
"seed_implementation = sys.modules[tf.compat.v1.get_seed.__module__]\n",
190-
"\n",
191-
"class DeterministicTestTool(object):\n",
192-
" def __init__(self, seed: int = 42, mode='constant'):\n",
193-
" \"\"\"Set mode to 'constant' or 'num_random_ops'. Defaults to 'constant'.\"\"\"\n",
194-
" if mode not in {'constant', 'num_random_ops'}:\n",
195-
" raise ValueError(\"Mode arg must be 'constant' or 'num_random_ops'. \" +\n",
196-
" \"Got: {}\".format(mode))\n",
197-
"\n",
198-
" self._mode = mode\n",
199-
" self._seed = seed\n",
200-
" self.operation_seed = 0\n",
201-
" self._observed_seeds = set()\n",
202-
"\n",
203-
" def scope(self):\n",
204-
" tf.random.set_seed(self._seed)\n",
205-
"\n",
206-
" def _get_seed(_):\n",
207-
" \"\"\"Wraps TF get_seed to make deterministic random generation easier.\n",
208-
"\n",
209-
" This makes a variable's initialization (and calls that involve random\n",
210-
" number generation) depend only on how many random number generations\n",
211-
" were used in the scope so far, rather than on how many unrelated\n",
212-
" operations the graph contains.\n",
213-
"\n",
214-
" Returns:\n",
215-
" Random seed tuple.\n",
216-
" \"\"\"\n",
217-
" op_seed = self.operation_seed\n",
218-
" if self._mode == \"constant\":\n",
219-
" tf.random.set_seed(op_seed)\n",
220-
" else:\n",
221-
" if op_seed in self._observed_seeds:\n",
222-
" raise ValueError(\n",
223-
" 'This `DeterministicTestTool` object is trying to re-use the ' +\n",
224-
" 'already-used operation seed {}. '.format(op_seed) +\n",
225-
" 'It cannot guarantee random numbers will match between eager ' +\n",
226-
" 'and sessions when an operation seed is reused. ' +\n",
227-
" 'You most likely set ' +\n",
228-
" '`operation_seed` explicitly but used a value that caused the ' +\n",
229-
" 'naturally-incrementing operation seed sequences to overlap ' +\n",
230-
" 'with an already-used seed.')\n",
231-
"\n",
232-
" self._observed_seeds.add(op_seed)\n",
233-
" self.operation_seed += 1\n",
234-
"\n",
235-
" return (self._seed, op_seed)\n",
236-
"\n",
237-
" # mock.patch internal symbols to modify the behavior of TF APIs relying on them\n",
197+
"import sys\n",
198+
"import unittest\n",
199+
"import numpy as np\n",
238200
"\n",
239-
" return unittest.mock.patch.object(seed_implementation, 'get_seed', wraps=_get_seed)"
201+
"import tensorflow as tf\n",
202+
"import tensorflow.compat.v1 as v1"
240203
]
241204
},
242205
{
@@ -362,6 +325,32 @@
362325
" model_tf1.logs[key].append(logs[key])"
363326
]
364327
},
328+
{
329+
"cell_type": "markdown",
330+
"metadata": {
331+
"id": "kki9yILSKS7f"
332+
},
333+
"source": [
334+
"The following [`v1.keras.utils.DeterministicRandomTestTool`](https://www.tensorflow.org/api_docs/python/tf/compat/v1/keras/utils/DeterministicRandomTestTool) class provides a context manager `scope()` that can make stateful random operations use the same seed across both TF1 graphs/sessions and eager execution,\n",
335+
"\n",
336+
"The tool provides two testing modes: \n",
337+
"1. `constant` which uses the same seed for every single operation no matter how many times it has been called and,\n",
338+
"2. `num_random_ops` which uses the number of previously-observed stateful random operations as the operation seed.\n",
339+
"\n",
340+
"This applies both to the stateful random operations used for creating and initializing variables, and to the stateful random operations used in computation (such as for dropout layers)."
341+
]
342+
},
343+
{
344+
"cell_type": "code",
345+
"execution_count": null,
346+
"metadata": {
347+
"id": "X6Y3RWMoKOl8"
348+
},
349+
"outputs": [],
350+
"source": [
351+
"random_tool = v1.keras.utils.DeterministicRandomTestTool(mode='num_random_ops')"
352+
]
353+
},
365354
{
366355
"cell_type": "markdown",
367356
"metadata": {
@@ -379,7 +368,6 @@
379368
},
380369
"outputs": [],
381370
"source": [
382-
"random_tool = DeterministicTestTool(mode='num_random_ops')\n",
383371
"with random_tool.scope():\n",
384372
" graph = tf.Graph()\n",
385373
" with graph.as_default(), tf.compat.v1.Session(graph=graph) as sess:\n",
@@ -483,7 +471,7 @@
483471
},
484472
"outputs": [],
485473
"source": [
486-
"random_tool = DeterministicTestTool(mode='num_random_ops')\n",
474+
"random_tool = v1.keras.utils.DeterministicRandomTestTool(mode='num_random_ops')\n",
487475
"with random_tool.scope():\n",
488476
" model_tf2 = SimpleModel(params)\n",
489477
" for step in range(step_num):\n",
@@ -562,7 +550,7 @@
562550
" step_num = 100\n",
563551
"\n",
564552
" # setup TF 1 model\n",
565-
" random_tool = DeterministicTestTool(mode='num_random_ops')\n",
553+
" random_tool = v1.keras.utils.DeterministicRandomTestTool(mode='num_random_ops')\n",
566554
" with random_tool.scope():\n",
567555
" # run TF1.x code in graph mode with context management\n",
568556
" graph = tf.Graph()\n",
@@ -583,7 +571,7 @@
583571
" self.model_tf1.update_logs(logs)\n",
584572
"\n",
585573
" # setup TF2 model\n",
586-
" random_tool = DeterministicTestTool(mode='num_random_ops')\n",
574+
" random_tool = v1.keras.utils.DeterministicRandomTestTool(mode='num_random_ops')\n",
587575
" with random_tool.scope():\n",
588576
" self.model_tf2 = SimpleModel(params)\n",
589577
" for step in range(step_num):\n",
@@ -811,7 +799,6 @@
811799
"colab": {
812800
"collapsed_sections": [],
813801
"name": "migration_debugging.ipynb",
814-
"provenance": [],
815802
"toc_visible": true
816803
},
817804
"kernelspec": {

‎site/en/guide/migrate/model_mapping.ipynb

Lines changed: 21 additions & 73 deletions
Original file line numberDiff line numberDiff line change
@@ -108,12 +108,25 @@
108108
"cell_type": "code",
109109
"execution_count": null,
110110
"metadata": {
111-
"id": "842MGAOp5bbs"
111+
"id": "EG2n3-qlD5mA"
112112
},
113113
"outputs": [],
114114
"source": [
115-
"!pip uninstall -y tensorflow_estimator tensorflow keras\n",
116-
"!pip install -U tf-nightly"
115+
"!pip uninstall -y -q tensorflow"
116+
]
117+
},
118+
{
119+
"cell_type": "code",
120+
"execution_count": null,
121+
"metadata": {
122+
"id": "mVfR3MBvD9Sc"
123+
},
124+
"outputs": [],
125+
"source": [
126+
"# Install tf-nightly as the DeterministicRandomTestTool is available only in\n",
127+
"# Tensorflow 2.8\n",
128+
"\n",
129+
"!pip install -q tf-nightly"
117130
]
118131
},
119132
{
@@ -124,13 +137,11 @@
124137
},
125138
"outputs": [],
126139
"source": [
127-
"# Note that the model mapping shim is available only in TF 2.7.\n",
128140
"import tensorflow as tf\n",
129141
"import tensorflow.compat.v1 as v1\n",
130142
"import sys\n",
131143
"import numpy as np\n",
132144
"\n",
133-
"from unittest import mock\n",
134145
"from contextlib import contextmanager"
135146
]
136147
},
@@ -1024,69 +1035,7 @@
10241035
"id": "kzJF0H0sbce8"
10251036
},
10261037
"source": [
1027-
"Use the deterministic number generation test tool to verify that this incremental change leaves the model with the same behavior as before."
1028-
]
1029-
},
1030-
{
1031-
"cell_type": "code",
1032-
"execution_count": null,
1033-
"metadata": {
1034-
"id": "VRTg0bQlcPeP"
1035-
},
1036-
"outputs": [],
1037-
"source": [
1038-
"# import tensorflow.python.framework.random_seed as random_seed\n",
1039-
"seed_implementation = sys.modules[tf.compat.v1.get_seed.__module__]\n",
1040-
"\n",
1041-
"class DeterministicTestTool(object):\n",
1042-
" def __init__(self, seed: int = 42, mode='constant'):\n",
1043-
" \"\"\"Set mode to 'constant' or 'num_random_ops'. Defaults to 'constant'.\"\"\"\n",
1044-
" if mode not in {'constant', 'num_random_ops'}:\n",
1045-
" raise ValueError(\"Mode arg must be 'constant' or 'num_random_ops'. \" +\n",
1046-
" \"Got: {}\".format(mode))\n",
1047-
"\n",
1048-
" self._mode = mode\n",
1049-
" self._seed = seed\n",
1050-
" self.operation_seed = 0\n",
1051-
" self._observed_seeds = set()\n",
1052-
"\n",
1053-
" def scope(self):\n",
1054-
" tf.random.set_seed(self._seed)\n",
1055-
"\n",
1056-
" def _get_seed(_):\n",
1057-
" \"\"\"Wraps TF get_seed to make deterministic random generation easier.\n",
1058-
"\n",
1059-
" This makes a variable's initialization (and calls that involve random\n",
1060-
" number generation) depend only on how many random number generations\n",
1061-
" were used in the scope so far, rather than on how many unrelated\n",
1062-
" operations the graph contains.\n",
1063-
"\n",
1064-
" Returns:\n",
1065-
" Random seed tuple.\n",
1066-
" \"\"\"\n",
1067-
" op_seed = self.operation_seed\n",
1068-
" if self._mode == \"constant\":\n",
1069-
" tf.random.set_seed(op_seed)\n",
1070-
" else:\n",
1071-
" if op_seed in self._observed_seeds:\n",
1072-
" raise ValueError(\n",
1073-
" 'This `DeterministicTestTool` object is trying to re-use the ' +\n",
1074-
" 'already-used operation seed {}. '.format(op_seed) +\n",
1075-
" 'It cannot guarantee random numbers will match between eager ' +\n",
1076-
" 'and sessions when an operation seed is reused. ' +\n",
1077-
" 'You most likely set ' +\n",
1078-
" '`operation_seed` explicitly but used a value that caused the ' +\n",
1079-
" 'naturally-incrementing operation seed sequences to overlap ' +\n",
1080-
" 'with an already-used seed.')\n",
1081-
"\n",
1082-
" self._observed_seeds.add(op_seed)\n",
1083-
" self.operation_seed += 1\n",
1084-
"\n",
1085-
" return (self._seed, op_seed)\n",
1086-
"\n",
1087-
" # mock.patch internal symbols to modify the behavior of TF APIs relying on them\n",
1088-
"\n",
1089-
" return mock.patch.object(seed_implementation, 'get_seed', wraps=_get_seed)"
1038+
"Use the [`v1.keras.utils.DeterministicRandomTestTool`](https://www.tensorflow.org/api_docs/python/tf/compat/v1/keras/utils/DeterministicRandomTestTool) class to verify that this incremental change leaves the model with the same behavior as before."
10901039
]
10911040
},
10921041
{
@@ -1097,7 +1046,7 @@
10971046
},
10981047
"outputs": [],
10991048
"source": [
1100-
"random_tool = DeterministicTestTool(mode='num_random_ops')\n",
1049+
"random_tool = v1.keras.utils.DeterministicRandomTestTool(mode='num_random_ops')\n",
11011050
"with random_tool.scope():\n",
11021051
" layer = CompatModel(10)\n",
11031052
"\n",
@@ -1118,7 +1067,7 @@
11181067
},
11191068
"outputs": [],
11201069
"source": [
1121-
"random_tool = DeterministicTestTool(mode='num_random_ops')\n",
1070+
"random_tool = v1.keras.utils.DeterministicRandomTestTool(mode='num_random_ops')\n",
11221071
"with random_tool.scope():\n",
11231072
" layer = PartiallyMigratedModel(10)\n",
11241073
"\n",
@@ -1191,7 +1140,7 @@
11911140
},
11921141
"outputs": [],
11931142
"source": [
1194-
"random_tool = DeterministicTestTool(mode='num_random_ops')\n",
1143+
"random_tool = v1.keras.utils.DeterministicRandomTestTool(mode='num_random_ops')\n",
11951144
"with random_tool.scope():\n",
11961145
" layer = NearlyFullyNativeModel(10)\n",
11971146
"\n",
@@ -1264,7 +1213,7 @@
12641213
},
12651214
"outputs": [],
12661215
"source": [
1267-
"random_tool = DeterministicTestTool(mode='num_random_ops')\n",
1216+
"random_tool = v1.keras.utils.DeterministicRandomTestTool(mode='num_random_ops')\n",
12681217
"with random_tool.scope():\n",
12691218
" layer = FullyNativeModel(10)\n",
12701219
"\n",
@@ -1565,7 +1514,6 @@
15651514
"colab": {
15661515
"collapsed_sections": [],
15671516
"name": "model_mapping.ipynb",
1568-
"provenance": [],
15691517
"toc_visible": true
15701518
},
15711519
"kernelspec": {

0 commit comments

Comments
 (0)