|
423 | 423 | "\n",
|
424 | 424 | "The optimizer is in red, regular variables are in blue, and the optimizer slot variables are in orange. The other nodes—for example, representing the `tf.train.Checkpoint`—are in black.\n",
|
425 | 425 | "\n",
|
426 |
| - "Slot variables are part of the optimizer's state, but are created for a specific variable. For example the `'m'` edges above correspond to momentum, which the Adam optimizer tracks for each variable. Slot variables are only saved in a checkpoint if the variable and the optimizer would both be saved, thus the dashed edges." |
| 426 | + "Slot variables are part of the optimizer's state, but are created for a specific variable. For example, the `'m'` edges above correspond to momentum, which the Adam optimizer tracks for each variable. Slot variables are only saved in a checkpoint if the variable and the optimizer would both be saved, thus the dashed edges." |
427 | 427 | ]
|
428 | 428 | },
|
429 | 429 | {
|
|
491 | 491 | "id": "KCcmJ-2j9RUP"
|
492 | 492 | },
|
493 | 493 | "source": [
|
494 |
| - "### Delayed restorations\n", |
| 494 | + "### Deferred restorations\n", |
495 | 495 | "\n",
|
496 |
| - "`Layer` objects in TensorFlow may delay the creation of variables to their first call, when input shapes are available. For example the shape of a `Dense` layer's kernel depends on both the layer's input and output shapes, and so the output shape required as a constructor argument is not enough information to create the variable on its own. Since calling a `Layer` also reads the variable's value, a restore must happen between the variable's creation and its first use.\n", |
| 496 | + "`Layer` objects in TensorFlow may defer the creation of variables to their first call, when input shapes are available. For example, the shape of a `Dense` layer's kernel depends on both the layer's input and output shapes, and so the output shape required as a constructor argument is not enough information to create the variable on its own. Since calling a `Layer` also reads the variable's value, a restore must happen between the variable's creation and its first use.\n", |
497 | 497 | "\n",
|
498 |
| - "To support this idiom, `tf.train.Checkpoint` queues restores which don't yet have a matching variable." |
| 498 | + "To support this idiom, `tf.train.Checkpoint` defers restores which don't yet have a matching variable." |
499 | 499 | ]
|
500 | 500 | },
|
501 | 501 | {
|
|
506 | 506 | },
|
507 | 507 | "outputs": [],
|
508 | 508 | "source": [
|
509 |
| - "delayed_restore = tf.Variable(tf.zeros([1, 5]))\n", |
510 |
| - "print(delayed_restore.numpy()) # Not restored; still zeros\n", |
511 |
| - "fake_layer.kernel = delayed_restore\n", |
512 |
| - "print(delayed_restore.numpy()) # Restored" |
| 509 | + "deferred_restore = tf.Variable(tf.zeros([1, 5]))\n", |
| 510 | + "print(deferred_restore.numpy()) # Not restored; still zeros\n", |
| 511 | + "fake_layer.kernel = deferred_restore\n", |
| 512 | + "print(deferred_restore.numpy()) # Restored" |
513 | 513 | ]
|
514 | 514 | },
|
515 | 515 | {
|
|
0 commit comments