Fix notebooks.

PiperOrigin-RevId: 483639934

Author: MarkDaoust

site/en/guide/migrate/migration_debugging.ipynb

@@ -416,7 +416,7 @@
         "      decay_steps=params['decay_steps'],\n",
         "      end_learning_rate=params['end_lr'],\n",
         "      power=params['lr_power'])  \n",
-        "    self.optimizer = tf.keras.optimizers.SGD(learning_rate_fn)\n",
+        "    self.optimizer = tf.keras.optimizers.legacy.SGD(learning_rate_fn)\n",
         "    self.compiled_loss = tf.keras.losses.CategoricalCrossentropy(from_logits=True)\n",
         "    self.logs = {\n",
         "        'lr': [],\n",

site/en/tutorials/audio/transfer_learning_audio.ipynb

@@ -99,7 +99,9 @@
       },
       "outputs": [],
       "source": [
-        "!pip install tensorflow_io"
+        "!pip install \"tensorflow==2.10.*\"\n",
+        "# tensorflow_io 0.27 is compatible with TensorFlow 2.10\n",
+        "!pip install \"tensorflow_io==0.27.*\""
       ]
     },
     {

site/en/tutorials/reinforcement_learning/actor_critic.ipynb

@@ -161,8 +161,7 @@
       "source": [
         "%%bash\n",
         "# Install additional packages for visualization\n",
-        "sudo apt-get install -y xvfb python-opengl > /dev/null 2>&1\n",
-        "pip install pyvirtualdisplay > /dev/null 2>&1\n",
+        "sudo apt-get install -y python-opengl > /dev/null 2>&1\n",
         "pip install git+https://github.com/tensorflow/docs > /dev/null 2>&1"
       ]
     },
@@ -187,11 +186,10 @@
         "\n",
         "\n",
         "# Create the environment\n",
-        "env = gym.make(\"CartPole-v0\")\n",
+        "env = gym.make(\"CartPole-v1\")\n",
         "\n",
         "# Set seed for experiment reproducibility\n",
         "seed = 42\n",
-        "env.seed(seed)\n",
         "tf.random.set_seed(seed)\n",
         "np.random.seed(seed)\n",
         "\n",
@@ -307,7 +305,7 @@
         "def env_step(action: np.ndarray) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:\n",
         "  \"\"\"Returns state, reward and done flag given an action.\"\"\"\n",
         "\n",
-        "  state, reward, done, _ = env.step(action)\n",
+        "  state, reward, done, truncated, info = env.step(action)\n",
         "  return (state.astype(np.float32), \n",
         "          np.array(reward, np.int32), \n",
         "          np.array(done, np.int32))\n",
@@ -431,15 +429,22 @@
     {
       "cell_type": "markdown",
       "metadata": {
-        "id": "1hrPLrgGxlvb"
+        "id": "qhr50_Czxazw"
       },
       "source": [
         "### 3. The actor-critic loss\n",
         "\n",
         "Since a hybrid actor-critic model is used, the chosen loss function is a combination of actor and critic losses for training, as shown below:\n",
         "\n",
-        "$$L = L_{actor} + L_{critic}$$\n",
-        "\n",
+        "$$L = L_{actor} + L_{critic}$$"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "nOQIJuG1xdTH"
+      },
+      "source": [
         "#### Actor loss\n",
         "\n",
         "The actor loss is based on [policy gradients with the critic as a state dependent baseline](https://www.youtube.com/watch?v=EKqxumCuAAY&t=62m23s) and computed with single-sample (per-episode) estimates.\n",
@@ -456,8 +461,15 @@
         "\n",
         "A negative term is added to the sum since the idea is to maximize the probabilities of actions yielding higher rewards by minimizing the combined loss.\n",
         "\n",
-        "<br>\n",
-        "\n",
+        "<br>"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "Y304O4OAxiAv"
+      },
+      "source": [
         "##### Advantage\n",
         "\n",
         "The $G - V$ term in our $L_{actor}$ formulation is called the [advantage](https://spinningup.openai.com/en/latest/spinningup/rl_intro.html#advantage-functions), which indicates how much better an action is given a particular state over a random action selected according to the policy $\\pi$ for that state.\n",
@@ -468,8 +480,15 @@
         "\n",
         "For instance, suppose that two actions for a given state would yield the same expected return. Without the critic, the algorithm would try to raise the probability of these actions based on the objective $J$. With the critic, it may turn out that there's no advantage ($G - V = 0$) and thus no benefit gained in increasing the actions' probabilities and the algorithm would set the gradients to zero.\n",
         "\n",
-        "<br>\n",
-        "\n",
+        "<br>"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "1hrPLrgGxlvb"
+      },
+      "source": [
         "#### Critic loss\n",
         "\n",
         "Training $V$ to be as close possible to $G$ can be set up as a regression problem with the following loss function:\n",
@@ -596,11 +615,11 @@
         "\n",
         "min_episodes_criterion = 100\n",
         "max_episodes = 10000\n",
-        "max_steps_per_episode = 1000\n",
+        "max_steps_per_episode = 500\n",
         "\n",
-        "# Cartpole-v0 is considered solved if average reward is >= 195 over 100 \n",
+        "# Cartpole-v1 is considered solved if average reward is >= 475 over 500 \n",
         "# consecutive trials\n",
-        "reward_threshold = 195\n",
+        "reward_threshold = 475\n",
         "running_reward = 0\n",
         "\n",
         "# Discount factor for future rewards\n",
@@ -609,16 +628,17 @@
         "# Keep last episodes reward\n",
         "episodes_reward: collections.deque = collections.deque(maxlen=min_episodes_criterion)\n",
         "\n",
-        "with tqdm.trange(max_episodes) as t:\n",
-        "  for i in t:\n",
-        "    initial_state = tf.constant(env.reset(), dtype=tf.float32)\n",
+        "t = tqdm.trange(max_episodes)\n",
+        "for i in t:\n",
+        "    initial_state, info = env.reset()\n",
+        "    initial_state = tf.constant(initial_state, dtype=tf.float32)\n",
         "    episode_reward = int(train_step(\n",
         "        initial_state, model, optimizer, gamma, max_steps_per_episode))\n",
         "    \n",
         "    episodes_reward.append(episode_reward)\n",
         "    running_reward = statistics.mean(episodes_reward)\n",
         "  \n",
-        "    t.set_description(f'Episode {i}')\n",
+        "\n",
         "    t.set_postfix(\n",
         "        episode_reward=episode_reward, running_reward=running_reward)\n",
         "  \n",
@@ -655,31 +675,26 @@
         "\n",
         "from IPython import display as ipythondisplay\n",
         "from PIL import Image\n",
-        "from pyvirtualdisplay import Display\n",
-        "\n",
-        "\n",
-        "display = Display(visible=0, size=(400, 300))\n",
-        "display.start()\n",
         "\n",
+        "render_env = gym.make(\"CartPole-v1\", render_mode='rgb_array')\n",
         "\n",
         "def render_episode(env: gym.Env, model: tf.keras.Model, max_steps: int): \n",
-        "  screen = env.render(mode='rgb_array')\n",
-        "  im = Image.fromarray(screen)\n",
-        "\n",
-        "  images = [im]\n",
-        "  \n",
-        "  state = tf.constant(env.reset(), dtype=tf.float32)\n",
+        "  state, info = render_env.reset()\n",
+        "  state = tf.constant(state, dtype=tf.float32)\n",
+        "  screen = render_env.render()\n",
+        "  images = [Image.fromarray(screen)]\n",
+        " \n",
         "  for i in range(1, max_steps + 1):\n",
         "    state = tf.expand_dims(state, 0)\n",
         "    action_probs, _ = model(state)\n",
         "    action = np.argmax(np.squeeze(action_probs))\n",
         "\n",
-        "    state, _, done, _ = env.step(action)\n",
+        "    state, reward, done, truncated, info = render_env.step(action)\n",
         "    state = tf.constant(state, dtype=tf.float32)\n",
         "\n",
         "    # Render screen every 10 steps\n",
         "    if i % 10 == 0:\n",
-        "      screen = env.render(mode='rgb_array')\n",
+        "      screen = render_env.render()\n",
         "      images.append(Image.fromarray(screen))\n",
         "  \n",
         "    if done:\n",
@@ -690,7 +705,7 @@
         "\n",
         "# Save GIF image\n",
         "images = render_episode(env, model, max_steps_per_episode)\n",
-        "image_file = 'cartpole-v0.gif'\n",
+        "image_file = 'cartpole-v1.gif'\n",
         "# loop=0: loop forever, duration=1: play each frame for 1ms\n",
         "images[0].save(\n",
         "    image_file, save_all=True, append_images=images[1:], loop=0, duration=1)"

site/en/tutorials/video/video_classification.ipynb

@@ -115,11 +115,7 @@
         "\n",
         "import tensorflow as tf\n",
         "import keras\n",
-        "from keras.models import Model, Sequential\n",
-        "from tensorflow.keras import layers\n",
-        "from tensorflow.keras.optimizers import Adam\n",
-        "from keras.losses import sparse_categorical_crossentropy\n",
-        "from keras.utils.vis_utils import plot_model"
+        "from keras import layers\n"
       ]
     },
     {
@@ -550,7 +546,7 @@
         "  \"\"\"\n",
         "  def __init__(self, units):\n",
         "    super().__init__()\n",
-        "    self.seq = Sequential([\n",
+        "    self.seq = keras.Sequential([\n",
         "        layers.Dense(units),\n",
         "        layers.LayerNormalization()\n",
         "    ])\n",
@@ -681,9 +677,9 @@
         "\n",
         "x = layers.GlobalAveragePooling3D()(x)\n",
         "x = layers.Flatten()(x)\n",
-        "x = layers.Dense(10, activation = 'softmax')(x)\n",
+        "x = layers.Dense(10)(x)\n",
         "\n",
-        "model = Model(input, x)"
+        "model = keras.Model(input, x)"
       ]
     },
     {
@@ -707,7 +703,7 @@
       "outputs": [],
       "source": [
         "# Visualize the model\n",
-        "plot_model(model, expand_nested=True, dpi=60, show_shapes=True)"
+        "keras.utils.plot_model(model, expand_nested=True, dpi=60, show_shapes=True)"
       ]
     },
     {
@@ -729,8 +725,8 @@
       },
       "outputs": [],
       "source": [
-        "model.compile(loss = sparse_categorical_crossentropy, \n",
-        "              optimizer = Adam(learning_rate = 0.0001), \n",
+        "model.compile(loss = keras.losses.SparseCategoricalCrossentropy(from_logits=True), \n",
+        "              optimizer = keras.optimizers.Adam(learning_rate = 0.0001), \n",
         "              metrics = ['accuracy'])"
       ]
     },
@@ -740,8 +736,6 @@
         "id": "nZT1Xlx9stP2"
       },
       "source": [
-        "\n",
-        "\n",
         "Train the model for 50 epoches with the Keras `Model.fit` method.\n",
         "\n",
         "Note: This example model is trained on fewer data points (300 training and 100 validation examples) to keep training time reasonable for this tutorial. Moreover, this example model may take over one hour to train."
@@ -871,7 +865,7 @@
         "  actual = [labels for _, labels in dataset.unbatch()]\n",
         "  predicted = model.predict(dataset)\n",
         "\n",
-        "  actual = tf.concat(actual, axis=0)\n",
+        "  actual = tf.stack(actual, axis=0)\n",
         "  predicted = tf.concat(predicted, axis=0)\n",
         "  predicted = tf.argmax(predicted, axis=1)\n",
         "\n",