fix

Author: bebechien

site/en/gemma/docs/core/huggingface_text_full_finetune.ipynb

@@ -67,7 +67,7 @@
         "This guide walks you through how to fine-tune Gemma on a mobile game NPC dataset using Hugging Face [Transformers](https://huggingface.co/docs/transformers/index) and [TRL](https://huggingface.co/docs/trl/index). You will learn:\n",
         "\n",
         "- Setup development environment\n",
-        "- Create and prepare the fine-tuning dataset\n",
+        "- Prepare the fine-tuning dataset\n",
         "- Full model fine-tuning Gemma using TRL and the SFTTrainer\n",
         "- Test Model Inference and vibe checks\n",
         "\n",
@@ -256,8 +256,6 @@
         }
       ],
       "source": [
-        "npc_type = \"martian\" #@param [\"martian\", \"venusian\"]\n",
-        "\n",
         "from datasets import load_dataset\n",
         "\n",
         "def create_conversation(sample):\n",
@@ -268,12 +266,15 @@
         "      ]\n",
         "  }\n",
         "\n",
-        "# Load dataset from the hub\n",
+        "npc_type = \"martian\" #@param [\"martian\", \"venusian\"]\n",
+        "\n",
+        "# Load dataset from the Hub\n",
         "dataset = load_dataset(\"bebechien/MobileGameNPC\", npc_type, split=\"train\")\n",
         "\n",
         "# Convert dataset to conversational format\n",
-        "dataset = dataset.map(create_conversation, remove_columns=dataset.features,batched=False)\n",
-        "# split dataset into 80% training samples and 20% test samples\n",
+        "dataset = dataset.map(create_conversation, remove_columns=dataset.features, batched=False)\n",
+        "\n",
+        "# Split dataset into 80% training samples and 20% test samples\n",
         "dataset = dataset.train_test_split(test_size=0.2, shuffle=False)\n",
         "\n",
         "# Print formatted user prompt\n",
@@ -332,9 +333,9 @@
         "id": "M3w3b9-O4fDz"
       },
       "source": [
-        "## Before fine-tune (Base model)\n",
+        "## Before fine-tune\n",
         "\n",
-        "The output below shows that the model is a generalist and isn't specifically trained for your NPC's character."
+        "The output below shows that the out-of-the-box capabilities may not be good enough for this use case."
       ]
     },
     {
@@ -720,19 +721,11 @@
         "# Access the log history\n",
         "log_history = trainer.state.log_history\n",
         "\n",
-        "# Extract training loss and global steps\n",
-        "train_losses = []\n",
-        "eval_losses = []\n",
-        "epoch_train = []\n",
-        "epoch_eval = []\n",
-        "\n",
-        "for log in log_history:\n",
-        "    if \"loss\" in log:  # Check for training loss\n",
-        "        train_losses.append(log[\"loss\"])\n",
-        "        epoch_train.append(log[\"epoch\"])\n",
-        "    if \"eval_loss\" in log: # Check for validation loss\n",
-        "        eval_losses.append(log[\"eval_loss\"])\n",
-        "        epoch_eval.append(log['epoch'])\n",
+        "# Extract training / validation loss\n",
+        "train_losses = [log[\"loss\"] for log in log_history if \"loss\" in log]\n",
+        "epoch_train = [log[\"epoch\"] for log in log_history if \"loss\" in log]\n",
+        "eval_losses = [log[\"eval_loss\"] for log in log_history if \"eval_loss\" in log]\n",
+        "epoch_eval = [log[\"epoch\"] for log in log_history if \"eval_loss\" in log]\n",
         "\n",
         "# Plot the training loss\n",
         "plt.plot(epoch_train, train_losses, label=\"Training Loss\")\n",
@@ -771,9 +764,7 @@
         "\n",
         "After the training is done, you'll want to evaluate and test your model. You can load different samples from the test dataset and evaluate the model on those samples.\n",
         "\n",
-        "For this particular use case, the best model is a matter of preference. Interestingly, what we'd normally call 'overfitting' can be very useful for a game NPC. It forces the model to forget general information and instead lock onto the specific persona and characteristics it was trained on, ensuring it stays consistently in character.\n",
-        "\n",
-        "> Note: Evaluating generative AI models is not a trivial task since one input can have multiple correct outputs. This guide only focuses on manual evaluation and vibe checks."
+        "For this particular use case, the best model is a matter of preference. Interestingly, what we'd normally call 'overfitting' can be very useful for a game NPC. It forces the model to forget general information and instead lock onto the specific persona and characteristics it was trained on, ensuring it stays consistently in character.\n"
       ]
     },
     {