Gemma finetuning bugfixes (#86)

Author: nealvaidya

models/Gemma/README.md

@@ -31,13 +31,13 @@ You can download either model from the NVIDIA NGC Catalog, using the NGC CLI. Th
 To download the model, execute one of the following commands, based on which model you want to use:
 
 ```bash
-ngc registry model download-version "nvidia/nemo/gemma_2b_base:1.0"
+ngc registry model download-version "nvidia/nemo/gemma_2b_base:1.1"
 ```
 
 or
 
 ```bash
-ngc registry model download-version "nvidia/nemo/gemma_7b_base:1.0"
+ngc registry model download-version "nvidia/nemo/gemma_7b_base:1.1"
 ```
 
 ## Getting NeMo Framework

models/Gemma/lora.ipynb

@@ -47,13 +47,13 @@
     "To download the model, execute one of the following commands, based on which model you want to use:\n",
     "\n",
     "```bash\n",
-    "ngc registry model download-version \"nvidia/nemo/gemma_2b_base:1.0\"\n",
+    "ngc registry model download-version \"nvidia/nemo/gemma_2b_base:1.1\"\n",
     "```\n",
     "\n",
     "or\n",
     "\n",
     "```bash\n",
-    "ngc registry model download-version \"nvidia/nemo/gemma_7b_base:1.0\"\n",
+    "ngc registry model download-version \"nvidia/nemo/gemma_7b_base:1.1\"\n",
     "```"
    ]
   },
@@ -286,7 +286,8 @@
     "  },\n",
     "  \"test_ds\": {\n",
     "    \"file_names\": [\"pubmedqa_test.jsonl\"],\n",
-    "    \"names\": [\"pubmedqa\"]\n",
+    "    \"names\": [\"pubmedqa\"],\n",
+    "    \"add_bos\": True\n",
     "  }\n",
     "}, merge=True)"
    ]

models/Gemma/sft.ipynb

@@ -45,13 +45,13 @@
     "To download the model, execute one of the following commands, based on which model you want to use:\n",
     "\n",
     "```bash\n",
-    "ngc registry model download-version \"nvidia/nemo/gemma_2b_base:1.0\"\n",
+    "ngc registry model download-version \"nvidia/nemo/gemma_2b_base:1.1\"\n",
     "```\n",
     "\n",
     "or\n",
     "\n",
     "```bash\n",
-    "ngc registry model download-version \"nvidia/nemo/gemma_7b_base:1.0\"\n",
+    "ngc registry model download-version \"nvidia/nemo/gemma_7b_base:1.1\"\n",
     "```"
    ]
   },
@@ -291,6 +291,7 @@
     "   model.data.train_ds.micro_batch_size=1 \\\n",
     "   model.data.train_ds.global_batch_size=128 \\\n",
     "   model.data.train_ds.file_path=../databricks-dolly-15k-output.jsonl \\\n",
+    "   model.data.train_ds.add_bos=True \\\n",
     "   model.data.validation_ds.micro_batch_size=1 \\\n",
     "   model.data.validation_ds.global_batch_size=128 \\\n",
     "   model.data.validation_ds.drop_last=True \\\n",