|
83 | 83 | "\n", |
84 | 84 | "[Low Rank Adaptation (LoRA)](https://arxiv.org/abs/2106.09685) is a fine-tuning technique which greatly reduces the number of trainable parameters for downstream tasks by freezing the weights of the model and inserting a smaller number of new weights into the model. This makes training with LoRA much faster and more memory-efficient, and produces smaller model weights (a few hundred MBs), all while maintaining the quality of the model outputs.\n", |
85 | 85 | "\n", |
86 | | - "This tutorial walks you through using KerasNLP to perform LoRA fine-tuning on a Gemma 2B model using the [Databricks Dolly 15k dataset](https://huggingface.co/datasets/databricks/databricks-dolly-15k). This dataset contains 15,000 high-quality human-generated prompt / response pairs specifically designed for fine-tuning LLMs." |
| 86 | + "This tutorial walks you through using KerasHub to perform LoRA fine-tuning on a Gemma 2B model using the [Databricks Dolly 15k dataset](https://huggingface.co/datasets/databricks/databricks-dolly-15k). This dataset contains 15,000 high-quality human-generated prompt / response pairs specifically designed for fine-tuning LLMs." |
87 | 87 | ] |
88 | 88 | }, |
89 | 89 | { |
|
180 | 180 | "source": [ |
181 | 181 | "### Install dependencies\n", |
182 | 182 | "\n", |
183 | | - "Install Keras, KerasNLP, and other dependencies." |
| 183 | + "Install Keras, KerasHub, and other dependencies." |
184 | 184 | ] |
185 | 185 | }, |
186 | 186 | { |
|
192 | 192 | "outputs": [], |
193 | 193 | "source": [ |
194 | 194 | "# Install Keras 3 last. See https://keras.io/getting_started/ for more details.\n", |
195 | | - "!pip install -q -U keras-nlp\n", |
196 | | - "!pip install -q -U \"keras>=3\"" |
| 195 | + "!pip install -q -U keras-hub\n", |
| 196 | + "!pip install -q -U keras" |
197 | 197 | ] |
198 | 198 | }, |
199 | 199 | { |
|
230 | 230 | "source": [ |
231 | 231 | "### Import packages\n", |
232 | 232 | "\n", |
233 | | - "Import Keras and KerasNLP." |
| 233 | + "Import Keras and KerasHub." |
234 | 234 | ] |
235 | 235 | }, |
236 | 236 | { |
|
242 | 242 | "outputs": [], |
243 | 243 | "source": [ |
244 | 244 | "import keras\n", |
245 | | - "import keras_nlp" |
| 245 | + "import keras_hub" |
246 | 246 | ] |
247 | 247 | }, |
248 | 248 | { |
|
329 | 329 | "source": [ |
330 | 330 | "## Load Model\n", |
331 | 331 | "\n", |
332 | | - "KerasNLP provides implementations of many popular [model architectures](https://keras.io/api/keras_nlp/models/). In this tutorial, you'll create a model using `GemmaCausalLM`, an end-to-end Gemma model for causal language modeling. A causal language model predicts the next token based on previous tokens.\n", |
| 332 | + "KerasHub provides implementations of many popular [model architectures](https://keras.io/api/keras_hub/models/). In this tutorial, you'll create a model using `GemmaCausalLM`, an end-to-end Gemma model for causal language modeling. A causal language model predicts the next token based on previous tokens.\n", |
333 | 333 | "\n", |
334 | 334 | "Create the model using the `from_preset` method:" |
335 | 335 | ] |
|
466 | 466 | } |
467 | 467 | ], |
468 | 468 | "source": [ |
469 | | - "gemma_lm = keras_nlp.models.GemmaCausalLM.from_preset(\"gemma2_2b_en\")\n", |
| 469 | + "gemma_lm = keras_hub.models.GemmaCausalLM.from_preset(\"gemma2_2b_en\")\n", |
470 | 470 | "gemma_lm.summary()" |
471 | 471 | ] |
472 | 472 | }, |
|
557 | 557 | " instruction=\"What should I do on a trip to Europe?\",\n", |
558 | 558 | " response=\"\",\n", |
559 | 559 | ")\n", |
560 | | - "sampler = keras_nlp.samplers.TopKSampler(k=5, seed=2)\n", |
| 560 | + "sampler = keras_hub.samplers.TopKSampler(k=5, seed=2)\n", |
561 | 561 | "gemma_lm.compile(sampler=sampler)\n", |
562 | 562 | "print(gemma_lm.generate(prompt, max_length=256))" |
563 | 563 | ] |
|
912 | 912 | " instruction=\"What should I do on a trip to Europe?\",\n", |
913 | 913 | " response=\"\",\n", |
914 | 914 | ")\n", |
915 | | - "sampler = keras_nlp.samplers.TopKSampler(k=5, seed=2)\n", |
| 915 | + "sampler = keras_hub.samplers.TopKSampler(k=5, seed=2)\n", |
916 | 916 | "gemma_lm.compile(sampler=sampler)\n", |
917 | 917 | "print(gemma_lm.generate(prompt, max_length=256))" |
918 | 918 | ] |
|
993 | 993 | "source": [ |
994 | 994 | "## Summary and next steps\n", |
995 | 995 | "\n", |
996 | | - "This tutorial covered LoRA fine-tuning on a Gemma model using KerasNLP. Check out the following docs next:\n", |
| 996 | + "This tutorial covered LoRA fine-tuning on a Gemma model using KerasHub. Check out the following docs next:\n", |
997 | 997 | "\n", |
998 | 998 | "* Learn how to [generate text with a Gemma model](https://ai.google.dev/gemma/docs/get_started).\n", |
999 | 999 | "* Learn how to perform [distributed fine-tuning and inference on a Gemma model](https://ai.google.dev/gemma/docs/core/distributed_tuning).\n", |
1000 | 1000 | "* Learn how to [use Gemma open models with Vertex AI](https://cloud.google.com/vertex-ai/docs/generative-ai/open-models/use-gemma).\n", |
1001 | | - "* Learn how to [fine-tune Gemma using KerasNLP and deploy to Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/model_garden/model_garden_gemma_kerasnlp_to_vertexai.ipynb)." |
| 1001 | + "* Learn how to [fine-tune Gemma using KerasHub and deploy to Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/model_garden/model_garden_gemma_kerasnlp_to_vertexai.ipynb)." |
1002 | 1002 | ] |
1003 | 1003 | } |
1004 | 1004 | ], |
|
0 commit comments