|
80 | 80 | "source": [ |
81 | 81 | "This guide demonstrates how to fine-tune FunctionGemma for tool calling.\n", |
82 | 82 | "\n", |
83 | | - "While FunctionGemma is natively capable of calling tools. But true capability comes from two distinct skills: the mechanical knowledge of how to use a tool (syntax) and the cognitive ability to interpret *why* and *when* to use it (intent).\n", |
| 83 | + "While FunctionGemma is natively capable of calling tools, true capability comes from two distinct skills: the mechanical knowledge of how to use a tool (syntax) and the cognitive ability to interpret *why* and *when* to use it (intent).\n", |
84 | 84 | "\n", |
85 | | - "Models, especially smaller ones, have fewer parameters available to retain complex intent understanding. This is why we need to fine-tune them\n", |
| 85 | + "Models, especially smaller ones, have fewer parameters available to retain complex intent understanding. This is why we need to fine-tune them.\n", |
86 | 86 | "\n", |
87 | 87 | "Common use cases for fine-tuning tool calling include:\n", |
88 | 88 | "\n", |
|
134 | 134 | "id": "3raKuRFXEDNm" |
135 | 135 | }, |
136 | 136 | "source": [ |
137 | | - "> _Note: If you are using a GPU with Ampere architecture (such as NVIDIA L4) or newer, you can use Flash attention. Flash Attention is a method that significantly speeds computations up and reduces memory usage from quadratic to linear in sequence length, leading to acelerating training up to 3x. Learn more at [FlashAttention](https://github.com/Dao-AILab/flash-attention/tree/main)._\n", |
| 137 | + "> _Note: If you are using a GPU with Ampere architecture (such as NVIDIA L4) or newer, you can use Flash attention. Flash Attention is a method that significantly speeds computations up and reduces memory usage from quadratic to linear in sequence length, leading to accelerating training up to 3x. Learn more at [FlashAttention](https://github.com/Dao-AILab/flash-attention/tree/main)._\n", |
138 | 138 | "\n", |
139 | 139 | "Before you can start training, you have to make sure that you accepted the terms of use for Gemma. You can accept the license on [Hugging Face](http://huggingface.co/google/functiongemma-270m-it) by clicking on the **Agree** and access repository button on the model page at: http://huggingface.co/google/functiongemma-270m-it\n", |
140 | 140 | "\n", |
|
924 | 924 | "source": [ |
925 | 925 | "To plot the training and validation losses, you would typically extract these values from the `TrainerState` object or the logs generated during training.\n", |
926 | 926 | "\n", |
927 | | - "Libraries like Matplotlib can then be used to visualize these values over training steps or epochs. The x-asis would represent the training steps or epochs, and the y-axis would represent the corresponding loss values." |
| 927 | + "Libraries like Matplotlib can then be used to visualize these values over training steps or epochs. The x-axis would represent the training steps or epochs, and the y-axis would represent the corresponding loss values." |
928 | 928 | ] |
929 | 929 | }, |
930 | 930 | { |
|
1059 | 1059 | } |
1060 | 1060 | ], |
1061 | 1061 | "source": [ |
1062 | | - "check_success_rate()\n" |
| 1062 | + "check_success_rate()" |
1063 | 1063 | ] |
1064 | 1064 | }, |
1065 | 1065 | { |
|
1087 | 1087 | "Check out the following docs next:\n", |
1088 | 1088 | "\n", |
1089 | 1089 | "- [Full function calling sequence with FunctionGemma](https://ai.google.dev/gemma/docs/functiongemma/full-function-calling-sequence-with-functiongemma)\n", |
1090 | | - "- [Finetune FunctionGemma for Mobile Actions](https://github.com/google-gemini/gemma-cookbook/blob/main/FunctionGemma/%5BFunctionGemma%5DFinetune_FunctionGemma_270M_for_Mobile_Actions_with_Hugging_Face.ipynb) in the Gemma Cookbook\n" |
| 1090 | + "- [Fine-tune FunctionGemma for Mobile Actions](https://github.com/google-gemini/gemma-cookbook/blob/main/FunctionGemma/%5BFunctionGemma%5DFinetune_FunctionGemma_270M_for_Mobile_Actions_with_Hugging_Face.ipynb) in the Gemma Cookbook\n" |
1091 | 1091 | ] |
1092 | 1092 | } |
1093 | 1093 | ], |
|
0 commit comments