|
17 | 17 | "\n", |
18 | 18 | "This notebook can run on **TPU v5e-8** or **v5p-8**\n", |
19 | 19 | "\n", |
20 | | - "This notebook demonstrates how to perform Supervised Fine-Tuning (SFT) on Llama3.1-8B-Instruct using the Hugging Face ultrachat_200k dataset with MaxText and Tunix integration for efficient training.\n", |
21 | | - "\n", |
22 | | - "## Dataset Overview\n", |
23 | | - "\n", |
24 | | - "**Dataset Link:** https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k\n", |
25 | | - "\n", |
26 | | - "### Dataset Information:\n", |
27 | | - "- **Name:** HuggingFaceH4/ultrachat_200k\n", |
28 | | - "- **Type:** Supervised Fine-Tuning dataset\n", |
29 | | - "- **Size:** ~200k conversations\n", |
30 | | - "- **Format:** Chat conversations with human-AI pairs\n", |
31 | | - "- **Splits:** train_sft, test_sft\n", |
32 | | - "- **Data columns:** ['messages']\n", |
33 | | - "\n", |
34 | | - "### Dataset Structure:\n", |
35 | | - "Each example contains a 'messages' field with:\n", |
36 | | - "- **role:** 'user' or 'assistant'\n", |
37 | | - "- **content:** The actual message text\n", |
38 | | - "\n", |
39 | | - "### Example data format:\n", |
40 | | - "```json\n", |
41 | | - "{\n", |
42 | | - " \"messages\": [\n", |
43 | | - " {\"role\": \"user\", \"content\": \"What is the capital of France?\"},\n", |
44 | | - " {\"role\": \"assistant\", \"content\": \"The capital of France is Paris.\"}\n", |
45 | | - " ]\n", |
46 | | - "}\n", |
47 | | - "```\n" |
| 20 | + "This notebook demonstrates how to perform Supervised Fine-Tuning (SFT) on Llama3.1-8B-Instruct using the Hugging Face ultrachat_200k dataset with MaxText and Tunix integration for efficient training.\n" |
48 | 21 | ] |
49 | 22 | }, |
50 | 23 | { |
|
84 | 57 | "\n", |
85 | 58 | "**Follow these steps to store your token:**\n", |
86 | 59 | "\n", |
87 | | - "Just put your token in the line below" |
| 60 | + "Just put your token in the cell below" |
88 | 61 | ] |
89 | 62 | }, |
90 | 63 | { |
|
100 | 73 | "cell_type": "markdown", |
101 | 74 | "metadata": {}, |
102 | 75 | "source": [ |
103 | | - "Install maxtext and dependencies\n", |
| 76 | + "### Install\n", |
104 | 77 | "\n", |
105 | 78 | "https://maxtext.readthedocs.io/en/latest/tutorials/sft.html#install-dependencies" |
106 | 79 | ] |
|
118 | 91 | "metadata": {}, |
119 | 92 | "outputs": [], |
120 | 93 | "source": [ |
121 | | - "!cd ~/maxtext/src/ # This is the path to the maxtext folder" |
| 94 | + "!cd ~/maxtext/src/ # make sure we are in the right directory" |
122 | 95 | ] |
123 | 96 | }, |
124 | 97 | { |
|
0 commit comments