Skip to content

Commit 145322d

Browse files
authored
[Minor] Qwen3 typo & optim (rasbt#758)
* typo * remove weight dict after loading
1 parent b12dbf6 commit 145322d

File tree

2 files changed

+6
-5
lines changed

2 files changed

+6
-5
lines changed

ch05/11_qwen3/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -255,7 +255,7 @@ The following table shows a performance comparison on an A100 for consequent `ge
255255
| Qwen3Model compiled | 107 | 1.99 GB |
256256

257257
 
258-
#### Pro tip 2: speed up inference with compilation
258+
#### Pro tip 2: speed up inference with KV cache
259259

260260
You can significantly boost inference performance using the KV cache `Qwen3Model` drop-in replacement when running the model on a CPU. (See my [Understanding and Coding the KV Cache in LLMs from Scratch](https://magazine.sebastianraschka.com/p/coding-the-kv-cache-in-llms) article to learn more about KV caches.)
261261

ch05/11_qwen3/standalone-qwen3.ipynb

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -822,7 +822,7 @@
822822
},
823823
{
824824
"cell_type": "code",
825-
"execution_count": 37,
825+
"execution_count": null,
826826
"id": "699cb1b8-a67d-49fb-80a6-0dad9d81f392",
827827
"metadata": {
828828
"colab": {
@@ -936,7 +936,8 @@
936936
" weights_dict.update(shard)\n",
937937
"\n",
938938
"load_weights_into_qwen(model, QWEN3_CONFIG, weights_dict)\n",
939-
"model.to(device);"
939+
"model.to(device)\n",
940+
"del weights_dict"
940941
]
941942
},
942943
{
@@ -1187,7 +1188,7 @@
11871188
"provenance": []
11881189
},
11891190
"kernelspec": {
1190-
"display_name": "Python 3 (ipykernel)",
1191+
"display_name": ".venv",
11911192
"language": "python",
11921193
"name": "python3"
11931194
},
@@ -1201,7 +1202,7 @@
12011202
"name": "python",
12021203
"nbconvert_exporter": "python",
12031204
"pygments_lexer": "ipython3",
1204-
"version": "3.11.9"
1205+
"version": "3.12.6"
12051206
}
12061207
},
12071208
"nbformat": 4,

0 commit comments

Comments
 (0)