Skip to content

Commit 6b5f373

Browse files
emre-openaiemre-openai
committed
Merge branch 'emre/agents-context-management' of https://github.com/openai/openai-cookbook into emre/agents-context-management
2 parents f750cfa + 4612c8e commit 6b5f373

File tree

272 files changed

+28803
-71
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

272 files changed

+28803
-71
lines changed

‎articles/gpt-oss/fine-tune-korean.ipynb‎

Lines changed: 1196 additions & 0 deletions
Large diffs are not rendered by default.

‎articles/gpt-oss/fine-tune-transfomers.ipynb‎

Lines changed: 673 additions & 0 deletions
Large diffs are not rendered by default.

‎articles/gpt-oss/handle-raw-cot.md‎

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
# How to handle the raw chain of thought in gpt-oss
2+
3+
The [gpt-oss models](https://openai.com/open-models) provide access to a raw chain of thought (CoT) meant for analysis and safety research by model implementors, but it’s also crucial for the performance of tool calling, as tool calls can be performed as part of the CoT. At the same time, the raw CoT might contain potentially harmful content or could reveal information to users that the person implementing the model might not intend (like rules specified in the instructions given to the model). You therefore should not show raw CoT to end users.
4+
5+
## Harmony / chat template handling
6+
7+
The model encodes its raw CoT as part of our [harmony response format](https://cookbook.openai.com/articles/openai-harmony). If you are authoring your own chat templates or are handling tokens directly, make sure to [check out harmony guide first](https://cookbook.openai.com/articles/openai-harmony).
8+
9+
To summarize a couple of things:
10+
11+
1. CoT will be issued to the `analysis` channel
12+
2. After a message to the `final` channel in a subsequent sampling turn all `analysis` messages should be dropped. Function calls to the `commentary` channel can remain
13+
3. If the last message by the assistant was a tool call of any type, the analysis messages until the previous `final` message should be preserved on subsequent sampling until a `final` message gets issued
14+
15+
## Chat Completions API
16+
17+
If you are implementing a Chat Completions API, there is no official spec for handling chain of thought in the published OpenAI specs, as our hosted models will not offer this feature for the time being. We ask you to follow [the following convention from OpenRouter instead](https://openrouter.ai/docs/use-cases/reasoning-tokens). Including:
18+
19+
1. Raw CoT will be returned as part of the response unless `reasoning: { exclude: true }` is specified as part of the request. [See details here](https://openrouter.ai/docs/use-cases/reasoning-tokens#legacy-parameters)
20+
2. The raw CoT is exposed as a `reasoning` property on the message in the output
21+
3. For delta events the delta has a `reasoning` property
22+
4. On subsequent turns you should be able to receive the previous reasoning (as `reasoning`) and handle it in accordance with the behavior specified in the chat template section above.
23+
24+
When in doubt, please follow the convention / behavior of the OpenRouter implementation.
25+
26+
## Responses API
27+
28+
For the Responses API we augmented our Responses API spec to cover this case. Below are the changes to the spec as type definitions. At a high level we are:
29+
30+
1. Introducing a new `content` property on `reasoning`. This allows a reasoning `summary` that could be displayed to the end user to be returned at the same time as the raw CoT (which should not be shown to the end user, but which might be helpful for interpretability research).
31+
2. Introducing a new content type called `reasoning_text`
32+
3. Introducing two new events `response.reasoning_text.delta` to stream the deltas of the raw CoT and `response.reasoning_text.done` to indicate a turn of CoT to be completed
33+
4. On subsequent turns you should be able to receive the previous reasoning and handle it in accordance with the behavior specified in the chat template section above.
34+
35+
**Item type changes**
36+
37+
```typescript
38+
type ReasoningItem = {
39+
id: string;
40+
type: "reasoning";
41+
summary: SummaryContent[];
42+
// new
43+
content: ReasoningTextContent[];
44+
};
45+
46+
type ReasoningTextContent = {
47+
type: "reasoning_text";
48+
text: string;
49+
};
50+
51+
type ReasoningTextDeltaEvent = {
52+
type: "response.reasoning_text.delta";
53+
sequence_number: number;
54+
item_id: string;
55+
output_index: number;
56+
content_index: number;
57+
delta: string;
58+
};
59+
60+
type ReasoningTextDoneEvent = {
61+
type: "response.reasoning_text.done";
62+
sequence_number: number;
63+
item_id: string;
64+
output_index: number;
65+
content_index: number;
66+
text: string;
67+
};
68+
```
69+
70+
**Event changes**
71+
72+
```typescript
73+
...
74+
{
75+
type: "response.content_part.added"
76+
...
77+
}
78+
{
79+
type: "response.reasoning_text.delta",
80+
sequence_number: 14,
81+
item_id: "rs_67f47a642e788191aec9b5c1a35ab3c3016f2c95937d6e91",
82+
output_index: 0,
83+
content_index: 0,
84+
delta: "The "
85+
}
86+
...
87+
{
88+
type: "response.reasoning_text.done",
89+
sequence_number: 18,
90+
item_id: "rs_67f47a642e788191aec9b5c1a35ab3c3016f2c95937d6e91",
91+
output_index: 0,
92+
content_index: 0,
93+
text: "The user asked me to think"
94+
}
95+
```
96+
97+
**Example responses output**
98+
99+
```typescript
100+
"output": [
101+
{
102+
"type": "reasoning",
103+
"id": "rs_67f47a642e788191aec9b5c1a35ab3c3016f2c95937d6e91",
104+
"summary": [
105+
{
106+
"type": "summary_text",
107+
"text": "**Calculating volume of gold for Pluto layer**\n\nStarting with the approximation..."
108+
}
109+
],
110+
"content": [
111+
{
112+
"type": "reasoning_text",
113+
"text": "The user asked me to think..."
114+
}
115+
]
116+
}
117+
]
118+
119+
```
120+
121+
## Displaying raw CoT to end-users
122+
123+
If you are providing a chat interface to users, you should not show the raw CoT because it might contain potentially harmful content or other information that you might not intend to show to users (like, for example, instructions in the developer message). Instead, we recommend showing a summarized CoT, similar to our production implementations in the API or ChatGPT, where a summarizer model reviews and blocks harmful content from being shown.

‎articles/gpt-oss/run-colab.ipynb‎

Lines changed: 247 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,247 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {
6+
"id": "wGfI8meEHXfM"
7+
},
8+
"source": [
9+
"[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openai/openai-cookbook/blob/main/articles/gpt-oss/run-colab.ipynb)"
10+
]
11+
},
12+
{
13+
"cell_type": "markdown",
14+
"metadata": {
15+
"id": "gj6KvThm8Jjn"
16+
},
17+
"source": [
18+
"# Run OpenAI gpt-oss 20B in a FREE Google Colab\n",
19+
"\n",
20+
"OpenAI released `gpt-oss` [120B](https://hf.co/openai/gpt-oss-120b) and [20B](https://hf.co/openai/gpt-oss-20b). Both models are Apache 2.0 licensed.\n",
21+
"\n",
22+
"Specifically, `gpt-oss-20b` was made for lower latency and local or specialized use cases (21B parameters with 3.6B active parameters).\n",
23+
"\n",
24+
"Since the models were trained in native MXFP4 quantization it makes it easy to run the 20B even in resource constrained environments like Google Colab.\n",
25+
"\n",
26+
"Authored by: [Pedro](https://huggingface.co/pcuenq) and [VB](https://huggingface.co/reach-vb)"
27+
]
28+
},
29+
{
30+
"cell_type": "markdown",
31+
"metadata": {
32+
"id": "Kv2foJJa9Xkc"
33+
},
34+
"source": [
35+
"## Setup environment"
36+
]
37+
},
38+
{
39+
"cell_type": "markdown",
40+
"metadata": {
41+
"id": "zMRXDOpY1Q3Q"
42+
},
43+
"source": [
44+
"Since support for mxfp4 in transformers is bleeding edge, we need a recent version of PyTorch and CUDA, in order to be able to install the `mxfp4` triton kernels.\n",
45+
"\n",
46+
"We also need to install transformers from source, and we uninstall `torchvision` and `torchaudio` to remove dependency conflicts."
47+
]
48+
},
49+
{
50+
"cell_type": "code",
51+
"execution_count": null,
52+
"metadata": {
53+
"id": "4gUEKrLEvJmf"
54+
},
55+
"outputs": [],
56+
"source": [
57+
"!pip install -q --upgrade torch"
58+
]
59+
},
60+
{
61+
"cell_type": "code",
62+
"execution_count": null,
63+
"metadata": {
64+
"id": "3N00UT7gtpkp"
65+
},
66+
"outputs": [],
67+
"source": [
68+
"!pip install -q transformers triton==3.4 kernels"
69+
]
70+
},
71+
{
72+
"cell_type": "code",
73+
"execution_count": null,
74+
"metadata": {
75+
"id": "7GW0knW2w3ND"
76+
},
77+
"outputs": [],
78+
"source": [
79+
"!pip uninstall -q torchvision torchaudio -y"
80+
]
81+
},
82+
{
83+
"cell_type": "markdown",
84+
"metadata": {
85+
"id": "pxU0WKwtH19m"
86+
},
87+
"source": [
88+
"Please, restart your Colab runtime session after installing the packages above."
89+
]
90+
},
91+
{
92+
"cell_type": "markdown",
93+
"metadata": {
94+
"id": "D3xCxY159frD"
95+
},
96+
"source": [
97+
"## Load the model from Hugging Face in Google Colab\n",
98+
"\n",
99+
"We load the model from here: [openai/gpt-oss-20b](https://hf.co/openai/gpt-oss-20b)"
100+
]
101+
},
102+
{
103+
"cell_type": "code",
104+
"execution_count": null,
105+
"metadata": {
106+
"id": "k2HFwdkXu2R1"
107+
},
108+
"outputs": [],
109+
"source": [
110+
"from transformers import AutoModelForCausalLM, AutoTokenizer\n",
111+
"\n",
112+
"model_id = \"openai/gpt-oss-20b\"\n",
113+
"\n",
114+
"tokenizer = AutoTokenizer.from_pretrained(model_id)\n",
115+
"model = AutoModelForCausalLM.from_pretrained(\n",
116+
" model_id,\n",
117+
" torch_dtype=\"auto\",\n",
118+
" device_map=\"cuda\",\n",
119+
")"
120+
]
121+
},
122+
{
123+
"cell_type": "markdown",
124+
"metadata": {
125+
"id": "Jbeq6kN79ql0"
126+
},
127+
"source": [
128+
"## Setup messages/ chat\n",
129+
"\n",
130+
"You can provide an optional system prompt or directly the input."
131+
]
132+
},
133+
{
134+
"cell_type": "code",
135+
"execution_count": null,
136+
"metadata": {
137+
"id": "P5dJV3xsu_89"
138+
},
139+
"outputs": [],
140+
"source": [
141+
"messages = [\n",
142+
" {\"role\": \"system\", \"content\": \"Always respond in riddles\"},\n",
143+
" {\"role\": \"user\", \"content\": \"What is the weather like in Madrid?\"},\n",
144+
"]\n",
145+
"\n",
146+
"inputs = tokenizer.apply_chat_template(\n",
147+
" messages,\n",
148+
" add_generation_prompt=True,\n",
149+
" return_tensors=\"pt\",\n",
150+
" return_dict=True,\n",
151+
").to(model.device)\n",
152+
"\n",
153+
"generated = model.generate(**inputs, max_new_tokens=500)\n",
154+
"print(tokenizer.decode(generated[0][inputs[\"input_ids\"].shape[-1]:]))"
155+
]
156+
},
157+
{
158+
"cell_type": "markdown",
159+
"metadata": {
160+
"id": "ksxo7bjR_-th"
161+
},
162+
"source": [
163+
"## Specify Reasoning Effort"
164+
]
165+
},
166+
{
167+
"cell_type": "markdown",
168+
"metadata": {
169+
"id": "fcv6QdcQKLr0"
170+
},
171+
"source": [
172+
"Simply pass it as an additional argument to `apply_chat_template()`. Supported values are `\"low\"`, `\"medium\"` (default), or `\"high\"`."
173+
]
174+
},
175+
{
176+
"cell_type": "code",
177+
"execution_count": null,
178+
"metadata": {
179+
"id": "CmnkAle608Hl"
180+
},
181+
"outputs": [],
182+
"source": [
183+
"messages = [\n",
184+
" {\"role\": \"system\", \"content\": \"Always respond in riddles\"},\n",
185+
" {\"role\": \"user\", \"content\": \"Explain why the meaning of life is 42\"},\n",
186+
"]\n",
187+
"\n",
188+
"inputs = tokenizer.apply_chat_template(\n",
189+
" messages,\n",
190+
" add_generation_prompt=True,\n",
191+
" return_tensors=\"pt\",\n",
192+
" return_dict=True,\n",
193+
" reasoning_effort=\"high\",\n",
194+
").to(model.device)\n",
195+
"\n",
196+
"generated = model.generate(**inputs, max_new_tokens=500)\n",
197+
"print(tokenizer.decode(generated[0][inputs[\"input_ids\"].shape[-1]:]))"
198+
]
199+
},
200+
{
201+
"cell_type": "markdown",
202+
"metadata": {
203+
"id": "Tf2-ocGqEC_r"
204+
},
205+
"source": [
206+
"## Try out other prompts and ideas!\n",
207+
"\n",
208+
"Check out our blogpost for other ideas: [https://hf.co/blog/welcome-openai-gpt-oss](https://hf.co/blog/welcome-openai-gpt-oss)"
209+
]
210+
},
211+
{
212+
"cell_type": "code",
213+
"execution_count": null,
214+
"metadata": {
215+
"id": "2QrnTpcCKd_n"
216+
},
217+
"outputs": [],
218+
"source": []
219+
}
220+
],
221+
"metadata": {
222+
"accelerator": "GPU",
223+
"colab": {
224+
"gpuType": "T4",
225+
"provenance": []
226+
},
227+
"kernelspec": {
228+
"display_name": "Python 3 (ipykernel)",
229+
"language": "python",
230+
"name": "python3"
231+
},
232+
"language_info": {
233+
"codemirror_mode": {
234+
"name": "ipython",
235+
"version": 3
236+
},
237+
"file_extension": ".py",
238+
"mimetype": "text/x-python",
239+
"name": "python",
240+
"nbconvert_exporter": "python",
241+
"pygments_lexer": "ipython3",
242+
"version": "3.12.7"
243+
}
244+
},
245+
"nbformat": 4,
246+
"nbformat_minor": 0
247+
}

0 commit comments

Comments
 (0)