Skip to content

Commit 633b153

Browse files
author
zhiweiz
committed
add more docs for envs
1 parent c6fd13e commit 633b153

File tree

1 file changed

+27
-15
lines changed

1 file changed

+27
-15
lines changed

OpenAI/GPT-OSS.md

Lines changed: 27 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -171,24 +171,22 @@ vllm serve ... --tool-server ip-1:port-1,ip-2:port-2
171171

172172
The URLs are expected to be MCP SSE servers that implement `instructions` in server info and well documented tools. The tools will be injected into the system prompt for the model to enable them.
173173

174-
GPT OSS also expects a builtin tool called container. It doesn't have exposed tool type in openai types.
174+
GPT OSS also expects a built-in tool called container. It doesn't have exposed tool type in openai types.
175175
For reference the container tool is a stateful docker container that can be used to run command line tools.
176176
The enabled tool namespace is `container` and the tool name used the most is `exec`.
177-
MCP server need to implement the following functions to support container tool:
178-
```
179-
- for tool name: exec
180-
- args:
181-
{
182-
"cmd":List[str] "command to execute",
183-
"workdir":optional[str] "current working directory",
184-
"env":optional[object/dict] "environment variables",
185-
"session_name":optional[str] "session name",
186-
"timeout":optional[int] "timeout in seconds",
187-
"user":optional[str] "user name",
188-
}
177+
MCP server needs to implement the following functions to support container tool:
178+
```
179+
Tool: exec
180+
Arguments:
181+
- cmd (List[str]): command to execute
182+
- workdir (Optional[str]): current working directory
183+
- env (Optional[Dict[str, str]]): environment variables
184+
- session_name (Optional[str]): session name
185+
- timeout (Optional[int]): timeout in seconds
186+
- user (Optional[str]): user name
189187
Signature:
190-
async def exec(ctx: Context, rest_of_the_args) -> str
191-
expect ctx to contain a session id to identify the container session and make it stateful
188+
async def exec(ctx: Context, **kwargs) -> str
189+
# Note: `ctx` is expected to contain a session id to identify the container session and make it stateful.
192190
```
193191
Container tool runtime implementation can be referenced from https://github.com/SWE-agent/SWE-ReX
194192
The docker image might need to have some similar features as codex supports
@@ -197,6 +195,13 @@ To enable container tool in vllm before openai types has it, Add below
197195
export VLLM_ENABLE_CONTAINER_TOOL=1
198196
```
199197

198+
Skipping Tool descriptions
199+
Some tools don't expect json format like built-in python.
200+
Remove json description from system prompt by set the below
201+
```
202+
export VLLM_GPT_OSS_NO_TOOL_DESCRIPTION_NAMESPACES="python"
203+
```
204+
200205
## Accuracy Evaluation Panels
201206

202207
OpenAI recommends using the gpt-oss reference library to perform evaluation.
@@ -298,3 +303,10 @@ Meaning:
298303
If you want to use offline inference, you can treat vLLM as a token-in-token-out service and pass in tokens that are already formatted with Harmony.
299304

300305
For function calling, only tool_choice="auto" is supported.
306+
307+
Harmony also only supports instructions in developer message but to achieve better alignment with training
308+
It it preferred to place it in system message.
309+
Enable below to move instructions to system message
310+
```
311+
export VLLM_GPT_OSS_HARMONY_SYSTEM_INSTRUCTIONS=1
312+
```

0 commit comments

Comments
 (0)