|
96 | 96 | "Streaming allows the generated tokens to be sent as data-only server-sent events whenever they become available. \n",
|
97 | 97 | "This is extremely important for interactive applications such as chatbots, so the user is always engaged. \n",
|
98 | 98 | "\n",
|
99 |
| - "To use streaming, simply set `\"stream\":\"True\"` as part of the request payload. \n", |
| 99 | + "To use streaming, simply set `\"stream\":True` as part of the request payload. \n", |
100 | 100 | "In the streaming mode, the REST API response will be different from non-streaming mode.\n",
|
101 | 101 | "\n",
|
102 | 102 | "Here is an example: "
|
|
108 | 108 | "metadata": {},
|
109 | 109 | "outputs": [],
|
110 | 110 | "source": [
|
111 |
| - "!curl -X POST -L https://your-endpoint.inference.ai.azure.com/v1/chat/completions -H 'Content-Type: application/json' -H 'Authorization: your-auth-key' -d '{\"messages\":[{\"content\":\"You are a helpful assistant.\",\"role\":\"system\"},{\"content\":\"Who wrote the book Innovators dilemma?\",\"role\":\"user\"}], \"max_tokens\": 500, \"stream\": \"True\"}'" |
| 111 | + "!curl -X POST -L https://your-endpoint.inference.ai.azure.com/v1/chat/completions -H 'Content-Type: application/json' -H 'Authorization: your-auth-key' -d '{\"messages\":[{\"content\":\"You are a helpful assistant.\",\"role\":\"system\"},{\"content\":\"Who wrote the book Innovators dilemma?\",\"role\":\"user\"}], \"max_tokens\": 500, \"stream\": True}'" |
112 | 112 | ]
|
113 | 113 | },
|
114 | 114 | {
|
|
170 | 170 | " {\"role\":\"user\", \"content\":\"Who wrote the book Innovators dilemma?\"}], \n",
|
171 | 171 | " \"max_tokens\": 500,\n",
|
172 | 172 | " \"temperature\": 0.9,\n",
|
173 |
| - " \"stream\": \"True\",\n", |
| 173 | + " \"stream\": True,\n", |
174 | 174 | "}\n",
|
175 | 175 | "\n",
|
176 | 176 | "body = str.encode(json.dumps(data))\n",
|
|
230 | 230 | " {\"role\":\"user\", \"content\":\"Who wrote the book Innovators dilemma?\"}],\n",
|
231 | 231 | " \"max_tokens\": 500,\n",
|
232 | 232 | " \"temperature\": 0.9,\n",
|
233 |
| - " \"stream\": \"True\"\n", |
| 233 | + " \"stream\": True\n", |
234 | 234 | "}\n",
|
235 | 235 | "\n",
|
236 | 236 | "\n",
|
|
0 commit comments