You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/ai-studio/reference/reference-model-inference-api.md
+29-10Lines changed: 29 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -75,9 +75,17 @@ The API indicates how developers can consume predictions for the following modal
75
75
76
76
### Inference SDK support
77
77
78
-
You can use streamlined inference clients in the language of your choice to consume predictions from models running the API.
78
+
You can use streamlined inference clients in the language of your choice to consume predictions from models running the Azure AI model inference API.
79
79
80
-
# [REST](#tab/python)
80
+
# [Python](#tab/python)
81
+
82
+
Install the package `azure-ai-inference` using your package manager, like pip:
83
+
84
+
```bash
85
+
pip install azure-ai-inference
86
+
```
87
+
88
+
Then, you can use the package to consume the model. The following example shows how to create a client to consume chat completions:
81
89
82
90
```python
83
91
import os
@@ -90,7 +98,15 @@ model = ChatCompletionsClient(
90
98
)
91
99
```
92
100
93
-
# [REST](#tab/javascript)
101
+
# [JavaScript](#tab/javascript)
102
+
103
+
Install the package `@azure-rest/ai-inference` using npm:
104
+
105
+
```bash
106
+
npm install @azure-rest/ai-inference
107
+
```
108
+
109
+
Then, you can use the package to consume the model. The following example shows how to create a client to consume chat completions:
94
110
95
111
```javascript
96
112
importModelClientfrom"@azure-rest/ai-inference";
@@ -114,7 +130,6 @@ POST /chat/completions?api-version=2024-04-01-preview
114
130
Authorization: Bearer <bearer-token>
115
131
Content-Type: application/json
116
132
```
117
-
118
133
---
119
134
120
135
### Extensibility
@@ -125,7 +140,7 @@ By setting a header `extra-parameters: allow`, the API will attempt to pass any
125
140
126
141
The following example shows a request passing the parameter `safe_prompt` supported by Mistral-Large, which isn't specified in the Azure AI Model Inference API:
127
142
128
-
# [REST](#tab/python)
143
+
# [Python](#tab/python)
129
144
130
145
```python
131
146
response = model.complete(
@@ -139,7 +154,7 @@ response = model.complete(
139
154
)
140
155
```
141
156
142
-
# [REST](#tab/javascript)
157
+
# [JavaScript](#tab/javascript)
143
158
144
159
```javascript
145
160
var messages = [
@@ -196,7 +211,7 @@ The Azure AI Model Inference API indicates a general set of capabilities but eac
196
211
197
212
The following example shows the response for a chat completion request indicating the parameter `reponse_format` and asking for a reply in `JSON` format. In the example, since the model doesn't support such capability an error 422 is returned to the user.
198
213
199
-
# [REST](#tab/python)
214
+
# [Python](#tab/python)
200
215
201
216
```python
202
217
from azure.ai.inference.models import ChatCompletionsResponseFormat
@@ -225,7 +240,7 @@ except HttpResponseError as ex:
225
240
raise ex
226
241
```
227
242
228
-
# [REST](#tab/python)
243
+
# [JavaScript](#tab/python)
229
244
230
245
```javascript
231
246
try {
@@ -311,7 +326,7 @@ The Azure AI model inference API supports [Azure AI Content Safety](../concepts/
311
326
312
327
The following example shows the response for a chat completion request that has triggered content safety.
313
328
314
-
# [REST](#tab/python)
329
+
# [Python](#tab/python)
315
330
316
331
```python
317
332
from azure.ai.inference.models import AssistantMessage, UserMessage, SystemMessage
@@ -337,7 +352,7 @@ except HttpResponseError as ex:
337
352
raise ex
338
353
```
339
354
340
-
# [REST](#tab/javascript)
355
+
# [JavaScript](#tab/javascript)
341
356
342
357
```javascript
343
358
try {
@@ -407,3 +422,7 @@ __Response__
407
422
}
408
423
```
409
424
---
425
+
426
+
## Getting started
427
+
428
+
The Azure AI Model Inference API is currently supported in certain models deployed as [Serverless API endpoints](../how-to/deploy-models-serverless.md) and Managed Online Endpoints. Deploy any of the [supported models](#availability) and use the exact same code to consume their predictions.
0 commit comments