You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/reference-model-inference-api.md
+30-29Lines changed: 30 additions & 29 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -52,7 +52,7 @@ The API is compatible with Azure OpenAI model deployments.
52
52
53
53
## Capabilities
54
54
55
-
The following section describes some of the capabilities the API exposes. For a full specification of the API, view the [OpenAPI definition](https://github.com/Azure/azureai-model-inference-api).
55
+
The following section describes some of the capabilities the API exposes. For a full specification of the API, view the [reference section](reference-model-inference-info.md).
56
56
57
57
### Modalities
58
58
@@ -64,18 +64,21 @@ The API indicates how developers can consume predictions for the following modal
64
64
*[Chat completions](reference-model-inference-chat-completions.md): Creates a model response for the given chat conversation.
65
65
*[Image embeddings](reference-model-inference-images-embeddings.md): Creates an embedding vector representing the input text and image.
66
66
67
-
### Models with disparate set of capabilities
67
+
### Extensibility
68
68
69
-
The Azure AI Model Inference API indicates a general set of capabilities but each of the models can decide to implement them or not. A specific error is returned on those cases where the model can't support a specific parameter.
69
+
The Azure AI Model Inference API specifies a set of modalities and parameters that models can subscribe to. However, some models may have further capabilities that the ones the API indicates. On those cases, the API allows the developer to pass them as extra parameters in the payload.
70
70
71
-
The following example shows the response for a chat completion request indicating the parameter `reponse_format` and asking for a reply in `JSON` format. In the example, since the model doesn't support such capability an error 422 is returned to the user.
71
+
By setting a header `extra-parameters: allow`, the API will attempt to pass any unknown parameter directly to the underlying model. If the model can handle that parameter, the request completes.
72
+
73
+
The following example shows a request passing the parameter `safe_prompt` supported by Mistral-Large, which isn't specified in the Azure AI Model Inference API:
72
74
73
75
__Request__
74
76
75
77
```HTTP/1.1
76
78
POST /chat/completions?api-version=2024-05-01-preview
"message": "One of the parameters contain invalid values."
98
+
"response_format": { "type": "text" },
99
+
"safe_prompt": true
110
100
}
111
101
```
112
102
113
103
> [!TIP]
114
-
> You can inspect the property `details.loc` to understand the location of the offending parameter and `details.input` to see the value that was passed in the request.
115
-
116
-
### Extensibility
104
+
> Alternatively, you can set `extra-parameters: drop` to drop any unknown parameter in the request. Use this capability in case you happen to be sending requests with extra parameters that you know the model won't support but you want the request to completes anyway. A typical example of this is indicating `seed` parameter.
117
105
118
-
The Azure AI Model Inference API specifies a set of modalities and parameters that models can subscribe to. However, some models may have further capabilities that the ones the API indicates. On those cases, the API allows the developer to pass them as extra parameters in the payload.
106
+
### Models with disparate set of capabilities
119
107
120
-
By setting a header `extra-parameters: allow`, the API will attempt to pass any unknown parameter directly to the underlying model. If the model can handle that parameter, the request completes.
108
+
The Azure AI Model Inference API indicates a general set of capabilities but each of the models can decide to implement them or not. A specific error is returned on those cases where the model can't support a specific parameter.
121
109
122
-
The following example shows a request passing the parameter `safe_prompt` supported by Mistral-Large, which isn't specified in the Azure AI Model Inference API:
110
+
The following example shows the response for a chat completion request indicating the parameter `reponse_format` and asking for a reply in `JSON` format. In the example, since the model doesn't support such capability an error 422 is returned to the user.
123
111
124
112
__Request__
125
113
126
114
```HTTP/1.1
127
115
POST /chat/completions?api-version=2024-05-01-preview
128
116
Authorization: Bearer <bearer-token>
129
117
Content-Type: application/json
130
-
extra-parameters: allow
131
118
```
132
119
133
120
```JSON
@@ -144,13 +131,27 @@ extra-parameters: allow
144
131
],
145
132
"temperature": 0,
146
133
"top_p": 1,
147
-
"response_format": { "type": "text" },
148
-
"safe_prompt": true
134
+
"response_format": { "type": "json_object" },
135
+
}
136
+
```
137
+
138
+
__Response__
139
+
140
+
```JSON
141
+
{
142
+
"status": 422,
143
+
"code": "parameter_not_supported",
144
+
"detail": {
145
+
"loc": [ "body", "response_format" ],
146
+
"input": "json_object"
147
+
},
148
+
"message": "One of the parameters contain invalid values."
149
149
}
150
150
```
151
151
152
152
> [!TIP]
153
-
> Alternatively, you can set `extra-parameters: drop` to drop any unknown parameter in the request. Use this capability in case you happen to be sending requests with extra parameters that you know the model won't support but you want the request to completes anyway. A typical example of this is indicating `seed` parameter.
153
+
> You can inspect the property `details.loc` to understand the location of the offending parameter and `details.input` to see the value that was passed in the request.
0 commit comments