Skip to content

Commit b7ec5d6

Browse files
committed
links updated
1 parent 1ebdedb commit b7ec5d6

File tree

1 file changed

+30
-29
lines changed

1 file changed

+30
-29
lines changed

articles/machine-learning/reference-model-inference-api.md

Lines changed: 30 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ The API is compatible with Azure OpenAI model deployments.
5252

5353
## Capabilities
5454

55-
The following section describes some of the capabilities the API exposes. For a full specification of the API, view the [OpenAPI definition](https://github.com/Azure/azureai-model-inference-api).
55+
The following section describes some of the capabilities the API exposes. For a full specification of the API, view the [reference section](reference-model-inference-info.md).
5656

5757
### Modalities
5858

@@ -64,18 +64,21 @@ The API indicates how developers can consume predictions for the following modal
6464
* [Chat completions](reference-model-inference-chat-completions.md): Creates a model response for the given chat conversation.
6565
* [Image embeddings](reference-model-inference-images-embeddings.md): Creates an embedding vector representing the input text and image.
6666

67-
### Models with disparate set of capabilities
67+
### Extensibility
6868

69-
The Azure AI Model Inference API indicates a general set of capabilities but each of the models can decide to implement them or not. A specific error is returned on those cases where the model can't support a specific parameter.
69+
The Azure AI Model Inference API specifies a set of modalities and parameters that models can subscribe to. However, some models may have further capabilities that the ones the API indicates. On those cases, the API allows the developer to pass them as extra parameters in the payload.
7070

71-
The following example shows the response for a chat completion request indicating the parameter `reponse_format` and asking for a reply in `JSON` format. In the example, since the model doesn't support such capability an error 422 is returned to the user.
71+
By setting a header `extra-parameters: allow`, the API will attempt to pass any unknown parameter directly to the underlying model. If the model can handle that parameter, the request completes.
72+
73+
The following example shows a request passing the parameter `safe_prompt` supported by Mistral-Large, which isn't specified in the Azure AI Model Inference API:
7274

7375
__Request__
7476

7577
```HTTP/1.1
7678
POST /chat/completions?api-version=2024-05-01-preview
7779
Authorization: Bearer <bearer-token>
7880
Content-Type: application/json
81+
extra-parameters: allow
7982
```
8083

8184
```JSON
@@ -92,42 +95,26 @@ Content-Type: application/json
9295
],
9396
"temperature": 0,
9497
"top_p": 1,
95-
"response_format": { "type": "json_object" },
96-
}
97-
```
98-
99-
__Response__
100-
101-
```JSON
102-
{
103-
"status": 422,
104-
"code": "parameter_not_supported",
105-
"detail": {
106-
"loc": [ "body", "response_format" ],
107-
"input": "json_object"
108-
},
109-
"message": "One of the parameters contain invalid values."
98+
"response_format": { "type": "text" },
99+
"safe_prompt": true
110100
}
111101
```
112102

113103
> [!TIP]
114-
> You can inspect the property `details.loc` to understand the location of the offending parameter and `details.input` to see the value that was passed in the request.
115-
116-
### Extensibility
104+
> Alternatively, you can set `extra-parameters: drop` to drop any unknown parameter in the request. Use this capability in case you happen to be sending requests with extra parameters that you know the model won't support but you want the request to completes anyway. A typical example of this is indicating `seed` parameter.
117105
118-
The Azure AI Model Inference API specifies a set of modalities and parameters that models can subscribe to. However, some models may have further capabilities that the ones the API indicates. On those cases, the API allows the developer to pass them as extra parameters in the payload.
106+
### Models with disparate set of capabilities
119107

120-
By setting a header `extra-parameters: allow`, the API will attempt to pass any unknown parameter directly to the underlying model. If the model can handle that parameter, the request completes.
108+
The Azure AI Model Inference API indicates a general set of capabilities but each of the models can decide to implement them or not. A specific error is returned on those cases where the model can't support a specific parameter.
121109

122-
The following example shows a request passing the parameter `safe_prompt` supported by Mistral-Large, which isn't specified in the Azure AI Model Inference API:
110+
The following example shows the response for a chat completion request indicating the parameter `reponse_format` and asking for a reply in `JSON` format. In the example, since the model doesn't support such capability an error 422 is returned to the user.
123111

124112
__Request__
125113

126114
```HTTP/1.1
127115
POST /chat/completions?api-version=2024-05-01-preview
128116
Authorization: Bearer <bearer-token>
129117
Content-Type: application/json
130-
extra-parameters: allow
131118
```
132119

133120
```JSON
@@ -144,13 +131,27 @@ extra-parameters: allow
144131
],
145132
"temperature": 0,
146133
"top_p": 1,
147-
"response_format": { "type": "text" },
148-
"safe_prompt": true
134+
"response_format": { "type": "json_object" },
135+
}
136+
```
137+
138+
__Response__
139+
140+
```JSON
141+
{
142+
"status": 422,
143+
"code": "parameter_not_supported",
144+
"detail": {
145+
"loc": [ "body", "response_format" ],
146+
"input": "json_object"
147+
},
148+
"message": "One of the parameters contain invalid values."
149149
}
150150
```
151151

152152
> [!TIP]
153-
> Alternatively, you can set `extra-parameters: drop` to drop any unknown parameter in the request. Use this capability in case you happen to be sending requests with extra parameters that you know the model won't support but you want the request to completes anyway. A typical example of this is indicating `seed` parameter.
153+
> You can inspect the property `details.loc` to understand the location of the offending parameter and `details.input` to see the value that was passed in the request.
154+
154155

155156
## Getting started
156157

0 commit comments

Comments
 (0)