Merge pull request #278098 from MicrosoftDocs/main

6/13 11:00 AM IST Publish

Author: PhilKang0704

articles/ai-studio/reference/reference-model-inference-api.md

@@ -32,7 +32,7 @@ While foundational models excel in specific domains, they lack a uniform set of
 > * Use smaller models that can run faster on specific tasks.
 > * Compose multiple models to develop intelligent experiences.
 
-Having a uniform way to consume foundational models allow developers to realize all those benefits without changing a single line of code on their applications.
+Having a uniform way to consume foundational models allow developers to realize all those benefits without sacrificing portability or changing the underlying code.
 
 ## Availability
 
@@ -43,8 +43,8 @@ Models deployed to [serverless API endpoints](../how-to/deploy-models-serverless
 > [!div class="checklist"]
 > * [Cohere Embed V3](../how-to/deploy-models-cohere-embed.md) family of models
 > * [Cohere Command R](../how-to/deploy-models-cohere-command.md) family of models
-> * [Meta Llama 2](../how-to/deploy-models-llama.md) family of models
-> * [Meta Llama 3](../how-to/deploy-models-llama.md) family of models
+> * [Meta Llama 2 chat](../how-to/deploy-models-llama.md) family of models
+> * [Meta Llama 3 instruct](../how-to/deploy-models-llama.md) family of models
 > * [Mistral-Small](../how-to/deploy-models-mistral.md)
 > * [Mistral-Large](../how-to/deploy-models-mistral.md)
 > * [Phi-3](../how-to/deploy-models-phi-3.md) family of models
@@ -154,6 +154,49 @@ __Response__
 > [!TIP]
 > You can inspect the property `details.loc` to understand the location of the offending parameter and `details.input` to see the value that was passed in the request.
 
+## Content safety
+
+The Azure AI model inference API supports [Azure AI Content Safety](../concepts/content-filtering.md). When using deployments with Azure AI Content Safety on, inputs and outputs pass through an ensemble of classification models aimed at detecting and preventing the output of harmful content. The content filtering system detects and takes action on specific categories of potentially harmful content in both input prompts and output completions.
+
+The following example shows the response for a chat completion request that has triggered content safety. 
+
+__Request__
+
+```HTTP/1.1
+POST /chat/completions?api-version=2024-04-01-preview
+Authorization: Bearer <bearer-token>
+Content-Type: application/json
+```
+
+```JSON
+{
+    "messages": [
+    {
+        "role": "system",
+        "content": "You are a helpful assistant"
+    },
+    {
+        "role": "user",
+        "content": "Chopping tomatoes and cutting them into cubes or wedges are great ways to practice your knife skills."
+    }
+    ],
+    "temperature": 0,
+    "top_p": 1,
+}
+```
+
+__Response__
+
+```JSON
+{
+    "status": 400,
+    "code": "content_filter",
+    "message": "The response was filtered",
+    "param": "messages",
+    "type": null
+}
+```
+
 ## Getting started
 
 The Azure AI Model Inference API is currently supported in models deployed as [Serverless API endpoints](../how-to/deploy-models-serverless.md). Deploy any of the [supported models](#availability) to a new [Serverless API endpoints](../how-to/deploy-models-serverless.md) to get started. Then you can consume the API in the following ways: 

articles/ai-studio/reference/reference-model-inference-chat-completions.md

@@ -30,6 +30,14 @@ POST /chat/completions?api-version=2024-04-01-preview
 | --- | --- | --- | --- | --- |
 | api-version | query | True | string | The version of the API in the format "YYYY-MM-DD" or "YYYY-MM-DD-preview". |
 
+## Request Header
+
+
+| Name | Required | Type | Description |
+| --- | --- | --- | --- |
+| extra-parameters | | string | The behavior of the API when extra parameters are indicated in the payload. Using `allow` makes the API to pass the parameter to the underlying model. Use this value when you want to pass parameters that you know the underlying model can support. Using `drop` makes the API to drop any unsupported parameter. Use this value when you need to use the same payload across different models, but one of the extra parameters may make a model to error out if not supported. Using `error` makes the API to reject any extra parameter in the payload. Only parameters specified in this API can be indicated, or a 400 error is returned. |
+| azureml-model-deployment |     | string | Name of the deployment you want to route the request to. Supported for endpoints that support multiple deployments. |
+
 ## Request Body
 
 | Name | Required | Type | Description |
@@ -113,7 +121,7 @@ POST /chat/completions?api-version=2024-04-01-preview
   "stream": false,
   "temperature": 0,
   "top_p": 1,
-  "response_format": "text"
+  "response_format": { "type": "text" }
 }
 ```
 
@@ -157,7 +165,8 @@ Status code: 200
 | [ChatCompletionFinishReason](#chatcompletionfinishreason) | The reason the model stopped generating tokens. This will be `stop` if the model hit a natural stop point or a provided stop sequence, `length` if the maximum number of tokens specified in the request was reached, `content_filter` if content was omitted due to a flag from our content filters, `tool_calls` if the model called a tool. |
 | [ChatCompletionMessageToolCall](#chatcompletionmessagetoolcall) |     |
 | [ChatCompletionObject](#chatcompletionobject) | The object type, which is always `chat.completion`. |
-| [ChatCompletionResponseFormat](#chatcompletionresponseformat) |     |
+| [ChatCompletionResponseFormat](#chatcompletionresponseformat) | The response format for the model response. Setting to `json_object` enables JSON mode, which guarantees the message the model generates is valid JSON. When using JSON mode, you **must** also instruct the model to produce JSON yourself via a system or user message. Also note that the message content may be partially cut off if `finish_reason="length"`, which indicates the generation exceeded `max_tokens` or the conversation exceeded the max context length.    |
+| [ChatCompletionResponseFormatType](#chatcompletionresponseformattype) | The response format type. |
 | [ChatCompletionResponseMessage](#chatcompletionresponsemessage) | A chat completion message generated by the model. |
 | [ChatCompletionTool](#chatcompletiontool) |     |
 | [ChatMessageRole](#chatmessagerole) | The role of the author of this message. |
@@ -166,15 +175,15 @@ Status code: 200
 | [ContentFilterError](#contentfiltererror) | The API call fails when the prompt triggers a content filter as configured. Modify the prompt and try again. |
 | [CreateChatCompletionRequest](#createchatcompletionrequest) |     |
 | [CreateChatCompletionResponse](#createchatcompletionresponse) | Represents a chat completion response returned by model, based on the provided input. |
-| [Detail](#detail) |     |
+| [Detail](#detail) | Details for the [UnprocessableContentError](#unprocessablecontenterror) error.    |
 | [Function](#function) | The function that the model called. |
-| [FunctionObject](#functionobject) |     |
+| [FunctionObject](#functionobject) | Definition of a function the model has access to.    |
 | [ImageDetail](#imagedetail)  | Specifies the detail level of the image. |
-| [NotFoundError](#notfounderror) |     |
+| [NotFoundError](#notfounderror) | The route is not valid for the deployed model.    |
 | [ToolType](#tooltype) | The type of the tool. Currently, only `function` is supported. |
-| [TooManyRequestsError](#toomanyrequestserror) |     |
-| [UnauthorizedError](#unauthorizederror) |     |
-| [UnprocessableContentError](#unprocessablecontenterror) |     |
+| [TooManyRequestsError](#toomanyrequestserror) | You have hit your assigned rate limit and your requests need to be paced.     |
+| [UnauthorizedError](#unauthorizederror) | Authentication is missing or invalid.    |
+| [UnprocessableContentError](#unprocessablecontenterror) | The request contains unprocessable content. The error is returned when the payload indicated is valid according to this specification. However, some of the instructions indicated in the payload are not supported by the underlying model. Use the `details` section to understand the offending parameter.    |
 
 
 ### ChatCompletionFinishReason
@@ -209,6 +218,15 @@ The object type, which is always `chat.completion`.
 
 ### ChatCompletionResponseFormat
 
+The response format for the model response. Setting to `json_object` enables JSON mode, which guarantees the message the model generates is valid JSON. When using JSON mode, you **must** also instruct the model to produce JSON yourself via a system or user message. Also note that the message content may be partially cut off if `finish_reason="length"`, which indicates the generation exceeded `max_tokens` or the conversation exceeded the max context length.
+
+| Name | Type | Description |
+| --- | --- | --- |
+| type | [ChatCompletionResponseFormatType](#chatcompletionresponseformattype) | The response format type. |
+
+### ChatCompletionResponseFormatType
+
+The response format type.
 
 | Name | Type | Description |
 | --- | --- | --- |
@@ -237,7 +255,6 @@ A chat completion message generated by the model.
 
 The role of the author of this message.
 
-
 | Name | Type | Description |
 | --- | --- | --- |
 | assistant | string |     |
@@ -249,7 +266,6 @@ The role of the author of this message.
 
 A list of chat completion choices. Can be more than one if `n` is greater than 1.
 
-
 | Name | Type | Description |
 | --- | --- | --- |
 | finish\_reason | [ChatCompletionFinishReason](#chatcompletionfinishreason) | The reason the model stopped generating tokens. This will be `stop` if the model hit a natural stop point or a provided stop sequence, `length` if the maximum number of tokens specified in the request was reached, `content_filter` if content was omitted due to a flag from our content filters, `tool_calls` if the model called a tool. |
@@ -282,7 +298,6 @@ The API call fails when the prompt triggers a content filter as configured. Modi
 
 ### CreateChatCompletionRequest
 
-
 | Name | Type | Default Value | Description |
 | --- | --- | --- | --- |
 | frequency\_penalty | number | 0   | Helps prevent word repetitions by reducing the chance of a word being selected if it has already been used. The higher the frequency penalty, the less likely the model is to repeat the same words in its output. Return a 422 error if value or parameter is not supported by model. |
@@ -348,7 +363,6 @@ Specifies the detail level of the image.
 
 Represents a chat completion response returned by model, based on the provided input.
 
-
 | Name | Type | Description |
 | --- | --- | --- |
 | choices | [Choices](#choices)\[\] | A list of chat completion choices. Can be more than one if `n` is greater than 1. |
@@ -361,6 +375,7 @@ Represents a chat completion response returned by model, based on the provided i
 
 ### Detail
 
+Details for the [UnprocessableContentError](#unprocessablecontenterror) error.
 
 | Name | Type | Description |
 | --- | --- | --- |
@@ -371,14 +386,14 @@ Represents a chat completion response returned by model, based on the provided i
 
 The function that the model called.
 
-
 | Name | Type | Description |
 | --- | --- | --- |
 | arguments | string | The arguments to call the function with, as generated by the model in JSON format. Note that the model does not always generate valid JSON, and may generate incorrect parameters not defined by your function schema. Validate the arguments in your code before calling your function. |
 | name | string | The name of the function to call. |
 
 ### FunctionObject
 
+Definition of a function the model has access to.
 
 | Name | Type | Description |
 | --- | --- | --- |
@@ -407,6 +422,7 @@ The type of the tool. Currently, only `function` is supported.
 ### TooManyRequestsError
 
 
+
 | Name | Type | Description |
 | --- | --- | --- |
 | error | string | The error description. |
@@ -424,11 +440,12 @@ The type of the tool. Currently, only `function` is supported.
 
 ### UnprocessableContentError
 
+The request contains unprocessable content. The error is returned when the payload indicated is valid according to this specification. However, some of the instructions indicated in the payload are not supported by the underlying model. Use the `details` section to understand the offending parameter.
 
 | Name | Type | Description |
 | --- | --- | --- |
 | code | string | The error code. |
 | detail | [Detail](#detail) |     |
 | error | string | The error description. |
 | message | string | The error message. |
-| status | integer | The HTTP status code. |
+| status | integer | The HTTP status code. |