add Phi 3.5 MoE and vision (MaaP only)

Author: msakande

articles/ai-studio/how-to/deploy-models-phi-3-5-vision.md

@@ -39,15 +39,6 @@ To use Phi-3.5 chat model with vision with Azure AI Studio, you need the followi
 
 ### A model deployment
 
-**Deployment to serverless APIs**
-
-Phi-3.5 chat model with vision can be deployed to serverless API endpoints with pay-as-you-go billing. This kind of deployment provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance that organizations need. 
-
-Deployment to a serverless API endpoint doesn't require quota from your subscription. If your model isn't deployed already, use the Azure AI Studio, Azure Machine Learning SDK for Python, the Azure CLI, or ARM templates to [deploy the model as a serverless API](deploy-models-serverless.md).
-
-> [!div class="nextstepaction"]
-> [Deploy the model to serverless API endpoints](deploy-models-serverless.md)
-
 **Deployment to a self-hosted managed compute**
 
 Phi-3.5 chat model with vision can be deployed to our self-hosted managed inference solution, which allows you to customize and control all the details about how the model is served.
@@ -110,9 +101,6 @@ client = ChatCompletionsClient(
 )
 ```
 
-> [!NOTE]
-> Currently, serverless API endpoints do not support using Microsoft Entra ID for authentication.
-
 ### Get the model's capabilities
 
 The `/info` route returns information about the model that is deployed to the endpoint. Return the model's information by calling the following method:
@@ -276,42 +264,6 @@ The following extra parameters can be passed to Phi-3.5 chat model with vision:
 | `n` | How many chat completion choices to generate for each input message. Note that you will be charged based on the number of generated tokens across all of the choices. | `int` |
 
 
-### Apply content safety
-
-The Azure AI model inference API supports [Azure AI content safety](https://aka.ms/azureaicontentsafety). When you use deployments with Azure AI content safety turned on, inputs and outputs pass through an ensemble of classification models aimed at detecting and preventing the output of harmful content. The content filtering system detects and takes action on specific categories of potentially harmful content in both input prompts and output completions.
-
-The following example shows how to handle events when the model detects harmful content in the input prompt and content safety is enabled.
-
-
-```python
-from azure.ai.inference.models import AssistantMessage, UserMessage, SystemMessage
-
-try:
-    response = client.complete(
-        messages=[
-            SystemMessage(content="You are an AI assistant that helps people find information."),
-            UserMessage(content="Chopping tomatoes and cutting them into cubes or wedges are great ways to practice your knife skills."),
-        ]
-    )
-
-    print(response.choices[0].message.content)
-
-except HttpResponseError as ex:
-    if ex.status_code == 400:
-        response = ex.response.json()
-        if isinstance(response, dict) and "error" in response:
-            print(f"Your request triggered an {response['error']['code']} error:\n\t {response['error']['message']}")
-        else:
-            raise
-    raise
-```
-
-> [!TIP]
-> To learn more about how you can configure and control Azure AI content safety settings, check the [Azure AI content safety documentation](https://aka.ms/azureaicontentsafety).
-
-> [!NOTE]
-> Azure AI content safety is only available for models deployed as serverless API endpoints.
-
 ## Use chat completions with images
 
 Phi-3.5-vision-Instruct can reason across text and images and generate text completions based on both kinds of input. In this section, you explore the capabilities of Phi-3.5-vision-Instruct for vision in a chat fashion:
@@ -407,15 +359,6 @@ To use Phi-3.5 chat model with vision with Azure AI Studio, you need the followi
 
 ### A model deployment
 
-**Deployment to serverless APIs**
-
-Phi-3.5 chat model with vision can be deployed to serverless API endpoints with pay-as-you-go billing. This kind of deployment provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance that organizations need. 
-
-Deployment to a serverless API endpoint doesn't require quota from your subscription. If your model isn't deployed already, use the Azure AI Studio, Azure Machine Learning SDK for Python, the Azure CLI, or ARM templates to [deploy the model as a serverless API](deploy-models-serverless.md).
-
-> [!div class="nextstepaction"]
-> [Deploy the model to serverless API endpoints](deploy-models-serverless.md)
-
 **Deployment to a self-hosted managed compute**
 
 Phi-3.5 chat model with vision can be deployed to our self-hosted managed inference solution, which allows you to customize and control all the details about how the model is served.
@@ -476,9 +419,6 @@ const client = new ModelClient(
 );
 ```
 
-> [!NOTE]
-> Currently, serverless API endpoints do not support using Microsoft Entra ID for authentication.
-
 ### Get the model's capabilities
 
 The `/info` route returns information about the model that is deployed to the endpoint. Return the model's information by calling the following method:
@@ -661,48 +601,6 @@ The following extra parameters can be passed to Phi-3.5 chat model with vision:
 | `n` | How many chat completion choices to generate for each input message. Note that you will be charged based on the number of generated tokens across all of the choices. | `int` |
 
 
-### Apply content safety
-
-The Azure AI model inference API supports [Azure AI content safety](https://aka.ms/azureaicontentsafety). When you use deployments with Azure AI content safety turned on, inputs and outputs pass through an ensemble of classification models aimed at detecting and preventing the output of harmful content. The content filtering system detects and takes action on specific categories of potentially harmful content in both input prompts and output completions.
-
-The following example shows how to handle events when the model detects harmful content in the input prompt and content safety is enabled.
-
-
-```javascript
-try {
-    var messages = [
-        { role: "system", content: "You are an AI assistant that helps people find information." },
-        { role: "user", content: "Chopping tomatoes and cutting them into cubes or wedges are great ways to practice your knife skills." },
-    ];
-
-    var response = await client.path("/chat/completions").post({
-        body: {
-            messages: messages,
-        }
-    });
-
-    console.log(response.body.choices[0].message.content);
-}
-catch (error) {
-    if (error.status_code == 400) {
-        var response = JSON.parse(error.response._content);
-        if (response.error) {
-            console.log(`Your request triggered an ${response.error.code} error:\n\t ${response.error.message}`);
-        }
-        else
-        {
-            throw error;
-        }
-    }
-}
-```
-
-> [!TIP]
-> To learn more about how you can configure and control Azure AI content safety settings, check the [Azure AI content safety documentation](https://aka.ms/azureaicontentsafety).
-
-> [!NOTE]
-> Azure AI content safety is only available for models deployed as serverless API endpoints.
-
 ## Use chat completions with images
 
 Phi-3.5-vision-Instruct can reason across text and images and generate text completions based on both kinds of input. In this section, you explore the capabilities of Phi-3.5-vision-Instruct for vision in a chat fashion:
@@ -804,15 +702,6 @@ To use Phi-3.5 chat model with vision with Azure AI Studio, you need the followi
 
 ### A model deployment
 
-**Deployment to serverless APIs**
-
-Phi-3.5 chat model with vision can be deployed to serverless API endpoints with pay-as-you-go billing. This kind of deployment provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance that organizations need. 
-
-Deployment to a serverless API endpoint doesn't require quota from your subscription. If your model isn't deployed already, use the Azure AI Studio, Azure Machine Learning SDK for Python, the Azure CLI, or ARM templates to [deploy the model as a serverless API](deploy-models-serverless.md).
-
-> [!div class="nextstepaction"]
-> [Deploy the model to serverless API endpoints](deploy-models-serverless.md)
-
 **Deployment to a self-hosted managed compute**
 
 Phi-3.5 chat model with vision can be deployed to our self-hosted managed inference solution, which allows you to customize and control all the details about how the model is served.
@@ -888,9 +777,6 @@ client = new ChatCompletionsClient(
 );
 ```
 
-> [!NOTE]
-> Currently, serverless API endpoints do not support using Microsoft Entra ID for authentication.
-
 ### Get the model's capabilities
 
 The `/info` route returns information about the model that is deployed to the endpoint. Return the model's information by calling the following method:
@@ -1070,48 +956,6 @@ The following extra parameters can be passed to Phi-3.5 chat model with vision:
 | `n` | How many chat completion choices to generate for each input message. Note that you will be charged based on the number of generated tokens across all of the choices. | `int` |
 
 
-### Apply content safety
-
-The Azure AI model inference API supports [Azure AI content safety](https://aka.ms/azureaicontentsafety). When you use deployments with Azure AI content safety turned on, inputs and outputs pass through an ensemble of classification models aimed at detecting and preventing the output of harmful content. The content filtering system detects and takes action on specific categories of potentially harmful content in both input prompts and output completions.
-
-The following example shows how to handle events when the model detects harmful content in the input prompt and content safety is enabled.
-
-
-```csharp
-try
-{
-    requestOptions = new ChatCompletionsOptions()
-    {
-        Messages = {
-            new ChatRequestSystemMessage("You are an AI assistant that helps people find information."),
-            new ChatRequestUserMessage(
-                "Chopping tomatoes and cutting them into cubes or wedges are great ways to practice your knife skills."
-            ),
-        },
-    };
-
-    response = client.Complete(requestOptions);
-    Console.WriteLine(response.Value.Choices[0].Message.Content);
-}
-catch (RequestFailedException ex)
-{
-    if (ex.ErrorCode == "content_filter")
-    {
-        Console.WriteLine($"Your query has trigger Azure Content Safety: {ex.Message}");
-    }
-    else
-    {
-        throw;
-    }
-}
-```
-
-> [!TIP]
-> To learn more about how you can configure and control Azure AI content safety settings, check the [Azure AI content safety documentation](https://aka.ms/azureaicontentsafety).
-
-> [!NOTE]
-> Azure AI content safety is only available for models deployed as serverless API endpoints.
-
 ## Use chat completions with images
 
 Phi-3.5-vision-Instruct can reason across text and images and generate text completions based on both kinds of input. In this section, you explore the capabilities of Phi-3.5-vision-Instruct for vision in a chat fashion:
@@ -1198,15 +1042,6 @@ To use Phi-3.5 chat model with vision with Azure AI Studio, you need the followi
 
 ### A model deployment
 
-**Deployment to serverless APIs**
-
-Phi-3.5 chat model with vision can be deployed to serverless API endpoints with pay-as-you-go billing. This kind of deployment provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance that organizations need. 
-
-Deployment to a serverless API endpoint doesn't require quota from your subscription. If your model isn't deployed already, use the Azure AI Studio, Azure Machine Learning SDK for Python, the Azure CLI, or ARM templates to [deploy the model as a serverless API](deploy-models-serverless.md).
-
-> [!div class="nextstepaction"]
-> [Deploy the model to serverless API endpoints](deploy-models-serverless.md)
-
 **Deployment to a self-hosted managed compute**
 
 Phi-3.5 chat model with vision can be deployed to our self-hosted managed inference solution, which allows you to customize and control all the details about how the model is served.
@@ -1236,9 +1071,6 @@ First, create the client to consume the model. The following code uses an endpoi
 
 When you deploy the model to a self-hosted online endpoint with **Microsoft Entra ID** support, you can use the following code snippet to create a client.
 
-> [!NOTE]
-> Currently, serverless API endpoints do not support using Microsoft Entra ID for authentication.
-
 ### Get the model's capabilities
 
 The `/info` route returns information about the model that is deployed to the endpoint. Return the model's information by calling the following method:
@@ -1489,47 +1321,6 @@ The following extra parameters can be passed to Phi-3.5 chat model with vision:
 | `n` | How many chat completion choices to generate for each input message. Note that you will be charged based on the number of generated tokens across all of the choices. | `int` |
 
 
-### Apply content safety
-
-The Azure AI model inference API supports [Azure AI content safety](https://aka.ms/azureaicontentsafety). When you use deployments with Azure AI content safety turned on, inputs and outputs pass through an ensemble of classification models aimed at detecting and preventing the output of harmful content. The content filtering system detects and takes action on specific categories of potentially harmful content in both input prompts and output completions.
-
-The following example shows how to handle events when the model detects harmful content in the input prompt and content safety is enabled.
-
-
-```json
-{
-    "messages": [
-        {
-            "role": "system",
-            "content": "You are an AI assistant that helps people find information."
-        },
-                {
-            "role": "user",
-            "content": "Chopping tomatoes and cutting them into cubes or wedges are great ways to practice your knife skills."
-        }
-    ]
-}
-```
-
-
-```json
-{
-    "error": {
-        "message": "The response was filtered due to the prompt triggering Microsoft's content management policy. Please modify your prompt and retry.",
-        "type": null,
-        "param": "prompt",
-        "code": "content_filter",
-        "status": 400
-    }
-}
-```
-
-> [!TIP]
-> To learn more about how you can configure and control Azure AI content safety settings, check the [Azure AI content safety documentation](https://aka.ms/azureaicontentsafety).
-
-> [!NOTE]
-> Azure AI content safety is only available for models deployed as serverless API endpoints.
-
 ## Use chat completions with images
 
 Phi-3.5-vision-Instruct can reason across text and images and generate text completions based on both kinds of input. In this section, you explore the capabilities of Phi-3.5-vision-Instruct for vision in a chat fashion:
@@ -1620,10 +1411,6 @@ For more examples of how to use Phi-3 family models, see the following examples
 | LiteLLM                                   | Python            | [Link](https://aka.ms/phi-3/litellm-sample)             | 
 
 
-## Cost and quota considerations for Phi-3 family models deployed as serverless API endpoints
-
-Quota is managed per deployment. Each deployment has a rate limit of 200,000 tokens per minute and 1,000 API requests per minute. However, we currently limit one deployment per model per project. Contact Microsoft Azure Support if the current rate limits aren't sufficient for your scenarios.
-
 ## Cost and quota considerations for Phi-3 family models deployed to managed compute
 
 Phi-3 family models deployed to managed compute are billed based on core hours of the associated compute instance. The cost of the compute instance is determined by the size of the instance, the number of instances running, and the run duration.