Skip to content

Commit 98e808a

Browse files
Learn Build Service GitHub AppLearn Build Service GitHub App
authored andcommitted
Merging changes synced from https://github.com/MicrosoftDocs/azure-docs-pr (branch live)
2 parents 76e641e + 319d719 commit 98e808a

File tree

98 files changed

+1661
-1094
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

98 files changed

+1661
-1094
lines changed

.openpublishing.redirection.json

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4807,6 +4807,11 @@
48074807
"redirect_url": "/azure/reliability/reliability-event-grid",
48084808
"redirect_document_id": false
48094809
},
4810+
{
4811+
"source_path_from_root": "/articles/openshift/howto-encrypt-data-disks.md",
4812+
"redirect_url": "/azure/openshift/howto-byok",
4813+
"redirect_document_id": false
4814+
},
48104815
{
48114816
"source_path_from_root": "/articles/event-grid/availability-zones-disaster-recovery.md",
48124817
"redirect_url": "/azure/reliability/reliability-event-grid",

articles/ai-services/openai/references/on-your-data.md

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ description: Learn how to use Azure OpenAI On Your Data Python & REST API.
55
manager: nitinme
66
ms.service: azure-ai-openai
77
ms.topic: conceptual
8-
ms.date: 06/13/2024
8+
ms.date: 07/18/2024
99
author: mrbullwinkle
1010
ms.author: mbullwin
1111
recommendations: false
@@ -174,6 +174,20 @@ completion = client.chat.completions.create(
174174

175175
print(completion.model_dump_json(indent=2))
176176

177+
# render the citations
178+
179+
content = completion.choices[0].message.content
180+
context = completion.choices[0].message.context
181+
for citation_index, citation in enumerate(context["citations"]):
182+
citation_reference = f"[doc{citation_index + 1}]"
183+
url = "https://example.com/?redirect=" + citation["url"] # replace with actual host and encode the URL
184+
filepath = citation["filepath"]
185+
title = citation["title"]
186+
snippet = citation["content"]
187+
chunk_id = citation["chunk_id"]
188+
replaced_html = f"<a href='{url}' title='{title}\n{snippet}''>(See from file {filepath}, Part {chunk_id})</a>"
189+
content = content.replace(citation_reference, replaced_html)
190+
print(content)
177191
```
178192

179193
# [REST](#tab/rest)

articles/ai-studio/concepts/deployments-overview.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,6 @@ author: msakande
1616

1717
# Overview: Deploy models, flows, and web apps with Azure AI Studio
1818

19-
[!INCLUDE [Feature preview](~/reusable-content/ce-skilling/azure/includes/ai-studio/includes/feature-preview.md)]
2019

2120
Azure AI Studio supports deploying large language models (LLMs), flows, and web apps. Deploying an LLM or flow makes it available for use in a website, an application, or other production environments. This typically involves hosting the model on a server or in the cloud, and creating an API or other interface for users to interact with the model.
2221

articles/ai-studio/how-to/deploy-models-serverless-availability.md

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,6 @@ ms.custom:
1818

1919
# Region availability for models in serverless API endpoints | Azure AI Studio
2020

21-
[!INCLUDE [Feature preview](~/reusable-content/ce-skilling/azure/includes/ai-studio/includes/feature-preview.md)]
22-
2321
In this article, you learn about which regions are available for each of the models supporting serverless API endpoint deployments.
2422

2523
Certain models in the model catalog can be deployed as a serverless API with pay-as-you-go billing. This kind of deployment provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance that organizations need. This deployment option doesn't require quota from your subscription.

articles/ai-studio/how-to/deploy-models-serverless-connect.md

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,6 @@ ms.custom:
1717

1818
# Consume serverless API endpoints from a different Azure AI Studio project or hub
1919

20-
[!INCLUDE [Feature preview](~/reusable-content/ce-skilling/azure/includes/ai-studio/includes/feature-preview.md)]
21-
2220
In this article, you learn how to configure an existing serverless API endpoint in a different project or hub than the one that was used to create the deployment.
2321

2422
[Certain models in the model catalog](deploy-models-serverless-availability.md) can be deployed as serverless APIs. This kind of deployment provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance that organizations need. This deployment option doesn't require quota from your subscription.
@@ -235,4 +233,4 @@ Follow these steps to create a connection:
235233
## Related content
236234
237235
- [What is Azure AI Studio?](../what-is-ai-studio.md)
238-
- [Azure AI FAQ article](../faq.yml)
236+
- [Azure AI FAQ article](../faq.yml)

articles/ai-studio/how-to/deploy-models-serverless.md

Lines changed: 40 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ description: Learn to deploy models as serverless APIs, using Azure AI Studio.
55
manager: scottpolly
66
ms.service: azure-ai-studio
77
ms.topic: how-to
8-
ms.date: 5/21/2024
8+
ms.date: 07/18/2024
99
ms.author: mopeakande
1010
author: msakande
1111
ms.reviewer: fasantia
@@ -15,8 +15,6 @@ ms.custom: build-2024, serverless, devx-track-azurecli
1515

1616
# Deploy models as serverless APIs
1717

18-
[!INCLUDE [Feature preview](~/reusable-content/ce-skilling/azure/includes/ai-studio/includes/feature-preview.md)]
19-
2018
In this article, you learn how to deploy a model from the model catalog as a serverless API with pay-as-you-go token based billing.
2119

2220
[Certain models in the model catalog](deploy-models-serverless-availability.md) can be deployed as a serverless API with pay-as-you-go billing. This kind of deployment provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance that organizations need. This deployment option doesn't require quota from your subscription.
@@ -86,16 +84,13 @@ In this article, you learn how to deploy a model from the model catalog as a ser
8684
You can use any compatible web browser to [deploy ARM templates](../../azure-resource-manager/templates/deploy-portal.md) in the Microsoft Azure portal or use any of the deployment tools. This tutorial uses the [Azure CLI](/cli/azure/).
8785
8886
89-
## Subscribe your project to the model offering
90-
91-
For models offered through the Azure Marketplace, you can deploy them to serverless API endpoints to consume their predictions. If it's your first time deploying the model in the project, you have to subscribe your project for the particular model offering from the Azure Marketplace. Each project has its own subscription to the particular Azure Marketplace offering of the model, which allows you to control and monitor spending.
92-
93-
> [!NOTE]
94-
> Models offered through the Azure Marketplace are available for deployment to serverless API endpoints in specific regions. Check [Model and region availability for Serverless API deployments](deploy-models-serverless-availability.md) to verify which models and regions are available. If the one you need is not listed, you can deploy to a workspace in a supported region and then [consume serverless API endpoints from a different workspace](deploy-models-serverless-connect.md).
87+
## Find your model and model ID in the model catalog
9588
9689
1. Sign in to [Azure AI Studio](https://ai.azure.com).
9790
98-
1. Ensure your account has the **Azure AI Developer** role permissions on the resource group, or that you meet the [permissions required to subscribe to model offerings](#permissions-required-to-subscribe-to-model-offerings).
91+
1. For models offered through the Azure Marketplace, ensure that your account has the **Azure AI Developer** role permissions on the resource group, or that you meet the [permissions required to subscribe to model offerings](#permissions-required-to-subscribe-to-model-offerings).
92+
93+
Models that are offered by non-Microsoft providers (for example, Llama and Mistral models) are billed through the Azure Marketplace. For such models, you're required to subscribe your project to the particular model offering. Models that are offered by Microsoft (for example, Phi-3 models) don't have this requirement, as billing is done differently. For details about billing for serverless deployment of models in the model catalog, see [Billing for serverless APIs](model-catalog-overview.md#billing).
9994
10095
1. Select **Model catalog** from the left sidebar and find the model card of the model you want to deploy. In this article, you select a **Meta-Llama-3-8B-Instruct** model.
10196
@@ -106,13 +101,23 @@ For models offered through the Azure Marketplace, you can deploy them to serverl
106101
107102
:::image type="content" source="../media/deploy-monitor/serverless/model-card.png" alt-text="A screenshot showing a model's details page." lightbox="../media/deploy-monitor/serverless/model-card.png":::
108103
104+
105+
The next section covers the steps for subscribing your project to a model offering. You can skip this section and go to [Deploy the model to a serverless API endpoint](#deploy-the-model-to-a-serverless-api-endpoint), if you're deploying a Microsoft model.
106+
107+
## Subscribe your project to the model offering
108+
109+
For non-Microsoft models offered through the Azure Marketplace, you can deploy them to serverless API endpoints to consume their predictions. If it's your first time deploying the model in the project, you have to subscribe your project for the particular model offering from the Azure Marketplace. Each project has its own subscription to the particular Azure Marketplace offering of the model, which allows you to control and monitor spending.
110+
111+
> [!NOTE]
112+
> Models offered through the Azure Marketplace are available for deployment to serverless API endpoints in specific regions. Check [Model and region availability for Serverless API deployments](deploy-models-serverless-availability.md) to verify which models and regions are available. If the one you need is not listed, you can deploy to a workspace in a supported region and then [consume serverless API endpoints from a different workspace](deploy-models-serverless-connect.md).
113+
109114
1. Create the model's marketplace subscription. When you create a subscription, you accept the terms and conditions associated with the model offer.
110115
111116
# [AI Studio](#tab/azure-ai-studio)
112117
113-
1. On the model's **Details** page, select **Deploy** and then select **Serverless API** to open the deployment wizard.
118+
1. On the model's **Details** page, select **Deploy** and then select **Serverless API with Azure AI Content Safety (preview)** to open the deployment wizard.
114119
115-
1. Select the project in which you want to deploy your models. Notice that not all the regions are supported.
120+
1. Select the project in which you want to deploy your models. To use the serverless API model deployment offering, your project must belong to one of the [regions that are supported for serverless deployment](deploy-models-serverless-availability.md) for the particular model.
116121
117122
:::image type="content" source="../media/deploy-monitor/serverless/deploy-pay-as-you-go.png" alt-text="A screenshot showing how to deploy a model with the serverless API option." lightbox="../media/deploy-monitor/serverless/deploy-pay-as-you-go.png":::
118123
@@ -191,7 +196,7 @@ For models offered through the Azure Marketplace, you can deploy them to serverl
191196
}
192197
```
193198
194-
1. Once you sign up the project for the particular Azure Marketplace offering, subsequent deployments of the same offering in the same project don't require subscribing again.
199+
1. Once you subscribe the project for the particular Azure Marketplace offering, subsequent deployments of the same offering in the same project don't require subscribing again.
195200
196201
1. At any point, you can see the model offers to which your project is currently subscribed:
197202
@@ -233,15 +238,21 @@ For models offered through the Azure Marketplace, you can deploy them to serverl
233238
234239
## Deploy the model to a serverless API endpoint
235240
236-
Once you've created a model's subscription, you can deploy the associated model to a serverless API endpoint. The serverless API endpoint provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance organizations need. This deployment option doesn't require quota from your subscription.
241+
Once you've created a subscription for a non-Microsoft model, you can deploy the associated model to a serverless API endpoint. For Microsoft models (such as Phi-3 models), you don't need to create a subscription.
237242
238-
In this article, you create an endpoint with name **meta-llama3-8b-qwerty**.
243+
The serverless API endpoint provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance organizations need. This deployment option doesn't require quota from your subscription.
244+
245+
In this section, you create an endpoint with the name **meta-llama3-8b-qwerty**.
239246
240247
1. Create the serverless endpoint
241248
242249
# [AI Studio](#tab/azure-ai-studio)
243250
244-
1. From the previous wizard, select **Deploy** (if you've just subscribed the project to the model offer in the previous section), or select **Continue to deploy** (if your deployment wizard had the note *You already have an Azure Marketplace subscription for this project*).
251+
1. To deploy a Microsoft model that doesn't require subscribing to a model offering:
252+
1. Select **Deploy** and then select **Serverless API with Azure AI Content Safety (preview)** to open the deployment wizard.
253+
1. Select the project in which you want to deploy your model. Notice that not all the regions are supported.
254+
255+
1. Alternatively, for a non-Microsoft model that requires a model subscription, if you've just subscribed your project to the model offer in the previous section, continue to select **Deploy**. Alternatively, select **Continue to deploy** (if your deployment wizard had the note *You already have an Azure Marketplace subscription for this project*).
245256
246257
:::image type="content" source="../media/deploy-monitor/serverless/deploy-pay-as-you-go-subscribed-project.png" alt-text="A screenshot showing a project that is already subscribed to the offering." lightbox="../media/deploy-monitor/serverless/deploy-pay-as-you-go-subscribed-project.png":::
247258
@@ -418,11 +429,11 @@ In this article, you create an endpoint with name **meta-llama3-8b-qwerty**.
418429
> [!TIP]
419430
> If you're using prompt flow in the same project or hub where the deployment was deployed, you still need to create the connection.
420431
421-
## Using the serverless API endpoint
432+
## Use the serverless API endpoint
422433
423434
Models deployed in Azure Machine Learning and Azure AI studio in Serverless API endpoints support the [Azure AI Model Inference API](../reference/reference-model-inference-api.md) that exposes a common set of capabilities for foundational models and that can be used by developers to consume predictions from a diverse set of models in a uniform and consistent way.
424435
425-
Read more about the [capabilities of this API](../reference/reference-model-inference-api.md#capabilities) and how [you can leverage it when building applications](../reference/reference-model-inference-api.md#getting-started).
436+
Read more about the [capabilities of this API](../reference/reference-model-inference-api.md#capabilities) and how [you can use it when building applications](../reference/reference-model-inference-api.md#getting-started).
426437
427438
## Delete endpoints and subscriptions
428439
@@ -495,7 +506,15 @@ az resource delete --name <resource-name>
495506

496507
## Cost and quota considerations for models deployed as serverless API endpoints
497508

498-
Models deployed as serverless API endpoints are offered through the Azure Marketplace and integrated with Azure AI Studio for use. You can find the Azure Marketplace pricing when deploying or fine-tuning the models.
509+
Quota is managed per deployment. Each deployment has a rate limit of 200,000 tokens per minute and 1,000 API requests per minute. However, we currently limit one deployment per model per project. Contact Microsoft Azure Support if the current rate limits aren't sufficient for your scenarios.
510+
511+
#### Cost for Microsoft models
512+
513+
You can find the pricing information on the __Pricing and terms__ tab of the deployment wizard when deploying Microsoft models (such as Phi-3 models) as serverless API endpoints.
514+
515+
#### Cost for non-Microsoft models
516+
517+
Non-Microsoft models deployed as serverless API endpoints are offered through the Azure Marketplace and integrated with Azure AI Studio for use. You can find the Azure Marketplace pricing when deploying or fine-tuning these models.
499518

500519
Each time a project subscribes to a given offer from the Azure Marketplace, a new resource is created to track the costs associated with its consumption. The same resource is used to track costs associated with inference and fine-tuning; however, multiple meters are available to track each scenario independently.
501520

@@ -504,8 +523,6 @@ For more information on how to track costs, see [Monitor costs for models offere
504523
:::image type="content" source="../media/deploy-monitor/serverless/costs-model-as-service-cost-details.png" alt-text="A screenshot showing different resources corresponding to different model offers and their associated meters." lightbox="../media/deploy-monitor/serverless/costs-model-as-service-cost-details.png":::
505524

506525

507-
Quota is managed per deployment. Each deployment has a rate limit of 200,000 tokens per minute and 1,000 API requests per minute. However, we currently limit one deployment per model per project. Contact Microsoft Azure Support if the current rate limits aren't sufficient for your scenarios.
508-
509526
## Permissions required to subscribe to model offerings
510527

511528
Azure role-based access controls (Azure RBAC) are used to grant access to operations in Azure AI Studio. To perform the steps in this article, your user account must be assigned the __Owner__, __Contributor__, or __Azure AI Developer__ role for the Azure subscription. Alternatively, your account can be assigned a custom role that has the following permissions:
@@ -527,6 +544,7 @@ Azure role-based access controls (Azure RBAC) are used to grant access to operat
527544

528545
For more information on permissions, see [Role-based access control in Azure AI Studio](../concepts/rbac-ai-studio.md).
529546

530-
## Next step
547+
## Related content
531548

549+
* [Region availability for models in serverless API endpoints](deploy-models-serverless-availability.md)
532550
* [Fine-tune a Meta Llama 2 model in Azure AI Studio](fine-tune-model-llama.md)

articles/aks/manage-ssh-node-access.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -125,6 +125,9 @@ To improve security and support your corporate security requirements or strategy
125125
126126
When you disable SSH at cluster creation time, it takes effect after the cluster is created. However, when you disable SSH on an existing cluster or node pool, AKS doesn't automatically disable SSH. At any time, you can choose to perform a nodepool upgrade operation. The disable/enable SSH keys operation takes effect after the node image update is complete.
127127
128+
> [!NOTE]
129+
> When you disable SSH at the cluster level, it applies to all existing node pools. Any node pools created after this operation will have SSH enabled by default, and you'll need to run these commands again in order to disable it.
130+
128131
|SSH parameter |Description |
129132
|-----|-----|
130133
|`disabled` |The SSH service is disabled. |
-248 Bytes
Loading
Loading
-1.51 KB
Loading

0 commit comments

Comments
 (0)