Skip to content

Commit 6b71f18

Browse files
denreadenrea
authored
Merge pull request #1036 from Blackmist/319502-ai-model-policy
writing for AI Studio policies
2 parents 01b4f3c + c7280b6 commit 6b71f18

File tree

3 files changed

+222
-0
lines changed

3 files changed

+222
-0
lines changed

‎articles/ai-studio/how-to/built-in-policy-model-deployment.md

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
---
2+
title: Control AI model deployment with built-in policies
3+
titleSuffix: Azure AI Studio
4+
description: "Learn how to use built-in Azure policies to control what managed AI Services (MaaS) and Model-as-a-Platform (MaaP) AI models can be deployed in Azure AI Studio."
5+
author: Blackmist
6+
ms.author: larryfr
7+
ms.service: azure-ai-studio
8+
ms.topic: how-to #Don't change
9+
ms.date: 10/25/2024
10+
11+
#customer intent: As an admin, I want control what Managed AI Services (MaaS) and Model-as-a-Platform (MaaP) AI models can be deployed by my developers.
12+
13+
---
14+
15+
# Control AI model deployment with built-in policies in Azure AI Studio
16+
17+
Azure Policy provides built-in policy definitions that help you govern the deployment of AI models in Managed AI Services (MaaS) and Model-as-a-Platform (MaaP). You can use these policies to control what models your developers can deploy in Azure AI Studio.
18+
19+
## Prerequisites
20+
21+
- An Azure subscription. If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/) before you begin.
22+
- Permissions to create and assign policies. To create and assign policies, you must be an [Owner](/azure/role-based-access-control/built-in-roles#owner) or [Resource Policy Contributor](/azure/role-based-access-control/built-in-roles#resource-policy-contributor) at the Azure subscription or resource group level.
23+
- Familiarity with Azure Policy. To learn more, see [What is Azure Policy?](/azure/governance/policy/overview).
24+
25+
## Enable the policy
26+
27+
1. From the [Azure portal](https://portal.azure.com), select **Policy** from the left side of the page. You can also search for **Policy** in the search bar at the top of the page.
28+
1. From the left side of the Azure Policy Dashboard, select **Authoring**, **Definition**, and then search for "[Preview]: Azure Machine Learning Deployments should only use approved Registry Models" in the search bar within the page. You can also directly navigate to [policy definition creation page](https://portal.azure.com/#blade/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2Fproviders%2FMicrosoft.Authorization%2FpolicyDefinitions%2F12e5dd16-d201-47ff-849b-8454061c293d).
29+
1. Select on **Assign** to assign the policy to the management group:
30+
31+
- **Scope**: Select the scope where you want to assign the policy. The scope can be a management group, subscription, or resource group.
32+
- **Policy definition**: this section should already have a value of "**[Preview]: Azure Machine Learning Deployments should only use approved Registry Models**".
33+
- **Assignment name**: Enter a unique name for the assignment.
34+
35+
The rest of the fields can be left as their default values or you can customize as needed for your organization.
36+
37+
1. Select **Next** at the bottom of the page or the **Parameters** tab at the top of the page.
38+
1. In the **Parameters** tab, deselect **Only show parameters that needs input or review** to see all fields:
39+
40+
- **Effect**: Set to [**Deny**](/azure/governance/policy/concepts/effect-deny).
41+
> [!NOTE]
42+
> Using the [audit](/azure/governance/policy/concepts/effect-audit) option allows you to configure the policy to log information to your own compliance dashboard.
43+
- **Allowed Models Publishers**: This field expects a list of **publisher's name** in quotation and separated by commas.
44+
- **Allowed Asset Ids**: This field expects a list of **model asset ids** in quotation and separated by commas.
45+
46+
To get the model asset ID strings and model publishers' name use the following steps:
47+
48+
1. Go to the [Azure AI Studio model catalog](model-catalog-overview.md).
49+
50+
51+
1. For each model you want to allow, select the model to view the details. In the model detail information, copy the **Model ID** value. For example, the value might look like `azureml://registries/azure-openai/models/gpt-35-turbo/versions/3` for GPT-3.5-Turbo model. The provided names are also *Collections* in model catalog. For example, the publisher for "Meta-Llama-3.1-70B-Instruct" model is Meta.
52+
53+
> [!IMPORTANT]
54+
> The model ID value must be an exact match for the model. If the model ID is not an exact match, the model won't be allowed.
55+
56+
57+
1. Select **Review + create** tab and verify that the policy assignment is correct. When ready, select **Create** to assign the policy.
58+
1. Notify your developers that the policy is in place. They receive an error message if they try to deploy a model that isn't in the list of allowed models.
59+
60+
## Monitor compliance
61+
62+
To monitor compliance with the policy, follow these steps:
63+
64+
1. From the [Azure portal](https://portal.azure.com), select **Policy** from the left side of the page. You can also search for **Policy** in the search bar at the top of the page.
65+
1. From the left side of the Azure Policy Dashboard, select **Compliance**. Each policy assignment is listed with the compliance status. To view more details, select the policy assignment.
66+
67+
## Update the policy assignment
68+
69+
To update an existing policy assignment with new models, follow these steps:
70+
71+
1. From the [Azure portal](https://portal.azure.com), select **Policy** from the left side of the page. You can also search for **Policy** in the search bar at the top of the page.
72+
1. From the left side of the Azure Policy Dashboard, select **Assignments** and find the existing policy assignment. Select the ellipsis (...) next to the assignment and select **Edit assignment**.
73+
1. From the **Parameters** tab, update the **Allowed models** parameter with the new model IDs.
74+
1. From the **Review + Save** tab, select **Save** to update the policy assignment.
75+
76+
## Best practices
77+
78+
- **Granular scoping**: Assign policies at the appropriate scope to balance control and flexibility. For example, apply at the subscription level to control all resources in the subscription, or apply at the resource group level to control resources in a specific group.
79+
- **Policy naming**: Use a consistent naming convention for policy assignments to make it easier to identify the purpose of the policy. Include information such as the purpose and scope in the name.
80+
- **Documentation**: Keep records of policy assignments and configurations for auditing purposes. Document any changes made to the policy over time.
81+
- **Regular reviews**: Periodically review policy assignments to ensure they align with your organization's requirements.
82+
- **Testing**: Test policies in a nonproduction environment before applying them to production resources.
83+
- **Communication**: Make sure developers are aware of the policies in place and understand the implications for their work.
84+
85+
## Related content
86+
87+
- [Azure Policy overview](/azure/governance/policy/overview)
88+
- [Azure AI Studio model catalog](model-catalog-overview.md)

‎articles/ai-studio/how-to/custom-policy-model-deployment.md

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
---
2+
title: Control AI model deployment with custom policies
3+
titleSuffix: Azure AI Studio
4+
description: "Learn how to use custom Azure Policies to control Azure AI services and Azure OpenAI model deployment with Azure AI Studio."
5+
author: Blackmist
6+
ms.author: larryfr
7+
ms.service: azure-ai-studio
8+
ms.topic: how-to #Don't change
9+
ms.date: 10/25/2024
10+
11+
#customer intent: As an admin, I want control what Azure AI services and Azure OpenAI models can be deployed by my developers.
12+
13+
---
14+
15+
# Control AI model deployment with custom policies in Azure AI Studio
16+
17+
When using models from Azure AI services and Azure OpenAI with Azure AI Studio, you might need to use custom policies to control what models your developers can deploy. Custom Azure Policies allow you to create policy definitions that meet your organization's unique requirements. This article shows you how to create and assign an example custom policy to control model deployment.
18+
19+
## Prerequisites
20+
21+
- An Azure subscription. If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/) before you begin.
22+
- Permissions to create and assign policies. To create and assign policies, you must be an [Owner](/azure/role-based-access-control/built-in-roles#owner) or [Resource Policy Contributor](/azure/role-based-access-control/built-in-roles#resource-policy-contributor) at the Azure subscription or resource group level.
23+
- Familiarity with Azure Policy. To learn more, see [What is Azure Policy?](/azure/governance/policy/overview).
24+
25+
## Create a custom policy
26+
27+
1. From the [Azure portal](https://portal.azure.com), select **Policy** from the left side of the page. You can also search for **Policy** in the search bar at the top of the page.
28+
1. From the left side of the Azure Policy Dashboard, select **Authoring**, **Definitions**, and then select **+ Policy definition** from the top of the page.
29+
1. In the **Policy Definition** form, use the following values:
30+
31+
- **Definition location**: Select the subscription or management group where you want to store the policy definition.
32+
- **Name**: Enter a unique name for the policy definition. For example, `Custom allowed Azure AI services and Azure OpenAI models`.
33+
- **Description**: Enter a description for the policy definition.
34+
- **Category**: You can either create a new category or use an existing one. For example, "AI model governance."
35+
- **Policy rule**: Enter the policy rule in JSON format. The following example shows a policy rule that allows the deployment of specific Azure AI services and Azure OpenAI models:
36+
37+
> [!TIP]
38+
> Azure AI services was originally named Azure Cognitive Services. This name is still used internally by Azure, such as this custom policy where you see a value of `Microsoft.CognitiveServices`. Azure OpenAI is part of Azure AI services, so this policy also applies to Azure OpenAI models.
39+
40+
```json
41+
{
42+
"mode": "All",
43+
"policyRule": {
44+
"if": {
45+
"allOf": [
46+
{
47+
"field": "type",
48+
"equals": "Microsoft.CognitiveServices/accounts/deployments"
49+
},
50+
{
51+
"not": {
52+
"value": "[concat(field('Microsoft.CognitiveServices/accounts/deployments/model.name'), ',', field('Microsoft.CognitiveServices/accounts/deployments/model.version'))]",
53+
"in": "[parameters('allowedModels')]"
54+
}
55+
}
56+
]
57+
},
58+
"then": {
59+
"effect": "deny"
60+
}
61+
},
62+
"parameters": {
63+
"allowedModels": {
64+
"type": "Array",
65+
"metadata": {
66+
"displayName": "Allowed AI models",
67+
"description": "The list of allowed models to be deployed."
68+
}
69+
}
70+
}
71+
}
72+
```
73+
74+
1. Select **Save** to save the policy definition. After saving, you arrive at the policy definition's overview page.
75+
1. From the policy definition's overview page, select **Assign policy** to assign the policy definition.
76+
1. From the **Assign policy** page, use the following values on the **Basics** tab:
77+
78+
- **Scope**: Select the scope where you want to assign the policy. The scope can be a management group, subscription, or resource group.
79+
- **Policy definition**: This field is prepopulated with the title of policy definition you created previously.
80+
- **Assignment name**: Enter a unique name for the assignment.
81+
- **Policy enforcement**: Make sure that the **Policy enforcement** field is set to **Enabled**. If it isn't enabled, the policy isn't enforced.
82+
83+
Select **Next** at the bottom of the page, or the **Parameters** tab at the top of the page.
84+
1. From the **Parameters** tab, set **Allowed AI models** to the list of models that you want to allow. The list should be a comma-separated list of model names and approved versions, surrounded by square brackets. For example, `["gpt-4,0613", "gpt-35-turbo,0613"]`.
85+
86+
> [!TIP]
87+
> You can find the model names and their versions in the [Azure AI Studio Model Catalog](https://ai.azure.com/explore/models). Select the model to view the details, and then copy the model name and their version in the title.
88+
89+
1. Optionally, select the **Non-compliance messages** tab at the top of the page and set a custom message for noncompliance.
90+
1. Select **Review + create** tab and verify that the policy assignment is correct. When ready, select **Create** to assign the policy.
91+
1. Notify your developers that the policy is in place. They receive an error message if they try to deploy a model that isn't in the list of allowed models.
92+
93+
## Verify policy assignment
94+
95+
To verify that the policy is assigned, navigate to **Policy** in the Azure portal, and then select **Assignments** under **Authoring**. You should see the policy listed.
96+
97+
## Monitor compliance
98+
99+
To monitor compliance with the policy, follow these steps:
100+
101+
1. From the [Azure portal](https://portal.azure.com), select **Policy** from the left side of the page. You can also search for **Policy** in the search bar at the top of the page.
102+
1. From the left side of the Azure Policy Dashboard, select **Compliance**. Each policy assignment is listed with the compliance status. To view more details, select the policy assignment.
103+
104+
## Update the policy assignment
105+
106+
To update an existing policy assignment with new models, follow these steps:
107+
108+
1. From the [Azure portal](https://portal.azure.com), select **Policy** from the left side of the page. You can also search for **Policy** in the search bar at the top of the page.
109+
1. From the left side of the Azure Policy Dashboard, select **Assignments** and find the existing policy assignment. Select the ellipsis (...) next to the assignment and select **Edit assignment**.
110+
1. From the **Parameters** tab, update the **Allowed models** parameter with the new models.
111+
1. From the **Review + Save** tab, select **Save** to update the policy assignment.
112+
113+
## Best practices
114+
115+
- **Obtaining model names**: Use the [Azure AI Studio Model Catalog](https://ai.azure.com/explore/models), then select the model to view details. Use the model name in the title with the policy.
116+
- **Granular scoping**: Assign policies at the appropriate scope to balance control and flexibility. For example, apply at the subscription level to control all resources in the subscription, or apply at the resource group level to control resources in a specific group.
117+
- **Policy naming**: Use a consistent naming convention for policy assignments to make it easier to identify the purpose of the policy. Include information such as the purpose and scope in the name.
118+
- **Documentation**: Keep records of policy assignments and configurations for auditing purposes. Document any changes made to the policy over time.
119+
- **Regular reviews**: Periodically review policy assignments to ensure they align with your organization's requirements.
120+
- **Testing**: Test policies in a nonproduction environment before applying them to production resources.
121+
- **Communication**: Make sure developers are aware of the policies in place and understand the implications for their work.
122+
123+
## Related content
124+
125+
- [Azure Policy overview](/azure/governance/policy/overview)
126+
- [Azure AI Studio model catalog](model-catalog-overview.md)
127+
- [Azure AI services documentation](/azure/ai-services)
128+

‎articles/ai-studio/toc.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -341,6 +341,12 @@ items:
341341
href: concepts/encryption-keys-portal.md
342342
- name: Rotate keys
343343
href: ../ai-services/rotate-keys.md?context=/azure/ai-studio/context/context
344+
- name: Azure policies
345+
items:
346+
- name: Built-in policy to allow specific models
347+
href: how-to/built-in-policy-model-deployment.md
348+
- name: Custom policy to allow specific models
349+
href: how-to/custom-policy-model-deployment.md
344350
- name: Vulnerability management
345351
href: concepts/vulnerability-management.md
346352
- name: Disaster recovery

0 commit comments

Comments
 (0)