You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/sentinel/ci-cd-custom-content.md
+39-25Lines changed: 39 additions & 25 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,9 +6,10 @@ author: austinmccollum
6
6
ms.author: austinmc
7
7
ms.service: microsoft-sentinel
8
8
ms.topic: conceptual
9
-
ms.date: 8/24/2022
9
+
ms.date: 12/31/2024
10
10
ms.custom: template-concept
11
-
11
+
appliesto: Microsoft Sentinel in the Azure portal and the Microsoft Defender portal
12
+
ms.collection: usx-security
12
13
13
14
#Customer intent: As a SOC collaborator or MSSP analyst, I want to manage dynamic Microsoft Sentinel content as code based on source control repositories using CI/CD pipelines so that I can automate updates and ensure consistent configurations across workspaces. As an MSSP content manager, I want to deploy one solution to many customer workspaces and still be able to tailor custom content for their environments.
14
15
@@ -19,17 +20,39 @@ ms.custom: template-concept
19
20
The Microsoft Sentinel repositories feature provides a central experience for the deployment and management of Sentinel content as code. Repositories allow connections to an external source control for continuous integration / continuous delivery (CI/CD). This automation removes the burden of manual processes to update and deploy your custom content across workspaces. For more information on Sentinel content, see [About Microsoft Sentinel content and solutions](sentinel-solutions.md).
20
21
21
22
> [!IMPORTANT]
22
-
> The Microsoft Sentinel **Repositories** feature is currently in **PREVIEW**. See the [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) for additional legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.
23
+
> The Microsoft Sentinel **Repositories** feature is currently in **PREVIEW**. See the [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) for more legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.
23
24
>
24
25
25
26
## Plan your repository connection
26
27
27
-
Microsoft Sentinel repositories require careful planning to ensure you have the proper permissions from your workspace to the repository (repo) you want connected. Only connections to GitHub and Azure DevOps repositories with contributor access are currently supported. The Microsoft Sentinel application will need authorization to your repo and have Actions enabled for GitHub and Pipelines enabled for Azure DevOps.
28
+
Microsoft Sentinel repositories require careful planning to ensure you have the proper permissions from your workspace to the repository (repo) you want connected.
29
+
30
+
- Only connections to GitHub and Azure DevOps repositories are supported.
31
+
- Collaborator access to your GitHub repository or Project Administrator access to your Azure DevOps repository is required.
32
+
- The Microsoft Sentinel application needs authorization to your repo.
33
+
- Actions must be enabled for GitHub.
34
+
- Pipelines must be enabled for Azure DevOps.
35
+
- An Azure DevOps connection must be in the same tenant as your Microsoft Sentinel workspace.
36
+
37
+
Creating a connection to a repository requires an **Owner** role in the resource group that contains your Microsoft Sentinel workspace. If you're unable to use the Owner role in your environment, use the combination of **User Access Administrator** and **Sentinel Contributor** roles to create the connection.
38
+
39
+
If you find content in a public repository where you aren't a contributor, first import, fork, or clone the content to a repo where you're a contributor. Then connect your repo to your Microsoft Sentinel workspace. For more information, see [Deploy custom content from your repository](ci-cd.md).
28
40
29
-
Repositories require an **Owner** role in the resource group that contains your Microsoft Sentinel workspace. This role is required to create the connection between Microsoft Sentinel and your source control repository. If you're' unable to use the Owner role in your environment, you can instead use the combination of **User Access Administrator** and **Sentinel Contributor** roles to create the connection.
41
+
## Plan your repository content
30
42
31
-
If you find content in a public repository where you *aren't* a contributor, you'll need to get that content into your repo first. You can do that with an import, fork, or clone of the content to a repo where you're a contributor. Then you can connect your repo to your Sentinel workspace. For more information, see [Deploy custom content from your repository](ci-cd.md).
43
+
Repository content must be stored as [Bicep files](../azure-resource-manager/bicep/file.md) or [Azure Resource Manager (ARM) templates](../azure-resource-manager/templates/overview.md). However, Bicep is more intuitive and makes it easier to describe Azure resources and Microsoft Sentinel content.
44
+
45
+
Deploy Bicep file templates alongside or instead of ARM JSON templates. If you're considering infrastructure as code options, we recommend looking at Bicep. For more information, see [What is Bicep?](../azure-resource-manager/bicep/overview.md).
46
+
47
+
> [!IMPORTANT]
48
+
> In order to use Bicep templates, your repositories connection needs to be updated if your connection was created before November 1, 2024. Repositories connections must be [removed](ci-cd.md#remove-a-repository-connection) and recreated in order to update.
32
49
50
+
Even if your original content is an ARM template, consider converting to Bicep to make the review and update processes less complex. Bicep is closely related to ARM because during a deployment, each Bicep file is converted to an ARM template. For more information on converting ARM templates, see [Decompiling ARM template JSON to Bicep](../azure-resource-manager/bicep/decompile.md).
51
+
52
+
> [!NOTE]
53
+
> Known Bicep limitations:
54
+
> - Bicep templates don't support the `id` property. When decompiling ARM JSON to Bicep, make sure you don't have this property. For example, analytic rule templates exported from Microsoft Sentinel have the `id` property that needs removal.
55
+
> - Change the ARM JSON schema to version `2019-04-01` for best results when decompiling.
33
56
34
57
### Validate your content
35
58
@@ -45,11 +68,9 @@ The following Microsoft Sentinel content types can be deployed through a reposit
45
68
> This article does *not* describe how to create these types of content from scratch. For more information, see the relevant [Microsoft Sentinel GitHub wiki](https://github.com/Azure/Azure-Sentinel/wiki#get-started) for each content type.
46
69
>
47
70
48
-
Repositories content needs to be stored as [ARM templates](../azure-resource-manager/templates/overview.md). The repositories deployment doesn't validate the content except to confirm it's in the correct JSON format.
49
-
50
-
The first step to validate your content is to test it within Microsoft Sentinel. You can also apply the [Microsoft Sentinel GitHub validation process](https://github.com/Azure/Azure-Sentinel/wiki#test-your-contribution) and tools to complement your validation process.
71
+
The repositories deployment doesn't validate the content except to confirm it's in the correct JSON or Bicep format. Be sure to test your content within Microsoft Sentinel before deploying.
51
72
52
-
A sample repository is available with ARM templates for each of the content types listed above. The repo also demonstrates how to use advanced features of repository connections. For more information, see [Sentinel CICD repositories sample](https://github.com/SentinelCICD/RepositoriesSampleContent).
73
+
A sample repository is available with templates for each of the content types listed. The repo also demonstrates how to use advanced features of repository connections. For more information, see [Microsoft Sentinel CI/CD repositories sample](https://github.com/SentinelCICD/RepositoriesSampleContent).
53
74
54
75
55
76
:::image type="content" source="media/ci-cd-custom-content/repositories-connection-success.png" alt-text="Screenshot of a successful repository connection. The RepositoriesSampleContent is shown. This screenshot is after the sample was imported from the SentinelCICD repo to a private GitHub repo in the FourthCoffee organization." lightbox="media/ci-cd-custom-content/repositories-connection-success.png":::
@@ -58,9 +79,7 @@ A sample repository is available with ARM templates for each of the content type
58
79
### Maximum connections and deployments
59
80
60
81
- Each Microsoft Sentinel workspace is currently limited to **five repository connections**.
61
-
62
-
- Each Azure resource group is limited to **800 deployments** in its deployment history. If you have a high volume of ARM template deployments in your resource group(s), you may see the `Deployment QuotaExceeded` error. For more information, see [DeploymentQuotaExceeded](/azure/azure-resource-manager/templates/deployment-quota-exceeded) in the Azure Resource Manager templates documentation.
63
-
82
+
- Each Azure resource group is limited to **800 deployments** in its deployment history. If you have a high volume of template deployments one or more of your resource groups, you may see the `Deployment QuotaExceeded` error. For more information, see [DeploymentQuotaExceeded](/azure/azure-resource-manager/templates/deployment-quota-exceeded) in the Azure Resource Manager templates documentation.
64
83
65
84
66
85
## Improve performance with smart deployments
@@ -69,29 +88,24 @@ A sample repository is available with ARM templates for each of the content type
69
88
> To ensure smart deployments works in GitHub, Workflows must have read and write permissions on your repository. See [Managing GitHub Actions settings for a repository](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository) for more details.
70
89
>
71
90
72
-
The **smart deployments** feature is a back-end capability that improves performance by actively tracking modifications made to the content files of a connected repository. It uses a CSV file within the '.sentinel' folder in your repository to audit each commit. The workflow avoids redeploying content that hasn't been modified since the last deployment. This process improves your deployment performance and prevents tampering with unchanged content in your workspace, such as resetting dynamic schedules of your analytics rules.
73
-
74
-
Smart deployments are enabled by default on newly created connections. If you prefer all source control content to be deployed every time a deployment is triggered, regardless of whether that content was modified or not, you can modify your workflow to disable smart deployments. For more information, see [Customize the workflow or pipeline](ci-cd-custom-deploy.md#customize-the-workflow-or-pipeline).
75
-
76
-
> [!NOTE]
77
-
> This capability was launched in public preview on April 20th, 2022. Connections created prior to launch would need to be updated or recreated for smart deployments to be turned on.
78
-
>
91
+
The **smart deployments** feature is a back-end capability that improves performance by actively tracking modifications made to the content files of a connected repository. It uses a CSV file within the `.sentinel` folder in your repository to audit each commit. The workflow avoids redeploying content that hasn't been modified since the last deployment. This process improves your deployment performance and prevents tampering with unchanged content in your workspace, such as resetting dynamic schedules of your analytics rules.
79
92
93
+
Smart deployments are enabled by default on newly created connections. If you prefer all source control content deployed every time a deployment is triggered, whether that content was modified or not, modify your workflow to disable smart deployments. For more information, see [Customize the workflow or pipeline](ci-cd-custom-deploy.md#customize-the-workflow-or-pipeline).
80
94
81
95
## Consider deployment customization options
82
96
83
-
A number of customization options are available to consider when deploying content with Microsoft Sentinel repositories.
97
+
Consider the following customization options when deploying content with Microsoft Sentinel repositories.
84
98
85
99
#### Customize the workflow or pipeline
86
100
87
-
You may want to customize the workflow or pipeline in one of the following ways:
101
+
Customize the workflow or pipeline in one of the following ways:
88
102
- configure different deployment triggers
89
103
- deploy content only from a specific root folder for a given workspace
90
104
- schedule the workflow to run periodically
91
105
- combine different workflow events together
92
106
- turn off smart deployments
93
107
94
-
These customizations are defined in a .yml file specific to your workflow or pipeline. For more details on how to implement, see [Customize repository deployments](ci-cd-custom-deploy.md#customize-the-workflow-or-pipeline)
108
+
These customizations are defined in a .yml file specific to your workflow or pipeline. For more information on how to implement, see [Customize repository deployments](ci-cd-custom-deploy.md#customize-the-workflow-or-pipeline)
95
109
96
110
#### Customize the deployment
97
111
@@ -100,13 +114,13 @@ Once the workflow or pipeline is triggered, the deployment supports the followin
100
114
- exclude content from deployment
101
115
- specify ARM template parameter files
102
116
103
-
These options are available through a feature of the PowerShell deployment script called from the workflow or pipeline. For more details on how to implement these customizations, see [Customize repository deployments](ci-cd-custom-deploy.md#customize-your-connection-configuration).
117
+
These options are available through a feature of the PowerShell deployment script called from the workflow or pipeline. For more information on how to implement these customizations, see [Customize repository deployments](ci-cd-custom-deploy.md#customize-your-connection-configuration).
104
118
105
119
106
120
## Next steps
107
121
108
122
Get more examples and step by step instructions on deploying Microsoft Sentinel repositories.
109
123
110
124
-[Deploy custom content from your repository](ci-cd.md)
-[Automate Sentinel integration with DevOps](/azure/architecture/example-scenario/devops/automate-sentinel-integration#microsoft-sentinel-repositories)
0 commit comments