diff --git a/docs/content/contributing/bicep/ci-environment/_index.md b/docs/content/contributing/bicep/ci-environment/_index.md new file mode 100644 index 000000000..8d732ebb7 --- /dev/null +++ b/docs/content/contributing/bicep/ci-environment/_index.md @@ -0,0 +1,22 @@ +--- +title: Bicep CI environment +linktitle: CI # breadcrumb +description: Continuous Integration (CI) pipelines are essential for maintaining the quality and reliability of Azure Verified Modules. +--- + +Continuous Integration (CI) pipelines are essential for maintaining the quality and reliability of Azure Verified Modules. These pipelines automate the process of integrating code changes from multiple contributors and running tests. By automating these tasks, CI pipelines help to catch errors early, ensure that code changes do not introduce new issues, maintain a consistent codebase and ensuring idempotency for deployments. + +Using CI pipelines for Azure Verified Modules also facilitates collaboration among developers. Automated testing and integration reduce the time and effort required for manual code reviews and testing, allowing developers to focus on writing high-quality code. Additionally, CI pipelines provide immediate feedback on code changes, enabling developers to address issues promptly. This continuous feedback loop helps to improve the overall quality of the modules and ensures that they meet the rigorous standards expected by Azure users. + +The Continuous Integration environment (CI environment) is a set of automation components that are used for continuously validating and publishing module artifacts (Bicep registry repositories, template specs, universal packages). Technically, the CI environment consists of a DevOps platform hosting related pipelines and scripts, as well as an Azure environment (Azure AD tenant with management group(s) and subscription(s)) in which the modules are validated by the automation pipelines and scripts. + +- [Pipeline design]({{% siteparam base %}}/contributing/bicep/ci-environment/pipeline-design) + - [Static validation]({{% siteparam base %}}/contributing/bicep/ci-environment/static-validation) + - [Deployment Flow]({{% siteparam base %}}/contributing/bicep/ci-environment/deployment-flow) + - [Deployment validation]({{% siteparam base %}}/contributing/bicep/ci-environment/deployment-validation) + - [Deployment history cleanup]({{% siteparam base %}}/contributing/bicep/ci-environment/deployment-history-cleanup) + - [Publishing]({{% siteparam base %}}/contributing/bicep/ci-environment/publishing) + - [Token replacement]({{% siteparam base %}}/contributing/bicep/ci-environment/token-replacement) +- [Pipeline usage]({{% siteparam base %}}/contributing/bicep/ci-environment/pipeline-usage) +- [Bicep configuration]({{% siteparam base %}}/contributing/bicep/ci-environment/bicep-configuration) +- [Troubleshooting]({{% siteparam base %}}/contributing/bicep/ci-environment/troubleshooting) diff --git a/docs/content/contributing/bicep/ci-environment/bicep-configuration.md b/docs/content/contributing/bicep/ci-environment/bicep-configuration.md new file mode 100644 index 000000000..c719c3894 --- /dev/null +++ b/docs/content/contributing/bicep/ci-environment/bicep-configuration.md @@ -0,0 +1,28 @@ +--- +title: CI environment - Bicep Configuration +--- + +This section provides details on the Bicep configuration used in the CARML CI environment. This configuration happens in two places, the Bicep Configuration File (`bicepconfig.json`) and inside the modules themselves. + +--- + +## Bicep Configuration File + +Using this file, you can customize your Bicep development experience. This includes + +- Linter Rules (e.g., [max parameters](https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/linter-rule-max-parameters)) +- Source locations (e.g., [aliases](https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/bicep-config-modules)) + +For a full list of available rules, please refer to the [official documentation](https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/bicep-config). + +> If you remove this file, Bicep uses default values + +The configuration applied in the CI environment can be found in the `bicepconfig.json` file in the root folder. Next to the corresponding settings you will also find the rational for its application. + +## Module-level Bicep configuration + +The Bicep DSL (Domain Specific Language) is continuously improved and extended with additional capabilities. Of of them, the Bicep Linter, provides guidance around template design & best practices - and surfaces any findings as warnings. However, while it is a great feature, there can be cases where rules show false-positives - or are not addressed by us immediately. For these cases, we occasionally apply ignore tags such as `#disable-next-line secure-secrets-in-params` on a module level. + +> Note: Each ignore tag should be accompanied by a comment to justify its existence. + +> Note: As we want to follow best-practices whenever we can, the ignore tags should only be applied when absolutely necessary, on a case-by-case basis. diff --git a/docs/content/contributing/bicep/ci-environment/ci-environment-pipeline-design.md b/docs/content/contributing/bicep/ci-environment/ci-environment-pipeline-design.md new file mode 100644 index 000000000..637ac7576 --- /dev/null +++ b/docs/content/contributing/bicep/ci-environment/ci-environment-pipeline-design.md @@ -0,0 +1,131 @@ +--- +title: The CI environment Pipeline design +--- + +## Module Pipelines + +The repository hosts one pipeline for each module in the AVM library. + +The purpose of each module pipeline is twofold: + +1. **Validation**: To ensure the modules hosted by the AVM library are valid and can perform the intended deployments. +2. **Publishing**: To publish versioned and already validated modules to one or multiple target locations, from where they can be referenced by solutions consuming them. + +As such, each pipeline can be mapped to Phases 1 and 2 described in the Deployment flow section. + +![Pipeline Phases](/Azure-Verified-Modules/images/bicep-ci/pipeline-design.png) + +The following paragraphs provide an overview of the different phases and shared logic the module pipelines use. + +## Pipeline phases + +This paragraph provides an overview of the three phases performed by each module pipeline. Further details about the implementation and design of each phase are provided on the dedicated pages linked below. + +1. **Static Validation**: Runs a set of static Pester tests on the module and its templates to ensure they comply with the design principles. Further details for this phase are provided on the corresponding wiki page - see the [Static validation](/Azure-Verified-Modules/contributing/bicep/ci-environment/static-validation) section. +1. **Deployment Validation**: An actual Azure deployment is run in a sandbox subscription leveraging a predefined set of module test files, each validating a different configuration of the same Azure resource in parallel. The test suite is cleaned up by default, removing all test resources post-deployment. Further details for this phase are provided on the corresponding wiki page - see the [Deployment validation](/Azure-Verified-Modules/contributing/bicep/ci-environment/deployment-validation) section. +1. **Publishing**: Runs only if the previous steps are successful. A new module version is published to all configured target locations such as template specs, private Bicep registry and Azure DevOps Universal Packages. Published module versions can then be referenced by solutions using them. Further details for this phase are provided on the corresponding wiki page - see the [Publishing](/Azure-Verified-Modules/contributing/bicep/ci-environment/publishing) page. + +![Pipeline Design Phases](/Azure-Verified-Modules/images/bicep-ci/pipeline-design-phases.png) + +### GitHub-specific design + +![Pipeline Phases GitHub](/Azure-Verified-Modules/images/bicep-ci/pipeline-phases-github.png) + +GitHub workflows map each pipeline phase to a dedicated composite action, to maximize code reusability. +The mapping to the specific composite action is provided below: + +| Composite Action | Pipeline phase | +| - | - | +| **Initialize pipeline** | Prepare the pipeline by (for example) publishing pipeline variables, collecting test files, etc. | +| **Module / Static validation** | Static validation | +| **Module / PSRule validation** | PS-Rule validation (with one job per template test file) | +| **Module / Deployment validation** | Deployment validation (with one job per template test file) | +| **publishModule** | Publishing | + +// TODO the next section. +// is it this? + outputs: + workflowInput: ${{ steps.get-workflow-param.outputs.workflowInput }} + moduleTestFilePaths: ${{ steps.get-module-test-file-paths.outputs.moduleTestFilePaths }} + psRuleModuleTestFilePaths: ${{ steps.get-module-test-file-paths.outputs.psRuleModuleTestFilePaths }} + modulePath: "${{ env.modulePath }}" +// end todo +In addition, workflows leverage the following composite actions: + +| Composite Action | Description | +| - | - | +| **getWorkflowInput** | This action allows fetching workflow input values from the module's workflow file, even if the pipeline was not triggered via a `workflow_dispatch` action. Without it, we would not be able to process the contained information and would need to duplicate the configuration as workflow variables. Such input values are for example, the removal switch `removeDeployment`. | +| **setEnvironment** | This action sets up GitHub runners with relevant PowerShell modules required for specific jobs. It then parses the settings file ([`settings.yml`](https://github.com/Azure/ResourceModules/blob/main/settings.yml)) and sets the key-value pairs in the `variables` list as environment variables. | + +Technical documentation for each composite action, such as required input and output variables, is included in each `action.yml` file located in path `.github/actions/templates`. + +## Module pipeline inputs + +Each module pipeline comes with the following runtime parameters: + +- `'Branch'` dropdown: A dropdown to select the branch to run the pipeline from. +- `'Execute static validation'` switch: Can be enabled or disabled. It controls whether the static tests jobs are executed during the pipeline run. + > **Note:** This switch cannot be used to bypass the publishing requirements, where both static tests & deployment validation jobs must be successful. +- `'Execute deployment validation'` switch: Can be enabled or disabled. It controls whether the deployment validation jobs are executed during the pipeline run. +- `'Remove deployed module'` switch: Can be enabled or disabled. It controls whether the test-deployed resources are removed after the deployment validation jobs. It is enabled by default. + > **Note:** This switch cannot be used to bypass the publishing requirements, where both the static tests & deployment validation jobs must be successful. +- `'Default location overwrite'` string: By default, a region is choosen randomly, that the resources are deployed into. This parameter let's you overwrite a random region with one you specify (e.g. eastus). + +![Module Pipeline Input](/Azure-Verified-Modules/images/bicep-ci/module-pipeline-input.png) + +--- + +## Platform pipelines + +In addition to module pipelines, the repository includes several platform pipelines covering further tasks as described below. + +- [PSRule Pre-Flight validation pipeline](#psrule-pre-flight-validation-pipeline) +- [Deployment history cleanup](/Azure-Verified-Modules/contributing/bicep/ci-environment/deployment-history-cleanup) +// TODO add missing pipelines + +## PSRule Pre-Flight validation pipeline + +The purpose of the PSRule Pre-Flight validation pipeline is to validate Azure resources deployed by module validation pipeline tests, by leveraging [PSRule for Azure](https://azure.github.io/PSRule.Rules.Azure/about/). +PSRule for Azure is aligned to the [Well-Architected Framework (WAF)](https://learn.microsoft.com/en-us/azure/architecture/framework/). Tests, called _Rules_, check the configuration of Azure resources against WAF principles. + +The pipeline, currently only available as a [GitHub workflow](https://github.com/Azure/bicep-registry-modules/blob/main/.github/workflows/platform.check.psrule.yml), runs weekly on the whole library, providing as output the list of non-compliant resources and corresponding failing rules, if any. + +### Configuration settings + +PSRule options set for the AVM repository are configured in the [ps-rule.yaml](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/psrule/ps-rule.yaml) file. + +Documentation for all configuration options is available at the following links: + +- [https://aka.ms/ps-rule/options](https://aka.ms/ps-rule/options) +- [https://aka.ms/ps-rule-azure/options](https://aka.ms/ps-rule-azure/options) + +### Baselines + +A [baseline](https://azure.github.io/PSRule.Rules.Azure/working-with-baselines/) is a standard PSRule artifact that combines rules and configuration. The PSRule Pre-Flight validation pipeline uses the default baseline to analyze module test resources. + +For the list of all rules included see [Azure.Default baseline](https://azure.github.io/PSRule.Rules.Azure/en/baselines/Azure.Default/). +To view a list of rules by Azure resources see [Rules by resource](https://azure.github.io/PSRule.Rules.Azure/en/rules/resource/). + +### Exclusions and suppression rules + +Not all baseline rules may be valid for some of the test Azure resources deployed by the module validation pipelines. + +For example, resources deployed by the min tests, aim to validate only the required input parameters for each module. +Therefore, optional features such as diagnostic settings are not configured in those tests. Since enabling logging is a general recommendation for most of the resources supporting them, missing diagnostic settings usually trigger incopliance of PSRule checks, e.g., [Azure.KeyVault.Logs](https://azure.github.io/PSRule.Rules.Azure/en/rules/Azure.KeyVault.Logs/). For this reason, these checks are excluded from being evaluated for resources deployed by min tests. + +PSRule allows skipping rules on two levels: + +- **Exclusions**: Can be leveraged to exclude specific baseline rules from being evaluated for any resource. + - [ps-rule.yaml](https://github.com/Azure/ResourceModules/blob/main/ps-rule.yaml): Lists the name of specific rules to exclude under the option [Rule.Exclude](https://microsoft.github.io/PSRule/v2/concepts/PSRule/en-US/about_PSRule_Options/#ruleexclude) +- **Suppression Groups**: PSRule can use [Suppression Groups](https://microsoft.github.io/PSRule/v2/concepts/PSRule/en-US/about_PSRule_SuppressionGroups/) to suppress rules based on a condition. Suppression groups can be leveraged when some of the rules in the baseline are not relevant under specific conditions, e.g., only for specific resources. They are stored in the `.ps-rule` repo folder in `.yaml` format. In particular: + - [cb-waf-security.Rule.yaml](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/psrule/.ps-rule/cb-waf-security.Rule.yaml): Defines the module-list for the WAF Security recommendations + - [dep-suppress.Rule.yaml](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/psrule/.ps-rule/dep-suppress.Rule.yaml): Lists rules to be ignored for resources deployed as dependencies + - [min-suppress.Rule.yaml](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/psrule/.ps-rule/min-suppress.Rule.yaml): Lists rules to be ignored for resources deployed by the min tests + - [na-suppress.Rule.yaml](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/psrule/.ps-rule/na-suppress.Rule.yaml): Lists rules to be ignored for resources not supporting Tags + - [storage-firewall-suppress.Rule.yaml](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/psrule/.ps-rule/storage-firewall-suppress.Rule.yaml): Surpress Rules for resources, that can't activate firewalls + +### Output + +To better outline failed rules and allow fixing incompliant resources quickly, the pipeline leverages the script [utilities\pipelines\PSRulestaticValidation\psrule\Set-PSRuleGitHubOutput.ps1](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/psrule/Set-PSRuleGitHubOutput.ps1) to aggregate PSRule output into Custom Markdown content and display it to the Actions run summary page. + +![PSRule Summary](/Azure-Verified-Modules/images/bicep-ci/psrule-summary.png) diff --git a/docs/content/contributing/bicep/ci-environment/deployment-flow.md b/docs/content/contributing/bicep/ci-environment/deployment-flow.md new file mode 100644 index 000000000..a89709bf9 --- /dev/null +++ b/docs/content/contributing/bicep/ci-environment/deployment-flow.md @@ -0,0 +1,47 @@ +--- +title: Deployment Flow +--- + +This page provides an overview of the standard development-to-deployment flow that goes from source modules to target solutions. + +![Deployment Flow]({{% siteparam base %}}/images/bicep-ci/deployment_flow.png?width=400px) + +This flow generally covers 3 phases: + +1. In the Develop modules phase modules are first implemented/updated and then validated using one or multiple module test files, testing their successful deployment to a sandbox subscription to prove their correctness. +2. The next phase, Publish modules, packages and publishes the tested and approved modules to a target location for later consumption. The target location (also known as package store or artifact store) should support versioning to allow referencing a specific module version and to avoid breaking changes when referencing them. +3. In the final Consume modules phase, published modules are referenced and combined to deploy more complex architectures (multi-module solutions) such as workloads/applications or individual services. + +### Module Versioning + +Deploying resources by referencing their corresponding modules from source control has one major drawback: If your deployments directly rely on your source repository, then they will by default use the latest code. +Applying software development lifecycle concepts like publishing build artifacts and versioning enables you to have a point in time version of a module. By introducing versions to your modules, the consuming orchestration can and should specify a module version needed, and deploy the Azure solution leveraging it. +In case a breaking change is introduced to a module and an updated version is published, no deployments are affected as they still reference the previously published version. Instead, a deliberate decision must be made to upgrade the solution to reference newer module versions. +Also, if you reference a module version that was tested in and has passed through the CI environment, you can trust that it complies the qualitative and functional standards. + +## Where does the AVM CI environment fit in? + +To ensure the modules hosted by the AVM library are valid and can perform the intended deployments, the repository comes with a continuous integration (CI) environment for each module. If the validation is successful, the CI environment is also publishing versioned modules to one or multiple target locations, from where they can be referenced by solutions consuming them. +The CI environment covers Phase #1, (the validation) & Phase #2 (the publishing) of the [deployment flow]({{% siteparam base %}}/contributing/bicep/ci-environment/deployment-flow/) section - these include the steps typically performed by the module developer persona. +The AVM solution developer and solution consumer personas are usually working in Phase #2 and Phase #3, i.e., building/leveraging complex, multi-module solutions by consuming the already tested and versioned modules previously placed in an artifact store. +The below diagram shows the details of how the different phases are interconnected: + +![Deployment Flow]({{% siteparam base %}}/images/bicep-ci/deployment_flow_detail_white.png?width=400px) + +The top row represents your orchestration environment, for example, GitHub or Azure DevOps. The bottom row represents your Azure cloud environment. +From left to right, there are the three phases introduced before, Develop modules, Publish modules & Consume modules. The diagram shows how each phase interacts with the Azure environment. + +1. Starting with **Develop modules**, the top left box shows the test pipelines that exist for each module, performing the following steps: + + - *Static validation*: Pester & PSRule tests are run on each module to ensure a baseline code quality across the library. + - *Deployment validation*: An actual Azure deployment is performed in a validation/sandbox subscription, shown in the bottom left corner. The subscription is intended to be without any link to production. Resources deployed here should be considered temporary and be removed after testing. + - *Publishing*: Runs only if the previous steps are successful and initiates the second phase as described below. + +2. The **Publish modules** phase is shown in the center box of the diagram. If all tests for a module succeed, the module is published to a given target location. Currently, the target locations supported by the AVM CI environment are: + + - [Template specs](https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/template-specs?tabs=azure-powershell) + - [Private Bicep registry](https://learn.microsoft.com/en-gb/azure/azure-resource-manager/bicep/private-module-registry) + + To dive deeper and understand which target locations may be best suited for your use case, we provide further information in the Publish-location considerations section. + +3. The third phase, **Consume modules** is represented on the right. The top right corner provides examples of orchestrations deploying the target solutions by referencing the published modules. The deployments performed in this third phase are supposed to target an integration/production environment. This phase references the validated and published modules coming out of the AVM CI environment, and leverages them with the correctly configured parameters to orchestrate their deployment in the intended order. diff --git a/docs/content/contributing/bicep/ci-environment/deployment-history-cleanup.md b/docs/content/contributing/bicep/ci-environment/deployment-history-cleanup.md new file mode 100644 index 000000000..898a317ae --- /dev/null +++ b/docs/content/contributing/bicep/ci-environment/deployment-history-cleanup.md @@ -0,0 +1,48 @@ +--- +title: CI environment - Deployment History Cleanup +--- + +This pipeline and its scripts can be used to limit the number of deployments that are listed in the Azure deployments history at any given time. This is especially useful for a scope like 'Management Group', since for that scope, Azure Resource Manager does not currently provide [automatic deletion](https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/deployment-history-deletions?tabs=azure-powershell) of deployments from your history as you near the 800 limit. + +Without automatic deletion, if the scope you are deploying to is already at the 800 limit, your next deployment fails with an error. The corresponding error message would look similar to `Creating the deployment '' would exceed the quota of '800'. The current deployment count is '804'. Please delete some deployments before creating a new one, or see https://aka.ms/800LimitFix for information on managing deployment limits.` + +The platform pipeline [`platform.deployment.history.cleanup.yml`](https://github.com/Azure/bicep-registry-modules/blob/main/.github/workflows/platform.deployment.history.cleanup.yml) has an automatic schedule that runs every night, but can also be triggered on-demand. + +By default, the pipeline will cleanup both the Subscription, as well as Management Group scope using the script's default configuration. In addition, you can specify a pipeline variable `maxDeploymentRetentionInDays` to define beyond which time frame deployments should always be deleted. Within the timeframe, only non-failed and non-running deployments are considered. + +> **NOTE**
In order for the pipeline to remove both Management-Group-Level as well as Subscription-Level deployments the used service principal needs the [permissions](https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/deployment-history-deletions?tabs=azure-powershell#required-permissions) either scope. If you use AVM to test deployments on any of these scopes your principal will already have the required permissions.
+> In case the principal does not have the required permissions you can either: +> - Not register the `platform.deployment.history.cleanup.yml` pipeline OR +> - Adjust the pipeline's default behavior to not remove deployments on the scope you'd like to preserve (i.e., change `default: true` to `default: false`) + +--- + +## Location + +You can find the used scripts under + +- [`/utilities/pipelines/platform/deploymentRemoval/Clear-SubscriptionDeploymentHistory.ps1`](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/platform/deploymentRemoval/Clear-SubscriptionDeploymentHistory.ps1) +- [`/utilities/pipelines/platform/deploymentRemoval/Clear-ManagementGroupDeploymentHistory.ps1`](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/platform/deploymentRemoval/Clear-ManagementGroupDeploymentHistory.ps1) + +## How it works + +Both scripts work in the same way: + +1. The script fetches all current deployments at a given scope from Azure (i.e., Management-Group-scope or Subscription-scope). + - For example, it may find + - 120 successful deployments + - 10 failed deployments that are 3 weeks old + - 10 failed deployments that are 1 week old + - 10 running deployments +1. By default, it then filters them down to non-running & non-failing deployments (can be modified). It only considers failed deployments for deletion if they are older than a provided time limit. + - Following the example, and with a `maxDeploymentRetentionInDays` of 14 days (2 weeks), the following deployments are targeted for deletion from the deployment history: + - 120 successful deployments + - 10 failed deployments that are 3 weeks old +1. Lastly, it removes all matching deployments in chunks of 100 deployments each. + - In the context of the example, this means the procedure deletes the deployments in 2 chunks, first removing 100, and then 30 deployments. + +## How to use it + +For details on how to use the functions, please refer to the scripts local documentation. + +> **Note:** Each script must be loaded ('*dot-sourced*') before the function can be invoked. diff --git a/docs/content/contributing/bicep/ci-environment/deployment-validation.md b/docs/content/contributing/bicep/ci-environment/deployment-validation.md new file mode 100644 index 000000000..1a4d32a73 --- /dev/null +++ b/docs/content/contributing/bicep/ci-environment/deployment-validation.md @@ -0,0 +1,154 @@ +--- +title: CI environment - Deployment Validation +linktitle: Deployment Validation # breadcrumb +--- + +This section provides an overview of the principles the deployment validation is built upon, how it is set up, and how you can interact with it. + +![Deployment Validation Step]({{% siteparam base %}}/images/bicep-ci/deployment-validation-step.png?width=400px) + +## Deployment validation steps + +The deployment validation phase can be divided into three steps, running in sequence: + +- **Template validation:** Tests the module template is valid before the actual deployment. +- **Azure deployment validation:** Performs the actual Azure deployments. +- **Resource removal:** Deletes deployed resources. + +## Template validation + +The template validation step performs a dry-run with each module test file in the module's `'.test'` folder (and its subfolders) + +In particular, the step runs a `Test-AzDeployment` cmdlet (_the command may vary based on the template schema_) for each provided module test file to verify if the template could be deployed using them. + +The intention of this test is to **fail fast**, before getting to the later deployment step. The template validation could fail either because the template is invalid, or because any of the module test files is configured incorrectly. + +## Azure deployment validation + +This step performs the actual Azure deployments using each available & configured module test file. The purpose of this step is to prove the module can be deployed in different configurations based on the different parameters provided. Deployments for the different variants happen in parallel. + +If any of these parallel deployments require multiple/different/specific resource instances already present, these resources are deployed by the module test files before the module to validate. You can find additional information about this effort [here]({{% siteparam base %}}/spec/SNFR2/#required-resourcesdependencies-required-for-e2e-tests). + +The module test files used in this stage should ideally cover as many configurations as possible to validate the template flexibility, i.e., to verify that the module can cover multiple scenarios in which the given Azure resource may be used. Using the example of the CosmosDB module, we may want to have one module test file for the minimum amount of required parameters, one module test file for each CosmosDB type to test individual configurations, and at least one module test file testing the supported extension resources such as RBAC & diagnostic settings. + +> **Note:** Since every customer environment might be different due to applied Azure Policies or security policies, modules might behave differently and naming conventions need to be verified beforehand. + +> **Note:** [Management-Group](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits#management-group-limits) or [Subscription](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits#subscription-limits) deployments may eventually exceed the limit of 800 and require you to remove some of them manually. If you are faced with any corresponding error message you can manually remove deployments on a Management-Group or Subscription Level on scale using one of our [utilities](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/e2eValidation/resourceRemoval/Initialize-DeploymentRemoval.ps1). The CI environment also comes with a [pipeline](https://github.com/Azure/bicep-registry-modules/blob/main/.github/workflows/platform.deployment.history.cleanup.yml) that runs on a nightly basis to mitigate this limitation. + +### Output example + +![Deployment Validation Output]({{% siteparam base %}}/images/bicep-ci/deployment-validation-output.png?width=400px) + +### Resource removal + +This paragraph describes how the removal of resources deployed by a module is performed and how to modify the default behavior if a specific module or resource type needs it. + +The removal step is triggered after the deployment completes. It removes all resources deployed in the previous step. The reason is twofold: + +- Make sure to keep the validation subscription cost as low as possible. +- Run test deployments from scratch at every run. + +However, the removal step can be skipped in case further investigation on the deployed resource is needed. This can be controlled when running the module pipeline leveraging [Module pipeline inputs]({{% siteparam base %}}/contributing/bicep/ci-environment/pipeline-design/#module-pipeline-inputs). + +> Note: The logic will consider all deployment names used during the deployment step - even those of retries. + +### How it works + +The removal process will delete all resources created by the deployment. The list of resources is identified by: + +1. Recursively fetching the list of resource IDs created in the deployment(s) (identified via the deployment names(s) used). +1. Ordering the list based on resource IDs segment count (ensures child resources are removed first. E.g., `storageAccount/blobServices` comes before `storageAccount` as it has one more segments delimited by `/`). +1. Filtering out resources must remain even after the test concluded from the list. This contains, but is not limited to: + 1. Resources that are autogenerated by Azure and can cause issues if not controlled (e.g., the Network Watcher resource group that is autogenerated and shared by multiple module tests) + 1. Resources of specific resource types. This currently involves the following: + - `Microsoft.Security/autoProvisioningSettings` + - `Microsoft.Security/deviceSecurityGroups` + - `Microsoft.Security/iotSecuritySolutions` + - `Microsoft.Security/pricings` + - `Microsoft.Security/securityContacts` + - `Microsoft.Security/workspaceSettings` +1. Moving specific resource types to the top of the list (if a certain order is required). For example, `diagnosticSettings` need to be removed before the resource to which they are applied, even though they are no child-resources. + +After a resource is removed (this happens after each resource in the list), if defined, the script will perform a **post removal operation**. This can be used for those resource types that require post-processing, like purging a soft-deleted Key Vault. + +The procedure is initiated post-deployment by the script [`/utilities/pipelines/e2eValidation/resourceRemoval/Initialize-DeploymentRemoval.ps1`](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/e2eValidation/resourceRemoval/Initialize-DeploymentRemoval.ps1) in the pipeline templates: + +- (GitHub) [`/.github/actions/templates/validateModuleDeployment/action.yml`](https://github.com/Azure/bicep-registry-modules/blob/main/.github/actions/templates/avm-validateModuleDeployment/action.yml) + +It uses several helper scripts that can be found in its `helper` subfolder + +### Create a specialized removal procedure + +This paragraph is intended for AVM contributors who want to add a new module to the library. It contains instructions on how to customize the removal scripts if needed for any specific resource. + +The default removal procedure works for most of the modules. As such, it is unlikely you'll have to change anything to enable your new module for post-deployment removal. + +However, if you need to, you can define a custom removal procedure by: + +1. Influencing the **order** in which resources are removed by prioritizing specific resource types. + > **Example** _Diagnostic settings_ need to be removed before the resource to which they are applied. +1. Defining a **custom removal** action to remove a resource of a _specific resource type_. + > **Example** A _Recovery Services Vault_ resource requires some protected items to be identified and removed before the vault itself can be removed. +1. Defining a **custom post-removal** action to be run after removing a resource of a _specific resource type_. + > **Example** A _Key Vault_ resource needs to be purged when soft deletion is enforced. + +Those methods can be combined independently. + +To modify the resource types removal **order**: + +1. Open the [`/utilities/pipelines/e2eValidation/resourceRemoval/Initialize-DeploymentRemoval.ps1`](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/e2eValidation/resourceRemoval/Initialize-DeploymentRemoval.ps1) file. +1. Look for the following comment: `### CODE LOCATION: Add custom removal sequence here` +1. Add a case value that matches your resource type +1. In the case block, update the `$removalSequence` variable value to accommodate your module requirements +1. Remember to add the `break` statement. + +To define a **custom removal** action: + +1. Open the [`/utilities/pipelines/e2eValidation/resourceRemoval/helper/Invoke-ResourceRemoval.ps1`](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/e2eValidation/resourceRemoval/helper/Invoke-ResourceRemoval.ps1) file. +1. Look for the following comment: `### CODE LOCATION: Add custom removal action here` +1. Add a case value that matches the resource type you want to customize the removal action for +1. In the case block, define the resource-type-specific removal action + +To add a **custom post-removal** step: + +1. Open the [`/uutilities/pipelines/e2eValidation/resourceRemoval/helper/Invoke-ResourcePostRemoval.ps1`](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/e2eValidation/resourceRemoval/helper/Invoke-ResourcePostRemoval.ps1) file. +1. Look for the following comment: `### CODE LOCATION: Add custom post-removal operation here` +1. Add a case value that matches the resource type you want to add a post-removal operation for +1. In the case block, define the resource-type-specific post-removal action + +## Verify the deployment validation of your module locally + +This paragraph is intended for AVM contributors or more generally for those leveraging the AVM CI environment and want to update or add a new module to the library. + +You can leverage the below snippet to leverage the 'Test-ModuleLocally.ps1' script to verify if your module will comply with the deployment validation step before pushing to source control. + +```powershell +#########[ Function Test-ModulesLocally.ps1 ]############# +$pathToRepository = '' +. "$pathToRepository\utilities\tools\Test-ModuleLocally.ps1" + +# REQUIRED INPUT FOR TESTING +$TestModuleLocallyInput = @{ + templateFilePath = '' + parameterFilePath = '' + PesterTest = $false + PesterTestRecurse = $true + DeploymentTest = $true + ValidationTest = $true + ValidateOrDeployParameters = @{ + Location = '' + ResourceGroupName = 'validation-rg' + SubscriptionId = '' + ManagementGroupId = '' + RemoveDeployment = $true + } + AdditionalTokens = @{ + namePrefix = 'abcd' + TenantId = '00000000-0000-0000-0000-000000000000' + } +} + +Test-ModuleLocally @TestModuleLocallyInput -Verbose +``` + +> You can use the `Get-Help` cmdlet to show more options on how you can use this script. diff --git a/docs/content/contributing/bicep/ci-environment/module-changelog-automation.md b/docs/content/contributing/bicep/ci-environment/module-changelog-automation.md new file mode 100644 index 000000000..53d38cb25 --- /dev/null +++ b/docs/content/contributing/bicep/ci-environment/module-changelog-automation.md @@ -0,0 +1,92 @@ +--- +title: Module Changelog Generation +--- + +Changelogs are crucial in software development as they provide a detailed and chronological record of all notable changes made to a project. This documentation helps developers, project managers, and users to track the progress of the project, understand the evolution of features, and identify when and where bugs were introduced or fixed. Changelogs also facilitate better communication within the development team and with stakeholders, ensuring everyone is aware of the latest updates and improvements. Additionally, they are essential for maintaining transparency and accountability in the development process. + +{{< tabs title="Changelog examples" >}} + {{% tab title="new release template" %}} + +## new release template + +This template should be included right after the `# Changelog` line in any PR that changes the version number of a module. + +```text +## + +### Changes + +- this changed +- and this also + +### Breaking Changes + +- none + +``` + + {{% /tab %}} + {{% tab title="changelog for a PR" %}} + +## changelog for a PR + +The current version changes are inserted after the opening heading and adjusted acording to the changes in the PR. + +```text +# Changelog + +## 0.2.1 + +### Changes + +- Updated the referenced AVM common types +- The recently introduced minCPU parameter is now applied + +### Breaking Changes + +- none + +## 0.2.0 + +### Changes + +- Implemented the minCPU parameter +- Updated the referenced VirtualNetwork module +- Updated the referenced AVM common types + +### Breaking Changes + +- The minCPU parameter is mandatory + +## 0.1.0 + +### Changes + +- Initial Release + +### Breaking Changes + +- none + +``` + {{% /tab %}} +{{% /tabs %}} + +## Generation process + +Whenever the version of a (sub-)module is increased (major, minor or patch), static tests ensure the existance of + +1. a `CHANGELOG.md` file in the modules root directory +2. the changelog file contains a section '## \' with two sections + - Changes + - Breaking Changes + +### CI pipelines + +Whenever code is being checked in, the module's specific test is triggered (if the corresponding action is not deactivated). + +1. avm.[res|ptn|utl].[provider-namespace].[resource-type] - the default [module pipeline]({{% siteparam base %}}/contributing/bicep/ci-environment/pipeline-design/#module-pipelines) +2. avm.template.module - triggered by the above workflow with module specific parameters + 1. static validation tests, which make sure the CHANGELOG.md is well-formatted + +![GitHub Commit message]({{% siteparam base %}}/images/bicep-ci/gh-changelog-commit.png?width=400px) diff --git a/docs/content/contributing/bicep/ci-environment/old-ci-environment-pipeline-design.md b/docs/content/contributing/bicep/ci-environment/old-ci-environment-pipeline-design.md new file mode 100644 index 000000000..bb4d1dbb5 --- /dev/null +++ b/docs/content/contributing/bicep/ci-environment/old-ci-environment-pipeline-design.md @@ -0,0 +1,134 @@ +--- +title: CI environment - Pipeline design +--- + +## Module Pipelines + +The repository hosts one pipeline for each module in the AVM library. + +The purpose of each module pipeline is twofold: + +1. **Validation**: To ensure the modules hosted by the AVM library are valid and can perform the intended deployments. +2. **Publishing**: To publish versioned and already validated modules to one or multiple target locations, from where they can be referenced by solutions consuming them. + +As such, each pipeline can be mapped to Phases 1 and 2 described in the Deployment flow section. + +![Pipeline Phases](/Azure-Verified-Modules/images/bicep-ci/pipeline-design.png) + +The following paragraphs provide an overview of the different phases and shared logic the module pipelines use. + +- [Pipeline phases](#pipeline-phases) +- [Module pipeline inputs](#module-pipeline-inputs) + +## Pipeline phases + +This paragraph provides an overview of the three phases performed by each module pipeline. Further details about the implementation and design of each phase are provided on the dedicated pages linked below. + +1. **Static Validation**: Runs a set of static Pester tests on the module and its templates to ensure they comply with the design principles. Further details for this phase are provided on the corresponding wiki page - see the [Static validation](/Azure-Verified-Modules/contributing/bicep/ci-environment/static-validation) section. +1. **Deployment Validation**: An actual Azure deployment is run in a sandbox subscription leveraging a predefined set of module test files, each validating a different configuration of the same Azure resource in parallel. The test suite is cleaned up by default, removing all test resources post-deployment. Further details for this phase are provided on the corresponding wiki page - see the [Deployment validation](/Azure-Verified-Modules/contributing/bicep/ci-environment/deployment-validation) section. +1. **Publishing**: Runs only if the previous steps are successful. A new module version is published to all configured target locations such as template specs, private Bicep registry and Azure DevOps Universal Packages. Published module versions can then be referenced by solutions using them. Further details for this phase are provided on the corresponding wiki page - see the [Publishing](/Azure-Verified-Modules/contributing/bicep/ci-environment/publishing) page. + +![Pipeline Design Phases](/Azure-Verified-Modules/images/bicep-ci/pipeline-design-phases.png) + +### GitHub-specific design + +![Pipeline Phases GitHub](/Azure-Verified-Modules/images/bicep-ci/pipeline-phases-github.png) + +GitHub workflows map each pipeline phase to a dedicated composite action, to maximize code reusability. +The mapping to the specific composite action is provided below: + +| Composite Action | Pipeline phase | +| - | - | +| **Initialize pipeline** | Prepare the pipeline by (for example) publishing pipeline variables, collecting test files, etc. | +| **Module / Static validation** | Static validation | +| **Module / PSRule validation** | PS-Rule validation (with one job per template test file) | +| **Module / Deployment validation** | Deployment validation (with one job per template test file) | +| **publishModule** | Publishing | + +// TODO the next section. +// is it this? + outputs: + workflowInput: ${{ steps.get-workflow-param.outputs.workflowInput }} + moduleTestFilePaths: ${{ steps.get-module-test-file-paths.outputs.moduleTestFilePaths }} + psRuleModuleTestFilePaths: ${{ steps.get-module-test-file-paths.outputs.psRuleModuleTestFilePaths }} + modulePath: "${{ env.modulePath }}" +// end todo +In addition, workflows leverage the following composite actions: + +| Composite Action | Description | +| - | - | +| **getWorkflowInput** | This action allows fetching workflow input values from the module's workflow file, even if the pipeline was not triggered via a `workflow_dispatch` action. Without it, we would not be able to process the contained information and would need to duplicate the configuration as workflow variables. Such input values are for example, the removal switch `removeDeployment`. | +| **setEnvironment** | This action sets up GitHub runners with relevant PowerShell modules required for specific jobs. It then parses the settings file ([`settings.yml`](https://github.com/Azure/ResourceModules/blob/main/settings.yml)) and sets the key-value pairs in the `variables` list as environment variables. | + +Technical documentation for each composite action, such as required input and output variables, is included in each `action.yml` file located in path `.github/actions/templates`. + +## Module pipeline inputs + +Each module pipeline comes with the following runtime parameters: + +- `'Branch'` dropdown: A dropdown to select the branch to run the pipeline from. +- `'Execute static validation'` switch: Can be enabled or disabled. It controls whether the static tests jobs are executed during the pipeline run. + > **Note:** This switch cannot be used to bypass the publishing requirements, where both static tests & deployment validation jobs must be successful. +- `'Execute deployment validation'` switch: Can be enabled or disabled. It controls whether the deployment validation jobs are executed during the pipeline run. +- `'Remove deployed module'` switch: Can be enabled or disabled. It controls whether the test-deployed resources are removed after the deployment validation jobs. It is enabled by default. + > **Note:** This switch cannot be used to bypass the publishing requirements, where both the static tests & deployment validation jobs must be successful. +- `'Default location overwrite'` string: By default, a region is choosen randomly, that the resources are deployed into. This parameter let's you overwrite a random region with one you specify (e.g. eastus). + +![Module Pipeline Input](/Azure-Verified-Modules/images/bicep-ci/module-pipeline-input.png) + +--- + +## Platform pipelines + +In addition to module pipelines, the repository includes several platform pipelines covering further tasks as described below. + +- [PSRule Pre-Flight validation pipeline](#psrule-pre-flight-validation-pipeline) +- [Deployment history cleanup](/Azure-Verified-Modules/contributing/bicep/ci-environment/deployment-history-cleanup) +// TODO add missing pipelines + +## PSRule Pre-Flight validation pipeline + +The purpose of the PSRule Pre-Flight validation pipeline is to validate Azure resources deployed by module validation pipeline tests, by leveraging [PSRule for Azure](https://azure.github.io/PSRule.Rules.Azure/about/). +PSRule for Azure is aligned to the [Well-Architected Framework (WAF)](https://learn.microsoft.com/en-us/azure/architecture/framework/). Tests, called _Rules_, check the configuration of Azure resources against WAF principles. + +The pipeline, currently only available as a [GitHub workflow](https://github.com/Azure/bicep-registry-modules/blob/main/.github/workflows/platform.check.psrule.yml), runs weekly on the whole library, providing as output the list of non-compliant resources and corresponding failing rules, if any. + +### Configuration settings + +PSRule options set for the AVM repository are configured in the [ps-rule.yaml](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/psrule/ps-rule.yaml) file. + +Documentation for all configuration options is available at the following links: + +- [https://aka.ms/ps-rule/options](https://aka.ms/ps-rule/options) +- [https://aka.ms/ps-rule-azure/options](https://aka.ms/ps-rule-azure/options) + +### Baselines + +A [baseline](https://azure.github.io/PSRule.Rules.Azure/working-with-baselines/) is a standard PSRule artifact that combines rules and configuration. The PSRule Pre-Flight validation pipeline uses the default baseline to analyze module test resources. + +For the list of all rules included see [Azure.Default baseline](https://azure.github.io/PSRule.Rules.Azure/en/baselines/Azure.Default/). +To view a list of rules by Azure resources see [Rules by resource](https://azure.github.io/PSRule.Rules.Azure/en/rules/resource/). + +### Exclusions and suppression rules + +Not all baseline rules may be valid for some of the test Azure resources deployed by the module validation pipelines. + +For example, resources deployed by the min tests, aim to validate only the required input parameters for each module. +Therefore, optional features such as diagnostic settings are not configured in those tests. Since enabling logging is a general recommendation for most of the resources supporting them, missing diagnostic settings usually trigger incopliance of PSRule checks, e.g., [Azure.KeyVault.Logs](https://azure.github.io/PSRule.Rules.Azure/en/rules/Azure.KeyVault.Logs/). For this reason, these checks are excluded from being evaluated for resources deployed by min tests. + +PSRule allows skipping rules on two levels: + +- **Exclusions**: Can be leveraged to exclude specific baseline rules from being evaluated for any resource. + - [ps-rule.yaml](https://github.com/Azure/ResourceModules/blob/main/ps-rule.yaml): Lists the name of specific rules to exclude under the option [Rule.Exclude](https://microsoft.github.io/PSRule/v2/concepts/PSRule/en-US/about_PSRule_Options/#ruleexclude) +- **Suppression Groups**: PSRule can use [Suppression Groups](https://microsoft.github.io/PSRule/v2/concepts/PSRule/en-US/about_PSRule_SuppressionGroups/) to suppress rules based on a condition. Suppression groups can be leveraged when some of the rules in the baseline are not relevant under specific conditions, e.g., only for specific resources. They are stored in the `.ps-rule` repo folder in `.yaml` format. In particular: + - [cb-waf-security.Rule.yaml](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/psrule/.ps-rule/cb-waf-security.Rule.yaml): Defines the module-list for the WAF Security recommendations + - [dep-suppress.Rule.yaml](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/psrule/.ps-rule/dep-suppress.Rule.yaml): Lists rules to be ignored for resources deployed as dependencies + - [min-suppress.Rule.yaml](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/psrule/.ps-rule/min-suppress.Rule.yaml): Lists rules to be ignored for resources deployed by the min tests + - [na-suppress.Rule.yaml](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/psrule/.ps-rule/na-suppress.Rule.yaml): Lists rules to be ignored for resources not supporting Tags + - [storage-firewall-suppress.Rule.yaml](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/psrule/.ps-rule/storage-firewall-suppress.Rule.yaml): Surpress Rules for resources, that can't activate firewalls + +### Output + +To better outline failed rules and allow fixing incompliant resources quickly, the pipeline leverages the script [utilities\pipelines\PSRulestaticValidation\psrule\Set-PSRuleGitHubOutput.ps1](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/psrule/Set-PSRuleGitHubOutput.ps1) to aggregate PSRule output into Custom Markdown content and display it to the Actions run summary page. + +![PSRule Summary](/Azure-Verified-Modules/images/bicep-ci/psrule-summary.png) diff --git a/docs/content/contributing/bicep/ci-environment/old-ci-environment.md b/docs/content/contributing/bicep/ci-environment/old-ci-environment.md new file mode 100644 index 000000000..2332958ef --- /dev/null +++ b/docs/content/contributing/bicep/ci-environment/old-ci-environment.md @@ -0,0 +1,49 @@ +--- +title: CI Environment +--- + +This page provides an overview of the standard development-to-deployment flow that goes from source modules to target solutions. + +![Deployment Flow](/Azure-Verified-Modules/images/bicep-ci/deployment_flow.png) + +This flow generally covers 3 phases: + +1. In the Develop modules phase modules are first implemented/updated and then validated using one or multiple module test files, testing their successful deployment to a sandbox subscription to prove their correctness. +2. The next phase, Publish modules, packages and publishes the tested and approved modules to a target location for later consumption. The target location (also known as package store or artifact store) should support versioning to allow referencing a specific module version and to avoid breaking changes when referencing them. +3. In the final Consume modules phase, published modules are referenced and combined to deploy more complex architectures (multi-module solutions) such as workloads/applications or individual services. + +### Module Versioning + +Deploying resources by referencing their corresponding modules from source control has one major drawback: If your deployments directly rely on your source repository, then they will by default use the latest code. +Applying software development lifecycle concepts like publishing build artifacts and versioning enables you to have a point in time version of a module. By introducing versions to your modules, the consuming orchestration can and should specify a module version needed, and deploy the Azure solution leveraging it. +In case a breaking change is introduced to a module and an updated version is published, no deployments are affected as they still reference the previously published version. Instead, a deliberate decision must be made to upgrade the solution to reference newer module versions. +Also, if you reference a module version that was tested in and has passed through the CI environment, you can trust that it complies the qualitative and functional standards. + +## Where does the AVM CI environment fit in? + +To ensure the modules hosted by the AVM library are valid and can perform the intended deployments, the repository comes with a continuous integration (CI) environment for each module. If the validation is successful, the CI environment is also publishing versioned modules to one or multiple target locations, from where they can be referenced by solutions consuming them. +The CI environment covers Phase #1, (the validation) & Phase #2 (the publishing) of the [deployment flow](/Azure-Verified-Modules/contributing/bicep/ci-environment/deployment-flow/) section - these include the steps typically performed by the module developer persona. +The AVM solution developer and solution consumer personas are usually working in Phase #2 and Phase #3, i.e., building/leveraging complex, multi-module solutions by consuming the already tested and versioned modules previously placed in an artifact store. +The below diagram shows the details of how the different phases are interconnected: + +![Deployment Flow](/Azure-Verified-Modules/images/bicep-ci/deployment_flow_detail_white.png) + +The top row represents your orchestration environment, for example, GitHub or Azure DevOps. The bottom row represents your Azure cloud environment. +From left to right, there are the three phases introduced before, Develop modules, Publish modules & Consume modules. The diagram shows how each phase interacts with the Azure environment. + +1. Starting with **Develop modules**, the top left box shows the test pipelines that exist for each module, performing the following steps: + + - *Static validation*: Pester & PSRule tests are run on each module to ensure a baseline code quality across the library. + - *Deployment validation*: An actual Azure deployment is performed in a validation/sandbox subscription, shown in the bottom left corner. The subscription is intended to be without any link to production. Resources deployed here should be considered temporary and be removed after testing. + - *Publishing*: Runs only if the previous steps are successful and initiates the second phase as described below. + +2. The **Publish modules** phase is shown in the center box of the diagram. If all tests for a module succeed, the module is published to a given target location. Currently, the target locations supported by the AVM CI environment are: + + - [Template specs](https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/template-specs?tabs=azure-powershell) + - [Private Bicep registry](https://learn.microsoft.com/en-gb/azure/azure-resource-manager/bicep/private-module-registry) + - [Azure DevOps Universal Packages.](https://learn.microsoft.com/en-us/azure/devops/artifacts/concepts/feeds?view=azure-devops) + *Note: this is only available if using Azure DevOps pipelines.* + + To dive deeper and understand which target locations may be best suited for your use case, we provide further information in the Publish-location considerations section. + +3. The third phase, **Consume modules** is represented on the right. The top right corner provides examples of orchestrations deploying the target solutions by referencing the published modules. The deployments performed in this third phase are supposed to target an integration/production environment. This phase references the validated and published modules coming out of the AVM CI environment, and leverages them with the correctly configured parameters to orchestrate their deployment in the intended order. diff --git a/docs/content/contributing/bicep/ci-environment/pipeline-design.md b/docs/content/contributing/bicep/ci-environment/pipeline-design.md new file mode 100644 index 000000000..24e3a2900 --- /dev/null +++ b/docs/content/contributing/bicep/ci-environment/pipeline-design.md @@ -0,0 +1,131 @@ +--- +title: CI environment - Pipeline design +--- + +## Module Pipelines + +The repository hosts one pipeline for each module in the AVM library (see [Contribution Guide - Implement your contribution]({{% siteparam base %}}/contributing/bicep/bicep-contribution-flow/#4-implement-your-contribution)). + +The purpose of each module pipeline is twofold: + +1. **Validation**: To ensure the modules hosted by the AVM library are valid and can perform the intended deployments. +2. **Publishing**: To publish versioned and already validated modules to one or multiple target locations, from where they can be referenced by solutions consuming them. + +As such, each pipeline can be mapped to Phases 1 and 2 described in the Deployment flow section. + +![Pipeline Phases]({{% siteparam base %}}/images/bicep-ci/pipeline-design.png?width=400px) + +The following paragraphs provide an overview of the different phases and shared logic the module pipelines use. + +## Pipeline phases + +This paragraph provides an overview of the three phases performed by each module pipeline. Further details about the implementation and design of each phase are provided on the dedicated pages linked below. + +1. **Static Validation**: Runs a set of static [Pester (test and mock framework for PowerShell)](https://pester.dev/) tests on the module and its templates to ensure they comply with the AVM design principles. Further details for this phase are provided on the corresponding wiki page - see the [Static validation]({{% siteparam base %}}/contributing/bicep/ci-environment/static-validation) section. +1. **Deployment Validation**: An actual Azure deployment is run in a sandbox subscription leveraging a predefined set of module test files, each validating a different configuration of the same Azure resource in parallel. The test suite is cleaned up by default, removing all test resources post-deployment. Further details for this phase are provided on the corresponding wiki page - see the [Deployment validation]({{% siteparam base %}}/contributing/bicep/ci-environment/deployment-validation) section. +1. **Publishing**: Runs only if the previous steps are successful. A new module version is published to all configured target locations such as template specs, private Bicep registry and Azure DevOps Universal Packages. Published module versions can then be referenced by solutions using them. Further details for this phase are provided on the corresponding wiki page - see the [Publishing]({{% siteparam base %}}/contributing/bicep/ci-environment/publishing) page. + +![Pipeline Design Phases]({{% siteparam base %}}/images/bicep-ci/pipeline-design-phases.png?width=400px) + +### GitHub-specific design + +![Pipeline Phases GitHub]({{% siteparam base %}}/images/bicep-ci/pipeline-phases-github.png?width=400px) + +GitHub workflows map each pipeline phase to a dedicated composite action, to maximize code reusability. +The mapping to the specific composite action is provided below: + +| Composite Action | Pipeline phase | +| - | - | +| **Initialize pipeline** | Prepare the pipeline by (for example) publishing pipeline variables, collecting test files, etc. | +| **Module / Static validation** | Static validation | +| **Module / PSRule validation** | PS-Rule validation (with one job per template test file) | +| **Module / Deployment validation** | Deployment validation (with one job per template test file) | +| **publishModule** | Publishing | + +// TODO the next section. +// is it this? + outputs: + workflowInput: ${{ steps.get-workflow-param.outputs.workflowInput }} + moduleTestFilePaths: ${{ steps.get-module-test-file-paths.outputs.moduleTestFilePaths }} + psRuleModuleTestFilePaths: ${{ steps.get-module-test-file-paths.outputs.psRuleModuleTestFilePaths }} + modulePath: "${{ env.modulePath }}" +// end todo +In addition, workflows leverage the following composite actions: + +| Composite Action | Description | +| - | - | +| **getWorkflowInput** | This action allows fetching workflow input values from the module's workflow file, even if the pipeline was not triggered via a `workflow_dispatch` action. Without it, we would not be able to process the contained information and would need to duplicate the configuration as workflow variables. Such input values are for example, the removal switch `removeDeployment`. | +| **setEnvironment** | This action sets up GitHub runners with relevant PowerShell modules required for specific jobs. It then parses the settings file ([`settings.yml`](https://github.com/Azure/ResourceModules/blob/main/settings.yml)) and sets the key-value pairs in the `variables` list as environment variables. | + +Technical documentation for each composite action, such as required input and output variables, is included in each `action.yml` file located in path `.github/actions/templates`. + +## Module pipeline inputs + +Each module pipeline comes with the following runtime parameters: + +- `'Branch'` dropdown: A dropdown to select the branch to run the pipeline from. +- `'Execute static validation'` switch: Can be enabled or disabled. It controls whether the static tests jobs are executed during the pipeline run. + > **Note:** This switch cannot be used to bypass the publishing requirements, where both static tests & deployment validation jobs must be successful. +- `'Execute deployment validation'` switch: Can be enabled or disabled. It controls whether the deployment validation jobs are executed during the pipeline run. +- `'Remove deployed module'` switch: Can be enabled or disabled. It controls whether the test-deployed resources are removed after the deployment validation jobs. It is enabled by default. + > **Note:** This switch cannot be used to bypass the publishing requirements, where both the static tests & deployment validation jobs must be successful. +- `'Default location overwrite'` string: By default, a region is choosen randomly, that the resources are deployed into. This parameter let's you overwrite a random region with one you specify (e.g. eastus). + +![Module Pipeline Input]({{% siteparam base %}}/images/bicep-ci/module-pipeline-input.png?width=400px) + +--- + +## Platform pipelines + +In addition to module pipelines, the repository includes several platform pipelines covering further tasks as described below. + +- [PSRule Pre-Flight validation pipeline](#psrule-pre-flight-validation-pipeline) +- [Deployment history cleanup]({{% siteparam base %}}/contributing/bicep/ci-environment/deployment-history-cleanup) +// TODO add missing pipelines + +## PSRule Pre-Flight validation pipeline + +The purpose of the PSRule Pre-Flight validation pipeline is to validate Azure resources deployed by module validation pipeline tests, by leveraging [PSRule for Azure](https://azure.github.io/PSRule.Rules.Azure/about/). +PSRule for Azure is aligned to the [Well-Architected Framework (WAF)](https://learn.microsoft.com/en-us/azure/architecture/framework/). Tests, called _Rules_, check the configuration of Azure resources against WAF principles. + +The pipeline, currently only available as a [GitHub workflow](https://github.com/Azure/bicep-registry-modules/blob/main/.github/workflows/platform.check.psrule.yml), runs weekly on the whole library, providing as output the list of non-compliant resources and corresponding failing rules, if any. + +### Configuration settings + +PSRule options set for the AVM repository are configured in the [ps-rule.yaml](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/psrule/ps-rule.yaml) file. + +Documentation for all configuration options is available at the following links: + +- [https://aka.ms/ps-rule/options](https://aka.ms/ps-rule/options) +- [https://aka.ms/ps-rule-azure/options](https://aka.ms/ps-rule-azure/options) + +### Baselines + +A [baseline](https://azure.github.io/PSRule.Rules.Azure/working-with-baselines/) is a standard PSRule artifact that combines rules and configuration. The PSRule Pre-Flight validation pipeline uses the default baseline to analyze module test resources. + +For the list of all rules included see [Azure.Default baseline](https://azure.github.io/PSRule.Rules.Azure/en/baselines/Azure.Default/). +To view a list of rules by Azure resources see [Rules by resource](https://azure.github.io/PSRule.Rules.Azure/en/rules/resource/). + +### Exclusions and suppression rules + +Not all baseline rules may be valid for some of the test Azure resources deployed by the module validation pipelines. + +For example, resources deployed by the min tests, aim to validate only the required input parameters for each module. +Therefore, optional features such as diagnostic settings are not configured in those tests. Since enabling logging is a general recommendation for most of the resources supporting them, missing diagnostic settings usually trigger incopliance of PSRule checks, e.g., [Azure.KeyVault.Logs](https://azure.github.io/PSRule.Rules.Azure/en/rules/Azure.KeyVault.Logs/). For this reason, these checks are excluded from being evaluated for resources deployed by min tests. + +PSRule allows skipping rules on two levels: + +- **Exclusions**: Can be leveraged to exclude specific baseline rules from being evaluated for any resource. + - [ps-rule.yaml](https://github.com/Azure/ResourceModules/blob/main/ps-rule.yaml): Lists the name of specific rules to exclude under the option [Rule.Exclude](https://microsoft.github.io/PSRule/v2/concepts/PSRule/en-US/about_PSRule_Options/#ruleexclude) +- **Suppression Groups**: PSRule can use [Suppression Groups](https://microsoft.github.io/PSRule/v2/concepts/PSRule/en-US/about_PSRule_SuppressionGroups/) to suppress rules based on a condition. Suppression groups can be leveraged when some of the rules in the baseline are not relevant under specific conditions, e.g., only for specific resources. They are stored in the `.ps-rule` repo folder in `.yaml` format. In particular: + - [cb-waf-security.Rule.yaml](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/psrule/.ps-rule/cb-waf-security.Rule.yaml): Defines the module-list for the WAF Security recommendations + - [dep-suppress.Rule.yaml](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/psrule/.ps-rule/dep-suppress.Rule.yaml): Lists rules to be ignored for resources deployed as dependencies + - [min-suppress.Rule.yaml](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/psrule/.ps-rule/min-suppress.Rule.yaml): Lists rules to be ignored for resources deployed by the min tests + - [na-suppress.Rule.yaml](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/psrule/.ps-rule/na-suppress.Rule.yaml): Lists rules to be ignored for resources not supporting Tags + - [storage-firewall-suppress.Rule.yaml](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/psrule/.ps-rule/storage-firewall-suppress.Rule.yaml): Surpress Rules for resources, that can't activate firewalls + +### Output + +To better outline failed rules and allow fixing incompliant resources quickly, the pipeline leverages the script [utilities\pipelines\PSRulestaticValidation\psrule\Set-PSRuleGitHubOutput.ps1](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/psrule/Set-PSRuleGitHubOutput.ps1) to aggregate PSRule output into Custom Markdown content and display it to the Actions run summary page. + +![PSRule Summary]({{% siteparam base %}}/images/bicep-ci/psrule-summary.png?width=400px) diff --git a/docs/content/contributing/bicep/ci-environment/pipeline-usage.md b/docs/content/contributing/bicep/ci-environment/pipeline-usage.md new file mode 100644 index 000000000..816ef1932 --- /dev/null +++ b/docs/content/contributing/bicep/ci-environment/pipeline-usage.md @@ -0,0 +1,47 @@ +--- +title: CI environment - Pipeline usage +--- + +This section provides a guideline on how to use the AVM CI environment pipelines. + +--- + +## Operate the module pipelines + +To validate updates to a module template, you can perform the following steps: + +1. (Optionally) Update the module's test files to reflect your changes. +1. Push the local changes to the repository (using a branch that is not `main|master`). +1. Select the branch with your updated template. +1. (Optionally) disable the `Remove deployed module` input parameter in case you don't want to apply the default behavior and want to skip the deletion of the test-deployed resources to check them post-deployment. +1. Trigger the pipeline. + +Once the pipeline concludes, it will either be in a green (success) or red (failed) state, depending on how the module performed. +Pipeline logs are available for troubleshooting and provide detailed information in case of failures. If errors occur in the [Static validation]({{% siteparam base %}}/contributing/bicep/ci-environment/static-validation/) phase, you may only see the failed test and need to `expand` the error message. + +## Add a new module pipeline + +To add a new module pipeline, we recommend to create a copy of a currently existing module pipeline and adjust all module-specific properties, e.g., triggers and module paths. The registration of the pipeline depends on the [DevOps platform](#devops-tool-specific-guidance) you're using. + +## GitHub workflows + +This section focuses on _GitHub_ Actions & Workflows. + +### Trigger a workflow + +To trigger a workflow in _GitHub_: + +1. Navigate to the 'Actions' tab in your repository. + + ![Actions tab]({{% siteparam base %}}/images/bicep-ci/gh-actions-tab.png?width=400px) + +1. Select the pipeline of your choice from the list on the left, followed by 'Run workflow' to the right. You can then select the branch of your choice and trigger the pipeline by clicking on the green 'Run workflow' button. + + ![Run workflow]({{% siteparam base %}}/images/bicep-ci/gh-trigger-pipeline.png?width=400px) + +>**Note**: Depending on the pipeline you selected you may have additional input parameters you can provide aside from the branch. An outline can be found in the [Module pipeline inputs](./The%20CI%20environment%20-%20Pipeline%20design#module-pipeline-inputs) section. + +### Register a workflow + +To register a workflow in _GitHub_ you have to create the workflow file (`.yml`) and store it inside the `.github/workflows` folder. +> ***Note:*** Once merged to `main|master`, GitHub will automatically list the new workflow in the 'Actions' tab. Workflows are not registered from a branch unless you specify a temporal push trigger targeting your branch. diff --git a/docs/content/contributing/bicep/ci-environment/publishing.md b/docs/content/contributing/bicep/ci-environment/publishing.md new file mode 100644 index 000000000..132b6b423 --- /dev/null +++ b/docs/content/contributing/bicep/ci-environment/publishing.md @@ -0,0 +1,129 @@ +--- +title: CI environment - Publishing +--- + +This section provides an overview of the principles the publishing is built upon, how it is set up, and how you can interact with it. + +![Publishing Step]({{% siteparam base %}}/images/bicep-ci/publishing-step.png?width=500px) + +## Publishing overview + +The publishing phase concludes each module's pipeline. If all previous tests succeed (i.e., no phase failed) and the pipeline is run in the `main` branch, a new module version is published to all configured target locations. Currently, we support the following target locations: + +- _[Template Specs](https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/template-specs?tabs=azure-powershell)_ +- _[Bicep Registry](https://learn.microsoft.com/en-gb/azure/azure-resource-manager/bicep/private-module-registry)_ + +> **Note**: The `version` used for publishing any artifact is the same for all three target locations, which reduces the maintenance effort. + +## Module identifiers + +The names of published modules differ slightly depending on the location they are published to & settings you may have configured. This is rooted in the different requirements per target location. + +{{% tabs title="Rules applied" groupid="rulesapplied" %}} + {{% tab title="Template Specs" %}} + +### Actions + +The final name is different based on the `useApiSpecsAlignedName` setting in the `settings.yml` file. The main difference is, that based on the module's path, the actual resource type identifier will be fetched from the `utilities/src/apiSpecsList.json` file. Following you can find the general flow and distinct differences in either case: + +1. (if `useApiSpecsAlignedName` is `true`) + 1. Recover the original API Specs reference for the module from the `utilities/src/apiSpecsList.json` file + 1. Replace `microsoft` with `ms` +1. Remove the root folder name `modules` from provided module reference +1. Make lowercase +1. Replace all `\` or `/` with `.` +1. Remove all duplications in the path. For example, in an identifier like `virtualNetworks/virtualNetworkPeerings` we'd trim down to `virtualNetworks/peerings` to shorten the path (as there is a maximum character limit for template specs) + +### Examples + +| Module Path | useApiSpecsAlignedName | Result | +| - | - | - | +| `/recovery-services/vault` | `true` | `ms.recoveryservices.vaults` | +| | `false` | `recovery-services.vault` | +| `/recovery-services/vault/replication-fabric/replication-protection-container/replication-protection-container-mapping` | `true` | `ms.recoveryservices.vaults.replicationfabrics.replicationprotectioncontainers.mappings` | +| | `false` | `recovery-services.vault.replication-fabric.replication-protection-container.mapping` | + + {{% /tab %}} + {{% tab title="Private Bicep Registry (Azure Container Registry)" %}} + +### Actions + +The final name is different based on the `useApiSpecsAlignedName` setting in the `settings.yml` file. The main difference is, that based on the module's path, the actual resource type identifier will be fetched from the `utilities/src/apiSpecsList.json` file. Following you can find the general flow and distinct differences in either case: + +1. (if `useApiSpecsAlignedName` is `true`) + 1. Recover the original API Specs reference for the module from the `utilities/src/apiSpecsList.json` file +1. Remove the root folder name `modules` from provided module reference +1. Make lowercase +1. Replace all `\` or `/` with `.` +1. Add the `bicep/modules` prefix + +### Examples + +| Module Path | useApiSpecsAlignedName | Result | +| - | - | - | +| `/recovery-services/vault` | `true` | `bicep/modules/microsoft.recoveryservices.vaults` | +| | `false` | `bicep/modules/recovery-services.vault` | +| `/recovery-services/vault/replication-fabric/replication-protection-container/replication-protection-container-mapping` | `true` | `bicep/modules/microsoft.recoveryservices.vaults.replicationfabrics.replicationprotectioncontainers.replicationprotectioncontainermappings` | +| | `false` | `bicep/modules/recovery-services.vault.replication-fabric.replication-protection-container.replication-protection-container-mapping` | + + {{% /tab %}} +{{% /tabs %}} + +## How it works + +The publishing works as follows: + +1. The script [`utilities/pipelines/resourcePublish/Get-ModulesToPublish.ps1`](https://github.com/Azure/ResourceModules/blob/main/utilities/pipelines/resourcePublish/Get-ModulesToPublish.ps1) gets all changed module files, including child modules, and handles the logic of propagating the appropriate module version to be used: + 1. The major (`x.0`) and minor (`0.x`) version are set based on the `version.json` file in the module folder. + 1. The patch (`0.0.x`) version is calculated based on the number of commits on the `HEAD` ref (aka. git height). This will cause the patch version to never reset to 0 with major and/or minor increment, as specified for [semver](https://semver.org/). + 1. The module is published with a `major.minor.patch` version (`x.y.z`). For Template Specs and Bicep Registry only, a `major` version (`x`), a `major.minor` version (`x.y`) and a `latest` version are also updated, allowing a consumer to: + - Reference the latest version of a major, i.e., the latest minor and patch of a major version. + > Example: Using Template Specs, the reference to a `major` could look like: `ts/modules:resources.resource-group:1` which means that the template will always consume whatever the potentially overwritten/updated version `1` contains. + - Reference the latest version of a minor, i.e., the latest patch of a minor version. + > Example: Using the Bicep registry, the reference to a `major.minor` could look like: `br/modules:resources.resource-group:0.4` which means that the template will always consume whatever the potentially overwritten/updated version `0.4` contains. + 1. For a changed child module, the direct parent hierarchy is also registered for an update, following the same procedure as above. + 1. The list of module files paths and their versions are passed on as a array list. +1. The [Get-ModulesMissingFrom*.ps1](https://github.com/Azure/ResourceModules/tree/main/utilities/pipelines/resourcePublish) scripts further check if a given module is missing from the corresponding target location (e.g., Azure Container Registry) and adds each missing entry to to aforementioned array - using the version specified in the module's `version.json` file. +1. The different publishing scripts run (Artifact, Template Spec or Bicep Registry) and publish the module to the respective target location for each item on the list. + +### Example scenario + +Let's look at an example run where we would do a patch change on the `fileShares` module: + +1. A new branch is created for further development of the `fileShare` module. Let's assume the new branch started from commit `500` on the default branch and the `version.json` of the `fileShare` module contains major and minor `0.3`. +1. Bug-fixes, documentation, and security updates are added to the `fileShare` module by the author. The `version.json` file is not changed in either the child or parent module folders. +1. The author runs a manual workflow based on their development branch, with the 'Publish prerelease module' option enabled. +1. A prerelease run of publishing triggers after test and validation of the module. + - For the child and parent modules, the module version's major and minor version is read from the `version.json` file in the module folder respectively. Being unchanged, it still contains the version `0.3`. + - The patch is calculated based on the total number of commits in history on the branch (independent on the module). The new branch started from commit `500` on the default branch and 1 commit has been pushed, so the total number of commits on the new branch is `501`. + - As the pipeline is not running based on the 'default branch', a prerelease segment (`-prerelease`) is added to the version. + - The version results in being `0.3.501-prerelease`. The child and parent modules may have different major and minor versions, but the patch version will be the same in this case. Other unmodified child modules of `storageAccount` will not be republished and remain with the existing version. +1. Sequential commits on the branch and runs of the module pipeline, with the 'publish prerelease' option enabled results in the following versions being published: + - `0.3.502-prerelease` + - `0.3.503-prerelease` + - ... + - `0.3.506-prerelease` +1. When the branch is merged to the default branch, the only thing that changes is the patch version and the removal of the `-prerelease` segment. + - The number of commits will at this point be calculated based on the number of commits on the default branch. + - Assuming the development branch started from commit 500 on the default branch, and the author added 6 commits on the development branch, the prerelease versions will reach `0.3.506-prerelease`. + - Meanwhile, there can be changes (let's say 2 squashed PR merges) on the default branch that is pushing its number of commits in history further. + - If the PR for the changes to `fileShare` is squash merged as commit number 503, the patch version on the child and parent module is then `503`, resulting in a version `0.3.503` being published. +1. The merge triggers cascading updates in the following way: + - The module is published with a `major.minor.patch` version. In addition, only for Template Specs and Bicep Registry, the module is also published with `major.minor` and `major` version updates, allowing consumers to target the latest major or latest minor version respectively. + - All parent module are published following the steps mentioned above. + +``` text + \ \ +C499 -> C500 ---> C501 ---> C502 ---> C503 (503) + \ / + D1 --> D2 --> D3 ... --> D6 + (501) (502) (503) (506) +``` + +`Cx` - Commits on main, +`Dx` - Commits on development branch, +`(x)` - Calculated patch version + +### Output example + +![Publishing Output]({{% siteparam base %}}/images/bicep-ci/publishing-output.png) diff --git a/docs/content/contributing/bicep/ci-environment/static-validation.md b/docs/content/contributing/bicep/ci-environment/static-validation.md new file mode 100644 index 000000000..068148ec3 --- /dev/null +++ b/docs/content/contributing/bicep/ci-environment/static-validation.md @@ -0,0 +1,157 @@ +--- +title: CI environment - Static validation +--- + +This section provides an overview of the principles the static validation is built upon, how it is set up, and how you can interact with it. + +![Static Validation Step]({{% siteparam base %}}/images/bicep-ci/static-validation-step.png?width=400px) + +## Static code validation + +All module Unit tests are performed with the help of [Pester](https://github.com/pester/Pester) to ensure that the modules are configured correctly, documentation is up to date and modules don't turn stale. + +### Outline + +The following activities are performed by the [`utilities/pipelines/staticValidation/compliance/module.tests.ps1`](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/compliance/module.tests.ps1) script. + +- **File/folder tests** + - **General module folder tests** + 1. Module should contain a [` main.bicep `] file. + 1. Module should contain a [` main.json `] file. + 1. Module should contain a [` README.md `] file. + 1. Module should contain a [` CHANGELOG.md `] file. + 1. Main module version should be increased in the [` version.json `] file, if the child version number has been increased. + 1. Module should contain a [` ORPHANED.md `] file only, if the module is orphaned. + 1. Module should contain a [` version.json `] file. + 1. Module should contain a [` tests `] folder. + 1. Module should contain a [` tests/e2e `] folder. + 1. Module should contain a [` tests/e2e/waf-aligned `] folder. + 1. Module should contain a [` tests/e2e/defaults `] folder. + 1. Module should contain a [` main.test.bicep `] file in each e2e test folder. +- **Pipeline tests** + 1. Module should have a GitHub workflow [` .github/workflows/ `]. + 1. Module workflow should have [` workflowPath `] environment variable with the value [` .github/workflows/ `]. +- **Module tests** + - **Readme content tests** + 1. `Set-ModuleReadMe` script should not apply any updates. + - **Compiled ARM template tests** + 1. Compiled ARM template should be based on the current Bicep template. + - **General template tests** + 1. The template file should not be empty. + 1. Template schema version should be the latest. + 1. Template schema should use HTTPS reference. + 1. The template file should contain required elements [schema], [contentVersion], [resources]. + 1. The template file should have a module name specified. + 1. The template file should have a module description specified. + - **Parameters template tests** + 1. The Location should be defined as a parameter, with the default value of 'resourceGroup().location' or global' for ResourceGroup deployment scope. + 1. The telemetry parameter should be present and valid. + 1. Parameter & UDT names should be camel-cased (no dashes or underscores and must start with lower-case letter). + 1. Each parameters' & UDT's description should be well formatted. + 1. Conditional parameters' & UDT's description should contain 'Required if' followed by a definition. + 1. Non-required parameters' and UDT's description should not start with 'Required'. + 1. Required parameters' and UDT's description should start with 'Required|Conditional'. + - **UDT Parameters template tests** + 1. Parameters should implement AVM's corresponding user-defined type. + 1. If a UDT definition [managedIdentitiesType] exists and the module supports system-assigned-identities, an output for its principal ID should exist. + 1. A parameter [tags] should be 'nullable'. + - **Variables template tests** + 1. Variable names should be camel-cased (no dashes or underscores and must start with lower-case letter). + - **Resources template tests** + 1. Telemetry deployment should be present in the template. + 1. Telemetry deployment should have correct condition in the template. + 1. Telemetry deployment should have expected inner output for verbosity. + 1. Telemetry deployment should have expected telemetry identifier. + - **Output template tests** + 1. Output names should be camel-cased (no dashes or underscores and must start with lower-case letter). + 1. Output names description should be well formatted. + 1. Location output should be returned for resources that use it. + 1. Resource Group output should exist for resources that are deployed into a resource group scope. + 1. Resource name output should exist. + 1. Resource ID output should exist. + 1. Resource modules Principal ID output should exist, if supported. + - [**Changelog content tests**]({{% siteparam base %}}/contributing/bicep/ci-environment/changelog-automation) + 1. The changelog should not be empty. + 1. The changelog shoud start with '# Changelog'. + 1. The changelog shoud start with '# Changelog', followed by an empty line. + 1. The '## unreleased' section should be in the changelog. + 1. The '## unreleased' section should be in the changelog only once. + 1. The '## unreleased' section should be the first section in the changelog. + 1. The versions in the changelog should appear in descending order. + 1. The '## unreleased' section should contain "New Features" surrounded by empty lines. + 1. The '## unreleased' section should contain "Changes" surrounded by empty lines. + 1. The '## unreleased' section should contain "Bugfixes" surrounded by empty lines. + 1. The '## unreleased' section should contain "Breaking Changes" surrounded by empty lines. + 1. The '## unreleased' section should contain actual content and not just headers or empty lines. +- **Governance tests** + 1. Owning team should be specified correctly in CODEWONERS file. + 1. Module identifier should be listed in issue template in the correct alphabetical position. +- **Test file tests** + - **General test file** + 1. Bicep test deployment should have parameter [serviceShort]. + 1. Bicep test deployment files in a [defaults] folder should have a parameter [serviceShort] with a value ending with 'min'. + 1. Bicep test deployment files in a [max] folder should have a parameter [serviceShort] with a value ending with 'max'. + 1. Bicep test deployment files in a [waf-aligned] folder should have a parameter [serviceShort] with a value ending with 'waf'. + 1. Bicep test deployment files should contain a metadata string [name]. + 1. Bicep test deployment files should contain a metadata string [description] . + 1. Bicep test deployment files should contain a parameter [namePrefix] with value '#_namePrefix_#'. + 1. Bicep test deployment files should invoke test like [`module testDeployment '../.*main.bicep' = [ or {`] . + 1. Bicep test deployment name should contain [`-test-`]. +- **API version tests** + 1. In used resource type should use one of the recent API version(s). + +### Output example + +Every test creates output, that is written to the workflows log. *Yellow* lines are either verbose or warning messages, whereas *red* lines are actuall errors, which mark the whole test as faulty and stopps the CI pipeline. + +![Static Validation Output]({{% siteparam base %}}/images/bicep-ci/static-validation-output.png?width=400px) + +## API version validation + +In this phase, Pester analyzes the API version of each resource type deployed by the module. + +In particular, each resource's API version is compared with those currently available on Azure. This test has a certain level of tolerance (does not enforce the latest version): the API version in use should be one of the 5 latest versions available (including preview versions) or one of the the 5 latest non-preview versions. + +This test also leverages the [`utilities/pipelines/staticValidation/compliance/module.tests.ps1`](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/staticValidation/compliance/module.tests.ps1) script. + +To test the API versions, the test leverages the file `governance/apiSpecsList.json` file as a reference for all API versions available for any used Provider Namespace & Resource Type. + +> **NOTE:** If this file does not exist, the API tests will be skipped. + +> **NOTE:** This functionality has a dependency on the [AzureAPICrawler](https://www.powershellgallery.com/packages/AzureAPICrawler) PowerShell module. + +The [Set-ModuleReadMe.ps1](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/pipelines/sharedScripts/Set-ModuleReadMe.ps1) will read the file and check the referenced APIs in the module against that list. + +## Verify the static validation of your module locally + +This paragraph is intended for AVM contributors or more generally for those leveraging the AVM CI environment and having the need to update or add a new module to the library. + +Refer to the below snippet to leverage the 'Test-ModuleLocally.ps1' script and verify if your module will comply to the static validation before pushing to source control. + +```powershell +#########[ Function Test-ModulesLocally.ps1 ]############# +$pathToRepository = '' +. "$pathToRepository\utilities\tools\Test-ModuleLocally.ps1" + +# REQUIRED INPUT FOR TESTING +$TestModuleLocallyInput = @{ + templateFilePath = "$pathToRepository\modules\authorization\role-definition\main.bicep" + PesterTest = $true + PesterTestRecurse = $true + ValidationTest = $false + DeploymentTest = $false + ValidateOrDeployParameters = @{ + Location = 'germanywestcentral' + SubscriptionId = '00000000-0000-0000-0000-000000000000' + RemoveDeployment = $true + } + AdditionalTokens = @{ + namePrefix = 'abcd' + TenantId = '00000000-0000-0000-0000-000000000000' + } +} + +Test-ModuleLocally @TestModuleLocallyInput -Verbose +``` + +> You can use the `Get-Help` cmdlet to show more options on how you can use this script. diff --git a/docs/content/contributing/bicep/ci-environment/token-replacement.md b/docs/content/contributing/bicep/ci-environment/token-replacement.md new file mode 100644 index 000000000..7dbd0e329 --- /dev/null +++ b/docs/content/contributing/bicep/ci-environment/token-replacement.md @@ -0,0 +1,98 @@ +--- +title: CI environment - Token replacement +--- + +This section provides details on the tokens replacement functionality that enables the use of tokens inside module test files instead of plain text strings. + +--- + +## Description + +Tokens allow you to test deploying modules in your own environment (i.e., using tokens for your naming conventions), or apply other customizations to your resources (i.e., injecting a subscription ID inside a Resource ID string). + +The [module pipelines]({{% siteparam base %}}/contributing/bicep/ci-environment/pipeline-design/#module-pipelines) leverage a token replacement function that enables module test files to contain tokens (i.e., `[[subscriptionId]]`, `[[tenantId]]`) instead of using static values. This helps with the following: + +- Allows the repository to be portable without having static values from where it was cloned. +- Enables dynamic updates of the tokens from single locations without having to modify all files. +- Not adding more environment variables to workflows/pipelines whenever new tokens are required for the environment. + +## Token Types + +There are 2 types of tokens that can be applied on a module test file: + +### Default Tokens + +These are tokens constructed from environment variables, which are defined in the workflow (Pipeline). Review [Configure your CI environment](/contributing/bicep/bicep-contribution-flow/#3-configure-your-ci-environment) for more information on these environment variables. + +- `[[ARM_SUBSCRIPTION_ID]]`: Will point to the Azure subscription. +- `[[ARM_MGMTGROUP_ID]]`: Will point to the Azure an Azure Management Group. +- `[[ARM_TENANT_ID]]`: Will point to the Azure Tenant ID. +- `[[TOKEN_NAMEPREFIX]]`: Short, unique prefix for Azure resource names. + +### (Optional) Local Custom Tokens + +This allows creating tokens that are local and updatable via Source Control mechanisms. Here is an example on where these tokens are stored. You can add key-value pairs as required: + +```yml +localToken_tokenA: 'foo' +localToken_tokenB: 'bar' +``` + +> **Note:** The CI pipelines automatically removes the `localToken_` prefix from the name when processing the tokens replacement. This means that your actual token name is `tokenA` and NOT `localToken_tokenA`. + +Let's say you'd want to use this token inside a Key Vault module test file, to deploy the Key Vault with a name that contains this token: + +```json +"parameters": { + "name": { + "value": "[[tokenA]]-keyVault-[[tokenB]]" + } +} +``` + +Once the Key Vault is deployed, you'll notice that the Key Vault name in Azure will be `foo-keyVault-bar`. +The solution comes with one predefined local token `namePrefix`. This token is leveraged in most of the parameter & test files for deployments. It allows using a consistent naming prefix that is applied to all resources being tested. There are two ways this token can be set and one will take precedence over the other: + +1. By updating the value of `localToken_namePrefix` it then becomes `namePrefix` when the pipelines run. +1. Creating a GitHub Secret / ADO variable called `TOKEN_NAMEPREFIX`, which then becomes `namePrefix` when the pipelines run. + +When validating modules through the CI environment, you must update it to a custom value as described in the [Special case: TOKEN_NAMEPREFIX]({{% siteparam base %}}/contributing/bicep/bicep-contribution-flow/#311-shared-repository-secrets) paragraph. This is done to avoid conflicts with resources requiring a globally unique name, such as storage accounts or Key Vaults. + +> **Note**: Do not store sensitive information in this location as they will be present in your Git History. Follow best [practices and guidelines](https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/best-practices#security-recommendations-for-parameters) on how to handle secrets in template deployments. + +## How it works + +The below image compares the different token types that can be used for module test file tokens: + +![token types]({{% siteparam base %}}/images/bicep-ci/token-types.png?width=400px) + +### How tokens are replaced in a module test file + +The below diagram illustrates the Token Replacement Functionality via the [validate module deployment](https://github.com/Azure/bicep-registry-modules/blob/main/.github/actions/templates/avm-validateModuleDeployment/action.yml) Action/Template. + +![token replacement]({{% siteparam base %}}/images/bicep-ci/token-replacement.png?width=400px) + +- **1.** The user creates default tokens as [GitHub Secrets](https://docs.github.com/en/actions/security-guides/encrypted-secrets#creating-encrypted-secrets-for-a-repository) or [Azure DevOps Pipeline Variables](https://learn.microsoft.com/en-us/azure/devops/pipelines/library/?view=azure-devops), that are injected as environment variables. +- **2.** The module test files can now be tokenized as per required value. And the token format can look like `[[tokenA]]`. Example: + + ```json + "adminPassword": { + "reference": { + "keyVault": { + "id": "/subscriptions/[[subscriptionId]]/resourceGroups/validation-rg/providers/Microsoft.KeyVault/vaults/[[tokenA]]-keyVault" + }, + "secretName": "[[tokenB]]" + } + } + ``` + +- **3A.** The Replace Tokens function gets the default tokens from the environment variables. + > Default Tokens are harder to scale as they are explicitly defined in deploy/validate task, workflows and pipelines, and these components must be updated as you create more tokens. + +- **3B.** The Replace Tokens function gets the module test file (tokenized and not deployable) and then all tokens are processed for replacement. + +- **3C.** The updated module test file is then saved, replacing the tokenized version. This file is now 'deployable'. + +- **4A.** The Validate/Deploy function retrieves the latest updated module test file. + +- **4B.** The Validate/Deploy function validates the deployment artifacts for the module before deploying it to the Azure Sandbox Subscription. diff --git a/docs/content/contributing/bicep/ci-environment/troubleshooting.md b/docs/content/contributing/bicep/ci-environment/troubleshooting.md new file mode 100644 index 000000000..cb6de36d7 --- /dev/null +++ b/docs/content/contributing/bicep/ci-environment/troubleshooting.md @@ -0,0 +1,63 @@ +--- +title: CI environment - Troubleshooting +--- + +This section provides an outline of technical CI-specific challenges that were raised in the past and some guidance on how to resolve them. + +--- + +## Error reading JToken from JsonReader + +> Related error messages: +> +> - `The template is not valid. .github/workflows/ms.eventgrid.topics.yml (Line: 116, Col: 29): Error reading JToken from JsonReader. Path '', line 0, position 0.` + +Potential reasons + +- The `AZURE_CREDENTIALS` are not configured as a single-line string, but as a multiline JSON. + + - **Solution:** As per the [documentation](https://github.com/Azure/ResourceModules/wiki/Getting%20started%20-%20Scenario%202%20Onboard%20module%20library%20and%20CI%20environment#321-set-up-secrets) please ensure that the secret is created in one line. That means using + + ```json + { "it's a me" } + ``` + + instead of + + ```json + { + "it's a me" + } + ``` + + - **Background:** If configured as a multiline object, GitHub interprets each line as a dedicated secret value. This leads to a situation where characters like `{` & `}` are treated as individual secrets - and values that are passed as JSON strings between pipeline jobs that contain these characters cannot be passed anymore (as per GitHubs default behavior). For example: `{"removeDeployment": true}`. Because this value isn't passed anymore, subsequently the line `removeDeployment: '${{ (fromJson(needs.job_initialize_pipeline.outputs.workflowInput)).removeDeployment }}'` that references it throws the error message you're seeing. + +## Outdated Bicep CLI version + +> Related error messages: +> +> - `Error BCP018: Expected the ":" character at this location.` +> - `Error BCP022: Expected a property name at this location.` +> - `Error BCP236: Expected a new line or comma character at this location.` +> - `...` with the described reason below, there can be any amount of error messages you may encounter + +Potential reasons + +- The Bicep CLI version is outdated. + - **Solution:** Update the version of Bicep you're using on your machine. This can include: + - The Bicep extension for VS Code ([ref](https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/install#visual-studio-and-bicep-extension)) + - The Bicep extension for the Azure CLI ([ref](https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/install#azure-cli)) + - The Bicep CLI ([ref](https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/install#azure-powershell) + - **Background:** When using Bicep on your local device, you usually handle 2 different tools: The 'Bicep' extension for the Azure CLI (identified by commands such as `az bicep version`) and the Bicep CLI itself (identified by commands such as `bicep --version`). The later is the one responsible for compiling Bicep templates into ARM templates. The Bicep extension for Visual Studio Code usually automatically updates the Azure CLI extension. However, the same is not true for the Bicep CLI version. This you have to update yourself. + +## Failed pipelines + +Sometimes pipelines report an error, which occured from timeouts during deployments or other reasons, that are unique to a single execution. The AVM team has created a PowerShell script [Invoke-WorkflowsFailedJobsReRun.ps1](https://github.com/Azure/bicep-registry-modules/blob/main/utilities/tools/Invoke-WorkflowsFailedJobsReRun.ps1) that allows you to re-run all failed jobs on a branch in a repository. + +As always, the documentation is within the script. This is a sample usage, which takes the GitHub token from the login and shows which workflows and jobs would be re-run. + +```bash +. .\utilities\tools\Invoke-WorkflowsFailedJobsReRun.ps1 +gh auth login +Invoke-WorkflowsFailedJobsReRun -TargetBranch main -RepositoryOwner Azure -PipelineFilter 'avm\.(?:res|ptn|utl)' -WhatIf +``` diff --git a/docs/hugo.toml b/docs/hugo.toml index a64d0f47a..dbb0d5620 100644 --- a/docs/hugo.toml +++ b/docs/hugo.toml @@ -9,8 +9,6 @@ enableGitInfo = true [params] base = '/Azure-Verified-Modules' - images = ['images/avm_logo.png'] - logo = 'images/favicon.svg' disableExplicitIndexURLs = true disableLanguageSwitchingButton = false themeVariant = [ @@ -538,6 +536,102 @@ enableGitInfo = true weight = 5 params = { alwaysopen = true, collapsibleMenu = true } + [[menu.defined]] + identifier = 'bicep-ci-environment' + parent = 'bicep-module-contribution' + name = 'CI Environment' + pageRef = '/contributing/bicep/ci-environment' + weight = 4 + params = { alwaysopen = false, collapsibleMenu = true } + + [[menu.defined]] + identifier = 'pipeline-design' + parent = 'bicep-ci-environment' + name = 'Pipeline Design' + pageRef = '/contributing/bicep/ci-environment/pipeline-design' + weight = 1 + params = { alwaysopen = false, collapsibleMenu = true } + + [[menu.defined]] + identifier = 'static-validation' + parent = 'bicep-ci-environment' + name = 'Static Validation' + pageRef = '/contributing/bicep/ci-environment/static-validation' + weight = 2 + params = { alwaysopen = false, collapsibleMenu = true } + + [[menu.defined]] + identifier = 'deployment-flow' + parent = 'bicep-ci-environment' + name = 'Deployment Flow' + pageRef = '/contributing/bicep/ci-environment/deployment-flow' + weight = 3 + params = { alwaysopen = true, collapsibleMenu = true } + + [[menu.defined]] + identifier = 'deployment-validation' + parent = 'deployment-flow' + name = 'Deployment Validation' + pageRef = '/contributing/bicep/ci-environment/deployment-validation' + weight = 1 + params = { alwaysopen = true, collapsibleMenu = true } + + [[menu.defined]] + identifier = 'deployment-history-cleanup' + parent = 'deployment-flow' + name = 'Deployment History Cleanup' + pageRef = '/contributing/bicep/ci-environment/deployment-history-cleanup' + weight = 2 + params = { alwaysopen = true, collapsibleMenu = true } + + [[menu.defined]] + identifier = 'publishing' + parent = 'bicep-ci-environment' + name = 'Publishing' + pageRef = '/contributing/bicep/ci-environment/publishing' + weight = 4 + params = { alwaysopen = false, collapsibleMenu = true } + + [[menu.defined]] + identifier = 'token-replacement' + parent = 'bicep-ci-environment' + name = 'Token Replacement' + pageRef = '/contributing/bicep/ci-environment/token-replacement' + weight = 5 + params = { alwaysopen = false, collapsibleMenu = true } + + [[menu.defined]] + identifier = 'pipeline-usage' + parent = 'bicep-ci-environment' + name = 'Pipeline Usage' + pageRef = '/contributing/bicep/ci-environment/pipeline-usage' + weight = 6 + params = { alwaysopen = false, collapsibleMenu = true } + + [[menu.defined]] + identifier = 'bicep-configuration' + parent = 'bicep-ci-environment' + name = 'Bicep Configuration' + pageRef = '/contributing/bicep/ci-environment/bicep-configuration' + weight = 7 + params = { alwaysopen = false, collapsibleMenu = true } + + [[menu.defined]] + identifier = 'changelog-automation' + parent = 'bicep-ci-environment' + name = 'Module Changelog Automation' + pageRef = '/contributing/bicep/ci-environment/module-changelog-automation' + weight = 8 + params = { alwaysopen = false, collapsibleMenu = true } + + [[menu.defined]] + identifier = 'troubleshooting' + parent = 'bicep-ci-environment' + name = 'Troubleshooting' + pageRef = '/contributing/bicep/ci-environment/troubleshooting' + weight = 9 + params = { alwaysopen = false, collapsibleMenu = true } + [[menu.defined]] identifier = 'terraform-module-contribution' parent = 'contributing' @@ -587,7 +681,7 @@ enableGitInfo = true params = { alwaysopen = false, collapsibleMenu = true } [[menu.defined]] - identifier = 'contirbution-qna' + identifier = 'contribution-qna' parent = 'contributing' name = 'Contribution Q&A' pageRef = '/contributing/q-and-a' diff --git a/docs/static/images/bicep-ci/deployment-validation-output.png b/docs/static/images/bicep-ci/deployment-validation-output.png new file mode 100644 index 000000000..b57d82a8c Binary files /dev/null and b/docs/static/images/bicep-ci/deployment-validation-output.png differ diff --git a/docs/static/images/bicep-ci/deployment-validation-step.png b/docs/static/images/bicep-ci/deployment-validation-step.png new file mode 100644 index 000000000..0185a5785 Binary files /dev/null and b/docs/static/images/bicep-ci/deployment-validation-step.png differ diff --git a/docs/static/images/bicep-ci/deployment_flow.png b/docs/static/images/bicep-ci/deployment_flow.png new file mode 100644 index 000000000..14b219b5c Binary files /dev/null and b/docs/static/images/bicep-ci/deployment_flow.png differ diff --git a/docs/static/images/bicep-ci/deployment_flow_detail_white.png b/docs/static/images/bicep-ci/deployment_flow_detail_white.png new file mode 100644 index 000000000..199fb61eb Binary files /dev/null and b/docs/static/images/bicep-ci/deployment_flow_detail_white.png differ diff --git a/docs/static/images/bicep-ci/gh-actions-tab.png b/docs/static/images/bicep-ci/gh-actions-tab.png new file mode 100644 index 000000000..cce90167c Binary files /dev/null and b/docs/static/images/bicep-ci/gh-actions-tab.png differ diff --git a/docs/static/images/bicep-ci/gh-changelog-commit.png b/docs/static/images/bicep-ci/gh-changelog-commit.png new file mode 100644 index 000000000..453736f65 Binary files /dev/null and b/docs/static/images/bicep-ci/gh-changelog-commit.png differ diff --git a/docs/static/images/bicep-ci/gh-trigger-pipeline.png b/docs/static/images/bicep-ci/gh-trigger-pipeline.png new file mode 100644 index 000000000..6346922f8 Binary files /dev/null and b/docs/static/images/bicep-ci/gh-trigger-pipeline.png differ diff --git a/docs/static/images/bicep-ci/module-pipeline-input.png b/docs/static/images/bicep-ci/module-pipeline-input.png new file mode 100644 index 000000000..7c3fd5dc7 Binary files /dev/null and b/docs/static/images/bicep-ci/module-pipeline-input.png differ diff --git a/docs/static/images/bicep-ci/pipeline-design-phases.png b/docs/static/images/bicep-ci/pipeline-design-phases.png new file mode 100644 index 000000000..2d9bb4e50 Binary files /dev/null and b/docs/static/images/bicep-ci/pipeline-design-phases.png differ diff --git a/docs/static/images/bicep-ci/pipeline-design.png b/docs/static/images/bicep-ci/pipeline-design.png new file mode 100644 index 000000000..0a11243f4 Binary files /dev/null and b/docs/static/images/bicep-ci/pipeline-design.png differ diff --git a/docs/static/images/bicep-ci/pipeline-phases-github.png b/docs/static/images/bicep-ci/pipeline-phases-github.png new file mode 100644 index 000000000..a4f848b8c Binary files /dev/null and b/docs/static/images/bicep-ci/pipeline-phases-github.png differ diff --git a/docs/static/images/bicep-ci/psrule-summary.png b/docs/static/images/bicep-ci/psrule-summary.png new file mode 100644 index 000000000..4c7e6a8e5 Binary files /dev/null and b/docs/static/images/bicep-ci/psrule-summary.png differ diff --git a/docs/static/images/bicep-ci/publishing-output.png b/docs/static/images/bicep-ci/publishing-output.png new file mode 100644 index 000000000..e5d278dbb Binary files /dev/null and b/docs/static/images/bicep-ci/publishing-output.png differ diff --git a/docs/static/images/bicep-ci/publishing-step.png b/docs/static/images/bicep-ci/publishing-step.png new file mode 100644 index 000000000..7f46f0143 Binary files /dev/null and b/docs/static/images/bicep-ci/publishing-step.png differ diff --git a/docs/static/images/bicep-ci/static-validation-output.png b/docs/static/images/bicep-ci/static-validation-output.png new file mode 100644 index 000000000..371569cce Binary files /dev/null and b/docs/static/images/bicep-ci/static-validation-output.png differ diff --git a/docs/static/images/bicep-ci/static-validation-step.png b/docs/static/images/bicep-ci/static-validation-step.png new file mode 100644 index 000000000..f538f0ab2 Binary files /dev/null and b/docs/static/images/bicep-ci/static-validation-step.png differ diff --git a/docs/static/images/bicep-ci/token-replacement.png b/docs/static/images/bicep-ci/token-replacement.png new file mode 100644 index 000000000..caefce25d Binary files /dev/null and b/docs/static/images/bicep-ci/token-replacement.png differ diff --git a/docs/static/images/bicep-ci/token-types.png b/docs/static/images/bicep-ci/token-types.png new file mode 100644 index 000000000..8c2fa8841 Binary files /dev/null and b/docs/static/images/bicep-ci/token-types.png differ