-
Notifications
You must be signed in to change notification settings - Fork 8.1k
[feat] added azure pipelines guide #22777
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 12 commits
a38b41c
c351714
5073388
ec1abc2
8f518ca
e178661
8859428
2bd138b
66ae8f0
b586405
e1431e6
df81e99
64843c6
b98c125
b49137e
6bdeebb
60a7c7d
0a5341b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,308 @@ | ||
| --- | ||
| title: Introduction to Azure Pipelines with Docker | ||
| linkTitle: Azure Pipelines and Docker | ||
| summary: | | ||
| Learn how to automate Docker image build and push using Azure Pipelines. | ||
| params: | ||
| tags: [devops] | ||
| time: 10 minutes | ||
| --- | ||
|
|
||
| > This guide was created by [Kristiyan Velkov](https://www.linkedin.com/in/kristiyan-velkov-763130b3/). | ||
|
Check failure on line 11 in content/guides/azure-pipelines.md
|
||
|
|
||
| ## Prerequisites | ||
|
|
||
| Before you begin, ensure the following requirements are met: | ||
|
|
||
| - A [Docker Hub account](https://hub.docker.com) with a generated access token. | ||
| - An active [Azure DevOps project](https://dev.azure.com/) with a connected [Git repository](https://learn.microsoft.com/en-us/azure/devops/repos/git/?view=azure-devops). | ||
| - A project that includes a valid [`Dockerfile`](https://docs.docker.com/engine/reference/builder/) at its root or appropriate build context. | ||
|
|
||
| ## Overview | ||
|
|
||
| This guide walks you through building and pushing Docker images using [Azure Pipelines](https://azure.microsoft.com/en-us/products/devops/pipelines), enabling a streamlined and secure CI workflow for containerized applications. You’ll learn how to: | ||
|
|
||
| - Configure Docker authentication securely. | ||
| - Set up an automated pipeline to build and push images. | ||
|
|
||
| ## Step 1: Configure a Docker Hub service connection | ||
|
|
||
| To securely authenticate with Docker Hub using Azure Pipelines: | ||
|
|
||
| 1. Navigate to **Project Settings > Service Connections** in your Azure DevOps project. | ||
| 2. Click **New service connection > Docker Registry**. | ||
|
Check warning on line 33 in content/guides/azure-pipelines.md
|
||
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| 3. Choose **Docker Hub** and provide your Docker Hub credentials or access token. | ||
| 4. Give the service connection a recognizable name, such as `my-docker-registry`. | ||
| 5. Grant access only to the specific pipeline(s) that require it for improved security and least privilege. | ||
|
|
||
| > [!IMPORTANT] | ||
| > Avoid selecting the option to grant access to all pipelines unless absolutely necessary. Always apply the principle of least privilege. | ||
kristiyan-velkov marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ## Step 2: Create your pipeline | ||
|
|
||
| Add the following `azure-pipelines.yml` file to the root of your repository: | ||
|
|
||
| ```yaml | ||
| # Trigger pipeline on commits to the main branch | ||
| trigger: | ||
| - main | ||
|
|
||
| # Trigger pipeline on pull requests targeting the main branch | ||
| pr: | ||
| - main | ||
|
|
||
| # Define variables for reuse across the pipeline | ||
| variables: | ||
| imageName: 'docker.io/$(dockerUsername)/my-image' | ||
kristiyan-velkov marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| buildTag: '$(Build.BuildId)' | ||
| latestTag: 'latest' | ||
|
|
||
| stages: | ||
| - stage: BuildAndPush | ||
| displayName: Build and Push Docker Image | ||
| jobs: | ||
| - job: DockerJob | ||
| displayName: Build and Push | ||
| pool: | ||
| vmImage: ubuntu-latest | ||
| demands: | ||
| - docker | ||
kristiyan-velkov marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| steps: | ||
| - checkout: self | ||
| displayName: Checkout Code | ||
|
|
||
| - task: Docker@2 | ||
| displayName: Docker Login | ||
| inputs: | ||
| command: login | ||
| containerRegistry: 'my-docker-registry' # Service connection name | ||
|
|
||
| - task: Docker@2 | ||
| displayName: Build Docker Image | ||
| inputs: | ||
| command: build | ||
| repository: $(imageName) | ||
| tags: | | ||
| $(buildTag) | ||
| $(latestTag) | ||
| dockerfile: './Dockerfile' | ||
| arguments: | | ||
| --sbom=true | ||
| --attest type=provenance | ||
| --cache-from $(imageName):latest | ||
| env: | ||
| DOCKER_BUILDKIT: 1 | ||
|
|
||
| - task: Docker@2 | ||
| displayName: Push Docker Image | ||
| condition: eq(variables['Build.SourceBranch'], 'refs/heads/main') | ||
kristiyan-velkov marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| inputs: | ||
| command: push | ||
| repository: $(imageName) | ||
| tags: | | ||
| $(buildTag) | ||
| $(latestTag) | ||
|
|
||
| # Optional: logout for self-hosted agents | ||
| - script: docker logout | ||
| displayName: Docker Logout (Self-hosted only) | ||
| condition: ne(variables['Agent.OS'], 'Windows_NT') | ||
kristiyan-velkov marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ``` | ||
kristiyan-velkov marked this conversation as resolved.
Show resolved
Hide resolved
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ## What this pipeline does | ||
|
|
||
| This pipeline automates the Docker image build and deployment process for the main branch. It ensures a secure and efficient workflow with best practices like caching, tagging, and conditional cleanup. Here's what it does: | ||
|
|
||
| - Triggers on commits and pull requests targeting the `main` branch. | ||
| - Authenticates securely with Docker Hub using an Azure DevOps service connection. | ||
| - Builds and tags the Docker image using Docker BuildKit for caching. | ||
| - Pushes both buildId and latest tags to Docker Hub. | ||
|
Check failure on line 119 in content/guides/azure-pipelines.md
|
||
| - Logs out from Docker if running on a self-hosted Linux agent. | ||
|
|
||
|
|
||
| ## Detailed Step-by-Step Explanation | ||
|
Check warning on line 123 in content/guides/azure-pipelines.md
|
||
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ### Step 1: Define Pipeline Triggers | ||
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ```yaml | ||
| trigger: | ||
| - main | ||
|
|
||
| pr: | ||
| - main | ||
| ``` | ||
|
|
||
| This pipeline is triggered automatically on: | ||
| - Commits pushed to the `main` branch | ||
| - Pull requests targeting `main` main branch | ||
|
|
||
| > [!TIP] | ||
| > Learn more: [Define pipeline triggers in Azure Pipelines](https://learn.microsoft.com/en-us/azure/devops/pipelines/build/triggers?view=azure-devops) | ||
|
|
||
|
|
||
| ### Step 2: Define Common Variables | ||
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ```yaml | ||
| variables: | ||
| imageName: 'docker.io/$(dockerUsername)/my-image' | ||
| buildTag: '$(Build.BuildId)' | ||
| latestTag: 'latest' | ||
| ``` | ||
|
|
||
| These variables ensure consistent naming, versioning, and reuse throughout the pipeline steps: | ||
|
|
||
| - `imageName`: your image path on Docker Hub | ||
| - `buildTag`: a unique tag for each pipeline run | ||
| - `latestTag`: a stable alias for your most recent image | ||
|
|
||
| > [!TIP] | ||
| > Learn more: [Define and use variables in Azure Pipelines](https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch) | ||
|
|
||
|
|
||
| ### Step 3: Define Pipeline Stages and Jobs | ||
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ```yaml | ||
| stages: | ||
| - stage: BuildAndPush | ||
| displayName: Build and Push Docker Image | ||
| ``` | ||
|
|
||
| This stage executes only if: | ||
|
|
||
| - The source branch is main. | ||
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| > [!TIP] | ||
| > Learn more: [Stage conditions in Azure Pipelines](https://learn.microsoft.com/en-us/azure/devops/pipelines/process/stages?view=azure-devops&tabs=yaml) | ||
kristiyan-velkov marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ### Step 4: Job Configuration | ||
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ```yaml | ||
| jobs: | ||
| - job: DockerJob | ||
| displayName: Build and Push | ||
| pool: | ||
| vmImage: ubuntu-latest | ||
| demands: | ||
| - docker | ||
| ``` | ||
|
|
||
| This job utilizes the latest Ubuntu VM image with Docker support, provided by Microsoft-hosted agents. It can be replaced with a custom pool for self-hosted agents if necessary. | ||
|
|
||
| > [!TIP] | ||
| > Learn more: [Specify a pool in your pipeline](https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/pools-queues?view=azure-devops&tabs=yaml%2Cbrowser) | ||
kristiyan-velkov marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| #### Step 4.1 Checkout Code | ||
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ```yaml | ||
| steps: | ||
| - checkout: self | ||
| displayName: Checkout Code | ||
|
|
||
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| ``` | ||
|
|
||
| This step pulls your repository code into the build agent, so the pipeline can access the Dockerfile and application files. | ||
|
|
||
| > [!TIP] | ||
| > Learn more: [checkout step documentation](https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema/steps-checkout?view=azure-pipelines) | ||
kristiyan-velkov marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
|
|
||
| #### Step 4.2 Authenticate to Docker Hub | ||
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ```yaml | ||
| - task: Docker@2 | ||
| displayName: Docker Login | ||
| inputs: | ||
| command: login | ||
| containerRegistry: 'my-docker-registry' # Replace with your service connection name | ||
| ``` | ||
|
|
||
| Uses a preconfigured Azure DevOps Docker registry service connection to authenticate securely without exposing credentials directly. | ||
|
Check failure on line 219 in content/guides/azure-pipelines.md
|
||
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| > [!TIP] | ||
| > Learn more: [Use service connections for Docker Hub](https://learn.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints?view=azure-devops#docker-hub-or-others) | ||
kristiyan-velkov marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
|
|
||
| #### Step 4.3 Build the Docker Image | ||
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ```yaml | ||
| - task: Docker@2 | ||
| displayName: Build Docker Image | ||
| inputs: | ||
| command: build | ||
| repository: $(imageName) | ||
| tags: | | ||
| $(buildTag) | ||
| $(latestTag) | ||
| dockerfile: './Dockerfile' | ||
| arguments: | | ||
| --sbom=true | ||
| --attest type=provenance | ||
| --cache-from $(imageName):latest | ||
| env: | ||
| DOCKER_BUILDKIT: 1 | ||
| ``` | ||
|
|
||
| This builds the image with: | ||
|
|
||
| - Two tags: one with the unique Build ID and one as latest | ||
| - Docker BuildKit enabled for faster builds and efficient layer caching | ||
| - Cache pull from the most recent pushed latest image | ||
| - Software Bill of Materials (SBOM) for supply chain transparency | ||
| - Provenance attestation to verify how and where the image was built | ||
|
|
||
| > [!TIP] | ||
| > Learn more: | ||
| > - [Docker task for Azure Pipelines](https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/docker-v2?view=azure-pipelines&tabs=yaml) | ||
| > - [Docker SBOM Attestations](/manuals/build/metadata/attestations/slsa-provenance.md) | ||
kristiyan-velkov marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| #### Step 4.4 Push the Docker Image | ||
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ```yaml | ||
| - task: Docker@2 | ||
| displayName: Push Docker Image | ||
| condition: eq(variables['Build.SourceBranch'], 'refs/heads/main') | ||
| inputs: | ||
| command: push | ||
| repository: $(imageName) | ||
| tags: | | ||
| $(buildTag) | ||
| $(latestTag) | ||
|
|
||
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| ``` | ||
| By applying this condition, the pipeline builds the Docker image on every run to ensure early detection of issues, but only pushes the image to the registry when changes are merged into the main branch—keeping your Docker Hub clean and focused | ||
|
Check warning on line 272 in content/guides/azure-pipelines.md
|
||
|
|
||
| This uploads both tags to Docker Hub: | ||
| - `$(buildTag)` ensures traceability per run. | ||
| - `latest` is used for most recent image references. | ||
| - | ||
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
|
|
||
| #### Step 4.5 Logout from Docker (Self-Hosted Agents) | ||
|
Check warning on line 280 in content/guides/azure-pipelines.md
|
||
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ```yaml | ||
| - script: docker logout | ||
| displayName: Docker Logout (Self-hosted only) | ||
| condition: ne(variables['Agent.OS'], 'Windows_NT') | ||
|
|
||
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| ``` | ||
|
|
||
| Executes docker logout at the end of the pipeline on Linux-based self-hosted agents to proactively clean up credentials and enhance security posture. | ||
|
Check warning on line 289 in content/guides/azure-pipelines.md
|
||
|
|
||
| --- | ||
|
|
||
| ## Summary | ||
|
|
||
| With this Azure Pipelines CI setup, you get: | ||
| - Secure Docker authentication using a built-in service connection. | ||
| - Automated image building and tagging triggered by code changes. | ||
| - Efficient builds leveraging Docker BuildKit cache. | ||
| - Safe cleanup with logout on persistent agents. | ||
|
Check warning on line 299 in content/guides/azure-pipelines.md
|
||
kristiyan-velkov marked this conversation as resolved.
Show resolved
Hide resolved
kristiyan-velkov marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| - Build images that meet modern software supply chain requirements with SBOM and attestation | ||
| --- | ||
|
|
||
| ## Further Reading | ||
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| - [Azure Pipelines Documentation](https://learn.microsoft.com/en-us/azure/devops/pipelines/?view=azure-devops) - Comprehensive guide to configuring and managing CI/CD pipelines in Azure DevOps. | ||
|
Check warning on line 305 in content/guides/azure-pipelines.md
|
||
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| - [Docker Task for Azure Pipelines](https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/build/docker) - Detailed reference for using the Docker task in Azure Pipelines to build and push images. | ||
| - [Docker Buildx Bake](/manuals/build/bake/_index.md) - Explore Docker's advanced build tool for complex, multi-stage, and multi-platform build setups. See also the [Mastering Buildx Bake Guide](/guides/bake/index.md) for practical examples and best practices. | ||
| - [Docker Build Cloud](/guides/docker-build-cloud/_index.md) - Learn about Docker's managed build service for faster, scalable, and multi-platform image builds in the cloud. | ||
kristiyan-velkov marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
Uh oh!
There was an error while loading. Please reload this page.