You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/data-factory/ci-cd-pattern-with-airflow.md
+6-4Lines changed: 6 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -20,9 +20,9 @@ Working with data pipelines in Airflow requires you to create or update your DAG
20
20
21
21
### Continuous Integration (CI)
22
22
23
-
Continuous Integration (CI) is a software development practice that emphasizes frequent and automated integration of code changes into a shared repository. It involves developers regularly committing their code, and upon each commit, an automated CI pipeline builds the code, runs tests, and performs validation checks. The primary goal is to detect and address integration issues early in the development process, providing rapid feedback to developers. CI ensures that the codebase remains in a constantly testable and deployable state. This practise leads to enhanced code quality, collaboration, and the ability to catch and fix bugs before they become significant problems.
23
+
Continuous Integration (CI) is a software development practice that emphasizes frequent and automated integration of code changes into a shared repository. It involves developers regularly committing their code, and upon each commit, an automated CI pipeline builds the code, runs tests, and performs validation checks. The primary goal is to detect and address integration issues early in the development process, providing rapid feedback to developers. CI ensures that the codebase remains in a constantly testable and deployable state. This practice leads to enhanced code quality, collaboration, and the ability to catch and fix bugs before they become significant problems.
24
24
25
-
### Continuous Deployment
25
+
### Continuous Deployment (CD)
26
26
27
27
Continuous Deployment (CD) is an extension of CI that takes the automation one step further. While CI focuses on automating the integration and testing phases, CD automates the deployment of code changes to production or other target environments. This practice helps organizations release software updates quickly and reliably. It reduces mistakes in manual deployment and ensures that approved code changes are delivered to end-users swiftly.
28
28
@@ -37,7 +37,8 @@ When a pull request (PR) is made from a feature branch to the Development branch
37
37
-**Python Dependencies Testing**: These tests install and verify the correctness of Python dependencies to ensure that the project's dependencies are properly configured.
38
38
-**Code Analysis and Linting:** Tools for static code analysis and linting are applied to evaluate code quality and adherence to coding standards.
39
39
-**Airflow DAG’s Tests:** These tests execute validation tests, including tests for the DAG definition and unit tests designed for Airflow DAGs.
40
-
-**Unit Tests for Airflow custom operators, hooks, sensors and triggers**
40
+
-**Unit Tests for Airflow custom operators, hooks, sensors and triggers.**
41
+
41
42
If any of these checks fail, the pipeline terminates, signaling that the developer needs to address the issues identified.
42
43
43
44
#### Git-sync with Production IR: Map your Managed Airflow environment with your Git repository’s Production branch.
@@ -182,6 +183,7 @@ steps:
182
183
For more information, See [Azure Pipelines](/azure/devops/pipelines/get-started/pipelines-sign-up)
183
184
184
185
### Using GitHub Actions
186
+
185
187
**Step 2.1:** Create a `.github/workflows` directory in your GitHub repository.
186
188
187
189
**Step 2.2:** In the `.github/workflows` directory, create a file named `github-actions-ci-cd.yml`
0 commit comments