You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# Tutorial: Workload management in a multi-cluster environment with GitOps
14
14
15
+
Enterprise organizations, developing cloud native applications, face challenges to deploy, configure and promote a great variety of applications and services across a fleet of Kubernetes clusters at scale.
16
+
15
17
This tutorial walks you through typical scenarios of the workload deployment and configuration in a multi-cluster Kubernetes environment. First, you deploy a sample infrastructure with a few GitHub repositories and AKS clusters. Next, you work through a set of use cases where you act as different personas working in the same environment: the Platform Team and the Application Team.
16
18
17
19
In this tutorial, you learn how to:
18
20
19
21
> [!div class="checklist"]
20
-
> * Deploy the sample infrastructure
21
22
> * Onboard a new application
23
+
> * Schedule an application on the cluster types
24
+
> * Promote an application across rollout environemnts
22
25
> * Build and deploy an application
23
26
> * Provide platform configurations
24
27
> * Add a new cluster type to your environment
@@ -45,7 +48,7 @@ chmod 700 deploy.sh
45
48
./deploy.sh -c -p <prefix. e.g. kalypso> -o <github org. e.g. eedorenko> -t <github token> -l <azure-location. e.g. westus2>
46
49
```
47
50
48
-
This script may take a while to complete. After it's done, it reports the execution result in the output like this:
51
+
This script may take 10-15 minutes to complete. After it's done, it reports the execution result in the output like this:
49
52
50
53
```output
51
54
Deployment is complete!
@@ -91,7 +94,7 @@ The infrastructure also includes a couple of the Application Team repositories:
91
94
The script created the following Azure Kubernetes Service (AKS) clusters:
92
95
93
96
- `control-plane` - This cluster doesn't run any workloads. It's a management cluster. It hosts [Kalypso Scheduler](https://github.com/microsoft/kalypso-scheduler) operator that transforms high level abstractions from the [Control Plane](https://github.com/microsoft/kalypso-control-plane) repository to the raw Kubernetes manifests in the [Platform GitOps](https://github.com/microsoft/kalypso-gitops) repository.
94
-
- `drone` - A sample workload cluster. It's Azure Arc-enabled and it uses `Flux` to reconcile manifests from the [Platform GitOps](https://github.com/microsoft/kalypso-gitops) repository.
97
+
- `drone` - A sample workload cluster. It has [GitOps extension](conceptual-gitops-flux2.md) installed and it uses `Flux` to reconcile manifests from the [Platform GitOps](https://github.com/microsoft/kalypso-gitops) repository.
95
98
- `large` - A sample workload cluster. It has `ArgoCD` installed on it to reconcile manifests from the [Platform GitOps](https://github.com/microsoft/kalypso-gitops) repository.
96
99
97
100
### Explore Control Plane
@@ -122,7 +125,7 @@ The `dev` and `stage` branches:
122
125
123
126
## 2 - Platform Team: Onboard a new application
124
127
125
-
The Application Team runs their software development lifecycle. They build their application and promote it across environments. They're not aware of what cluster types are available in the fleet and where their application is going to be deployed. But they do know that they want to deploy their application in`Dev` environment forfunctional and performance testing andin`Stage` environment for testing.
128
+
The Application Team runs their software development lifecycle. They build their application and promote it across environments. They're not aware of what cluster types are available in the fleet and where their application is going to be deployed. But they do know that they want to deploy their application in`Dev` environment forfunctional and performance testing andin`Stage` environment forUAT testing.
126
129
127
130
The Application Team describes this intention in the [workload](https://github.com/microsoft/kalypso-app-src/blob/main/workload/workload.yaml) file in the [Application Source](https://github.com/microsoft/kalypso-app-src) repository:
128
131
@@ -201,7 +204,7 @@ git push
201
204
```
202
205
203
206
> [!NOTE]
204
-
> For sinmplicity, this tutorial pushes changes directly to `main`. In practice, you'd create a pull request to submit the changes.
207
+
> For simplicity, this tutorial pushes changes directly to `main`. In practice, you'd create a pull request to submit the changes.
205
208
206
209
With that in place, the application is onboarded in the control plane. But the control plane still doesn't know how to map the application deployment targets to the cluster types in the fleet.
207
210
@@ -272,7 +275,7 @@ Besides `Promoted_Commit_id`, which is just tracking information for the promoti
272
275
273
276
Note that there are `command-center` and `small` cluster types as well in the `Dev` environment:
274
277
275
-
:::image type="content" source="media/tutorial-workload-management/dev-cluster-types.png" alt-text="Screenshot showing acluster types in the Dev environment.":::
278
+
:::image type="content" source="media/tutorial-workload-management/dev-cluster-types.png" alt-text="Screenshot showing cluster types in the Dev environment.":::
276
279
277
280
However, only the `drone` and `large` cluster types were selected by the scheduling policies that you defined.
278
281
@@ -347,7 +350,7 @@ spec:
347
350
348
351
### Promote application to Stage
349
352
350
-
Once we have approved and merged the PR to the `Platform GitOps` repository, the `drone` and `large` AKS clusters that represent corresponding cluster types start fetching the assignment manifests. The `drone` cluster has Azure Arc GitOps configuration installed, pointing to the `Platform GitOps` repository. It reports its `compliance` status:
353
+
Once we have approved and merged the PR to the `Platform GitOps` repository, the `drone` and `large` AKS clusters that represent corresponding cluster types start fetching the assignment manifests. The `drone` cluster has [GitOps extension](conceptual-gitops-flux2.md) installed, pointing to the `Platform GitOps` repository. It reports its `compliance` status to Azure Resource Graph:
351
354
352
355
:::image type="content" source="media/tutorial-workload-management/drone-compliance-state.png" alt-text="Screenshot showing compliance state details for the drone cluster.":::
353
356
@@ -415,13 +418,15 @@ The Application Team regularly submits pull requests to the `main` branch in the
415
418
416
419
The workflow does the following:
417
420
418
-
:::image type="content" source="media/tutorial-workload-management/cicd-workflow.png" alt-text="Screenshot showing the CI/CD workflow." lightbox="media/tutorial-workload-management/cicd-workflow.png":::
419
-
420
421
- Builds the application Docker image and pushes it to the GitHub repository package.
421
422
- Generates manifests forthe `functional-test` and `performance-test` deployment targets. It uses configuration values from the `dev-configs` branch. The generated manifests are added to a pull request and auto-mergedin the `dev` branch.
422
-
- Generates manifests for the `uat-test` deployment target. It uses configuration values from the `stage-configs` branch. The generated manifests are added to a pull request to the `stage` branch waiting for approval:
423
+
- Generates manifests for the `uat-test` deployment target. It uses configuration values from the `stage-configs` branch.
424
+
425
+
:::image type="content" source="media/tutorial-workload-management/cicd-workflow.png" alt-text="Screenshot showing the CI/CD workflow." lightbox="media/tutorial-workload-management/cicd-workflow.png":::
426
+
427
+
The generated manifests are added to a pull request to the `stage` branch waiting for approval:
423
428
424
-
:::image type="content" source="media/tutorial-workload-management/app-pr-to-stage.png" alt-text="Screenshot showing a PR to stage.":::
429
+
:::image type="content" source="media/tutorial-workload-management/app-pr-to-stage.png" alt-text="Screenshot showing a PR to stage.":::
425
430
426
431
To test the application manually on the `Dev` environment before approving the PR to the `Stage` environment, first verify how the `functional-test` application instance works on the `drone` cluster:
427
432
@@ -509,7 +514,7 @@ In a few seconds, a new PR to the `stage` branch in the `Platform GitOps` reposi
509
514
510
515
Approve the PR and merge it.
511
516
512
-
The `large` cluster is handled by ArgoCD, which, by default, is configured to reconcile every three minutes. This cluster doesn't report its compliance state to Azure like the clusters such as `drone` which have the Azure Arc GitOps extension. However, you can still monitor the reconciliation state on the cluster with ArgoCD UI.
517
+
The `large` cluster is handled by ArgoCD, which, by default, is configured to reconcile every three minutes. This cluster doesn't report its compliance state to Azure like the clusters such as `drone` which have the [GitOps extension](conceptual-gitops-flux2.md). However, you can still monitor the reconciliation state on the cluster with ArgoCD UI.
513
518
514
519
To access the ArgoCD UI on the `large` cluster, run the following command:
515
520
@@ -532,7 +537,7 @@ Once the new configuration has arrived to the cluster, check the `uat-test` appl
@@ -542,7 +547,7 @@ You'll see the updated database url:
542
547
543
548
## 5 - Platform Team: Add cluster type to environment
544
549
545
-
Currently, only `drone` and `large` cluster types are included in the `Stage` environment. Let's include the `small` cluster type to `Stage` as well. Even though we don't have a physical cluster representing this cluster type, you can see how the scheduler reacts to this change.
550
+
Currently, only `drone` and `large` cluster types are included in the `Stage` environment. Let's include the `small` cluster type to `Stage` as well. Even though we don't have a physical cluster representing this cluster type, you can see how the scheduler reacts to this change.
546
551
547
552
```bash
548
553
# Switch to stage branch (representing Stage environemnt) in the control-plane folder
0 commit comments