|
| 1 | +# Guide: Manifest Generation Pipeline |
| 2 | + |
| 3 | +This section describes how to configure Azure Devops to be your CI/CD orchestrator for your GitOps Workflow. You will create a manifest generation pipeline using Fabrikate. |
| 4 | + |
| 5 | +## Prerequisites |
| 6 | + |
| 7 | +1. _Permissions_: The ability to create Projects in your Github Organization. |
| 8 | +2. _High Level Deployment Description_: Either your own [Fabrikate](https://github.com/Microsoft/fabrikate) high level definition for your deployment or a sample one of ours. We provide a [sample HLD repo](https://github.com/andrebriggs/fabrikate-sample-app) that builds upon the [cloud-native](https://github.com/timfpark/fabrikate-cloud-native) Fabrikate definition. |
| 9 | + |
| 10 | +## Setup |
| 11 | + |
| 12 | +### 1. Create Repositories and Personal Access Tokens |
| 13 | + |
| 14 | +Create both high level definition (HLD) and resource manifest repos and the personal access tokens that you'll use for the two ends of this CI/CD pipeline. We have instructions for how to do that in two flavors: |
| 15 | +* [Azure DevOps](../docs/ADORepos.md) |
| 16 | +* [GitHub](../docs/GitHubRepos.md) |
| 17 | + |
| 18 | +#### Add Github actions workflow YAML |
| 19 | +If you are using your own high level description, add the following [`workflow.yml`](./workflow.yml) file to the .github/workflows directory to defines the build steps for your Github actions workflow. |
| 20 | + |
| 21 | +``` |
| 22 | +on: |
| 23 | + push: |
| 24 | + branches: |
| 25 | + - main |
| 26 | + - master |
| 27 | + |
| 28 | + pull_request: |
| 29 | + branches: |
| 30 | + - main |
| 31 | + - master |
| 32 | +
|
| 33 | +jobs: |
| 34 | + FabrikateToManifest: |
| 35 | +
|
| 36 | + runs-on: ubuntu-latest |
| 37 | + steps: |
| 38 | + - uses: actions/checkout@v2 |
| 39 | + - name: Download Bedrock orchestration script |
| 40 | + run: | |
| 41 | + curl $BEDROCK_BUILD_SCRIPT > build.sh |
| 42 | + chmod +x ./build.sh |
| 43 | + shell: bash |
| 44 | + env: |
| 45 | + BEDROCK_BUILD_SCRIPT: https://raw.githubusercontent.com/Microsoft/bedrock/master/gitops/azure-devops/build.sh |
| 46 | +
|
| 47 | + |
| 48 | + - uses: azure/setup-helm@v1 |
| 49 | + with: |
| 50 | + version: '2.17.0' # default is latest stable |
| 51 | + id: install |
| 52 | +
|
| 53 | + - name: Validate fabrikate definitions |
| 54 | + run: | |
| 55 | + chmod +x ./build.sh |
| 56 | + ./build.sh |
| 57 | + shell: bash |
| 58 | + env: |
| 59 | + MAJOR: 1 |
| 60 | + VERIFY_ONLY: 1 |
| 61 | +
|
| 62 | + - name: Get branch name |
| 63 | + shell: bash |
| 64 | + run: echo "##[set-output name=branch_name;]$(echo ${GITHUB_REF#refs/heads/})" |
| 65 | + id: get_branch_name |
| 66 | +
|
| 67 | + - name: Transform fabrikate definitions and publish to YAML manifests to repo |
| 68 | + run: | |
| 69 | + ./build.sh |
| 70 | + shell: bash |
| 71 | + env: |
| 72 | + MAJOR: 1 |
| 73 | + ACCESS_TOKEN_SECRET: ${{ secrets.ACCESS_TOKEN }} |
| 74 | + COMMIT_MESSAGE: ${{ github.event.head_commit.message }} |
| 75 | + REPO: ${{ secrets.MANIFEST_REPO }} |
| 76 | + BRANCH_NAME: ${{ steps.get_branch_name.outputs.branch_name }} |
| 77 | +``` |
| 78 | + |
| 79 | +### 2. Create workflow |
| 80 | + |
| 81 | +We use an [Github workflow](https://github.com/features/actions) to build your high level description into resource manifests: |
| 82 | + |
| 83 | +1. On a pull request (pre push to master) it executes a simple validation on proposed changes to infrastructure definition in the HLD repo. |
| 84 | +1. On a merge to master branch (post push to master) it executes a script to transform the high level definition to YAML using [Fabrikate](https://github.com/Microsoft/fabrikate) and pushes the generated results into the resource manifest repo. |
| 85 | + |
| 86 | +__Note__: If you would like to trigger a build from a pipeline not linked to the high level definition repo, you can define a variable `HLD_PATH` and pass it into the script with other variables as shown above in `workflow.yml`. You need to set this to a git URL, such as `git://github.com/Microsoft/fabrikate-production-cluster-demo.git`. |
| 87 | + |
| 88 | +#### Create Build for your Definition Repo |
| 89 | + |
| 90 | +With Github actions you do not need to setup the workflow. Commit and push the yml to the workflows directory and its set. |
| 91 | + |
| 92 | +#### Configure Build |
| 93 | + |
| 94 | +1. Click the "Secrets" tab. |
| 95 | + |
| 96 | +5. Add two variables that are used by the `build.sh` script referenced in `workflow.yml`: |
| 97 | + 1. __Name__: `ACCESS_TOKEN` (_mandatory_) __Value__: Personal Access Token ([Azure DevOps](https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops) or [GitHub](https://www.help.github.com/articles/creating-a-personal-access-token-for-the-command-line)) for your repo type. |
| 98 | + 2. __Name__: `MANIFEST_REPO` (_mandatory_) __Value__: The full URL to your manifest repo (i.e. https://github.com/andrebriggs/acme-company-yaml.git) |
| 99 | + 3. __Name__: `FAB_ENVS` (_optional_) __Value__: Comma-separated list of environments for which you have specified a config in your high level definition repo. If this variable is not created in the pipeline, the script will generate manifest files for a generic `prod` environment. For example, you may set this variable to `prod-east, prod-west` depending on your configuration. |
| 100 | + |
| 101 | +7. You should now see the build run and complete successfully. |
| 102 | + |
| 103 | +### 3. Configure Flux |
| 104 | + |
| 105 | +Once you have your Github workflow is working, you will need to retrieve the SSH public key you used to [set up your cluster](../../cluster/common/flux/README.md). |
| 106 | + |
| 107 | +1. Copy the SSH key to your clipboard. |
| 108 | + |
| 109 | +2. In Github actions, under Settings > Deploy kets, click on `Add deploy key` and add the Flux deploy key. |
| 110 | +  |
| 111 | + |
| 112 | +3. On your cluster find the name of your pod by executing `kubectl get pods -n flux` |
| 113 | + ``` |
| 114 | + $ kubectl get pods -n flux |
| 115 | + NAME READY STATUS RESTARTS AGE |
| 116 | + flux-7d459f5f9-c2wtd 1/1 Running 0 24h |
| 117 | + flux-memcached-59947476d9-49xs6 1/1 Running 0 24h |
| 118 | + ``` |
| 119 | +
|
| 120 | +4. Monitor the logs of your running Flux instance using the command `kubectl logs POD_NAME -n flux` to ensure that the initial manifest YAML files are being applied to your cluster. |
| 121 | +``` |
| 122 | +$ kubectl logs flux-7d459f5f9-c2wtd -n flux |
| 123 | +ts=2019-02-14T19:37:55.332948174Z caller=main.go:156 version=1.10.1 |
| 124 | +ts=2019-02-14T19:37:55.408911845Z caller=main.go:247 component=cluster identity=/etc/fluxd/ssh/identity |
| 125 | +ts=2019-02-14T19:37:55.414659575Z caller=main.go:417 url=git@github.com:andrebriggs/aks-feb-manifest.git user="Weave Flux" email=support@weave.works sync-tag=flux-sync notes-ref=flux set-author=false |
| 126 | +... |
| 127 | +... |
| 128 | +``` |
| 129 | +Now, when a change is commited to the resource manifest repo, Flux should acknowledge the commit and make changes to the state of your cluster as necessary. You can monitor Flux by viewing the logs by running `kubectl logs POD_NAME -n flux -f` in stream mode. |
| 130 | +
|
| 131 | +### 4. Make a Pull Request |
| 132 | +
|
| 133 | +1. Create a new branch in your HLD repo and make a commit to the high level definition. |
| 134 | +
|
| 135 | +1. For example, let's say we wanted to make a change that dropped the `cloud-native` stack and instead added directly a Elasticsearch / Fluentd / Kibana logging stack and Prometheus / Grafana metrics monitoring stack to your definition. We would make a commit that made this change: |
| 136 | +  |
| 137 | +
|
| 138 | +1. Then, create a pull request to merge your changes into master/main branch. |
| 139 | +
|
| 140 | +1. Once these checks have passed and the PR has been approved by your team process, you can merge it into master. |
| 141 | +
|
| 142 | +### 5. Monitor Repository Changes |
| 143 | +1. Once merged, you can monitor the progress of the HLD transformation in the Actions tab. |
| 144 | +
|
| 145 | +1. When the commit is merged into master/main, your workflow will build the resource manifests for this definition and check them into the resource manifest repo. |
| 146 | +
|
| 147 | +1. Once the build is successful, navigate to your manifest repository. You should see a very recent commit to the main branch. |
| 148 | +
|
| 149 | +### 6. Monitor Cluster Changes |
| 150 | +
|
| 151 | +1. Next, [Flux](https://github.com/weaveworks/flux/blob/master/site/get-started.md#confirm-the-change-landed) will automatically apply the build resource manifest changes to your cluster. You can watch this with the following `kubectl` command: |
| 152 | +
|
| 153 | +``` |
| 154 | +$ kubectl logs POD_NAME -n flux -f |
| 155 | +``` |
| 156 | +
|
| 157 | +2. You can also use [Kubediff](https://github.com/weaveworks/kubediff) to make sure the applied resource manifests in your cluster match your resource manifest repo by cloning your resource manifest repo and then running: |
| 158 | +
|
| 159 | +``` |
| 160 | +$ kubediff ./cloned-resource-manifest-repo |
| 161 | +``` |
| 162 | +
|
| 163 | +3. Finally, you should watch your normal operational metrics to make sure the change was successful. |
0 commit comments