Skip to content

Commit b4b4464

Browse files
mnuttallmnuttall
authored
Document automerge (#75)
Document automerge
1 parent a4da6ab commit b4b4464

File tree

13 files changed

+510
-16
lines changed

13 files changed

+510
-16
lines changed

‎README.md‎

Lines changed: 1 addition & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -73,18 +73,6 @@ go test ./pkg/git -run TestCopyServiceWithFailureCopying
7373

7474
## Getting started
7575

76-
This section is temporary. To create a sample promotion Pull Request, until https://github.com/rhd-gitops-example/services/issues/8 is done:
77-
78-
- Fork the repository https://github.com/rhd-gitops-example/gitops-example-dev
79-
- Fork the repository https://github.com/rhd-gitops-example/gitops-example-staging
80-
- Inside the services folder build the code: `go build ./cmd/services`
81-
- export GITHUB_TOKEN=[your token]
82-
- Substitute your repository URLs for those in square brackets:
83-
84-
```shell
85-
./services promote --from [url.to.dev] --to [url.to.staging] --service service-a`
86-
```
87-
8876
At a high level the services command currently:
8977

9078
- git clones the source and target repositories into ~/.promotion/cache
@@ -94,10 +82,7 @@ At a high level the services command currently:
9482
- pushes the cloned target
9583
- creates a PR from the new branch in the target to master in the target
9684

97-
## Important notes:
98-
99-
- We need to remove the local cache between requests. See https://github.com/rhd-gitops-example/services/issues/20. Until then, add `rm -rf ~/.promotion/cache; ` before subsequent requests.
100-
- See https://github.com/rhd-gitops-example/services/issues/19 for an issue related to problems 'promoting' config from a source repo into a gitops repo.
85+
See the [tekton-example](./tekton-example/README.md) directory for more on using the `promote` task with Tekton. See [automerge](./automerge/README.md) for some suggestions as to how promotion PullRequests may be merged automatically where appropriate.
10186

10287
## Release process
10388

‎automerge-example/Dockerfile‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
FROM ubuntu
2+
RUN apt-get update
3+
4+
# Install curl
5+
RUN apt-get update
6+
RUN apt-get install -y curl
7+
8+
# Install kubectl
9+
RUN apt-get install -y apt-transport-https gnupg2
10+
RUN curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
11+
RUN echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | tee -a /etc/apt/sources.list.d/kubernetes.list
12+
RUN apt-get update && apt-get install -y kubectl
13+
14+
15+
# Easy way to install yq, but wastes a lot of image size. Only required for 'standalone' case.
16+
RUN apt-get install -y software-properties-common
17+
RUN add-apt-repository -y ppa:rmescandon/yq
18+
RUN apt install -y yq
19+
20+
# Install hub command, and add 'git' alias. 'hub' is a strict superset of 'git'.
21+
RUN apt install -y hub
22+
RUN ln -s /usr/local/bin/hub /usr/local/bin/git
23+
24+
RUN rm -rf /var/lib/apt/lists/*

‎automerge-example/README.md‎

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
# Automerge
2+
3+
The `promote` task initiates change by creating Pull Requests. Sometimes a team will want to merge these manually, and sometimes automation can assist. A development team may want changes to an early, shared 'dev' environment to be merged automatically, whereas changes into 'prod' will more likely require manual approval. This directory contains sample Tekton tasks that demonstrate ways to automatically merge Pull Requests.
4+
5+
We have two subdirectories: 'standalone' and 'webhooks'. In the former case, a Tekton PipelineRun must be created manually. In the latter, the [Tekton Dashboard Webhooks Extension](https://github.com/tektoncd/experimental/tree/master/webhooks-extension) is used to start an automerge PipelineRun in response to a PullRequest arriving at a GitOps repository.
6+
7+
## Prerequisites
8+
9+
This example is for more advanced users. Start with the [tekton-example](../tekton-example/README.md) if you're new to the tool. You should have a good understanding of the tool, its purposes and syntax before working through the 'automerge' topic. In addition to the `services` binary you will need the following tools installed:
10+
11+
- [`kubectl`](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
12+
- [`tkn`](https://github.com/tektoncd/cli) if you're not using webhooks
13+
- [`docker`](https://docs.docker.com/get-docker/)
14+
15+
*Note* The 'standalone' code was developed on Docker Desktop and does not yet include the Role-Based Access Control configuration necessary for it to run on OpenShift or other locked-down environments. The 'webhook' code was developed on OpenShift but used a ServiceAccount that had a generous Role attached to it. Full RBAC support should be added to this example under https://github.com/rhd-gitops-example/services/issues/77.
16+
17+
## Setup - both cases
18+
19+
Our samples currently work with GitHub. They use the `hub` CLI to create a merge commit that when pushed, will merge the associated Pull Request. See [here](https://hub.github.com/hub-merge.1.html) for more details.
20+
21+
## gitconfig
22+
23+
Since we are creating git commits within the Tekton tasks, we need to know the username and email address to associate with them, as in our ['tekton-example'](../tekton-example/README.md). So, edit `gitconfig` and
24+
25+
```sh
26+
kubectl create configmap promoteconfigmap --from-file=gitconfig
27+
```
28+
29+
## Dockerfile
30+
31+
The 'hub' CLI runs in a Docker container within a Tekton Pipeline. We've provided a sample Dockerfile. If you are taking the 'webhooks' path you can comment out the section that installs `yq`. Then,
32+
33+
```sh
34+
docker login
35+
docker build -t YOUR_DOCKER_HUB_ID/hub-test .
36+
docker push YOUR_DOCKER_HUB_ID/hub-test
37+
```
38+
39+
## Create a Pull Request
40+
41+
1. Fork our example gitops repository, https://github.com/rhd-gitops-example/gitops-example-dev.
42+
1. Create a 'promotion' Pull Request either manually or using our ['tekton-example'](../tekton-example/README.md). For example you can use code of the form,
43+
44+
```sh
45+
services promote --from promote-demo --to https://github.com/YOUR_GITHUB_ID/gitops-example-dev.git --service promote-demo
46+
```
47+
48+
## Generate a GitHub token
49+
50+
You will need a GitHub token with repository access in order to merge Pull Requests.
51+
52+
## Standalone case
53+
54+
Edit the files in the 'standalone/templates' directory.
55+
56+
- In `automerge-task.yaml` replace `YOUR_DOCKER_HUB_ID` with your DockerHub id.
57+
- In `git-resource.yaml` replace `YOUR_GITHUB_ID` with your GitHub id.
58+
- In `github-secret.yaml` replace `[your-github-token-with-repo-access]` with your GitHub token.
59+
- In pull-request.yaml replace YOUR_PULL_REQUEST_URL with your pull request URL, e.g. `https://github.com/mnuttall/gitops-example-dev/pull/22`
60+
61+
Apply all the Tekton resources:
62+
63+
```sh
64+
kubectl apply -f standalone/resources
65+
kubectl apply -f standalone/templates
66+
```
67+
68+
Finally start the Tekton pipeline:
69+
70+
```sh
71+
tkn pipeline start automerge-pipeline -r source-repo=gitops-repo -r pr=pull-request -p github-config=promoteconfigmap -p github-secret=github-secret --showlog
72+
```
73+
74+
The pipeline will do a dry run to test that the yaml in the Pull Request is good, then merge the Pull Request and delete the branch associated with it.
75+
76+
## Tekton Dashboard Webhooks Extension
77+
78+
See the [Getting Started](https://github.com/tektoncd/experimental/blob/master/webhooks-extension/docs/GettingStarted.md) guide for setup guidelines. Our example uses GitHub Enterprise (GHE) and expects webhooks to be delivered to a cluster that is routeable from GHE.
79+
80+
### Secrets and Service Accounts
81+
82+
Set up your Service Account and Secret as per the guide above. In this case you should have,
83+
84+
- A ServiceAccount configured for use by Tekton.
85+
- A Tekton-compatible Secret patched onto that that ServiceAccount containing your GitHub token.
86+
87+
This secret is used in two related ways. We check the source repository out using a Tekton Git PipelineResource. This sets up `~/.gitconfig` with the credentials needed for `git push` to work. It gets these credentials from the `accessToken` field of the relevant secret patched onto the ServiceAccount running the Tekton Task. We then extract the same field and export it into the `GITHUB_TOKEN` environment variable to make `hub merge` work. Instructions for creating this secret are in the Getting Started document linked above. You should have resources of the form,
88+
89+
```yaml
90+
---
91+
apiVersion: v1
92+
data:
93+
password: [base64-encoded token]
94+
username: [base64-encoded email address]
95+
kind: Secret
96+
metadata:
97+
annotations:
98+
tekton.dev/git-0: https://github.ibm.com # For example
99+
labels:
100+
serviceAccount: test-sa # As configured in the Webhooks Extension
101+
name: github-repo-access-secret
102+
103+
---
104+
apiVersion: v1
105+
kind: ServiceAccount
106+
metadata:
107+
name: test-sa
108+
secrets:
109+
- name: github-repo-access-secret
110+
```
111+
112+
### Edit templates
113+
114+
Next edit the `webhooks/templates/*` files.
115+
116+
- In automerge-task.yaml,
117+
- replace `YOUR_DOCKER_HUB_ID` with your DockerHub id.
118+
- replace `YOUR_GHE` with your GitHub Enterprise domain.
119+
- In automerge-tt.yaml, replace YOUR_TEKTON_SERVICE_ACCOUNT with the name of your ServiceAccount used by Tekton.
120+
121+
### Apply configuration
122+
123+
Apply your Tekton config as usual:
124+
125+
```sh
126+
kubectl apply -f webhooks/resources
127+
kubectl apply -f webhooks/templates
128+
```
129+
130+
### Set up webhook, and test
131+
132+
Using the Tekton Dashboard webhooks extension, associate the `automerge-pipeline` with your GitOps repository. Now when a PR is raised against that repo you should see three PipelineRuns created for `automerge-pipeline`.
133+
134+
- The first is triggered when the branch for the PullRequest is created. This runs the `echo "do nothing"` section in `automerge-task`.
135+
- The second run executes the bulk of `automerge-task`: a merge commit is created and pushed, and its associated branch deleted.
136+
137+
TODO:
138+
A discrepancy occurs between running this task against GitHub and GitHub Enterprise. On GHE, while the commit is merged the PR remains open whereas on GitHub, it is shown as merged. A commented out section in `automerge-task` notes this, which will be investigated under https://github.com/rhd-gitops-example/services/issues/76.
139+
140+
- Finally the third run executes `echo "kubectl apply -k env"`. Were you to remove the `echo` then this would result in the updated configuration being deployed.

‎automerge-example/gitconfig‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
[user]
2+
name = REPLACE_ME.GITHUB_USERNAME
3+
email = REPLACE_ME.GITHUB_EMAIL

‎automerge-example/standalone/resources/automerge-pipeline.yaml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
apiVersion: tekton.dev/v1alpha1
2+
kind: Pipeline
3+
metadata:
4+
name: automerge-pipeline
5+
spec:
6+
params:
7+
- name: github-secret
8+
type: string
9+
- name: github-config
10+
type: string
11+
resources:
12+
- name: source-repo
13+
type: git
14+
- name: pr
15+
type: pullRequest
16+
tasks:
17+
- name: do-this-first
18+
taskRef:
19+
name: automerge-task
20+
resources:
21+
inputs:
22+
- name: git-source
23+
resource: source-repo
24+
- name: pull-request
25+
resource: pr
26+
params:
27+
- name: github-secret
28+
value: $(params.github-secret)
29+
- name: github-config
30+
value: $(params.github-config)

‎automerge-example/standalone/templates/automerge-task.yaml‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
apiVersion: tekton.dev/v1alpha1
2+
kind: Task
3+
metadata:
4+
name: automerge-task
5+
spec:
6+
params:
7+
- name: github-config
8+
type: string
9+
description: configmap name of the gitconfig file that has user name, user e-mail. The key name is gitconfig. It can be created by "kubectl create configmap <configmap name> --from-file=gitconfig"
10+
- name: github-secret
11+
type: string
12+
description: Name of the secret containing an access token for Github. Expects Tekton format, 'username' and 'password' keys.
13+
volumes:
14+
- name: gitconfig
15+
configMap:
16+
name: $(params.github-config)
17+
items:
18+
- key: gitconfig
19+
path: gitconfig
20+
inputs:
21+
resources:
22+
- name: git-source
23+
type: git
24+
- name: pull-request
25+
type: pullRequest
26+
steps:
27+
- name: check-yaml
28+
image: YOUR_DOCKER_HUB_ID/hub-test
29+
script: |
30+
#!/bin/bash
31+
kubectl apply -k git-source/env --dry-run=client
32+
- name: merge-pr
33+
image: YOUR_DOCKER_HUB_ID/hub-test
34+
volumeMounts:
35+
- name: gitconfig
36+
mountPath: /root
37+
script: |
38+
#!/bin/bash -x
39+
40+
cat /root/gitconfig >> $HOME/.gitconfig
41+
fullPRLink=$(yq r pull-request/pr.json 'Link')
42+
prURL=${fullPRLink%.diff}
43+
44+
cd git-source
45+
hub merge $prURL
46+
git push -u origin master
47+
48+
# Pushing the change will merge the PR. Delete its branch to tidy up.
49+
prBranch=$(yq r /workspace/pull-request/pr.json 'Head.Ref')
50+
git push origin --delete $prBranch
51+
52+
env:
53+
- name: GITHUB_TOKEN
54+
valueFrom:
55+
secretKeyRef:
56+
name: $(params.github-secret)
57+
key: password

‎automerge-example/standalone/templates/git-resource.yaml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
apiVersion: tekton.dev/v1alpha1
2+
kind: PipelineResource
3+
metadata:
4+
name: gitops-repo
5+
spec:
6+
params:
7+
- name: revision
8+
value: master
9+
- name: url
10+
value: https://github.com/YOUR_GITHUB_ID/gitops-example-dev
11+
type: git

‎automerge-example/standalone/templates/github-secret.yaml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
apiVersion: v1
2+
kind: Secret
3+
metadata:
4+
name: github-secret
5+
annotations:
6+
tekton.dev/git-0: https://github.com
7+
type: kubernetes.io/basic-auth
8+
stringData:
9+
username: token
10+
password: [your-github-token-with-repo-access]

‎automerge-example/standalone/templates/pull-request.yaml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
apiVersion: tekton.dev/v1alpha1
2+
kind: PipelineResource
3+
metadata:
4+
name: pull-request
5+
spec:
6+
type: pullRequest
7+
params:
8+
- name: url
9+
value: YOUR_PULL_REQUEST_URL
10+
secrets:
11+
- fieldName: authToken
12+
secretName: github-secret
13+
secretKey: password

‎automerge-example/webhooks/resources/automerge-pipeline.yaml‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
apiVersion: tekton.dev/v1alpha1
2+
kind: Pipeline
3+
metadata:
4+
name: automerge-pipeline
5+
spec:
6+
params:
7+
- name: github-secret
8+
type: string
9+
- name: github-config
10+
type: string
11+
- name: event-type
12+
type: string
13+
- name: branch-name
14+
type: string
15+
- name: pull-request-url
16+
type: string
17+
resources:
18+
- name: source-repo
19+
type: git
20+
tasks:
21+
- name: test-and-automerge
22+
taskRef:
23+
name: automerge-task
24+
resources:
25+
inputs:
26+
- name: git-source
27+
resource: source-repo
28+
params:
29+
- name: github-secret
30+
value: $(params.github-secret)
31+
- name: github-config
32+
value: $(params.github-config)
33+
- name: event-type
34+
value: $(params.event-type)
35+
- name: branch-name
36+
value: $(params.branch-name)
37+
- name: pull-request-url
38+
value: $(params.pull-request-url)

0 commit comments

Comments
 (0)