Skip to content

Commit 66a8986

Browse files
mnuttallmnuttall
authored
Overhaul README.md, split out DEVELOPMENT.md (#88)
1 parent 2cec184 commit 66a8986

File tree

2 files changed

+128
-55
lines changed

2 files changed

+128
-55
lines changed

‎DEVELOPMENT.md‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# Notes for developers
2+
3+
## Building
4+
5+
You need Go version 1.13 or higher to build this project.
6+
7+
```shell
8+
$ go build ./cmd/services
9+
```
10+
11+
## Testing
12+
13+
Linting should be done first (this is done on Travis, and what's good locally should be good there too)
14+
15+
Grab the linter if you haven't already:
16+
17+
```shell
18+
GO111MODULE=on go get github.com/golangci/golangci-lint/cmd/[email protected]
19+
```
20+
21+
Then you can do:
22+
23+
```shell
24+
golangci-lint run
25+
```
26+
27+
Run the unit tests:
28+
29+
```shell
30+
$ go test ./...
31+
```
32+
33+
To run the complete integration tests, including pushing to the Git repository:
34+
35+
```shell
36+
$ TEST_GITHUB_TOKEN=<a valid github auth token> go test ./...
37+
```
38+
39+
Note that the tests in pkg/git/repository_test.go will clone and manipulate a
40+
remote Git repository locally.
41+
42+
To run a particular test: for example,
43+
44+
```shell
45+
go test ./pkg/git -run TestCopyServiceWithFailureCopying
46+
```
47+
48+
## Release process
49+
50+
When a new tag is pushed with the `v` prefix, a GitHub release will be created with binaries produced for 64-bit Linux, and Mac automatically.

‎README.md‎

Lines changed: 78 additions & 55 deletions
Original file line numberDiff line numberDiff line change
@@ -1,96 +1,119 @@
11
# services [![Build Status](https://travis-ci.org/rhd-gitops-example/services.svg?branch=master)](https://travis-ci.org/rhd-gitops-example/services) [![Docker Repository on Quay](https://quay.io/repository/redhat-developer/gitops-cli/status "Docker Repository on Quay")](https://quay.io/repository/redhat-developer/gitops-cli)
22

3-
A tool for promoting between GitHub repositories.
3+
'GitOps' emphasises the use of Git repositories as sources of truth. Changes to a system should be made by changing its associated Git repository, rather than by operating directly on that system. This tool works with GitOps repositories that follow [this specification](https://github.com/rhd-gitops-example/docs/tree/master/model#gitops-repository). If provides a way to promote versions of files between environments, by operating on their associated Git repositories.
44

5-
This is a pre-alpha PoC for promoting versions of files between environments, represented as repositories.
5+
## Overview
66

7-
## Building
7+
At a high level the `services promote` command currently:
88

9-
You need Go version 1.14 to build this project.
9+
- git clones the source and target repositories into `~/.promotion/cache`
10+
- creates a branch
11+
- checks out the branch
12+
- copies the relevant files from the cloned source into the cloned target
13+
- pushes the cloned target
14+
- creates a PR from the new branch in the target to master in the target
1015

11-
```shell
12-
$ go build ./cmd/services
13-
```
16+
See the [tekton-example](./tekton-example/README.md) directory for more on using the `promote` task with Tekton. See [automerge](./automerge/README.md) for some suggestions as to how promotion PullRequests may be merged automatically where appropriate.
1417

15-
## Running
18+
## Installation
1619

17-
You'll need a GitHub token to test this out.
20+
Built versions of the tool can be obtained [here](https://github.com/rhd-gitops-example/services/releases). See [here](DEVELOPMENT.md) for instructions on building it yourself. The binary is also available as an OCI image at `quay.io/redhat-developer/gitops-cli`.
1821

19-
```shell
20-
$ export GITHUB_TOKEN=<paste in GitHub access token>
21-
$ ./services promote --from https://github.com/organisation/first-environment.git --to https://github.com/organisation/second-environment.git --service service-a --commit-name <User to commit as> --commit-email <Email to commit as>
22-
```
22+
## Usage
2323

24-
This will _copy_ all files under `/services/service-a/base/config/*` in `first-environment` to `second-environment`, commit and push, and open a PR for the change.
24+
`services` currently offers one verb: `promote`. This packages up the tasks required to take the version of a [service](https://github.com/rhd-gitops-example/docs/tree/master/model#gitops-repository) from one environment and deploy it into another.
2525

26+
### Promotion uses Pull Requests in GitHub and Merge Requests in GitLab
2627

27-
## Using environments
28+
A key point about GitOps in general is that it incorporates the ideas of workflow and approval. The `promote` task creates Pull Requests in GitHub, or Merge Requests in GitLab. In the simplest case, these will often require manual approval by someone other than the author of the change. This approval mechanism is part of how GitOps enables the idea of governance: it can be used to enforce rules about who can approve change, and under what circumstances. Thus `promote` doesn't cause deployments to happen: it creates Pull or Merge Requests. In due course these result in `git merge` operations within the associated repository, which may then cause deployment changes via automation.
2829

30+
### Types of promotion: 'Remote' vs 'Local'
2931

30-
If an `environments` folder exists in the GitOps repository you are promoting into, and that only has one folder, the files will be copied into the destination repository's `/environments/<the only folder>` directory.
32+
`promote` currently handles two different scenarios. You may hear us refer to these in shorthand as 'remote' and 'local'. In the 'remote' scenario, a specific version of a service is deployed in environment A and we want to promote it into environment B. Thus we might say, "promote my-service from dev to staging." The same version of my-service will be running in both environments on the eventual completion of the whole promotion worklow - i.e. once the Pull Request has been merged and the implied deployment changes completed. We call this 'remote' because the promotion is happening between two Git repositories both remote from the machine running 'promote'. Both git repositories are referred to by https:// URLs.
3133

32-
Future support is planned for an `--env` like flag which will allow us to promote from/to different repositories with multiple environments.
34+
We often think of promotion as a linear flow, such as 'from dev to staging to production.' The second scenario provides a means for services to enter this flow - it's about how services can get into 'dev' at all. So how can services get into 'dev'? One option is via [`odo pipelines service add`](https://github.com/rhd-gitops-example/docs/tree/master/commands/service). Under this model, having created a new service in a new source repository, a command is issued that generates a new local version of the GitOps repository, from which a pull request towards 'dev' may be manually prepared. Another option, which 'local promotion' is there to support, requires a new service, in its source repository, to contain a basic version of the kubernetes yaml that will be used to deploy it. 'Local' promotion can then be used by automation that is building the microservice, to automatically 'promote' it into the 'dev' environment. We call this 'local' promotion because the yaml in question is coming from a local directory, for example within a running Tekton build.
3335

34-
## Testing
36+
### Using environments
3537

36-
Linting should be done first (this is done on Travis, and what's good locally should be good there too)
38+
If an `environments` folder exists in the GitOps repository you are promoting into, and that only has one folder, the files will be copied into the destination repository's `/environments/<the only folder>` directory.
3739

38-
Grab the linter if you haven't already:
40+
Future support is planned for an `--env` like flag which will allow us to promote from/to different repositories with multiple environments.
3941

40-
```shell
41-
GO111MODULE=on go get github.com/golangci/golangci-lint/cmd/[email protected]
42-
```
42+
## Example 1: Promote a service 'service-a' from 'dev' to 'staging'
4343

44-
Then you can do:
44+
You'll need a GitHub token to test this out.
4545

46-
```shell
47-
golangci-lint run
46+
```sh
47+
export GITHUB_TOKEN=<paste in GitHub access token>
48+
./services promote --from https://github.com/organisation/dev.git --to https://github.com/organisation/staging.git --service service-a --commit-name <User to commit as> --commit-email <Email to commit as>
4849
```
4950

50-
Run the unit tests:
51+
If the `commit-name` and `commit-email` are not provided, it will attempt to find them in `~/.gitconfig`, otherwise it will fail.
5152

52-
```shell
53-
$ go test ./...
54-
```
53+
This will _copy_ all files under `/environments/<env-name>/services/service-a/base/config/*` in `dev` to `staging`, commit and push, and open a PR for the change.
5554

56-
To run the complete integration tests, including pushing to the Git repository:
55+
### Example 2: Promote (or 'publish') a microservice into 'dev'
5756

58-
```shell
59-
$ TEST_GITHUB_TOKEN=<a valid github auth token> go test ./...
57+
```sh
58+
services promote --from /workspace/service-b --to https://my.git/org/dev --service service-b --commit-message "Publish service-b into dev"
6059
```
6160

62-
Note that the tests in pkg/git/repository_test.go will clone and manipulate a
63-
remote Git repository locally.
64-
65-
To run a particular test: for example,
66-
67-
```shell
68-
go test ./pkg/git -run TestCopyServiceWithFailureCopying
61+
This will again raise a Pull Request, copying the local files `/workspace/service-b/config/*` to https://my.git/org/dev, into `/environments/<env-name>/services/service-b/base/config/*`
62+
63+
## CLI Reference
64+
65+
```sh
66+
services promote --help
67+
promote from one environment to another
68+
69+
Usage:
70+
services promote [flags]
71+
72+
Flags:
73+
--cache-dir string where to cache Git checkouts (default "~/.promotion/cache")
74+
--commit-email string the email to use for commits when creating branches
75+
--commit-message string the msg to use on the resultant commit and pull request
76+
--commit-name string the name to use for commits when creating branches
77+
--debug additional debug logging output
78+
--from string source Git repository
79+
-h, --help help for promote
80+
--insecure-skip-verify Insecure skip verify TLS certificate
81+
--keep-cache whether to retain the locally cloned repositories in the cache directory
82+
--repository-type string the type of repository: github, gitlab or ghe (default "github")
83+
--service string service name to promote
84+
--to string destination Git repository
85+
86+
Global Flags:
87+
--github-token string oauth access token to authenticate the request
6988
```
7089

71-
## Getting started
72-
73-
At a high level the services command currently:
74-
75-
- git clones the source and target repositories into ~/.promotion/cache
76-
- creates a branch
77-
- checks out the branch
78-
- copies the relevant files from the cloned source into the cloned target
79-
- pushes the cloned target
80-
- creates a PR from the new branch in the target to master in the target
90+
This will _copy_ all files under `/services/service-a/base/config/*` in `first-environment` to `second-environment`, commit and push, and open a PR for the change. Any of these arguments may be provided as environment variables, using all upper case and replacing `-` with `_`. Hence you can set CACHE_DIR, COMMIT_EMAIL, etc.
8191

82-
See the [tekton-example](./tekton-example/README.md) directory for more on using the `promote` task with Tekton. See [automerge](./automerge/README.md) for some suggestions as to how promotion PullRequests may be merged automatically where appropriate.
92+
- `--cache-dir` : path on the local filesystem in which Git checkouts will be cached.
93+
- `--commit-email` : Git commits require an associated email address and username. This is the email address. May be set via ~/.gitconfig.
94+
- `--commit-message` : use this to override the commit message which will otherwise be generated automatically.
95+
- `--commit-name` : The other half of `commit-email`. Both must be set.
96+
- `--debug` : prints extra debug output if true.
97+
- `--from` : an https URL to a GitOps repository for 'remote' cases, or a path to a Git clone of a microservice for 'local' cases.
98+
- `--help`: prints the above text if true.
99+
- `--insecure-skip-verify` : skip TLS cerificate verification if true. Do not set this to true unless you know what you are doing.
100+
- `--keep-cache` : `cache-dir` is deleted unless this is set to true. Keeping the cache will often cause further promotion attempts to fail. This flag is mostly used along with `--debug` when investigating failure cases.
101+
- `--repository-type` : the type of repository: github, gitlab or ghe (default "github"). If `--from` is a Git URL, it must be of the same type as that specified via `--to`.
102+
- `--service` : the destination path for promotion is `/environments/<env-name>/services/<service-name>/base/config/`. This argument defines `service-name` in that path.
103+
- `--to`: an https URL to the destination GitOps repository.
83104

84-
## Release process
105+
### Troubleshooting
85106

86-
When a new tag is pushed with the `v` prefix, a GitHub release will be created with binaries produced for 64-bit Linux, and Mac automatically.
107+
- Authentication and authorisation failures: ensure that GITHUB_TOKEN is set and has the necessary permissions.
108+
- 'Failure to commit, Error 128'. Errors of this form are often caused by a failure to set the `commit-email` and `commmit-name` parameters.
109+
- 'Nothing to commit'. If there's no difference between the state of `--from` and `--to` then there's no change to made, and no Git commit can be created.
110+
- Remote branch is created but no Pull Request. Again check GITHUB_TOKEN, and that `--repository-type` is set correctly.
87111

88112
## Experimental plugin section
89113

90114
Inside of the `plugin` folder you'll see documentation and other files related to using the `services` binary as a plugin to `oc`. This has been tested with the following version on OpenShift 4.3:
91115

92-
```
116+
```yaml
93117
Client Version: openshift-clients-4.3.13-202004121622
94118
Server Version: 4.3.13
95119
Kubernetes Version: v1.16.2
96-
```

0 commit comments

Comments
 (0)