|
1 | 1 | = Templates for Common Files in Operator Repositories
|
2 | 2 |
|
3 |
| -== Purpose |
4 | 3 | Stackable develops and maintains a growing number of operators for open source software.
|
5 | 4 |
|
6 | 5 | The high level structure of the repository is consistent across all repositories, which in the past has led to repetitive maintenance tasks in many repositories even for small changes.
|
7 | 6 |
|
8 | 7 | This repository is intended to help with these changes by keeping common files as templates and offering tooling to roll these out to all repositories that are under management by this tool.
|
9 | 8 |
|
10 |
| -== Using |
11 |
| -=== Via Github Actions |
12 |
| -==== Authentication |
13 |
| -Since this tool needs to authenticate itself in order to push commits it needs credentials of a user. |
14 |
| -This is currently solved via a personal access token that needs to be provided as a repository (or org) secret with the name `GH_ACCESS_TOKEN`. |
| 9 | +== Structure of this repository |
15 | 10 |
|
16 |
| -The personal access token needs to have the following permissions for this to work: |
| 11 | +=== GitHub Actions |
17 | 12 |
|
18 |
| -- repo |
19 |
| -- org:read |
20 |
| -- workflow |
| 13 | +The definition files for GitHub actions that in turn execute the playbooks doing the actual work are kept in `.github` |
| 14 | + |
| 15 | +=== Playbook |
| 16 | + |
| 17 | +`playbook` contains the ansible playbook which is executed to perform the needed changes. |
| 18 | + |
| 19 | +=== Templates |
| 20 | + |
| 21 | +Everything under the top level folder `template` is replicated to the target repositories. |
| 22 | +The folder and file structure under `template` is considered as source structure which will be synchronized across all repositories in the following manner: |
| 23 | + |
| 24 | +* Regular files and folders are simply copied |
| 25 | +* Files with an extension of `.j2` will be processed as jinja2 templates and copied to the target repositories with the `.j2` extension removed. The default jinja2 variable delimiter has been replaced with `{[ }]` since a lot of the template files contain `{{ }}` and caused issues. |
| 26 | +* File and directory names which contain `\[[product]]` will have this substituted for the value of the `product_string` variable. For example: "stackable-\[[product]]-operator.service" would become `stackable-kafka-operator.service`. |
| 27 | + |
| 28 | +To remove files or directories that already exist in the target repositories these need to be configured in `repositories.yaml` under the `retired_files` key. |
| 29 | + |
| 30 | +Anything that is listed here will be deleted from the target repositories. |
| 31 | + |
| 32 | +NOTE: Deletion is the last step that is performed, so if there is an overlap between files existing in the template folder and this setting, the files would not be rolled out, since they'd get deleted before creting the pull request. |
| 33 | + |
| 34 | + |
| 35 | +=== Configuration |
| 36 | + |
| 37 | +All user-facing configuration is kept in `repositories.yaml`. |
21 | 38 |
|
22 |
| -==== Configuring repositories |
23 | 39 | The actions in this repository are set up so that the playbooks are automatically executed whenever a commit is made to the `main` branch of this repository.
|
24 | 40 |
|
25 | 41 | So in principle it is as simple as pushing any commit to `main` which will trigger a sync of everything under `template` to all target repositories.
|
@@ -55,55 +71,44 @@ Target repositories are configured in `repositories.yaml` in the following form:
|
55 | 71 | |===
|
56 | 72 |
|
57 | 73 |
|
58 |
| - |
59 | 74 | These are the only variables currently being used on the playbooks, but can be extended easily as more are needed.
|
60 | 75 |
|
61 | 76 | NOTE: If a new variable is introduced, it needs to be added to all repository objects!
|
62 | 77 |
|
63 |
| -==== Templating |
64 |
| -The folder and file structure under `template` is considered as source structure which will be synchronized across all repositories in the following manner: |
| 78 | +Additional settings can be found in `playbook/group_vars/all`, but these are not intended to be freely changed and should be treated with care. |
65 | 79 |
|
66 |
| -* Regular files and folders are simply copied |
67 |
| -* Files with an extension of `.j2` will be processed as jinja2 templates and copied to the target repositories with the `.j2` extension removed. The default jinja2 variable delimiter has been replaced with `{[ }]` since a lot of the template files contain `{{ }}` and caused issues. |
68 |
| -* File and directory names which contain `\[[product]]` will have this substituted for the value of the `product_string` variable. For example: "stackable-\[[product]]-operator.service" would become `stackable-kafka-operator.service`. |
69 | 80 |
|
70 |
| -To remove files or directories that already exist in the target repositories these need to be configured in `repositories.yaml` under the `retired_files` key. |
| 81 | +== Making changes to the template |
71 | 82 |
|
72 |
| -Anything that is listed here will be deleted from the target repositories. |
| 83 | +If you want to make a change that should be rolled out to all operators, make the change in the `template` directory. |
| 84 | +Consult the section above to learn more about the structure of the template. |
73 | 85 |
|
74 |
| -NOTE: Deletion is the last step that is performed, so if there is an overlap between files existing in the template folder and this setting, the files would not be rolled out, since they'd get deleted before creting the pull request. |
| 86 | +=== Test changes locally |
75 | 87 |
|
76 |
| -=== Local |
77 |
| -While this can in theory also be used locally this has not yet been tested and may or may not be documented and supported here in the future. |
| 88 | +1. Create the directory `work` and run the `test.sh` script. |
| 89 | +2. The changes can be examined with `git status`. |
| 90 | + When the pull request is later merged into the `main` branch then pull requests with these changes will be created automatically. |
| 91 | +3. Depending on the change, it makes sense to run the integration tests for all changed operators. |
| 92 | + If the tests are not run in this stage and if there is even just one integration test failing in the subsequential generated pull requests then the operator-templating must be adapted which creates again pull requests for all operators. |
| 93 | + Changes in the GitHub workflow actions cannot be tested until finally merged. |
78 | 94 |
|
79 |
| -What is supported is locally testing that all templates are rendered correctly. |
| 95 | +== Deploying changes |
80 | 96 |
|
81 |
| -The process here is to manually check out all operators that should be tested into the `work` subdirectory and then running `test.sh`. |
| 97 | +Changes are rolled out via GitHub actions. |
82 | 98 |
|
83 |
| -This will perform templating into the subdirectories, which can then be checked via `git status` or similar methods. |
| 99 | +=== Authentication |
| 100 | +Since this tool needs to authenticate itself in order to push commits it needs credentials of a user. |
| 101 | +This is currently solved via a personal access token that needs to be provided as a repository (or org) secret with the name `GH_ACCESS_TOKEN`. |
| 102 | + |
| 103 | +The personal access token needs to have the following permissions for this to work: |
| 104 | + |
| 105 | +- repo |
| 106 | +- org:read |
| 107 | +- workflow |
84 | 108 |
|
85 | 109 | == Limitations
|
86 | 110 | There is currently no synchronization with existing PRs on the target repositories whatsoever. A new pull request will be created for every commit made to this repository.
|
87 | 111 |
|
88 | 112 | To update a PR that was created via this tool, it will have to be closed and necessary changes pushed here, which will result in a new PR.
|
89 | 113 |
|
90 | 114 | WARNING: The Helm Chart files that are rolled out by the templates in their current form do not include a ClusterRole object which may be needed for this to work with RBAC.
|
91 |
| - |
92 |
| -== Structure |
93 |
| -**Github Actions** |
94 |
| - |
95 |
| -The definition files for github actions that in turn execute the playbooks doing the actual work are kept in `.github` |
96 |
| - |
97 |
| -**Playbook** |
98 |
| - |
99 |
| -`playbook` contains the ansible playbook which is executed to perform the needed changes. |
100 |
| - |
101 |
| -**Templates** |
102 |
| - |
103 |
| -Everything under the top level folder `template` is replicated to the target repositories. |
104 |
| - |
105 |
| -**Configuration** |
106 |
| - |
107 |
| -All user-facing configuration is kept in `repositories.yaml`. |
108 |
| - |
109 |
| -Additional settings can be found in `playbook/group_vars/all`, but these are not intended to be freely changed and should be treated with care. |
|
0 commit comments