You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
To contribute, add the upstream repository as an additional remote and work on a branch other than the main branch. Merge changes into the main branch to align with GitOps workflows. This approach simplifies upstream contributions. Contributions from your forked main branch typically include:
16
+
17
+
. Customizations to `values-global.yaml` and other installation-specific files.
18
+
. Commits made by Tekton and other automated processes unique to your setup.
19
+
20
+
*To isolate changes for upstream contributions*
21
+
22
+
. Run the following command to add the upstream repository as a remote:
. Create a branch named `hcp-main` and track `hcp/main`:
37
+
+
38
+
[source,terminal]
39
+
----
40
+
$ git branch -b hcp-main -t hcp/main
41
+
----
42
+
43
+
. Make your changes on the `hcp-main` branch as needed.
44
+
45
+
. Push your changes to your fork’s `hcp-main` branch:
46
+
47
+
+
48
+
[source,terminal]
49
+
----
50
+
$ git push origin hcp-main
51
+
----
52
+
53
+
*To update `hcp-main` branch with upstream changes*
54
+
55
+
. Run the following command to check out the `hcp-main` branch:
56
+
57
+
+
58
+
[source,terminal]
59
+
----
60
+
$ git checkout hcp-main
61
+
----
62
+
63
+
. Pull the latest changes from the upstream repository and rebase:
64
+
+
65
+
[source,terminal]
66
+
----
67
+
$ git pull --rebase
68
+
----
69
+
70
+
. Reflect these changes in your forked repository (useful for submitting a PR later):
71
+
+
72
+
[source,terminal]
73
+
----
74
+
$ git push origin hcp-main
75
+
----
76
+
77
+
*To integrate upstream pattern changes into your local GitOps process*
78
+
79
+
. Run the following command to check out the main branch:
80
+
+
81
+
[source,terminal]
82
+
----
83
+
$ git checkout main
84
+
----
85
+
86
+
. Merge changes from the `hcp-main` branch into your main branch:
87
+
+
88
+
[source,terminal]
89
+
----
90
+
$ git merge hcp-main
91
+
----
92
+
93
+
. Push the merged changes to your fork’s main branch:
94
+
+
95
+
[source,terminal]
96
+
----
97
+
$ git push origin main
98
+
----
99
+
100
+
This workflow ensures that the `hcp-main` branch:
101
+
102
+
. Remains separate from changes made by your local GitOps processes.
103
+
. Can be merged or cherry-picked into your local main branch for use in GitOps workflows (useful for tracking submodule updates, such as common).
104
+
. Serves as a solid basis for Pull Requests upstream, as it will not include local configuration differences or local GitOps commits.
105
+
106
+
[id="changing-subtrees"]
107
+
== Changing subtrees
108
+
109
+
The Git subtree feature promotes modularity by enabling multiple patterns to share a common base. Over time, functionality is moved into the common directory to isolate pattern-specific components as standard usage practices emerge. This approach strengthens tools in the common directory, adds features, and simplifies the development of new patterns. The common subtree is typically maintained during routine updates, and pulling changes from upstream includes any updates from the common directory.
110
+
111
+
You only need to modify subtrees if you want to test changes in the common area of the pattern repositories or contribute to the common repository alongside one of the patterns. Using the pattern alone does not require changing subtrees.
112
+
113
+
For most cases involving the use and consumption of the pattern, users do not need to know that the pattern uses a subtree.
Make changes in your fork’s main branch or create a new branch for your modifications.
129
+
130
+
To track these changes in your fork of the pattern repository (example, `industrial-edge`), replace the common subtree with your forked version. To simplify this process use:
This script sets up a new remote in your local working directory with the specified repository. It replaces the common directory with the specified fork and branch, commits the change, but does not push it.
This replaces the common directory in the current repository with the wip-main branch from mhjacks's common repository and names the remote `common-subtree`.
147
+
148
+
To pull changes from mhjacks's wip-main branch into the parent repository:
149
+
150
+
[source,terminal]
151
+
----
152
+
$ git subtree pull --prefix common common-subtree wip-main
153
+
----
154
+
155
+
Running the script without arguments defaults to:
156
+
157
+
[source,terminal]
158
+
----
159
+
$ common/scripts/make_common_subtree.sh https://github.com/validatedpatterns/common.git main common-subtree
160
+
----
161
+
162
+
These are the default settings for the repository configuration.
163
+
164
+
[id="subtree-compared-to-submodule"]
165
+
== Subtree compared to Submodule
166
+
167
+
Ensuring patterns are easily shareable across multiple projects has been a key priority. While technically possible, sharing changes between unrelated Git repositories often involves a manual and error-prone process. The goal is to provide a seamless "pull" experience, where a single git pull action updates shared pattern components. For repository sharing, two main strategies were considered: submodules and subtrees. Initially, submodules were used, but the approach later transitioned to subtrees.
168
+
169
+
Subtrees integrate the history of another repository into a parent repository, offering many benefits of submodules without most of their drawbacks. Atlassian provides useful documentation on subtrees, which explains their advantages in more detail. For more information see, link:https://www.atlassian.com/blog/developer[Developer's blog] and link:https://www.atlassian.com/git/tutorials/git-subtree[Git subtree].
170
+
171
+
Submodules presented unavoidable challenges, such as:
172
+
173
+
* Remembering to check out repositories with `--recurse-submodules` or running `git submodule init && git submodule sync`. The common directory often appeared empty without these steps.
174
+
* Ensuring compatibility between submodule-based repositories and other tools. While tools like ArgoCD and Tekton Pipelines supported submodules well, compatibility concerns remained.
175
+
* Switching branches where different revisions of submodules were referenced often resulted in out-of-sync states. This behavior, although is correct, could be confusing.
176
+
* Submodules required mirroring more repositories in disconnected environments.
177
+
* Working with forked submodules meant maintaining two separate repositories and multiple branches in each.
178
+
179
+
Subtrees also have challenges. For example, it is easier to diverge from the upstream version of the subtree, and users may not realize that a subtree is in use when cloning a repository. This can be considered a feature, but might lead to conflicts when updating to a newer version. Additionally, subtrees are less commonly understood, which can result in unexpected issues, such as:
180
+
181
+
182
+
* Cherry picking from a subtree commit into the parent puts the change in the parent location, not the subtree.
After forking the common repository and modifying your subtree for testing, you can propose changes to [https://github.com/validatedpatterns/common.git] for integration into other patterns. Making changes to the upstream common for a specific pattern involves two steps:
188
+
189
+
. Submit a PR to update the upstream common.
190
+
. Submit a PR to update the pattern repository with the modified common.
These patterns are designed to be composed of multiple components, and for those components to be used in gitops
15
-
workflows by consumers and contributors. To use the first pattern as an example, we maintain the link:/industrial-edge[Industrial Edge] pattern, which uses a https://github.com/validatedpatterns/industrial-edge[repo] with pattern-specific logic and configuration as well as a https://github.com/validatedpatterns/common[common repo] which has elements common to multiple patterns. The common repository is included in each pattern repository as a subtree.
14
+
Patterns are designed to be composed of multiple components, that consumers and contributors can integrate into GitOps workflows. For instance, the link:/industrial-edge[Industrial Edge] pattern comprises a https://github.com/validatedpatterns/industrial-edge[repository] with pattern-specific logic and configuration, alongside a https://github.com/validatedpatterns/common[common repository] containing elements common to various patterns. This common repository is included as a subtree within each pattern repository, streamlining consistency and reusability across different patterns.
15
+
16
16
17
17
[id="consuming-a-pattern"]
18
18
== Consuming a pattern
19
19
20
-
. Fork the pattern repository on GitHub to your workspace (GitHub user or organization). It is necessary to fork because your fork will be updated as part of the GitOps and DevOps processes, and the main branch (by default) will be used in the automated workflows.
21
-
. Clone the forked copy
20
+
. Fork the pattern repository on GitHub to your workspace (GitHub user or organization). Forking is necessary because your fork updates as part of the GitOps and DevOps processes, and the main branch (by default) is used in the automated workflows.
To update branch `hcp-main` with upstream changes:
68
-
69
-
[source,terminal]
70
37
----
71
-
$ git checkout hcp-main
72
-
$ git pull --rebase
73
-
----
74
-
75
-
To reflect these changes in your forked repository (such as if you would like to submit a PR later):
76
-
77
-
[source,terminal]
78
-
----
79
-
$ git push origin hcp-main
80
-
----
81
-
82
-
If you want to integrate upstream pattern changes into your local GitOps process:
83
-
84
-
[source,terminal]
85
-
----
86
-
$ git checkout main
87
-
$ git merge hcp-main
88
-
$ git push origin main
89
-
----
90
-
91
-
Using this workflow, the `hcp-main` branch will:
92
-
93
-
. Be isolated from any changes that are being made by your local GitOps processes
94
-
. Be merge-able (or cherry-pick-able) into your local main branch to be used by your local GitOps processes
95
-
(this is especially useful for tracking when any submodules, like common, update)
96
-
. Be a good basis for submitting Pull Requests to be integrated upstream, since it will not contain your local configuration differences or your local GitOps commits
97
-
98
-
[id="changing-subtrees"]
99
-
== Changing subtrees
100
-
101
-
Our patterns use the git subtree feature as a mechanism to promote modularity, so that multiple patterns can use the
102
-
same common basis. Over time we will move more functionality into common, to isolate the components that are
103
-
particular to each pattern, and standard usage conventions emerge. This will make the tools in common more powerful and featureful, and make it easier to develop new patterns. Normally, we will maintain the common subtree in the normal course of updates, and pulling changes from upstream will include any changes from common.
104
-
105
-
You only need to change subtrees if you want to test changes in the common/ area of the pattern repositories, or if you wish to contribute to the common/ repository itself in conjunction with one of the patterns. Using the pattern by itself _does not_ require changing subtrees.
106
-
107
-
For the common cases (use and consumption of the pattern), users do not need to be aware that the pattern uses a subtree at all.
Now, you can make changes in your fork's main branch, or else make a new branch and make changes there.
122
-
123
-
If you want to track these changes in your fork of the _pattern_ repository (industrial-edge in this case), you will need to swap out the subtree in industrial-edge for the version of common you forked. We have provided a script to make this a bit easier:
38
+
$ vi ~/values-secret.yaml
39
+
----
124
40
41
+
. Customize the deployment for your cluster. Customization involves updating the `values-global.yaml` file with configurations specific to your cluster environment.
This script will set up a new remote in your local working directory with the repository you specify. It will replace the common directory with a new common from the fork and branch you specify, and commit it. The script will _not_ push the result.
131
-
132
-
For example:
133
-
49
+
.. Commit the updated `values-global.yaml` file to the repository:
This will replace common in the current repository with the wip-main branch from the common in mhjacks's common repository, and call the remote common-subtree.
140
-
141
-
From that point, changes from mhjacks's wip-main branch on mhjacks's fork of common can be pulled in this way:
142
-
56
+
.. Push the committed changes:
57
+
+
143
58
[source,terminal]
144
-
----
145
-
$ git subtree pull --prefix common common-subtree wip-main
146
-
----
147
-
148
-
When run without arguments, the script will run as if it had been given the following arguments:
149
59
150
-
[source,terminal]
151
60
----
152
-
$ common/scripts/make_common_subtree.sh https://github.com/validatedpatterns/common.git main common-subtree
61
+
$ git push
153
62
----
154
63
155
-
Which are the defaults the repository is normally configured with.
156
-
157
-
[id="subtree-vs-submodule"]
158
-
== Subtree vs. Submodule
159
-
160
-
It has always been important to us to be have a substrate for patterns that is as easy as possible to share amongst
161
-
multiple patterns. While it is possible to share changes between multiple unrelated git repositories, it is an almost
162
-
entirely manual process, prone to error. We feel it is important to be able to provide a "pull" experience (i.e. one git "pull" type action) to update the shared components of a pattern. Two strategies exist for repository sharing in this way: submodule and subtree. We started with submodules but have since moved to subtree.
163
-
164
-
Atlassian has some good documentation on what subtree is https://blog.developer.atlassian.com/the-power-of-git-subtree/[here] and https://www.atlassian.com/git/tutorials/git-subtree[here]. In short, a subtree integrates another repository's history into a parent repository, which allows for most of the benefits of a submodule workflow, without most of the caveats.
165
-
166
-
Earlier versions of this document described the usage of patterns with submodules instead of subtrees. In the earliest stages of pattern development, we used submodules because the developers of the project were familiar with submodules and had used them previously, but we had not used subtrees. User feedback, as well as some of the unavoidable complexities of submodules, convinced us to try subtrees and we believe we will stick with that strategy. Some of the unavoidable complexities of submodules include:
167
-
168
-
* Having to remember to checkout repositories with `--recurse-submdules`, or else doing `git submodule init && git submodule sync`. Experienced developers asked in several of our support channels early on why common was empty.
169
-
* Hoping that other tools that are interacting with the repository are compatible with the submodule approach. (To be fair, tools like ArgoCD and Tekton Pipelines did this very well; their support of submodules was one of the key reasons we started with submodules)
170
-
* When changing branches on a submoduled repository, if the branch you were changing to was pointed to a different revision of the submoduled repository, the repository would show out of sync. While this behavior is correct, it can be surprising and difficult to navigate.
171
-
* In disconnected environments, submodules require mirroring more repositories.
172
-
* Developing with a fork of the submoduled repository means maintaining two forked repositories and multiple branches in both.
173
-
174
-
Subtrees have some pitfalls as well. In the subtree strategy, it is easier to diverge from the upstream version of the subtree repository, and in fact with a typical `git clone`, the user may not be aware that a subtree is in use at all. This can be considered a feature, but could become problematic if the user/consumer later wants to update to a newer version of the subtree but local changes might conflict. Additionally, since subtrees are not as well understood generally, there can be some surprising effects. In practice, we have run into the following:
175
-
176
-
* Cherry picking from a subtree commit into the parent puts the change in the parent location, not the subtree
Once you have forked common and changed your subtree for testing, changes from your fork can then be proposed to [https://github.com/validatedpatterns/common.git] and can then be integrated into other patterns. A change to upstream common for a particular upstream pattern would have to be done in two stages:
182
-
183
-
. PR the change into upstream's common
184
-
. PR the updated common into the pattern repository
0 commit comments