Skip to content

Commit 9476007

Browse files
camilamacedo86camilamacedo86
authored
📖 design doc with proposal to re-scaffold projects from scratch (#3221)
1 parent ef3c017 commit 9476007

File tree

2 files changed

+213
-1
lines changed

2 files changed

+213
-1
lines changed

‎designs/helper_to_upgrade_projects_by_rescaffolding.md

Lines changed: 212 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,212 @@
1+
| Authors | Creation Date | Status | Extra |
2+
|------------------------------------|---------------|-------------|---|
3+
| @camilamacedo86,@Kavinjsir,@varshaprasad96 | Feb, 2023 | Implementable | - |
4+
5+
Experimental Helper to upgrade projects by re-scaffolding
6+
===================
7+
8+
This proposal aims to provide a new alpha command with a helper which
9+
would be able to re-scaffold the project from the scratch based on
10+
the [PROJECT config][project-config].
11+
12+
## Example
13+
14+
By running a command like following, users would be able to re-scaffold the whole project from the scratch using the
15+
current version of KubeBuilder binary available.
16+
17+
```shell
18+
kubebuilder alpha generate [OPTIONS]
19+
```
20+
21+
### Workflows
22+
23+
Following some examples of the workflows
24+
25+
**To update the project with minor changes provided**
26+
27+
See that for each KubeBuilder release the plugins versions used to scaffold
28+
the projects might have bug fixes and new incremental features added to the
29+
templates which will result in changes to the files that are generated by
30+
the tool for new projects.
31+
32+
In this case, you used previously the tool to generate the project
33+
and now would like to update your project with the latest changes
34+
provided for the same plugin version. Therefore, you will need to:
35+
36+
- Download and install KubeBuilder binary ( latest / upper release )
37+
- You will run the command in the root directory of your project: `kubebuilder alpha generate`
38+
- Then, the command will remove the content of your local directory and re-scaffold the project from the scratch
39+
- It will allow you to compare your local branch with the remote branch of your project to re-add the code on top OR
40+
if you do not use the flag `--no-backup` then you can compare the local directory with the copy of your project
41+
copied to the path `.backup/project-name/` before the re-scaffold be done.
42+
- Therefore, you can run make all and test the final result. You will have after all your project updated.
43+
44+
**To update the project with major changes provided**
45+
46+
In this case, you are looking for to migrate the project from, for example,
47+
`go/v3` to `go/v4`. The steps are very similar to the above ones. However,
48+
in this case you need to inform the plugin that you want to use to do the scaffold
49+
from scratch `kubebuilder alpha generate --plugins=go/v4`.
50+
51+
## Open Questions
52+
53+
N/A
54+
55+
## Summary
56+
57+
Therefore, a new command can be designed to load user configs from the [PROJECT config][project-config] file, and run the corresponding kubebuilder subcommands to generate the project based on the new kubebuilder version. Thus, it makes it easier for the users to migrate their operator projects to the new scaffolding.
58+
59+
## Motivation
60+
61+
A common scenario is to upgrade the project based on the newer Kubebuilder. The recommended (straightforward) steps are:
62+
63+
- a) re-scaffold all files from scratch using the upper version/plugins
64+
- b) copy user-defined source code to the new layout
65+
66+
The proposed command will automate the process at maximum, therefore helping operator authors with minimizing the manual effort.
67+
68+
The main motivation of this proposal is to provide a helper for upgrades and
69+
make less painful this process. Examples:
70+
71+
- See the discussion [How to regenerate scaffolding?](https://github.com/kubernetes-sigs/kubebuilder/discussions/2864)
72+
- From [slack channel By Paul Laffitte](https://kubernetes.slack.com/archives/CAR30FCJZ/p1675166014762669)
73+
74+
### Goals
75+
76+
- Help users upgrade their project with the latest changes
77+
- Help users to re-scaffold projects from scratch based on what was done previously with the tool
78+
- Make less painful the process to upgrade
79+
80+
### Non-Goals
81+
82+
- Change the default layout or how the KubeBuilder CLI works
83+
- Deal with customizations or deviations from the proposed layout
84+
- Be able to perform the project upgrade to the latest changes without human interactions
85+
- Deal and support external plugins
86+
- Provides support to [declarative](https://book.kubebuilder.io/plugins/declarative-v1.html) plugin
87+
since it is desired and planned to decouple this solution and donate this plugin to its own authors [More info](https://github.com/kubernetes-sigs/kubebuilder/issues/3186)
88+
- Provide support to older version before having the Project config (Kubebuilder < 3x) and the go/v2 layout which exists to ensure a backwards compatibility with legacy layout provided by Kubebuilder 2x
89+
90+
## Proposal
91+
92+
The proposed solution to achieve this goal is to create an alpha command as described
93+
in the example section above, see:
94+
95+
```shell
96+
kubebuilder alpha generate \
97+
--input-dir=<path where the PROJECT file can be found>
98+
--output-dir=<path where the project should be re-scaffold>
99+
--no-backup
100+
--backup-path=<path-where the current version of the project should be copied as backup>
101+
--plugins=<chain of plugins key that can be used to create the layout with init sub-command>
102+
```
103+
104+
**Where**:
105+
106+
- input-dir: [Optional] If not informed, then, by default, it is the current directory (project directory). If the `PROJECT` file does not exist, it will fail.
107+
- output-dir: [Optional] If not informed then, it should be the current repository.
108+
- no-backup: [Optional] If not informed then, the current directory should be copied to the path `.backup/project-name`
109+
- backup: [Optional] If not informed then, the backup will be copied to the path `.backup/project-name`
110+
- plugins: [Optional] If not informed then, it is the same plugin chain available in the layout field
111+
- binary: [Optional] If not informed then, the command will use KubeBuilder binary installed globaly.
112+
113+
> Note that the backup created in the current directory must be prefixed with `.`. Otherwise the tool
114+
will not able to perform the scaffold to create a new project from the scratch.
115+
116+
This command would mainly perform the following operations:
117+
118+
- 1. Check the flags
119+
- 2. If the backup flag be used, then check if is a valid path and make a backup of the current project
120+
- 3. Copy the whole current directory to `.backup/project-name`
121+
- 4. Ensure that the output path is clean. By default it is the current directory project where the project was scaffolded previously and it should be cleaned up before to do the re-scaffold.
122+
Only the content under `.backup/project-name` should be kept.
123+
- 4. Read the [PROJECT config][project-config]
124+
- 5. Re-run all commands using the KubeBuilder binary to recreate the project in the output directory
125+
126+
The command should also provide a comprensive help with examples of the proposed workflows. So that, users
127+
are able to understand how to use it when run `--help`.
128+
129+
### User Stories
130+
131+
**As an Operator author:**
132+
133+
- I can re-generate my project from scratch based on the proposed helper, which executes all the
134+
commands according to my previous input to the project. That way, I can easily migrate my project to the new layout
135+
using the newer CLI/plugin versions, which support the latest changes, bug fixes, and features.
136+
- I can regenerate my project from the scratch based on all commands that I used the tool to build
137+
my project previously but informing a new init plugin chain, so that I could upgrade my current project to new
138+
layout versions and experiment alpha ones.
139+
- I would like to re-generate the project from the scratch using the same config provide in the PROJECT file and inform
140+
a path to do a backup of my current directory so that I can also use the backup to compare with the new scaffold and add my custom code
141+
on top again without the need to compare my local directory and new scaffold with any outside source.
142+
143+
**As a Kubebuiler maintainer:**
144+
145+
- I can leverage this helper to easily migrate tutorial projects of the Kubebuilder book.
146+
- I can leverage on this helper to encourage its users to migrate to upper versions more often, making it easier to maintain the project.
147+
148+
### Implementation Details/Notes/Constraints
149+
150+
Note that in the [e2e tests](https://github.com/kubernetes-sigs/kubebuilder/tree/master/test/e2e) the binary is used to do the scaffolds.
151+
Also, very similar to the implementation that exist in the integration test KubeBuilder has
152+
a code implementation to re-generate the samples used in the docs and add customizations on top,
153+
for further information check the [hack/docs](https://github.com/kubernetes-sigs/kubebuilder/tree/master/hack/docs).
154+
155+
This subcommand could have a similar implementation that could be used by the tests and this plugin.
156+
Note that to run the commands using the binaries we are mainly using the following golang implementation:
157+
158+
```go
159+
cmd := exec.Command(t.BinaryName, Options)
160+
_, err := t.Run(cmd)
161+
```
162+
163+
### Risks and Mitigations
164+
165+
**Hard to keep the command maintained**
166+
167+
A risk to consider is that it would be hard to keep this command maintained
168+
because we need to develop specific code operations for each plugin. The mitigation for
169+
this problem could be developing a design more generic that could work with all plugins.
170+
171+
However, initially a more generic design implementation does not appear to be achievable and
172+
would be considered out of the scope of this proposal (no goal). It should to be considered
173+
as a second phase of this implementation.
174+
175+
Therefore, the current achievable mitigation in place is that KubeBuilder's policy of not providing official
176+
support of maintaining and distributing many plugins.
177+
178+
### Proof of Concept
179+
180+
All input data is tracked. Also, as described above we have examples of code implementation
181+
that uses the binary to scaffold the projects. Therefore, the goal of this project seems
182+
very reasonable and achievable. An initial work to try to address this requirement can
183+
be checked in this [pull request](https://github.com/kubernetes-sigs/kubebuilder/pull/3022)
184+
185+
## Drawbacks
186+
187+
- If the value that feature provides does not pay off the effort to keep it
188+
maintained, then we would need to deprecate and remove the feature in the long term.
189+
190+
## Alternatives
191+
192+
N/A
193+
194+
## Implementation History
195+
196+
The idea of automate the re-scaffold of the project is what motivates
197+
us track all input data in to the [project config][project-config]
198+
in the past. We also tracked the [issue](https://github.com/kubernetes-sigs/kubebuilder/issues/2068)
199+
based on discussion that we have to indeed try to add further
200+
specific implementations to do operations per major bumps. For example:
201+
202+
To upgrade from go/v3 to go/v4 we know exactly what are the changes in the layout
203+
then, we could automate these specific operations as well. However, this first idea is harder yet
204+
to be addressed and maintained.
205+
206+
## Future Vision
207+
208+
We could use it to do cool future features such as creating a GitHub action which would push-pull requests against the project repositories to help users be updated with, for example, minor changes. By using this command, we might able to git clone the project and to do a new scaffold and then use some [git strategy merge](https://www.geeksforgeeks.org/merge-strategies-in-git/) to result in a PR to purpose the required changes.
209+
210+
We probably need to store the CLI tool tag release used to do the scaffold to persuade this idea. So that we can know if the project requires or not updates.
211+
212+
[project-config]: https://book.kubebuilder.io/reference/project-config.html

‎pkg/plugin/util/testdata/exampleFile.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
exampleTargetexampleCodeexampleCode
1+
exampleTargetexampleCodeexampleCodeexampleCode

0 commit comments

Comments
 (0)