Skip to content

Commit 6f4014d

Browse files
camilamacedo86camilamacedo86
committed
Proposal: Automating maintenance via tooling and actions
1 parent 9abbd42 commit 6f4014d

File tree

1 file changed

+338
-0
lines changed

1 file changed

+338
-0
lines changed

‎designs/update_action.md

Lines changed: 338 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,338 @@
1+
| Authors | Creation Date | Status | Extra |
2+
|-----------------|---------------|-------------|-------|
3+
| @camilamacedo86 | 2024-11-07 | Implementable | - |
4+
5+
# Proposal: Automating Operator Maintenance: Driving Better Results with Less Overhead
6+
7+
## Introduction
8+
9+
Code-generation tools like **Kubebuilder** and **Operator-SDK** have revolutionized cloud-native application development by providing scalable, community-driven frameworks. These tools simplify complexity, accelerate development, and enable developers to create tailored solutions while avoiding common pitfalls, establishing a strong foundation for innovation.
10+
11+
However, as these tools evolve to keep up with ecosystem changes and new features, projects risk becoming outdated. Manual updates are time-consuming, error-prone, and create challenges in maintaining security, adopting advancements, and staying aligned with modern standards.
12+
13+
This project proposes an **automated solution for Kubebuilder**, with potential applications for similar tools or those built on its foundation. By streamlining maintenance, projects remain modern, secure, and adaptable, fostering growth and innovation across the ecosystem. The automation lets developers focus on what matters most: **building great solutions**.
14+
15+
### Initial Approach
16+
The initial approach to solving this problem is through **three-way Git merges**, ensuring updates are applied
17+
while maintaining customizations. In the future, **AI-assisted conflict resolution** may be explored to further enhance the process.
18+
19+
## Problem Statement
20+
21+
Kubebuilder is widely used for developing Kubernetes operators, providing a standardized scaffold. However, as the ecosystem evolves, keeping projects up-to-date presents challenges due to:
22+
23+
- **Manual re-scaffolding processes**: These are time-intensive and error-prone.
24+
- **Increased risk of outdated configurations**: Leads to security vulnerabilities and incompatibility with modern practices.
25+
26+
## Proposed Solution
27+
28+
This proposal introduces a **workflow-based tool** (such as a GitHub Action) that automates updates for Kubebuilder projects. Whenever a new version of Kubebuilder is released, the tool initiates a workflow that:
29+
30+
1. **Detects the new release**.
31+
2. **Generates an updated scaffold**.
32+
3. **Performs a three-way merge to retain customizations**.
33+
4. **Creates a pull request (PR) summarizing the updates** for review and merging.
34+
35+
## Example Usage
36+
37+
### GitHub Actions Workflow:
38+
39+
1. A user creates a project with Kubebuilder `v4.4.3`.
40+
2. When Kubebuilder `v4.5.0` is released, a **pull request** is automatically created.
41+
3. The PR includes scaffold updates while preserving the user’s customizations, allowing easy review and merging.
42+
43+
### Local Tool Usage:
44+
45+
1. A user creates a project with Kubebuilder `v4.4.3`
46+
2. When Kubebuilder `v4.5.0` is released, they run the tool locally.
47+
3. The tool updates the scaffold and preserves customizations for review and application.
48+
49+
### Handling Merge Conflicts
50+
51+
If conflicts cannot be resolved automatically, developers can manually address them before completing the update.
52+
53+
## Open Questions
54+
55+
### 1. Do we need to create branches to perform the three-way merge, or can we use local temporary directories?
56+
57+
> While temporary directories are sufficient for simple three-way merges, branches are better suited for complex scenarios. They provide history tracking, support collaboration, integrate with CI/CD workflows, and offer more advanced conflict resolution through Git’s merge command. For these reasons, it seems more appropriate to use branches to ensure flexibility and maintainability in the merging process.
58+
59+
### 2. What Git configuration options can facilitate the three-way merge?
60+
61+
Several Git configuration options can improve the three-way merge process:
62+
63+
```bash
64+
# Show all three versions (base, current, and updated) during conflicts
65+
git config --global merge.conflictStyle diff3
66+
67+
# Enable "reuse recorded resolution" to remember and reuse previous conflict resolutions
68+
git config --global rerere.enabled true
69+
70+
# Increase the rename detection limit to better handle renamed or moved files
71+
git config --global merge.renameLimit 999999
72+
73+
# Set up custom merge drivers for specific file types (e.g., YAML or JSON)
74+
git config --global merge.<driver>.name "Custom Merge Driver"
75+
```
76+
77+
These configurations enhance the merging process by improving conflict visibility, reusing resolutions, and providing better file handling, making three-way merges more efficient and developer-friendly.
78+
79+
### 3. If we change Git configurations, can we isolate these changes to avoid affecting the local developer environment when the tool runs locally?
80+
81+
It seems that changes can be made using the `-c` flag, which applies the configuration only for the duration of a specific Git command. This ensures that the local developer environment remains unaffected.
82+
83+
For example:
84+
85+
```
86+
git -c merge.conflictStyle=diff3 -c rerere.enabled=true merge
87+
```
88+
89+
### 4. How can we minimize and resolve conflicts effectively during merges?
90+
91+
- **Enable Git Features:**
92+
- Use `git config --global rerere.enabled true` to reuse previous conflict resolutions.
93+
- Configure custom merge drivers for specific file types (e.g., `git config --global merge.<driver>.name "Custom Merge Driver"`).
94+
95+
- **Encourage Standardization:**
96+
- Adopt a standardized scaffold layout to minimize divergence and reduce conflicts.
97+
98+
- **Apply Frequent Updates:**
99+
- Regularly update projects to avoid significant drift between the scaffold and customizations.
100+
101+
These strategies help minimize conflicts and simplify their resolution during merges.
102+
103+
### 5. How to create the PR with the changes for projects that are monorepos?
104+
That means the result of Kubebuilder is not defined in the root dir and might be in other paths.
105+
106+
We can define an `--output` directory and a configuration for the GitHub Action where users will define where in their repo the path for the Kubebuilder project is. However, this might be out of scope for the initial version.
107+
108+
### 6. How could AI help us solve conflicts? Are there any available solutions?
109+
110+
> <TODO>
111+
112+
### 7. Could GitHub Copilot help solve conflicts? Does it provide an API we can leverage?
113+
114+
> <TODO>
115+
116+
## Summary
117+
118+
### Workflow Example:
119+
120+
1. A developer creates a project with Kubebuilder `v4.4`.
121+
2. The tooling uses the release of Kubebuilder `v4.5`.
122+
3. The tool:
123+
- Regenerates the original base source code for `v4.4` using the `clientVersion` in the `PROJECT` file.
124+
- Generates the base source code for `v4.5`
125+
4. A three-way merge integrates the changes into the developer’s project while retaining custom code.
126+
5. The changes now can be packaged into a pull request, summarizing updates and conflicts for the developer’s review.
127+
128+
### Steps:
129+
130+
The proposed implementation involves the following steps:
131+
132+
1. **Version Tracking**:
133+
- Record the `clientVersion` (initial Kubebuilder version) in the `PROJECT` file.
134+
- Use this version as a baseline for updates.
135+
- [More info](https://github.com/kubernetes-sigs/kubebuilder/issues/4398)
136+
137+
2. **Scaffold Generation**:
138+
- Generate the **original scaffold** using the recorded version.
139+
- Generate the **updated scaffold** using the latest Kubebuilder release.
140+
141+
3. **Three-Way Merge**:
142+
- Ensure git is configured to handle three-way merges.
143+
- Merge the original scaffold, updated scaffold, and the user’s customized project.
144+
- Preserve custom code during the merge.
145+
146+
4. **(For Actions) - Pull Request Creation**:
147+
- Open a pull request summarizing changes, including details on conflict resolution.
148+
- Schedule updates weekly or provide an on-demand option.
149+
150+
#### Example Workflow
151+
152+
The following example code illustrates the proposed idea but has not been evaluated.
153+
This is an early, incomplete draft intended to demonstrate the approach and basic concept.
154+
155+
We may want to develop a dedicated command-line tool, such as `kubebuilder alpha update`,
156+
to handle tasks like downloading binaries, merging, and updating the scaffold. In this approach,
157+
the GitHub Action would simply invoke this tool to manage the update process and open the
158+
Pull Request, rather than performing each step directly within the Action itself.
159+
160+
```yaml
161+
162+
name: Update Kubebuilder Scaffold
163+
164+
on:
165+
workflow_dispatch:
166+
schedule:
167+
- cron: '0 0 * * 0' # Run weekly to check for new Kubebuilder versions
168+
169+
jobs:
170+
update-scaffold:
171+
runs-on: ubuntu-latest
172+
173+
steps:
174+
- name: Check out the repository
175+
uses: actions/checkout@v2
176+
with:
177+
fetch-depth: 0 # Ensures the full history is checked out
178+
179+
- name: Set up environment and dependencies
180+
run: |
181+
sudo apt-get update
182+
sudo apt-get install -y jq curl
183+
184+
- name: Read Kubebuilder version from PROJECT file
185+
id: read_version
186+
run: |
187+
export INITIAL_VERSION=$(grep "clientVersion" PROJECT | awk '{print $2}')
188+
echo "::set-output name=initial_version::$INITIAL_VERSION"
189+
190+
- name: Download and install the initial Kubebuilder version
191+
run: |
192+
curl -L https://github.com/kubernetes-sigs/kubebuilder/releases/download/${{ steps.read_version.outputs.initial_version }}/kubebuilder_${{ steps.read_version.outputs.initial_version }}_linux_amd64.tar.gz -o kubebuilder_initial.tar.gz
193+
tar -zxvf kubebuilder_initial.tar.gz
194+
sudo mv kubebuilder /usr/local/kubebuilder_initial
195+
196+
- name: Generate initial scaffold in `scaffold_initial` directory
197+
run: |
198+
mkdir scaffold_initial
199+
cp -r . scaffold_initial/
200+
cd scaffold_initial
201+
/usr/local/kubebuilder_initial/bin/kubebuilder init
202+
cd ..
203+
204+
- name: Check for the latest Kubebuilder release
205+
id: get_latest_version
206+
run: |
207+
export LATEST_VERSION=$(curl -s https://api.github.com/repos/kubernetes-sigs/kubebuilder/releases/latest | jq -r .tag_name)
208+
echo "::set-output name=latest_version::$LATEST_VERSION"
209+
210+
- name: Download and install the latest Kubebuilder version
211+
run: |
212+
curl -L https://github.com/kubernetes-sigs/kubebuilder/releases/download/${{ steps.get_latest_version.outputs.latest_version }}/kubebuilder_${{ steps.get_latest_version.outputs.latest_version }}_linux_amd64.tar.gz -o kubebuilder_latest.tar.gz
213+
tar -zxvf kubebuilder_latest.tar.gz
214+
sudo mv kubebuilder /usr/local/kubebuilder_latest
215+
216+
- name: Generate updated scaffold in `scaffold_updated` directory
217+
run: |
218+
mkdir scaffold_updated
219+
cp -r . scaffold_updated/
220+
cd scaffold_updated
221+
/usr/local/kubebuilder_latest/bin/kubebuilder init
222+
cd ..
223+
224+
- name: Copy current project into `scaffold_current` directory
225+
run: |
226+
mkdir scaffold_current
227+
cp -r . scaffold_current/
228+
229+
- name: Perform three-way merge with scaffolds
230+
run: |
231+
# Create a temporary directory to hold the final merged version
232+
mkdir merged_scaffold
233+
# Run three-way merge using scaffold_initial, scaffold_current, and scaffold_updated
234+
# Adjusting merge strategy and paths to use directories
235+
diff3 -m scaffold_current scaffold_initial scaffold_updated > merged_scaffold/merged_files
236+
237+
- name: Copy merged files back to main directory
238+
run: |
239+
cp -r merged_scaffold/* .
240+
git add .
241+
git commit -m "Three-way merge with Kubebuilder updates and custom code"
242+
243+
- name: Create Pull Request
244+
uses: peter-evans/create-pull-request@v3
245+
with:
246+
commit-message: "Update scaffold to Kubebuilder ${{ steps.get_latest_version.outputs.latest_version }}"
247+
title: "Update scaffold to Kubebuilder ${{ steps.get_latest_version.outputs.latest_version }}"
248+
body: |
249+
This pull request updates the scaffold with the latest Kubebuilder version ${{ steps.get_latest_version.outputs.latest_version }}.
250+
branch: kubebuilder-update-${{ steps.get_latest_version.outputs.latest_version }}
251+
```
252+
253+
## Motivation
254+
255+
A significant challenge faced by Kubebuilder users is keeping their projects up-to-date with the latest
256+
scaffolds while preserving customizations. The manual processes required for updates are time-consuming,
257+
error-prone, and often discourage users from adopting new versions, leading to outdated and insecure projects.
258+
259+
The primary motivation for this proposal is to simplify and automate the process of maintaining Kubebuilder
260+
projects. By providing a streamlined workflow for updates, this solution ensures that users can keep
261+
their projects aligned with modern standards while retaining their customizations.
262+
263+
### Goals
264+
265+
- **Automate Updates**: Detect and apply scaffold updates while preserving customizations.
266+
- **Simplify Updates**: Generate pull requests for easy review and merging.
267+
- **Provide Local Tooling**: Allow developers to run updates locally with preserved customizations.
268+
- **Keep Projects Current**: Ensure alignment with the latest scaffold improvements.
269+
- **Minimize Disruptions**: Enable scheduled or on-demand updates.
270+
271+
### Non-Goals
272+
273+
- **Automating conflict resolution for heavily customized projects**.
274+
- **Automatically merging updates without developer review**.
275+
- **In Phase 1, supporting monorepo project layouts or handling repositories that contain more than just the Kubebuilder-generated code**.
276+
277+
## Proposal
278+
279+
### User Stories
280+
281+
- **As a Kubebuilder maintainer**, I want to help users keep their projects updated with minimal effort, ensuring they adhere to best practices and maintain alignment with project standards.
282+
- **As a user of Kubebuilder**, I want my project to stay up-to-date with the latest scaffold best practices while preserving customizations.
283+
- **As a user of Kubebuilder**, I want an easy way to apply updates across multiple repositories, saving time on manual updates.
284+
- **As a user of Kubebuilder**, I want to ensure my codebases remain secure and maintainable without excessive manual effort.
285+
286+
### Implementation Details/Notes/Constraints
287+
288+
- Introduce a new [Kubebuilder Plugin](https://book.kubebuilder.io/plugins/plugins) that scaffolds the
289+
**GitHub Action** based on the POC. This plugin will be released as an **alpha feature**,
290+
allowing users to opt-in for automated updates.
291+
292+
> <TODO>
293+
294+
## Risks and Mitigations
295+
- **Risk**: Frequent conflicts may make the process cumbersome.
296+
- *Mitigation*: Provide clear conflict summaries and leverage GitHub preview tools.
297+
- **Risk**: High maintenance overhead.
298+
- *Mitigation*: Build a dedicated command-line tool (`kubebuilder alpha update`) to streamline updates and minimize complexity.
299+
300+
## Proof of Concept
301+
302+
The feasibility of re-scaffolding projects has been demonstrated by the `kubebuilder alpha generate` command.
303+
304+
**Command Example:**
305+
306+
```bash
307+
kubebuilder alpha generate
308+
```
309+
310+
For more details, refer to the [Alpha Generate Documentation](https://kubebuilder.io/reference/rescaffold).
311+
312+
This command allows users to manually re-scaffold a project, to allow users add their code on top.
313+
It confirms the technical capability of regenerating and updating scaffolds effectively.
314+
315+
This proposal builds upon this foundation by automating the process. The proposed tool would extend this functionality
316+
to automatically update projects with new scaffold versions, preserving customizations.
317+
318+
The three-way merge approach is a common strategy for integrating changes from multiple sources.
319+
It is widely used in version control systems to combine changes from a common ancestor with two sets of modifications.
320+
In the context of this proposal, the three-way merge would combine the original scaffold, the updated scaffold, and the user’s custom code
321+
seems to be very promising.
322+
323+
<TODO: Add POC>
324+
325+
## Drawbacks
326+
327+
- **Frequent Conflicts:** Automated updates may often result in conflicts, making the process cumbersome for users.
328+
- **Complex Resolutions:** If conflicts are hard to review and resolve, users may find the solution impractical.
329+
- **Maintenance Overhead:** The implementation could become too complex for maintainers to develop and support effectively.
330+
331+
## Alternatives
332+
333+
- **Manual Update Workflow**: Continue with manual updates where users regenerate
334+
and merge changes independently, though this is time-consuming and error-prone.
335+
- **Use alpha generate command**: Continue with updates partial automated provided
336+
by the alpha generate command.
337+
- **Dependabot Integration**: Leverage Dependabot for dependency updates, though this
338+
doesn’t fully support scaffold updates and could lead to incomplete upgrades.

0 commit comments

Comments
 (0)