Skip to content

Commit 6ed8718

Browse files
kcp-ci-botkcp-ci-bot
authored
Merge pull request #3629 from xmudrii/monorepo-docs
Document monorepo structure
2 parents 7338dfe + 981a9e0 commit 6ed8718

File tree

4 files changed

+136
-0
lines changed

4 files changed

+136
-0
lines changed

‎docs/content/contributing/.pages‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ nav:
22
- index.md
33
- getting-started.md
44
- coding.md
5+
- monorepo.md
56
- governance
67
- continuous-integration
78
- guides

‎docs/content/contributing/guides/publishing-a-new-kcp-release.md‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,26 @@ git checkout -b "release-$VERSION" "$REF"
7676
git push "$REMOTE" "release-$VERSION"
7777
```
7878

79+
### Update the `publishing-bot` Rules
80+
81+
The [`publishing-bot` rules](https://github.com/kcp-dev/kcp/blob/main/staging/publishing/rules.yaml) must be updated
82+
to add a new branch for all staging repositories.
83+
84+
While this can be done manually, the easiest and recommended way is to use the `update-rules` tool.
85+
This tool can be installed by cloning kcp's fork of [publishing-bot](https://github.com/kcp-dev/publishing-bot)
86+
and running `make build` in the repository, which will install the tool in `_output/update-rules`.
87+
Alternatively, the tool can be ran inside a Docker container using the `ghcr.io/kcp-dev/publishing-bot:latest` image.
88+
89+
The tool takes the path to the rules file, the branch name, and the Go version used on that branch:
90+
91+
```shell
92+
_output/update-rules -branch release-0.x -go 1.x.y -rules ../kcp/staging/publishing/rules.yaml -o /tmp/rules.yaml
93+
```
94+
95+
This will generate a temporary rules file in `/tmp/rules.yaml`. You'll need to replace `staging/publishing/rules.yaml`
96+
in the kcp repo with that temporary file and push it to the repo before proceeding. The new rules file will be used
97+
on the next publishing-bot run automatically.
98+
7999
## Generate Release Notes
80100

81101
To generate release notes from the information in PR descriptions you should use Kubernetes' [release-notes](https://github.com/kubernetes/release/tree/master/cmd/release-notes) tool.

‎docs/content/contributing/index.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -100,3 +100,11 @@ are two roles: reviewer and approver. Merging a PR requires sign off from both
100100

101101
In general, maintainers strive to pick up PRs for review when they can. If you feel like your PR has been missed,
102102
do not hesitate to ping maintainers directly or ask on the project communication channels about your PR.
103+
104+
### The Monorepo Structure
105+
106+
The core `kcp-dev/kcp` repository is a monorepo containing the kcp core and some close to the core libraries.
107+
See the [monorepo document](./monorepo/) for more details.
108+
109+
When contributing to projects like `kcp-dev/apimachinery`, `kcp-dev/client-go`, and `kcp-dev/code-generator`,
110+
you must create a PR in the `kcp-dev/kcp` repository. Direct PRs to these repositories will be closed.

‎docs/content/contributing/monorepo.md‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
---
2+
description: >
3+
Monorepo structure for kcp projects.
4+
---
5+
6+
# Monorepo Structure
7+
8+
The [kcp-dev/kcp](https://github.com/kcp-dev/kcp) repository is a monorepo that contains the source code for:
9+
10+
- the kcp core
11+
- libraries that are used in the kcp core or are close to the kcp core
12+
13+
At the moment, the following libraries are part of the monorepo:
14+
15+
- [kcp-dev/apimachinery](https://github.com/kcp-dev/apimachinery)
16+
- [kcp-dev/client-go](https://github.com/kcp-dev/client-go)
17+
- [kcp-dev/code-generator](https://github.com/kcp-dev/code-generator)
18+
19+
## Terminology
20+
21+
- The monorepo: the [kcp-dev/kcp](https://github.com/kcp-dev/kcp) repository
22+
- Staging repository: a repository that is part of the monorepo
23+
(located within `kcp-dev/kcp/staging/src/github.com/kcp-dev`)
24+
- Published repository: a standalone repository that's a mirror of the staging repository
25+
26+
Note: the term 'staging repository' does **not** imply a Git repository (with `.git` directory), but an
27+
authoritative place that stores code. Every staging repository is a standalone Go module.
28+
29+
## Similarities with Kubernetes
30+
31+
This pattern is the same as the Kubernetes staging repositories pattern, with an exception of how tags are handled
32+
within published repositories (this is explained later in this document).
33+
34+
## High Level Overview
35+
36+
The source code for staging repositories is located on the special path in the monorepo which is
37+
[`./staging/src/github.com/kcp-dev`](https://github.com/kcp-dev/kcp/tree/main/staging/src/github.com/kcp-dev).
38+
Every subdirectory on that path corresponds to a single staging repository.
39+
The code in the staging/ directory is authoritative, i.e. the only copy of the code. You can directly modify such code.
40+
41+
In order to preserve standalone Go modules for each staging repository, the source code from staging repositories
42+
is mirrored to standalone (aka published) repositories. The mirroring is an automatic process happening every
43+
several hours.
44+
45+
Aside from the source code, the mirroring process includes branches and tags. This means that branches from
46+
the monorepo for which mirroring is enabled will be created in each published repository. Tags on those branches
47+
will be automatically mirrored to the published repositories. At the moment, mirroring is enabled for
48+
the `main` branch and release branches starting from `release-0.29`.
49+
50+
## Handling Tags
51+
52+
Tags published to the published repositories inherit the major version of their original Go module.
53+
In other words, the tag's major version will **not** correspond to the kcp major version.
54+
55+
For example, if we push v0.29.0 in the kcp repository, the tag in the published repository that has Go module major
56+
version v2 will be v2.29.0.
57+
58+
Concretely:
59+
60+
- kcp-dev/client-go has the Go module major version at v0, so tags will be v0.x.y
61+
- kcp-dev/apimachinery has the Go module major version at v2, so tags will be v2.x.y
62+
- kcp-dev/code-generator has the Go module major version at v3, so tags will be v3.x.y
63+
64+
Note: the reason for the speical handling of tags is to ensure the backwards compatibility, as these libraries
65+
already had tags/releases that did not correspond to the kcp versioning.
66+
67+
## Implementation
68+
69+
The mirroring process is implemented via the publishing-bot tool. This tool has been originally created by
70+
the Kubernetes project. We're maintaing a fork of this tool that has been adjusted for our purposes at
71+
[kcp-dev/publishing-bot](https://github.com/kcp-dev/publishing-bot). Overall, the changes made to the original tool
72+
include:
73+
74+
- Add support for running publishing-bot on ARM64
75+
- Add image building process compatible with kcp
76+
- Add support for customizing tags mirroring
77+
- Add kcp-related configuration
78+
79+
Some of these changes will be contributed to the upstream project in the future.
80+
81+
The publishing-bot is running as a Pod in the Prow control plane cluster used by kcp. The status of the bot is
82+
automatically posted to the [`kcp-dev/kcp#3619`](https://github.com/kcp-dev/kcp/issues/3619) issue.
83+
The issue being closed indicates that the bot is healthy and has at least one successful run, while the
84+
issue being open indicates that the bot is not healthy and at least the last run has failed.
85+
86+
The kcp configuration for publishing-bot can be found at https://github.com/kcp-dev/publishing-bot/blob/master/configs/kcp-dev
87+
88+
The mirroring rules are located in the kcp-dev/kcp repoitory and can be found at https://github.com/kcp-dev/kcp/blob/main/staging/publishing/rules.yaml
89+
90+
At the moment, the bot is configured to run every 4 hours.
91+
92+
## Creating a New Staging Repository
93+
94+
Adding a completely new staging repository to the monorepo must follow this process:
95+
96+
- Send an email to the [the developer mailing list](https://groups.google.com/g/kcp-dev) requesting a new staging
97+
repository
98+
- After the request is approved by a simple majority of all Maintainers, create a tracking issue in the
99+
[kcp-dev/kcp](https://github.com/kcp-dev/kcp) repository
100+
- Create a staging repository in `staging/src/github.com/kcp-dev` with all required files for a repository
101+
(e.g. LICENSE, OWNERS...)
102+
- Create a new Git repository on GitHub to serve as a published repository with an empty initial commit
103+
- Update the publishing-bot rules to mirror from the new staging repository to the new published repository
104+
105+
Adding an existing standalone repository to the monorepo is generally discouraged and will be considered on
106+
a case-by-case basis by Maintainers. This case requires following different steps that will be documented
107+
in the future.

0 commit comments

Comments
 (0)