⚠️ go/v4-alpha: change the layout to follow Golang Standards (#2985)

Author: camilamacedo86

docs/book/src/migration/manually_migration_guide_gov3_to_gov4.md

@@ -16,11 +16,17 @@ The recommended upgrade approach is to follow the [Migration Guide go/v3 to go/v
 
 ## Migration from project config version "go/v3" to "go/v4"
 
-Update `PROJECT` file layout which stores the information about the resources are use to enable plugins to make useful decisions when scaffolding.
+Update the `PROJECT` file layout which stores information about the resources that are used to enable plugins make 
+useful decisions while scaffolding. The `layout` field indicates the scaffolding and the primary plugin version in use.
 
-Furthermore, the `PROJECT` file itself is now versioned. The `version` field corresponds to the version of the `PROJECT` file itself, while the `layout` field indicates the scaffolding and the primary plugin version in use.
+### Steps to migrate
+
+#### Migrate the layout version into the PROJECT file
+
+The following steps describe the manual changes required to bring the project configuration file (`PROJECT`). 
+These change will add the information that Kubebuilder would add when generating the file. This file can be found in the root directory.
 
-Update: 
+Update the PROJECT file by replacing:
 
 ```yaml
 layout:
@@ -35,34 +41,150 @@ layout:
 
 ```
 
-### Steps to migrate
+#### Changes to the layout
 
-- Update the `main.go` with the changes which can be found in the samples under testdata for the release tag used. (see for example `testdata/project-v4/main.go`).
-- Update the Makefile with the changes which can be found in the samples under testdata for the release tag used. (see for example `testdata/project-v4/Makefile`)
-- Update the `go.mod` with the changes which can be found in the samples under `testdata` for the release tag used. (see for example `testdata/project-v4/go.mod`). Then, run
-`go mod tidy` to ensure that you get the latest dependencies and your Golang code has no breaking changes.
-- Update the manifest under `config/` directory with all changes performed in the default scaffold done with `go/v4-alpha` plugin. (see for example `testdata/project-v4/config/`) to get all changes in the 
-default scaffolds to be applied on your project
-- Create `config/samples/kustomization.yaml` with all CR samples specified. (see for example `testdata/project-v4/config/samples/kustomization.yaml`)
-- Replace the import `admissionv1beta1 "k8s.io/api/admission/v1beta1"` with `admissionv1 "k8s.io/api/admission/v1"` in the webhook test files
+##### New layout:
+
+- The directory `apis` was renamed to `api` to follow the standard
+- The `controller(s)` directory has been moved under a new directory called `internal` and renamed to singular as well `controller`
+- The `main.go` previously scaffolded in the root directory has been moved under a new directory  called `cmd`
+
+Therefore, you can check the changes in the layout results into:
+
+```sh
+...
+├── cmd
+│ └── main.go
+├── internal
+│ └── controller
+└── api
+```
+
+##### Migrating to the new layout:
+
+- Create a new directory `cmd` and move the `main.go` under it.
+- If your project support multi-group the APIs are scaffold under a directory called `apis`. Rename this directory to `api`
+- Move the `controllers` directory under the `internal` and rename it for `controller` 
+- Now ensure that the imports will be updated accordingly by:
+  - Update the `main.go` imports to look for the new path of your controllers under the `pkg` directory
+
+**Then, let's update the scaffolds paths**
+
+- Update the Dockerfile to ensure that you will have:
+
+```
+COPY cmd/main.go cmd/main.go
+COPY api/ api/
+COPY internal/controller/ internal/controller/
+```
+
+Then, replace:
+
+```
+RUN CGO_ENABLED=0 GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH} go build -a -o manager main.go
+
+```
+
+With:
+
+```
+RUN CGO_ENABLED=0 GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH} go build -a -o manager cmd/main.go
+```
+
+- Update the Makefile targets to build and run the manager by replacing:
+
+```
+.PHONY: build
+build: manifests generate fmt vet ## Build manager binary.
+	go build -o bin/manager main.go
+
+.PHONY: run
+run: manifests generate fmt vet ## Run a controller from your host.
+	go run ./main.go
+```
+
+With:
+
+```
+.PHONY: build
+build: manifests generate fmt vet ## Build manager binary.
+	go build -o bin/manager cmd/main.go
+
+.PHONY: run
+run: manifests generate fmt vet ## Run a controller from your host.
+	go run ./cmd/main.go
+```
+
+- Update the `internal/controller/suite_test.go` to set the path for the `CRDDirectoryPaths`:
+
+Replace:
+
+```
+CRDDirectoryPaths:     []string{filepath.Join("..", "config", "crd", "bases")},
+```
+
+With:
+
+```
+CRDDirectoryPaths:     []string{filepath.Join("..", "..", "config", "crd", "bases")},
+```
+
+Note that if your project has multiple groups (`multigroup:true`) then the above update should result into `"..", "..", "..",` instead of `"..",".."`
+
+#### Now, let's update the PATHs in the PROJECT file accordingly
+
+The PROJECT tracks the paths of all APIs used in your project. Ensure that they now point to `api/...` as the following example:
+
+Before update:
+  group: crew
+  kind: Captain
+  path: sigs.k8s.io/kubebuilder/testdata/project-v4/apis/crew/v1
+```
+
+After Update:
+
+```
+  group: crew
+  kind: Captain
+  path: sigs.k8s.io/kubebuilder/testdata/project-v4/api/crew/v1
+```
+
+### Update kustomize manifests with the changes made so far
+
+- Update the manifest under `config/` directory with all changes performed in the default scaffold done with `go/v4-alpha` plugin. (see for example `testdata/project-v4/config/`) to get all changes in the
+  default scaffolds to be applied on your project
+- Create `config/samples/kustomization.yaml` with all Custom Resources samples specified into `config/samples`. _(see for example `testdata/project-v4/config/samples/kustomization.yaml`)_
 
 <aside class="warning">
 <h1>`config/` directory with changes into the scaffold files</h1>
 
 Note that under the `config/` directory you will find scaffolding changes since using
 `go/v4-alpha` you will ensure that you are no longer using Kustomize v3x.
 
-You can mainly compare the `config/` directory from the samples scaffolded under the `testdata`directory by 
+You can mainly compare the `config/` directory from the samples scaffolded under the `testdata`directory by
 checking the differences between the `testdata/project-v3/config/` with `testdata/project-v4/config/` which
 are samples created with the same commands with the only difference being versions.
 
 However, note that if you create your project with Kubebuilder CLI 3.0.0, its scaffolds
-might change to accommodate changes up to the latest releases using `go/v3` which are not considered 
-breaking for users and/or are forced by the changes introduced in the dependencies 
-used by the project such as [controller-runtime][controller-runtime] and [controller-tools][controller-tools]. 
+might change to accommodate changes up to the latest releases using `go/v3` which are not considered
+breaking for users and/or are forced by the changes introduced in the dependencies
+used by the project such as [controller-runtime][controller-runtime] and [controller-tools][controller-tools].
 
 </aside>
 
+### If you have webhooks:
+
+Replace the import `admissionv1beta1 "k8s.io/api/admission/v1beta1"` with `admissionv1 "k8s.io/api/admission/v1"` in the webhook test files
+
+### Makefile updates
+
+Update the Makefile with the changes which can be found in the samples under testdata for the release tag used. (see for example `testdata/project-v4/Makefile`)
+
+### Update the dependencies
+
+Update the `go.mod` with the changes which can be found in the samples under `testdata` for the release tag used. (see for example `testdata/project-v4/go.mod`). Then, run 
+`go mod tidy` to ensure that you get the latest dependencies and your Golang code has no breaking changes.
+
 ### Verification
 
 In the steps above, you updated your project manually with the goal of ensuring that it follows

docs/book/src/migration/v3vsv4.md

@@ -20,6 +20,8 @@ The details of all changes (breaking or otherwise) can be found in:
 - no longer scaffold webhook test files with `"k8s.io/api/admission/v1beta1"` the k8s API which is no longer served since k8s `1.25`. By default
   webhooks test files are scaffolding using `"k8s.io/api/admission/v1"` which is support from k8s `1.20`
 - no longer provide backwards compatible support with k8s versions < `1.16`
+- change the layout to accommodate the community request to follow the [Standard Go Project Layout][standard-go-project] 
+by moving the api(s) under a new directory called `api`, controller(s) under a new directory called `internal` and the `main.go` under a new directory named `cmd`
 
 <aside class="note">
 <H1> TL;DR of the New `go/v4-alpha` Plugin </H1>
@@ -86,3 +88,4 @@ This way is more complex, susceptible to errors, and success cannot be assured.
 [go/v4-doc]: ./../plugins/go-v4-plugin.md
 [migration-guide-gov3-to-gov4]: migration_guide_gov3_to_gov4.md
 [manually-upgrade]: manually_migration_guide_gov3_to_gov4.md
+[standard-go-project]: https://github.com/golang-standards/project-layout

docs/book/src/plugins/go-v4-plugin.md

@@ -1,7 +1,7 @@
 # go/v4-alpha (go.kubebuilder.io/v4-alpha)
 
 Kubebuilder will scaffold using the `go/v4-alpha` plugin only if specified when initializing the project. 
-This plugin is a composition of the plugins ` kustomize.common.kubebuilder.io/v2-alpha` and `base.go.kubebuilder.io/v4`. 
+This plugin is a composition of the plugins ` kustomize.common.kubebuilder.io/v2-alpha` and `base.go.kubebuilder.io/v4-alpha`. 
 It scaffolds a project template that helps in constructing sets of [controllers][controller-runtime]. 
 
 It scaffolds boilerplate code to create and design controllers. 
@@ -24,6 +24,8 @@ under the [testdata][testdata] directory on the root directory of the Kubebuilde
 - If you are looking to have your project update with the latest version available
 - if you are not targeting k8s versions < `1.16` and `1.20` if you are using webhooks
 - If you are looking to work on with scaffolds which are compatible with k8s `1.25+`
+- If you are looking for the new layout following the [Standard Go Project Layout][standard-go-project] where 
+the "api(s)" are scaffold under the `api` directory, "controller(s)" under `internal`, and the `main.go` under `cmd`
 
 <aside class="note">
 
@@ -61,4 +63,5 @@ kubebuilder init --domain tutorial.kubebuilder.io --repo tutorial.kubebuilder.io
 [testdata]: https://github.com/kubernetes-sigs/kubebuilder/tree/master/testdata
 [plugins-main]: https://github.com/kubernetes-sigs/kubebuilder/blob/master/cmd/main.go
 [kustomize-plugin]: ../plugins/kustomize-v2-alpha.md
-[kustomize]: https://github.com/kubernetes-sigs/kustomize
+[kustomize]: https://github.com/kubernetes-sigs/kustomize
+[standard-go-project]: https://github.com/golang-standards/project-layout

pkg/model/resource/utils.go

@@ -50,6 +50,14 @@ func safeImport(unsafe string) string {
 
 // APIPackagePath returns the default path
 func APIPackagePath(repo, group, version string, multiGroup bool) string {
+	if multiGroup && group != "" {
+		return path.Join(repo, "api", group, version)
+	}
+	return path.Join(repo, "api", version)
+}
+
+// APIPackagePathLegacy returns the default path
+func APIPackagePathLegacy(repo, group, version string, multiGroup bool) string {
 	if multiGroup {
 		if group != "" {
 			return path.Join(repo, "apis", group, version)

pkg/model/resource/utils_test.go

@@ -45,6 +45,23 @@ var _ = Describe("APIPackagePath", func() {
 			Expect(APIPackagePath(repo, group, version, multiGroup)).To(Equal(p))
 		},
 		Entry("single group setup", repo, group, version, false, path.Join(repo, "api", version)),
+		Entry("multiple group setup", repo, group, version, true, path.Join(repo, "api", group, version)),
+		Entry("multiple group setup with empty group", repo, "", version, true, path.Join(repo, "api", version)),
+	)
+})
+
+var _ = Describe("APIPackagePathLegacy", func() {
+	const (
+		repo    = "github.com/kubernetes-sigs/kubebuilder"
+		group   = "group"
+		version = "v1"
+	)
+
+	DescribeTable("should work",
+		func(repo, group, version string, multiGroup bool, p string) {
+			Expect(APIPackagePathLegacy(repo, group, version, multiGroup)).To(Equal(p))
+		},
+		Entry("single group setup", repo, group, version, false, path.Join(repo, "api", version)),
 		Entry("multiple group setup", repo, group, version, true, path.Join(repo, "apis", group, version)),
 		Entry("multiple group setup with empty group", repo, "", version, true, path.Join(repo, "apis", version)),
 	)

pkg/plugin/helpers.go

@@ -47,6 +47,19 @@ func GetShortName(name string) string {
 	return strings.SplitN(name, ".", 2)[0]
 }
 
+// Deprecated: it was added to ensure backwards compatibility and should
+// be removed when we remove the go/v3 plugin
+// IsLegacyLayout returns true when is possible to identify that the project
+// was scaffolded with the previous layout
+func IsLegacyLayout(config config.Config) bool {
+	for _, pluginKey := range config.GetPluginChain() {
+		if strings.Contains(pluginKey, "go.kubebuilder.io/v3") || strings.Contains(pluginKey, "go.kubebuilder.io/v2") {
+			return true
+		}
+	}
+	return false
+}
+
 // Validate ensures a Plugin is valid.
 func Validate(p Plugin) error {
 	if err := validateName(p.Name()); err != nil {

pkg/plugin/util/helpers.go

@@ -20,16 +20,19 @@ import (
 	"sigs.k8s.io/kubebuilder/v3/pkg/config"
 )
 
+// Deprecated: go/v4 no longer supports v1beta1 option
 // HasDifferentCRDVersion returns true if any other CRD version is tracked in the project configuration.
 func HasDifferentCRDVersion(config config.Config, crdVersion string) bool {
 	return hasDifferentAPIVersion(config.ListCRDVersions(), crdVersion)
 }
 
+// Deprecated: go/v4 no longer supports v1beta1 option
 // HasDifferentWebhookVersion returns true if any other webhook version is tracked in the project configuration.
 func HasDifferentWebhookVersion(config config.Config, webhookVersion string) bool {
 	return hasDifferentAPIVersion(config.ListWebhookVersions(), webhookVersion)
 }
 
+// Deprecated: go/v4 no longer supports v1beta1 option
 func hasDifferentAPIVersion(versions []string, version string) bool {
 	return !(len(versions) == 0 || (len(versions) == 1 && versions[0] == version))
 }

pkg/plugin/util/util.go

@@ -191,6 +191,25 @@ func EnsureExistAndReplace(input, match, replace string) (string, error) {
 	return strings.Replace(input, match, replace, -1), nil
 }
 
+func HasFragment(path, target string) (bool, error) {
+	_, err := os.Stat(path)
+	if err != nil {
+		return false, err
+	}
+
+	// false positive
+	// nolint:gosec
+	b, err := os.ReadFile(path)
+	if err != nil {
+		return false, err
+	}
+
+	if !strings.Contains(string(b), target) {
+		return false, nil
+	}
+	return true, nil
+}
+
 // ReplaceInFile replaces all instances of old with new in the file at path.
 func ReplaceInFile(path, old, new string) error {
 	info, err := os.Stat(path)

pkg/plugins/golang/declarative/v1/api.go

@@ -100,7 +100,8 @@ func (p *createAPISubcommand) Scaffold(fs machinery.Filesystem) error {
 	}
 
 	// Update Dockerfile
-	err = updateDockerfile()
+	// nolint:staticcheck
+	err = updateDockerfile(plugin.IsLegacyLayout(p.config))
 	if err != nil {
 		return err
 	}

pkg/plugins/golang/declarative/v1/init.go

@@ -41,27 +41,32 @@ func (p *initSubcommand) InjectConfig(c config.Config) error {
 }
 
 func (p *initSubcommand) Scaffold(_ machinery.Filesystem) error {
-	err := updateDockerfile()
+	//nolint:staticcheck
+	err := updateDockerfile(plugin.IsLegacyLayout(p.config))
 	if err != nil {
 		return err
 	}
 	return nil
 }
 
 // updateDockerfile will add channels staging required for declarative plugin
-func updateDockerfile() error {
+func updateDockerfile(isLegacyLayout bool) error {
 	fmt.Println("updating Dockerfile to add channels/ directory in the image")
-	managerFile := filepath.Join("Dockerfile")
+	dockerfile := filepath.Join("Dockerfile")
 
+	controllerPath := "internal/controller/"
+	if isLegacyLayout {
+		controllerPath = "controllers/"
+	}
 	// nolint:lll
-	err := insertCodeIfDoesNotExist(managerFile,
-		"COPY controllers/ controllers/",
+	err := insertCodeIfDoesNotExist(dockerfile,
+		fmt.Sprintf("COPY %s %s", controllerPath, controllerPath),
 		"\n# https://github.com/kubernetes-sigs/kubebuilder-declarative-pattern/blob/master/docs/addon/walkthrough/README.md#adding-a-manifest\n# Stage channels and make readable\nCOPY channels/ /channels/\nRUN chmod -R a+rx /channels/")
 	if err != nil {
 		return err
 	}
 
-	err = insertCodeIfDoesNotExist(managerFile,
+	err = insertCodeIfDoesNotExist(dockerfile,
 		"COPY --from=builder /workspace/manager .",
 		"\n# copy channels\nCOPY --from=builder /channels /channels\n")
 	if err != nil {