diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 0000000..7457664 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,16 @@ +--- +name: Bug Report +about: Report a bug +labels: kind/bug + +--- + +**What happened**: + +**What you expected to happen**: + +**How to reproduce it (as minimally and precisely as possible)**: + +**Anything else we need to know**: + +**Environment**: diff --git a/.github/ISSUE_TEMPLATE/enhancement_request.md b/.github/ISSUE_TEMPLATE/enhancement_request.md new file mode 100644 index 0000000..4179e17 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/enhancement_request.md @@ -0,0 +1,10 @@ +--- +name: Enhancement Request +about: Suggest an enhancement +labels: kind/enhancement + +--- + +**What would you like to be added**: + +**Why is this needed**: diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 0000000..1fe33e5 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,20 @@ +**What this PR does / why we need it**: + +**Which issue(s) this PR fixes**: +Fixes # + +**Special notes for your reviewer**: + +**Release note**: + +```feature user + +``` diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..5798cfb --- /dev/null +++ b/.gitignore @@ -0,0 +1,3 @@ +# editor and IDE paraphernalia +/.vscode +/.idea \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ba7fa4a..815443e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -2,9 +2,11 @@ ## Code of Conduct -All members of the project community must abide by the [SAP Open Source Code of Conduct](https://github.com/SAP/.github/blob/main/CODE_OF_CONDUCT.md). +All members of the project community must abide by the [SAP Open Source Code of Conduct](https://github.com/openmcp-project/.github/blob/main/CODE_OF_CONDUCT.md). Only by respecting each other we can develop a productive, collaborative community. -Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting [a project maintainer](.reuse/dep5). +Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at [ospo@sap.com](mailto:ospo@sap.com) (SAP Open Source Program Office). All complaints will be reviewed and investigated promptly and fairly. + +All community leaders are obligated to respect the privacy and security of the reporter of any incident. ## Engaging in Our Project diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 0000000..18d8b43 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,12 @@ +# Use distroless as minimal base image to package the component binary +# Refer to https://github.com/GoogleContainerTools/distroless for more details +FROM gcr.io/distroless/static-debian12:nonroot +ARG TARGETOS +ARG TARGETARCH +ARG COMPONENT +WORKDIR / +COPY bin/$COMPONENT.$TARGETOS-$TARGETARCH / +USER 65532:65532 + +# docker doesn't substitue args in ENTRYPOINT, so we replace this during the build script +ENTRYPOINT ["/"] diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..8a761dd --- /dev/null +++ b/Makefile @@ -0,0 +1,4 @@ +%: + @echo "This repository uses task (https://taskfile.dev) instead of make." + @echo "Run 'go install github.com/go-task/task/v3/cmd/task@latest' to install the latest version." + @echo "Then run 'task -l' to list available tasks." diff --git a/README.md b/README.md index 829308d..e479d33 100644 --- a/README.md +++ b/README.md @@ -6,9 +6,169 @@ OpenMCP build and CI scripts -## Requirements and Setup +The Kubernetes operators in the openmcp-project use mostly the same `make` targets and surrounding scripts. This makes sense, because this way developers do not have to think about in which repo they are working right now - `make tidy` will always tidy the go modules. +The drawback is that all `make` targets and scripts have to be kept in sync. If the `make` targets have the same name but a different behavior (conceptually, not code-wise), this will became more of an disadvantage than an advantage. This 'keeping it in sync' means that adding an improvement to any of the scripts required this improvement to be added to all of the script's copies in the different repositories, which is annoying and error-prone. -*Insert a short description what is required to get your project running...* +To improve this, the scripts that are shared between the different repositories have been moved into this repository, which is intended to be used as a git submodule in the actual operator repositories. + +Instead of `make`, we have decided to use the [task](https://taskfile.dev/) tool. + +## Requirements + +It is strongly recommended to include this submodule under the `hack/common` path in the operator repositories. While most of the coding is designed to work from anywhere within the including repository, there are some workarounds for bugs in `task` which rely on the assumption that this repo is a submodule under `hack/common` in the including repository. + +## Setup + +To use this repository, first check it out via +```shell +git submodule add https://github.com/openmcp-project/build.git hack/common +``` +and ensure that it is checked-out via +```shell +git submodule init +``` + +### Taskfile + +To use the generic Taskfile contained in this repository, create a `Taskfile.yaml` in the including repository. It should look something like this: +```yaml +version: 3 + +vars: + NESTED_MODULES: api + API_DIRS: '{{.ROOT_DIR}}/api/core/v1alpha1/...' + MANIFEST_OUT: '{{.ROOT_DIR}}/api/crds/manifests' + CODE_DIRS: '{{.ROOT_DIR}}/cmd/... {{.ROOT_DIR}}/internal/... {{.ROOT_DIR}}/test/... {{.ROOT_DIR}}/api/constants/... {{.ROOT_DIR}}/api/errors/... {{.ROOT_DIR}}/api/install/... {{.ROOT_DIR}}/api/v1alpha1/... {{.ROOT_DIR}}/api/core/v1alpha1/...' + COMPONENTS: 'mcp-operator' + REPO_NAME: 'https://github.com/openmcp-project/mcp-operator' + GENERATE_DOCS_INDEX: "true" + +includes: + shared: + taskfile: hack/common/Taskfile_controller.yaml + flatten: true +``` + +⚠️⚠️⚠️ There is currently a [bug](https://github.com/go-task/task/issues/2108) in the `task` tool which causes it to not propagate variables from the top-level `vars` field to the included Taskfiles properly. As a workaround, the variables have to be specified in `includes.*.vars` instead. + +> `ROOT_DIR` is a task-internal variable that points to the directory of the root Taskfile. + +Since the imported Taskfile is generic, there are a few variables that need to be set in order to configure the tasks correctly. Unless specified otherwise, the variables must not be specified if their respective purpose doesn't apply to the importing repository (e.g. `NESTED_MODULES` is not required if there are no nested modules). +- `NESTED_MODULES` + - List of nested modules, separated by spaces. + - Note that the module has to be located in a subfolder that matches its name. + - Required for multiple tools from the golang environment which are able to work on a single module only and therefore have to be called once per go module. +- `API_DIRS` + - List of files with API type definitions for which k8s CRDs should be generated. + - The `/...` syntax can be used to refer to all files in the directory and its subdirectories. + - This is fed into the k8s code generation tool for CRD generation. +- `MANIFEST_OUT` + - Directory where the generated CRDs should be put in. +- `CODE_DIRS` + - List of files with go code, separated by spaces. + - The `/...` syntax can be used to refer to all files in the directory and its subdirectories. + - Formatting and linting checks are executed on these files. + - This variable must always be specified. +- `COMPONENTS` + - A list of 'components' contained in this repository, separated by spaces. + - This is relevant for binary, image, chart, and OCM component building. Each entry will result in a separate build artifact for the respective builds. + - A 'component' specified here has some implications: + - A `cmd//main.go` file is expected for binary builds. + - A separate docker image will be built for each component. + - If the component has a helm chart, it is expected under `charts//`. + - Note that support for helm charts is not fully implemented yet. + - Each component will get its own OCM component. + - Note that support for OCM components is not implemented yet. + - Library repos will not have any component, operator repos will mostly contain just a single component (the operator itself). +- `REPO_URL` + - URL of the github repository that contains the Taskfile. + - This is used for building the OCM component, which will fail if it is not specified. +- `GENERATE_DOCS_INDEX` + - If this is set and its value is not `false`, the `generate:docs` target will generate a documentation index at `docs/README.md`. Otherwise, the task is skipped. + - See below for a short documentation of the the index generation. + +There are two main Taskfiles, one of which should be included: +- `Taskfile_controller.yaml` is meant for operator repositories and contains task definitions for code generation and validation, binary builds, and image builds. +- `Taskfile_library.yaml` is meant for library repos and does not include the tasks for binary and image building. + +A minimal Taskfile for a library repository could look like this: +```yaml +version: 3 + +vars: + CODE_DIRS: '{{.ROOT_DIR}}/pkg/...' + +includes: + shared: + taskfile: hack/common/Taskfile_library.yaml + flatten: true +``` + +#### Overwriting and Excluding Task Definitions + +Adding new specialized tasks in addition to the imported generic ones is straightforward: simply add the task definitions in the importing Taskfile. + +It is also possible to exclude or overwrite generic tasks. The following example uses an `external-apis` task that should be executed as part of the generic `generate:code` task. + +Overwriting basically works by excluding and re-defining the generic task that should be overwritten. If the generic task's logic should be kept as part of the overwritten definition, the generic file needs to be imported a second time with `internal: true`, so that the original task can be called. + +```yaml +includes: + shared: + taskfile: hack/common/Taskfile_controller.yaml + flatten: true + excludes: # put task names in here which are overwritten in this file + - generate:code + common: # imported a second time so that overwriting task definitions can call the overwritten task with a 'c:' prefix + taskfile: hack/common/Taskfile_controller.yaml + internal: true + aliases: + - c + +tasks: + generate:code: # overwrites shared code task to add external API fetching + desc: " Generate code (mainly DeepCopy functions) and fetches external APIs." + run: once + cmds: + - task: external-apis + - task: c:generate:code + + external-apis: + desc: " Fetch external APIs." + <...> +``` + +### Makefile + +This repo contains a dummy Makefile that for any command prints the instructions for installing `task`: +``` +This repository uses task (https://taskfile.dev) instead of make. +Run 'go install github.com/go-task/task/v3/cmd/task@latest' to install the latest version. +Then run 'task -l' to list available tasks. +``` + +To re-use it, simply create a symbolic link from the importing repo: +```shell +ln -s ./hack/common/Makefile Makefile +``` + +## Documentation Index Generation + +This repository contains a script for creating a index for the documentation of the importing repository. This script is not executed by default, only if the `GENERATE_DOCS_INDEX` variable is explicitly set to anything except `false` in the importing Taskfile. Doing so will not only activate the documentation index generation, but also a check whether it is up-to-date during the `validate:docs` task. + +⚠️ Running the documentation index generation script will overwrite the `docs/README.md` file! + +The script checks the `docs` folder in the importing repository for subdirectories and markdown files (*.md) contained within. Each directory that contains a special metadata file will result in a header, followed by a list where each entry links to one of the markdown files of the respective directory. Directories without the metadata file will be ignored. + +The metadata file is named `.docnames` and is expected to be JSON-formatted, containing a single object with a field named `header`. The value of this field will determine the name of the header in the documentation index for this respective folder. + +Additional fields in the JSON object can be used to manipulate the entries for markdown files within the directory: An entry `"foo.md": "Bar"` in the object causes the `foo.md` file in the directory to be displayed as `Bar` in the generated index. Setting the value of such an entry to the empty string removes the corresponding file from the index. + +Markdown files whose name is not overwritten by a corresponding field in the metadata file are named according to the first line starting with `# ` in their content, or ignored if the name cannot be determined this way. + +#### Limitations + +The script is rather primitive and can only handle a single hierarchy level, nested folder structures are not supported. Manipulating or configuring the generated index apart from adapting the names is also not possible. ## Support, Feedback, Contributing @@ -19,7 +179,7 @@ If you find any bug that may be a security problem, please follow our instructio ## Code of Conduct -We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone. By participating in this project, you agree to abide by its [Code of Conduct](https://github.com/SAP/.github/blob/main/CODE_OF_CONDUCT.md) at all times. +We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone. By participating in this project, you agree to abide by its [Code of Conduct](https://github.com/openmcp-project/.github/blob/main/CODE_OF_CONDUCT.md) at all times. ## Licensing diff --git a/Taskfile_controller.yaml b/Taskfile_controller.yaml new file mode 100644 index 0000000..a94efa5 --- /dev/null +++ b/Taskfile_controller.yaml @@ -0,0 +1,67 @@ +version: 3 + +# This Taskfile is meant to be included for controller repos. +# In addition to the library tasks, it contains tasks for building binaries and images. + +run: once +method: checksum + +includes: + lib: + taskfile: Taskfile_library.yaml + flatten: true + build: + taskfile: tasks_build.yaml + aliases: + - bld + - b + +tasks: + + # The following tasks are non-namespaced aliases for the most important namespaced tasks. + # This helps with visibility on 'task -l'. + + all: + desc: " Run code generation and validation, build all artifacts and push them to the respective registries." + summary: This is an alias for 'build:all'. + run: once + deps: + - build:all + + build: + desc: " Build the binaries. Includes code generation and validation." + summary: This is an alias for 'build:bin:all'. + run: once + deps: + - build:bin:all + + build-raw: + desc: " Like 'build', but skips code generation/validation tasks." + summary: This is an alias for 'build:bin:build-multi-raw'. + run: once + deps: + - build:bin:build-multi-raw + + image: + desc: " Build and push the images. Includes binary build." + summary: This is an alias for 'build:img:all'. + run: once + deps: + - build:img:all + + helm: + desc: " Package and push the helm charts." + summary: This is an alias for 'build:helm:all'. + aliases: + - chart + - helm-chart + run: once + deps: + - build:helm:all + + ocm: + desc: " Build and push the OCM component." + summary: This is an alias for 'build:ocm:all'. + run: once + deps: + - build:ocm:all diff --git a/Taskfile_library.yaml b/Taskfile_library.yaml new file mode 100644 index 0000000..617bbcf --- /dev/null +++ b/Taskfile_library.yaml @@ -0,0 +1,124 @@ +version: 3 + +# This Taskfile is meant to be included for library repos. +# It contains tasks for code generation, validation, and release management. + +run: once +method: checksum + +includes: + generate: + taskfile: tasks_gen.yaml + aliases: + - gen + - g + tools: + taskfile: tasks_tools.yaml + validate: + taskfile: tasks_val.yaml + aliases: + - val + - v + release: + taskfile: tasks_rls.yaml + aliases: + - rls + - r + +vars: + # whacky workarounds for incorrect paths in the special variables + # see https://github.com/go-task/task/issues/2056 and https://github.com/go-task/task/issues/2057 + ROOT_DIR2: '{{.ROOT_DIR | trimSuffix "/common" | trimSuffix "/hack"}}' + TASKFILE_DIR2: + sh: 'if [[ "{{.TASKFILE_DIR}}" == "{{.ROOT_DIR2}}" ]] || [[ "{{.TASKFILE_DIR}}" == "" ]]; then echo -n "{{.ROOT_DIR2}}/hack/common"; else echo -n "{{.TASKFILE_DIR}}"; fi' + + VERSION: + sh: 'PROJECT_ROOT="{{.ROOT_DIR2}}" {{.TASKFILE_DIR2}}/get-version.sh' + OS: + sh: echo ${OS:-$(go env GOOS)} + ARCH: + sh: echo ${ARCH:-$(go env GOARCH)} + MODULE_NAME: + sh: 'cat "{{.ROOT_DIR2}}/go.mod" | grep "module " | sed "s/module //" | sed -E "s/[[:blank:]].*//"' + NESTED_MODULES: '{{.NESTED_MODULES | default "" }}' + + LOCALBIN: '{{ .LOCALBIN | default (print .ROOT_DIR2 "/bin") }}' + LOCALTMP: '{{ .LOCALTMP | default (print .ROOT_DIR2 "/tmp") }}' + CONTROLLER_GEN: '{{ .CONTROLLER_GEN | default (print .LOCALBIN "/controller-gen") }}' + CONTROLLER_GEN_VERSION: '{{ .CONTROLLER_GEN_VERSION | default "v0.16.4" }}' + FORMATTER: '{{ .FORMATTER | default (print .LOCALBIN "/goimports") }}' + FORMATTER_VERSION: '{{ .FORMATTER_VERSION | default "v0.26.0" }}' + LINTER: '{{ .LINTER | default (print .LOCALBIN "/golangci-lint") }}' + LINTER_VERSION: '{{ .LINTER_VERSION | default "v1.64.4" }}' + JQ: '{{ .JQ | default (print .LOCALBIN "/jq") }}' + JQ_VERSION: '{{ .JQ_VERSION | default "v1.7.1" }}' + HELM: '{{ .HELM | default (print .LOCALBIN "/helm") }}' + HELM_VERSION: '{{ .HELM_VERSION | default "v3.17.1" }}' + YAML2JSON: '{{ .YAML2JSON | default (print .LOCALBIN "/yaml2json") }}' + YAML2JSON_VERSION: '{{ .YAML2JSON_VERSION | default "v1.3.5" }}' + OCM: '{{ .OCM | default (print .LOCALBIN "/ocm") }}' + OCM_VERSION: '{{ .OCM_VERSION | default "v0.21.0" }}' + + DOCKER_BUILDER_NAME: # move to build taskfile later + sh: 'echo -n ${DOCKER_BUILDER_NAME:-"openmcp-multiarch-builder"}' + + # separator strings + SEP_MSG: "This is just a separator, what did you expect to happen?" + MAIN_SEP: "MAIN ###########################################################################" + GEN_SEP: "CODE GENERATION ### generate / gen / g #########################################" + VAL_SEP: "CODE VALIDATION ### validate / val / v #########################################" + BLD_SEP: "BUILD ARTIFACTS ### build / bld / b ############################################" + BLD_BIN_SEP: "BINARY BUILD --- bin -----------------------------------------------------------" + BLD_IMG_SEP: "IMAGE BUILD --- img ------------------------------------------------------------" + BLD_HLM_SEP: "HELM CHART BUILD --- helm ------------------------------------------------------" + BLD_OCM_SEP: "OCM COMPONENT BUILD --- ocm ----------------------------------------------------" + RLS_SEP: "RELEASE MANAGEMENT ### release / rls / r #######################################" + +tasks: + + # This is a dummy task that serves as a separator between task namespaces in the 'task -l' output. + "---": + desc: "{{.MAIN_SEP}}" + cmds: + - cmd: echo "{{.SEP_MSG}}" + silent: true + + default: # This is executed if 'task' is called without any arguments. + silent: true + deps: + - help + + help: + silent: true + cmds: + - cmd: task -l + silent: true # for some reason, this is not needed here, but let's keep it for consistency + + generate: + desc: " Combines all code generation tasks, including formatting." + run: once + deps: + - gen:generate + + test: + desc: " Run all tests." + run: once + deps: + - val:test + + validate: + desc: " Combines all validation tasks except for tests." + run: once + aliases: + - verify + - check + deps: + - val:validate + + version: + desc: " Print the version of the project. Use VERSION_OVERRIDE to make this task print a specific version." + run: once + cmds: + - cmd: 'echo "{{.VERSION}}"' + silent: true + diff --git a/components.yaml b/components.yaml new file mode 100644 index 0000000..d15ab46 --- /dev/null +++ b/components.yaml @@ -0,0 +1,120 @@ +# Usage: +# +# Feed arguments into ocm CLI like this: +# ocm add componentversions ... -- CHART_REGISTRY=... IMG_REGISTRY=... +# +# Required values: +# - VERSION (set via the ocm CLI's --version flag) +# - Used as version for the source element pointing to this repo. +# - Used as referenced GitHub release, if it does not contain a '-dev'. +# - Used as fallback value for other versions. +# - COMMIT +# - Commit hash of the git commit used to generate this component descriptor. +# - Used for the source element pointing to this repo. +# - CHART_REGISTRY +# - URL of the OCI registry used for the helm charts +# - IMG_REGISTRY +# - URL of the OCI registry used for the container images +# - COMPONENTS +# - Comma-separated list of components for which resources should be added to the component descriptor, e.g. "apiserver-controller,managedcontrolplane-controller,landscaper-connector". +# - Not required if all of BP_COMPONENTS, CHART_COMPONENTS, and IMG_COMPONENTS are specified instead. +# - MODULE_NAME +# - Name of the Go module. +# - REPO_URL +# - URL of the git repository. +# +# Optional values: +# - CD_VERSION +# - Version used for the component descriptor. +# - Defaults to VERSION if not specified. +# - CHART_VERSION +# - Default version for referenced helm charts. +# - Defaults to VERSION if not specified. +# - IMG_VERSION +# - Default version for referenced container images. +# - Defaults to VERSION if not specified. +# - BP_PATH +# - Path to the blueprint directory. Must be specified if BP_COMPONENTS is set. +# - BP_COMPONENTS +# - Comma-separated list of components for which the blueprint should be added to the component descriptor, e.g. "apiserver-controller,managedcontrolplane-controller,landscaper-connector" +# - Each element will result in a resource entry of type 'landscaper.gardener.cloud/blueprint' named '-blueprint'. The corresponding blueprint is expected at 'BP_PATH/' (relative to this file). +# - Defaults to COMPONENTS if BP_PATH is set and is empty otherwise. +# - CHART_COMPONENTS +# - Comma-separated list of components for which helm charts should be referenced in the component descriptor, optionally with version (separated by ":"). +# - Example: "apiserver-controller:v0.1.0,managedcontrolplane-controller:v0.2.0,landscaper-connector" +# - Each element will result in a resource entry of type 'helmChart' named '-chart'. The chart is expected in the OCI registry at '/:'. +# - Defaults to COMPONENTS if not specified. +# - Each chart's version defaults to CHART_VERSION if not specified. +# - IMG_COMPONENTS +# - Comma-separated list of components for which container images should be referenced in the component descriptor, optionally with version (separated by ":"). +# - Example: "apiserver-controller:v0.1.0,managedcontrolplane-controller:v0.2.0,landscaper-connector" +# - Each element will result in a resource entry of type 'ociImage' named '-image'. The image is expected in the OCI registry at '/:'. +# - Defaults to COMPONENTS if not specified. +# - Each image's version defaults to IMG_VERSION if not specified. + + +name: (( lower(values.MODULE_NAME) )) +version: (( defaults.CD_VERSION )) +provider: + name: openmcp.cloud + +sources: +- name: (( ( "tmp" = split("/", values.REPO_URL) ) element(tmp, length(tmp) - 1) )) + type: blob + version: (( values.VERSION )) + access: + type: gitHub + repoUrl: (( values.REPO_URL )) + commit: (( values.COMMIT )) + ref: (( contains(values.VERSION, "-dev") ? ~~ :"refs/tags/" values.VERSION )) +resources: +- <<<: (( sum[funcs.splitIgnoreEmpty(",", defaults.BP_COMPONENTS)|[]|s,comp|-> s *templates.blueprint] )) +- <<<: (( sum[funcs.splitIgnoreEmpty(",", defaults.CHART_COMPONENTS)|[]|s,cv|-> ("cvs" = split(":", cv)) ("comp" = cvs[0], "chart_version" = (cvs[1] || defaults.CHART_VERSION)) s *templates.chart] )) +- <<<: (( sum[funcs.splitIgnoreEmpty(",", defaults.IMG_COMPONENTS)|[]|s,cv|-> ("cvs" = split(":", cv)) ("comp" = cvs[0], "img_version" = (cvs[1] || defaults.IMG_VERSION)) s *templates.image] )) + + +# ########################################################################## +# # Everything below this is temporary stuff only required during rendering and will not be part of the generated component descriptor. + +defaults: + <<<: (( &temporary )) + CD_VERSION: (( funcs.notEmpty(values.CD_VERSION || "") ? values.CD_VERSION :values.VERSION )) + CHART_VERSION: (( funcs.notEmpty(values.CHART_VERSION || "") ? values.CHART_VERSION :values.VERSION )) + IMG_VERSION: (( funcs.notEmpty(values.IMG_VERSION || "") ? values.IMG_VERSION :values.VERSION )) + BP_COMPONENTS: (( funcs.notEmpty(values.BP_PATH || "") ? ( funcs.notEmpty(values.BP_COMPONENTS || "") ? values.BP_COMPONENTS :values.COMPONENTS ) :"" )) + CHART_COMPONENTS: (( funcs.notEmpty(values.CHART_COMPONENTS || "") ? values.CHART_COMPONENTS :values.COMPONENTS )) + IMG_COMPONENTS: (( funcs.notEmpty(values.IMG_COMPONENTS || "") ? values.IMG_COMPONENTS :values.COMPONENTS )) + +funcs: + <<<: (( &temporary )) + notEmpty: (( |x|-> x != "" )) # returns true if the input is not an empty string + splitIgnoreEmpty: (( |d,s|-> select[split(d, s)|x|-> x != ""] )) # splits a string by a delimiter and removes empty elements + +templates: + <<<: (( &temporary )) + blueprint: + <<<: (( &template )) + name: (( comp "-blueprint" )) + type: landscaper.gardener.cloud/blueprint + input: + path: (( "../blueprints/" comp )) + type: dir + chart: + <<<: (( &template )) + name: (( comp "-chart" )) + type: helmChart + version: (( chart_version )) + access: + type: ociArtifact + imageReference: (( values.CHART_REGISTRY "/" comp ":" chart_version )) + image: + <<<: (( &template )) + name: (( comp "-image" )) + type: ociImage + version: (( img_version )) + access: + imageReference: (( values.IMG_REGISTRY "/" comp ":" img_version )) + type: ociArtifact + + + diff --git a/compute-next-release-version.sh b/compute-next-release-version.sh new file mode 100755 index 0000000..be39d2f --- /dev/null +++ b/compute-next-release-version.sh @@ -0,0 +1,38 @@ +#!/bin/bash + +set -euo pipefail +source "$(realpath "$(dirname $0)/environment.sh")" + +if [[ -z "${VERSION:-}" ]]; then + VERSION=$("$COMMON_SCRIPT_DIR/get-version.sh") +fi + +semver=${1:-"minor"} + +major=${VERSION%%.*} +major=${major#v} +minor=${VERSION#*.} +minor=${minor%%.*} +patch=${VERSION##*.} +patch=${patch%%-*} + +case "$semver" in + ("major") + major=$((major + 1)) + minor=0 + patch=0 + ;; + ("minor") + minor=$((minor + 1)) + patch=0 + ;; + ("patch") + patch=$((patch + 1)) + ;; + (*) + echo "invalid argument: $semver" + exit 1 + ;; +esac + +echo -n "v$major.$minor.$patch" diff --git a/environment.sh b/environment.sh new file mode 100755 index 0000000..770df27 --- /dev/null +++ b/environment.sh @@ -0,0 +1,10 @@ +#!/bin/bash + +export COMMON_SCRIPT_DIR="$(realpath "$(dirname ${BASH_SOURCE[0]})")" +source "$COMMON_SCRIPT_DIR/lib.sh" +export PROJECT_ROOT="${PROJECT_ROOT:-$(realpath "$COMMON_SCRIPT_DIR/../..")}" +export COMPONENT_DEFINITION_FILE="${COMPONENT_DEFINITION_FILE:-"$PROJECT_ROOT/components/components.yaml"}" + +if [[ -f "$COMMON_SCRIPT_DIR/../environment.sh" ]]; then + source "$COMMON_SCRIPT_DIR/../environment.sh" +fi diff --git a/generate-docs-index.sh b/generate-docs-index.sh new file mode 100755 index 0000000..71fed7d --- /dev/null +++ b/generate-docs-index.sh @@ -0,0 +1,87 @@ +#!/bin/bash + +set -euo pipefail +source "$(realpath "$(dirname $0)/environment.sh")" + +DOCS_FOLDER="${DOCS_FOLDER:-${PROJECT_ROOT}/docs}" +METAFILE_NAME=".docnames" + +doc_index_file=${1:-"$DOCS_FOLDER/README.md"} + +# prints to the new doc index +function println() { + echo "$@" >> "$newindex" +} + +# expects a path to a folder as argument +# returns how this folder should be named in an index +function getDocFolderName() { + local metafile="$1/$METAFILE_NAME" + if [[ -f "$metafile" ]]; then + cat "$metafile" | $JQ -r '.header' + fi +} + +# expects two arguments: +# - path to a doc folder +# - name of the file in there +# the file is expected to contain its header in the first line +# or there should be an overwrite present in /$METAFILE_NAME +function getDocName() { + local metafile="$1/$METAFILE_NAME" + local filename="$2" + if [[ -f "$metafile" ]]; then + local overwrite="$(cat "$metafile" | $JQ -r '.overwrites[$name]' --arg name "$2")" + if [[ "$overwrite" != "null" ]]; then + echo "$overwrite" + return + fi + fi + if [[ -f "$filename" ]]; then + local firstheader=$(grep -m1 -E '^# (.*)$' "$filename") + echo "${firstheader#'# '}" + fi +} + +echo "> Generating Documentation Index" + +newindex=$(mktemp) + +println '' +println "# Documentation Index" +println + +( + cd "$DOCS_FOLDER" + for f in *; do + if [[ -d "$f" ]]; then + foldername="$(getDocFolderName "$f")" + if [[ -z "$foldername" ]]; then + echo "Ignoring folder '$f' due to missing '$METAFILE_NAME' file." + continue + fi + + println "## $foldername" + println + + ( + cd "$f" + for f2 in *.md; do + docname="$(getDocName "../$f" "$f2")" + if [[ -z "$docname" ]]; then + echo "Ignoring file '$f/$f2' because the header could not be determined." + # There are two possible reasons for this: + # 1. The file doesn't start with a '# ' in the first line and no overwrite is defined in the folder's metafile. + # 2. The overwrite in the folder's metafile explicitly sets the name to an empty string, meaning this file should be ignored. + continue + fi + println "- [$docname]($f/$f2)" + done + ) + + println + fi + done +) + +cp "$newindex" "$doc_index_file" \ No newline at end of file diff --git a/get-registry.sh b/get-registry.sh new file mode 100755 index 0000000..40fa918 --- /dev/null +++ b/get-registry.sh @@ -0,0 +1,40 @@ +#!/bin/bash -eu + +set -euo pipefail + +if [[ -z ${BASE_REGISTRY:-} ]]; then + BASE_REGISTRY=ghcr.io/openmcp-project +fi + +if [[ -z ${IMAGE_REGISTRY:-} ]]; then + IMAGE_REGISTRY=$BASE_REGISTRY/images +fi +if [[ -z ${CHART_REGISTRY:-} ]]; then + CHART_REGISTRY=$BASE_REGISTRY/charts +fi +if [[ -z ${COMPONENT_REGISTRY:-} ]]; then + COMPONENT_REGISTRY=$BASE_REGISTRY/components +fi + +mode="BASE_" + +while [[ "$#" -gt 0 ]]; do + case ${1:-} in + "-i"|"--image") + mode="IMAGE_" + ;; + "-h"|"--helm") + mode="CHART_" + ;; + "-c"|"--component") + mode="COMPONENT_" + ;; + *) + echo "invalid argument: $1" 1>&2 + exit 1 + ;; + esac + shift +done + +eval echo "\$${mode}REGISTRY" diff --git a/get-version.sh b/get-version.sh new file mode 100755 index 0000000..1e96f07 --- /dev/null +++ b/get-version.sh @@ -0,0 +1,21 @@ +#!/bin/bash -eu + +set -euo pipefail +source "$(realpath "$(dirname $0)/environment.sh")" + +if [[ -n "${VERSION_OVERRIDE:-}" ]]; then + echo -n "$VERSION_OVERRIDE" + exit 0 +fi + +VERSION="$(cat "${PROJECT_ROOT}/VERSION")" + +( + cd "$PROJECT_ROOT" + + if [[ "$VERSION" = *-dev ]] ; then + VERSION="$VERSION-$(git rev-parse HEAD)" + fi + + echo "$VERSION" +) diff --git a/lib.sh b/lib.sh new file mode 100755 index 0000000..fbbf560 --- /dev/null +++ b/lib.sh @@ -0,0 +1,12 @@ +#!/bin/bash + +# pipe some text into 'indent X' to indent each line by X levels (one 'level' being two spaces) +function indent() { + local level=${1:-""} + if [[ -z "$level" ]]; then + level=1 + fi + local spaces=$(($level * 2)) + local iv=$(printf %${spaces}s) + sed "s/^/$iv/" +} \ No newline at end of file diff --git a/renovate.json b/renovate.json new file mode 100644 index 0000000..2ce1e84 --- /dev/null +++ b/renovate.json @@ -0,0 +1,22 @@ +{ + "$schema": "https://docs.renovatebot.com/renovate-schema.json", + "extends": [ + "config:best-practices", + "security:openssf-scorecard", + ":dependencyDashboard" + ], + "customManagers": [ + { + "description": "Match in Makefile", + "customType": "regex", + "fileMatch": [ + "(^|/|\\.)([Dd]ocker|[Cc]ontainer)file$", + "(^|/)([Dd]ocker|[Cc]ontainer)file[^/]*$", + "(^|/)Makefile$" + ], + "matchStrings": [ + "# renovate: datasource=(?[a-z-.]+?) depName=(?[^\\s]+?)(?: (lookupName|packageName)=(?[^\\s]+?))?(?: versioning=(?[^\\s]+?))?(?: extractVersion=(?[^\\s]+?))?(?: registryUrl=(?[^\\s]+?))?\\s(?:ENV |ARG )?.+?_VERSION ?(?:\\?=|=)\"? ?(?.+?)\"?\\s" + ] + } + ] +} diff --git a/run-lint.sh b/run-lint.sh new file mode 100755 index 0000000..ad98f8f --- /dev/null +++ b/run-lint.sh @@ -0,0 +1,40 @@ +#!/bin/bash + +# Runs 'go test' on all modules. +# Expects NESTED_MODULES to be set and the code directories being passed in as arguments. + +set -euo pipefail +source "$(realpath "$(dirname $0)/environment.sh")" + +function run_lint() { + "$LINTER" run -c "$PROJECT_ROOT/.golangci.yaml" "$@" +} + +echo "> Running linter ..." + +# NESTED_MODULES must be set to the list of nested go modules, e.g. 'api,nested2,nested3' +paths=("$@") +for nm in ${NESTED_MODULES//,/ }; do + echo "> Linting $nm module ..." | indent 1 + # filter out paths that belong to the nested module by prefix matching + module_paths=() + non_module_paths=() + for val in "${paths[@]}"; do + if [[ "$val" =~ ^$PROJECT_ROOT/$nm ]] || [[ "$val" =~ ^$nm ]]; then + module_paths+=("$val") + else + non_module_paths+=("$val") + fi + done + paths=("${non_module_paths[@]}") + ( + cd "$PROJECT_ROOT/$nm" + run_lint "${module_paths[@]}" + ) +done + +echo "> Linting root module ..." | indent 1 +( + cd "$PROJECT_ROOT" + run_lint "${paths[@]}" +) diff --git a/run-tests.sh b/run-tests.sh new file mode 100755 index 0000000..378c85f --- /dev/null +++ b/run-tests.sh @@ -0,0 +1,42 @@ +#!/bin/bash + +# Runs 'go test' on all modules. +# Expects NESTED_MODULES to be set and the code directories being passed in as arguments. + +set -euo pipefail +source "$(realpath "$(dirname $0)/environment.sh")" + +function run_test() { + go test "$@" -coverprofile cover.root.out + go tool cover --html=cover.root.out -o cover.root.html + go tool cover -func cover.root.out | tail -n 1 +} + +echo "> Running tests ..." + +# NESTED_MODULES must be set to the list of nested go modules, e.g. 'api,nested2,nested3' +paths=("$@") +for nm in ${NESTED_MODULES//,/ }; do + echo "> Testing $nm module ..." | indent 1 + # filter out paths that belong to the nested module by prefix matching + module_paths=() + non_module_paths=() + for val in "${paths[@]}"; do + if [[ "$val" =~ ^$PROJECT_ROOT/$nm ]] || [[ "$val" =~ ^$nm ]]; then + module_paths+=("$val") + else + non_module_paths+=("$val") + fi + done + paths=("${non_module_paths[@]}") + ( + cd "$PROJECT_ROOT/$nm" + run_test "${module_paths[@]}" + ) +done + +echo "> Testing root module ..." | indent 1 +( + cd "$PROJECT_ROOT" + run_test "${paths[@]}" +) diff --git a/sed.sh b/sed.sh new file mode 100755 index 0000000..55245d7 --- /dev/null +++ b/sed.sh @@ -0,0 +1,33 @@ +#!/bin/bash + +set -euo pipefail +source "$(realpath "$(dirname $0)/environment.sh")" + +# This basically runs 'sed -i', but works on both GNU and BSD sed by manually creating a temporary file instead of using the -i flag. +# The last argument is the file to operate on, everything else is passed to sed. +# If the specified file doesn't exist, this is ignored silently. + +FILE=${@: -1} +if [[ -z "$FILE" ]]; then + echo "No file specified." + exit 1 +fi + +if [[ ! -f "$FILE" ]]; then + exit 0 +fi + +TMPFILE=$(mktemp) + +# helper function to restore the temp file to the original file, if that one doesn't exist +function restore() { + if [[ ! -f "$FILE" ]] && [[ -f "$TMPFILE" ]]; then + mv "$TMPFILE" "$FILE" + fi +} + +trap restore EXIT + +mv "$FILE" "$TMPFILE" +cat "$TMPFILE" | sed "${@:1:$#-1}" > "$FILE" +rm -f "$TMPFILE" diff --git a/tasks_build.yaml b/tasks_build.yaml new file mode 100644 index 0000000..c0c38e8 --- /dev/null +++ b/tasks_build.yaml @@ -0,0 +1,27 @@ +version: 3 + +includes: + bin: + taskfile: tasks_build_bin.yaml + img: + taskfile: tasks_build_img.yaml + helm: + taskfile: tasks_build_helm.yaml + ocm: + taskfile: tasks_build_ocm.yaml + +tasks: + # This is a dummy task that serves as a separator between task namespaces in the 'task -l' output. + "---": + desc: "{{.BLD_SEP}}" + cmds: + - cmd: echo "{{.SEP_MSG}}" + silent: true + + all: + desc: " Builds binaries and images, packages helm charts, builds the OCM component, and pushes everything into the respective registries." + run: once + cmds: + - task: img:all + - task: helm:all + - task: ocm:all diff --git a/tasks_build_bin.yaml b/tasks_build_bin.yaml new file mode 100644 index 0000000..1abf766 --- /dev/null +++ b/tasks_build_bin.yaml @@ -0,0 +1,88 @@ +version: 3 + +includes: + gen: + taskfile: tasks_gen.yaml + internal: true + val: + taskfile: tasks_val.yaml + internal: true + +tasks: + # This is a dummy task that serves as a separator between task namespaces in the 'task -l' output. + "---": + desc: "{{.BLD_BIN_SEP}}" + cmds: + - cmd: echo "{{.SEP_MSG}}" + silent: true + + build: + desc: ' Build the binary for $OS/$ARCH.' + summary: | + This task builds the binary for the current operating system and architecture. + To overwrite this, set the 'OS' and 'ARCH' environment variables. + The binary is saved in the 'bin' folder, as $COMPONENT.$OS-$ARCH. + requires: + vars: + - COMPONENTS + - OS + - ARCH + deps: + - gen:tidy + cmds: + - task: gen:generate # don't use deps, since they are executed in parallel + - task: val:validate # and we want the code generation + - task: val:test # to be executed before the validation. + - task: build-raw + vars: + OS: '{{.OS}}' + ARCH: '{{.ARCH}}' + + build-raw: + desc: " Build the binary. Opposed to the regular build, this one just builds and skips code generation/validation tasks." + summary: | + This task builds the binary for the current operating system and architecture. + To overwrite this, set the 'OS' and 'ARCH' environment variables. + The binary is saved in the 'bin' folder, as $COMPONENT.$OS-$ARCH. + requires: + vars: + - COMPONENTS + - OS + - ARCH + cmds: + - for: + var: COMPONENTS + as: COMPONENT + cmd: 'CGO_ENABLED=0 GOOS={{.OS}} GOARCH={{.ARCH}} go build -a -o {{.ROOT_DIR2}}/bin/{{.COMPONENT}}.{{.OS}}-{{.ARCH}} {{.ROOT_DIR2}}/cmd/{{.COMPONENT}}/main.go' + + build-multi-raw: + desc: " Build multi-platform binaries. Skips code generation/validation tasks." + requires: + vars: + - COMPONENTS + cmds: + - for: + matrix: + OS: [linux, darwin] + ARCH: [amd64, arm64] + vars: + OS: '{{.ITEM.OS}}' + ARCH: '{{.ITEM.ARCH}}' + task: build-raw + + all: + desc: " Build multi-platform binaries." + aliases: + - build-multi + requires: + vars: + - COMPONENTS + cmds: + - for: + matrix: + OS: [linux, darwin] + ARCH: [amd64, arm64] + vars: + OS: '{{.ITEM.OS}}' + ARCH: '{{.ITEM.ARCH}}' + task: build diff --git a/tasks_build_helm.yaml b/tasks_build_helm.yaml new file mode 100644 index 0000000..85b93e4 --- /dev/null +++ b/tasks_build_helm.yaml @@ -0,0 +1,97 @@ +version: 3 + +includes: + tools: + taskfile: tasks_tools.yaml + internal: true + +tasks: + # This is a dummy task that serves as a separator between task namespaces in the 'task -l' output. + "---": + desc: "{{.BLD_HLM_SEP}}" + cmds: + - cmd: echo "{{.SEP_MSG}}" + silent: true + + all: + desc: " Packages and pushes the helm charts for all components." + run: once + aliases: + - helm + cmds: + - task: build + - task: push + + build: + desc: " Packages the helm chart to prepare it for being pushed to the registry." + run: once + requires: + vars: + - COMPONENTS + - VERSION + cmds: + - for: + var: COMPONENTS + vars: + COMPONENT: '{{.ITEM}}' + VERSION: '{{.VERSION}}' + task: build-internal + + build-internal: + desc: " Packages the helm chart of a specific component to prepare it for being pushed to the registry." + run: when_changed + deps: + - tools:localtmp + - tools:helm + requires: + vars: + - COMPONENT + - VERSION + status: + - 'test ! -f "{{.ROOT_DIR2}}/charts/{{.COMPONENT}}/Chart.yaml"' + cmds: + - '"{{.HELM}}" package "{{.ROOT_DIR2}}/charts/{{.COMPONENT}}" -d "{{.LOCALTMP}}" --version "{{.VERSION}}"' + internal: true + + push: + desc: " Push the helm chart to the registry. Requires the chart to have been packaged before." + run: once + requires: + vars: + - COMPONENTS + - VERSION + vars: + HELM_REGISTRY: + sh: 'PROJECT_ROOT="{{.ROOT_DIR2}}" {{.TASKFILE_DIR2}}/get-registry.sh --helm' + cmds: + - for: + var: COMPONENTS + vars: + COMPONENT: '{{.ITEM}}' + VERSION: '{{.VERSION}}' + HELM_REGISTRY: '{{.HELM_REGISTRY}}' + task: push-internal + + push-internal: + desc: " Push the helm chart of a specific component to the registry." + run: when_changed + deps: + - tools:helm + - tools:yaml2json + - tools:jq + requires: + vars: + - COMPONENT + - VERSION + - HELM_REGISTRY + - LOCALTMP + status: + - 'test ! -f "{{.ROOT_DIR2}}/charts/{{.COMPONENT}}/Chart.yaml"' + vars: + CHART_NAME: # vars are evaluated before status, therefore the duplicated check + sh: 'if [[ -f "{{.ROOT_DIR2}}/charts/{{.COMPONENT}}/Chart.yaml" ]]; then cat "{{.ROOT_DIR2}}/charts/{{.COMPONENT}}/Chart.yaml" | {{.YAML2JSON}} | {{.JQ}} -r .name; else echo -n "{{.COMPONENT}}"; fi' + cmds: + - '"{{.HELM}}" push "{{.LOCALTMP}}/{{.CHART_NAME}}-{{.VERSION}}.tgz" "oci://{{.HELM_REGISTRY}}"' + - 'rm -f "{{.LOCALTMP}}/{{.CHART_NAME}}-{{.VERSION}}.tgz"' + internal: true + diff --git a/tasks_build_img.yaml b/tasks_build_img.yaml new file mode 100644 index 0000000..9a22ecb --- /dev/null +++ b/tasks_build_img.yaml @@ -0,0 +1,219 @@ +version: 3 + +includes: + build:bin: + taskfile: tasks_build_bin.yaml + internal: true + +tasks: + # This is a dummy task that serves as a separator between task namespaces in the 'task -l' output. + "---": + desc: "{{.BLD_IMG_SEP}}" + cmds: + - cmd: echo "{{.SEP_MSG}}" + silent: true + + prepare-docker-builder: + desc: " Prepares the docker multiarch builder." + run: once + requires: + vars: + - DOCKER_BUILDER_NAME + status: + - 'docker buildx ls | grep "{{.DOCKER_BUILDER_NAME}}" >/dev/null' + cmds: + - '( docker buildx ls | grep "{{.DOCKER_BUILDER_NAME}}" >/dev/null ) || docker buildx create --name {{.DOCKER_BUILDER_NAME}} >/dev/null' # duplicate the check because this might throw an error if run with '-f' otherwise + internal: true + + build: + desc: " Build the image for $OS/$ARCH." + summary: | + This task builds the image for the current operating system and architecture. + To overwrite this, set the 'OS' and 'ARCH' environment variables. + To overwrite the image's base path, set the 'IMAGE_REGISTRY' environment variable. + deps: + - task: build:bin:build + vars: + OS: '{{.OS}}' + ARCH: '{{.ARCH}}' + cmds: + - task: build-raw + vars: + OS: '{{.OS}}' + ARCH: '{{.ARCH}}' + + build-raw: + desc: " Build the image. Does not run the binary build before." + summary: | + This task builds the image for the current operating system and architecture. + To overwrite this, set the 'OS' and 'ARCH' environment variables. + To overwrite the image's base path, set the 'IMAGE_REGISTRY' environment variable. + requires: + vars: + - COMPONENTS + - VERSION + - OS + - ARCH + vars: + IMAGE_BASE: + sh: 'PROJECT_ROOT="{{.ROOT_DIR2}}" {{.TASKFILE_DIR2}}/get-registry.sh --image' + cmds: + - task: prepare-docker-builder + - for: + var: COMPONENTS + vars: + COMPONENT: '{{.ITEM}}' + IMAGE_BASE: '{{.IMAGE_BASE}}' + OS: '{{.OS}}' + ARCH: '{{.ARCH}}' + task: build-internal + + build-internal: + desc: " Build the image for a single component. Requires the docker builder to have been prepared before." + requires: + vars: + - COMPONENT + - VERSION + - OS + - ARCH + - IMAGE_BASE + - DOCKER_BUILDER_NAME + cmds: + - 'echo "Building image {{.COMPONENT}}:{{.VERSION}}-{{.OS}}-{{.ARCH}}"' + - '[[ "{{.OS}}" == "linux" ]] || { echo "The distroless base image does only support linux as operating system."; exit 1; }' + - 'cat "{{.TASKFILE_DIR2}}/Dockerfile" | sed "s//{{.COMPONENT}}/g" > "{{.ROOT_DIR2}}/Dockerfile.tmp"' + - '( cd "{{.ROOT_DIR2}}"; docker buildx build --builder {{.DOCKER_BUILDER_NAME}} --load --build-arg COMPONENT={{.COMPONENT}} --platform {{.OS}}/{{.ARCH}} -t {{.IMAGE_BASE}}/{{.COMPONENT}}:{{.VERSION}}-{{.OS}}-{{.ARCH}} -f Dockerfile.tmp . )' + - 'rm -f "{{.ROOT_DIR2}}/Dockerfile.tmp"' + internal: true + + build-multi-raw: + desc: " Build multi-platform image. Does not build the binaries before." + requires: + vars: + - COMPONENTS + cmds: + - for: + matrix: + OS: [linux] # distroless base image only supports linux + ARCH: [amd64, arm64] + vars: + OS: '{{.ITEM.OS}}' + ARCH: '{{.ITEM.ARCH}}' + task: build-raw + + build-multi: + desc: " Build multi-platform image." + requires: + vars: + - COMPONENTS + cmds: + - for: + matrix: + OS: [linux] # distroless base image only supports linux + ARCH: [amd64, arm64] + vars: + OS: '{{.ITEM.OS}}' + ARCH: '{{.ITEM.ARCH}}' + task: build + + push: + desc: " Push the image. Image must have been built before." + summary: | + This task pushes the image for the current operating system and architecture. + To overwrite this, set the 'OS' and 'ARCH' environment variables. + To overwrite the image's base path, set the 'IMAGE_REGISTRY' environment variable. + requires: + vars: + - COMPONENTS + - OS + - ARCH + cmds: + - for: + var: COMPONENTS + vars: + COMPONENT: '{{.ITEM}}' + OS: '{{.OS}}' + ARCH: '{{.ARCH}}' + task: push-internal + + push-internal: + desc: "Push the image for a single component. Image must already have been built." + requires: + vars: + - COMPONENT + - VERSION + - OS + - ARCH + vars: + IMAGE_BASE: + sh: 'PROJECT_ROOT="{{.ROOT_DIR2}}" {{.TASKFILE_DIR2}}/get-registry.sh --image' + cmds: + - 'echo "Pushing image {{.COMPONENT}}:{{.VERSION}}-{{.OS}}-{{.ARCH}}"' + - 'docker push {{.IMAGE_BASE}}/{{.COMPONENT}}:{{.VERSION}}-{{.OS}}-{{.ARCH}}' + internal: true + + push-multi: + desc: " Push the multi-platform image. Images must have been built before." + vars: + IMAGE_BASE: + sh: 'PROJECT_ROOT="{{.ROOT_DIR2}}" {{.TASKFILE_DIR2}}/get-registry.sh --image' + requires: + vars: + - COMPONENTS + - VERSION + cmds: + - for: + matrix: + OS: [linux] # distroless base image only supports linux + ARCH: [amd64, arm64] + vars: + OS: '{{.ITEM.OS}}' + ARCH: '{{.ITEM.ARCH}}' + task: push + - for: + var: COMPONENTS + as: COMPONENT + vars: + IMG: '{{.IMAGE_BASE}}/{{.COMPONENT}}:{{.VERSION}}' + task: push-multi-internal + + push-multi-internal: + desc: " Build and push the multi-platform manifest for a single component's image. Individual platform-specific images must have been pushed before." + requires: + vars: + - IMG + cmds: + - for: + matrix: + OS: [linux] # distroless base image only supports linux + ARCH: [amd64, arm64] + cmd: 'docker manifest create {{.IMG}} --amend {{.IMG}}-{{.ITEM.OS}}-{{.ITEM.ARCH}}' + - 'echo "Pushing image {{.IMG}}"' + - 'docker manifest push {{.IMG}}' + internal: true + + tag: + desc: " Adds an additional tag to the multi-platform image. Image must have been built and pushed before." + preconditions: + - sh: '[[ "{{index (.CLI_ARGS | splitList " ") 0}}" != "" ]]' + msg: "No tag specified. Do so by calling 'task {{.TASK}} -- '." + vars: + IMAGE_BASE: + sh: 'PROJECT_ROOT="{{.ROOT_DIR2}}" {{.TASKFILE_DIR2}}/get-registry.sh --image' + TAG: '{{index (.CLI_ARGS | splitList " ") 0}}' + requires: + vars: + - COMPONENTS + - VERSION + - TAG + cmds: + - for: + var: COMPONENTS + as: COMPONENT + cmd: 'docker buildx imagetools create "{{.IMAGE_BASE}}/{{.COMPONENT}}:{{.VERSION}}" --tag "{{.IMAGE_BASE}}/{{.COMPONENT}}:{{.TAG}}"' + + all: + desc: " Build binaries and images for multiple operating systems and architectures and push them to the registry." + cmds: + - task: build-multi + - task: push-multi diff --git a/tasks_build_ocm.yaml b/tasks_build_ocm.yaml new file mode 100644 index 0000000..376d6af --- /dev/null +++ b/tasks_build_ocm.yaml @@ -0,0 +1,78 @@ +version: 3 + +includes: + tools: + taskfile: tasks_tools.yaml + internal: true + +tasks: + # This is a dummy task that serves as a separator between task namespaces in the 'task -l' output. + "---": + desc: "{{.BLD_OCM_SEP}}" + cmds: + - cmd: echo "{{.SEP_MSG}}" + silent: true + + all: + desc: " Builds and pushes the OCM component." + run: once + cmds: + - task: build + - task: push + + build: + desc: " Build the OCM component." + run: once + deps: + - tools:ocm + - tools:localtmp + requires: + vars: + - COMPONENTS + - VERSION + - MODULE_NAME + - REPO_URL + vars: + CHART_REGISTRY: + sh: 'PROJECT_ROOT="{{.ROOT_DIR2}}" {{.TASKFILE_DIR2}}/get-registry.sh --helm' + IMAGE_REGISTRY: + sh: 'PROJECT_ROOT="{{.ROOT_DIR2}}" {{.TASKFILE_DIR2}}/get-registry.sh --image' + COMMIT: + sh: '( cd "{{.ROOT_DIR2}}"; git rev-parse HEAD )' + compdir: '{{.LOCALTMP}}/component' + cmds: + - | + "{{.OCM}}" add componentversions --file "{{.compdir}}" --version "{{.VERSION}}" --create --force --templater spiff "{{.TASKFILE_DIR2}}/components.yaml" -- \ + VERSION="{{.VERSION}}" \ + CHART_REGISTRY="{{.CHART_REGISTRY}}" \ + IMG_REGISTRY="{{.IMAGE_REGISTRY}}" \ + COMMIT="{{.COMMIT}}" \ + MODULE_NAME="{{.MODULE_NAME}}" \ + REPO_URL="{{.REPO_URL}}" \ + COMPONENTS="{{.COMPONENTS | trimSuffix " " | replace " " ","}}" \ + CD_VERSION="{{.CD_VERSION | default ""}}" \ + CHART_VERSION="{{.CHART_VERSION | default ""}}" \ + IMG_VERSION="{{.IMG_VERSION | default ""}}" \ + BP_COMPONENTS="{{.BP_COMPONENTS | default ""}}" \ + CHART_COMPONENTS="{{.CHART_COMPONENTS | default ""}}" \ + IMG_COMPONENTS="{{.IMG_COMPONENTS | default ""}}" + - cmd: echo "Use '$(realpath --relative-base="{{.USER_WORKING_DIR}}" "{{.OCM}}") get cv $(realpath --relative-base="{{.USER_WORKING_DIR}}" "{{.compdir}}") -o yaml' to view the generated component descriptor." + silent: true + + push: + desc: " Push the OCM component to the registry. It must have been built before. Set OVERWRITE_COMPONENTS to 'true' to overwrite existing component versions." + run: once + deps: + - tools:ocm + requires: + vars: + - VERSION + - LOCALTMP + vars: + COMPONENT_REGISTRY: + sh: 'PROJECT_ROOT="{{.ROOT_DIR2}}" {{.TASKFILE_DIR2}}/get-registry.sh --component' + overwrite_mod: + sh: 'if [[ -n ${OVERWRITE_COMPONENTS:-} ]] && [[ ${OVERWRITE_COMPONENTS} != "false" ]]; then echo -n "--overwrite"; fi' + compdir: '{{.LOCALTMP}}/component' + cmds: + - '"{{.OCM}}" transfer componentversions "{{.compdir}}" "{{.COMPONENT_REGISTRY}}" {{.overwrite_mod}}' diff --git a/tasks_gen.yaml b/tasks_gen.yaml new file mode 100644 index 0000000..b850da3 --- /dev/null +++ b/tasks_gen.yaml @@ -0,0 +1,156 @@ +version: 3 + +includes: + tools: + taskfile: tasks_tools.yaml + internal: true + +tasks: + # This is a dummy task that serves as a separator between task namespaces in the 'task -l' output. + "---": + desc: "{{.GEN_SEP}}" + cmds: + - cmd: echo "{{.SEP_MSG}}" + silent: true + + tidy: + desc: " Run 'go mod tidy' for all modules." + run: once + vars: + GO_VERSION: + sh: grep -E 'go [0-9]+\.[0-9]+\.[0-9]+' <{{.ROOT_DIR2}}/go.mod | sed 's/go //' # extract go version from go.mod + cmds: + - for: + var: NESTED_MODULES + as: MODULE + cmd: '"{{.TASKFILE_DIR2}}/sed.sh" -E "s/go [0-9]+\.[0-9]+\.[0-9]+/go {{.GO_VERSION}}/" "{{.ROOT_DIR2}}/{{.MODULE}}/go.mod"' # align go version in nested modules + - for: + var: NESTED_MODULES + as: MODULE + cmd: ( cd {{.ROOT_DIR2}}/{{.MODULE}}; go mod tidy ) + - cmd: ( cd {{.ROOT_DIR2}}; go mod tidy ) + - cmd: '"{{.TASKFILE_DIR2}}/sed.sh" -E "s/toolchain go[0-9]+\.[0-9]+\.[0-9]+/toolchain go{{.GO_VERSION}}/" "{{.ROOT_DIR2}}/go.mod"' # overwrite toolchain in root go.mod file + - for: + var: NESTED_MODULES + as: MODULE + cmd: '"{{.TASKFILE_DIR2}}/sed.sh" -E "s/toolchain go[0-9]+\.[0-9]+\.[0-9]+/toolchain go{{.GO_VERSION}}/" "{{.ROOT_DIR2}}/{{.MODULE}}/go.mod"' # overwrite toolchain in nested go.mod files + + all: + desc: " Combines all code generation tasks, including formatting." + run: once + aliases: + - generate + cmds: + - task: tidy + - task: code + - task: manifests + - task: docs + - task: format + + manifests: + desc: " Generate manifests for CRDs." + run: once + status: + - '[ "{{.API_DIRS | default ""}}" == "" ] || [ "{{.MANIFEST_OUT | default ""}}" == "" ]' + cmds: + - task: manifests-internal + + manifests-internal: + desc: " Generate manifests for CRDs." + run: when_changed + deps: + - tools:controller-gen + requires: + vars: + - API_DIRS + - MANIFEST_OUT + cmds: + - cmd: rm -rf {{.MANIFEST_OUT}} + - for: + var: API_DIRS + cmd: '{{.CONTROLLER_GEN}} crd paths={{.ITEM}} output:crd:artifacts:config={{.MANIFEST_OUT}}' + internal: true + + code: + desc: " Generate code (mainly DeepCopy functions) for all modules." + run: once + status: + - '[ "{{.API_DIRS | default ""}}" == "" ]' + cmds: + - task: code-internal + + code-internal: + desc: " Generate code (mainly DeepCopy functions) for all modules." + run: once + deps: + - tools:controller-gen + requires: + vars: + - API_DIRS + cmds: + - for: + var: API_DIRS + cmd: '{{.CONTROLLER_GEN}} object paths={{.ITEM}}' + internal: true + + docs: + desc: " Generate the documentation index for the project. No effect, unless GENERATE_DOCS_INDEX is set to 'true' in the Taskfile." + run: once + status: + - '[ "{{.GENERATE_DOCS_INDEX | default "false"}}" == "false" ]' + cmds: + - task: docs-internal + + docs-internal: + desc: " Generate the documentation index for the project." + run: once + deps: + - tools:jq + cmds: + - 'PROJECT_ROOT="{{.ROOT_DIR2}}" JQ="{{.JQ}}" {{.TASKFILE_DIR2}}/generate-docs-index.sh' + internal: true + + fmt-internal: + desc: " Run 'go fmt' for a single module." + run: when_changed + requires: + vars: + - MODULE + cmds: + - '( cd {{.ROOT_DIR2}}/{{.MODULE}}; go fmt ./...; )' + internal: true + + fmt: + desc: " Run 'go fmt' for all modules." + run: once + cmds: + - for: + var: NESTED_MODULES + vars: + MODULE: '{{.ITEM}}' + task: fmt-internal + - vars: + MODULE: "" + task: fmt-internal + + format-imports: + desc: " Run 'goimports'." + run: once + deps: + - tools:goimports + requires: + vars: + - CODE_DIRS + - MODULE_NAME + vars: + code_dirs: + sh: '{{.TASKFILE_DIR2}}/unfold.sh --clean --no-unfold --inline {{.CODE_DIRS}}' # goimports doesn't like the '/...' syntax + cmds: + - '{{.FORMATTER}} -l -w -local={{.MODULE_NAME}} {{.code_dirs}}' + + format: + desc: " Combines all formatting tasks." + run: once + cmds: + - task: fmt + - task: format-imports diff --git a/tasks_rls.yaml b/tasks_rls.yaml new file mode 100644 index 0000000..3cc6299 --- /dev/null +++ b/tasks_rls.yaml @@ -0,0 +1,138 @@ +version: 3 + +tasks: + # This is a dummy task that serves as a separator between task namespaces in the 'task -l' output. + "---": + desc: "{{.RLS_SEP}}" + cmds: + - cmd: echo "{{.SEP_MSG}}" + silent: true + + patch: + desc: " Prepares a new patch release by creating a commit that will result in a patch release when pushed." + run: once + requires: + vars: + - VERSION + cmds: + - task: release-internal + vars: + RELEASE_VERSION: + sh: 'VERSION={{.VERSION}} "{{.TASKFILE_DIR2}}/compute-next-release-version.sh" patch' + + minor: + desc: " Prepares a new minor release by creating a commit that will result in a minor release when pushed." + run: once + requires: + vars: + - VERSION + cmds: + - task: release-internal + vars: + RELEASE_VERSION: + sh: 'VERSION={{.VERSION}} "{{.TASKFILE_DIR2}}/compute-next-release-version.sh" minor' + + major: + desc: " Prepares a new major release by creating a commit that will result in a major release when pushed." + run: once + requires: + vars: + - VERSION + cmds: + - task: release-internal + vars: + RELEASE_VERSION: + sh: 'VERSION={{.VERSION}} "{{.TASKFILE_DIR2}}/compute-next-release-version.sh" major' + + prompt-changes: + desc: " Causes a prompt if there are uncommitted changes which would be included in the release commit." + run: once + vars: + changes: + sh: '( cd "{{.ROOT_DIR2}}"; git status --short )' + status: + - 'test -z "$( cd "{{.ROOT_DIR2}}" && git status --porcelain=v1)"' + prompt: + - 'There are uncommitted changes in the working directory:{{"\n"}}{{.changes}}{{"\n\n"}}These changes will be included in the release commit, unless you stash, commit, or remove them otherwise before. Do you want to continue?' + internal: true + + release-internal: + desc: " Prepares a new release by creating a commit that will result in a release when pushed." + run: once + deps: + - prompt-changes + requires: + vars: + - RELEASE_VERSION + prompt: + - 'The release version will be {{.RELEASE_VERSION}}. Do you want to continue?' + cmds: + - cmd: 'echo "Creating release commit for version {{.RELEASE_VERSION}} ..."' + silent: true + - vars: + CLI_ARGS: '{{.RELEASE_VERSION}}' + task: set-version + - '( cd "{{.ROOT_DIR2}}"; git add --all; git commit -m "release {{.RELEASE_VERSION}}" )' + internal: true + + set-version: + desc: ' Updates all versions to the specified version. Usage: task {{.TASK}} -- ' + summary: | + This task is called by the release tasks automatically. + It sets the following versions to the one provided as argument: + - the VERSION file in the root directory + - 'version' in the Chart.yaml file of each component's helm chart + - 'appVersion' in the Chart.yaml file of each component's helm chart + - 'tag' in the values.yaml file of each component's helm chart + - the imported version of each nested module in the root go.mod file + run: once + requires: + vars: + - COMPONENTS + - NESTED_MODULES + - MODULE_NAME + preconditions: + - sh: 'test "{{len .CLI_ARGS}}" -gt 0' + msg: 'This task requires a version argument. Usage: task {{.TASK}} -- ' + vars: + VERSION: '{{.CLI_ARGS | splitList " " | first}}' + cmds: + - 'echo -n "{{.VERSION}}" > "{{.ROOT_DIR2}}/VERSION"' + - for: + var: COMPONENTS + vars: + COMPONENT: '{{.ITEM}}' + VERSION: '{{.VERSION}}' + task: set-chart-version-internal + - for: + var: NESTED_MODULES + vars: + MODULE: '{{.ITEM}}' + VERSION: '{{.VERSION}}' + MODULE_NAME: '{{.MODULE_NAME}}' + task: set-module-version-internal + + set-chart-version-internal: + desc: " Sets the versions in the helm chart for a specific component." + run: always + requires: + vars: + - VERSION + - COMPONENT + cmds: + - '{{.TASKFILE_DIR2}}/sed.sh -E "s@version: v?[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+.*@version: {{.VERSION}}@1" "{{.ROOT_DIR2}}/charts/{{.COMPONENT}}/Chart.yaml"' + - '{{.TASKFILE_DIR2}}/sed.sh -E "s@appVersion: v?[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+.*@appVersion: {{.VERSION}}@1" "{{.ROOT_DIR2}}/charts/{{.COMPONENT}}/Chart.yaml"' + - '{{.TASKFILE_DIR2}}/sed.sh -E "s@ tag: .*@ tag: {{.VERSION}}@" "{{.ROOT_DIR2}}/charts/{{.COMPONENT}}/values.yaml"' + internal: true + + set-module-version-internal: + desc: " Sets the imported version of a nested module in the root go.mod file." + run: always + requires: + vars: + - VERSION + - MODULE + - MODULE_NAME + cmds: + - '{{.TASKFILE_DIR2}}/sed.sh -E "s@ {{.MODULE_NAME}}/{{.MODULE}} .*@ {{.MODULE_NAME}}/{{.MODULE}} {{.VERSION}}@" "{{.ROOT_DIR2}}/go.mod"' + internal: true diff --git a/tasks_tools.yaml b/tasks_tools.yaml new file mode 100644 index 0000000..0fe0997 --- /dev/null +++ b/tasks_tools.yaml @@ -0,0 +1,159 @@ +version: 3 + +tasks: + localbin: + desc: " Ensure that the folder specified in LOCALBIN exists." + run: once + requires: + vars: + - LOCALBIN + status: + - test -d {{.LOCALBIN}} + cmds: + - 'echo "localbin: {{.LOCALBIN}}"' + - mkdir -p {{.LOCALBIN}} + internal: true + + localtmp: + desc: " Ensure that the folder specified in LOCALTMP exists." + run: once + requires: + vars: + - LOCALTMP + status: + - test -d {{.LOCALTMP}} + cmds: + - 'echo "localtmp: {{.LOCALTMP}}"' + - mkdir -p {{.LOCALTMP}} + internal: true + + controller-gen: + desc: " Ensure that controller-gen is installed." + run: once + requires: + vars: + - CONTROLLER_GEN + - CONTROLLER_GEN_VERSION + deps: + - localbin + status: + - test -x {{.CONTROLLER_GEN}} + - '{{.CONTROLLER_GEN}} --version | grep -q "{{.CONTROLLER_GEN_VERSION}}"' + cmds: + - 'GOBIN="{{.LOCALBIN}}" go install sigs.k8s.io/controller-tools/cmd/controller-gen@{{.CONTROLLER_GEN_VERSION}}' + internal: true + + goimports: + desc: " Ensure that goimports is installed." + run: once + requires: + vars: + - FORMATTER + - FORMATTER_VERSION + deps: + - localbin + status: + - test -x {{.FORMATTER}} + - test -f {{.LOCALBIN}}/formatter_version + - 'cat {{.LOCALBIN}}/formatter_version | grep -q "{{.FORMATTER_VERSION}}"' + cmds: + - 'GOBIN="{{.LOCALBIN}}" go install golang.org/x/tools/cmd/goimports@{{.FORMATTER_VERSION}}' + - echo -n "{{.FORMATTER_VERSION}}" > {{.LOCALBIN}}/formatter_version + internal: true + + golangci-lint: + desc: " Ensure that golangci-lint is installed." + run: once + requires: + vars: + - LINTER + - LINTER_VERSION + deps: + - localbin + status: + - test -x {{.LINTER}} + - '{{.LINTER}} --version | grep -q {{.LINTER_VERSION | trimPrefix "v"}}' + cmds: + - 'curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b {{.LOCALBIN}} {{.LINTER_VERSION}}' + internal: true + + jq: + desc: " Ensure that jq is installed." + run: once + requires: + vars: + - JQ + - JQ_VERSION + deps: + - localbin + vars: + JQ_OS: + sh: 'if [[ "{{.OS}}" == "darwin" ]]; then echo "macos"; else echo "{{.OS}}"; fi' # jq uses 'macos' instead of 'darwin' + status: + - test -x {{.JQ}} + - '{{.JQ}} --version | grep -q "{{.JQ_VERSION | trimPrefix "v"}}"' + cmds: + - 'curl -sfL "https://github.com/jqlang/jq/releases/download/jq-{{.JQ_VERSION | trimPrefix "v"}}/jq-{{.JQ_OS}}-{{.ARCH}}" --output "{{.JQ}}"' + - 'chmod +x "{{.JQ}}"' + internal: true + + helm: + desc: " Ensure that helm is installed." + run: once + requires: + vars: + - HELM + - HELM_VERSION + deps: + - localbin + status: + - test -x "{{.HELM}}" + - '"{{.HELM}}" version --short | grep -q {{.HELM_VERSION}}' + vars: + tmpdir: + sh: 'mktemp -d' + cmds: + - 'mkdir -p {{.tmpdir}}/helm-unpacked' + - 'curl -sfL "https://get.helm.sh/helm-{{.HELM_VERSION}}-{{.OS}}-{{.ARCH}}.tar.gz" --output "{{.tmpdir}}/helm.tar.gz"' + - 'tar -xzf "{{.tmpdir}}/helm.tar.gz" --directory "{{.tmpdir}}/helm-unpacked"' + - 'mv "{{.tmpdir}}/helm-unpacked/{{.OS}}-{{.ARCH}}/helm" "{{.HELM}}"' + - 'chmod +x "{{.HELM}}"' + - 'rm -rf "{{.tmpdir}}"' + internal: true + + yaml2json: + desc: " Ensure that yaml2json is installed." + run: once + requires: + vars: + - YAML2JSON + - YAML2JSON_VERSION + deps: + - localbin + status: + - 'test -x "{{.YAML2JSON}}"' + - '"{{.YAML2JSON}}" --version | grep -q "{{.YAML2JSON_VERSION | trimPrefix "v"}}"' + cmds: + - 'curl -sfL "https://github.com/bronze1man/yaml2json/releases/download/{{.YAML2JSON_VERSION}}/yaml2json_{{.OS}}_{{.ARCH}}" --output "{{.LOCALBIN}}/yaml2json"' + - 'chmod +x "{{.YAML2JSON}}"' + internal: true + + ocm: + desc: " Ensure that the ocm CLI is installed." + run: once + requires: + vars: + - OCM + - OCM_VERSION + deps: + - localbin + status: + - 'test -x "{{.OCM}}"' + - '"{{.OCM}}" --version | grep -q "{{.OCM_VERSION | trimPrefix "v"}}"' + vars: + tmpdir: + sh: 'mktemp -d' + cmds: + - 'curl -sSfL https://ocm.software/install.sh | OCM_VERSION="{{.OCM_VERSION | trimPrefix "v"}}" bash -s "{{.tmpdir}}"' + - 'mv "{{.tmpdir}}/ocm" "{{.OCM}}"' + internal: true diff --git a/tasks_val.yaml b/tasks_val.yaml new file mode 100644 index 0000000..33bb0ba --- /dev/null +++ b/tasks_val.yaml @@ -0,0 +1,108 @@ +version: 3 + +includes: + tools: + taskfile: tasks_tools.yaml + internal: true + +tasks: + # This is a dummy task that serves as a separator between task namespaces in the 'task -l' output. + "---": + desc: "{{.VAL_SEP}}" + cmds: + - cmd: echo "{{.SEP_MSG}}" + silent: true + + all: + desc: " Combines all validation tasks." + run: once + cmds: + - task: validate + - task: test + + test: + desc: " Run all tests." + run: once + requires: + vars: + - CODE_DIRS + cmds: + - 'PROJECT_ROOT="{{.ROOT_DIR2}}" NESTED_MODULES="{{.NESTED_MODULES}}" {{.TASKFILE_DIR2}}/run-tests.sh {{.CODE_DIRS}}' + + validate: + desc: " Combines all validation tasks except for tests." + run: once + aliases: + - verify + - check + cmds: + - task: vet + - task: lint + - task: format-imports + - task: docs + + docs: + desc: " Checks if the documentation index is up-to-date." + run: once + status: + - '[ "{{.GENERATE_DOCS_INDEX | default "false"}}" == "false" ]' + cmds: + - task: docs-internal + + docs-internal: + desc: " Checks if the documentation index is up-to-date." + run: once + deps: + - tools:jq + cmds: + - 'PROJECT_ROOT="{{.ROOT_DIR2}}" JQ="{{.JQ}}" {{.TASKFILE_DIR2}}/verify-docs-index.sh' + internal: true + + vet-internal: + desc: " Run 'go vet' for a single module." + run: once + requires: + vars: + - MODULE + cmds: + - '( cd {{.ROOT_DIR2}}/{{.MODULE}}; go vet ./...; )' + internal: true + + vet: + desc: " Run 'go vet' for all modules." + run: once + cmds: + - for: + var: NESTED_MODULES + vars: + MODULE: '{{.ITEM}}' + task: vet-internal + - vars: + MODULE: "" + task: vet-internal + + lint: + desc: " Run 'golangci-lint'." + run: once + deps: + - tools:golangci-lint + requires: + vars: + - LINTER + cmds: + - 'PROJECT_ROOT="{{.ROOT_DIR2}}" NESTED_MODULES="{{.NESTED_MODULES}}" LINTER="{{.LINTER}}" {{.TASKFILE_DIR2}}/run-lint.sh {{.CODE_DIRS}}' + + format-imports: + desc: " Check if 'goimports' has been run." + run: once + deps: + - tools:goimports + requires: + vars: + - CODE_DIRS + - MODULE_NAME + vars: + code_dirs: + sh: '{{.TASKFILE_DIR2}}/unfold.sh --clean --no-unfold --inline {{.CODE_DIRS}}' # goimports doesn't like the '/...' syntax + cmds: + - 'tmp=$({{.FORMATTER}} -l -local={{.MODULE_NAME}} {{.code_dirs}}); if [[ "$tmp" ]]; then echo "Unformatted files detected:"; echo "$tmp"; exit 1; fi' diff --git a/unfold.sh b/unfold.sh new file mode 100755 index 0000000..740fd84 --- /dev/null +++ b/unfold.sh @@ -0,0 +1,93 @@ +#!/bin/bash + +set -euo pipefail + +# This is a small helper script that takes a list of paths and unfolds them: +# If the path ends with '/...', the path itself (without '/...') and all of its subfolders are printed. +# Otherwise, only the path is printed. +# +# Paths that don't exist will cause an error. +# +# Options: +# +# --absolute +# If active, converts all paths to absolute paths. Overrides --clean. +# +# --clean +# If active, all paths are printed relative to the working directory, with './' and '../' resolved where possible. +# +# --no-unfold +# If active, does simply remove '/...' suffixes instead of unfolding the corresponding paths. +# +# --inline +# If active, paths are separated by spaces instead of newlines. +# +# Note that each option's flag +# - toggles that option between active and inactive (with inactive being the default when no flag for that option is specified) +# - can be used multiple times, toggling the option on and off as described above +# - affects only the paths that are specified after it in the command + +# 'toggle X' flips $X between 'true' and 'false'. +function toggle() { + if eval \$$1; then + eval "$1=false" + else + eval "$1=true" + fi +} + +absolute=false +clean=false +no_unfold=false +inline=false +inline_printed=false +for f in "$@"; do + case "$f" in + "--absolute") + toggle absolute + ;; + "--clean") + toggle clean + ;; + "--no-unfold") + toggle no_unfold + ;; + "--inline") + toggle inline + ;; + *) + depth_mod="" + if [[ "$f" == */... ]]; then + f="${f%/...}" # cut off '/...' + if $no_unfold; then + depth_mod="-maxdepth 0" + fi + else + depth_mod="-maxdepth 0" + fi + if $absolute; then + f="$(realpath "$f")" + elif $clean; then + f="$(realpath --relative-base="$PWD" "$f")" + fi + if tmp=$(find "$f" $depth_mod -type d 2>&1); then + if $inline; then + if $inline_printed; then + echo -n " " + fi + echo -n "$tmp" + inline_printed=true + else + if $inline_printed; then + echo + fi + echo "$tmp" + inline_printed=false + fi + else + echo "error unfolding path '$f': $tmp" >&2 + exit 1 + fi + ;; + esac +done diff --git a/verify-docs-index.sh b/verify-docs-index.sh new file mode 100755 index 0000000..65f1ca0 --- /dev/null +++ b/verify-docs-index.sh @@ -0,0 +1,15 @@ +#!/bin/bash + +set -euo pipefail +source "$(realpath "$(dirname $0)/environment.sh")" + +echo "> Checking if documentation index needs changes ..." +doc_index_file="$PROJECT_ROOT/docs/README.md" +tmp_compare_file=$(mktemp) +"$COMMON_SCRIPT_DIR/generate-docs-index.sh" "$tmp_compare_file" >/dev/null +if ! cmp -s "$doc_index_file" "$tmp_compare_file"; then + echo "The documentation index requires changes." + echo "Please run 'make generate-docs' to update it." + exit 1 +fi +echo "Documentation index is up-to-date." | indent 1