Skip to content

Commit 6ac93e4

Browse files
ritmunritmun
committed
e2e suite is boilerplated
1 parent e1bd2e4 commit 6ac93e4

16 files changed

+433
-79
lines changed

‎.gitattributes

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
### BEGIN BOILERPLATE GENERATED -- DO NOT EDIT ###
2+
### This block must be the last thing in your ###
3+
### .gitattributes file; otherwise the 'validate' ###
4+
### CI check will fail. ###
5+
# Used to ensure nobody mucked with boilerplate files.
6+
boilerplate/_lib/freeze-check linguist-generated=false
7+
# Show the boilerplate commit hash update. It's only one line anyway.
8+
boilerplate/_data/last-boilerplate-commit linguist-generated=false
9+
# Used by freeze-check. Good place for attackers to inject badness.
10+
boilerplate/update linguist-generated=false
11+
# Make sure attackers can't hide changes to this configuration
12+
.gitattributes linguist-generated=false
13+
### END BOILERPLATE GENERATED ###

‎boilerplate/_data/backing-image-tag

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
image-v7.4.0

‎boilerplate/_data/last-boilerplate-commit

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
519eec21abfddb4d65bbde57e4f3b13368f54e6b

‎boilerplate/generated-includes.mk

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,5 @@
22
# This file automatically includes any *.mk files in your subscribed
33
# conventions. Please ensure your base Makefile includes only this file.
44
include boilerplate/_lib/boilerplate.mk
5+
include boilerplate/openshift/golang-osd-e2e/project.mk
6+
include boilerplate/openshift/golang-osd-e2e/standard.mk

‎boilerplate/openshift/golang-osd-e2e/OWNERS

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
reviewers:
2+
- srep-infra-cicd
3+
approvers:
4+
- srep-infra-cicd

‎boilerplate/openshift/golang-osd-e2e/README.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Conventions for Ginkgo based e2e tests
2+
3+
- [Conventions for Ginkgo based e2e tests](#conventions-for-ginkgo-based-e2e-tests)
4+
- [Consuming](#consuming)
5+
- [`make` targets and functions.](#make-targets-and-functions)
6+
- [E2E Test](#e2e-test)
7+
- [Local Testing](#e2e-local-testing)
8+
9+
## Consuming
10+
Currently, this convention is only intended for OSD operators. To adopt this convention, your `boilerplate/update.cfg` should include:
11+
12+
```
13+
openshift/golang-osd-e2e
14+
```
15+
16+
## `make` targets and functions.
17+
18+
**Note:** Your repository's main `Makefile` needs to be edited to include:
19+
20+
```
21+
include boilerplate/generated-includes.mk
22+
```
23+
24+
One of the primary purposes of these `make` targets is to allow you to
25+
standardize your prow and app-sre pipeline configurations using the
26+
following:
27+
28+
### E2e Test
29+
30+
| `make` target | Purpose |
31+
|------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
32+
| `e2e-binary-build` | Compiles ginkgo tests under test/e2e and creates the ginkgo binary. |
33+
| `e2e-image-build-push` | Builds e2e image and pushes to operator's quay repo. Image name is defaulted to <operator-image-name>-test-harness. Quay repository must be created beforehand. |
34+
35+
#### E2E Local Testing
36+
37+
Please follow [this README](https://github.com/openshift/ops-sop/blob/master/v4/howto/osde2e/operator-test-harnesses.md#using-ginkgo) to run your e2e tests locally
38+

‎boilerplate/openshift/golang-osd-e2e/e2e-template.yml

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# THIS FILE IS GENERATED BY BOILERPLATE. DO NOT EDIT.
2+
apiVersion: template.openshift.io/v1
3+
kind: Template
4+
metadata:
5+
name: osde2e-focused-tests
6+
parameters:
7+
- name: OSDE2E_CONFIGS
8+
required: true
9+
- name: TEST_IMAGE
10+
required: true
11+
- name: OCM_CLIENT_ID
12+
required: false
13+
- name: OCM_CLIENT_SECRET
14+
required: false
15+
- name: OCM_CCS
16+
required: false
17+
- name: AWS_ACCESS_KEY_ID
18+
required: false
19+
- name: AWS_SECRET_ACCESS_KEY
20+
required: false
21+
- name: CLOUD_PROVIDER_REGION
22+
required: false
23+
- name: GCP_CREDS_JSON
24+
required: false
25+
- name: JOBID
26+
generate: expression
27+
from: "[0-9a-z]{7}"
28+
- name: IMAGE_TAG
29+
value: ''
30+
required: true
31+
- name: LOG_BUCKET
32+
value: 'osde2e-logs'
33+
objects:
34+
- apiVersion: batch/v1
35+
kind: Job
36+
metadata:
37+
name: osde2e-${OPERATOR_NAME}-${IMAGE_TAG}-${JOBID}
38+
spec:
39+
backoffLimit: 0
40+
template:
41+
spec:
42+
restartPolicy: Never
43+
containers:
44+
- name: osde2e
45+
image: quay.io/redhat-services-prod/osde2e-cicada-tenant/osde2e:latest
46+
command:
47+
- /osde2e
48+
args:
49+
- test
50+
- --only-health-check-nodes
51+
- --skip-destroy-cluster
52+
- --skip-must-gather
53+
- --configs
54+
- ${OSDE2E_CONFIGS}
55+
securityContext:
56+
runAsNonRoot: true
57+
allowPrivilegeEscalation: false
58+
capabilities:
59+
drop: ["ALL"]
60+
seccompProfile:
61+
type: RuntimeDefault
62+
env:
63+
- name: AD_HOC_TEST_IMAGES
64+
value: ${TEST_IMAGE}:${IMAGE_TAG}
65+
- name: OCM_CLIENT_ID
66+
value: ${OCM_CLIENT_ID}
67+
- name: OCM_CLIENT_SECRET
68+
value: ${OCM_CLIENT_SECRET}
69+
- name: OCM_CCS
70+
value: ${OCM_CCS}
71+
- name: AWS_ACCESS_KEY_ID
72+
value: ${AWS_ACCESS_KEY_ID}
73+
- name: AWS_SECRET_ACCESS_KEY
74+
value: ${AWS_SECRET_ACCESS_KEY}
75+
- name: CLOUD_PROVIDER_REGION
76+
value: ${CLOUD_PROVIDER_REGION}
77+
- name: GCP_CREDS_JSON
78+
value: ${GCP_CREDS_JSON}
79+
- name: LOG_BUCKET
80+
value: ${LOG_BUCKET}

‎boilerplate/openshift/golang-osd-e2e/project.mk

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# Project specific values
2+
OPERATOR_NAME?=$(shell sed -n 's/.*OperatorName .*"\([^"]*\)".*/\1/p' config/config.go)
3+
4+
E2E_IMAGE_REGISTRY?=quay.io
5+
E2E_IMAGE_REPOSITORY?=app-sre
6+
E2E_IMAGE_NAME?=$(OPERATOR_NAME)-e2e
7+
8+
9+
REGISTRY_USER?=$(QUAY_USER)
10+
REGISTRY_TOKEN?=$(QUAY_TOKEN)

‎boilerplate/openshift/golang-osd-e2e/standard.mk

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
# Validate variables in project.mk exist
2+
ifndef OPERATOR_NAME
3+
$(error OPERATOR_NAME is not set; only operators should consume this convention; check project.mk file)
4+
endif
5+
ifndef E2E_IMAGE_REGISTRY
6+
$(error E2E_IMAGE_REGISTRY is not set; check project.mk file)
7+
endif
8+
ifndef E2E_IMAGE_REPOSITORY
9+
$(error E2E_IMAGE_REPOSITORY is not set; check project.mk file)
10+
endif
11+
12+
# Use current commit as e2e image tag
13+
CURRENT_COMMIT=$(shell git rev-parse --short=7 HEAD)
14+
E2E_IMAGE_TAG=$(CURRENT_COMMIT)
15+
16+
### Accommodate docker or podman
17+
#
18+
# The docker/podman creds cache needs to be in a location unique to this
19+
# invocation; otherwise it could collide across jenkins jobs. We'll use
20+
# a .docker folder relative to pwd (the repo root).
21+
CONTAINER_ENGINE_CONFIG_DIR = .docker
22+
JENKINS_DOCKER_CONFIG_FILE = /var/lib/jenkins/.docker/config.json
23+
export REGISTRY_AUTH_FILE = ${CONTAINER_ENGINE_CONFIG_DIR}/config.json
24+
25+
# If this configuration file doesn't exist, podman will error out. So
26+
# we'll create it if it doesn't exist.
27+
ifeq (,$(wildcard $(REGISTRY_AUTH_FILE)))
28+
$(shell mkdir -p $(CONTAINER_ENGINE_CONFIG_DIR))
29+
# Copy the node container auth file so that we get access to the registries the
30+
# parent node has access to
31+
$(shell if test -f $(JENKINS_DOCKER_CONFIG_FILE); then cp $(JENKINS_DOCKER_CONFIG_FILE) $(REGISTRY_AUTH_FILE); fi)
32+
endif
33+
34+
# ==> Docker uses --config=PATH *before* (any) subcommand; so we'll glue
35+
# that to the CONTAINER_ENGINE variable itself. (NOTE: I tried half a
36+
# dozen other ways to do this. This was the least ugly one that actually
37+
# works.)
38+
ifndef CONTAINER_ENGINE
39+
CONTAINER_ENGINE=$(shell command -v podman 2>/dev/null || echo docker --config=$(CONTAINER_ENGINE_CONFIG_DIR))
40+
endif
41+
42+
REGISTRY_USER ?=
43+
REGISTRY_TOKEN ?=
44+
45+
# TODO: Figure out how to discover this dynamically
46+
OSDE2E_CONVENTION_DIR := boilerplate/openshift/golang-osd-operator-osde2e
47+
48+
# log into quay.io
49+
.PHONY: container-engine-login
50+
container-engine-login:
51+
@test "${REGISTRY_USER}" != "" && test "${REGISTRY_TOKEN}" != "" || (echo "REGISTRY_USER and REGISTRY_TOKEN must be defined" && exit 1)
52+
mkdir -p ${CONTAINER_ENGINE_CONFIG_DIR}
53+
@${CONTAINER_ENGINE} login -u="${REGISTRY_USER}" -p="${REGISTRY_TOKEN}" quay.io
54+
55+
######################
56+
# Targets used by e2e test suite
57+
######################
58+
59+
# create binary
60+
.PHONY: e2e-binary-build
61+
e2e-binary-build: GOFLAGS_MOD=-mod=mod
62+
e2e-binary-build: GOENV=GOOS=${GOOS} GOARCH=${GOARCH} CGO_ENABLED=0 GOFLAGS="${GOFLAGS_MOD}"
63+
e2e-binary-build:
64+
go mod tidy
65+
go test ./test/e2e -v -c --tags=osde2e -o e2e.test
66+
67+
# push e2e image tagged as latest and as repo commit hash
68+
.PHONY: e2e-image-build-push
69+
e2e-image-build-push:
70+
${CONTAINER_ENGINE} build --pull -f test/e2e/Dockerfile -t $(E2E_IMAGE_REGISTRY)/$(E2E_IMAGE_REPOSITORY)/$(E2E_IMAGE_NAME):$(E2E_IMAGE_TAG) .
71+
${CONTAINER_ENGINE} tag $(E2E_IMAGE_REGISTRY)/$(E2E_IMAGE_REPOSITORY)/$(E2E_IMAGE_NAME):$(E2E_IMAGE_TAG) $(E2E_IMAGE_REGISTRY)/$(E2E_IMAGE_REPOSITORY)/$(E2E_IMAGE_NAME):latest
72+
${CONTAINER_ENGINE} push $(E2E_IMAGE_REGISTRY)/$(E2E_IMAGE_REPOSITORY)/$(E2E_IMAGE_NAME):$(E2E_IMAGE_TAG)
73+
${CONTAINER_ENGINE} push $(E2E_IMAGE_REGISTRY)/$(E2E_IMAGE_REPOSITORY)/$(E2E_IMAGE_NAME):latest

‎boilerplate/openshift/golang-osd-e2e/update

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
#!/usr/bin/env bash
2+
3+
set -e
4+
5+
source $CONVENTION_ROOT/_lib/common.sh
6+
7+
# No PRE
8+
[[ "$1" == "PRE" ]] && exit 0
9+
10+
# Expect POST
11+
[[ "$1" == "POST" ]] || err "Got a parameter I don't understand: '$1'. Did the infrastructure change?"
12+
13+
REPO_ROOT=$(git rev-parse --show-toplevel)
14+
OPERATOR_NAME=$(sed -n 's/.*OperatorName .*=.*"\([^"]*\)".*/\1/p' "${REPO_ROOT}/config/config.go")
15+
E2E_SUITE_DIRECTORY=$REPO_ROOT/test/e2e
16+
17+
# Update operator name in templates
18+
OPERATOR_UNDERSCORE_NAME=${OPERATOR_NAME//-/_}
19+
OPERATOR_PROPER_NAME=$(echo "$OPERATOR_NAME" | sed 's/-/ /g' | awk '{for(i=1;i<=NF;i++){ $i=toupper(substr($i,1,1)) substr($i,2) }}1')
20+
OPERATOR_NAME_CAMEL_CASE=${OPERATOR_PROPER_NAME// /}
21+
22+
mkdir -p "${E2E_SUITE_DIRECTORY}"
23+
24+
E2E_SUITE_BUILDER_IMAGE=registry.ci.openshift.org/openshift/release:rhel-8-release-golang-1.23-openshift-4.19
25+
if [[ -n ${KONFLUX_BUILDS} ]]; then
26+
E2E_SUITE_BUILDER_IMAGE="brew.registry.redhat.io/rh-osbs/openshift-golang-builder:rhel_8_1.23"
27+
fi
28+
29+
echo "syncing ${E2E_SUITE_DIRECTORY}/Dockerfile"
30+
tee "${E2E_SUITE_DIRECTORY}/Dockerfile" <<EOF
31+
# THIS FILE IS GENERATED BY BOILERPLATE. DO NOT EDIT.
32+
FROM ${E2E_SUITE_BUILDER_IMAGE} as builder
33+
WORKDIR /go/src/github.com/openshift/$OPERATOR_NAME/
34+
COPY . .
35+
RUN CGO_ENABLED=0 GOFLAGS="-mod=mod" go test ./test/e2e -v -c --tags=osde2e -o /e2e.test
36+
37+
FROM registry.access.redhat.com/ubi8/ubi-minimal:latest
38+
COPY --from=builder ./e2e.test e2e.test
39+
ENTRYPOINT [ "/e2e.test" ]
40+
EOF
41+
42+
if [ ! -f "${E2E_SUITE_DIRECTORY}/${OPERATOR_UNDERSCORE_NAME}_tests.go" ]; then
43+
echo "syncing ${E2E_SUITE_DIRECTORY}/${OPERATOR_UNDERSCORE_NAME}_tests.go"
44+
tee "${E2E_SUITE_DIRECTORY}/${OPERATOR_UNDERSCORE_NAME}_tests.go" <<EOF
45+
// DO NOT REMOVE TAGS BELOW. IF ANY NEW TEST FILES ARE CREATED UNDER /test/e2e, PLEASE ADD THESE TAGS TO THEM IN ORDER TO BE EXCLUDED FROM UNIT TESTS.
46+
//go:build osde2e
47+
48+
package osde2etests
49+
50+
import "github.com/onsi/ginkgo/v2"
51+
52+
var _ = ginkgo.Describe("$OPERATOR_NAME", func() {
53+
// Add your tests
54+
})
55+
EOF
56+
fi
57+
58+
echo "syncing ${E2E_SUITE_DIRECTORY}/${OPERATOR_UNDERSCORE_NAME}_runner_test.go"
59+
tee "${E2E_SUITE_DIRECTORY}/${OPERATOR_UNDERSCORE_NAME}_runner_test.go" <<EOF
60+
// THIS FILE IS GENERATED BY BOILERPLATE. DO NOT EDIT.
61+
//go:build osde2e
62+
63+
package osde2etests
64+
65+
import (
66+
"os"
67+
"path/filepath"
68+
"testing"
69+
70+
. "github.com/onsi/ginkgo/v2"
71+
. "github.com/onsi/gomega"
72+
)
73+
74+
const (
75+
testResultsDirectory = "/test-run-results"
76+
jUnitOutputFilename = "junit-${OPERATOR_NAME}.xml"
77+
)
78+
79+
// Test entrypoint. osde2e runs this as a test suite on test pod.
80+
func Test${OPERATOR_NAME_CAMEL_CASE}(t *testing.T) {
81+
RegisterFailHandler(Fail)
82+
suiteConfig, reporterConfig := GinkgoConfiguration()
83+
if _, ok := os.LookupEnv("DISABLE_JUNIT_REPORT"); !ok {
84+
reporterConfig.JUnitReport = filepath.Join(testResultsDirectory, jUnitOutputFilename)
85+
}
86+
RunSpecs(t, "${OPERATOR_PROPER_NAME}", suiteConfig, reporterConfig)
87+
}
88+
EOF
89+
90+
tee "${E2E_SUITE_DIRECTORY}/README.md" <<EOF
91+
## Locally running e2e test suite
92+
When updating your operator it's beneficial to add e2e tests for new functionality AND ensure existing functionality is not breaking using e2e tests.
93+
To do this, following steps are recommended
94+
95+
1. Run "make e2e-binary-build" to make sure e2e tests build
96+
2. Deploy your new version of operator in a test cluster
97+
3. Run "go install github.com/onsi/ginkgo/ginkgo@latest"
98+
4. Get kubeadmin credentials from your cluster using
99+
100+
ocm get /api/clusters_mgmt/v1/clusters/(cluster-id)/credentials | jq -r .kubeconfig > /(path-to)/kubeconfig
101+
102+
5. Run test suite using
103+
104+
DISABLE_JUNIT_REPORT=true KUBECONFIG=/(path-to)/kubeconfig ./(path-to)/bin/ginkgo --tags=osde2e -v test/e2e
105+
EOF
106+
107+
sed -e "s/\${OPERATOR_NAME}/${OPERATOR_NAME}/" $(dirname $0)/e2e-template.yml >"${E2E_SUITE_DIRECTORY}/e2e-template.yml"
108+
109+
# todo: remove after file is renamed in ALL consumer repos
110+
if [ -f "${E2E_SUITE_DIRECTORY}/test-harness-template.yml" ]; then
111+
rm -f "${E2E_SUITE_DIRECTORY}/test-harness-template.yml"
112+
fi

0 commit comments

Comments
 (0)