Skip to content

Commit 02c114a

Browse files
gangwgrgangwgr
committed
Add JUnit reporter configuration and comprehensive README for OTE
- Add JUnit reporter configuration to generate XML reports for CI integration - Create comprehensive README.md for OTE implementation based on openshift-apiserver example - Add report.json and junit.xml to .gitignore to exclude generated files - Document test execution, CI integration, and architecture details
1 parent c919a88 commit 02c114a

File tree

3 files changed

+162
-0
lines changed

3 files changed

+162
-0
lines changed

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,3 +4,5 @@
44
_output
55
telepresence.log
66
.openshift-tests-extension/openshift_payload_*.json
7+
report.json
8+
junit.xml

‎cmd/cluster-kube-controller-manager-operator-tests-ext/main.go‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,10 @@ func runGinkgoTest(ctx context.Context, testName string) *extensiontests.Extensi
7575
// Run the test suite with focus on specific test
7676
suiteConfig, reporterConfig := ginkgo.GinkgoConfiguration()
7777
suiteConfig.FocusStrings = []string{escapeRegexChars(testName)}
78+
79+
// Configure JUnit reporter for CI integration
80+
reporterConfig.JUnitReport = "junit.xml"
81+
reporterConfig.JSONReport = "report.json"
7882

7983
passed := ginkgo.RunSpecs(NewGinkgoTestingT(), "OpenShift Kube Controller Manager Operator Test Suite", suiteConfig, reporterConfig)
8084

‎test/extended/README.md‎

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
# OpenShift Kube Controller Manager Operator Tests Extension
2+
========================================================
3+
4+
This repository contains the tests for the OpenShift Kube Controller Manager Operator for OpenShift.
5+
These tests run against OpenShift clusters and are meant to be used in the OpenShift CI/CD pipeline.
6+
They use the framework: https://github.com/openshift-eng/openshift-tests-extension
7+
8+
## How to Run the Tests Locally
9+
10+
| Command | Description |
11+
|---------|-------------|
12+
| `make tests-ext-build` | Builds the test extension binary. |
13+
| `./cluster-kube-controller-manager-operator-tests-ext list` | Lists all available test cases. |
14+
| `./cluster-kube-controller-manager-operator-tests-ext run-suite <suite-name>` | Runs a test suite. e.g., `openshift/cluster-kube-controller-manager-operator/conformance/parallel` |
15+
| `./cluster-kube-controller-manager-operator-tests-ext run-test <test-name>` | Runs one specific test. |
16+
17+
## How to Run the Tests Locally
18+
19+
The tests can be run locally using the `cluster-kube-controller-manager-operator-tests-ext` binary against an OpenShift cluster.
20+
Use the environment variable `KUBECONFIG` to point to your cluster configuration file such as:
21+
22+
```shell
23+
export KUBECONFIG=path/to/kubeconfig
24+
./cluster-kube-controller-manager-operator-tests-ext run-test <test-name>
25+
```
26+
27+
### Local Test using OCP
28+
29+
1. Use the `Cluster Bot` to create an OpenShift cluster.
30+
31+
**Example:**
32+
```shell
33+
launch 4.20 gcp,techpreview
34+
```
35+
36+
2. Set the `KUBECONFIG` environment variable to point to your OpenShift cluster configuration file.
37+
38+
**Example:**
39+
```shell
40+
mv ~/Downloads/cluster-bot-2025-08-06-082741.kubeconfig ~/.kube/cluster-bot.kubeconfig
41+
export KUBECONFIG=~/.kube/cluster-bot.kubeconfig
42+
```
43+
44+
3. Run the tests using the `cluster-kube-controller-manager-operator-tests-ext` binary.
45+
46+
**Example:**
47+
```shell
48+
./cluster-kube-controller-manager-operator-tests-ext run-suite openshift/cluster-kube-controller-manager-operator/all
49+
```
50+
51+
### Running with JUnit Output
52+
53+
To generate JUnit XML reports for CI integration:
54+
55+
```shell
56+
./cluster-kube-controller-manager-operator-tests-ext run-suite openshift/cluster-kube-controller-manager-operator/conformance/parallel --junit-path junit.xml
57+
```
58+
59+
This generates both OTE framework and Ginkgo JUnit XML reports that can be integrated into CI systems.
60+
61+
## Available Test Suites
62+
63+
| Suite Name | Description |
64+
|------------|-------------|
65+
| `openshift/cluster-kube-controller-manager-operator/conformance/parallel` | Parallel conformance tests |
66+
| `openshift/cluster-kube-controller-manager-operator/conformance/serial` | Serial conformance tests |
67+
| `openshift/cluster-kube-controller-manager-operator/optional/slow` | Optional slow tests |
68+
| `openshift/cluster-kube-controller-manager-operator/all` | All tests |
69+
70+
## CI/CD Integration
71+
72+
The tests are integrated into the OpenShift CI/CD pipeline through the release configuration.
73+
The CI configuration runs the OTE binary and generates JUnit reports for test result tracking.
74+
75+
**Example CI step:**
76+
```yaml
77+
- as: tests-extension
78+
commands: |
79+
echo "Build binary cluster-kube-controller-manager-operator-tests-ext"
80+
make tests-ext-build
81+
echo "Running ./cluster-kube-controller-manager-operator-tests-ext with sanity test"
82+
./cluster-kube-controller-manager-operator-tests-ext run-suite "openshift/cluster-kube-controller-manager-operator/conformance/parallel" --junit-path junit.xml
83+
from: src
84+
resources:
85+
requests:
86+
cpu: 100m
87+
```
88+
89+
## Makefile Commands
90+
91+
| Target | Description |
92+
|--------|-------------|
93+
| `make build` | Builds the operator binary. |
94+
| `make tests-ext-build` | Builds the test extension binary. |
95+
| `make tests-ext-update` | Updates the metadata JSON file and cleans machine-specific codeLocations. |
96+
| `make verify` | Runs formatting, vet, and linter. |
97+
98+
**Note:** Metadata is stored in: `.openshift-tests-extension/cluster-kube-controller-manager-operator.json`
99+
100+
## FAQ
101+
102+
### Why don't we have a Dockerfile for `cluster-kube-controller-manager-operator-tests-ext`?
103+
104+
We do not provide a Dockerfile for `cluster-kube-controller-manager-operator-tests-ext` because building and shipping a
105+
standalone image for this test binary would introduce unnecessary complexity.
106+
107+
Technically, it is possible to create a new OpenShift component just for the
108+
tests and add a corresponding test image to the payload. However, doing so requires
109+
onboarding a new component, setting up build pipelines, and maintaining image promotion
110+
and test configuration — all of which adds overhead.
111+
112+
From the OpenShift architecture point of view:
113+
114+
1. Tests for payload components are part of the product. Many users (such as storage vendors, or third-party CNIs)
115+
rely on these tests to validate that their solutions are compatible and conformant with OpenShift.
116+
117+
2. Adding new images to the payload comes with significant overhead and cost.
118+
It is generally preferred to include tests in the same image as the component
119+
being tested whenever possible.
120+
121+
### Why do we need to run `make tests-ext-update`?
122+
123+
Running `make tests-ext-update` ensures that each test gets a unique and stable **TestID** over time.
124+
125+
The TestID is used to identify tests across the OpenShift CI/CD pipeline and reporting tools like Sippy.
126+
It helps track test results, detect regressions, and ensures the correct tests are
127+
executed and reported.
128+
129+
This step is important whenever you add, rename, or delete a test.
130+
More information:
131+
- https://github.com/openshift/enhancements/blob/master/enhancements/testing/openshift-tests-extension.md#test-id
132+
- https://github.com/openshift-eng/ci-test-mapping
133+
134+
### How to get help with OTE?
135+
136+
For help with the OpenShift Tests Extension (OTE), you can reach out on the #wg-openshift-tests-extension Slack channel.
137+
138+
## Test Implementation Details
139+
140+
The OTE implementation uses the registry-based approach from the `openshift-tests-extension` framework:
141+
142+
- **Registry Pattern**: Uses `extension.NewRegistry()` and `extension.NewExtension()` for clean, maintainable code
143+
- **JUnit Integration**: Generates both OTE framework and Ginkgo JUnit XML reports
144+
- **Test Discovery**: Automatically discovers and registers Ginkgo tests from the `test/extended` package
145+
- **CI Ready**: Includes proper JUnit reporter configuration for CI integration
146+
147+
## Architecture
148+
149+
The OTE binary (`cluster-kube-controller-manager-operator-tests-ext`) provides:
150+
151+
1. **Test Discovery**: Lists all available tests and suites
152+
2. **Test Execution**: Runs individual tests or entire suites
153+
3. **Result Reporting**: Generates JSON and JUnit XML output
154+
4. **CI Integration**: Provides standardized output formats for CI systems
155+
156+
The implementation follows OpenShift best practices and integrates seamlessly with the existing CI/CD pipeline.

0 commit comments

Comments
 (0)