Skip to content

Commit a5da52f

Browse files
ming1013ming1013
committed
Update OTE migration documentation
 - Document component-specific test filtering by [sig-<extension-name>] tag - Explain automatic [OTP] and [Level0] tag additions with examples - Add detailed directory structure strategies (monorepo vs single-module) - Document dynamic dependency resolution from upstream - Document automatic Go toolchain management (GOTOOLCHAIN=auto) - Add before/after examples for test restructuring - Enhance key features section with new capabilities
1 parent bfcf075 commit a5da52f

File tree

2 files changed

+448
-23
lines changed

2 files changed

+448
-23
lines changed

‎plugins/ote-migration/README.md‎

Lines changed: 272 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -23,14 +23,18 @@
2323
8. Provides validation - Gives comprehensive next steps and validation guide
2424

2525
**Key Features:**
26-
- Complete automation - One command handles the entire migration
27-
- Smart repository management with remote detection
28-
- Automatic dependency vendoring (compat_otp, exutil, etc.)
29-
- Two directory strategies - multi-module or single-module
30-
- Git status validation for working directory
31-
- Auto-install go-bindata for generating embedded testdata
32-
- Automatic FixturePath() migration and import updates
33-
- Build verification before completion
26+
- **Complete automation** - One command handles the entire migration
27+
- **Two directory strategies** - Monorepo (integrated) or single-module (isolated)
28+
- **Smart repository management** - Remote detection and update capabilities
29+
- **Dynamic dependency resolution** - Fetches latest Kubernetes and ginkgo versions from upstream
30+
- **Automatic Go toolchain management** - Uses `GOTOOLCHAIN=auto` to download required Go version
31+
- **Automatic test migration** - Replaces FixturePath() calls and updates imports
32+
- **Component-specific test filtering** - Generated main.go filters tests by `[sig-<extension-name>]` tag
33+
- **Test tracking annotations** - Automatically adds [OTP] and [Level0] tags to test cases
34+
- **Build verification** - Validates successful compilation before completion
35+
- **Git status validation** - Ensures clean working directory
36+
- **Auto-install go-bindata** - For generating embedded testdata
37+
- **Dockerfile integration** - Provides templates for both strategies
3438

3539
## Installation
3640

@@ -54,18 +58,269 @@
5458
- Local target repository path (optional)
5559
- Target repository URL (if not using local)
5660

57-
Directory Structure Strategies
61+
## Directory Structure Strategies
5862

59-
Option 1: Monorepo Strategy (Recommended)
63+
The migration tool supports two directory strategies to fit different repository layouts:
6064

61-
Best for component repos with existing cmd/ and test/ directories.
65+
### Option 1: Monorepo Strategy (Recommended for Component Repositories)
6266

63-
Option 2: Single-Module Strategy
67+
Integrates OTE into existing repository structure with **separate test module**.
6468

65-
Best for standalone test extensions or prototyping.
69+
**Structure created:**
70+
```
71+
<repo-root>/
72+
├── cmd/
73+
│ └── extension/
74+
│ └── main.go # OTE entry point
75+
├── test/
76+
│ ├── e2e/
77+
│ │ ├── go.mod # Separate test module
78+
│ │ ├── go.sum
79+
│ │ └── *_test.go # Test files
80+
│ ├── testdata/
81+
│ │ ├── bindata.go # Generated
82+
│ │ └── fixtures.go
83+
│ └── bindata.mk
84+
├── go.mod # Root module (with replace directive)
85+
└── Makefile # Extension target added
86+
```
6687
67-
Resources
88+
**Key characteristics:**
89+
- **Separate test module**: `test/e2e/go.mod` is independent from root `go.mod`
90+
- **Replace directive**: Root `go.mod` includes `replace <module>/test/e2e => ./test/e2e`
91+
- **Integrated build**: Makefile target `tests-ext-build` added to root
92+
- **Binary location**: `bin/<extension-name>-tests-ext`
6893
69-
- https://github.com/openshift/enhancements/pull/1676
70-
- https://github.com/openshift-eng/openshift-tests-extension
71-
- https://github.com/openshift-eng/openshift-tests-extension/blob/main/cmd/example-tests/main.go
94+
**Best for:**
95+
- Component repos with existing `cmd/` and `test/` structure
96+
- Teams wanting OTE tests alongside production code
97+
- Repos that already use multi-module layout
98+
99+
**Example repositories:**
100+
- machine-config-operator
101+
- cluster-network-operator
102+
- router
103+
104+
### Option 2: Single-Module Strategy (For Standalone Test Extensions)
105+
106+
Creates isolated `tests-extension/` directory with **single go.mod**.
107+
108+
**Structure created:**
109+
```
110+
<working-dir>/
111+
└── tests-extension/
112+
├── cmd/
113+
│ └── main.go # OTE entry point
114+
├── test/
115+
│ ├── e2e/
116+
│ │ └── *_test.go
117+
│ └── testdata/
118+
│ ├── bindata.go
119+
│ └── fixtures.go
120+
├── go.mod # Single module
121+
├── go.sum
122+
├── Makefile
123+
└── bindata.mk
124+
```
125+
126+
**Key characteristics:**
127+
- **Single module**: All code in one `go.mod`
128+
- **Self-contained**: No changes to existing repo structure
129+
- **Standalone binary**: `tests-extension/bin/<extension-name>-tests-ext`
130+
131+
**Best for:**
132+
- Standalone test extensions
133+
- Prototyping OTE migrations
134+
- Repos without existing test structure
135+
- Separate test repositories
136+
137+
## Important Notes
138+
139+
### Test Filtering in Generated main.go
140+
141+
The generated `cmd/main.go` (or `cmd/extension/main.go` for monorepo) includes **component-specific test filtering** using the `[sig-<extension-name>]` tag. This ensures only your component's tests are registered with the OTE framework.
142+
143+
**Filter implementation:**
144+
```go
145+
// Build test specs from Ginkgo
146+
allSpecs, err := g.BuildExtensionTestSpecsFromOpenShiftGinkgoSuite()
147+
if err != nil {
148+
panic(fmt.Sprintf("couldn't build extension test specs from ginkgo: %+v", err.Error()))
149+
}
150+
151+
// Filter to only include component-specific tests (tests with [sig-<extension-name>] in name)
152+
var filteredSpecs []*et.ExtensionTestSpec
153+
allSpecs.Walk(func(spec *et.ExtensionTestSpec) {
154+
if strings.Contains(spec.Name, "[sig-<extension-name>]") {
155+
filteredSpecs = append(filteredSpecs, spec)
156+
}
157+
})
158+
specs := et.ExtensionTestSpecs(filteredSpecs)
159+
```
160+
161+
**Why this matters:**
162+
- Without this filter, you'd see **5,000+ upstream Kubernetes tests** in addition to your component tests
163+
- The filter ensures `./bin/<extension-name>-tests-ext list` shows only tests tagged with `[sig-<extension-name>]`
164+
- Tests must have the `[sig-<extension-name>]` tag in their name to be included
165+
166+
**Verification after migration:**
167+
```bash
168+
# Should show only component-specific tests with [sig-<extension-name>] tag
169+
./bin/<extension-name>-tests-ext list
170+
171+
# Count filtered tests
172+
./bin/<extension-name>-tests-ext list | grep -c "\[sig-<extension-name>\]"
173+
```
174+
175+
### Test Tracking Annotations
176+
177+
The migration **automatically modifies test files** to add tracking annotations. This happens in Phase 6 (Test Migration) and restructures test names for better organization.
178+
179+
**Automatic annotations added:**
180+
181+
1. **[OTP]** - Added to ALL Describe blocks
182+
- Marks all tests ported from openshift-tests-private
183+
- Helps track migration progress
184+
- Placement: After `[sig-<extension-name>]` in Describe blocks
185+
- Example: `g.Describe("[sig-router][OTP]", func() { ... })`
186+
187+
2. **[Level0]** - Added to individual It() test cases with "-LEVEL0-" in name
188+
- Identifies level0 conformance tests
189+
- Auto-detected by searching for "-LEVEL0-" string in It() descriptions
190+
- Placement: At the beginning of It() descriptions
191+
- Example: `g.It("[Level0] Router should handle basic routing", func() { ... })`
192+
193+
**Test restructuring performed:**
194+
195+
The migration simplifies test structure by:
196+
- Moving Describe block text into It() descriptions
197+
- Simplifying Describe to just tags: `[sig-<extension-name>][OTP]`
198+
- Prepending `[Level0]` to It() for tests with "-LEVEL0-"
199+
200+
**Before migration:**
201+
```go
202+
g.Describe("[sig-router] Router functionality", func() {
203+
g.It("should handle basic routing -LEVEL0-", func() {
204+
// test code
205+
})
206+
})
207+
```
208+
209+
**After migration:**
210+
```go
211+
g.Describe("[sig-router][OTP]", func() {
212+
g.It("[Level0] Router functionality should handle basic routing -LEVEL0-", func() {
213+
// test code
214+
})
215+
})
216+
```
217+
218+
**Full test name visible in list:**
219+
```
220+
[sig-router][OTP] [Level0] Router functionality should handle basic routing -LEVEL0-
221+
```
222+
223+
**Benefits:**
224+
- **Track migration progress** - Count tests with `[OTP]` tag
225+
- **Identify level0 tests** - Filter by `[Level0]` tag
226+
- **Cleaner test hierarchy** - Describe blocks use tags only
227+
- **Flexibility for test execution** - Run level0 tests separately or in conformance suites
228+
229+
**Verification after migration:**
230+
```bash
231+
# Count total ported tests (all should have [OTP])
232+
./bin/<extension-name>-tests-ext list | grep -c "\[OTP\]"
233+
234+
# Count level0 conformance tests
235+
./bin/<extension-name>-tests-ext list | grep -c "\[Level0\]"
236+
237+
# View restructured test names
238+
./bin/<extension-name>-tests-ext list | head -10
239+
```
240+
241+
### Dynamic Dependency Resolution
242+
243+
The migration tool **fetches latest versions** of critical dependencies directly from upstream repositories instead of copying potentially stale versions from openshift-tests-private.
244+
245+
**What's fetched dynamically:**
246+
247+
1. **Kubernetes dependencies** - From `github.com/openshift/kubernetes` (master branch)
248+
```bash
249+
K8S_LATEST=$(git ls-remote https://github.com/openshift/kubernetes.git refs/heads/master | awk '{print $1}')
250+
# Creates versioned replace directives for all k8s.io/* packages
251+
```
252+
253+
2. **Ginkgo testing framework** - From `github.com/openshift/onsi-ginkgo` (v2.27.2-openshift-4.22 branch)
254+
```bash
255+
GINKGO_LATEST=$(git ls-remote https://github.com/openshift/onsi-ginkgo.git refs/heads/v2.27.2-openshift-4.22 | awk '{print $1}')
256+
```
257+
258+
3. **Origin dependencies** - From `github.com/openshift/origin` (main branch)
259+
```bash
260+
ORIGIN_LATEST=$(git ls-remote https://github.com/openshift/origin.git refs/heads/main | awk '{print $1}')
261+
```
262+
263+
**Why this matters:**
264+
265+
Prevents common API incompatibility errors:
266+
- ❌ `undefined: ginkgo.NewWriter`
267+
- ❌ `undefined: diff.Diff` (library-go)
268+
- ❌ `undefined: otelgrpc.UnaryClientInterceptor` (cri-client)
269+
- ❌ `structured-merge-diff/v6 vs v4` type mismatches
270+
271+
**Old behavior (problematic):**
272+
- Copied all replace directives from `openshift-tests-private/go.mod`
273+
- Could be weeks or months out of date
274+
- Led to build failures with newer dependencies
275+
276+
**New behavior (reliable):**
277+
- Fetches latest commit hashes at migration time
278+
- Generates fresh pseudo-versions
279+
- Ensures compatibility with current OpenShift ecosystem
280+
281+
### Automatic Go Toolchain Management
282+
283+
The migration uses `GOTOOLCHAIN=auto` and `GOSUMDB=sum.golang.org` to automatically download required Go versions.
284+
285+
**What this solves:**
286+
287+
If dependencies require Go 1.24.6 but you have Go 1.24.3 installed:
288+
289+
```bash
290+
# Without GOTOOLCHAIN=auto (fails):
291+
go: go.mod requires go >= 1.24.6 (running go 1.24.3; GOTOOLCHAIN=local)
292+
293+
# With GOTOOLCHAIN=auto (succeeds):
294+
# Automatically downloads and uses go1.24.11
295+
```
296+
297+
**How it works:**
298+
- Your system Go: 1.24.3
299+
- Dependencies require: 1.24.6+
300+
- `GOTOOLCHAIN=auto` downloads: 1.24.11 (from go.mod's toolchain directive)
301+
- Build succeeds using the downloaded toolchain
302+
303+
**Used in these migration steps:**
304+
```bash
305+
# During go.mod creation
306+
GOTOOLCHAIN=auto GOSUMDB=sum.golang.org go mod tidy
307+
308+
# During dependency verification
309+
GOTOOLCHAIN=auto GOSUMDB=sum.golang.org go mod download
310+
```
311+
312+
**Manual usage after migration:**
313+
```bash
314+
# Set globally in your environment
315+
export GOTOOLCHAIN=auto
316+
export GOSUMDB=sum.golang.org
317+
318+
# Or per-command
319+
GOTOOLCHAIN=auto go mod tidy
320+
```
321+
322+
## Resources
323+
324+
- [OTE Framework Enhancement](https://github.com/openshift/enhancements/pull/1676)
325+
- [OTE Framework Repository](https://github.com/openshift-eng/openshift-tests-extension)
326+
- [Example Implementation](https://github.com/openshift-eng/openshift-tests-extension/blob/main/cmd/example-tests/main.go)

0 commit comments

Comments
 (0)