Skip to content

Commit 0c0b122

Browse files
chmouelchmouel
committed
feat: Add OpenAPI schema validation for CRD
* Defined OpenAPI v3 schema for the Repository Custom Resource Definition (CRD). This enhanced resource validation, improved client-side tooling like IDE autocompletion, and provided clearer API documentation. * Automated schema generation via a new `make update-schemas` target using `controller-gen`. This ensured schema accuracy and simplified maintenance. * Added a CI job to verify that generated OpenAPI schemas were current. This prevented stale schemas from being merged. * Documented the usage and benefits of the OpenAPI schema for users. This guided users on leveraging schema validation and tooling. * Enhanced `kubectl get repo` output with additional status columns for better observability. JIRA: https://issues.redhat.com/browse/SRVKP-7605 Signed-off-by: Chmouel Boudjnah <chmouel@redhat.com>
1 parent d81d616 commit 0c0b122

File tree

7 files changed

+803
-158
lines changed

7 files changed

+803
-158
lines changed

‎.tekton/linter.yaml‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,29 @@ spec:
9090
9191
exit $failed
9292
93+
- name: check-generated-schemas
94+
displayName: "Check generated OpenAPI schemas"
95+
image: golang:1.23
96+
workingDir: $(workspaces.source.path)
97+
env:
98+
- name: HUB_TOKEN
99+
valueFrom:
100+
secretKeyRef:
101+
name: "nightly-ci-github-hub-token"
102+
key: "hub-token"
103+
script: |
104+
set -eu
105+
git config --global --add safe.directory $(workspaces.source.path)
106+
version=$(curl -H "Authorization: Bearer ${HUB_TOKEN}" -L -s https://api.github.com/repos/mikefarah/yq/releases/latest|python3 -c 'import sys, json;dico=json.load(sys.stdin);print(dico["tag_name"])')
107+
arch=amd64
108+
if uname -m | grep -q aarch64; then
109+
arch=arm64
110+
fi
111+
curl -sH "Authorization: Bearer ${HUB_TOKEN}" -L https://github.com/mikefarah/yq/releases/download/${version}/yq_linux_${arch} -o /usr/bin/yq && chmod +x /usr/bin/yq
112+
make update-schemas
113+
# check now that the is no changes in config with git
114+
git diff --exit-code config/ || { echo "Error: you need to run 'make update-schemas' and commit it."; exit 1; }
115+
93116
- name: conventional-commits
94117
displayName: "conventional commit check"
95118
image: registry.access.redhat.com/ubi9/python-312

‎Makefile‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -181,6 +181,11 @@ update-golden: ## run unit tests (updating golden files)
181181
@echo "Running unit tests to update golden files..."
182182
@./hack/update-golden.sh
183183

184+
.PHONY: update-schemas
185+
update-schemas: ## update openapi schemas
186+
@echo "Running update schemas..."
187+
@./hack/update-schemas.sh
188+
184189
.PHONY: generated
185190
generated: update-golden fumpt ## generate all files that needs to be generated
186191

‎config/300-repositories.yaml‎

Lines changed: 319 additions & 127 deletions
Large diffs are not rendered by default.

‎docs/content/docs/guide/openapi-schema.md‎

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
---
2+
title: OpenAPI Schema Validation
3+
weight: 10
4+
---
5+
6+
# OpenAPI Schema Validation for Repository CRDs
7+
8+
Pipelines-as-Code provides [OpenAPI](https://www.openapis.org/) schema validation for its Custom Resource
9+
Definitions (CRDs), which helps in writing the Repository resources. This page
10+
explains what OpenAPI schemas are, their benefits, and how to leverage them in
11+
your development environment.
12+
13+
## What is OpenAPI Schema Validation?
14+
15+
OpenAPI schema validation is a mechanism that adds metadata to Kubernetes CRDs, providing:
16+
17+
- **Type information**: Specifies expected data types for fields
18+
- **Required fields**: Marks which fields must be present
19+
- **Pattern validation**: Enforces specific formats (e.g., URLs must start with http:// or https://)
20+
- **Enumeration validation**: Limits fields to a predefined set of values
21+
- **Field descriptions**: Documents the purpose of each field
22+
23+
When you create or modify a Repository resource, the OpenAPI schema helps
24+
validate your configuration before it's applied to the cluster, catching errors
25+
early in the development process.
26+
27+
## Benefits of OpenAPI Schema Validation
28+
29+
Using OpenAPI schemas with Pipelines-as-Code provides numerous advantages:
30+
31+
1. **Early Error Detection**: Identifies configuration errors before applying resources to your cluster
32+
2. **Improved Documentation**: Field descriptions provide built-in documentation
33+
3. **Better IDE Support**: Enables rich autocompletion and validation in code editors
34+
4. **Standardized Formatting**: Ensures your resources follow expected formats
35+
5. **Self-Documenting API**: Makes it easier to understand the Repository CRD structure
36+
37+
## Using OpenAPI Schemas in VS Code
38+
39+
Visual Studio Code and other modern editors can leverage OpenAPI schemas to
40+
provide real-time validation, autocompletion, and documentation while editing
41+
your Repository resources.
42+
43+
### Setting Up VS Code for CRD Validation
44+
45+
1. Install the [Kubernetes](https://marketplace.visualstudio.com/items?itemName=ms-kubernetes-tools.vscode-kubernetes-tools) extension (by Microsoft)
46+
2. Install the [YAML](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml) extension (by Red Hat)
47+
48+
These extensions will automatically detect and use the OpenAPI schema from your Kubernetes cluster when editing Repository resources.
49+
50+
### Example: Editing a Repository Resource in VS Code
51+
52+
When editing a Repository resource in VS Code with the appropriate extensions, you'll experience:
53+
54+
- **Autocompletion** for fields like `url`, `git_provider`, `settings`, etc.
55+
- **Validation** for required fields and field formats
56+
- **Documentation tooltips** when hovering over fields
57+
- **Enum dropdown suggestions** for fields with predefined values
58+
59+
Here's an example of what you'll see:
60+
61+
```yaml
62+
apiVersion: pipelinesascode.tekton.dev/v1alpha1
63+
kind: Repository
64+
metadata:
65+
name: my-repository
66+
spec:
67+
# VS Code will show an error if this required field is missing
68+
url: https://github.com/myorg/myrepo
69+
70+
# Autocompletion will suggest all available fields
71+
git_provider:
72+
# Dropdown will show available provider types
73+
type: github
74+
75+
# Pattern validation ensures the URL format is correct
76+
url: https://api.github.com
77+
78+
# Documentation tooltips show field descriptions
79+
webhook_secret:
80+
name: webhook-secret
81+
key: token
82+
83+
# Validation for minimum values
84+
concurrency_limit: 5
85+
86+
settings:
87+
# Enum validation limits to specific options
88+
pipelinerun_provenance: default_branch
89+
```
90+
91+
## Other Tools Using OpenAPI Schemas
92+
93+
Beyond VS Code, OpenAPI schemas are useful with:
94+
95+
1. **kubectl**: Validates resources before applying them
96+
97+
```bash
98+
kubectl create -f my-repository.yaml --validate=true
99+
```
100+
101+
2. **kube-linter**: Validates resources as part of CI/CD pipelines
102+
103+
3. **Kubernetes Dashboard**: Shows field descriptions and validations
104+
105+
4. **OpenAPI documentation generators**: Create API documentation from schemas
106+
107+
## How to Get the OpenAPI Schema
108+
109+
The OpenAPI schema for Repository resources is embedded in the CRD itself. You can view it with:
110+
111+
```bash
112+
kubectl get crd repositories.pipelinesascode.tekton.dev -o jsonpath='{.spec.versions[0].schema.openAPIV3Schema}' | jq
113+
```
114+
115+
Or to see all available fields with their descriptions:
116+
117+
```bash
118+
kubectl explain repository.spec --recursive
119+
```
120+
121+
## Troubleshooting Schema Validation Errors
122+
123+
If you encounter validation errors when creating or updating a Repository:
124+
125+
1. **Check field types**: Ensure values match expected types
126+
2. **Verify required fields**: Make sure all required fields are present
127+
3. **Review pattern validations**: URLs must follow the correct format
128+
4. **Check enum values**: Some fields accept only specific values
129+
130+
Error messages typically provide specific details about which validation failed, making it easier to correct your resources.
131+
132+
{{< hint info >}}
133+
Even with client-side validation, the webhook validation in Pipelines-as-Code provides an additional layer of validation for complex logic that cannot be expressed in OpenAPI schemas.
134+
{{< /hint >}}

‎docs/content/docs/install/gitlab.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,6 +119,7 @@ $ tkn pac create repo
119119
url: "https://gitlab.com/group/project"
120120
git_provider:
121121
# url: "https://gitlab.example.com/ # Set this if you are using a private GitLab instance
122+
type: "gitlab"
122123
secret:
123124
name: "gitlab-webhook-config"
124125
# Set this if you have a different key in your secret

‎hack/update-schemas.sh‎

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
#!/usr/bin/env bash
2+
# Copyright 2025 The Tekton Authors
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
#
16+
# Imported from tektoncd/pipeline/hack/update-schemas.sh and vibed/adapted for
17+
# PAAC.
18+
19+
set -o errexit
20+
set -o nounset
21+
set -o pipefail
22+
23+
OLDGOFLAGS="${GOFLAGS:-}"
24+
GOFLAGS=""
25+
26+
SCRIPT_DIR=$(dirname "${BASH_SOURCE[0]}")
27+
CRD_PATH="${SCRIPT_DIR}/../config"
28+
API_PATH="${SCRIPT_DIR}/../pkg/apis"
29+
TEMP_DIR_LOGS=$(mktemp -d)
30+
31+
echo "=== Generating CRD schemas with OpenAPI validation ==="
32+
33+
# List of CRDs to process
34+
CRD_FILES=(
35+
"${CRD_PATH}/300-repositories.yaml"
36+
# Add more CRD files here as they are created
37+
)
38+
39+
for FILENAME in "${CRD_FILES[@]}"; do
40+
BASENAME=$(basename "$FILENAME")
41+
echo "Generating OpenAPI schema for $FILENAME"
42+
43+
# Extract group from the CRD file
44+
GROUP=$(grep -E '^ group:' $FILENAME)
45+
GROUP=${GROUP#" group: "}
46+
API_SUBDIR=${GROUP%".tekton.dev"}
47+
48+
TEMP_DIR=$(mktemp -d)
49+
cp -p $FILENAME $TEMP_DIR/.
50+
LOG_FILE=$TEMP_DIR_LOGS/log-schema-generation-$(basename $FILENAME)
51+
52+
echo " Processing API group: $GROUP, subdir: $API_SUBDIR"
53+
54+
counter=0 limit=5
55+
while [ "$counter" -lt "$limit" ]; do
56+
set +e
57+
# Use controller-gen to generate the schema
58+
go run sigs.k8s.io/controller-tools/cmd/controller-gen@v0.17.1 \
59+
crd:crdVersions=v1 \
60+
output:crd:artifacts:config=$TEMP_DIR \
61+
paths=$API_PATH/$API_SUBDIR/... >$LOG_FILE 2>&1
62+
rc=$?
63+
set -e
64+
65+
if [ $rc -eq 0 ]; then
66+
echo " ✅ Successfully generated schema"
67+
68+
# Find the auto-generated CRD file
69+
if command -v yq >/dev/null 2>&1 && yq --version | grep -q "mikefarah/yq"; then
70+
echo " 🔄 Syncing schema from temporary CRD to $BASENAME"
71+
72+
# Find the auto-generated CRD file in the temp directory
73+
AUTO_GENERATED_CRD=$(find $TEMP_DIR -name "${GROUP}_*.yaml")
74+
75+
if [ -f "$AUTO_GENERATED_CRD" ]; then
76+
# Extract schema from auto-generated CRD and apply it to the original CRD
77+
yq eval '.spec.versions[0].schema' "$AUTO_GENERATED_CRD" >/tmp/schema.yaml
78+
yq eval -i '.spec.versions[0].schema = load("/tmp/schema.yaml")' "$FILENAME"
79+
echo " ✅ Schema successfully synced to $BASENAME"
80+
81+
# Clean up temporary schema file
82+
rm -f /tmp/schema.yaml
83+
else
84+
echo " ⚠️ Warning: Auto-generated CRD not found in temporary directory"
85+
fi
86+
else
87+
echo " ⚠️ Warning: mikefarah/yq not available, cannot automatically sync schema"
88+
echo " Manual action required: Copy schema from auto-generated CRD to $BASENAME"
89+
fi
90+
91+
# Now generate directly to the config directory for the final step
92+
# but we'll delete the auto-generated file afterward
93+
go run sigs.k8s.io/controller-tools/cmd/controller-gen@v0.17.1 \
94+
crd:crdVersions=v1 \
95+
output:crd:artifacts:config=$CRD_PATH \
96+
paths=$API_PATH/$API_SUBDIR/... >/dev/null 2>&1
97+
98+
# Find and delete the auto-generated CRD file from the config directory
99+
AUTO_GENERATED_CRD=$(find $CRD_PATH -name "${GROUP}_*.yaml")
100+
if [ -f "$AUTO_GENERATED_CRD" ]; then
101+
echo " 🗑️ Removing auto-generated CRD file: $(basename "$AUTO_GENERATED_CRD")"
102+
rm -f "$AUTO_GENERATED_CRD"
103+
fi
104+
105+
break
106+
fi
107+
108+
# Check if we can proceed despite errors
109+
if grep -q 'exit status 1' $LOG_FILE; then
110+
echo " ⚠️ Warning: Encountered errors during schema generation"
111+
echo " 📝 Check $LOG_FILE for details"
112+
# Attempt to extract and display useful error information
113+
ERROR_MSG=$(grep -A 3 'Error:' $LOG_FILE 2>/dev/null || echo "Unknown error")
114+
echo " Error summary: $ERROR_MSG"
115+
break
116+
fi
117+
118+
counter=$((counter + 1))
119+
if [ $counter -eq $limit ]; then
120+
echo " ❌ Failed to generate CRD schema after $limit attempts"
121+
echo " 📝 Check $LOG_FILE for details"
122+
cat ${LOG_FILE}
123+
exit 1
124+
fi
125+
126+
echo " Retrying (attempt $counter of $limit)..."
127+
sleep 1
128+
done
129+
130+
# Validate the generated CRD
131+
echo " Validating generated CRD"
132+
if command -v kubectl >/dev/null 2>&1; then
133+
# Try using the newer "kubectl server-side validation" approach
134+
if kubectl explain crd &>/dev/null; then
135+
if kubectl apply --server-side --dry-run=server --validate=strict -f "$FILENAME" >/dev/null 2>&1; then
136+
echo " ✅ CRD validation successful using server-side validation"
137+
else
138+
echo " ⚠️ Warning: Server-side CRD validation failed, validation may not be complete"
139+
fi
140+
else
141+
echo " ⚠️ Warning: kubectl API resources not available, skipping validation"
142+
fi
143+
else
144+
echo " ⚠️ Warning: kubectl not found, skipping validation"
145+
fi
146+
147+
rm -rf $TEMP_DIR
148+
done
149+
150+
echo "=== Schema generation complete ==="
151+
echo "Generated schemas saved to: $CRD_PATH"
152+
echo "OpenAPI schema has been successfully added to the numbered CRD files"
153+
echo "Auto-generated reference CRD files have been removed as requested"
154+
echo "Log files available at: $TEMP_DIR_LOGS"
155+
156+
GOFLAGS="${OLDGOFLAGS}"

0 commit comments

Comments
 (0)