Skip to content

Commit 4a538a4

Browse files
DanielBleiDanielBlei
committed
chore(lab): experimental Jenkins CI lab (Helm + JCasC seed)
- Bootstraps local Jenkins on Kind/Minikube - Seeds pipelines via Job DSL. mirroring existent GitHub actions checks
1 parent 7c64c12 commit 4a538a4

12 files changed

+619
-1
lines changed

‎jenkins/README.md‎

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
2+
# Jenkins CI/CD Lab for Pipeline-Forge
3+
4+
This directory contains a local Jenkins setup for experimenting with CI/CD Jenkins pipelines for the Pipeline-Forge project.
5+
6+
## Overview
7+
8+
This lab environment provides a quick way to spin up Jenkins locally using Kubernetes (kind or minikube) and configure it with Jenkins Configuration as Code (JCasc).
9+
10+
The goal is to test and develop CI/CD pipelines for Pipeline-Forge components, evaluating Jenkins solutions and features.
11+
12+
## Quick Start
13+
14+
### Prerequisites
15+
16+
- `kubectl`
17+
- `helm`
18+
- `docker`
19+
- Either `kind` or `minikube`
20+
21+
### Installation
22+
23+
Run the bootstrap script to create a local Jenkins instance:
24+
25+
```bash
26+
# Kind by default
27+
./bootstrap_jenkins.sh
28+
29+
# or specify minikube
30+
./bootstrap_jenkins.sh minikube
31+
```
32+
33+
The script will:
34+
1. Create a local Kubernetes cluster
35+
2. Set up the `jenkins` namespace with required RBAC and storage
36+
3. Install Jenkins via Helm with JCasc configuration
37+
4. Wait for Jenkins to be ready
38+
39+
40+
After the installation completes:
41+
42+
1. **Port forward to Jenkins:**
43+
```bash
44+
kubectl port-forward svc/jenkins -n jenkins 8080:8080
45+
```
46+
47+
2. **Get the admin password:**
48+
```bash
49+
kubectl get secret -n jenkins jenkins -o jsonpath={.data.jenkins-admin-password} | base64 --decode
50+
```
51+
52+
## Bootstrapping Pipelines
53+
54+
**⚠️ Important:** After Jenkins is installed, you need to manually run the seed job to bootstrap all pipelines.
55+
56+
**Steps:**
57+
1. Navigate to Jenkins UI
58+
2. Locate and run the seed job (configured via JCasc)
59+
3. This will create all Pipeline-Forge CI/CD pipelines (work in progress)
60+
61+
62+
## Cleanup
63+
64+
Delete the local Jenkins cluster:
65+
```bash
66+
kind delete cluster --name jenkins
67+
# or
68+
minikube delete -p jenkins-control-plane
69+
```
70+
71+
## Lab Implementation Notes
72+
73+
Compared to the [upstream Jenkins Helm chart defaults](https://raw.githubusercontent.com/jenkinsci/helm-charts/main/charts/jenkins/values.yaml), this lab includes:
74+
75+
**Seed Job via JCasC**
76+
A seed pipeline job is defined via Jenkins Configuration as Code
77+
- The seed job pulls the Pipeline-Forge repository and bootstraps Jenkins CI pipelines from repository-managed definitions
78+
79+
**Two-Step Bootstrap (Intentional Design)**
80+
- Jenkins installation and CI/CD pipeline creation are deliberately separated
81+
- Keeps bootstrap logs clean and avoids hiding configuration issues
82+
- Allows you to debug seed job errors without re-running the entire cluster setup
83+
- Enables independent iteration on CI/CD pipeline definitions
84+
85+
**CI-Focused Plugin Set**
86+
- Additional plugins for multibranch pipelines, GitHub integration, Job DSL, Kubernetes agents, and JCasC
87+
88+
**Job DSL Script Security**
89+
- Script security for Job DSL is disabled via init script to allow seed job execution without needing manual approval
90+
- Can be manually re-enabled in Jenkins UI (Manage Jenkins → Configure System → Job DSL) after the initial seed job completes if required
91+
92+
**Local-First Defaults**
93+
- Jenkins URL: `http://localhost:8080`
94+
- Access via port-forwarding with no external service
95+
96+
**Configuration**
97+
98+
Override values are defined in `values.lab.yaml`.
99+
100+
**Note:** Plugin versions are intentionally unpinned for this development environment to always pull the latest versions.
101+
102+
### References
103+
104+
- [Installing Jenkins on Kubernetes](https://www.jenkins.io/doc/book/installing/kubernetes/#install-jenkins)
105+
- [Jenkins Helm Chart Default Values](https://raw.githubusercontent.com/jenkinsci/helm-charts/main/charts/jenkins/values.yaml)

‎jenkins/bootstrap_jenkins.sh‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
#!/usr/bin/env bash
2+
3+
set -euo pipefail
4+
5+
K8S_RUNTIME=${1:-kind}
6+
ROLLOUT_TIMEOUT=5m
7+
NAMESPACE=jenkins
8+
RELEASE_CHART=jenkins
9+
10+
function apply_k8s_resources() {
11+
echo "Creating Kubernetes resources (Service Account, RBAC, PV, PVC)..."
12+
kubectl create ns "$NAMESPACE"
13+
kubectl apply -n "$NAMESPACE" -f k8s/jenkins-serviceAccount-and-rbac.yaml
14+
kubectl apply -n "$NAMESPACE" -f k8s/jenkins-volume.yaml
15+
}
16+
17+
function helm_install() {
18+
echo "Installing Jenkins via Helm..."
19+
helm repo add jenkinsci https://charts.jenkins.io > /dev/null
20+
helm repo update >/dev/null
21+
helm install "$RELEASE_CHART" -n "$NAMESPACE" -f values.lab.yaml jenkinsci/jenkins
22+
}
23+
24+
function wait_rollout() {
25+
echo "Waiting for rollout to complete..."
26+
if ! kubectl rollout status statefulset/jenkins -n "$NAMESPACE" --timeout=$ROLLOUT_TIMEOUT; then
27+
echo "Rollout timed out, double check the pod/container logs and events for more details." >&2
28+
exit 1
29+
fi
30+
}
31+
32+
function main() {
33+
if [[ $K8S_RUNTIME == "kind" ]]; then
34+
kind create cluster --name jenkins # suffix -control-plane is added by kind
35+
elif [[ $K8S_RUNTIME == "minikube" ]]; then
36+
minikube start -p jenkins-control-plane
37+
else
38+
echo "Unknown runtime: $K8S_RUNTIME"
39+
exit 1
40+
fi
41+
42+
apply_k8s_resources
43+
helm_install
44+
wait_rollout
45+
46+
printf '\033[1;32m%s\033[0m\n\n' '✔ Jenkins is ready!'
47+
# shellcheck disable=SC1083
48+
echo -e "admin password: $(kubectl get secret -n "jenkins" jenkins -o jsonpath={.data.jenkins-admin-password} | base64 --decode)\n"
49+
echo "Run 'kubectl port-forward svc/jenkins -n \"$NAMESPACE\" 8080:8080' and access Jenkins UI on localhost:8080"
50+
}
51+
52+
main "$@"

‎jenkins/ci/ingest-test.Jenkinsfile‎

Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
pipeline {
2+
agent {
3+
kubernetes {
4+
label 'buildah-pr'
5+
defaultContainer 'tools'
6+
yaml """
7+
apiVersion: v1
8+
kind: Pod
9+
metadata:
10+
labels:
11+
app: jenkins-buildah
12+
spec:
13+
serviceAccountName: jenkins
14+
restartPolicy: Never
15+
containers:
16+
- name: tools
17+
image: quay.io/containers/podman:latest
18+
command:
19+
- cat
20+
tty: true
21+
env:
22+
- name: XDG_RUNTIME_DIR
23+
value: /home/jenkins/.xdg
24+
- name: TMPDIR
25+
value: /home/jenkins/.tmp
26+
- name: BUILDAH_ISOLATION
27+
value: chroot
28+
"""
29+
}
30+
}
31+
32+
options {
33+
timeout(time: 30, unit: 'MINUTES')
34+
timestamps()
35+
disableConcurrentBuilds()
36+
buildDiscarder(logRotator(numToKeepStr: '10'))
37+
}
38+
39+
environment {
40+
IMAGE_TAG = "quay.io/danielblei/pipeline-forge/ingest:ci-${env.GIT_COMMIT}"
41+
WORKDIR = 'workloads/ingest'
42+
}
43+
44+
stages {
45+
stage('Checkout') {
46+
steps {
47+
checkout scmGit(
48+
branches: scm.branches,
49+
extensions: [cloneOption(shallow: true, depth: 1)],
50+
userRemoteConfigs: scm.userRemoteConfigs
51+
)
52+
}
53+
}
54+
55+
stage('Sanity: tools present') {
56+
steps {
57+
container('tools') {
58+
sh '''
59+
set -euxo pipefail
60+
podman --version
61+
buildah --version
62+
'''
63+
}
64+
}
65+
}
66+
67+
stage('Setup Python Environment') {
68+
steps {
69+
dir("${env.WORKDIR}") {
70+
sh '''
71+
curl -LsSf https://astral.sh/uv/install.sh | sh
72+
export PATH="$HOME/.local/bin:$PATH"
73+
ls -la pyproject.toml
74+
uv venv
75+
uv sync --frozen
76+
uv pip install -e . --no-deps
77+
'''
78+
}
79+
}
80+
}
81+
82+
stage('Quality Checks') {
83+
parallel {
84+
stage('Ruff Linter') {
85+
steps {
86+
dir("${env.WORKDIR}") {
87+
sh '''
88+
export PATH="$HOME/.local/bin:$PATH"
89+
uv run ruff check .
90+
'''
91+
}
92+
}
93+
}
94+
stage('Ruff Format Check') {
95+
steps {
96+
dir("${env.WORKDIR}") {
97+
sh '''
98+
export PATH="$HOME/.local/bin:$PATH"
99+
uv run ruff format --check .
100+
'''
101+
}
102+
}
103+
}
104+
stage('Type Check (mypy)') {
105+
steps {
106+
dir("${env.WORKDIR}") {
107+
sh '''
108+
export PATH="$HOME/.local/bin:$PATH"
109+
uv run mypy .
110+
'''
111+
}
112+
}
113+
}
114+
}
115+
}
116+
117+
stage('Run Tests') {
118+
steps {
119+
dir("${env.WORKDIR}") {
120+
catchError(buildResult: 'SUCCESS', stageResult: 'UNSTABLE') {
121+
sh '''
122+
export PATH="$HOME/.local/bin:$PATH"
123+
uv run pytest . -v --junitxml=test-results.xml
124+
'''
125+
}
126+
}
127+
}
128+
post {
129+
always {
130+
junit allowEmptyResults: true, testResults: "${env.WORKDIR}/test-results.xml"
131+
archiveArtifacts artifacts: "${env.WORKDIR}/test-results.xml", allowEmptyArchive: true
132+
}
133+
}
134+
}
135+
136+
stage('Build OCI Image (Buildah)') {
137+
steps {
138+
dir("${env.WORKDIR}") {
139+
container('tools') {
140+
sh '''
141+
set -euxo pipefail
142+
mkdir -p "$XDG_RUNTIME_DIR" "$TMPDIR"
143+
buildah bud --layers -t "${IMAGE_TAG}" .
144+
'''
145+
}
146+
}
147+
}
148+
}
149+
150+
stage('Smoke Test (Podman)') {
151+
steps {
152+
container('tools') {
153+
sh '''
154+
set -euxo pipefail
155+
mkdir -p "$XDG_RUNTIME_DIR" "$TMPDIR"
156+
podman run --rm "${IMAGE_TAG}" ingest --help
157+
'''
158+
}
159+
}
160+
}
161+
}
162+
163+
post {
164+
cleanup {
165+
container('tools') {
166+
sh '''
167+
podman rmi -f "${IMAGE_TAG}" 2>/dev/null || true
168+
buildah rmi -f "${IMAGE_TAG}" 2>/dev/null || true
169+
'''
170+
}
171+
}
172+
}
173+
}

‎jenkins/ci/operator-e2e-test.Jenkinsfile‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
pipeline {
2+
agent any
3+
stages {
4+
stage('Placeholder') {
5+
steps {
6+
echo 'Work in progress — please check ingest-test'
7+
}
8+
}
9+
}
10+
}

‎jenkins/ci/operator-test.Jenkinsfile‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
pipeline {
2+
agent any
3+
stages {
4+
stage('Placeholder') {
5+
steps {
6+
echo 'Work in progress — please check ingest-test'
7+
}
8+
}
9+
}
10+
}

‎jenkins/ci/trigger-test.Jenkinsfile‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
pipeline {
2+
agent any
3+
stages {
4+
stage('Placeholder') {
5+
steps {
6+
echo 'Work in progress — please check ingest-test'
7+
}
8+
}
9+
}
10+
}

‎jenkins/job-dsl/seed.Jenkinsfile‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
pipeline {
2+
agent any
3+
4+
options {
5+
timeout(time: 10, unit: 'MINUTES')
6+
}
7+
8+
stages {
9+
stage('Checkout') {
10+
steps {
11+
checkout scmGit(
12+
branches: scm.branches,
13+
extensions: [cloneOption(shallow: true, depth: 1, noTags: true)],
14+
userRemoteConfigs: scm.userRemoteConfigs
15+
)
16+
}
17+
}
18+
19+
stage('Generate jobs') {
20+
steps {
21+
jobDsl targets: 'jenkins/job-dsl/*.groovy',
22+
removedJobAction: 'DELETE',
23+
removedViewAction: 'DELETE',
24+
lookupStrategy: 'JENKINS_ROOT'
25+
}
26+
}
27+
}
28+
}

0 commit comments

Comments
 (0)