Skip to content

Commit 55cca8d

Browse files
DanielBleiDanielBlei
committed
chore(lab): experimental Jenkins CI lab (Helm + JCasC seed)
- Bootstraps local Jenkins on Kind/Minikube - Seeds pipelines via Job DSL. mirroring existent GitHub actions checks
1 parent 7c64c12 commit 55cca8d

12 files changed

+605
-1
lines changed

‎jenkins/README.md‎

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
2+
# Jenkins CI/CD Lab for Pipeline-Forge
3+
4+
This directory contains a local Jenkins setup for experimenting with CI/CD Jenkins pipelines for the Pipeline-Forge project.
5+
6+
## Overview
7+
8+
This lab environment provides a quick way to spin up Jenkins locally using Kubernetes (kind or minikube) and configure it with Jenkins Configuration as Code (JCasc).
9+
10+
The goal is to test and develop CI/CD pipelines for Pipeline-Forge components, evaluating Jenkins solutions and features.
11+
12+
## Quick Start
13+
14+
### Prerequisites
15+
16+
- `kubectl`
17+
- `helm`
18+
- `docker`
19+
- Either `kind` or `minikube`
20+
21+
### Installation
22+
23+
Run the bootstrap script to create a local Jenkins instance:
24+
25+
```bash
26+
# Kind by default
27+
./bootstrap_jenkins.sh
28+
29+
# or specify minikube
30+
./bootstrap_jenkins.sh minikube
31+
```
32+
33+
The script will:
34+
1. Create a local Kubernetes cluster
35+
2. Set up the `jenkins` namespace with required RBAC and storage
36+
3. Install Jenkins via Helm with JCasc configuration
37+
4. Wait for Jenkins to be ready
38+
39+
40+
After the installation completes:
41+
42+
1. **Port forward to Jenkins:**
43+
```bash
44+
kubectl port-forward svc/jenkins -n jenkins 8080:8080
45+
```
46+
47+
2. **Get the admin password:**
48+
```bash
49+
kubectl get secret -n jenkins jenkins -o jsonpath={.data.jenkins-admin-password} | base64 --decode
50+
```
51+
52+
## Bootstrapping Pipelines
53+
54+
**⚠️ Important:** After Jenkins is installed, you need to manually run the seed job to bootstrap all pipelines.
55+
56+
**Steps:**
57+
1. Navigate to Jenkins UI
58+
2. Locate and run the seed job (configured via JCasc)
59+
3. This will create all Pipeline-Forge CI/CD pipelines (work in progress)
60+
61+
62+
## Cleanup
63+
64+
Delete the local Jenkins cluster:
65+
```bash
66+
kind delete cluster --name jenkins
67+
# or
68+
minikube delete -p jenkins-control-plane
69+
```
70+
71+
## Lab Implementation Notes
72+
73+
Compared to the [upstream Jenkins Helm chart defaults](https://raw.githubusercontent.com/jenkinsci/helm-charts/main/charts/jenkins/values.yaml), this lab includes:
74+
75+
**Seed Job via JCasC**
76+
A seed pipeline job is defined via Jenkins Configuration as Code
77+
- The seed job pulls the Pipeline-Forge repository and bootstraps Jenkins CI pipelines from repository-managed definitions
78+
79+
**Two-Step Bootstrap (Intentional Design)**
80+
- Jenkins installation and CI/CD pipeline creation are deliberately separated
81+
- Keeps bootstrap logs clean and avoids hiding configuration issues
82+
- Allows you to debug seed job errors without re-running the entire cluster setup
83+
- Enables independent iteration on CI/CD pipeline definitions
84+
85+
**CI-Focused Plugin Set**
86+
- Additional plugins for multibranch pipelines, GitHub integration, Job DSL, Kubernetes agents, and JCasC
87+
88+
**Job DSL Script Security**
89+
- Script security for Job DSL is disabled via init script to allow seed job execution without needing manual approval
90+
- Can be manually re-enabled in Jenkins UI (Manage Jenkins → Configure System → Job DSL) after the initial seed job completes if required
91+
92+
**Local-First Defaults**
93+
- Jenkins URL: `http://localhost:8080`
94+
- Access via port-forwarding with no external service
95+
96+
**Configuration**
97+
98+
Override values are defined in `values.lab.yaml`.
99+
100+
**Note:** Plugin versions are intentionally unpinned for this development environment to always pull the latest versions.
101+
102+
### References
103+
104+
- [Installing Jenkins on Kubernetes](https://www.jenkins.io/doc/book/installing/kubernetes/#install-jenkins)
105+
- [Jenkins Helm Chart Default Values](https://raw.githubusercontent.com/jenkinsci/helm-charts/main/charts/jenkins/values.yaml)

‎jenkins/bootstrap_jenkins.sh‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
#!/usr/bin/env bash
2+
3+
set -euo pipefail
4+
5+
K8S_RUNTIME=${1:-kind}
6+
ROLLOUT_TIMEOUT=5m
7+
NAMESPACE=jenkins
8+
RELEASE_CHART=jenkins
9+
10+
function apply_k8s_resources() {
11+
echo "Creating Kubernetes resources (Service Account, RBAC, PV, PVC)..."
12+
kubectl create ns "$NAMESPACE"
13+
kubectl apply -n "$NAMESPACE" -f k8s/jenkins-serviceAccount-and-rbac.yaml
14+
kubectl apply -n "$NAMESPACE" -f k8s/jenkins-volume.yaml
15+
}
16+
17+
function helm_install() {
18+
echo "Installing Jenkins via Helm..."
19+
helm repo add jenkinsci https://charts.jenkins.io > /dev/null
20+
helm repo update >/dev/null
21+
helm install "$RELEASE_CHART" -n "$NAMESPACE" -f values.lab.yaml jenkinsci/jenkins
22+
}
23+
24+
function wait_rollout() {
25+
echo "Waiting for rollout to complete..."
26+
if ! kubectl rollout status statefulset/jenkins -n "$NAMESPACE" --timeout=$ROLLOUT_TIMEOUT; then
27+
echo "Rollout timed out, double check the pod/container logs and events for more details." >&2
28+
exit 1
29+
fi
30+
}
31+
32+
function main() {
33+
if [[ $K8S_RUNTIME == "kind" ]]; then
34+
kind create cluster --name jenkins # suffix -control-plane is added by kind
35+
elif [[ $K8S_RUNTIME == "minikube" ]]; then
36+
minikube start -p jenkins-control-plane
37+
else
38+
echo "Unknown runtime: $K8S_RUNTIME"
39+
exit 1
40+
fi
41+
42+
apply_k8s_resources
43+
helm_install
44+
wait_rollout
45+
46+
printf '\033[1;32m%s\033[0m\n\n' '✔ Jenkins is ready!'
47+
# shellcheck disable=SC1083
48+
echo -e "admin password: $(kubectl get secret -n "jenkins" jenkins -o jsonpath={.data.jenkins-admin-password} | base64 --decode)\n"
49+
echo "Run 'kubectl port-forward svc/jenkins -n \"$NAMESPACE\" 8080:8080' and access Jenkins UI on localhost:8080"
50+
}
51+
52+
main "$@"

‎jenkins/ci/ingest-test.Jenkinsfile‎

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
pipeline {
2+
agent {
3+
kubernetes {
4+
label 'podman-agent'
5+
defaultContainer 'tools'
6+
yaml """
7+
apiVersion: v1
8+
kind: Pod
9+
metadata:
10+
labels:
11+
app: jenkins-podman
12+
spec:
13+
serviceAccountName: jenkins
14+
restartPolicy: Never
15+
containers:
16+
- name: tools
17+
image: quay.io/containers/podman:latest
18+
command: ["cat"]
19+
tty: true
20+
env:
21+
- name: XDG_RUNTIME_DIR
22+
value: /home/jenkins/.xdg
23+
- name: TMPDIR
24+
value: /home/jenkins/.tmp
25+
"""
26+
}
27+
}
28+
29+
options {
30+
timeout(time: 30, unit: 'MINUTES')
31+
timestamps()
32+
disableConcurrentBuilds()
33+
buildDiscarder(logRotator(numToKeepStr: '10'))
34+
}
35+
36+
environment {
37+
IMAGE_TAG = "ingest:ci-${env.GIT_COMMIT}"
38+
WORKDIR = 'workloads/ingest'
39+
}
40+
41+
stages {
42+
stage('Checkout') {
43+
steps {
44+
checkout scmGit(
45+
branches: scm.branches,
46+
extensions: [cloneOption(shallow: true, depth: 1)],
47+
userRemoteConfigs: scm.userRemoteConfigs
48+
)
49+
}
50+
}
51+
52+
stage('Setup Python Environment') {
53+
steps {
54+
dir("${env.WORKDIR}") {
55+
sh '''
56+
curl -LsSf https://astral.sh/uv/install.sh | sh
57+
export PATH="$HOME/.local/bin:$PATH"
58+
ls -la pyproject.toml
59+
uv venv
60+
uv sync --frozen
61+
uv pip install -e . --no-deps
62+
'''
63+
}
64+
}
65+
}
66+
67+
stage('Quality Checks') {
68+
parallel {
69+
stage('Ruff Linter') {
70+
steps {
71+
dir("${env.WORKDIR}") {
72+
sh '''
73+
export PATH="$HOME/.local/bin:$PATH"
74+
uv run ruff check .
75+
'''
76+
}
77+
}
78+
}
79+
80+
stage('Ruff Format Check') {
81+
steps {
82+
dir("${env.WORKDIR}") {
83+
sh '''
84+
export PATH="$HOME/.local/bin:$PATH"
85+
uv run ruff format --check .
86+
'''
87+
}
88+
}
89+
}
90+
91+
stage('Type Check (mypy)') {
92+
steps {
93+
dir("${env.WORKDIR}") {
94+
sh '''
95+
export PATH="$HOME/.local/bin:$PATH"
96+
uv run mypy .
97+
'''
98+
}
99+
}
100+
}
101+
}
102+
}
103+
104+
stage('Run Tests') {
105+
steps {
106+
dir("${env.WORKDIR}") {
107+
catchError(buildResult: 'SUCCESS', stageResult: 'UNSTABLE') {
108+
sh '''
109+
export PATH="$HOME/.local/bin:$PATH"
110+
uv run pytest . -v --junitxml=test-results.xml
111+
'''
112+
}
113+
}
114+
}
115+
post {
116+
always {
117+
junit allowEmptyResults: true, testResults: "${env.WORKDIR}/test-results.xml"
118+
archiveArtifacts artifacts: "${env.WORKDIR}/test-results.xml", allowEmptyArchive: true
119+
}
120+
}
121+
}
122+
123+
stage('Build Image (Podman)') {
124+
steps {
125+
dir("${env.WORKDIR}") {
126+
container('tools') {
127+
sh '''
128+
set -euxo pipefail
129+
mkdir -p "$XDG_RUNTIME_DIR" "$TMPDIR"
130+
podman build -t "${IMAGE_TAG}" .
131+
'''
132+
}
133+
}
134+
}
135+
}
136+
137+
stage('Smoke Test (Podman)') {
138+
steps {
139+
container('tools') {
140+
sh '''
141+
set -euxo pipefail
142+
mkdir -p "$XDG_RUNTIME_DIR" "$TMPDIR"
143+
podman run --rm "${IMAGE_TAG}" ingest --help
144+
'''
145+
}
146+
}
147+
}
148+
}
149+
150+
post {
151+
cleanup {
152+
container('tools') {
153+
sh '''
154+
podman rmi -f "${IMAGE_TAG}" 2>/dev/null || true
155+
'''
156+
}
157+
}
158+
}
159+
}

‎jenkins/ci/operator-e2e-test.Jenkinsfile‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
pipeline {
2+
agent any
3+
stages {
4+
stage('Placeholder') {
5+
steps {
6+
echo 'Work in progress — please check ingest-test'
7+
}
8+
}
9+
}
10+
}

‎jenkins/ci/operator-test.Jenkinsfile‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
pipeline {
2+
agent any
3+
stages {
4+
stage('Placeholder') {
5+
steps {
6+
echo 'Work in progress — please check ingest-test'
7+
}
8+
}
9+
}
10+
}

‎jenkins/ci/trigger-test.Jenkinsfile‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
pipeline {
2+
agent any
3+
stages {
4+
stage('Placeholder') {
5+
steps {
6+
echo 'Work in progress — please check ingest-test'
7+
}
8+
}
9+
}
10+
}

‎jenkins/job-dsl/seed.Jenkinsfile‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
pipeline {
2+
agent any
3+
4+
options {
5+
timeout(time: 10, unit: 'MINUTES')
6+
}
7+
8+
stages {
9+
stage('Checkout') {
10+
steps {
11+
checkout scmGit(
12+
branches: scm.branches,
13+
extensions: [cloneOption(shallow: true, depth: 1, noTags: true)],
14+
userRemoteConfigs: scm.userRemoteConfigs
15+
)
16+
}
17+
}
18+
19+
stage('Generate jobs') {
20+
steps {
21+
jobDsl targets: 'jenkins/job-dsl/*.groovy',
22+
removedJobAction: 'DELETE',
23+
removedViewAction: 'DELETE',
24+
lookupStrategy: 'JENKINS_ROOT'
25+
}
26+
}
27+
}
28+
}

0 commit comments

Comments
 (0)