Skip to content

Commit 3d568db

Browse files
DanielBleiDanielBlei
committed
chore(lab): experimental Jenkins CI lab (Helm + JCasC seed)
- Bootstraps local Jenkins on Kind/Minikube - Seeds pipelines via Job DSL. mirroring existent GitHub actions checks
1 parent 7c64c12 commit 3d568db

12 files changed

+597
-1
lines changed

‎jenkins/README.md‎

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
2+
# Jenkins CI/CD Lab for Pipeline-Forge
3+
4+
This directory contains a local Jenkins setup for experimenting with CI/CD Jenkins pipelines for the Pipeline-Forge project.
5+
6+
## Overview
7+
8+
This lab environment provides a quick way to spin up Jenkins locally using Kubernetes (kind or minikube) and configure it with Jenkins Configuration as Code (JCasc).
9+
10+
The goal is to test and develop CI/CD pipelines for Pipeline-Forge components, evaluating Jenkins solutions and features.
11+
12+
## Quick Start
13+
14+
### Prerequisites
15+
16+
- `kubectl`
17+
- `helm`
18+
- `docker`
19+
- Either `kind` or `minikube`
20+
21+
### Installation
22+
23+
Run the bootstrap script to create a local Jenkins instance:
24+
25+
```bash
26+
# Kind by default
27+
./bootstrap_jenkins.sh
28+
29+
# or specify minikube
30+
./bootstrap_jenkins.sh minikube
31+
```
32+
33+
The script will:
34+
1. Create a local Kubernetes cluster
35+
2. Set up the `jenkins` namespace with required RBAC and storage
36+
3. Install Jenkins via Helm with JCasc configuration
37+
4. Wait for Jenkins to be ready
38+
39+
40+
After the installation completes:
41+
42+
1. **Port forward to Jenkins:**
43+
```bash
44+
kubectl port-forward svc/jenkins -n jenkins 8080:8080
45+
```
46+
47+
2. **Get the admin password:**
48+
```bash
49+
kubectl get secret -n jenkins jenkins -o jsonpath={.data.jenkins-admin-password} | base64 --decode
50+
```
51+
52+
## Bootstrapping Pipelines
53+
54+
**⚠️ Important:** After Jenkins is installed, you need to manually run the seed job to bootstrap all pipelines.
55+
56+
**Steps:**
57+
1. Navigate to Jenkins UI
58+
2. Locate and run the seed job (configured via JCasc)
59+
3. This will create all Pipeline-Forge CI/CD pipelines (work in progress)
60+
61+
62+
## Cleanup
63+
64+
Delete the local Jenkins cluster:
65+
```bash
66+
kind delete cluster --name jenkins
67+
# or
68+
minikube delete -p jenkins-control-plane
69+
```
70+
71+
## Lab Implementation Notes
72+
73+
Compared to the [upstream Jenkins Helm chart defaults](https://raw.githubusercontent.com/jenkinsci/helm-charts/main/charts/jenkins/values.yaml), this lab includes:
74+
75+
**Seed Job via JCasC**
76+
A seed pipeline job is defined via Jenkins Configuration as Code
77+
- The seed job pulls the Pipeline-Forge repository and bootstraps Jenkins CI pipelines from repository-managed definitions
78+
79+
**Two-Step Bootstrap (Intentional Design)**
80+
- Jenkins installation and CI/CD pipeline creation are deliberately separated
81+
- Keeps bootstrap logs clean and avoids hiding configuration issues
82+
- Allows you to debug seed job errors without re-running the entire cluster setup
83+
- Enables independent iteration on CI/CD pipeline definitions
84+
85+
**CI-Focused Plugin Set**
86+
- Additional plugins for multibranch pipelines, GitHub integration, Job DSL, Kubernetes agents, and JCasC
87+
88+
**Job DSL Script Security**
89+
- Script security for Job DSL is disabled via init script to allow seed job execution without needing manual approval
90+
- Can be manually re-enabled in Jenkins UI (Manage Jenkins → Configure System → Job DSL) after the initial seed job completes if required
91+
92+
**Local-First Defaults**
93+
- Jenkins URL: `http://localhost:8080`
94+
- Access via port-forwarding with no external service
95+
96+
**Configuration**
97+
98+
Override values are defined in `values.lab.yaml`.
99+
100+
**Note:** Plugin versions are intentionally unpinned for this development environment to always pull the latest versions.
101+
102+
### References
103+
104+
- [Installing Jenkins on Kubernetes](https://www.jenkins.io/doc/book/installing/kubernetes/#install-jenkins)
105+
- [Jenkins Helm Chart Default Values](https://raw.githubusercontent.com/jenkinsci/helm-charts/main/charts/jenkins/values.yaml)

‎jenkins/bootstrap_jenkins.sh‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
#!/usr/bin/env bash
2+
3+
set -euo pipefail
4+
5+
K8S_RUNTIME=${1:-kind}
6+
ROLLOUT_TIMEOUT=3m
7+
NAMESPACE=jenkins
8+
RELEASE_CHART=jenkins
9+
10+
function apply_k8s_resources() {
11+
echo "Creating Kubernetes resources (Service Account, RBAC, PV, PVC)..."
12+
kubectl create ns "$NAMESPACE"
13+
kubectl apply -n "$NAMESPACE" -f k8s/jenkins-serviceAccount-and-rbac.yaml
14+
kubectl apply -n "$NAMESPACE" -f k8s/jenkins-volume.yaml
15+
}
16+
17+
function helm_install() {
18+
echo "Installing Jenkins via Helm..."
19+
helm repo add jenkinsci https://charts.jenkins.io > /dev/null
20+
helm repo update >/dev/null
21+
helm install "$RELEASE_CHART" -n "$NAMESPACE" -f values.lab.yaml jenkinsci/jenkins
22+
}
23+
24+
function wait_rollout() {
25+
echo "Waiting for rollout to complete..."
26+
if ! kubectl rollout status statefulset/jenkins -n "$NAMESPACE" --timeout=$ROLLOUT_TIMEOUT; then
27+
echo "Rollout timed out, double check the pod/container logs and events for more details." >&2
28+
exit 1
29+
fi
30+
}
31+
32+
function main() {
33+
if [[ $K8S_RUNTIME == "kind" ]]; then
34+
kind create cluster --name jenkins # suffix -control-plane is added by kind
35+
elif [[ $K8S_RUNTIME == "minikube" ]]; then
36+
minikube start -p jenkins-control-plane
37+
else
38+
echo "Unknown runtime: $K8S_RUNTIME"
39+
exit 1
40+
fi
41+
42+
apply_k8s_resources
43+
helm_install
44+
wait_rollout
45+
46+
printf '\033[1;32m%s\033[0m\n\n' '✔ Jenkins is ready!'
47+
echo -e "Run 'kubectl port-forward svc/jenkins -n \"$NAMESPACE\" 8080:8080' to access Jenkins UI.\n"
48+
echo "admin password: 'kubectl get secret -n \"$NAMESPACE\" jenkins -o jsonpath={.data.jenkins-admin-password} | base64 --decode'"
49+
}
50+
51+
main "$@"

‎jenkins/ci/ingest-test.Jenkinsfile‎

Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
pipeline {
2+
agent any
3+
4+
options {
5+
timeout(time: 30, unit: 'MINUTES')
6+
timestamps()
7+
disableConcurrentBuilds()
8+
buildDiscarder(logRotator(numToKeepStr: '10'))
9+
}
10+
11+
environment {
12+
IMAGE_TAG = "ingest:ci-${env.GIT_COMMIT}"
13+
WORKDIR = 'workloads/ingest'
14+
}
15+
16+
stages {
17+
stage('Checkout') {
18+
steps {
19+
checkout scmGit(
20+
branches: scm.branches,
21+
extensions: [cloneOption(shallow: true, depth: 1)],
22+
userRemoteConfigs: scm.userRemoteConfigs
23+
)
24+
}
25+
}
26+
27+
stage('Setup Python Environment') {
28+
steps {
29+
dir("${env.WORKDIR}") {
30+
sh '''
31+
# Install UV package manager
32+
curl -LsSf https://astral.sh/uv/install.sh | sh
33+
export PATH="$HOME/.local/bin:$PATH"
34+
35+
# Verify we're in the right directory with pyproject.toml
36+
ls -la pyproject.toml
37+
38+
# Create virtual environment and install dependencies from pyproject.toml
39+
uv venv
40+
uv sync --frozen
41+
uv pip install -e . --no-deps
42+
'''
43+
}
44+
}
45+
}
46+
47+
stage('Quality Checks') {
48+
parallel {
49+
stage('Ruff Linter') {
50+
steps {
51+
dir("${env.WORKDIR}") {
52+
sh '''
53+
export PATH="$HOME/.local/bin:$PATH"
54+
uv run ruff check .
55+
'''
56+
}
57+
}
58+
}
59+
60+
stage('Ruff Format Check') {
61+
steps {
62+
dir("${env.WORKDIR}") {
63+
sh '''
64+
export PATH="$HOME/.local/bin:$PATH"
65+
uv run ruff format --check .
66+
'''
67+
}
68+
}
69+
}
70+
71+
stage('Type Check (mypy)') {
72+
steps {
73+
dir("${env.WORKDIR}") {
74+
sh '''
75+
export PATH="$HOME/.local/bin:$PATH"
76+
uv run mypy .
77+
'''
78+
}
79+
}
80+
}
81+
}
82+
}
83+
84+
stage('Run Tests') {
85+
steps {
86+
dir("${env.WORKDIR}") {
87+
// Continue on error to mirror GitHub Actions continue-on-error: true
88+
catchError(buildResult: 'SUCCESS', stageResult: 'UNSTABLE') {
89+
sh '''
90+
export PATH="$HOME/.local/bin:$PATH"
91+
uv run pytest . -v --junitxml=test-results.xml
92+
'''
93+
}
94+
}
95+
}
96+
post {
97+
always {
98+
// Publish test results if they exist
99+
junit allowEmptyResults: true, testResults: "${env.WORKDIR}/test-results.xml"
100+
}
101+
}
102+
}
103+
104+
stage('Build Docker Image') {
105+
steps {
106+
dir("${env.WORKDIR}") {
107+
sh """
108+
docker build -t ${env.IMAGE_TAG} .
109+
"""
110+
}
111+
}
112+
}
113+
114+
stage('Smoke Test') {
115+
steps {
116+
sh """
117+
docker run --rm ${env.IMAGE_TAG} ingest --help
118+
"""
119+
}
120+
}
121+
122+
stage('Archive Docker Image') {
123+
steps {
124+
dir("${env.WORKDIR}") {
125+
sh """
126+
docker save ${env.IMAGE_TAG} -o ingest-${env.GIT_COMMIT}.tar
127+
"""
128+
}
129+
archiveArtifacts artifacts: "${env.WORKDIR}/ingest-${env.GIT_COMMIT}.tar", fingerprint: true
130+
}
131+
}
132+
}
133+
134+
post {
135+
always {
136+
echo 'Pipeline completed'
137+
}
138+
success {
139+
echo 'Build succeeded!'
140+
}
141+
failure {
142+
echo 'Build failed!'
143+
}
144+
cleanup {
145+
// Clean up Docker images to save space
146+
sh """
147+
docker rmi ${env.IMAGE_TAG} || true
148+
rm -f ${env.WORKDIR}/ingest-${env.GIT_COMMIT}.tar || true
149+
"""
150+
}
151+
}
152+
}

‎jenkins/ci/operator-e2e-test.Jenkinsfile‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
pipeline {
2+
agent any
3+
stages {
4+
stage('Placeholder') {
5+
steps {
6+
echo 'Work in progress — please check ingest-test'
7+
}
8+
}
9+
}
10+
}

‎jenkins/ci/operator-test.Jenkinsfile‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
pipeline {
2+
agent any
3+
stages {
4+
stage('Placeholder') {
5+
steps {
6+
echo 'Work in progress — please check ingest-test'
7+
}
8+
}
9+
}
10+
}

‎jenkins/ci/trigger-test.Jenkinsfile‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
pipeline {
2+
agent any
3+
stages {
4+
stage('Placeholder') {
5+
steps {
6+
echo 'Work in progress — please check ingest-test'
7+
}
8+
}
9+
}
10+
}

‎jenkins/job-dsl/seed.Jenkinsfile‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
pipeline {
2+
agent any
3+
4+
options {
5+
timeout(time: 10, unit: 'MINUTES')
6+
}
7+
8+
stages {
9+
stage('Checkout') {
10+
steps {
11+
checkout scmGit(
12+
branches: scm.branches,
13+
extensions: [cloneOption(shallow: true, depth: 1, noTags: true)],
14+
userRemoteConfigs: scm.userRemoteConfigs
15+
)
16+
}
17+
}
18+
19+
stage('Generate jobs') {
20+
steps {
21+
jobDsl targets: 'jenkins/job-dsl/*.groovy',
22+
removedJobAction: 'DELETE',
23+
removedViewAction: 'DELETE',
24+
lookupStrategy: 'JENKINS_ROOT'
25+
}
26+
}
27+
}
28+
}

0 commit comments

Comments
 (0)