Skip to content

Commit 333111a

Browse files
committed
feat: Migrate cloud deployment example from Kind to Minikube
Replaces Kind with Minikube due to stability issues, especially with Podman. Adds support for both Docker and Podman container runtimes. Changes: - Migrate from Kind to Minikube with NodePort (port 30080) - Podman: Use tarball + minikube image load (no registry needed) - Docker: Use Minikube registry addon with port-forwarding - Linux + Podman: Rootless mode with containerd runtime - Linux + Podman: Check broker pod directly (entity operator compatibility issues) - Linux + Podman: Poll Kafka broker until topic exists before deploying agents - Mac/Linux + Podman: Background service tunnel for NodePort access - Dynamic image references and imagePullPolicy based on container tool Tested on: - macOS with Podman ✓ - Linux (Fedora) with Podman ✓ - Linux (GitHub Actions) with Docker ✓
1 parent 6f1b5b2 commit 333111a

File tree

6 files changed

+425
-214
lines changed

6 files changed

+425
-214
lines changed

.github/workflows/cloud-deployment-example.yml

Lines changed: 15 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -25,40 +25,44 @@ jobs:
2525
distribution: 'temurin'
2626
cache: maven
2727

28-
- name: Install Kind
28+
- name: Install Minikube
2929
run: |
30-
curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.20.0/kind-linux-amd64
31-
chmod +x ./kind
32-
sudo mv ./kind /usr/local/bin/kind
33-
kind version
30+
curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
31+
chmod +x minikube
32+
sudo mv minikube /usr/local/bin/
3433
3534
- name: Build project
3635
run: mvn -B clean install -DskipTests -f pom.xml
3736

38-
- name: Deploy with Kind
37+
- name: Deploy with Minikube
3938
working-directory: examples/cloud-deployment/scripts
4039
run: |
4140
chmod +x deploy.sh
42-
./deploy.sh --container-tool docker
41+
./deploy.sh
4342
4443
- name: Verify deployment
4544
working-directory: examples/cloud-deployment/scripts
4645
run: |
4746
chmod +x verify.sh
4847
./verify.sh
4948
49+
- name: Get service URL
50+
id: service
51+
run: |
52+
echo "url=$(minikube service a2a-agent-service -n a2a-demo --url)" >> $GITHUB_OUTPUT
53+
5054
- name: Verify agent card is accessible
5155
run: |
52-
echo "Testing agent card endpoint at http://localhost:8080/.well-known/agent-card.json"
53-
curl -f http://localhost:8080/.well-known/agent-card.json || (echo "Agent card not accessible" && exit 1)
56+
echo "Testing agent card endpoint at ${{ steps.service.outputs.url }}/.well-known/agent-card.json"
57+
curl -f ${{ steps.service.outputs.url }}/.well-known/agent-card.json || (echo "Agent card not accessible" && exit 1)
5458
5559
- name: Run test client
5660
working-directory: examples/cloud-deployment/server
5761
run: |
5862
mvn test-compile exec:java \
5963
-Dexec.mainClass="io.a2a.examples.cloud.A2ACloudExampleClient" \
6064
-Dexec.classpathScope=test \
61-
-Dagent.url=http://localhost:8080
65+
-Dagent.url=${{ steps.service.outputs.url }}
6266
6367
- name: Show diagnostics on failure
6468
if: failure()
@@ -96,6 +100,6 @@ jobs:
96100
if: always()
97101
run: |
98102
cd examples/cloud-deployment/scripts
99-
echo "y" | ./cleanup.sh --container-tool docker || true
103+
echo "y" | ./cleanup.sh || true
100104
101105

examples/cloud-deployment/README.md

Lines changed: 60 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -11,12 +11,12 @@ Note that the aim of this example is just to demonstrate how to set up a2a-java
1111
## Architecture
1212

1313
```
14-
localhost:8080
15-
(extraPortMappings)
14+
minikube service --url
15+
(tunnel to NodePort)
1616
1717
1818
┌───────────────────────────────────┼───────────────────────┐
19-
│ Kubernetes Cluster (Kind) │ │
19+
│ Kubernetes Cluster (Minikube) │ │
2020
│ │ │
2121
│ ┌───────────▼──────────┐ │
2222
│ │ Service (NodePort) │ │
@@ -60,7 +60,7 @@ Note that the aim of this example is just to demonstrate how to set up a2a-java
6060

6161
## Prerequisites
6262

63-
- **Kind** (Kubernetes IN Docker) v0.20+
63+
- **Minikube** v1.30+
6464
- **kubectl** (v1.27+)
6565
- **Maven** (3.8+)
6666
- **Java** 17+
@@ -70,15 +70,15 @@ Note that the aim of this example is just to demonstrate how to set up a2a-java
7070

7171
### 1. Install Prerequisites
7272

73-
**Install Kind:**
74-
See https://kind.sigs.k8s.io/docs/user/quick-start/ for installation instructions.
73+
**Install Minikube:**
74+
See https://minikube.sigs.k8s.io/docs/start/ for installation instructions.
7575

7676
**Install kubectl:**
7777
See https://kubernetes.io/docs/tasks/tools/ for installation instructions.
7878

7979
### 2. Deploy the Stack
8080

81-
The deployment script will automatically create the Kind cluster and deploy all components:
81+
The deployment script will automatically create the Minikube cluster and deploy all components:
8282

8383
```bash
8484
cd scripts
@@ -88,20 +88,21 @@ cd scripts
8888
**If using Podman instead of Docker:**
8989
```bash
9090
./deploy.sh --container-tool podman
91+
# OR set environment variable:
92+
export CONTAINER_TOOL=podman
93+
./deploy.sh
9194
```
9295

93-
Note that using Kind with Podman on Linux may have some occasional issues due to Kind's experimental support for Podman. In our testing, a reboot normally solves this.
96+
**Note**: Minikube with Podman requires the Podman driver. The script will configure this automatically.
9497

9598
The script will:
96-
- Create Kind cluster with local registry support (if not already exists)
97-
- Set up local container registry (localhost:5001)
99+
- Start Minikube cluster (if not already running)
100+
- Enable the registry addon
98101
- Install Strimzi Kafka operator
99102
- Deploy PostgreSQL
100103
- Deploy Kafka cluster (using KRaft mode)
101104
- Build and deploy the A2A agent (2 pods)
102105

103-
**Note:** You don't need to manually create the Kind cluster - the script handles everything.
104-
105106
### 3. Verify Deployment
106107

107108
```bash
@@ -121,13 +122,20 @@ Expected output:
121122

122123
#### Understanding the NodePort Setup
123124

124-
The agent service uses **NodePort** with Kind **extraPortMappings** to expose the service:
125+
The agent service uses **NodePort** for external access with reliable load balancing:
126+
127+
- Kubernetes Service exposes **NodePort 30080****pod port 8080** (configured in `k8s/05-agent-deployment.yaml`)
128+
- Minikube makes NodePort accessible via `minikube service` command
129+
- Result: Access the agent at the URL provided by `minikube service a2a-agent-service -n a2a-demo --url`
125130

126-
- Kind maps **host port 8080****node port 30080** (configured in `kind-config.yaml`)
127-
- Kubernetes Service maps **NodePort 30080****pod port 8080** (configured in `k8s/05-agent-deployment.yaml`)
128-
- Result: Access the agent at **http://localhost:8080** from your host machine
131+
**Why NodePort instead of LoadBalancer?**
132+
- ✅ Works without `minikube tunnel` (which can have networking issues, especially in CI)
133+
- ✅ Provides reliable round-robin load balancing across multiple pods
134+
- ✅ Simpler setup for local development
135+
- ✅ Consistent behavior across different host operating systems (macOS, Linux, Windows)
136+
- ✅ Works reliably in CI environments like GitHub Actions
129137

130-
This approach provides the same round-robin load balancing as a real LoadBalancer but works consistently across all platforms (macOS, Linux, Windows, and CI environments like GitHub Actions).
138+
**Note**: The test client creates fresh HTTP connections for each request to ensure proper load distribution across both pods.
131139

132140
#### Run the Test Client
133141

@@ -339,23 +347,17 @@ kubectl logs <pod-name> -n a2a-demo
339347
**Registry not accessible:**
340348

341349
```bash
342-
# Verify registry is running
343-
docker ps | grep kind-registry
344-
# or for Podman:
345-
podman ps | grep kind-registry
350+
# Verify Minikube registry addon is enabled
351+
minikube addons list | grep registry
346352

347-
# Verify registry is accessible
348-
curl http://localhost:5001/v2/
349-
# Should return: {}
350-
351-
# List images in registry
352-
curl http://localhost:5001/v2/_catalog
353+
# Verify images are cached in Minikube
354+
minikube ssh "crictl images" | grep a2a-cloud-deployment
353355
```
354356

355357
**Image push failures:**
356-
- Check registry container is running
357-
- For Podman: Ensure you use `--tls-verify=false` flag when pushing
358-
- Restart registry if needed: `docker/podman stop kind-registry && docker/podman start kind-registry`
358+
- Verify Minikube is running: `minikube status`
359+
- Check registry addon is enabled: `minikube addons enable registry`
360+
- For Podman: Ensure Minikube is started with `--driver=podman`
359361

360362
### Database Connection Failures
361363

@@ -407,10 +409,8 @@ kubectl get svc a2a-agent-service -n a2a-demo
407409
# NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
408410
# a2a-agent-service NodePort 10.96.123.45 <none> 8080:30080/TCP 5m
409411

410-
# 3. Verify Kind extraPortMappings are configured
411-
docker ps | grep kind-control-plane
412-
# or for Podman:
413-
podman ps | grep kind-control-plane
412+
# 3. Get service URL via minikube
413+
minikube service a2a-agent-service -n a2a-demo --url
414414
```
415415

416416
**Only seeing 1 pod:**
@@ -436,20 +436,22 @@ kubectl get crd kafkas.kafka.strimzi.io
436436
kubectl create -f 'https://strimzi.io/install/latest?namespace=kafka' -n kafka
437437
```
438438

439-
### Kind Resource Issues
439+
### Minikube Resource Issues
440440

441441
**Insufficient resources:**
442-
Kind uses your Docker/Podman resources. Increase Docker Desktop memory/CPU limits if needed.
442+
Minikube uses your Docker/Podman resources. Increase Docker Desktop or Podman Machine memory/CPU limits if needed.
443443

444444
**Disk space:**
445445
```bash
446-
# Check disk usage inside Kind node
447-
docker exec -it kind-control-plane df -h
448-
# or for Podman:
449-
podman exec -it kind-control-plane df -h
446+
# Check disk usage inside Minikube
447+
minikube ssh df -h
450448

451449
# Clean up old images
452450
docker system prune -a # or: podman system prune -a
451+
452+
# Delete and recreate Minikube with more resources
453+
minikube delete
454+
minikube start --cpus=4 --memory=8192
453455
```
454456

455457
## Cleanup
@@ -461,6 +463,16 @@ cd scripts
461463
./cleanup.sh
462464
```
463465

466+
**If you used Podman:**
467+
```bash
468+
./cleanup.sh --container-tool podman
469+
```
470+
471+
**Skip confirmation prompt:**
472+
```bash
473+
./cleanup.sh --yes
474+
```
475+
464476
This will delete:
465477
- A2A agent deployment and service
466478
- Kafka cluster
@@ -472,23 +484,18 @@ To also remove Strimzi operator:
472484
kubectl delete namespace kafka
473485
```
474486

475-
To delete the Kind cluster:
487+
To stop the Minikube cluster:
476488
```bash
477-
kind delete cluster
489+
minikube stop
478490
```
479491

480492
### Complete Clean Slate
481493

482494
For a completely fresh start (useful for testing from scratch):
483495

484496
```bash
485-
# Delete Kind cluster
486-
kind delete cluster
487-
488-
# Remove local registry container
489-
docker stop kind-registry && docker rm kind-registry
490-
# or for Podman:
491-
podman stop kind-registry && podman rm kind-registry
497+
# Delete Minikube cluster
498+
minikube delete
492499

493500
# Optional: Clean up container images
494501
docker system prune -a # or: podman system prune -a
@@ -571,9 +578,10 @@ From `pom.xml`:
571578
## References
572579

573580
- [A2A Protocol Specification](https://github.com/a2aproject/a2a)
574-
- [Kind Quick Start](https://kind.sigs.k8s.io/docs/user/quick-start/)
575-
- [Kind Local Registry](https://kind.sigs.k8s.io/docs/user/local-registry/)
576-
- [Kind extraPortMappings](https://kind.sigs.k8s.io/docs/user/configuration/#extra-port-mappings)
581+
- [Minikube Quick Start](https://minikube.sigs.k8s.io/docs/start/)
582+
- [Minikube Services](https://minikube.sigs.k8s.io/docs/handbook/accessing/)
583+
- [Minikube with Podman](https://minikube.sigs.k8s.io/docs/drivers/podman/)
577584
- [Strimzi Kafka Operator](https://strimzi.io/)
578585
- [Quarkus Reactive Messaging](https://quarkus.io/guides/kafka)
579586
- [Kubernetes Downward API](https://kubernetes.io/docs/concepts/workloads/pods/downward-api/)
587+
- [Kubernetes NodePort Service](https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport)

examples/cloud-deployment/k8s/05-agent-deployment.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ spec:
1717
spec:
1818
containers:
1919
- name: a2a-agent
20-
image: localhost:5001/a2a-cloud-deployment:latest
20+
image: localhost:5000/a2a-cloud-deployment:latest
2121
# Always pull to ensure latest image is used when rebuilding with :latest tag
2222
# For production, use digest-based image references (e.g., @sha256:...)
2323
imagePullPolicy: Always
@@ -91,6 +91,6 @@ spec:
9191
- protocol: TCP
9292
port: 8080 # Service port inside cluster
9393
targetPort: 8080 # Pod port
94-
nodePort: 30080 # Fixed NodePort (mapped to host port 8080 via Kind config)
94+
nodePort: 30080 # Fixed NodePort for consistent access
9595
type: NodePort
9696
sessionAffinity: None # Ensures round-robin load balancing

examples/cloud-deployment/scripts/cleanup.sh

Lines changed: 26 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
#!/bin/bash
22

33
echo "============================================"
4-
echo "A2A Cloud Deployment - Cleanup Script"
4+
echo "A2A Cloud Deployment - Cleanup Script for Minikube"
55
echo "============================================"
66
echo ""
77

@@ -12,33 +12,40 @@ YELLOW='\033[1;33m'
1212
NC='\033[0m' # No Color
1313

1414
# Parse command line arguments
15-
CONTAINER_TOOL="docker"
15+
CONTAINER_TOOL="${CONTAINER_TOOL:-docker}" # Use env var or default to docker
16+
SKIP_CONFIRMATION=false
1617
while [[ $# -gt 0 ]]; do
1718
case $1 in
1819
--container-tool)
1920
CONTAINER_TOOL="$2"
2021
shift 2
2122
;;
23+
-y|--yes)
24+
SKIP_CONFIRMATION=true
25+
shift
26+
;;
2227
*)
2328
echo -e "${RED}Unknown option: $1${NC}"
24-
echo "Usage: $0 [--container-tool docker|podman]"
29+
echo "Usage: $0 [--container-tool docker|podman] [--yes]"
2530
exit 1
2631
;;
2732
esac
2833
done
2934

30-
# Configure Kind to use podman if specified
35+
# Configure Minikube driver based on container tool
3136
if [ "$CONTAINER_TOOL" = "podman" ]; then
32-
export KIND_EXPERIMENTAL_PROVIDER=podman
37+
export MINIKUBE_DRIVER=podman
3338
fi
3439

35-
echo -e "${YELLOW}This will delete all resources in the a2a-demo namespace and the Kind cluster${NC}"
36-
read -p "Are you sure you want to continue? (y/N) " -n 1 -r
37-
echo ""
40+
if [ "$SKIP_CONFIRMATION" != "true" ]; then
41+
echo -e "${YELLOW}This will delete all resources in the a2a-demo namespace and stop the Minikube cluster${NC}"
42+
read -p "Are you sure you want to continue? (y/N) " -n 1 -r
43+
echo ""
3844

39-
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
40-
echo "Cleanup cancelled"
41-
exit 0
45+
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
46+
echo "Cleanup cancelled"
47+
exit 0
48+
fi
4249
fi
4350

4451
echo ""
@@ -66,13 +73,16 @@ echo "Deleting namespace..."
6673
kubectl delete -f ../k8s/00-namespace.yaml --ignore-not-found=true
6774

6875
echo ""
69-
echo "Deleting Kind cluster..."
70-
kind delete cluster
76+
echo "Stopping registry port-forward..."
77+
# Stop socat container (macOS)
78+
$CONTAINER_TOOL stop socat-registry 2>/dev/null || true
79+
$CONTAINER_TOOL rm socat-registry 2>/dev/null || true
80+
# Kill kubectl port-forward (Linux)
81+
pkill -f "kubectl.*port-forward.*registry" 2>/dev/null || true
7182

7283
echo ""
73-
echo "Stopping and removing registry container..."
74-
$CONTAINER_TOOL stop kind-registry > /dev/null 2>&1 || true
75-
$CONTAINER_TOOL rm kind-registry > /dev/null 2>&1 || true
84+
echo "Stopping Minikube..."
85+
minikube stop
7686

7787
echo ""
7888
echo -e "${GREEN}Cleanup completed${NC}"

0 commit comments

Comments
 (0)