Skip to content

Commit de61975

Browse files
committed
refactor: Use Minikube registry addon instead of external registry
Replace external registry container approach with Minikube's built-in registry addon for improved cross-platform compatibility. Key changes: - Use 'minikube addons enable registry' instead of external container - Port-forward to localhost:5000 (macOS: socat, Linux: kubectl) - Update image reference from 192.168.49.1:5001 to localhost:5000 - Change imagePullPolicy from Always to IfNotPresent - Remove insecure-registry requirement from Minikube start - Simplify GitHub Actions workflow Benefits: - Works with rootless Podman on Fedora/Linux - No insecure registry configuration needed - Cross-platform consistency (Mac, Linux, Windows) - Follows proven WildFly cloud-tests patterns - Simpler setup and troubleshooting Fixes compatibility issues with rootless Podman where external registry running in user context was inaccessible from Minikube VM.
1 parent 4733ed9 commit de61975

File tree

5 files changed

+134
-87
lines changed

5 files changed

+134
-87
lines changed

.github/workflows/cloud-deployment-example.yml

Lines changed: 31 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -33,20 +33,44 @@ jobs:
3333
kubernetes-version: v1.30.0
3434
driver: docker
3535
addons: ingress
36-
insecure-registry: '192.168.49.1:5001'
3736

3837
- name: Build project
3938
run: mvn -B clean install -DskipTests -f pom.xml
4039

40+
- name: Enable Minikube registry addon
41+
run: |
42+
echo "Enabling Minikube registry addon..."
43+
minikube addons enable registry
44+
sleep 5
45+
46+
- name: Set up registry port forwarding
47+
run: |
48+
echo "Starting registry port forwarding..."
49+
kubectl port-forward --namespace kube-system service/registry 5000:80 > /dev/null 2>&1 &
50+
51+
# Wait for registry to be accessible
52+
echo "Waiting for registry to be accessible..."
53+
for i in {1..30}; do
54+
if curl -s http://localhost:5000/v2/ > /dev/null 2>&1; then
55+
echo "Registry accessible"
56+
break
57+
fi
58+
if [ $i -eq 30 ]; then
59+
echo "ERROR: Registry not accessible"
60+
exit 1
61+
fi
62+
sleep 1
63+
done
64+
4165
- name: Build and package server
4266
working-directory: examples/cloud-deployment/server
4367
run: mvn clean package -DskipTests
4468

45-
- name: Build Docker image
69+
- name: Build and push Docker image
4670
working-directory: examples/cloud-deployment/server
4771
run: |
48-
docker build -t localhost:5001/a2a-cloud-deployment:latest .
49-
docker tag localhost:5001/a2a-cloud-deployment:latest 192.168.49.1:5001/a2a-cloud-deployment:latest
72+
docker build -t localhost:5000/a2a-cloud-deployment:latest .
73+
docker push localhost:5000/a2a-cloud-deployment:latest
5074
5175
- name: Deploy infrastructure (skip agent)
5276
working-directory: examples/cloud-deployment/scripts
@@ -56,24 +80,12 @@ jobs:
5680
chmod +x deploy.sh
5781
./deploy.sh
5882
59-
- name: Load image into minikube and deploy agent
83+
- name: Deploy agent
6084
working-directory: examples/cloud-deployment
6185
run: |
62-
# Load image into minikube
63-
echo "Loading image into minikube..."
64-
minikube image load 192.168.49.1:5001/a2a-cloud-deployment:latest
65-
66-
# Verify image is in minikube
67-
echo "Verifying image in minikube:"
68-
minikube ssh -- docker images | grep a2a-cloud-deployment
69-
70-
# Modify deployment YAML to use imagePullPolicy: Never before applying
71-
echo "Deploying agent with imagePullPolicy: Never..."
72-
cat k8s/04-agent-deployment.yaml | \
73-
sed 's/imagePullPolicy: Always/imagePullPolicy: Never/' | \
74-
kubectl apply -f -
86+
echo "Deploying agent..."
87+
kubectl apply -f k8s/04-agent-deployment.yaml
7588
76-
# Wait for rollout to complete
7789
echo "Waiting for deployment rollout to complete..."
7890
kubectl rollout status deployment/a2a-agent -n a2a-demo --timeout=120s
7991

examples/cloud-deployment/README.md

Lines changed: 32 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -61,19 +61,22 @@ This example demonstrates deploying an A2A agent to Kubernetes with:
6161

6262
### 1. Start Minikube
6363

64-
This example uses a local container registry on the host machine. Minikube must be configured to allow pulling from this insecure (HTTP) registry.
64+
This example uses Minikube's built-in registry addon for container image distribution.
6565

6666
**With Docker:**
6767
```bash
68-
minikube start --cpus=4 --memory=8192 --insecure-registry=192.168.49.1:5001
68+
minikube start --cpus=4 --memory=8192
6969
```
7070

71-
**With Podman:**
71+
**With Podman (rootful or rootless):**
7272
```bash
73-
minikube start --cpus=4 --memory=8192 --driver=podman --insecure-registry=192.168.49.1:5001
73+
minikube start --cpus=4 --memory=8192 --driver=podman
7474
```
7575

76-
**Note**: The IP address `192.168.49.1` is the default host IP from Minikube's perspective. If your Minikube uses a different network range, adjust accordingly.
76+
**Note for rootless Podman users on Fedora/Linux**: Add `--container-runtime=containerd` if you encounter issues:
77+
```bash
78+
minikube start --cpus=4 --memory=8192 --driver=podman --container-runtime=containerd
79+
```
7780

7881
### 2. Deploy the Stack
7982

@@ -88,6 +91,8 @@ cd scripts
8891
```
8992

9093
This script will:
94+
- Enable Minikube registry addon
95+
- Set up registry port forwarding (localhost:5000)
9196
- Install Strimzi Kafka operator
9297
- Deploy PostgreSQL
9398
- Deploy Kafka cluster
@@ -319,40 +324,45 @@ kubectl logs <pod-name> -n a2a-demo
319324
```
320325

321326
**Common issues:**
322-
- **ImagePullBackOff**: Image not built in Minikube's container environment
323-
- Solution with Docker: Run `eval $(minikube docker-env)` before building
324-
- Solution with Podman: Run `eval $(minikube podman-env)` before building
327+
- **ImagePullBackOff**: Image not pushed to Minikube registry
328+
- Solution: Ensure registry port-forward is running and push completed successfully
329+
- Check: `curl http://localhost:5000/v2/_catalog` should list the image
325330
- **CrashLoopBackOff**: Application startup failure
326331
- Check logs: `kubectl logs <pod-name> -n a2a-demo`
327332
- Common causes: Database not ready, Kafka not ready
328333

329-
### Container Build Issues
334+
### Registry Issues
330335

331-
**Image not found in Minikube:**
336+
**Registry not accessible:**
332337

333-
The container image must be built inside Minikube's environment.
338+
The deploy.sh script sets up port forwarding automatically. If you need to set it up manually:
334339

335-
**With Docker:**
340+
**On macOS:**
336341
```bash
337-
eval $(minikube docker-env)
338-
docker build -t a2a-cloud-deployment:latest .
342+
# Using Docker or Podman
343+
docker run -d --name socat-registry --rm --network=host alpine \
344+
ash -c "apk add socat && socat TCP-LISTEN:5000,reuseaddr,fork TCP:$(minikube ip):5000"
339345
```
340346

341-
**With Podman:**
347+
**On Linux:**
342348
```bash
343-
eval $(minikube podman-env)
344-
podman build -t a2a-cloud-deployment:latest .
349+
kubectl port-forward --namespace kube-system service/registry 5000:80 &
345350
```
346351

347-
**Verify image exists:**
352+
**Verify registry is accessible:**
348353
```bash
349-
# With Docker
350-
minikube ssh docker images | grep a2a-cloud-deployment
354+
curl http://localhost:5000/v2/
355+
# Should return: {}
351356

352-
# With Podman
353-
minikube ssh podman images | grep a2a-cloud-deployment
357+
# List images in registry
358+
curl http://localhost:5000/v2/_catalog
354359
```
355360

361+
**Image push failures:**
362+
- Check port-forward is running: `ps aux | grep "port-forward.*registry"`
363+
- Restart port-forward if needed
364+
- For rootless Podman: Ensure you're not using sudo (should work without it)
365+
356366
### Database Connection Failures
357367

358368
**Check PostgreSQL status:**

examples/cloud-deployment/k8s/04-agent-deployment.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,8 @@ spec:
1717
spec:
1818
containers:
1919
- name: a2a-agent
20-
image: 192.168.49.1:5001/a2a-cloud-deployment:latest
21-
imagePullPolicy: Always
20+
image: localhost:5000/a2a-cloud-deployment:latest
21+
imagePullPolicy: IfNotPresent
2222
ports:
2323
- containerPort: 8080
2424
name: http

examples/cloud-deployment/scripts/cleanup.sh

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -45,19 +45,25 @@ echo "Deleting namespace..."
4545
kubectl delete -f ../k8s/00-namespace.yaml --ignore-not-found=true
4646

4747
echo ""
48-
echo "Cleaning up local registry container..."
49-
# Determine container tool
48+
echo "Stopping registry port forwarding..."
49+
# Kill any port-forward processes
50+
pkill -f "kubectl.*port-forward.*registry" > /dev/null 2>&1 || true
51+
52+
# Determine container tool for stopping socat container on macOS
5053
CONTAINER_TOOL="docker"
5154
if command -v podman &> /dev/null; then
5255
CONTAINER_TOOL="podman"
5356
fi
5457

55-
# Remove registry container if it exists
56-
$CONTAINER_TOOL rm -f kind-registry > /dev/null 2>&1 || true
58+
# Stop socat container if running (macOS)
59+
$CONTAINER_TOOL stop socat-registry > /dev/null 2>&1 || true
60+
$CONTAINER_TOOL rm socat-registry > /dev/null 2>&1 || true
5761

5862
echo ""
5963
echo -e "${GREEN}Cleanup completed${NC}"
6064
echo ""
61-
echo -e "${YELLOW}Note: Strimzi operator was not removed${NC}"
65+
echo -e "${YELLOW}Note: Minikube registry addon and Strimzi operator were not removed${NC}"
66+
echo "To disable the registry addon, run:"
67+
echo " minikube addons disable registry"
6268
echo "To remove Strimzi operator, run:"
6369
echo " kubectl delete namespace kafka"

examples/cloud-deployment/scripts/deploy.sh

Lines changed: 58 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -34,66 +34,84 @@ echo ""
3434
# Check if Minikube is running
3535
if ! minikube status > /dev/null 2>&1; then
3636
echo -e "${RED}Error: Minikube is not running${NC}"
37-
echo "Please start Minikube first with insecure registry configuration:"
37+
echo "Please start Minikube first:"
3838
echo ""
3939
echo "With Docker:"
40-
echo " minikube start --cpus=4 --memory=8192 --insecure-registry=192.168.49.1:5001"
40+
echo " minikube start --cpus=4 --memory=8192"
4141
echo ""
4242
echo "With Podman:"
43-
echo " minikube start --cpus=4 --memory=8192 --driver=podman --insecure-registry=192.168.49.1:5001"
43+
echo " minikube start --cpus=4 --memory=8192 --driver=podman"
44+
echo ""
45+
echo "With Podman (rootless):"
46+
echo " minikube start --cpus=4 --memory=8192 --driver=podman --container-runtime=containerd"
4447
exit 1
4548
fi
4649

4750
echo -e "${GREEN}✓ Minikube is running${NC}"
4851

49-
# Check if insecure registry is configured
50-
echo "Checking insecure registry configuration..."
51-
MINIKUBE_HOST_IP=$(minikube ssh "ip route | grep default | awk '{print \$3}'")
52-
if ! minikube profile list -o json | grep -q "InsecureRegistry"; then
53-
echo -e "${RED}Error: Minikube is not configured for insecure registry${NC}"
54-
echo ""
55-
echo "Please delete and recreate Minikube with insecure registry support:"
56-
echo " minikube delete"
57-
echo " minikube start --cpus=4 --memory=8192 --driver=podman --insecure-registry=${MINIKUBE_HOST_IP}:5001"
58-
exit 1
52+
# Enable Minikube registry addon if not already enabled
53+
echo ""
54+
echo "Checking Minikube registry addon..."
55+
if ! minikube addons list | grep -q "registry.*enabled"; then
56+
echo "Enabling Minikube registry addon..."
57+
minikube addons enable registry
58+
# Wait a bit for registry to start
59+
sleep 5
60+
echo -e "${GREEN}✓ Registry addon enabled${NC}"
61+
else
62+
echo -e "${GREEN}✓ Registry addon already enabled${NC}"
5963
fi
60-
echo -e "${GREEN}✓ Insecure registry configured (${MINIKUBE_HOST_IP}:5001)${NC}"
6164

62-
# Set up local registry container on host
65+
# Set up port forwarding to registry
66+
# This makes the registry accessible at localhost:5000
6367
echo ""
64-
echo "Setting up local registry container..."
68+
echo "Setting up registry port forwarding..."
69+
70+
# Detect OS
71+
if echo "$OSTYPE" | grep -q "^darwin"; then
72+
# macOS - use socat in container for port forwarding
73+
echo "macOS detected, using socat for port forwarding..."
6574

66-
# Check if registry container is already running
67-
if $CONTAINER_TOOL ps --filter "name=kind-registry" --format '{{.Names}}' | grep -q kind-registry; then
68-
echo "Registry container already running"
75+
# Stop any existing port forwarder
76+
$CONTAINER_TOOL stop socat-registry 2>/dev/null || true
77+
$CONTAINER_TOOL rm socat-registry 2>/dev/null || true
78+
79+
# Pull alpine if needed
80+
$CONTAINER_TOOL pull alpine 2>/dev/null || true
81+
82+
# Start socat container for port forwarding
83+
$CONTAINER_TOOL run -d --name socat-registry --rm --network=host alpine \
84+
ash -c "apk add socat && socat TCP-LISTEN:5000,reuseaddr,fork TCP:$(minikube ip):5000" \
85+
> /dev/null 2>&1 &
86+
87+
echo -e "${GREEN}✓ Port forward started (socat container)${NC}"
6988
else
70-
# Remove old container if it exists but is stopped
71-
$CONTAINER_TOOL rm -f kind-registry > /dev/null 2>&1 || true
89+
# Linux - use kubectl port-forward
90+
echo "Linux detected, using kubectl port-forward..."
7291

73-
# Start registry container on host
74-
echo "Starting registry container on host (port 5001)..."
75-
$CONTAINER_TOOL run -d --restart=always -p "0.0.0.0:5001:5000" --name "kind-registry" registry:2
92+
# Kill any existing port-forward processes
93+
pkill -f "kubectl.*port-forward.*registry" || true
94+
95+
# Start port forward in background
96+
kubectl port-forward --namespace kube-system service/registry 5000:80 > /dev/null 2>&1 &
97+
98+
echo -e "${GREEN}✓ Port forward started (kubectl)${NC}"
7699
fi
77100

78-
# Verify registry is accessible
79-
echo "Verifying registry is accessible at localhost:5001..."
80-
for i in {1..10}; do
81-
if curl -s http://localhost:5001/v2/ > /dev/null 2>&1; then
82-
echo -e "${GREEN}✓ Registry accessible at localhost:5001${NC}"
101+
# Wait for registry to be accessible
102+
echo "Waiting for registry to be accessible at localhost:5000..."
103+
for i in {1..30}; do
104+
if curl -s http://localhost:5000/v2/ > /dev/null 2>&1; then
105+
echo -e "${GREEN}✓ Registry accessible at localhost:5000${NC}"
83106
break
84107
fi
85-
if [ $i -eq 10 ]; then
86-
echo -e "${RED}ERROR: Registry not accessible after 10 attempts${NC}"
108+
if [ $i -eq 30 ]; then
109+
echo -e "${RED}ERROR: Registry not accessible after 30 attempts${NC}"
87110
exit 1
88111
fi
89-
echo "Attempt $i/10..."
90112
sleep 1
91113
done
92114

93-
# Registry will be accessed from Minikube using host.minikube.internal
94-
REGISTRY="localhost:5001"
95-
echo "Using registry: $REGISTRY (host), host.minikube.internal:5001 (from Minikube)"
96-
97115
# Build the project
98116
echo ""
99117
echo "Building the project..."
@@ -102,17 +120,18 @@ mvn clean package -DskipTests
102120
echo -e "${GREEN}✓ Project built successfully${NC}"
103121

104122
# Build and push container image to Minikube registry
123+
REGISTRY="localhost:5000"
105124
echo ""
106125
echo "Building container image..."
107126
$CONTAINER_TOOL build -t ${REGISTRY}/a2a-cloud-deployment:latest .
108127
echo -e "${GREEN}✓ Container image built${NC}"
109128

110129
echo "Pushing image to Minikube registry..."
111-
# Retry push a few times as it can be flaky with Podman
130+
# Retry push a few times as port-forward can be flaky
112131
MAX_RETRIES=3
113132
for attempt in $(seq 1 $MAX_RETRIES); do
114133
echo "Push attempt $attempt/$MAX_RETRIES..."
115-
if $CONTAINER_TOOL push ${REGISTRY}/a2a-cloud-deployment:latest --tls-verify=false --retry=2 2>&1 | tee /tmp/push.log; then
134+
if $CONTAINER_TOOL push ${REGISTRY}/a2a-cloud-deployment:latest 2>&1 | tee /tmp/push.log; then
116135
echo -e "${GREEN}✓ Image pushed to registry${NC}"
117136
break
118137
else
@@ -245,4 +264,4 @@ echo "To access the agent from outside the cluster:"
245264
echo " kubectl port-forward -n a2a-demo svc/a2a-agent-service 8080:8080"
246265
echo ""
247266
echo "Then access the agent at:"
248-
echo " http://localhost:8080/a2a/agent-card"
267+
echo " http://localhost:8080/.well-known/agent-card.json"

0 commit comments

Comments
 (0)