Skip to content

Commit 9f59350

Browse files
nullchimpnullchimp
committed
Test Azure deployment
1 parent e9b2353 commit 9f59350

File tree

7 files changed

+583
-3
lines changed

7 files changed

+583
-3
lines changed

‎.github/workflows/deploy-azure.yml‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
name: Deploy to Azure
2+
3+
on:
4+
push:
5+
branches: [ RAG ]
6+
workflow_dispatch:
7+
8+
env:
9+
ENVIRONMENT: dev
10+
AZURE_RESOURCE_GROUP: GitHub
11+
AZURE_LOCATION: germanywestcentral
12+
13+
jobs:
14+
deploy-infrastructure:
15+
runs-on: ubuntu-latest
16+
steps:
17+
- uses: actions/checkout@v3
18+
19+
- name: Set up Python
20+
uses: actions/setup-python@v4
21+
with:
22+
python-version: '3.13'
23+
24+
- name: Azure Login
25+
uses: azure/login@v2
26+
with:
27+
client-id: ${{ secrets.AZURE_CLIENT_ID }}
28+
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
29+
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
30+
31+
- name: Setup Terraform
32+
uses: hashicorp/setup-terraform@v2
33+
34+
- name: Terraform Init
35+
run: |
36+
cd infra/azure
37+
terraform init
38+
39+
- name: Terraform Apply
40+
run: |
41+
cd infra/azure
42+
terraform apply -auto-approve \
43+
-var="environment=${{ env.ENVIRONMENT }}" \
44+
-var="memgraph_username=${{ secrets.MEMGRAPH_USERNAME }}" \
45+
-var="memgraph_password=${{ secrets.MEMGRAPH_PASSWORD }}"
46+
47+
- name: Get AKS Credentials
48+
run: |
49+
az aks get-credentials --resource-group ${{ env.AZURE_RESOURCE_GROUP }} \
50+
--name aks-ai-agent-${{ env.ENVIRONMENT }}
51+
52+
- name: Create K8s Secret for Memgraph
53+
run: |
54+
kubectl create secret generic memgraph-credentials \
55+
--from-literal=username=${{ secrets.MEMGRAPH_USERNAME }} \
56+
--from-literal=password=${{ secrets.MEMGRAPH_PASSWORD }} \
57+
--dry-run=client -o yaml | kubectl apply -f -
58+
59+
- name: Deploy to AKS
60+
run: |
61+
kubectl apply -f infra/k8s/memgraph.yaml
62+
63+
- name: Verify Deployment
64+
run: |
65+
kubectl get pods
66+
kubectl get services
67+
68+
- name: Run Tests
69+
run: |
70+
source .venv/bin/activate
71+
pytest tests/ --cov=src

‎README.md‎

Lines changed: 14 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,9 +19,20 @@ An intelligent AI agent framework written in Python, designed to facilitate seam
1919
- Modular file operations (read, write, list)
2020
- Web fetching and conversion utilities
2121
- Search client with pluggable backends
22-
- Tooling for codegen workflows
23-
- Configurable via environment variables and JSON configuration files
24-
- Retrieval-Augmented Generation (RAG) with graph database for knowledge management
22+
- Azure-based deployment with secure secret management
23+
24+
## Azure Deployment
25+
The AI Agent can be deployed to Azure using Kubernetes (AKS) with the following features:
26+
- Infrastructure as code using Terraform
27+
- Secrets managed securely in Azure Key Vault
28+
- Continuous deployment with GitHub Actions
29+
- Persistent storage for Memgraph data
30+
31+
To deploy to Azure:
32+
1. Run the setup script: `./scripts/setup_azure.sh`
33+
2. Add the generated service principal credentials to GitHub secrets as `AZURE_CREDENTIALS`
34+
3. Add `MEMGRAPH_USERNAME` and `MEMGRAPH_PASSWORD` to GitHub secrets
35+
4. Push to main branch to trigger deployment or manually trigger the workflow
2536

2637
## Architecture
2738
The project follows a component-based architecture where the AI Agent orchestrates interactions between users, language models, local tools, and MCP servers.

‎docs/azure-deployment-guide.md‎

Lines changed: 256 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,256 @@
1+
# Azure Deployment Guide for AI Agent
2+
3+
This guide provides a complete, step-by-step process for deploying the AI Agent's Memgraph database to Microsoft Azure using Kubernetes (AKS), with secure secret management and automated CI/CD.
4+
5+
## Table of Contents
6+
7+
1. [Prerequisites](#prerequisites)
8+
2. [Infrastructure Overview](#infrastructure-overview)
9+
3. [Setup Process](#setup-process)
10+
- [Setting up Azure Credentials](#setting-up-azure-credentials)
11+
- [GitHub Repository Configuration](#github-repository-configuration)
12+
- [Infrastructure Deployment](#infrastructure-deployment)
13+
4. [Understanding the Configuration Files](#understanding-the-configuration-files)
14+
- [Terraform Configuration](#terraform-configuration)
15+
- [Kubernetes Configuration](#kubernetes-configuration)
16+
- [GitHub Actions Workflow](#github-actions-workflow)
17+
5. [Accessing Memgraph](#accessing-memgraph)
18+
6. [Troubleshooting](#troubleshooting)
19+
7. [Cleanup](#cleanup)
20+
21+
## Prerequisites
22+
23+
Before starting, ensure you have:
24+
25+
- Azure account with access to the "GitHub" resource group in Germany West Central
26+
- GitHub repository set up and accessible
27+
- Local development tools:
28+
- Azure CLI installed
29+
- Terraform installed (optional, as it runs in GitHub Actions)
30+
- kubectl installed (optional, for direct access to AKS)
31+
- Python 3.9+ installed
32+
33+
## Infrastructure Overview
34+
35+
The deployment creates these resources in your existing "GitHub" resource group:
36+
37+
- **Azure Kubernetes Service (AKS)** - Container orchestration platform
38+
- **Azure Key Vault** - Secure secret management
39+
- **Azure Container Registry** - Docker image storage
40+
- **Persistent Volumes** - For Memgraph data, logs, and configuration
41+
- **Service Principal** - For secure GitHub Actions integration with Azure
42+
43+
## Setup Process
44+
45+
### Setting up Azure Credentials
46+
47+
1. Run the provided setup script:
48+
49+
```bash
50+
cd /Users/nullchimp/Projects/ai-agent
51+
./scripts/setup_azure.sh
52+
```
53+
54+
This script will:
55+
- Log you into Azure with `az login`
56+
- Create a service principal with Contributor access to the "GitHub" resource group
57+
58+
2. Copy the output JSON which looks similar to:
59+
60+
```json
61+
{
62+
"clientId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
63+
"clientSecret": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
64+
"subscriptionId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
65+
"tenantId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
66+
"activeDirectoryEndpointUrl": "https://login.microsoftonline.com",
67+
"resourceManagerEndpointUrl": "https://management.azure.com/",
68+
"activeDirectoryGraphResourceId": "https://graph.windows.net/",
69+
"sqlManagementEndpointUrl": "https://management.core.windows.net:8443/",
70+
"galleryEndpointUrl": "https://gallery.azure.com/",
71+
"managementEndpointUrl": "https://management.core.windows.net/"
72+
}
73+
```
74+
75+
Save this for the next step.
76+
77+
### GitHub Repository Configuration
78+
79+
1. In your GitHub repository, navigate to Settings → Secrets and variables → Actions.
80+
81+
2. Add the following secrets:
82+
83+
- **AZURE_CREDENTIALS**: Paste the entire JSON output from the setup script
84+
- **MEMGRAPH_USERNAME**: Choose a username for the Memgraph database
85+
- **MEMGRAPH_PASSWORD**: Choose a secure password for the Memgraph database
86+
87+
### Infrastructure Deployment
88+
89+
Deploy the infrastructure by either:
90+
91+
1. **Automatic Deployment** - Push to the main branch
92+
93+
```bash
94+
git add .
95+
git commit -m "Add Azure deployment configuration"
96+
git push origin main
97+
```
98+
99+
2. **Manual Deployment** - Trigger the workflow manually
100+
- Go to GitHub repository → Actions tab
101+
- Select "Deploy to Azure" workflow
102+
- Click "Run workflow" and select the main branch
103+
104+
3. **Monitor Deployment**:
105+
- Follow the progress in the GitHub Actions tab
106+
- The workflow will:
107+
- Set up Terraform
108+
- Deploy infrastructure to Azure
109+
- Configure Kubernetes
110+
- Deploy Memgraph to AKS
111+
- Verify the deployment
112+
- Run tests to validate everything is working
113+
114+
## Understanding the Configuration Files
115+
116+
### Terraform Configuration
117+
118+
Located at `/infra/azure/main.tf` and `/infra/azure/variables.tf`:
119+
120+
- **main.tf** - Defines the Azure resources:
121+
- References existing "GitHub" resource group
122+
- Creates Key Vault for secrets
123+
- Creates Container Registry for images
124+
- Provisions AKS cluster
125+
- Sets up proper access policies
126+
127+
- **variables.tf** - Defines variables:
128+
- `environment` - Deployment environment (default: "dev")
129+
- `memgraph_username` - Memgraph database username
130+
- `memgraph_password` - Memgraph database password
131+
132+
### Kubernetes Configuration
133+
134+
Located at `/infra/k8s/memgraph.yaml`:
135+
136+
- Defines Kubernetes resources:
137+
- ConfigMap for Memgraph configuration
138+
- Deployment for the Memgraph container
139+
- PersistentVolumeClaims for data persistence
140+
- Service for exposing Memgraph ports
141+
142+
The Kubernetes config ensures:
143+
- Memgraph container is properly configured
144+
- Credentials are securely injected from Kubernetes secrets
145+
- Data is persisted across pod restarts
146+
- Health checks monitor the container
147+
- The service is exposed via LoadBalancer
148+
149+
### GitHub Actions Workflow
150+
151+
Located at `/.github/workflows/deploy-azure.yml`:
152+
153+
- Defines the CI/CD pipeline that:
154+
- Sets up Python environment
155+
- Logs into Azure
156+
- Initializes and applies Terraform
157+
- Gets AKS credentials
158+
- Creates Kubernetes secrets
159+
- Deploys Memgraph to AKS
160+
- Verifies the deployment
161+
- Runs tests
162+
163+
### Setup Script
164+
165+
Located at `/scripts/setup_azure.sh`:
166+
167+
This bash script automates:
168+
- Azure CLI authentication
169+
- Service principal creation with appropriate permissions
170+
- Output of credentials for GitHub Actions
171+
172+
## Accessing Memgraph
173+
174+
After successful deployment:
175+
176+
1. Get the external IP of the Memgraph service:
177+
178+
```bash
179+
az aks get-credentials --resource-group GitHub --name aks-ai-agent-dev
180+
kubectl get service memgraph
181+
```
182+
183+
2. Note the EXTERNAL-IP from the output
184+
185+
3. Access Memgraph using these endpoints:
186+
- Bolt protocol: `EXTERNAL-IP:7687` (for direct database connections)
187+
- HTTP API: `EXTERNAL-IP:7444` (for REST API access)
188+
- MemGraph Lab UI: `EXTERNAL-IP:3000` (for visual database management)
189+
190+
4. Use the credentials (MEMGRAPH_USERNAME and MEMGRAPH_PASSWORD) to authenticate
191+
192+
## Troubleshooting
193+
194+
### Check pod status
195+
196+
```bash
197+
kubectl get pods
198+
kubectl describe pod <pod-name>
199+
kubectl logs <pod-name>
200+
```
201+
202+
### Check service status
203+
204+
```bash
205+
kubectl get services
206+
kubectl describe service memgraph
207+
```
208+
209+
### Common issues
210+
211+
1. **Persistent volumes not provisioning**:
212+
- Check storage class availability in your Azure region
213+
- Check the persistent volume claims status
214+
215+
2. **Memgraph not starting**:
216+
- Check logs for errors
217+
- Verify secrets were created correctly
218+
219+
3. **Cannot access external IP**:
220+
- Verify service type is LoadBalancer
221+
- Check if Azure has assigned an external IP
222+
- Verify network security groups allow traffic
223+
224+
## Cleanup
225+
226+
To remove the deployed resources:
227+
228+
1. Delete the Kubernetes resources:
229+
230+
```bash
231+
kubectl delete -f infra/k8s/memgraph.yaml
232+
```
233+
234+
2. Destroy the Terraform-managed infrastructure:
235+
236+
```bash
237+
cd infra/azure
238+
terraform destroy -var="environment=dev" \
239+
-var="memgraph_username=<username>" \
240+
-var="memgraph_password=<password>"
241+
```
242+
243+
3. Delete the service principal:
244+
245+
```bash
246+
az ad sp delete --id <client-id-from-service-principal>
247+
```
248+
249+
---
250+
251+
This deployment approach ensures:
252+
- Infrastructure as code with Terraform
253+
- Secure secret management with Azure Key Vault
254+
- Containerized deployment with Kubernetes
255+
- CI/CD automation with GitHub Actions
256+
- Persistent storage for your Memgraph data

0 commit comments

Comments
 (0)