|
| 1 | +# Azure Infrastructure Setup for Graphable Production |
| 2 | + |
| 3 | +This guide covers setting up the Azure infrastructure required for Graphable to run in production. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +Graphable uses **Azure Key Vault** to securely store data source connection strings and secrets. The application needs authentication to access the Key Vault. |
| 8 | + |
| 9 | +## Prerequisites |
| 10 | + |
| 11 | +1. **Azure CLI** installed and logged in (`az login`) |
| 12 | +2. **Access to Flowcore Azure subscription** (subscription ID: `4f94ba05-20f1-47c1-9011-b3f11c31a014`) |
| 13 | +3. **jq** installed for JSON parsing (`brew install jq` on macOS) |
| 14 | +4. **Permissions** to create resources in the `Usable` resource group |
| 15 | + |
| 16 | +## Quick Setup (Production) |
| 17 | + |
| 18 | +Run the automated setup script: |
| 19 | + |
| 20 | +```bash |
| 21 | +cd scripts |
| 22 | +./setup-azure-production.sh |
| 23 | +``` |
| 24 | + |
| 25 | +This script will: |
| 26 | + |
| 27 | +1. ✅ Verify Azure CLI is installed and you're logged in |
| 28 | +2. ✅ Set the correct Azure subscription |
| 29 | +3. ✅ Check/create the resource group `Usable` |
| 30 | +4. ✅ Create Azure Key Vault `graphable-kv-49711d` (if it doesn't exist) |
| 31 | +5. ✅ Create a Service Principal `graphable-prod-sp` for authentication |
| 32 | +6. ✅ Grant the Service Principal `Key Vault Secrets Officer` role |
| 33 | +7. ✅ Output the configuration values you need for deployment |
| 34 | + |
| 35 | +## What Gets Created |
| 36 | + |
| 37 | +### 1. Azure Key Vault |
| 38 | + |
| 39 | +- **Name**: `graphable-kv-49711d` |
| 40 | +- **Resource Group**: `Usable` |
| 41 | +- **Location**: `westeurope` (or your resource group's location) |
| 42 | +- **RBAC Authorization**: Enabled (using Azure RBAC instead of access policies) |
| 43 | + |
| 44 | +### 2. Service Principal |
| 45 | + |
| 46 | +- **Name**: `graphable-prod-sp` |
| 47 | +- **Purpose**: Allows Kubernetes pods to authenticate to Azure Key Vault |
| 48 | +- **Role**: `Key Vault Secrets Officer` (can read and write secrets) |
| 49 | + |
| 50 | +## Deployment Configuration |
| 51 | + |
| 52 | +After running the setup script, you'll get output like this: |
| 53 | + |
| 54 | +```yaml |
| 55 | +AZURE_KEY_VAULT_URL: |
| 56 | + value: "https://graphable-kv-49711d.vault.azure.net/" |
| 57 | +AZURE_CLIENT_ID: |
| 58 | + value: "<app-id-from-script>" |
| 59 | +AZURE_CLIENT_SECRET: |
| 60 | + valueFrom: |
| 61 | + secretKeyRef: |
| 62 | + name: graphable-credentials |
| 63 | + key: azure-client-secret |
| 64 | +AZURE_TENANT_ID: |
| 65 | + value: "<tenant-id-from-script>" |
| 66 | +``` |
| 67 | +
|
| 68 | +### Creating the Kubernetes Secret |
| 69 | +
|
| 70 | +Before deploying, create the Kubernetes secret with the client secret: |
| 71 | +
|
| 72 | +```bash |
| 73 | +# The script will output the exact command with your values |
| 74 | +kubectl create secret generic graphable-credentials \ |
| 75 | + --from-literal=azure-client-secret='<password-from-script>' \ |
| 76 | + --namespace=<your-namespace> |
| 77 | +``` |
| 78 | + |
| 79 | +## Authentication Methods |
| 80 | + |
| 81 | +### Current Approach: Service Principal |
| 82 | + |
| 83 | +**What it is**: A service account with credentials (client ID + client secret) that the application uses to authenticate. |
| 84 | + |
| 85 | +**Pros**: |
| 86 | +- ✅ Simple to set up |
| 87 | +- ✅ Works immediately |
| 88 | +- ✅ No additional AKS configuration needed |
| 89 | + |
| 90 | +**Cons**: |
| 91 | +- ⚠️ Requires managing credentials (rotation, storage) |
| 92 | +- ⚠️ Secret stored in Kubernetes (though as a K8s secret) |
| 93 | + |
| 94 | +### Future Improvement: Azure AD Workload Identity (Recommended) |
| 95 | + |
| 96 | +**What it is**: Federation between Kubernetes Service Accounts and Azure Managed Identities - no credentials needed. |
| 97 | + |
| 98 | +**Pros**: |
| 99 | +- ✅ No credentials to manage |
| 100 | +- ✅ Azure handles identity automatically |
| 101 | +- ✅ Better security posture |
| 102 | +- ✅ Industry best practice for AKS |
| 103 | + |
| 104 | +**Cons**: |
| 105 | +- ⚠️ Requires AKS cluster configuration |
| 106 | +- ⚠️ More complex initial setup |
| 107 | + |
| 108 | +**To implement** (future enhancement): |
| 109 | +1. Enable OIDC issuer on your AKS cluster |
| 110 | +2. Create a User-Assigned Managed Identity |
| 111 | +3. Create federated credential linking Kubernetes SA to Managed Identity |
| 112 | +4. Update deployment to use the service account |
| 113 | +5. Remove AZURE_CLIENT_SECRET (no longer needed) |
| 114 | + |
| 115 | +Reference: [Azure AD Workload Identity Documentation](https://azure.github.io/azure-workload-identity/) |
| 116 | + |
| 117 | +## How Graphable Uses Key Vault |
| 118 | + |
| 119 | +1. **Data Source Secrets**: When users add a PostgreSQL data source with a connection string, the connection string is stored in Key Vault |
| 120 | +2. **Secret References**: Only a reference (vault URL + secret name) is stored in the control plane database |
| 121 | +3. **Runtime Access**: When executing queries, Graphable fetches the connection string from Key Vault using the reference |
| 122 | +4. **Caching**: Secrets are cached in memory for 5 minutes to reduce Azure API calls |
| 123 | + |
| 124 | +## Security Best Practices |
| 125 | + |
| 126 | +✅ **DO**: |
| 127 | +- Rotate service principal credentials regularly |
| 128 | +- Use Kubernetes secrets for sensitive values |
| 129 | +- Monitor Key Vault access logs |
| 130 | +- Restrict Key Vault network access if possible |
| 131 | +- Plan migration to Workload Identity |
| 132 | + |
| 133 | +❌ **DON'T**: |
| 134 | +- Commit credentials to git |
| 135 | +- Use the same credentials for dev and production |
| 136 | +- Share production credentials in chat/email |
| 137 | +- Store secrets in plain environment variables |
| 138 | + |
| 139 | +## Troubleshooting |
| 140 | + |
| 141 | +### "Key Vault not found" error at runtime |
| 142 | + |
| 143 | +Check that `AZURE_KEY_VAULT_URL` is set correctly in your deployment configuration. |
| 144 | + |
| 145 | +### "Authorization failed" when accessing secrets |
| 146 | + |
| 147 | +1. Verify the service principal has the `Key Vault Secrets Officer` role |
| 148 | +2. Check that the Kubernetes secret `graphable-credentials` exists |
| 149 | +3. Verify the client secret hasn't expired |
| 150 | + |
| 151 | +### "DefaultAzureCredential failed" error |
| 152 | + |
| 153 | +The application tries multiple authentication methods in order: |
| 154 | +1. Environment variables (client ID + secret) |
| 155 | +2. Managed Identity |
| 156 | +3. Azure CLI |
| 157 | + |
| 158 | +Ensure `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET`, and `AZURE_TENANT_ID` are set. |
| 159 | + |
| 160 | +## Verifying the Setup |
| 161 | + |
| 162 | +Test that everything works: |
| 163 | + |
| 164 | +```bash |
| 165 | +# Set environment variables from script output |
| 166 | +export AZURE_KEY_VAULT_URL="https://graphable-kv-49711d.vault.azure.net/" |
| 167 | +export AZURE_CLIENT_ID="<from-script>" |
| 168 | +export AZURE_CLIENT_SECRET="<from-script>" |
| 169 | +export AZURE_TENANT_ID="<from-script>" |
| 170 | + |
| 171 | +# Try creating a test secret |
| 172 | +az keyvault secret set \ |
| 173 | + --vault-name graphable-kv-49711d \ |
| 174 | + --name test-secret \ |
| 175 | + --value "test-value" |
| 176 | + |
| 177 | +# Try reading it back |
| 178 | +az keyvault secret show \ |
| 179 | + --vault-name graphable-kv-49711d \ |
| 180 | + --name test-secret \ |
| 181 | + --query value -o tsv |
| 182 | + |
| 183 | +# Clean up |
| 184 | +az keyvault secret delete \ |
| 185 | + --vault-name graphable-kv-49711d \ |
| 186 | + --name test-secret |
| 187 | +``` |
| 188 | + |
| 189 | +## Additional Resources |
| 190 | + |
| 191 | +- [Azure Key Vault Overview](https://docs.microsoft.com/en-us/azure/key-vault/) |
| 192 | +- [Azure RBAC for Key Vault](https://docs.microsoft.com/en-us/azure/key-vault/general/rbac-guide) |
| 193 | +- [Flowcore Graphable PRD](https://usable.dev/dashboard/workspaces/60c10ca2-4115-4c1a-b6d7-04ac39fd3938/fragments/29e3f893-fe75-4497-b312-26df31102e5d) |
0 commit comments