Skip to content
Closed
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions hack/aks/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,10 @@ nodesubnet-byocni-nokubeproxy-up: rg-up ipv4 overlay-net-up ## Brings up an Node
$(LTS_ARGS) \
--yes
@$(MAKE) set-kubeconf
@$(MAKE) nodesubnet-ipconfigupdate

nodesubnet-ipconfigupdate: ## Run ipconfigupdate for NodeSubnet clusters
RESOURCE_GROUP=MC_$(GROUP)_$(CLUSTER)_$(REGION) go run test/integration/cilium-nodesubnet/ipconfigupdate.go

overlay-byocni-up: rg-up ipv4 overlay-net-up ## Brings up an Overlay BYO CNI cluster
$(AZCLI) aks create -n $(CLUSTER) -g $(GROUP) -l $(REGION) \
Expand Down
223 changes: 223 additions & 0 deletions hack/aks/README-byocni.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,223 @@
# BYO CNI Cluster Setup Script

The `create-byocni-cluster.sh` script automates the creation of a BYO (Bring Your Own) CNI cluster on Azure Kubernetes Service (AKS). It orchestrates the following steps:

1. **Cluster Creation**: Creates an AKS cluster with configurable networking modes (overlay, swift, nodesubnet, dualstack-overlay, vnetscale-swift)
2. **CNS Deployment**: Deploys Azure Container Networking Service (CNS) to the cluster using the `test-load` make command
3. **CNI Installation**: Installs the specified CNI networking components (Cilium by default, Azure CNI Manager, or none)

## Prerequisites

Before running the script, ensure you have:

- Azure CLI installed and logged in (`az login`)
- kubectl installed and configured
- make utility installed
- gettext package installed (for `envsubst` command)
- Proper Azure subscription permissions to create AKS clusters

## Usage

### Basic Usage

Create a cluster with default settings (overlay networking with Cilium):

**Note**: The script must be run from the root directory of the azure-container-networking repository.

```bash
cd /path/to/azure-container-networking
./hack/aks/create-byocni-cluster.sh --subscription YOUR_SUBSCRIPTION_ID
```

### Advanced Usage

Create a cluster with custom configuration:
```bash
cd /path/to/azure-container-networking
./hack/aks/create-byocni-cluster.sh \
--cluster my-cluster \
--subscription YOUR_SUBSCRIPTION_ID \
--networking-mode swift \
--vm-size Standard_D2s_v3 \
--cni-plugin cilium \
--cns-version v1.6.0 \
--cilium-dir 1.16 \
--cilium-version-tag v1.16.5
```

### Dry Run

Preview the commands that would be executed without actually running them:
```bash
cd /path/to/azure-container-networking
./hack/aks/create-byocni-cluster.sh --subscription YOUR_SUBSCRIPTION_ID --dry-run
```

## Configuration Options

All parameters have sensible defaults as specified below. Only the `--subscription` parameter is required.

| Parameter | Description | Default |
|-----------|-------------|---------|
| `--cluster` | Name of the AKS cluster | `byocni-cluster` |
| `--subscription` | Azure subscription ID | *Required* |
| `--resource-group` | Resource group name | Same as cluster name |
| `--azcli` | Azure CLI command | `az` |
| `--kubernetes-version` | Kubernetes version for the cluster | `1.33` |
| `--vm-size` | Azure VM size for cluster nodes | `Standard_B2s` |
| `--networking-mode` | Networking mode (overlay, swift, nodesubnet, dualstack-overlay, vnetscale-swift) | `overlay` |
| `--cni-plugin` | CNI plugin (cilium, azure-cni, none) | `cilium` |
| `--cns-version` | CNS version to deploy | `v1.5.38` |
| `--azure-ipam-version` | Azure IPAM version | `v0.3.0` |
| `--cilium-dir` | Cilium version directory | `1.17` |
| `--cilium-registry` | Cilium image registry | `acnpublic.azurecr.io` |
| `--cilium-version-tag` | Cilium version tag | Auto-detected |
| `--ipv6-hp-bpf-version` | IPv6 HP BPF version | Auto-detected |
| `--cns-image-repo` | CNS image repository | `MCR` |
| `--dry-run` | Show commands without executing | `false` |

## Networking Modes

- **overlay**: Standard overlay networking mode
- **swift**: SWIFT networking mode
- **nodesubnet**: NodeSubnet networking mode
- **dualstack-overlay**: Dualstack overlay networking mode
- **vnetscale-swift**: VNet Scale SWIFT networking mode

## CNI Plugins

- **cilium**: Deploy Cilium CNI with configurable versions (default, automatically disables kube-proxy)
- **azure-cni**: Deploy Azure CNI Manager (automatically enables kube-proxy)
- **none**: Deploy only cluster and CNS, no CNI plugin (automatically disables kube-proxy)

## Supported Cilium Versions

The script supports the following Cilium versions based on available manifests (when using --cni-plugin cilium):
- v1.12
- v1.13
- v1.14
- v1.16
- v1.17 (default)

## Examples

### Example 1: Basic cluster creation with Cilium (default)
```bash
cd /path/to/azure-container-networking
./hack/aks/create-byocni-cluster.sh --subscription 9b8218f9-902a-4d20-a65c-e98acec5362f
```

### Example 2: Swift networking with Azure CNI Manager
```bash
cd /path/to/azure-container-networking
./hack/aks/create-byocni-cluster.sh \
--subscription 9b8218f9-902a-4d20-a65c-e98acec5362f \
--networking-mode swift \
--cni-plugin azure-cni
```

### Example 3: Custom cluster with specific CNS version and Cilium
```bash
cd /path/to/azure-container-networking
./hack/aks/create-byocni-cluster.sh \
--cluster production-cluster \
--subscription 9b8218f9-902a-4d20-a65c-e98acec5362f \
--networking-mode overlay \
--cns-version v1.6.0 \
--azure-ipam-version v0.4.0 \
--cni-plugin cilium \
--cilium-dir 1.16
```

### Example 4: Custom cluster and resource group
```bash
cd /path/to/azure-container-networking
./hack/aks/create-byocni-cluster.sh \
--cluster my-aks-cluster \
--resource-group my-resource-group \
--subscription 9b8218f9-902a-4d20-a65c-e98acec5362f
```

### Example 5: Dualstack cluster with Cilium
```bash
cd /path/to/azure-container-networking
./hack/aks/create-byocni-cluster.sh \
--subscription 9b8218f9-902a-4d20-a65c-e98acec5362f \
--networking-mode dualstack-overlay \
--cilium-dir 1.17 \
--cilium-version-tag v1.17.0
```

### Example 6: Cluster with specific Kubernetes version
```bash
cd /path/to/azure-container-networking
./hack/aks/create-byocni-cluster.sh \
--subscription 9b8218f9-902a-4d20-a65c-e98acec5362f \
--kubernetes-version 1.30 \
--networking-mode overlay \
--cni-plugin cilium
```

### Example 8: Only cluster and CNS, no CNI plugin
```bash
cd /path/to/azure-container-networking
./hack/aks/create-byocni-cluster.sh \
--subscription 9b8218f9-902a-4d20-a65c-e98acec5362f \
--cni-plugin none
```

### Example 9: Using different image registry
```bash
cd /path/to/azure-container-networking
./hack/aks/create-byocni-cluster.sh \
--subscription 9b8218f9-902a-4d20-a65c-e98acec5362f \
--cilium-registry mcr.microsoft.com/containernetworking \
--cilium-version-tag v1.14.8
```

## Post-Installation

After successful cluster creation, you can:

1. Get the kubeconfig:
```bash
az aks get-credentials --resource-group CLUSTER_NAME --name CLUSTER_NAME
```

2. Verify the installation:
```bash
kubectl get pods -n kube-system
kubectl get nodes
```

3. Check Cilium status:
```bash
kubectl get pods -n kube-system | grep cilium
```

## Troubleshooting

### Common Issues

1. **Azure CLI not logged in**: Run `az login` before executing the script
2. **Missing permissions**: Ensure your Azure account has permissions to create AKS clusters
3. **Invalid Cilium version**: Use `--dry-run` to test configuration before running
4. **kubectl not found**: Install kubectl and ensure it's in your PATH

### Debugging

Use the `--dry-run` flag to see exactly what commands would be executed:
```bash
cd /path/to/azure-container-networking
./hack/aks/create-byocni-cluster.sh --subscription YOUR_SUBSCRIPTION_ID --dry-run
```

Check the logs for detailed information about each step of the process.

## Contributing

When modifying the script, please:
1. Test with `--dry-run` first
2. Ensure all error handling works correctly
3. Update this documentation for any new features
4. Test with different Cilium versions to ensure compatibility
Loading