|
| 1 | +# CLAUDE.md |
| 2 | + |
| 3 | +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +Jetlag is an OpenShift cluster deployment tool that uses Ansible automation to deploy Multi Node OpenShift (MNO) and Single Node OpenShift (SNO) clusters via the Assisted Installer. It supports Red Hat performance labs, Scale Labs, and IBMcloud environments. |
| 8 | + |
| 9 | +## Essential Commands |
| 10 | + |
| 11 | +### Environment Setup |
| 12 | +```bash |
| 13 | +# Bootstrap ansible virtual environment (run from repo root) |
| 14 | +source bootstrap.sh |
| 15 | + |
| 16 | +# Red Hat Labs (Scale Lab/Performance Lab) |
| 17 | +# Copy and edit configuration file |
| 18 | +cp ansible/vars/all.sample.yml ansible/vars/all.yml |
| 19 | +# Edit all.yml with your lab configuration (lab, lab_cloud, cluster_type, etc.) |
| 20 | + |
| 21 | +# Create inventory file for your lab environment |
| 22 | +ansible-playbook ansible/create-inventory.yml |
| 23 | +# Setup bastion host (replace cloud99.local with your inventory file) |
| 24 | +ansible-playbook -i ansible/inventory/cloud99.local ansible/setup-bastion.yml |
| 25 | + |
| 26 | +# IBMcloud |
| 27 | +# Copy and edit configuration file |
| 28 | +cp ansible/vars/ibmcloud.sample.yml ansible/vars/ibmcloud.yml |
| 29 | +# Edit ibmcloud.yml with your IBMcloud configuration (cluster_type, worker_node_count, etc.) |
| 30 | + |
| 31 | +# Create inventory file from IBMcloud CLI data |
| 32 | +ansible-playbook ansible/ibmcloud-create-inventory.yml |
| 33 | +# Setup bastion host for IBMcloud |
| 34 | +ansible-playbook -i ansible/inventory/ibmcloud.local ansible/ibmcloud-setup-bastion.yml |
| 35 | +``` |
| 36 | + |
| 37 | +### Cluster Deployment |
| 38 | +```bash |
| 39 | +# Red Hat Labs (Scale Lab/Performance Lab) |
| 40 | +# Deploy Multi Node OpenShift cluster |
| 41 | +ansible-playbook -i ansible/inventory/cloud99.local ansible/mno-deploy.yml |
| 42 | + |
| 43 | +# Deploy Single Node OpenShift clusters |
| 44 | +ansible-playbook -i ansible/inventory/cloud99.local ansible/sno-deploy.yml |
| 45 | + |
| 46 | +# Deploy Virtual Multi Node OpenShift (VMNO) - requires hypervisor setup first |
| 47 | +ansible-playbook -i ansible/inventory/cloud99.local ansible/hv-setup.yml |
| 48 | +ansible-playbook -i ansible/inventory/cloud99.local ansible/hv-vm-create.yml |
| 49 | +ansible-playbook -i ansible/inventory/cloud99.local ansible/mno-deploy.yml |
| 50 | + |
| 51 | +# IBMcloud |
| 52 | +# Deploy Multi Node OpenShift on IBMcloud |
| 53 | +ansible-playbook -i ansible/inventory/ibmcloud.local ansible/ibmcloud-mno-deploy.yml |
| 54 | + |
| 55 | +# Deploy Single Node OpenShift on IBMcloud |
| 56 | +ansible-playbook -i ansible/inventory/ibmcloud.local ansible/ibmcloud-sno-deploy.yml |
| 57 | +``` |
| 58 | + |
| 59 | +### Cluster Management |
| 60 | +```bash |
| 61 | +# Scale out MNO cluster |
| 62 | +ansible-playbook ansible/ocp-scale-out.yml |
| 63 | + |
| 64 | +# Setup hypervisor nodes for VMs |
| 65 | +ansible-playbook ansible/hv-setup.yml |
| 66 | + |
| 67 | +# Create VMs on hypervisor nodes |
| 68 | +ansible-playbook ansible/hv-vm-create.yml |
| 69 | + |
| 70 | +# Delete VMs from hypervisor nodes |
| 71 | +ansible-playbook ansible/hv-vm-delete.yml |
| 72 | + |
| 73 | +# Replace VMs on hypervisor nodes (delete + recreate) |
| 74 | +ansible-playbook ansible/hv-vm-replace.yml |
| 75 | + |
| 76 | +# Sync OpenShift releases |
| 77 | +ansible-playbook ansible/sync-ocp-release.yml |
| 78 | +``` |
| 79 | + |
| 80 | +## Project Architecture |
| 81 | + |
| 82 | +### Key Configuration Files |
| 83 | +- `ansible/vars/all.yml` - Main configuration for Red Hat labs (copy from `ansible/vars/all.sample.yml`) |
| 84 | +- `ansible/vars/ibmcloud.yml` - IBMcloud-specific configuration (copy from `ansible/vars/ibmcloud.sample.yml`) |
| 85 | +- `pull-secret.txt` - OpenShift pull secret (place in repo root) |
| 86 | +- `ansible/inventory/$CLOUDNAME.local` - Generated inventory file for your lab |
| 87 | + |
| 88 | +### Critical Variables |
| 89 | +- `lab`: Environment type (`performancelab`, `scalelab`, or `ibmcloud`) |
| 90 | +- `lab_cloud`: Specific cloud allocation (e.g., `cloud42`) |
| 91 | +- `cluster_type`: Either `mno`, `sno`, or `vmno` |
| 92 | +- `worker_node_count`: Number of bare metal worker nodes for MNO clusters |
| 93 | +- `hybrid_worker_count`: Number of virtual worker nodes for hybrid MNO clusters (requires hypervisor setup) |
| 94 | +- `ocp_build`: OpenShift build type (`ga`, `dev`, or `ci`) |
| 95 | +- `ocp_version`: OpenShift version (e.g., `latest-4.20`) |
| 96 | + |
| 97 | +### Ansible Role Structure |
| 98 | +Jetlag uses a modular Ansible role architecture: |
| 99 | + |
| 100 | +- **Bastion roles**: `bastion-*` roles configure the bastion host with services like Assisted Installer, DNS, registry |
| 101 | +- **Installation roles**: `install-cluster`, `sno-post-cluster-install` handle cluster deployment |
| 102 | +- **Hypervisor roles**: `hv-*` roles manage VM infrastructure on hypervisor nodes |
| 103 | +- **Utility roles**: `boot-iso`, `sync-*` roles provide supporting functionality |
| 104 | + |
| 105 | +### Cluster Types |
| 106 | +- **MNO (Multi Node OpenShift)**: 3 control-plane nodes + configurable bare metal worker nodes |
| 107 | +- **SNO (Single Node OpenShift)**: Single node clusters, one per available machine |
| 108 | +- **VMNO (Virtual Multi Node OpenShift)**: MNO cluster using VMs instead of bare metal (Jetlag-specific term) |
| 109 | +- **Hybrid MNO**: MNO cluster with both bare metal and virtual worker nodes |
| 110 | + |
| 111 | +#### Virtual and Hybrid Cluster Details |
| 112 | +- **VMNO clusters** allow multi-node deployment with fewer physical machines (minimum: 1 bastion + 1-2 hypervisors) |
| 113 | +- **Hybrid clusters** combine bare metal workers (`worker_node_count`) with virtual workers (`hybrid_worker_count`) |
| 114 | +- **Hypervisor nodes**: Unused machines become VM hosts for additional clusters or hybrid workers |
| 115 | +- Virtual workers are created as VMs on hypervisor nodes and added to the worker inventory section |
| 116 | +- VM placement distributed across available hypervisors based on hardware-specific VM count configurations |
| 117 | + |
| 118 | +### Lab Environment Support |
| 119 | +- **Performance Lab**: Dell r750, 740xd hardware |
| 120 | +- **Scale Lab**: Various Dell models (r750, r660, r650, r640, r630, fc640), Supermicro systems |
| 121 | +- **IBMcloud**: Supermicro E5-2620, Lenovo SR630 bare metal |
| 122 | + |
| 123 | +## Development Workflow |
| 124 | + |
| 125 | +### Standard MNO/SNO Deployment (Red Hat Labs) |
| 126 | +1. Edit `ansible/vars/all.yml` with your lab configuration |
| 127 | +2. Run `ansible-playbook ansible/create-inventory.yml` to generate inventory |
| 128 | +3. Run `ansible-playbook -i ansible/inventory/cloud99.local ansible/setup-bastion.yml` to configure bastion host |
| 129 | +4. Run deployment playbook (`ansible/mno-deploy.yml` or `ansible/sno-deploy.yml`) |
| 130 | +5. Access clusters using kubeconfig files in `/root/mno/` or `/root/sno/` |
| 131 | + |
| 132 | +### IBMcloud MNO/SNO Deployment |
| 133 | +1. Edit `ansible/vars/ibmcloud.yml` with your IBMcloud configuration |
| 134 | +2. Run `ansible-playbook ansible/ibmcloud-create-inventory.yml` to generate `ansible/inventory/ibmcloud.local` from IBMcloud CLI data |
| 135 | +3. Run `ansible-playbook -i ansible/inventory/ibmcloud.local ansible/ibmcloud-setup-bastion.yml` to configure bastion host |
| 136 | +4. Run deployment playbook (`ansible-playbook -i ansible/inventory/ibmcloud.local ansible/ibmcloud-mno-deploy.yml` or `ansible/ibmcloud-sno-deploy.yml`) |
| 137 | +5. Access clusters using kubeconfig files in `/root/mno/` or `/root/sno/` |
| 138 | + |
| 139 | +### VMNO Deployment (Red Hat Labs Only) |
| 140 | +1. Edit `ansible/vars/all.yml` with `cluster_type: vmno` and VM-specific settings |
| 141 | +2. Edit `ansible/vars/hv.yml` for hypervisor configuration |
| 142 | +3. Run `ansible-playbook ansible/create-inventory.yml` to generate inventory with VM entries |
| 143 | +4. Run `ansible-playbook -i ansible/inventory/cloud99.local ansible/setup-bastion.yml` to configure bastion host |
| 144 | +5. Run `ansible-playbook -i ansible/inventory/cloud99.local ansible/hv-setup.yml` to configure hypervisor nodes |
| 145 | +6. Run `ansible-playbook -i ansible/inventory/cloud99.local ansible/hv-vm-create.yml` to create VMs |
| 146 | +7. Run `ansible-playbook -i ansible/inventory/cloud99.local ansible/mno-deploy.yml` to deploy cluster to VMs |
| 147 | +8. Access cluster using kubeconfig in `/root/vmno/` |
| 148 | + |
| 149 | +### Hybrid Cluster Deployment (Red Hat Labs Only) |
| 150 | +1. Configure both `worker_node_count` (bare metal) and `hybrid_worker_count` (VMs) in `ansible/vars/all.yml` |
| 151 | +2. Ensure hypervisor nodes are available in allocation |
| 152 | +3. Follow standard Red Hat Labs MNO workflow - hybrid workers automatically added to inventory |
| 153 | + |
| 154 | +## Special Considerations |
| 155 | + |
| 156 | +- Inventory files are generated, not manually created (except for "Bring Your Own Lab" scenarios) |
| 157 | +- Bastion machine is always the first machine in allocation and hosts Assisted Installer |
| 158 | +- Unused machines in MNO deployments become hypervisor nodes |
| 159 | +- SNO deployments create one cluster per available machine after bastion |
| 160 | +- Public VLAN support available for routable environments (`public_vlan: true`) |
| 161 | +- Disconnected/air-gapped deployments supported with registry mirroring |
| 162 | + |
| 163 | +### Virtual and Hybrid Cluster Considerations |
| 164 | +- **Hardware Requirements**: VMNO requires additional CPU/memory capacity for VM overhead |
| 165 | +- **VM Management**: Use `hv-vm-delete.yml` or `hv-vm-replace.yml` between VMNO deployments to avoid conflicts |
| 166 | +- **Resource Planning**: Configure `hw_vm_counts` per hardware type to optimize VM distribution across hypervisors |
| 167 | +- **Disk Configuration**: VMs can span multiple disks on hypervisors (e.g., default disk + nvme for higher VM counts) |
| 168 | +- **Network Configuration**: VMs use libvirt networking with static IP assignment from controlplane network range |
| 169 | +- **Scale Lab/Performance Lab Only**: VMNO and hybrid deployments only supported in Scale Lab and Performance Lab environments |
| 170 | + |
| 171 | +## Troubleshooting and Tips |
| 172 | + |
| 173 | +When encountering issues with Jetlag deployments, consult these comprehensive documentation resources: |
| 174 | + |
| 175 | +### Primary Troubleshooting Resources |
| 176 | +- **[docs/troubleshooting.md](docs/troubleshooting.md)**: Comprehensive troubleshooting guide covering: |
| 177 | + - Common deployment issues and solutions |
| 178 | + - Hardware-specific problems (Dell, Supermicro) |
| 179 | + - Bastion configuration and recovery procedures |
| 180 | + - BMC/iDRAC reset procedures |
| 181 | + - Virtual media and discovery issues |
| 182 | + |
| 183 | +- **[docs/tips-and-vars.md](docs/tips-and-vars.md)**: Advanced configuration guidance including: |
| 184 | + - Network interface configuration and overrides |
| 185 | + - Install disk configuration options |
| 186 | + - OCP version management |
| 187 | + - NVMe disk configuration for install and etcd |
| 188 | + - Post-deployment tasks and optimizations |
| 189 | + - Bastion registry management |
| 190 | + |
| 191 | +### Common Issues to Check First |
| 192 | +1. **Network Configuration**: Verify `bastion_lab_interface` and `bastion_controlplane_interface` match your hardware |
| 193 | +2. **BMC Access**: Ensure BMC credentials and network connectivity are correct |
| 194 | +3. **DNS Services**: Check bastion DNS services are running and configured correctly |
| 195 | +4. **Disk Selection**: Verify install disk paths and available storage |
| 196 | +5. **Resource Limits**: Ensure sufficient CPU/memory for VM deployments (VMNO/hybrid) |
| 197 | + |
| 198 | +### When to Consult Documentation |
| 199 | +- Before troubleshooting deployment failures, read the relevant sections in `troubleshooting.md` |
| 200 | +- For advanced configuration needs, reference the specific sections in `tips-and-vars.md` |
| 201 | +- When working with specific hardware vendors, check the hardware-specific troubleshooting sections |
0 commit comments