|
| 1 | +# Airbox CLI |
| 2 | + |
| 3 | +Airbox is a command-line tool for managing Airbyte dataplanes on Kubernetes. |
| 4 | + |
| 5 | +## Installation |
| 6 | + |
| 7 | +Build from source: |
| 8 | +```bash |
| 9 | +go build -o airbox ./cmd/airbox |
| 10 | +``` |
| 11 | + |
| 12 | +## Quick Start |
| 13 | + |
| 14 | +```bash |
| 15 | +# 1. Configure airbox with your Airbyte instance |
| 16 | +airbox config init |
| 17 | + |
| 18 | +# 2. Authenticate with Airbyte |
| 19 | +airbox auth login |
| 20 | + |
| 21 | +# 3. Install a dataplane on your Kubernetes cluster |
| 22 | +airbox install dataplane |
| 23 | +``` |
| 24 | + |
| 25 | +## Configuration |
| 26 | + |
| 27 | +### Environment Variables |
| 28 | + |
| 29 | +- `AIRBOXCONFIG`: Override the default config file location (default: `~/.airbyte/airbox/config`) |
| 30 | +- `AIRBYTE_CLIENT_ID`: OAuth2 client ID for authentication |
| 31 | +- `AIRBYTE_CLIENT_SECRET`: OAuth2 client secret for authentication |
| 32 | + |
| 33 | +### Initialization |
| 34 | + |
| 35 | +Initialize your airbox configuration: |
| 36 | +```bash |
| 37 | +airbox config init |
| 38 | +``` |
| 39 | + |
| 40 | +This will prompt you to: |
| 41 | +- What Airbyte control plane are you connecting to (Cloud or Enterprise) |
| 42 | + |
| 43 | +For **Cloud**: Automatically configures cloud.airbyte.com URLs |
| 44 | +For **Enterprise**: You'll be prompted for: |
| 45 | +- Your Airbyte instance URL (e.g., https://airbyte.yourcompany.com) |
| 46 | +- API host is automatically set to your URL + "/api" |
| 47 | +- Edition is automatically set to "enterprise" |
| 48 | +- Context name defaults to "enterprise" |
| 49 | + |
| 50 | +Both Cloud and Enterprise use OAuth2 authentication only. |
| 51 | + |
| 52 | +## Authentication |
| 53 | + |
| 54 | +**OAuth2 Authentication (Both Cloud and Enterprise):** |
| 55 | + |
| 56 | +Set your OAuth2 client credentials as environment variables: |
| 57 | + |
| 58 | +```bash |
| 59 | +export AIRBYTE_CLIENT_ID="your-oauth-client-id" |
| 60 | +export AIRBYTE_CLIENT_SECRET="your-oauth-client-secret" |
| 61 | +``` |
| 62 | + |
| 63 | +Login to authenticate with Airbyte: |
| 64 | +```bash |
| 65 | +airbox auth login |
| 66 | +``` |
| 67 | + |
| 68 | +This will: |
| 69 | +1. Use OAuth2 client credentials from environment variables |
| 70 | +2. Authenticate via OAuth2 client credentials flow |
| 71 | +3. Save access tokens locally in YAML format |
| 72 | +4. Prompt you to select an organization context (if not already set) |
| 73 | + |
| 74 | +Logout and clear stored credentials: |
| 75 | +```bash |
| 76 | +airbox auth logout |
| 77 | +``` |
| 78 | + |
| 79 | +This removes your saved authentication tokens from local storage. |
| 80 | + |
| 81 | +Switch between Airbyte organization contexts: |
| 82 | +```bash |
| 83 | +airbox auth switch-organization |
| 84 | +``` |
| 85 | + |
| 86 | +## Dataplane Management |
| 87 | + |
| 88 | +### Install Dataplane |
| 89 | + |
| 90 | +Install a dataplane on your Kubernetes cluster: |
| 91 | +```bash |
| 92 | +airbox install dataplane |
| 93 | +``` |
| 94 | + |
| 95 | +This interactive command will: |
| 96 | +1. Prompt for organization and region selection |
| 97 | +2. Ask for a dataplane name |
| 98 | +3. Create a new Kind cluster (default) |
| 99 | +4. Register the dataplane with Airbyte Cloud/Enterprise |
| 100 | +5. Deploy the dataplane using Helm with the provided credentials |
| 101 | +6. Configure the dataplane to connect to your Airbyte control plane |
| 102 | + |
| 103 | +### Get Resources |
| 104 | + |
| 105 | +List dataplanes: |
| 106 | +```bash |
| 107 | +airbox get dataplane |
| 108 | +``` |
| 109 | + |
| 110 | +Get specific dataplane by ID: |
| 111 | +```bash |
| 112 | +airbox get dataplane <dataplane-id> |
| 113 | +``` |
| 114 | + |
| 115 | +Output in YAML format: |
| 116 | +```bash |
| 117 | +airbox get dataplane -o yaml |
| 118 | +``` |
| 119 | + |
| 120 | +### Delete Dataplane |
| 121 | + |
| 122 | +Remove a dataplane registration: |
| 123 | +```bash |
| 124 | +airbox delete dataplane <dataplane-id> |
| 125 | +``` |
| 126 | + |
| 127 | +Note: This only removes the registration from Airbyte. You must manually uninstall the Helm release from Kubernetes. |
| 128 | + |
| 129 | +## Global Options |
| 130 | + |
| 131 | +- `--help, -h`: Show help for any command |
| 132 | + |
| 133 | +## Examples |
| 134 | + |
| 135 | +Complete workflow: |
| 136 | +```bash |
| 137 | +# Set up authentication credentials |
| 138 | +export AIRBYTE_CLIENT_ID="your-client-id" |
| 139 | +export AIRBYTE_CLIENT_SECRET="your-client-secret" |
| 140 | + |
| 141 | +# Initialize configuration |
| 142 | +airbox config init |
| 143 | + |
| 144 | +# Authenticate |
| 145 | +airbox auth login |
| 146 | + |
| 147 | +# Install the dataplane (handles both registration and deployment) |
| 148 | +airbox install dataplane |
| 149 | + |
| 150 | +# Verify installation |
| 151 | +airbox get dataplane |
| 152 | +``` |
| 153 | + |
| 154 | +## Uninstalling a Dataplane |
| 155 | + |
| 156 | +### For Kind-based installations |
| 157 | + |
| 158 | +Kind clusters created by airbox follow the naming pattern: `airbox-<dataplane-name>` |
| 159 | + |
| 160 | +```bash |
| 161 | +# List all Kind clusters to find yours |
| 162 | +kind get clusters |
| 163 | + |
| 164 | +# Example output: |
| 165 | +# airbox-test-dp |
| 166 | +# airbox-dev-dp |
| 167 | +# airbox-demo-dp |
| 168 | + |
| 169 | +# Delete the dataplane registration from Airbyte |
| 170 | +airbox delete dataplane <dataplane-id> |
| 171 | + |
| 172 | +# Delete the Kind cluster (using the full cluster name with airbox- prefix) |
| 173 | +kind delete cluster --name airbox-test-dp |
| 174 | +``` |
| 175 | + |
| 176 | +### For existing Kubernetes clusters |
| 177 | + |
| 178 | +```bash |
| 179 | +# Delete the dataplane registration from Airbyte |
| 180 | +airbox delete dataplane <dataplane-id> |
| 181 | + |
| 182 | +# Uninstall the Helm release |
| 183 | +helm uninstall <dataplane-name> -n <namespace> |
| 184 | +``` |
| 185 | + |
| 186 | +## Running Syncs |
| 187 | + |
| 188 | +After installing a dataplane, create connections in the Airbyte web UI using the same workspace and region as your dataplane. When you trigger a sync, you'll see pods created in your Kubernetes cluster: |
| 189 | + |
| 190 | +```bash |
| 191 | +# Watch pods to see sync activity |
| 192 | +watch kubectl get po |
| 193 | + |
| 194 | +Every 2.0s: kubectl get po |
| 195 | + |
| 196 | +NAME READY STATUS RESTARTS AGE |
| 197 | +my-dataplane-airbyte-data-plane-567f98c6d9 1/1 Running 0 5m12s |
| 198 | +replication-job-15-attempt-0 0/3 Completed 0 2m18s |
| 199 | +source-postgres-discover-15-0-xyz123 0/2 Completed 0 2m45s |
| 200 | +``` |
| 201 | + |
| 202 | +The dataplane handles all job scheduling within your cluster while reporting status to the Airbyte control plane. |
| 203 | + |
| 204 | +## Configuration File |
| 205 | + |
| 206 | +Airbox stores configuration in `~/.airbyte/airbox/config` (YAML format). |
| 207 | + |
| 208 | +After running `airbox config init` and `airbox auth login`, your config file will look like this: |
| 209 | + |
| 210 | +```yaml |
| 211 | +current-context: cloud |
| 212 | +credentials: |
| 213 | + access_token: [REDACTED] |
| 214 | + token_type: Bearer |
| 215 | + expires_at: 2025-09-08T18:06:30.147961-07:00 |
| 216 | +contexts: |
| 217 | + - name: cloud |
| 218 | + context: |
| 219 | + airbyteApiHost: https://api.airbyte.com |
| 220 | + airbyteUrl: https://cloud.airbyte.com |
| 221 | + organizationId: [REDACTED] |
| 222 | + edition: cloud |
| 223 | + auth: |
| 224 | + clientId: [REDACTED] |
| 225 | + clientSecret: [REDACTED] |
| 226 | + type: oauth2 |
| 227 | +``` |
| 228 | +
|
| 229 | +The config file contains: |
| 230 | +- **current-context**: Active context name |
| 231 | +- **credentials**: OAuth2 access tokens and expiry (added after login) |
| 232 | +- **contexts**: Named configurations for different Airbyte instances |
| 233 | + - **airbyteApiHost**: API endpoint URL |
| 234 | + - **airbyteUrl**: Web UI URL |
| 235 | + - **organizationId**: Selected organization UUID (added after login) |
| 236 | + - **edition**: Deployment edition (cloud/enterprise/community) |
| 237 | + - **auth**: OAuth2 client credentials |
| 238 | +
|
| 239 | +## Requirements |
| 240 | +
|
| 241 | +- Go 1.21+ |
| 242 | +- Access to Airbyte Cloud or Enterprise |
| 243 | +- Kubernetes cluster (for dataplane installation) |
| 244 | +- Helm 3.x (for dataplane installation) |
| 245 | +- Kind (optional, for local development) |
0 commit comments