|
| 1 | +# AutoSpotting Migration Scripts |
| 2 | + |
| 3 | +This directory contains scripts for migrating and managing AutoSpotting data. |
| 4 | + |
| 5 | +## Company Memberships Backfill Script |
| 6 | + |
| 7 | +### Overview |
| 8 | + |
| 9 | +The `backfill_company_memberships.go` script migrates existing AutoSpotting deployments to the new multi-company SaaS model. It: |
| 10 | + |
| 11 | +1. Creates a default company for all existing users and accounts |
| 12 | +2. Creates company memberships for all existing users |
| 13 | +3. Sets the active company for each user |
| 14 | +4. Assigns all existing customer accounts to the default company |
| 15 | +5. Auto-approves accounts if the company is pre-approved |
| 16 | + |
| 17 | +### Prerequisites |
| 18 | + |
| 19 | +- Go 1.21 or later |
| 20 | +- AWS credentials configured (via environment variables, AWS CLI, or IAM role) |
| 21 | +- Proper IAM permissions to read/write DynamoDB tables |
| 22 | + |
| 23 | +### Required IAM Permissions |
| 24 | + |
| 25 | +```json |
| 26 | +{ |
| 27 | + "Version": "2012-10-17", |
| 28 | + "Statement": [ |
| 29 | + { |
| 30 | + "Effect": "Allow", |
| 31 | + "Action": [ |
| 32 | + "dynamodb:Scan", |
| 33 | + "dynamodb:GetItem", |
| 34 | + "dynamodb:PutItem", |
| 35 | + "dynamodb:UpdateItem" |
| 36 | + ], |
| 37 | + "Resource": [ |
| 38 | + "arn:aws:dynamodb:*:*:table/autospotting-Users", |
| 39 | + "arn:aws:dynamodb:*:*:table/autospotting-Companies", |
| 40 | + "arn:aws:dynamodb:*:*:table/autospotting-CompanyMemberships", |
| 41 | + "arn:aws:dynamodb:*:*:table/autospotting-CustomerAccounts" |
| 42 | + ] |
| 43 | + } |
| 44 | + ] |
| 45 | +} |
| 46 | +``` |
| 47 | + |
| 48 | +### Usage |
| 49 | + |
| 50 | +#### Environment Variables |
| 51 | + |
| 52 | +Set the following environment variables (or use command-line flags): |
| 53 | + |
| 54 | +```bash |
| 55 | +export USERS_TABLE="autospotting-Users" |
| 56 | +export COMPANIES_TABLE="autospotting-Companies" |
| 57 | +export COMPANY_MEMBERSHIPS_TABLE="autospotting-CompanyMemberships" |
| 58 | +export CUSTOMER_ACCOUNTS_TABLE="autospotting-CustomerAccounts" |
| 59 | +``` |
| 60 | + |
| 61 | +#### Running the Script |
| 62 | + |
| 63 | +**Dry Run (recommended first):** |
| 64 | + |
| 65 | +```bash |
| 66 | +cd scripts |
| 67 | +go run backfill_company_memberships.go \ |
| 68 | + --dry-run \ |
| 69 | + --company-name "AutoSpotting Legacy" \ |
| 70 | + --owner-email "admin@example.com" |
| 71 | +``` |
| 72 | + |
| 73 | +**Actual Execution:** |
| 74 | + |
| 75 | +```bash |
| 76 | +go run backfill_company_memberships.go \ |
| 77 | + --company-name "AutoSpotting Legacy" \ |
| 78 | + --owner-email "admin@example.com" |
| 79 | +``` |
| 80 | + |
| 81 | +#### Command-Line Flags |
| 82 | + |
| 83 | +- `--users-table`: DynamoDB Users table name (default: from `USERS_TABLE` env var) |
| 84 | +- `--companies-table`: DynamoDB Companies table name (default: from `COMPANIES_TABLE` env var) |
| 85 | +- `--memberships-table`: DynamoDB Company Memberships table name (default: from `COMPANY_MEMBERSHIPS_TABLE` env var) |
| 86 | +- `--accounts-table`: DynamoDB Customer Accounts table name (default: from `CUSTOMER_ACCOUNTS_TABLE` env var) |
| 87 | +- `--dry-run`: Run without making changes (default: `false`) |
| 88 | +- `--company-name`: Name for the default company (default: `"AutoSpotting Legacy"`) |
| 89 | +- `--owner-email`: Email of the company owner (default: first admin user found) |
| 90 | + |
| 91 | +### Example |
| 92 | + |
| 93 | +```bash |
| 94 | +# Dry run with custom company name |
| 95 | +go run backfill_company_memberships.go \ |
| 96 | + --users-table "autospotting-Users" \ |
| 97 | + --companies-table "autospotting-Companies" \ |
| 98 | + --memberships-table "autospotting-CompanyMemberships" \ |
| 99 | + --accounts-table "autospotting-CustomerAccounts" \ |
| 100 | + --dry-run \ |
| 101 | + --company-name "ACME Corporation" \ |
| 102 | + --owner-email "cto@acme.com" |
| 103 | + |
| 104 | +# Actual execution |
| 105 | +go run backfill_company_memberships.go \ |
| 106 | + --users-table "autospotting-Users" \ |
| 107 | + --companies-table "autospotting-Companies" \ |
| 108 | + --memberships-table "autospotting-CompanyMemberships" \ |
| 109 | + --accounts-table "autospotting-CustomerAccounts" \ |
| 110 | + --company-name "ACME Corporation" \ |
| 111 | + --owner-email "cto@acme.com" |
| 112 | +``` |
| 113 | + |
| 114 | +### What the Script Does |
| 115 | + |
| 116 | +1. **Scans Users Table**: Finds all existing users with PK and SK starting with `USER#` |
| 117 | + |
| 118 | +2. **Creates Default Company**: |
| 119 | + - Generates an 8-character random company ID |
| 120 | + - Sets the first admin user (or first user) as owner |
| 121 | + - Marks company as pre-approved (auto-approves new accounts) |
| 122 | + - Sets invoicing to disabled by default |
| 123 | + |
| 124 | +3. **Creates Memberships**: |
| 125 | + - Creates a company membership record for each user |
| 126 | + - Maps user roles: |
| 127 | + - `admin` → `admin` role in company |
| 128 | + - `operator` → `operator` role in company |
| 129 | + - `viewer` → `viewer` role in company |
| 130 | + - Uses DynamoDB patterns: |
| 131 | + - PK: `USER#<userId>` |
| 132 | + - SK: `COMPANY#<companyId>` |
| 133 | + - GSI1PK: `COMPANY#<companyId>` (for reverse lookup) |
| 134 | + - GSI1SK: `USER#<userId>` |
| 135 | + |
| 136 | +4. **Updates Users**: |
| 137 | + - Sets `active_company_id` field for each user |
| 138 | + - This ensures users see their accounts when they log in |
| 139 | + |
| 140 | +5. **Assigns Accounts**: |
| 141 | + - Scans all customer accounts |
| 142 | + - Updates each with `CompanyId` and `CompanyName` |
| 143 | + - Auto-approves pending accounts if company is pre-approved |
| 144 | + |
| 145 | +### Output Example |
| 146 | + |
| 147 | +``` |
| 148 | +Starting backfill process... |
| 149 | +Users table: autospotting-Users |
| 150 | +Companies table: autospotting-Companies |
| 151 | +Memberships table: autospotting-CompanyMemberships |
| 152 | +Accounts table: autospotting-CustomerAccounts |
| 153 | +Dry run: false |
| 154 | +
|
| 155 | +Found 5 existing users |
| 156 | +Created default company: AutoSpotting Legacy (ID: x7y2z9k4) |
| 157 | +Created 5 company memberships |
| 158 | +Updated 5 users with active company |
| 159 | +Found 12 total accounts |
| 160 | +Assigned 12 accounts to default company |
| 161 | +
|
| 162 | +Backfill completed successfully! |
| 163 | +Summary: |
| 164 | + - Created company: AutoSpotting Legacy |
| 165 | + - Created memberships: 5/5 |
| 166 | + - Updated users: 5/5 |
| 167 | + - Assigned accounts: 12 |
| 168 | +``` |
| 169 | + |
| 170 | +### Safety Features |
| 171 | + |
| 172 | +- **Dry Run Mode**: Test the migration without making changes |
| 173 | +- **Detailed Logging**: Shows all operations being performed |
| 174 | +- **Error Handling**: Continues processing even if individual operations fail |
| 175 | +- **Idempotent**: Can be run multiple times (will overwrite existing data) |
| 176 | + |
| 177 | +### Troubleshooting |
| 178 | + |
| 179 | +**Error: "Missing required table names"** |
| 180 | +- Ensure all table name environment variables or flags are set |
| 181 | + |
| 182 | +**Error: "scan failed: AccessDeniedException"** |
| 183 | +- Check IAM permissions for DynamoDB access |
| 184 | + |
| 185 | +**Warning: "Failed to create membership for user X"** |
| 186 | +- Check CloudWatch Logs for detailed error messages |
| 187 | +- Verify table schemas match expected structure |
| 188 | + |
| 189 | +**No users found** |
| 190 | +- Verify the Users table name is correct |
| 191 | +- Ensure users exist in the table with proper PK/SK format |
| 192 | + |
| 193 | +### Post-Migration Verification |
| 194 | + |
| 195 | +After running the backfill script, verify: |
| 196 | + |
| 197 | +1. **Company Created**: |
| 198 | + ```bash |
| 199 | + aws dynamodb get-item \ |
| 200 | + --table-name autospotting-Companies \ |
| 201 | + --key '{"CompanyId": {"S": "x7y2z9k4"}}' |
| 202 | + ``` |
| 203 | + |
| 204 | +2. **Memberships Created**: |
| 205 | + ```bash |
| 206 | + aws dynamodb query \ |
| 207 | + --table-name autospotting-CompanyMemberships \ |
| 208 | + --key-condition-expression "PK = :pk" \ |
| 209 | + --expression-attribute-values '{":pk": {"S": "USER#<userId>"}}' |
| 210 | + ``` |
| 211 | + |
| 212 | +3. **Accounts Assigned**: |
| 213 | + ```bash |
| 214 | + aws dynamodb scan \ |
| 215 | + --table-name autospotting-CustomerAccounts \ |
| 216 | + --filter-expression "attribute_exists(CompanyId)" |
| 217 | + ``` |
| 218 | + |
| 219 | +4. **Frontend Access**: |
| 220 | + - Log in to the dashboard |
| 221 | + - Verify company selector appears in header |
| 222 | + - Verify accounts are visible in dropdown |
| 223 | + - Check that all tabs load correctly |
| 224 | + |
| 225 | +### Rollback |
| 226 | + |
| 227 | +If you need to rollback the migration: |
| 228 | + |
| 229 | +1. **Remove Company**: |
| 230 | + ```bash |
| 231 | + aws dynamodb delete-item \ |
| 232 | + --table-name autospotting-Companies \ |
| 233 | + --key '{"CompanyId": {"S": "x7y2z9k4"}}' |
| 234 | + ``` |
| 235 | + |
| 236 | +2. **Remove Memberships**: |
| 237 | + ```bash |
| 238 | + # Query and delete all memberships for the company |
| 239 | + aws dynamodb scan \ |
| 240 | + --table-name autospotting-CompanyMemberships \ |
| 241 | + --filter-expression "CompanyId = :cid" \ |
| 242 | + --expression-attribute-values '{":cid": {"S": "x7y2z9k4"}}' |
| 243 | + ``` |
| 244 | + |
| 245 | +3. **Remove Company Data from Users**: |
| 246 | + ```bash |
| 247 | + # Update each user to remove active_company_id |
| 248 | + aws dynamodb update-item \ |
| 249 | + --table-name autospotting-Users \ |
| 250 | + --key '{"PK": {"S": "USER#<userId>"}, "SK": {"S": "USER#<userId>"}}' \ |
| 251 | + --update-expression "REMOVE active_company_id" |
| 252 | + ``` |
| 253 | + |
| 254 | +4. **Remove Company Data from Accounts**: |
| 255 | + ```bash |
| 256 | + # Update each account to remove CompanyId and CompanyName |
| 257 | + aws dynamodb update-item \ |
| 258 | + --table-name autospotting-CustomerAccounts \ |
| 259 | + --key '{"AccountId": {"S": "<accountId>"}}' \ |
| 260 | + --update-expression "REMOVE CompanyId, CompanyName" |
| 261 | + ``` |
| 262 | + |
| 263 | +### Support |
| 264 | + |
| 265 | +For issues or questions: |
| 266 | +- Check CloudWatch Logs for detailed error messages |
| 267 | +- Review the [Implementation Plan](../IMPLEMENTATION_PLAN.md) |
| 268 | +- Open an issue on GitHub |
| 269 | + |
| 270 | +## Building for Production |
| 271 | + |
| 272 | +To build a standalone binary: |
| 273 | + |
| 274 | +```bash |
| 275 | +cd scripts |
| 276 | +GOOS=linux GOARCH=amd64 go build -o backfill-linux-amd64 backfill_company_memberships.go |
| 277 | +``` |
| 278 | + |
| 279 | +This creates a portable executable that can be run on any Linux system without Go installed. |
0 commit comments