|
| 1 | +# Local Server Sync Plan |
| 2 | + |
| 3 | +This plan adds a CLI workflow for pulling database and uploads from a Vantage application into the local Altis codebase where the command is run. |
| 4 | + |
| 5 | +## Goal |
| 6 | + |
| 7 | +Provide a safe, repeatable local sync command: |
| 8 | + |
| 9 | +```sh |
| 10 | +altis-cli app sync all <app> |
| 11 | +altis-cli app sync database <app> |
| 12 | +altis-cli app sync uploads <app> |
| 13 | +``` |
| 14 | + |
| 15 | +This command is expected to be run from a local Altis project that uses `altis/local-server`. This is different from Vantage's app-to-app import endpoints. Local sync should use Vantage exports/backups, download the archive locally, and restore it into the current local-server project. |
| 16 | + |
| 17 | +## Adjacent Repo Changes |
| 18 | + |
| 19 | +### `../hm/vantage-backend` |
| 20 | + |
| 21 | +No required changes for the first implementation. |
| 22 | + |
| 23 | +Existing API support is enough: |
| 24 | + |
| 25 | +- `POST /stack/applications/{stack}/backups` |
| 26 | +- `GET /stack/applications/{stack}/backups` |
| 27 | +- Existing stream-log support through `/stream-log` |
| 28 | +- Existing search-replace configuration convention in `extra.altis.cloud.search-replace` |
| 29 | + |
| 30 | +The backup archive already contains: |
| 31 | + |
| 32 | +```txt |
| 33 | +database.sql.gz |
| 34 | +uploads/ |
| 35 | +``` |
| 36 | + |
| 37 | +Optional future improvement: |
| 38 | + |
| 39 | +- Return the generated backup object ID or download URL from the backup task/log metadata after completion. This would remove the need for the CLI to list backups and select the newest matching backup after a streamed export completes. |
| 40 | +- Add a small endpoint that exposes the resolved search-replace mappings for an application/build using the same logic as Vantage's app-to-app import flow. This is not required for the first CLI implementation if the CLI reads mappings from the local project's `composer.json`, but it would improve parity with the backend's exact deployed-build resolution. |
| 41 | + |
| 42 | +### `../hm/altis-local-server` |
| 43 | + |
| 44 | +No required changes for the first implementation. |
| 45 | + |
| 46 | +Existing Local Server commands are enough: |
| 47 | + |
| 48 | +- `composer server cli -- db import <file>` |
| 49 | +- `composer server s3 import-uploads` |
| 50 | +- `composer server import-uploads` as an alias in current local-server |
| 51 | +- `composer server status` for a lightweight readiness check |
| 52 | + |
| 53 | +Optional future improvement: |
| 54 | + |
| 55 | +- Add a dedicated `composer server sync-from-archive <archive>` command that imports DB and uploads from a Vantage backup archive in one local-server-native step. This is not needed for the CLI feature because the CLI can orchestrate existing commands. |
| 56 | + |
| 57 | +## New CLI Command Family |
| 58 | + |
| 59 | +Use a nested `app sync` command family. |
| 60 | + |
| 61 | +Commands: |
| 62 | + |
| 63 | +```sh |
| 64 | +altis-cli app sync all <app> |
| 65 | +altis-cli app sync database <app> |
| 66 | +altis-cli app sync uploads <app> |
| 67 | +``` |
| 68 | + |
| 69 | +The legacy `stack sync` alias should work too, but docs should prefer `app sync`. |
| 70 | + |
| 71 | +Options: |
| 72 | + |
| 73 | +- `--path <path>`: optional local Altis project path override. Defaults to the current working directory. |
| 74 | +- `--tables <table1,table2>`: database-only table selection. |
| 75 | +- `--uploads-path <prefix>`: uploads prefix to export from remote app. |
| 76 | +- `--search-replace-key <key>`: read mappings from `extra.altis.cloud.search-replace.<key>` in the local project's `composer.json`. Defaults to `local-server`. |
| 77 | +- `--replace <from=to>`: explicit search-replace mapping. Repeatable. |
| 78 | +- `--skip-search-replace`: skip search-replace entirely. |
| 79 | +- `--skip-post-sync`: skip the `wp altis post-sync` hook after database import. |
| 80 | +- `--dry-run-search-replace`: print the resolved mappings that would be applied, without triggering a backup or import. |
| 81 | +- `--output-dir <path>`: working directory for downloaded archives and extracted files. Defaults to a temp directory. |
| 82 | +- `--keep-archive`: keep downloaded `.tar` and extracted files after restore. |
| 83 | +- `--yes`: skip destructive confirmations. |
| 84 | +- `--resume <log-id>`: resume watching an already-started remote backup task. |
| 85 | +- `--debug`: pass through to stream-log rendering. |
| 86 | +- `--json`: print machine-readable summary at the end. |
| 87 | + |
| 88 | +## Workflow |
| 89 | + |
| 90 | +### 1. Validate Local Project |
| 91 | + |
| 92 | +Before calling Vantage: |
| 93 | + |
| 94 | +- Resolve the local project path from `--path` or `process.cwd()`. |
| 95 | +- Check `<local-project>/composer.json` exists. |
| 96 | +- Check the Composer project requires `altis/local-server` in `require` or `require-dev`. |
| 97 | +- Check `<local-project>/content` exists. |
| 98 | +- Check `composer` is available. |
| 99 | +- Run `composer server status` in the local project path. If it fails, show a clear message asking the user to start local-server with `composer server start`. |
| 100 | + |
| 101 | +Do not start local-server automatically in the first implementation. The expected workflow is that the user runs this command from the intended local Altis project after local-server is already set up and running. |
| 102 | + |
| 103 | +### 2. Confirm Destructive Actions |
| 104 | + |
| 105 | +Prompt unless `--yes` is passed. |
| 106 | + |
| 107 | +For database: |
| 108 | + |
| 109 | +```txt |
| 110 | +This will replace the local database for the current Altis project with data from <app>. Continue? |
| 111 | +``` |
| 112 | + |
| 113 | +For uploads: |
| 114 | + |
| 115 | +```txt |
| 116 | +This will merge remote uploads from <app> into ./content/uploads. Existing files may be overwritten. Continue? |
| 117 | +``` |
| 118 | + |
| 119 | +For `all`, show both warnings in one confirmation. |
| 120 | + |
| 121 | +### 3. Create Remote Export |
| 122 | + |
| 123 | +Call the existing backup endpoint with `stream=true`. |
| 124 | + |
| 125 | +For database: |
| 126 | + |
| 127 | +```http |
| 128 | +POST /stack/applications/{stack}/backups?stream=true |
| 129 | +``` |
| 130 | + |
| 131 | +Body/query params: |
| 132 | + |
| 133 | +```json |
| 134 | +{ |
| 135 | + "database": true, |
| 136 | + "uploads": false, |
| 137 | + "tables": ["wp_posts", "wp_options"] |
| 138 | +} |
| 139 | +``` |
| 140 | + |
| 141 | +For uploads: |
| 142 | + |
| 143 | +```json |
| 144 | +{ |
| 145 | + "database": false, |
| 146 | + "uploads": true, |
| 147 | + "uploads_path": "sites/2" |
| 148 | +} |
| 149 | +``` |
| 150 | + |
| 151 | +For all: |
| 152 | + |
| 153 | +```json |
| 154 | +{ |
| 155 | + "database": true, |
| 156 | + "uploads": true |
| 157 | +} |
| 158 | +``` |
| 159 | + |
| 160 | +The streamed backup endpoint returns a plain text log ID. Use the existing `streamLog(v, stack, logId, debug)` pattern to show progress. |
| 161 | + |
| 162 | +Implementation note: the current `streamLog()` helper renders progress but does not expose a promise that resolves on completion. `app sync` needs to know when the backup has completed before listing and downloading backups, so either: |
| 163 | + |
| 164 | +- update `streamLog()` to return a promise while preserving existing behavior for current callers, or |
| 165 | +- add a new local helper for `app sync` that uses `v.getLogStream({ id: stack, log: logId })` and resolves on `complete`, rejects on `fail`, and still renders progress. |
| 166 | + |
| 167 | +### 4. Find The Completed Backup |
| 168 | + |
| 169 | +After the stream completes successfully: |
| 170 | + |
| 171 | +- Call `GET /stack/applications/{stack}/backups`. |
| 172 | +- Sort newest first by `date`. |
| 173 | +- Select the newest backup created after the local command start time. |
| 174 | +- Prefer matching the expected manifest shape if available through the listed backup metadata. |
| 175 | + |
| 176 | +If no matching backup is found, print a clear manual recovery message: |
| 177 | + |
| 178 | +```txt |
| 179 | +Backup completed but the CLI could not find the download URL. Run: |
| 180 | + altis-cli app backups <app> |
| 181 | +``` |
| 182 | + |
| 183 | +### 5. Download And Extract |
| 184 | + |
| 185 | +Download the backup `url` to: |
| 186 | + |
| 187 | +```txt |
| 188 | +<output-dir>/<app>-<timestamp>.tar |
| 189 | +``` |
| 190 | + |
| 191 | +Extract into: |
| 192 | + |
| 193 | +```txt |
| 194 | +<output-dir>/<app>-<timestamp>/ |
| 195 | +``` |
| 196 | + |
| 197 | +Expected extracted files: |
| 198 | + |
| 199 | +```txt |
| 200 | +database.sql.gz |
| 201 | +uploads/ |
| 202 | +``` |
| 203 | + |
| 204 | +Validation: |
| 205 | + |
| 206 | +- If database sync was requested, require `database.sql.gz`. |
| 207 | +- If uploads sync was requested, require `uploads/`. |
| 208 | + |
| 209 | +### 6. Search-Replace And Restore Database |
| 210 | + |
| 211 | +Search-replace runs on the SQL stream before import using `@automattic/vip-search-replace`, which wraps [`go-search-replace`](https://github.com/Automattic/go-search-replace). The binary is downloaded on demand to `~/.altis-cli/bin/go-search-replace` on first use. |
| 212 | + |
| 213 | +Resolve mappings in this order: |
| 214 | + |
| 215 | +1. If `--skip-search-replace` is passed, skip search-replace and go straight to import. |
| 216 | +2. Read config mappings from `extra.altis.cloud.search-replace.local-server` in the local project's `composer.json`. Use `--search-replace-key <key>` to read a different key instead. |
| 217 | +3. Merge repeatable `--replace <from=to>` mappings after config mappings so explicit CLI values can add or override. |
| 218 | +4. Validate every mapping has a non-empty `from` and `to` before running. |
| 219 | + |
| 220 | +Configure local mappings in `composer.json`: |
| 221 | + |
| 222 | +```json |
| 223 | +{ |
| 224 | + "extra": { |
| 225 | + "altis": { |
| 226 | + "cloud": { |
| 227 | + "search-replace": { |
| 228 | + "local-server": { |
| 229 | + "example.com": "example.altis.dev", |
| 230 | + "www.example.com": "example.altis.dev" |
| 231 | + } |
| 232 | + } |
| 233 | + } |
| 234 | + } |
| 235 | + } |
| 236 | +} |
| 237 | +``` |
| 238 | + |
| 239 | +If no mappings are found, continue and print: |
| 240 | + |
| 241 | +```txt |
| 242 | +No search-replace mappings found. Skipping. Use --replace or configure extra.altis.cloud.search-replace.local-server in composer.json. |
| 243 | +``` |
| 244 | + |
| 245 | +Pipe the decompressed SQL through `go-search-replace` with all pairs in a single invocation, then feed the replaced stream directly into the local-server DB import: |
| 246 | + |
| 247 | +```js |
| 248 | +const { replace } = require('@automattic/vip-search-replace'); |
| 249 | +const replacements = Object.entries( mappings ).flat(); // ['from1','to1','from2','to2',...] |
| 250 | +const sqlStream = fs.createReadStream( sqlGzPath ).pipe( createGunzip() ); |
| 251 | +const replacedStream = await replace( sqlStream, replacements ); |
| 252 | +// pipe replacedStream into: composer server cli -- db import /dev/stdin |
| 253 | +``` |
| 254 | + |
| 255 | +After successful import, always run: |
| 256 | + |
| 257 | +```sh |
| 258 | +composer server cli -- cache flush |
| 259 | +``` |
| 260 | + |
| 261 | +Then, unless `--skip-post-sync` is passed, run the Altis post-sync hook to allow the user codebase to perform any environment-specific setup: |
| 262 | + |
| 263 | +```sh |
| 264 | +composer server cli -- altis post-sync |
| 265 | +``` |
| 266 | + |
| 267 | +If the command is not found or exits non-zero, print a warning but do not fail the sync. |
| 268 | + |
| 269 | +### 7. Restore Uploads Into Local Server |
| 270 | + |
| 271 | +Extract the backup's `uploads/` directory directly into `<local-project>/content/uploads/` using Node's tar extraction (no rsync needed). |
| 272 | + |
| 273 | +Then run from `<local-project>`: |
| 274 | + |
| 275 | +```sh |
| 276 | +composer server s3 import-uploads |
| 277 | +``` |
| 278 | + |
| 279 | +This mounts `content/uploads/` and syncs it into the local S3 container. Fallback for older local-server versions: |
| 280 | + |
| 281 | +```sh |
| 282 | +composer server import-uploads |
| 283 | +``` |
| 284 | + |
| 285 | +Try `composer server s3 import-uploads` first, then fall back to `composer server import-uploads` if the first command fails with an unsupported command error. |
| 286 | + |
| 287 | +### 8. Cleanup |
| 288 | + |
| 289 | +Unless `--keep-archive` is passed: |
| 290 | + |
| 291 | +- Remove downloaded archive. |
| 292 | +- Remove extracted working directory. |
| 293 | + |
| 294 | +Keep files on failure and print their paths for debugging. |
| 295 | + |
| 296 | +## Output |
| 297 | + |
| 298 | +Human output should show: |
| 299 | + |
| 300 | +- local project path |
| 301 | +- remote app |
| 302 | +- backup log id |
| 303 | +- downloaded archive path |
| 304 | +- database import status |
| 305 | +- search-replace status and mapping count |
| 306 | +- post-sync hook status |
| 307 | +- uploads import status |
| 308 | +- cleanup status |
| 309 | + |
| 310 | +`--json` should print: |
| 311 | + |
| 312 | +```json |
| 313 | +{ |
| 314 | + "stack": "example-dev-01", |
| 315 | + "path": "/path/to/local-project", |
| 316 | + "backup": { |
| 317 | + "id": "example-dev-01-2026-04-30-12-00-00.tar", |
| 318 | + "log": "example-dev-01/backups:..." |
| 319 | + }, |
| 320 | + "database": { |
| 321 | + "requested": true, |
| 322 | + "imported": true, |
| 323 | + "searchReplace": { |
| 324 | + "requested": true, |
| 325 | + "ran": true, |
| 326 | + "dryRun": false, |
| 327 | + "mappingCount": 2 |
| 328 | + }, |
| 329 | + "postSync": { |
| 330 | + "ran": true, |
| 331 | + "exitCode": 0 |
| 332 | + } |
| 333 | + }, |
| 334 | + "uploads": { |
| 335 | + "requested": true, |
| 336 | + "imported": true |
| 337 | + } |
| 338 | +} |
| 339 | +``` |
| 340 | + |
| 341 | +## Safety Rules |
| 342 | + |
| 343 | +- Never delete local uploads by default; merge and overwrite matching files only. |
| 344 | +- Never import database without confirmation unless `--yes` is passed. |
| 345 | +- Never run search-replace unless database sync was requested. |
| 346 | +- Never infer search-replace mappings from stack names, URLs, or local domains. Use composer.json mappings or explicit `--replace` values only. |
| 347 | +- Never infer the local project from a parent directory if `composer.json` is missing in `--path` or `cwd`. |
| 348 | +- Keep failed downloads/extractions for debugging. |
| 349 | +- Do not print signed backup URLs in normal output unless `--debug` is passed. |
| 350 | + |
| 351 | +## Implementation Notes |
| 352 | + |
| 353 | +- This command needs to run local shell commands. Use `child_process.spawn` with argument arrays rather than shell strings. |
| 354 | +- Run local commands with `cwd` set to the local project path. |
| 355 | +- Use Node's temp directory APIs for default `--output-dir`. |
| 356 | +- Use the existing Vantage auth flow and `Vantage.fetch()`. |
| 357 | +- Use existing `streamLog()` for remote backup progress. |
| 358 | +- Use the existing backup download approach from `lib/commands/stack/backups.js` as the starting point for progress reporting. |
| 359 | +- Parse `composer.json` with `JSON.parse()` and resolve `extra.altis.cloud.search-replace.local-server` as structured data. |
| 360 | +- Validate every search-replace mapping has a non-empty string `from` and `to` before running. |
| 361 | +- Use `@automattic/vip-search-replace` for search-replace. It wraps `go-search-replace` and downloads the binary on demand. Pass all pairs as a flat alternating array: `['from1','to1','from2','to2']`. All pairs run in a single invocation. |
| 362 | +- Search-replace runs on the decompressed SQL stream before DB import — no WP-CLI search-replace needed. |
| 363 | + |
| 364 | +## Acceptance Criteria |
| 365 | + |
| 366 | +- `app sync database <app>` creates a remote DB-only backup, downloads it, runs search-replace on the SQL stream, imports it into local-server, and flushes the object cache. |
| 367 | +- `app sync database <app>` reads search-replace mappings from `extra.altis.cloud.search-replace.local-server` in the local project's `composer.json` by default. |
| 368 | +- `app sync database <app>` supports repeatable `--replace <from=to>` mappings that add to or override composer.json mappings. |
| 369 | +- `app sync database <app> --search-replace-key <key>` reads `extra.altis.cloud.search-replace.<key>` instead of `local-server`. |
| 370 | +- `app sync database <app> --skip-search-replace` skips the search-replace step. |
| 371 | +- `app sync database <app>` runs `wp altis post-sync` after import and cache flush by default. |
| 372 | +- `app sync database <app> --skip-post-sync` skips the post-sync hook. |
| 373 | +- A non-zero exit from `wp altis post-sync` prints a warning but does not fail the sync. |
| 374 | +- `app sync database <app> --dry-run-search-replace` prints resolved mappings without triggering a backup or import. |
| 375 | +- `app sync uploads <app>` creates a remote uploads-only backup, downloads it, merges uploads into `content/uploads`, and imports them into local-server S3. |
| 376 | +- `app sync all <app>` performs both restore steps from one remote backup archive. |
| 377 | +- Running from a local Altis project works without passing `--path`. |
| 378 | +- `--path` can still target a local project outside the current working directory when needed. |
| 379 | +- Missing local-server dependency fails before any remote backup starts. |
| 380 | +- Local-server not running fails before any remote backup starts. |
| 381 | +- Destructive actions prompt unless `--yes` is passed. |
| 382 | +- Failed restore keeps the archive/extracted files and prints their paths. |
| 383 | + |
| 384 | +## Follow-Up Features |
| 385 | + |
| 386 | +- Automatic local-server startup with `--start`. |
| 387 | +- Interactive picker for recent backups instead of creating a new backup. |
| 388 | +- Direct download of an existing backup ID. |
| 389 | +- Better matching between streamed backup log ID and completed backup URL if Vantage adds that metadata. |
| 390 | +- Backend endpoint for resolved search-replace mappings from the deployed build, if exact Vantage backend parity becomes more important than local composer.json resolution. |
0 commit comments