Skip to content

Commit b670eea

Browse files
jericoclaude
andcommitted
Rename sync/ directory to sync-local/ to match the command name
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 784083f commit b670eea

15 files changed

Lines changed: 13355 additions & 0 deletions
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Subproject commit b7ec781f7a60cbe48df1478fa4240a926103dbee
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Subproject commit b7ec781f7a60cbe48df1478fa4240a926103dbee

docs/local-server-sync-plan.md

Lines changed: 390 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,390 @@
1+
# Local Server Sync Plan
2+
3+
This plan adds a CLI workflow for pulling database and uploads from a Vantage application into the local Altis codebase where the command is run.
4+
5+
## Goal
6+
7+
Provide a safe, repeatable local sync command:
8+
9+
```sh
10+
altis-cli app sync all <app>
11+
altis-cli app sync database <app>
12+
altis-cli app sync uploads <app>
13+
```
14+
15+
This command is expected to be run from a local Altis project that uses `altis/local-server`. This is different from Vantage's app-to-app import endpoints. Local sync should use Vantage exports/backups, download the archive locally, and restore it into the current local-server project.
16+
17+
## Adjacent Repo Changes
18+
19+
### `../hm/vantage-backend`
20+
21+
No required changes for the first implementation.
22+
23+
Existing API support is enough:
24+
25+
- `POST /stack/applications/{stack}/backups`
26+
- `GET /stack/applications/{stack}/backups`
27+
- Existing stream-log support through `/stream-log`
28+
- Existing search-replace configuration convention in `extra.altis.cloud.search-replace`
29+
30+
The backup archive already contains:
31+
32+
```txt
33+
database.sql.gz
34+
uploads/
35+
```
36+
37+
Optional future improvement:
38+
39+
- Return the generated backup object ID or download URL from the backup task/log metadata after completion. This would remove the need for the CLI to list backups and select the newest matching backup after a streamed export completes.
40+
- Add a small endpoint that exposes the resolved search-replace mappings for an application/build using the same logic as Vantage's app-to-app import flow. This is not required for the first CLI implementation if the CLI reads mappings from the local project's `composer.json`, but it would improve parity with the backend's exact deployed-build resolution.
41+
42+
### `../hm/altis-local-server`
43+
44+
No required changes for the first implementation.
45+
46+
Existing Local Server commands are enough:
47+
48+
- `composer server cli -- db import <file>`
49+
- `composer server s3 import-uploads`
50+
- `composer server import-uploads` as an alias in current local-server
51+
- `composer server status` for a lightweight readiness check
52+
53+
Optional future improvement:
54+
55+
- Add a dedicated `composer server sync-from-archive <archive>` command that imports DB and uploads from a Vantage backup archive in one local-server-native step. This is not needed for the CLI feature because the CLI can orchestrate existing commands.
56+
57+
## New CLI Command Family
58+
59+
Use a nested `app sync` command family.
60+
61+
Commands:
62+
63+
```sh
64+
altis-cli app sync all <app>
65+
altis-cli app sync database <app>
66+
altis-cli app sync uploads <app>
67+
```
68+
69+
The legacy `stack sync` alias should work too, but docs should prefer `app sync`.
70+
71+
Options:
72+
73+
- `--path <path>`: optional local Altis project path override. Defaults to the current working directory.
74+
- `--tables <table1,table2>`: database-only table selection.
75+
- `--uploads-path <prefix>`: uploads prefix to export from remote app.
76+
- `--search-replace-key <key>`: read mappings from `extra.altis.cloud.search-replace.<key>` in the local project's `composer.json`. Defaults to `local-server`.
77+
- `--replace <from=to>`: explicit search-replace mapping. Repeatable.
78+
- `--skip-search-replace`: skip search-replace entirely.
79+
- `--skip-post-sync`: skip the `wp altis post-sync` hook after database import.
80+
- `--dry-run-search-replace`: print the resolved mappings that would be applied, without triggering a backup or import.
81+
- `--output-dir <path>`: working directory for downloaded archives and extracted files. Defaults to a temp directory.
82+
- `--keep-archive`: keep downloaded `.tar` and extracted files after restore.
83+
- `--yes`: skip destructive confirmations.
84+
- `--resume <log-id>`: resume watching an already-started remote backup task.
85+
- `--debug`: pass through to stream-log rendering.
86+
- `--json`: print machine-readable summary at the end.
87+
88+
## Workflow
89+
90+
### 1. Validate Local Project
91+
92+
Before calling Vantage:
93+
94+
- Resolve the local project path from `--path` or `process.cwd()`.
95+
- Check `<local-project>/composer.json` exists.
96+
- Check the Composer project requires `altis/local-server` in `require` or `require-dev`.
97+
- Check `<local-project>/content` exists.
98+
- Check `composer` is available.
99+
- Run `composer server status` in the local project path. If it fails, show a clear message asking the user to start local-server with `composer server start`.
100+
101+
Do not start local-server automatically in the first implementation. The expected workflow is that the user runs this command from the intended local Altis project after local-server is already set up and running.
102+
103+
### 2. Confirm Destructive Actions
104+
105+
Prompt unless `--yes` is passed.
106+
107+
For database:
108+
109+
```txt
110+
This will replace the local database for the current Altis project with data from <app>. Continue?
111+
```
112+
113+
For uploads:
114+
115+
```txt
116+
This will merge remote uploads from <app> into ./content/uploads. Existing files may be overwritten. Continue?
117+
```
118+
119+
For `all`, show both warnings in one confirmation.
120+
121+
### 3. Create Remote Export
122+
123+
Call the existing backup endpoint with `stream=true`.
124+
125+
For database:
126+
127+
```http
128+
POST /stack/applications/{stack}/backups?stream=true
129+
```
130+
131+
Body/query params:
132+
133+
```json
134+
{
135+
"database": true,
136+
"uploads": false,
137+
"tables": ["wp_posts", "wp_options"]
138+
}
139+
```
140+
141+
For uploads:
142+
143+
```json
144+
{
145+
"database": false,
146+
"uploads": true,
147+
"uploads_path": "sites/2"
148+
}
149+
```
150+
151+
For all:
152+
153+
```json
154+
{
155+
"database": true,
156+
"uploads": true
157+
}
158+
```
159+
160+
The streamed backup endpoint returns a plain text log ID. Use the existing `streamLog(v, stack, logId, debug)` pattern to show progress.
161+
162+
Implementation note: the current `streamLog()` helper renders progress but does not expose a promise that resolves on completion. `app sync` needs to know when the backup has completed before listing and downloading backups, so either:
163+
164+
- update `streamLog()` to return a promise while preserving existing behavior for current callers, or
165+
- add a new local helper for `app sync` that uses `v.getLogStream({ id: stack, log: logId })` and resolves on `complete`, rejects on `fail`, and still renders progress.
166+
167+
### 4. Find The Completed Backup
168+
169+
After the stream completes successfully:
170+
171+
- Call `GET /stack/applications/{stack}/backups`.
172+
- Sort newest first by `date`.
173+
- Select the newest backup created after the local command start time.
174+
- Prefer matching the expected manifest shape if available through the listed backup metadata.
175+
176+
If no matching backup is found, print a clear manual recovery message:
177+
178+
```txt
179+
Backup completed but the CLI could not find the download URL. Run:
180+
altis-cli app backups <app>
181+
```
182+
183+
### 5. Download And Extract
184+
185+
Download the backup `url` to:
186+
187+
```txt
188+
<output-dir>/<app>-<timestamp>.tar
189+
```
190+
191+
Extract into:
192+
193+
```txt
194+
<output-dir>/<app>-<timestamp>/
195+
```
196+
197+
Expected extracted files:
198+
199+
```txt
200+
database.sql.gz
201+
uploads/
202+
```
203+
204+
Validation:
205+
206+
- If database sync was requested, require `database.sql.gz`.
207+
- If uploads sync was requested, require `uploads/`.
208+
209+
### 6. Search-Replace And Restore Database
210+
211+
Search-replace runs on the SQL stream before import using `@automattic/vip-search-replace`, which wraps [`go-search-replace`](https://github.com/Automattic/go-search-replace). The binary is downloaded on demand to `~/.altis-cli/bin/go-search-replace` on first use.
212+
213+
Resolve mappings in this order:
214+
215+
1. If `--skip-search-replace` is passed, skip search-replace and go straight to import.
216+
2. Read config mappings from `extra.altis.cloud.search-replace.local-server` in the local project's `composer.json`. Use `--search-replace-key <key>` to read a different key instead.
217+
3. Merge repeatable `--replace <from=to>` mappings after config mappings so explicit CLI values can add or override.
218+
4. Validate every mapping has a non-empty `from` and `to` before running.
219+
220+
Configure local mappings in `composer.json`:
221+
222+
```json
223+
{
224+
"extra": {
225+
"altis": {
226+
"cloud": {
227+
"search-replace": {
228+
"local-server": {
229+
"example.com": "example.altis.dev",
230+
"www.example.com": "example.altis.dev"
231+
}
232+
}
233+
}
234+
}
235+
}
236+
}
237+
```
238+
239+
If no mappings are found, continue and print:
240+
241+
```txt
242+
No search-replace mappings found. Skipping. Use --replace or configure extra.altis.cloud.search-replace.local-server in composer.json.
243+
```
244+
245+
Pipe the decompressed SQL through `go-search-replace` with all pairs in a single invocation, then feed the replaced stream directly into the local-server DB import:
246+
247+
```js
248+
const { replace } = require('@automattic/vip-search-replace');
249+
const replacements = Object.entries( mappings ).flat(); // ['from1','to1','from2','to2',...]
250+
const sqlStream = fs.createReadStream( sqlGzPath ).pipe( createGunzip() );
251+
const replacedStream = await replace( sqlStream, replacements );
252+
// pipe replacedStream into: composer server cli -- db import /dev/stdin
253+
```
254+
255+
After successful import, always run:
256+
257+
```sh
258+
composer server cli -- cache flush
259+
```
260+
261+
Then, unless `--skip-post-sync` is passed, run the Altis post-sync hook to allow the user codebase to perform any environment-specific setup:
262+
263+
```sh
264+
composer server cli -- altis post-sync
265+
```
266+
267+
If the command is not found or exits non-zero, print a warning but do not fail the sync.
268+
269+
### 7. Restore Uploads Into Local Server
270+
271+
Extract the backup's `uploads/` directory directly into `<local-project>/content/uploads/` using Node's tar extraction (no rsync needed).
272+
273+
Then run from `<local-project>`:
274+
275+
```sh
276+
composer server s3 import-uploads
277+
```
278+
279+
This mounts `content/uploads/` and syncs it into the local S3 container. Fallback for older local-server versions:
280+
281+
```sh
282+
composer server import-uploads
283+
```
284+
285+
Try `composer server s3 import-uploads` first, then fall back to `composer server import-uploads` if the first command fails with an unsupported command error.
286+
287+
### 8. Cleanup
288+
289+
Unless `--keep-archive` is passed:
290+
291+
- Remove downloaded archive.
292+
- Remove extracted working directory.
293+
294+
Keep files on failure and print their paths for debugging.
295+
296+
## Output
297+
298+
Human output should show:
299+
300+
- local project path
301+
- remote app
302+
- backup log id
303+
- downloaded archive path
304+
- database import status
305+
- search-replace status and mapping count
306+
- post-sync hook status
307+
- uploads import status
308+
- cleanup status
309+
310+
`--json` should print:
311+
312+
```json
313+
{
314+
"stack": "example-dev-01",
315+
"path": "/path/to/local-project",
316+
"backup": {
317+
"id": "example-dev-01-2026-04-30-12-00-00.tar",
318+
"log": "example-dev-01/backups:..."
319+
},
320+
"database": {
321+
"requested": true,
322+
"imported": true,
323+
"searchReplace": {
324+
"requested": true,
325+
"ran": true,
326+
"dryRun": false,
327+
"mappingCount": 2
328+
},
329+
"postSync": {
330+
"ran": true,
331+
"exitCode": 0
332+
}
333+
},
334+
"uploads": {
335+
"requested": true,
336+
"imported": true
337+
}
338+
}
339+
```
340+
341+
## Safety Rules
342+
343+
- Never delete local uploads by default; merge and overwrite matching files only.
344+
- Never import database without confirmation unless `--yes` is passed.
345+
- Never run search-replace unless database sync was requested.
346+
- Never infer search-replace mappings from stack names, URLs, or local domains. Use composer.json mappings or explicit `--replace` values only.
347+
- Never infer the local project from a parent directory if `composer.json` is missing in `--path` or `cwd`.
348+
- Keep failed downloads/extractions for debugging.
349+
- Do not print signed backup URLs in normal output unless `--debug` is passed.
350+
351+
## Implementation Notes
352+
353+
- This command needs to run local shell commands. Use `child_process.spawn` with argument arrays rather than shell strings.
354+
- Run local commands with `cwd` set to the local project path.
355+
- Use Node's temp directory APIs for default `--output-dir`.
356+
- Use the existing Vantage auth flow and `Vantage.fetch()`.
357+
- Use existing `streamLog()` for remote backup progress.
358+
- Use the existing backup download approach from `lib/commands/stack/backups.js` as the starting point for progress reporting.
359+
- Parse `composer.json` with `JSON.parse()` and resolve `extra.altis.cloud.search-replace.local-server` as structured data.
360+
- Validate every search-replace mapping has a non-empty string `from` and `to` before running.
361+
- Use `@automattic/vip-search-replace` for search-replace. It wraps `go-search-replace` and downloads the binary on demand. Pass all pairs as a flat alternating array: `['from1','to1','from2','to2']`. All pairs run in a single invocation.
362+
- Search-replace runs on the decompressed SQL stream before DB import — no WP-CLI search-replace needed.
363+
364+
## Acceptance Criteria
365+
366+
- `app sync database <app>` creates a remote DB-only backup, downloads it, runs search-replace on the SQL stream, imports it into local-server, and flushes the object cache.
367+
- `app sync database <app>` reads search-replace mappings from `extra.altis.cloud.search-replace.local-server` in the local project's `composer.json` by default.
368+
- `app sync database <app>` supports repeatable `--replace <from=to>` mappings that add to or override composer.json mappings.
369+
- `app sync database <app> --search-replace-key <key>` reads `extra.altis.cloud.search-replace.<key>` instead of `local-server`.
370+
- `app sync database <app> --skip-search-replace` skips the search-replace step.
371+
- `app sync database <app>` runs `wp altis post-sync` after import and cache flush by default.
372+
- `app sync database <app> --skip-post-sync` skips the post-sync hook.
373+
- A non-zero exit from `wp altis post-sync` prints a warning but does not fail the sync.
374+
- `app sync database <app> --dry-run-search-replace` prints resolved mappings without triggering a backup or import.
375+
- `app sync uploads <app>` creates a remote uploads-only backup, downloads it, merges uploads into `content/uploads`, and imports them into local-server S3.
376+
- `app sync all <app>` performs both restore steps from one remote backup archive.
377+
- Running from a local Altis project works without passing `--path`.
378+
- `--path` can still target a local project outside the current working directory when needed.
379+
- Missing local-server dependency fails before any remote backup starts.
380+
- Local-server not running fails before any remote backup starts.
381+
- Destructive actions prompt unless `--yes` is passed.
382+
- Failed restore keeps the archive/extracted files and prints their paths.
383+
384+
## Follow-Up Features
385+
386+
- Automatic local-server startup with `--start`.
387+
- Interactive picker for recent backups instead of creating a new backup.
388+
- Direct download of an existing backup ID.
389+
- Better matching between streamed backup log ID and completed backup URL if Vantage adds that metadata.
390+
- Backend endpoint for resolved search-replace mappings from the deployed build, if exact Vantage backend parity becomes more important than local composer.json resolution.

0 commit comments

Comments
 (0)