Skip to content

Commit 7dde891

Browse files
adamzieladamziel
authored
Support exporting from SQLite WordPress sites (#14)
## Summary Adds a `create_db_connection()` factory that selects the right database driver – PDO MySQL or `SqliteDriverPDO` adapter that runs the SQL queries using the MySQL-on-SQLite adapter. It only implements the methods used by our `MySQLDumpProducer` and will eventually be replaced by the upstream PDO adapter implementation once it becomes available. **Other changes:** - Removes `api.php` that allowed a standalone access to the site export endpoint. We don't really need that. Exporting a WordPress site depends on the WordPress runtime. - `resolve_db_credentials()` skips MySQL credential validation on SQLite sites - Preflight reports `db_engine` and checks `pdo_sqlite` instead of `pdo_mysql` - MySQL `@@variable` queries in preflight wrapped in their own try/catch for graceful degradation - `MySQLDumpProducer` constructor PDO type hint removed (adapter is duck-typed, PHP 7.4 lacks unions) - The factory verifies `SQLITE_DRIVER_VERSION >= 2.1.0` (when `get_connection()` stabilized) and throws a clear error if the plugin is too old. ## E2E test The new `import-35-sqlite-export.test.js` provisions a real SQLite WordPress site: downloads the sqlite-database-integration plugin, creates the db.php drop-in, writes wp-config.php with `WP_SQLITE_AST_DRIVER=true`, and runs `wp core install` to populate tables in the .ht.sqlite file. It then verifies: 1. Preflight reports `db_engine: "sqlite"` with a working connection 2. db-sync produces a valid MySQL dump with CREATE TABLE and INSERT for all WP tables 3. The dump imports into MySQL and all 10 standard WordPress tables contain the expected data ## Test plan - [x] All 268 existing tests pass (`composer test`) - [x] PHPStan clean (`composer analyze`) - [x] E2E test passes in CI (SQLite site export → MySQL import round-trip) - [x] Verify standard MySQL export unchanged
1 parent 92fda61 commit 7dde891

File tree

50 files changed

+1095
-520
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

50 files changed

+1095
-520
lines changed

‎.github/workflows/e2e.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -51,14 +51,14 @@ jobs:
5151
5252
# Quick smoke test: use the importer CLI to hit the export API.
5353
SECRET="test-secret-basic"
54-
BASE="http://127.0.0.1:8081/api.php"
54+
BASE="http://127.0.0.1:8081/?site-export-api"
5555
DIR="/srv/e2e-sites/basic"
5656
5757
echo "=== Importer preflight ==="
58-
timeout 30 php ../../importer/import.php preflight "${BASE}?directory=${DIR}" /tmp/e2e-smoke --secret="${SECRET}" 2>&1 | head -50 || echo "EXIT: $?"
58+
timeout 30 php ../../importer/import.php preflight "${BASE}&directory=${DIR}" /tmp/e2e-smoke --secret="${SECRET}" 2>&1 | head -50 || echo "EXIT: $?"
5959
6060
echo "=== Importer files-sync ==="
61-
timeout 30 php ../../importer/import.php files-sync "${BASE}?directory=${DIR}" /tmp/e2e-smoke --secret="${SECRET}" 2>&1 | head -50 || echo "EXIT: $?"
61+
timeout 30 php ../../importer/import.php files-sync "${BASE}&directory=${DIR}" /tmp/e2e-smoke --secret="${SECRET}" 2>&1 | head -50 || echo "EXIT: $?"
6262
6363
- name: Run E2E tests
6464
working-directory: tests/e2e

‎CLAUDE.md‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -123,8 +123,7 @@ PHPUnit tests automatically create/drop test databases. The naming convention is
123123
## File Organization
124124

125125
- wordpress-plugin/: Self-contained WordPress plugin directory
126-
- index.php: Thin WordPress plugin loader (plugin header, constants)
127-
- api.php: Standalone HTTP entry point (no WordPress dependency)
126+
- index.php: Plugin entry point — intercepts `?site-export-api` requests during plugin load, handles HMAC auth and export dispatch
128127
- generic/: Core export engine (export.php, producers, HMAC client, secrets)
129128
- wordpress/: WordPress admin UI (site-export.php)
130129
- importer/: Import client (import.php)

‎importer/import.php‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1248,7 +1248,7 @@ public function run(array $options = []): void
12481248

12491249
// Initialize HMAC authentication if a shared secret was provided.
12501250
// When set, every outgoing HTTP request will include X-Auth-Signature,
1251-
// X-Auth-Nonce, and X-Auth-Timestamp headers so api.php can verify
1251+
// X-Auth-Nonce, and X-Auth-Timestamp headers so the export API can verify
12521252
// the caller without a SECRET_KEY in the URL.
12531253
if (!empty($options["secret"])) {
12541254
// TODO: Distribute with the importer script somehow. Phar? Co-locate? A build script?
@@ -6283,7 +6283,7 @@ class StreamingContext
62836283
"Options:\n" .
62846284
" --abort Abort current sync and exit (keeps files and index)\n" .
62856285
" --follow-symlinks Follow symlinks pointing outside root directories\n" .
6286-
" --secret=TOKEN HMAC shared secret for api.php authentication\n" .
6286+
" --secret=TOKEN HMAC shared secret for export API authentication\n" .
62876287
" --verbose, -v Show detailed request/response logs\n" .
62886288
"\n" .
62896289
"Output files:\n" .
@@ -6302,7 +6302,7 @@ class StreamingContext
63026302
"\n" .
63036303
"Options:\n" .
63046304
" --abort Clear state and output, then exit\n" .
6305-
" --secret=TOKEN HMAC shared secret for api.php authentication\n" .
6305+
" --secret=TOKEN HMAC shared secret for export API authentication\n" .
63066306
" --verbose, -v Show detailed request/response logs\n",
63076307
],
63086308
"db-sync" => [
@@ -6313,7 +6313,7 @@ class StreamingContext
63136313
"\n" .
63146314
"Options:\n" .
63156315
" --abort Clear state and output, then exit\n" .
6316-
" --secret=TOKEN HMAC shared secret for api.php authentication\n" .
6316+
" --secret=TOKEN HMAC shared secret for export API authentication\n" .
63176317
" --verbose, -v Show detailed request/response logs\n" .
63186318
" --max-allowed-packet=SIZE Client max_allowed_packet (e.g. 16M, 64M)\n" .
63196319
"\n" .
@@ -6328,7 +6328,7 @@ class StreamingContext
63286328
"\n" .
63296329
"Options:\n" .
63306330
" --abort Clear state and output, then exit\n" .
6331-
" --secret=TOKEN HMAC shared secret for api.php authentication\n" .
6331+
" --secret=TOKEN HMAC shared secret for export API authentication\n" .
63326332
" --verbose, -v Show detailed request/response logs\n" .
63336333
"\n" .
63346334
"Output files:\n" .
@@ -6345,7 +6345,7 @@ class StreamingContext
63456345
"Exits 0 if the server reported OK, 1 otherwise.\n" .
63466346
"\n" .
63476347
"Options:\n" .
6348-
" --secret=TOKEN HMAC shared secret for api.php authentication\n",
6348+
" --secret=TOKEN HMAC shared secret for export API authentication\n",
63496349
],
63506350
"preflight-assert" => [
63516351
"short" => "Check if migration is feasible (exits 0 or 1)",
@@ -6361,7 +6361,7 @@ class StreamingContext
63616361
"Prints a PASS/FAIL summary and exits 0 if all checks pass, 1 if not.\n" .
63626362
"\n" .
63636363
"Options:\n" .
6364-
" --secret=TOKEN HMAC shared secret for api.php authentication\n",
6364+
" --secret=TOKEN HMAC shared secret for export API authentication\n",
63656365
],
63666366
];
63676367

@@ -6378,7 +6378,7 @@ class StreamingContext
63786378
echo "Run 'php import.php <command> --help' for command-specific help.\n";
63796379
echo "\n";
63806380
echo "Global options:\n";
6381-
echo " --secret=TOKEN HMAC shared secret for api.php authentication\n";
6381+
echo " --secret=TOKEN HMAC shared secret for export API authentication\n";
63826382
echo " --abort Abort current sync and exit (preserves downloaded files)\n";
63836383
echo " --follow-symlinks Follow symlinks pointing outside root directories\n";
63846384
echo " --verbose, -v Show detailed request/response logs\n";

‎phpstan-baseline.neon‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,3 +44,18 @@ parameters:
4444
message: "#^Strict comparison using \\!\\=\\= between null and null will always evaluate to false\\.$#"
4545
count: 1
4646
path: wordpress-plugin/generic/class-file-tree-producer.php
47+
48+
-
49+
message: "#^Class WP_SQLite_Driver not found\\.$#"
50+
count: 1
51+
path: wordpress-plugin/generic/export.php
52+
53+
-
54+
message: "#^Call to method get_connection\\(\\) on an unknown class WP_SQLite_Driver\\.$#"
55+
count: 1
56+
path: wordpress-plugin/generic/export.php
57+
58+
-
59+
message: "#^Instantiated class SqliteDriverPDO not found\\.$#"
60+
count: 1
61+
path: wordpress-plugin/generic/export.php

‎phpstan.neon‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ parameters:
99
- importer
1010
excludePaths:
1111
- wordpress-plugin/generic/secrets.php
12+
- wordpress-plugin/generic/class-sqlite-driver-pdo.php
1213
- wordpress-plugin/wordpress/
1314
- wordpress-plugin/index.php
1415
treatPhpDocTypesAsCertain: false

‎tests/e2e/ci/setup-infrastructure.sh‎

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ sudo add-apt-repository -y ppa:ondrej/php
2020
sudo apt-get update -qq
2121
sudo apt-get install -y \
2222
"php${PHP_VERSION}-cli" "php${PHP_VERSION}-fpm" \
23-
"php${PHP_VERSION}-mysql" "php${PHP_VERSION}-mbstring" "php${PHP_VERSION}-curl" "php${PHP_VERSION}-xml" "php${PHP_VERSION}-zip"
23+
"php${PHP_VERSION}-mysql" "php${PHP_VERSION}-mbstring" "php${PHP_VERSION}-curl" "php${PHP_VERSION}-xml" "php${PHP_VERSION}-zip" "php${PHP_VERSION}-sqlite3"
2424

2525
# Make sure the 'php' CLI command uses the version we just installed
2626
sudo update-alternatives --set php "/usr/bin/php${PHP_VERSION}"
@@ -123,15 +123,16 @@ jq -r '.sites | to_entries[] | select((.value.nginx // "standard") == "standard"
123123
cat <<VHOST | sudo tee "/etc/nginx/conf.d/e2e-${site}.conf" >/dev/null
124124
server {
125125
listen 127.0.0.1:${port};
126-
root ${SITE_ROOT}/${site}/wp-content/plugins/site-export;
126+
root ${SITE_ROOT}/${site};
127+
index index.php;
127128
128129
location / {
129-
try_files \$uri \$uri/ /api.php?\$query_string;
130+
try_files \$uri \$uri/ /index.php?\$query_string;
130131
}
131132
132133
location ~ \\.php\$ {
133134
fastcgi_pass unix:${FPM_SOCKET};
134-
fastcgi_index api.php;
135+
fastcgi_index index.php;
135136
fastcgi_param SCRIPT_FILENAME \$document_root\$fastcgi_script_name;
136137
include fastcgi_params;
137138
fastcgi_param SITE_EXPORT_TEST_MODE "1";
@@ -155,19 +156,20 @@ VHOST
155156
done
156157

157158
# Buffered sites
158-
jq -r '.sites | to_entries[] | select(.value.nginx == "buffered") | "\(.key) \(.value.port)"' "$REGISTRY" | while read site port; do
159+
jq -r '.sites | to_entries[] | select(.value.nginx == "buffered") | "\(.key) \(.value.port)"' "$REGISTRY" | while read site port target; do
159160
cat <<VHOST | sudo tee "/etc/nginx/conf.d/e2e-${site}.conf" >/dev/null
160161
server {
161162
listen 127.0.0.1:${port};
162-
root ${SITE_ROOT}/${site}/wp-content/plugins/site-export;
163+
root ${SITE_ROOT}/${site};
164+
index index.php;
163165
164166
location / {
165-
try_files \$uri \$uri/ /api.php?\$query_string;
167+
try_files \$uri \$uri/ /index.php?\$query_string;
166168
}
167169
168170
location ~ \\.php\$ {
169171
fastcgi_pass unix:${FPM_SOCKET};
170-
fastcgi_index api.php;
172+
fastcgi_index index.php;
171173
fastcgi_param SCRIPT_FILENAME \$document_root\$fastcgi_script_name;
172174
include fastcgi_params;
173175
fastcgi_param SITE_EXPORT_TEST_MODE "1";

‎tests/e2e/docker-entrypoint.sh‎

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -85,16 +85,18 @@ NGINXCONF
8585

8686
rm -f /etc/nginx/sites-enabled/default /etc/nginx/conf.d/default.conf
8787

88-
# Standard sites
88+
# Standard sites — serve from WordPress root so that index.php
89+
# bootstraps WordPress and the site-export plugin handles the request.
8990
jq -r '.sites | to_entries[] | select((.value.nginx // "standard") == "standard") | "\(.key) \(.value.port)"' "$REGISTRY" | while read site port; do
9091
cat > "/etc/nginx/conf.d/e2e-${site}.conf" <<VHOST
9192
server {
9293
listen 127.0.0.1:${port};
93-
root ${SITE_ROOT}/${site}/wp-content/plugins/site-export;
94-
location / { try_files \$uri \$uri/ /api.php?\$query_string; }
94+
root ${SITE_ROOT}/${site};
95+
index index.php;
96+
location / { try_files \$uri \$uri/ /index.php?\$query_string; }
9597
location ~ \\.php\$ {
9698
fastcgi_pass unix:${FPM_SOCKET};
97-
fastcgi_index api.php;
99+
fastcgi_index index.php;
98100
fastcgi_param SCRIPT_FILENAME \$document_root\$fastcgi_script_name;
99101
include fastcgi_params;
100102
fastcgi_param SITE_EXPORT_TEST_MODE "1";
@@ -120,11 +122,12 @@ jq -r '.sites | to_entries[] | select(.value.nginx == "buffered") | "\(.key) \(.
120122
cat > "/etc/nginx/conf.d/e2e-${site}.conf" <<VHOST
121123
server {
122124
listen 127.0.0.1:${port};
123-
root ${SITE_ROOT}/${site}/wp-content/plugins/site-export;
124-
location / { try_files \$uri \$uri/ /api.php?\$query_string; }
125+
root ${SITE_ROOT}/${site};
126+
index index.php;
127+
location / { try_files \$uri \$uri/ /index.php?\$query_string; }
125128
location ~ \\.php\$ {
126129
fastcgi_pass unix:${FPM_SOCKET};
127-
fastcgi_index api.php;
130+
fastcgi_index index.php;
128131
fastcgi_param SCRIPT_FILENAME \$document_root\$fastcgi_script_name;
129132
include fastcgi_params;
130133
fastcgi_param SITE_EXPORT_TEST_MODE "1";

‎tests/e2e/lib/site-setup.js‎

Lines changed: 31 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -157,6 +157,19 @@ function wpCoreInstall(siteDir, siteUrl, siteName) {
157157
);
158158
}
159159

160+
/**
161+
* Activate a WordPress plugin via WP-CLI.
162+
*/
163+
function wpPluginActivate(siteDir, pluginSlug) {
164+
const allowRoot = process.getuid?.() === 0 ? ' --allow-root' : '';
165+
execSync(
166+
`php ${WP_CLI_PATH} plugin activate ${pluginSlug}` +
167+
` --path=${JSON.stringify(siteDir)}` +
168+
allowRoot,
169+
{ timeout: 30000, stdio: 'pipe' }
170+
);
171+
}
172+
160173
/**
161174
* Create standard sample test files using Node fs APIs.
162175
*/
@@ -255,15 +268,26 @@ export async function ensureSite(name, options = {}) {
255268

256269
// Copy plugin source files
257270
safeCopyFile(
258-
join(PLUGIN_SRC, 'api.php'),
259-
join(siteDir, 'wp-content', 'plugins', 'site-export', 'api.php')
271+
join(PLUGIN_SRC, 'index.php'),
272+
join(siteDir, 'wp-content', 'plugins', 'site-export', 'index.php')
260273
);
261274
for (const f of readdirSync(join(PLUGIN_SRC, 'generic')).filter(f => f.endsWith('.php'))) {
262275
safeCopyFile(
263276
join(PLUGIN_SRC, 'generic', f),
264277
join(siteDir, 'wp-content', 'plugins', 'site-export', 'generic', f)
265278
);
266279
}
280+
// Copy wordpress/ subdirectory (admin settings page)
281+
const wpSubdir = join(PLUGIN_SRC, 'wordpress');
282+
if (existsSync(wpSubdir)) {
283+
mkdirSync(join(siteDir, 'wp-content', 'plugins', 'site-export', 'wordpress'), { recursive: true });
284+
for (const f of readdirSync(wpSubdir).filter(f => f.endsWith('.php'))) {
285+
safeCopyFile(
286+
join(wpSubdir, f),
287+
join(siteDir, 'wp-content', 'plugins', 'site-export', 'wordpress', f)
288+
);
289+
}
290+
}
267291
log('Files copied');
268292

269293
// Create database and run wp core install
@@ -282,6 +306,9 @@ export async function ensureSite(name, options = {}) {
282306
wpCoreInstall(siteDir, siteUrl, name);
283307
log('wp core install done');
284308

309+
// Activate the site-export plugin so WordPress loads index.php on requests.
310+
wpPluginActivate(siteDir, 'site-export');
311+
285312
// Run customDb hook to add extra tables on top of real WP
286313
if (options.customDb) {
287314
log('Running customDb hook...');
@@ -296,16 +323,13 @@ export async function ensureSite(name, options = {}) {
296323
}
297324
}
298325

299-
// Rewrite wp-config.php: minimal version for the export plugin
300-
// (the full config was only needed for wp core install above)
326+
// Rewrite wp-config.php with custom credentials if requested.
301327
if (options.wpConfig) {
302328
const wpDbUser = options.wpConfig.DB_USER || DB_USER;
303329
const wpDbPass = options.wpConfig.DB_PASSWORD || DB_PASS;
304330
const wpDbName = options.wpConfig.DB_NAME || dbName;
305331
const wpDbHost = options.wpConfig.DB_HOST || DB_HOST;
306-
writeMinimalWpConfig(siteDir, wpDbHost, wpDbName, wpDbUser, wpDbPass);
307-
} else {
308-
writeMinimalWpConfig(siteDir, DB_HOST, dbName, DB_USER, DB_PASS);
332+
writeFullWpConfig(siteDir, wpDbHost, wpDbName, wpDbUser, wpDbPass);
309333
}
310334

311335
// Create sample files (pure Node fs)

‎tests/e2e/lib/test-helpers.js‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ const DB_PASS = REGISTRY.dbPass;
2727
export function getSiteUrl(siteName, port = null) {
2828
const p = port || REGISTRY.sites[siteName]?.port;
2929
if (!p) throw new Error(`Unknown site: ${siteName}`);
30-
return `http://127.0.0.1:${p}/api.php`;
30+
return `http://127.0.0.1:${p}/?site-export-api`;
3131
}
3232

3333
/**

‎tests/e2e/nixos-e2e-services.nix‎

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ let
44
# PHP with required extensions
55
phpPackage = pkgs.php82.withExtensions ({ enabled, all }: enabled ++ [
66
all.pdo_mysql
7+
all.pdo_sqlite
78
all.zlib
89
all.curl
910
all.mbstring
@@ -32,15 +33,15 @@ let
3233
name = "e2e-${name}";
3334
value = {
3435
listen = [{ addr = "127.0.0.1"; port = cfg.port; }];
35-
root = "${siteRoot}/${name}/wp-content/plugins/site-export";
36+
root = "${siteRoot}/${name}";
3637
locations = {
3738
"/" = {
38-
tryFiles = "$uri $uri/ /api.php?$query_string";
39+
tryFiles = "$uri $uri/ /index.php?$query_string";
3940
};
4041
"~ \\.php$" = {
4142
extraConfig = ''
4243
fastcgi_pass unix:${config.services.phpfpm.pools.e2e.socket};
43-
fastcgi_index api.php;
44+
fastcgi_index index.php;
4445
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
4546
include ${pkgs.nginx}/conf/fastcgi_params;
4647
fastcgi_param SITE_EXPORT_TEST_MODE "1";
@@ -57,15 +58,15 @@ let
5758
name = "e2e-${name}";
5859
value = {
5960
listen = [{ addr = "127.0.0.1"; port = cfg.port; }];
60-
root = "${siteRoot}/${name}/wp-content/plugins/site-export";
61+
root = "${siteRoot}/${name}";
6162
locations = {
6263
"/" = {
63-
tryFiles = "$uri $uri/ /api.php?$query_string";
64+
tryFiles = "$uri $uri/ /index.php?$query_string";
6465
};
6566
"~ \\.php$" = {
6667
extraConfig = ''
6768
fastcgi_pass unix:${config.services.phpfpm.pools.e2e.socket};
68-
fastcgi_index api.php;
69+
fastcgi_index index.php;
6970
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
7071
include ${pkgs.nginx}/conf/fastcgi_params;
7172
fastcgi_param SITE_EXPORT_TEST_MODE "1";

0 commit comments

Comments
 (0)