Experiment with native /internal/shared dir for Playground CLI #2446

brandonpayton · 2025-07-31T06:33:14Z

Motivation for the change, related issues

There are two reasons to mount a real directory as /internal/shared:

We intend /internal/shared to be truly shared between all PHP instances
Zipping the primary worker /internal dir and unzipping for secondary workers is making multi-worker CLI startup quite slow

Related to #2419

Implementation details

TBD

Testing Instructions (or ideally a Blueprint)

TBD

brandonpayton · 2025-07-31T07:16:12Z

With --enableMultiWorker=2 and a native dir mounted as /internal/shared, Playground CLI dies in the second call to bootWordPress(). We haven't ever had two workers that truly shared a live /internal/shared dir, so this isn't too surprising.

Planning to look at this more tomorrow.

brandonpayton · 2025-07-31T07:21:44Z

Currently, the Blueprints v1 worker's bootAsSecondaryWorker() method just calls bootAsPrimaryWorker(). We might be running into conflicts with attempting to recreate existing files or something like that.

brandonpayton · 2025-07-31T07:29:38Z

Currently, the Blueprints v1 worker's bootAsSecondaryWorker() method just calls bootAsPrimaryWorker(). We might be running into conflicts with attempting to recreate existing files or something like that.

Yeah, the errno is 20 which maps to EEXIST in Emscripten libs.

adamziel · 2025-07-31T07:52:06Z

Yeah we might need to rewire some of the /internal initialization logic for a secondary worker when using Blueprints v1 to support this.

brandonpayton · 2025-07-31T18:08:47Z

This is a functional experiment now, and Playground CLI multi-worker boot feels nearly instantaneous on my laptop.

It requires rebuilding php-wasm/node, which I plan to do after lunch.

NOTE:
I did see one random error where Playground CLI was redirecting homepage requests to an error page like "expected WebSocket request" or something like that. But the error disappeared after restarting the CLI, and I haven't been able to reproduce it again.

brandonpayton · 2025-07-31T18:10:33Z

Yeah we might need to rewire some of the /internal initialization logic for a secondary worker when using Blueprints v1 to support this.

@adamziel, the only issue appears to have been creating the initial dirs under /internal/shared. By switching those lines from FS.mkdir() to FS.mkdirTree() and making one file creation conditional, the problem was fixed.

brandonpayton · 2025-08-02T04:26:25Z

Some things that need done:

Stress test
Add support for temp /internal/shared dir for Blueprints v2
Add auto-cleanup when Playground CLI is killed
Confirm that we are OK with the temp dir location and default permissions in case temp dir cleanup fails

brandonpayton · 2025-08-02T04:27:02Z

This isn't ready to merge, but it is ready for more feedback.

packages/php-wasm/compile/php/phpwasm-emscripten-library.js

adamziel · 2025-08-04T15:50:20Z

This is simple and effective, I like it. Thank you @brandonpayton!

Add auto-cleanup when Playground CLI is killed

This is a good idea. There will be some cases where that auto-cleanup won't run, e.g. the process gets killed without waiting, there's a power outage etc. I wonder what are some things we could do to maximize our chances of actually cleaning up stale directories. Perhaps we could use a naming pattern including PID and then sweep stale directories on startup? There could be some risk with short-lived processes so maybe a playground-{pid}-{timestamp} could help resolve them, as in we'd only sweep stale directories that are at least 1 hour old?

adamziel · 2025-08-04T16:50:28Z

Poking around more, we'll need to address all these paths:

wordpress-playground/packages/playground/wordpress/src/boot.ts

Lines 232 to 236 in 89a7f30

    
           proxyFileSystem(await requestHandler.getPrimaryPhp(), php, [ 
        
           	'/tmp', 
        
           	requestHandler.documentRoot, 
        
           	'/internal/shared', 
        
           ]);

Ideally, we can either get rid of proxyFileSystem() or find a way to contextualize what it does, as in uses PROXYFS, NODEFS, or other synchronization means in the future.

Also, we need to take this bit of the PHP class into account (in hotSwapPHPRuntime) – I think it already skips non-MEMFS nodes, but I'll still surface it here just in case:

wordpress-playground/packages/php-wasm/universal/src/lib/php.ts

Line 1279 in 89a7f30

// Copy the old /internal directory to the new filesystem

adamziel · 2025-08-04T16:55:29Z

Noodling on this more, perhaps a useful API would be something like markPathAsShared(/internal/shared, secondaryWorker, primaryWorker)? 🤔 The workers could then decide what to do based on how they were initialized, e.g.

Ignore the call in a primary worker and mount a primaryWorker path PROXYFS in a secondary worker
Mount a NODEFS directory in all the workers
(future) Mount a SharedArrayBuffer Filesystem

In any case, I've instrumented the Blueprints v2 worker code and the workers seem to boot fairly quickly. I don't have any numbers to quote but it feels fast!

adamziel · 2025-08-04T17:01:09Z

packages/php-wasm/node/asyncify/php_8_0.js

@@ -3914,6 +3912,8 @@ export function init(RuntimeName, PHPLoader) {
 				node = lookup.node;

 				if (FS.isMountpoint(node)) {
+					console.log({ mountpoint });


I accidentally reformatted the file. My changes here are all about this and the following two console.logs.

brandonpayton · 2025-08-07T21:11:48Z

This comment responds to multiple comments, not necessarily in the order they were made.

In any case, I've instrumented the Blueprints v2 worker code and the workers seem to boot fairly quickly. I don't have any numbers to quote but it feels fast!

Nice! Thanks for adding that. Yeah, that's been my experience as well.

Noodling on this more, perhaps a useful API would be something like markPathAsShared(/internal/shared, secondaryWorker, primaryWorker)? 🤔 The workers could then decide what to do based on how they were initialized,

@adamziel, what if we invert this? It seems like we want all workers to respond to requests based on the same state. What FS paths do we want to be private/separate per worker?

Poking around more, we'll need to address all these paths:
and
Also, we need to take this bit of the PHP class into account (in hotSwapPHPRuntime) – I think it already skips non-MEMFS nodes, but I'll still surface it here just in case:

Ah, good points! It's obvious we'd want to stop proxying already-shared filesystems (and same for copying from old to new PHP instances during hot swap), but somehow I was forgetting that.

adamziel · 2025-08-07T22:13:22Z

what if we invert this? It seems like we want all workers to respond to requests based on the same state. What FS paths do we want to be private/separate per worker?

Oh I like it! Off the top of my head, it's just the input/output devices. It seems like we could share everything else 🤔

So these paths would become worker-specific:

/internal/stdout
/internal/stderr
/internal/headers

We could move them to /internal/isolated (as opposed to /internal/shared to keep the /internal/shared path working for existing API consumers) and share the rest of the filesystem.

brandonpayton · 2025-08-08T03:47:41Z

Oh I like it! Off the top of my head, it's just the input/output devices. It seems like we could share everything else 🤔

So these paths would become worker-specific:
* /internal/stdout
* /internal/stderr
* /internal/headers
We could move them to /internal/isolated (as opposed to /internal/shared to keep the /internal/shared path working for existing API consumers) and share the rest of the filesystem.

Sweet. I'm looking at doing this.

Using /internal/isolated seems a bit tricky to do with Emscripten mounting because it doesn't look like we can easily mount a MEMFS dir inside a NODEFS dir. At least Google suggests we probably cannot easily do this because it would mean mixing contents of a real NODEFS dir with artificial mounts in that dir. Based on my recent looks into emscripten FS implementations, I found this reasoning compelling, but I haven't tested it.

If that is truly an issue, we will likely run into trouble if trying to mount a NODEFS dir as a subdir of our default/temp NODEFS dir. Then again, how does wp-now do this today for automounted subdirs of /wordpress ?

Will test and follow up here.

brandonpayton · 2025-08-08T03:53:49Z

Using /internal/isolated seems a bit tricky to do with Emscripten mounting because it doesn't look like we can easily mount a MEMFS dir inside a NODEFS dir. At least Google suggests we probably cannot easily do this because it would mean mixing contents of a real NODEFS dir with artificial mounts in that dir. Based on my recent looks into emscripten FS implementations, I found this reasoning compelling, but I haven't tested it.

OK, I guess we can forget that. In testing, it seems totally possible. I can mount a real dir as /wordpress and then mount another as /wordpress/wp-content on top of that.

brandonpayton · 2025-08-08T04:26:04Z

OK, I guess we can forget that. In testing, it seems totally possible. I can mount a real dir as /wordpress and then mount another as /wordpress/wp-content on top of that.

Ah, that wasn't mounting a MEMFS dir within a NODEFS dir, but I just tested that by editing a php_8_3.js file and mounting a MEMFS dir inside NODEFS like:

if (phpWasmInitOptions?.nativeInternalDirPath) {
	FS.mount(
		FS.filesystems.NODEFS,
		{ root: phpWasmInitOptions.nativeInternalDirPath },
		'/internal/shared'
	);
	FS.mkdir('/internal/shared/something');
	FS.mount(
		FS.filesystems.MEMFS,
		{ root: phpWasmInitOptions.nativeInternalDirPath },
		'/internal/shared/something'
	);
}

And there are no errors during boot. So this should all be doable. 👍

brandonpayton · 2025-08-08T04:29:40Z

Random thing, when I was thinking of adjusting the directory trees to workaround possibly mount challenges, I was thinking something like:

/wordpress - for WordPress
/internal - for files shared by entire php-wasm server
/request - for state that is unique to the current request and the single php-wasm instance that is handling it. stdout, stderr, and headers would go here.

@adamziel how does this sound to you? I kinda like it, at least compared to nesting non-shared things in a shared dir.

brandonpayton · 2025-08-08T04:30:10Z

I'm out of time today and plan to continue tomorrow.

brandonpayton · 2025-08-08T19:00:23Z

Also, we need to take this bit of the PHP class into account (in hotSwapPHPRuntime) – I think it already skips non-MEMFS nodes, but I'll still surface it here just in case:

wordpress-playground/packages/php-wasm/universal/src/lib/php.ts

Line 1279 in 89a7f30

// Copy the old /internal directory to the new filesystem

Cool. It looks like we already skip any FS node that is not detected as MEMFS:

wordpress-playground/packages/php-wasm/universal/src/lib/php.ts

Lines 1457 to 1461 in 533d676

    
           // MEMFS nodes have a `contents` property. NODEFS nodes don't. 
        
           // We only want to copy MEMFS nodes here. 
        
           if (!('contents' in oldNode.node)) { 
        
           	return; 
        
           }

adamziel · 2025-08-08T23:30:45Z

Random thing, when I was thinking of adjusting the directory trees to workaround possibly mount challenges, I was thinking something like:
/wordpress - for WordPress
/internal - for files shared by entire php-wasm server
/request - for state that is unique to the current request and the single php-wasm instance that is handling it. stdout, stderr, and headers would go here.
@adamziel how does this sound to you? I kinda like it, at least compared to nesting non-shared things in a shared dir.

Sounds good to me! I want to preserve the directory structure inside the /internal directory as I know at least Studio creates and reads files from there. stdout, stderr, and headers are not regular files, though. They're devices, and they're off limits – folks are not supposed to use those files directly. We can move them somewhere else just fine.

brandonpayton · 2025-08-10T04:51:01Z

Sounds good to me! I want to preserve the directory structure inside the /internal directory as I know at least Studio creates and reads files from there. ``

Ah! Thanks, that's good to know. I was tempted to get rid of /internal/shared since AFAICT all its contents would be shared between PHP processes now.

brandonpayton · 2025-08-10T04:56:31Z

I made changes to share both /wordpress and /internal dirs today, but something is broken. There seems to be a crash involving secondary PHP processes and/or runtime rotation. I spent a while debugging but haven't found the issue(s) yet.

Planning to continue debugging on Monday.

brandonpayton · 2025-08-10T04:56:35Z

Ideally, we can either get rid of proxyFileSystem() or find a way to contextualize what it does, as in uses PROXYFS, NODEFS, or other synchronization means in the future.

I've been thinking about this but have left it alone so far.

brandonpayton · 2025-08-11T18:31:26Z

Ideally, we can either get rid of proxyFileSystem() or find a way to contextualize what it does, as in uses PROXYFS, NODEFS, or other synchronization means in the future.

I've been thinking about this but have left it alone so far.

Funny. It turns out the bug I'm seeing is a failed attempt to proxy /internal since it is now already mounted during wasm init.

🤔 I wonder whether we can keep the same proxy paths but just make a proxying function that only proxies if there is not already a mount at that path.

On the other hand, it may be stronger and safer to require explicitly declaring shared/isolated paths.

brandonpayton · 2025-08-11T19:16:58Z

🤔 I wonder whether we can keep the same proxy paths but just make a proxying function that only proxies if there is not already a mount at that path.

On the other hand, it may be stronger and safer to require explicitly declaring shared/isolated paths.

Instead of this, I ended up only passing the native internal dir path to the primary PHP process. The secondary PHP processes can continue proxying to the primary PHP process /internal dir.

adamziel · 2025-08-11T21:02:05Z

Do we have to proxy at all if we can use native mounts? I suppose we do, at least in the web version, so maybe it makes sense to keep that logic consistent in the Node.js version 🤔

That's probably too big for this PR, but I wonder if we could decouple PHP and the Filesystem, as in have a FS-only WASM module to use as a source of truth, and proxy directories from both the "primary" and "secondary" PHP instances there. That might allow us to remove the distinction between the primary and secondary and use just identical workers.

brandonpayton · 2025-08-11T23:34:00Z

That's probably too big for this PR, but I wonder if we could decouple PHP and the Filesystem, as in have a FS-only WASM module to use as a source of truth, and proxy directories from both the "primary" and "secondary" PHP instances there.

That's worth considering!

That might allow us to remove the distinction between the primary and secondary and use just identical workers.

That sounds good.

Before this PR, we needed to pick an initial worker for running the Blueprint, run the Blueprint, and then copy the /internal results to the other workers. After this PR, even while applying the Blueprint during boot, there wouldn't need to be any special difference between the workers and PHP processes within the workers.

brandonpayton · 2025-08-11T23:36:02Z

@adamziel, I haven't really considered the /tmp dir yet. We are proxying it from secondary to primary PHP processes. Since we've found it worth proxying, is that another dir we should share across all PHP processes? I believe so but want to get your input.

brandonpayton · 2025-08-12T19:50:19Z

Some things that are left:

CI test failures
Proactive cleanup of orphaned temp dirs

Bonus:

Explore not using PROXYFS when sharing an FS via NODEFS

brandonpayton requested a review from a team July 31, 2025 06:33

brandonpayton self-assigned this Jul 31, 2025

brandonpayton added [Type] Exploration An exploration that may or may not result in mergable code [Package][@php-wasm] Node [Package][@wp-playground] CLI labels Jul 31, 2025

brandonpayton mentioned this pull request Aug 2, 2025

Playground CLI: Make additional worker initialization lazy #2420

Closed

brandonpayton marked this pull request as ready for review August 2, 2025 04:26

adamziel reviewed Aug 4, 2025

View reviewed changes

packages/php-wasm/compile/php/phpwasm-emscripten-library.js Show resolved Hide resolved

adamziel reviewed Aug 4, 2025

View reviewed changes

packages/php-wasm/compile/php/phpwasm-emscripten-library.js Show resolved Hide resolved

adamziel reviewed Aug 4, 2025

View reviewed changes

Use temp native dir for /wordpress and /internal

91dc4b0

brandonpayton force-pushed the playground-cli/try-making-internal-real-temp-dir branch from 28636f7 to 91dc4b0 Compare August 11, 2025 23:28

Experiment with native /internal/shared dir for Playground CLI #2446

Are you sure you want to change the base?

Experiment with native /internal/shared dir for Playground CLI #2446

Conversation

brandonpayton commented Jul 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation for the change, related issues

Implementation details

Testing Instructions (or ideally a Blueprint)

Uh oh!

brandonpayton commented Jul 31, 2025

Uh oh!

brandonpayton commented Jul 31, 2025

Uh oh!

brandonpayton commented Jul 31, 2025

Uh oh!

adamziel commented Jul 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

brandonpayton commented Jul 31, 2025

Uh oh!

brandonpayton commented Jul 31, 2025

Uh oh!

brandonpayton commented Aug 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

brandonpayton commented Aug 2, 2025

Uh oh!

Uh oh!

Uh oh!

adamziel commented Aug 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adamziel commented Aug 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adamziel commented Aug 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adamziel Aug 4, 2025

Choose a reason for hiding this comment

Uh oh!

brandonpayton commented Aug 7, 2025

Uh oh!

adamziel commented Aug 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

brandonpayton commented Aug 8, 2025

Uh oh!

brandonpayton commented Aug 8, 2025

Uh oh!

brandonpayton commented Aug 8, 2025

Uh oh!

brandonpayton commented Aug 8, 2025

Uh oh!

brandonpayton commented Aug 8, 2025

Uh oh!

brandonpayton commented Aug 8, 2025

Uh oh!

adamziel commented Aug 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

brandonpayton commented Aug 10, 2025

Uh oh!

brandonpayton commented Aug 10, 2025

Uh oh!

brandonpayton commented Aug 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

brandonpayton commented Aug 11, 2025

Uh oh!

brandonpayton commented Aug 11, 2025

Uh oh!

adamziel commented Aug 11, 2025

Uh oh!

brandonpayton commented Aug 11, 2025

Uh oh!

brandonpayton commented Aug 11, 2025

Uh oh!

brandonpayton commented Aug 12, 2025

brandonpayton commented Jul 31, 2025 •

edited

Loading

adamziel commented Jul 31, 2025 •

edited

Loading

brandonpayton commented Aug 2, 2025 •

edited

Loading

adamziel commented Aug 4, 2025 •

edited

Loading

adamziel commented Aug 4, 2025 •

edited

Loading

adamziel commented Aug 4, 2025 •

edited

Loading

adamziel commented Aug 7, 2025 •

edited

Loading

adamziel commented Aug 8, 2025 •

edited

Loading

brandonpayton commented Aug 10, 2025 •

edited

Loading