[SPLIT_MODULE] Make getWasmImports return Proxy #25559

aheejin · 2025-10-15T02:20:04Z

When SPLIT_MODULE is set, this makes the imports object returned by getWasmImports itself a Proxy, which redirects property requests starting with placeholder to an inner handler that processes the second module loading, and process all other property requests to the original imports object itself.

This is a prepration to add multi-split loading functionality to the JS runtime. In the multi-split mode, we don't have a single seconday module [modulename].deferred.wasm, but instead multiple secondary modules. We plan to have multiple placeholder namespace (e.g., placeholder2, placeholder3, ...) that match those multiple files, and this can't be hardcoded with one placeholder string as we currently do, so having the imports object itself as a Proxy provides us flexibility to handle multiple placeholder namespaces and load correct modules later.

Thanks @tlively for the idea!

When `SPLIT_MODULE` is set, this makes the `imports` object returned by `getWasmImports` itself a Proxy, which redirects property requests starting with `placeholder` to an inner `splitModuleProxyHandler` that processes the second module loading, and process all other property requests to the original `imports` object itself. This is a prepration to add multi-split loading functionality to the JS runtime. In the multi-split mode, we don't have a single seconday module `[modulename].deferred.wasm`, but instead multiple secondary modules. We plan to have multiple placeholder namespace (e.g., `placeholder_2`, `placeholder_3`, ...) that match those multiple files, and this can't be hardcoded with one `placeholder` string as we currently do, so having the `imports` object itself as a `Proxy` provides us flexibility to handle multiple placeholder namespaces and load correct modules later.

aheejin · 2025-10-15T02:20:26Z

"Hide whitespaces" will make the diff easier to view.

src/preamble.js

This is an automatic change generated by tools/maint/rebaseline_tests.py. The following (7) test expectation files were updated by running the tests with `--rebaseline`: ``` codesize/test_codesize_cxx_wasmfs.json: 176780 => 176780 [+0 bytes / +0.00%] test/codesize/test_codesize_file_preload.expected.js updated codesize/test_codesize_files_wasmfs.json: 55683 => 55683 [+0 bytes / +0.00%] codesize/test_codesize_hello_O0.json: 39197 => 39189 [-8 bytes / -0.02%] codesize/test_codesize_hello_dylink_all.json: 843553 => 843559 [+6 bytes / +0.00%] test/codesize/test_codesize_minimal_O0.expected.js updated codesize/test_unoptimized_code_size.json: 180236 => 180290 [+54 bytes / +0.03%] Average change: +0.00% (-0.02% - +0.03%) ```

aheejin · 2025-10-15T07:31:18Z

Not sure why the codesize test is failing... I think my local machine's rebaseline_tests.py result is different from the CI's. Any ideas?

kripken · 2025-10-15T16:43:07Z

Usually updating the emsdk (for LLVM and Binaryen) and merging in latest main here fixes those code size errors, for me.

sbc100 · 2025-10-15T17:36:56Z

src/preamble.js

-var splitModuleProxyHandler = {
-  get(target, prop, receiver) {
-    return (...args) => {
+var wasmImportsProxyHandler = {


Can we keep the splitModule prefix (or at least include it somewhere in the name) so its clear when this needed/used?

Renamed it to splitImportsProxyHandler: e918916

src/preamble.js

sbc100 · 2025-10-15T17:43:58Z

src/preamble.js

-      loadSplitModule(deferred, imports, prop);
-      err('instantiated deferred module, continuing');
+            // TODO: Implement multi-split module loading
+            err(`placeholder function called: ${base}`);


Pre-existing, but should this be dbg and wrapped in ASSERTIONS or somesuch?

Used dbg in two places and wrapped them in RUNTIME_DEBUG, given that the places where dbg() is used in this file are all seem to be wrapped in it: 4f2f3b3

aheejin · 2025-10-15T21:50:38Z

Usually updating the emsdk (for LLVM and Binaryen) and merging in latest main here fixes those code size errors, for me.

I updated both but still somehow does not match the CI's results....

aheejin · 2025-10-15T22:04:36Z

Is there a way to log in to the CI via ssh and run things? I used to an option that's called like "run via SSH" or something but I don't see it anymore.

Also the message very confusing. The CI failure message is like

The following (5) test expectation files were updated by
running the tests with `--rebaseline`:

codesize/test_codesize_cxx_wasmfs.json: 176780 => 176780 [+0 bytes / +0.00%]
codesize/test_codesize_files_wasmfs.json: 55683 => 55683 [+0 bytes / +0.00%]
codesize/test_codesize_hello_O0.json: 39197 => 39189 [-8 bytes / -0.02%]
codesize/test_codesize_hello_dylink_all.json: 843553 => 843559 [+6 bytes / +0.00%]
codesize/test_unoptimized_code_size.json: 180236 => 180212 [-24 bytes / -0.01%]

Average change: -0.01% (-0.02% - +0.00%)

But I already updated those files in this PR. For example, you can see this PR updates codesize/test_codesize_hello_O0.json's total line from 39197 to 39189. Why does this error out saying the size should be 39189?

sbc100 · 2025-10-15T23:23:31Z

The "Run via SSH" things only works for circleci.

Are you sure you did emsdk install tot and rebased your change onto main?

sbc100 · 2025-10-15T23:24:21Z

src/preamble.js

-  }
+  };
+#if SPLIT_MODULE
+  imports = new Proxy(imports, splitImportsProxyHandler);


How about splitModuleProxyHandler for the name?

That's the name of the inner handler... Do you want it to change to something else?

sbc100 · 2025-10-15T23:25:13Z

src/preamble.js

+          return (...args) => {
 #if ASYNCIFY == 2
-      throw new Error('Placeholder function "' + prop + '" should not be called when using JSPI.');
+            throw new Error('Placeholder function "' + base + '" should not be called when using JSPI.');


(As an aside I wonder if this line should be an assert).

Not sure... Can't this be triggered by user errors? If so an Error might be fine?

This reverts commit b2a22d4.

aheejin · 2025-10-15T23:30:10Z

Sorry, reverted an accidental change that was not meant to be in this PR: 68c3308

sbc100 · 2025-10-15T23:35:31Z

src/preamble.js

+var splitImportsProxyHandler = {
+  get(target, moduleName, receiver) {
+    if (moduleName.startsWith('placeholder')) {
+      var splitModuleProxyHandler = {


Just call this one innerHandler then maybe?

This inner name is less important since its not in the global namespace. It could even just be inlined (anonymous) bat that could make harder to grok maybe?

Renamed the outer handler to splitModuleProxyHandler and the inner one to innerHandler: eaea168

The inner handler name is used here, so now sure how to anonymize it.

return new Proxy({}, innerHandler);

I mean you could just write return new Proxy({}, { ... inner handler code }

Done: 67e5380

This is an automatic change generated by tools/maint/rebaseline_tests.py. The following (5) test expectation files were updated by running the tests with `--rebaseline`: ``` codesize/test_codesize_cxx_wasmfs.json: 176780 => 176780 [+0 bytes / +0.00%] codesize/test_codesize_files_wasmfs.json: 55683 => 55683 [+0 bytes / +0.00%] codesize/test_codesize_hello_O0.json: 39189 => 39197 [+8 bytes / +0.02%] codesize/test_codesize_hello_dylink_all.json: 843559 => 843553 [-6 bytes / -0.00%] codesize/test_unoptimized_code_size.json: 180290 => 180314 [+24 bytes / +0.01%] Average change: +0.01% (-0.00% - +0.02%) ```

aheejin · 2025-10-16T00:02:49Z

I removed (NFC) from the title because it changes some behaviors now (like, wrapping debug messages in RUNTIME_DEBUG)

This is an automatic change generated by tools/maint/rebaseline_tests.py. The following (5) test expectation files were updated by running the tests with `--rebaseline`: ``` codesize/test_codesize_cxx_wasmfs.json: 176780 => 176780 [+0 bytes / +0.00%] codesize/test_codesize_files_wasmfs.json: 55683 => 55683 [+0 bytes / +0.00%] codesize/test_codesize_hello_O0.json: 39197 => 39189 [-8 bytes / -0.02%] codesize/test_codesize_hello_dylink_all.json: 843553 => 843559 [+6 bytes / +0.00%] codesize/test_unoptimized_code_size.json: 180314 => 180290 [-24 bytes / -0.01%] Average change: -0.01% (-0.02% - +0.00%) ```

sbc100 · 2025-10-16T03:19:22Z

I just landed #25566 which should help with the codesize issues.

aheejin · 2025-10-16T07:11:56Z

Thanks, but it still fails with the codesize checks... I think the wasm sizes are different and have no clue why. I also tried emsdk tot but it failed too. Maybe I should ask someone tomorrow to run the codesize script with my PR and compare that with my results.

sbc100 · 2025-10-16T13:08:44Z

Thanks, but it still fails with the codesize checks... I think the wasm sizes are different and have no clue why. I also tried emsdk tot but it failed too. Maybe I should ask someone tomorrow to run the codesize script with my PR and compare that with my results.

The first thing to do is make sure you can run tools/maint/rebaseline_tests.py locally on upstream/main and get no chnages.

If you have changes in this case then either:

main it self is out-of-date
You need to install the latest emsdk
You need to run --clear-cache

Maybe you are getting bitten by (3)

This is an automatic change generated by tools/maint/rebaseline_tests.py. The following (1) test expectation files were updated by running the tests with `--rebaseline`: ``` codesize/test_unoptimized_code_size.json: 180212 => 180290 [+78 bytes / +0.04%] Average change: +0.04% (+0.04% - +0.04%) ```

aheejin · 2025-10-16T16:26:56Z

I thought I did all three, but now I do it again, somehow I again see changes... Let's see if this time it works 🙏

aheejin · 2025-10-16T16:53:35Z

Codesize checks succeeded! Thanks!

aheejin requested a review from tlively October 15, 2025 02:20

tlively approved these changes Oct 15, 2025

View reviewed changes

src/preamble.js Outdated Show resolved Hide resolved

aheejin added 5 commits October 15, 2025 05:58

Address a comment

6a30a50

namespace -> env

3c5e7ad

Rename prop -> base

3ddcbd1

[SPLIT_MODULE] Support multi-split loading

b2a22d4

sbc100 reviewed Oct 15, 2025

View reviewed changes

aheejin added 3 commits October 15, 2025 21:43

wasmImportsProxyHandler -> splitImportsProxyHandler

e918916

env -> moduleName

6cb0c42

Wrap dbg messages within RUNTIME_DEBUG

4f2f3b3

Merge branch 'main' into nested_proxies

5f2bc5f

sbc100 reviewed Oct 15, 2025

View reviewed changes

sbc100 approved these changes Oct 15, 2025

View reviewed changes

Revert "[SPLIT_MODULE] Support multi-split loading"

68c3308

This reverts commit b2a22d4.

sbc100 reviewed Oct 15, 2025

View reviewed changes

aheejin added 4 commits October 15, 2025 23:36

Rename handlers

eaea168

Fix errors from previous revert ;(

3919476

Anonymize the inner handler

67e5380

aheejin changed the title ~~[SPLIT_MODULE] Make getWasmImports return Proxy (NFC)~~ [SPLIT_MODULE] Make getWasmImports return Proxy Oct 16, 2025

aheejin added 2 commits October 16, 2025 00:05

Remove a newline added by mistake

13f034d

sbc100 approved these changes Oct 16, 2025

View reviewed changes

Merge branch 'main' into nested_proxies

ce9f8c8

aheejin added 2 commits October 16, 2025 16:23

Merge branch 'main' into nested_proxies

6d395fb

aheejin merged commit 292321d into emscripten-core:main Oct 16, 2025
34 checks passed

aheejin deleted the nested_proxies branch October 16, 2025 17:43

[SPLIT_MODULE] Make getWasmImports return Proxy #25559

[SPLIT_MODULE] Make getWasmImports return Proxy #25559

Conversation

aheejin commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aheejin commented Oct 15, 2025

Uh oh!

Uh oh!

aheejin commented Oct 15, 2025

Uh oh!

kripken commented Oct 15, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aheejin Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aheejin commented Oct 15, 2025

Uh oh!

aheejin commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sbc100 commented Oct 15, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aheejin commented Oct 15, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aheejin Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aheejin commented Oct 16, 2025

Uh oh!

sbc100 commented Oct 16, 2025

Uh oh!

aheejin commented Oct 16, 2025

Uh oh!

sbc100 commented Oct 16, 2025

Uh oh!

aheejin commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aheejin commented Oct 16, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

aheejin commented Oct 15, 2025 •

edited

Loading

aheejin Oct 15, 2025 •

edited

Loading

aheejin commented Oct 15, 2025 •

edited

Loading

aheejin Oct 15, 2025 •

edited

Loading

aheejin commented Oct 16, 2025 •

edited

Loading