feat: Build and test against Pyodide 0.29 (new pyodide_2025_0 ABI)

[agriyakhetarpal]

This PR updates the Pyodide/Emscripten builds (both debug and release mode) to use Emscripten 4.0.9. Previously, the Emscripten version was set to latest, which caused the wheels to be built against Emscripten 4.0.12 – a version later than the one used for the pyodide_2025_0 ABI introduced in Pyodide 0.28. I've used cibuildwheel as the driver for these builds.

This patch from the Pyodide source tree: https://github.com/pyodide/pyodide/blob/0.28.1/emsdk/patches/0003-dylink-Fix-rpath-calculation-in-nested-dependencies.patch is required for Emscripten, as the fix for the Rust symbols exports from side modules being reported as invalid at emscripten-core/emscripten#24427 was not incorporated at the time when finalising the pyodide_2025_0 ABI.

cc: @ryanking13 @hoodmane

xref pyodide/pyodide-recipes#99, marimo-team/marimo#5995, jupyterlite/jupyterlite#1576 (comment)

[agriyakhetarpal]

The error from the CI logs is:

note: emcc: error: invalid export name: _ZN100_$LT$$RF$mut$u20$rmp_serde..decode..Deserializer$LT$R$C$C$GT$$u20$as$u20$serde..de..Deserializer$GT$14deserialize_i817h283228b53233da2fE

[agriyakhetarpal]

Trying this locally, it looks like the mio crate does not support the WASM target:

error: This wasm target is unsupported by mio. If using Tokio, disable the net feature.
  --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/lib.rs:44:1
   |
44 | compile_error!("This wasm target is unsupported by mio. If using Tokio, disable the net feature.");
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0583]: file not found for module `selector`
  --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/sys/unix/mod.rs:53:5
   |
53 |     mod selector;
   |     ^^^^^^^^^^^^^
   |
   = help: to create the module `selector`, create file "/Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/sys/unix/selector.rs" or "/Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/sys/unix/selector/mod.rs"
   = note: if there is a `mod selector` elsewhere in the crate already, import it with `use crate::...` instead

error[E0583]: file not found for module `waker`
   --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/sys/unix/mod.rs:105:5
    |
105 |     mod waker;
    |     ^^^^^^^^^^
    |
    = help: to create the module `waker`, create file "/Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/sys/unix/waker.rs" or "/Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/sys/unix/waker/mod.rs"
    = note: if there is a `mod waker` elsewhere in the crate already, import it with `use crate::...` instead

   Compiling thiserror v2.0.12
error[E0432]: unresolved import `crate::sys::IoSourceState`
  --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/io_source.rs:14:5
   |
14 | use crate::sys::IoSourceState;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^ no `IoSourceState` in `sys`

error[E0433]: failed to resolve: could not find `Selector` in `sys`
   --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/poll.rs:322:18
    |
322 |             sys::Selector::new().map(|selector| Poll {
    |                  ^^^^^^^^ could not find `Selector` in `sys`

error[E0433]: failed to resolve: could not find `event` in `sys`
  --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/event/event.rs:24:14
   |
24 |         sys::event::token(&self.inner)
   |              ^^^^^ could not find `event` in `sys`

error[E0433]: failed to resolve: could not find `event` in `sys`
  --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/event/event.rs:38:14
   |
38 |         sys::event::is_readable(&self.inner)
   |              ^^^^^ could not find `event` in `sys`

error[E0433]: failed to resolve: could not find `event` in `sys`
  --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/event/event.rs:43:14
   |
43 |         sys::event::is_writable(&self.inner)
   |              ^^^^^ could not find `event` in `sys`

error[E0433]: failed to resolve: could not find `event` in `sys`
  --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/event/event.rs:68:14
   |
68 |         sys::event::is_error(&self.inner)
   |              ^^^^^ could not find `event` in `sys`

error[E0433]: failed to resolve: could not find `event` in `sys`
  --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/event/event.rs:99:14
   |
99 |         sys::event::is_read_closed(&self.inner)
   |              ^^^^^ could not find `event` in `sys`

error[E0433]: failed to resolve: could not find `event` in `sys`
   --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/event/event.rs:129:14
    |
129 |         sys::event::is_write_closed(&self.inner)
    |              ^^^^^ could not find `event` in `sys`

error[E0433]: failed to resolve: could not find `event` in `sys`
   --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/event/event.rs:151:14
    |
151 |         sys::event::is_priority(&self.inner)
    |              ^^^^^ could not find `event` in `sys`

error[E0433]: failed to resolve: could not find `event` in `sys`
   --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/event/event.rs:173:14
    |
173 |         sys::event::is_aio(&self.inner)
    |              ^^^^^ could not find `event` in `sys`

error[E0433]: failed to resolve: could not find `event` in `sys`
   --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/event/event.rs:183:14
    |
183 |         sys::event::is_lio(&self.inner)
    |              ^^^^^ could not find `event` in `sys`

error[E0433]: failed to resolve: could not find `event` in `sys`
   --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/event/event.rs:221:26
    |
221 |                     sys::event::debug_details(f, self.0)
    |                          ^^^^^ could not find `event` in `sys`

error[E0412]: cannot find type `Selector` in module `sys`
   --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/poll.rs:273:20
    |
273 |     selector: sys::Selector,
    |                    ^^^^^^^^ not found in `sys`

error[E0412]: cannot find type `Selector` in module `sys`
   --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/poll.rs:730:44
    |
730 |     pub(crate) fn selector(&self) -> &sys::Selector {
    |                                            ^^^^^^^^ not found in `sys`

error[E0425]: cannot find value `stream` in this scope
   --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/sys/unix/tcp.rs:135:58
    |
135 |     unsafe { to_socket_addr(addr.as_ptr()) }.map(|addr| (stream, addr))
    |                                                          ^^^^^^ not found in this scope

error[E0412]: cannot find type `Waker` in module `sys`
  --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/waker.rs:79:17
   |
79 |     inner: sys::Waker,
   |                 ^^^^^ not found in `sys`
   |
help: consider importing one of these structs
   |
1  + use crate::Waker;
   |
1  + use std::task::Waker;
   |
help: if you import `Waker`, refer to it directly
   |
79 -     inner: sys::Waker,
79 +     inner: Waker,
   |

error[E0433]: failed to resolve: could not find `Waker` in `sys`
  --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/waker.rs:87:14
   |
87 |         sys::Waker::new(registry.selector(), token).map(|inner| Waker { inner })
   |              ^^^^^ could not find `Waker` in `sys`
   |
help: consider importing one of these structs
   |
1  + use crate::Waker;
   |
1  + use std::task::Waker;
   |
help: if you import `Waker`, refer to it directly
   |
87 -         sys::Waker::new(registry.selector(), token).map(|inner| Waker { inner })
87 +         Waker::new(registry.selector(), token).map(|inner| Waker { inner })
   |

error[E0412]: cannot find type `Event` in module `sys`
  --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/event/event.rs:18:17
   |
18 |     inner: sys::Event,
   |                 ^^^^^ not found in `sys`
   |
help: consider importing this struct through its public re-export
   |
1  + use crate::event::Event;
   |
help: if you import `Event`, refer to it directly
   |
18 -     inner: sys::Event,
18 +     inner: Event,
   |

error[E0412]: cannot find type `Event` in module `sys`
   --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/event/event.rs:187:55
    |
187 |     pub(crate) fn from_sys_event_ref(sys_event: &sys::Event) -> &Event {
    |                                                       ^^^^^ not found in `sys`
    |
help: consider importing this struct through its public re-export
    |
1   + use crate::event::Event;
    |
help: if you import `Event`, refer to it directly
    |
187 -     pub(crate) fn from_sys_event_ref(sys_event: &sys::Event) -> &Event {
187 +     pub(crate) fn from_sys_event_ref(sys_event: &Event) -> &Event {
    |

error[E0412]: cannot find type `Event` in module `sys`
   --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/event/event.rs:191:41
    |
191 |             &*(sys_event as *const sys::Event as *const Event)
    |                                         ^^^^^ not found in `sys`
    |
help: consider importing this struct through its public re-export
    |
1   + use crate::event::Event;
    |
help: if you import `Event`, refer to it directly
    |
191 -             &*(sys_event as *const sys::Event as *const Event)
191 +             &*(sys_event as *const Event as *const Event)
    |

error[E0412]: cannot find type `Event` in module `sys`
   --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/event/event.rs:217:46
    |
217 |             struct EventDetails<'a>(&'a sys::Event);
    |                                              ^^^^^ not found in `sys`
    |
help: consider importing this struct through its public re-export
    |
1   + use crate::event::Event;
    |
help: if you import `Event`, refer to it directly
    |
217 -             struct EventDetails<'a>(&'a sys::Event);
217 +             struct EventDetails<'a>(&'a Event);
    |

error[E0412]: cannot find type `Events` in module `sys`
  --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/event/events.rs:43:17
   |
43 |     inner: sys::Events,
   |                 ^^^^^^ not found in `sys`
   |
help: consider importing this struct through its public re-export
   |
1  + use crate::Events;
   |
help: if you import `Events`, refer to it directly
   |
43 -     inner: sys::Events,
43 +     inner: Events,
   |

error[E0433]: failed to resolve: could not find `Events` in `sys`
  --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/event/events.rs:94:25
   |
94 |             inner: sys::Events::with_capacity(capacity),
   |                         ^^^^^^ could not find `Events` in `sys`
   |
help: consider importing this struct through its public re-export
   |
1  + use crate::Events;
   |
help: if you import `Events`, refer to it directly
   |
94 -             inner: sys::Events::with_capacity(capacity),
94 +             inner: Events::with_capacity(capacity),
   |

error[E0412]: cannot find type `Events` in module `sys`
   --> /Users/agriyakhetarpal/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/mio-1.0.4/src/event/events.rs:189:47
    |
189 |     pub(crate) fn sys(&mut self) -> &mut sys::Events {
    |                                               ^^^^^^ not found in `sys`
    |
help: consider importing this struct through its public re-export
    |
1   + use crate::Events;
    |
help: if you import `Events`, refer to it directly
    |
189 -     pub(crate) fn sys(&mut self) -> &mut sys::Events {
189 +     pub(crate) fn sys(&mut self) -> &mut Events {
    |

   Compiling utf8_iter v1.0.4
   Compiling zerofrom v0.1.6
Some errors have detailed explanations: E0412, E0425, E0432, E0433, E0583.
For more information about an error, try `rustc --explain E0412`.
error: could not compile `mio` (lib) due to 27 previous errors
warning: build failed, waiting for other jobs to finish...
💥 maturin failed

[agriyakhetarpal]

The same problem persists for the pyodide_2025_0 ABI, i.e., with the uvx cibuildwheel --only cp313-pyodide_wasm32 py-polars command.

[agriyakhetarpal]

The error is different this time:

note: emcc: error: invalid export name: _ZN102_$LT$core..iter..adapters..map..Map$LT$I$C$F$GT$$u20$as$u20$core..iter..traits..iterator..Iterator$GT$4next17h00a0e8925fcd653dE

I wonder if we're hitting up some Rust symbol mangling that is creating very long, complex symbol names that contain characters Emscripten doesn't accept.

[agriyakhetarpal]

9 minutes in so far and no failure, still building – looks like https://github.com/pyodide/rust-emscripten-wasm-eh-sysroot/ is doing its thing! I suppose we also need to pass the flags to the build backend, i.e., --profile dev --manifest-path py-polars/Cargo.toml --interpreter python3.X, but I'm waiting on the outcome of the job before I add them!

[agriyakhetarpal]

Ah, it failed with an illegal instruction :(

[agriyakhetarpal]

Okay, progress. We now have a different error, and it occurs at link time: error: linking with emcc failed: exit status: 1

[agriyakhetarpal]

I'm now experimenting with passing the --manifest-path and --profile flags to the maturin pep517 build-wheel command again, as it recognises the build frontend's --config-settings argument.

[agriyakhetarpal]

🤔 now we're at a similar error as before, though slightly different, and there is a very long symbol being produced/mangled:

emcc: error: invalid export name: _ZN11polars_core5frame6column6scalar10serde_impl1_113_$LT$impl$u20$serde..ser..Serialize$u20$for$u20$polars_core..frame..column..scalar..serde_impl..SerializeWrap$GT$9serialize17h4872ac3118e55689E

[ryanking13]

🤔 now we're at a similar error as before, though slightly different, and there is a very long symbol being produced/mangled:

emcc: error: invalid export name: _ZN11polars_core5frame6column6scalar10serde_impl1_113_$LT$impl$u20$serde..ser..Serialize$u20$for$u20$polars_core..frame..column..scalar..serde_impl..SerializeWrap$GT$9serialize17h4872ac3118e55689E

This one is becuase of Emscripten: https://github.com/pyodide/pyodide/blob/main/emsdk/patches/0002-Don-t-check-exports-for-being-valid-C-C-identifiers-.patch

We need to patch Emscripten with that patch.

Latest version of Emscripten fixed it, but it is not included in the Emscripten 4.0.9 that we are using in Pyodide 2025_0 ABI (emscripten-core/emscripten#24427)

[coastalwhite]

For the MIO error, also see #22231.

[agriyakhetarpal]

Thanks for the pointers!

[eitsupi]

You may want to see #26484

[agriyakhetarpal]

Thanks @eitsupi! I've gotten most of the test suite here to pass locally for the Pyodide/WASM build, though 10-11K tests are skipped (these are all for features already disabled in the WASM build), leaving around 34 failures that can be investigated or skipped. I think I'll wait for tokio to be made optional and rebase if/when your PR is merged.

[agriyakhetarpal]

I am a bit confused because the workflow isn't running here despite being defined – is it perhaps because it's disabled in the repository? At least I could leave this in a closely reviewable state if the Polars maintainers were to reconsider WASM support at a later time, now that #26484 has been closed.

[fzumstein]

I (i.e., Claude Code) managed to create a working build for Polars 1.33.1, see https://github.com/xlwings/polars-pyodide. I made a PR to include it in pyodide-recipes, but happy to hand over the build repository to whoever should/wants to maintain this going forward.