pyodide build regression, version flags breaking things?

[AndrewAnnex]

Description

I am trying to debug a pyodide build of my library which I had working with tests in a test repo about 3 weeks ago, but am beginning to apply to my main project, however as of an hour ago, and without changing much I seem to get a regression where cibuildwheel is passing a "-v" flag to pyodide build causing a fatal build error (I could get far past this point yesterday/earlier this morning), I think my only major change was switching from manually calling cibuildwheel in my github action, using the latest version of cibuildwheel as I never pinned it in my pyproject.toml, to switching to 3.1.4 and the github action command.

example partial log:

Building cp312-pyodide_wasm32 wheel
CPython 3.12 Pyodide

Installing a base python environment...
                                                              ✓ 3.20s
Setting up build environment...
                                                              ✓ 2.50s
Installing build tools...
                                                              ✓ 4.92s
Installing Emscripten version: 3.1.58 ...
                                                             ✓ 31.44s
Installing Pyodide xbuildenv version: 0.27.7 ...
                                                              ✓ 8.88s
Building wheel...
  
  + pyodide build /home/runner/work/SpiceyPy/SpiceyPy --outdir=/tmp/cibw-run-i6d6ci8v/cp312-pyodide_wasm32/built_wheel -v
  ERROR: Unrecognized options in config-settings:
    -v
  
  ERROR Backend subprocess exited when trying to invoke build_wheel
                                                              ✕ 2.68s
Error: cibuildwheel: Command ['/tmp/cibw-run-i6d6ci8v/cp312-pyodide_wasm32/build/venv/bin/pyodide', 'build', '/home/runner/work/SpiceyPy/SpiceyPy', '--outdir=/tmp/cibw-run-i6d6ci8v/cp312-pyodide_wasm32/built_wheel', '-v'] failed with code 1. 

Error: Process completed with exit code 1.

I get the same issue building locally in a ubuntu vm.

I saw that 3.2.0 was recently published, so I am wondering if maybe there is some other build version mis match, plus I am specifying in my pyproject.toml to only build pyodide version 0.27.7, as that is the one jupyterlite uses (or used 3 weeks ago last I checked, using the main live deploy of jupyterlite is desirable for me).

Looking at some older logs, it looks like maybe this could be a regression from 3.1.3 to 3.1.4, where I don't see the -v flag being used in the first step. But also pyodide 0.30.5 is being used.

I've attached two logs, one where the build proceeds (although there is another issue with compilation I am debugging in it) and the failed one, so hoping to spot more differences there.

working_build_cibuildwheel_log.txt

failed_build_cibuildwheel_log.txt

Build log

No response

CI config

https://github.com/AndrewAnnex/SpiceyPy/blob/add_pyodide_build/.github/workflows/publish-to-test-and-live-pypi.yml

[AndrewAnnex]

hmm the working log seems to be using pyodide-build-0.30.5 pyodide-cli-0.3.0 pyodide-lock-0.1.0a7, the failed one pyodide-build-0.30.7 pyodide-cli-0.3.0 pyodide-lock-0.1.0. Both were using cibuildwheel 3.2.0 oddly, so I think really me specifying the build numbers was an attempt to make sure there wasn't an issue with that and I just picked the prior releases assuming it was a recent regression.

maybe I need to pin the pyodide build version?

[AndrewAnnex]

ahh wait a second, in my pyproject.toml for the pyodide section I had specified "build-verbosity = 2". It looks like setting it to 0 avoids passing "-v" in through *extra_flags in pyodide.py?

[agriyakhetarpal]

ahh wait a second, in my pyproject.toml for the pyodide section I had specified "build-verbosity = 2". It looks like setting it to 0 avoids passing "-v" in through *extra_flags in pyodide.py?

Hi! Yes, we don't currently support build verbosity. There is an open issue about it, though: pyodide/pyodide-build#222

[AndrewAnnex]

okay @agriyakhetarpal I'll close this issue for now, although maybe cibuildwheel shouldn't pass unsupported flags to underlying tools so maybe a change to strip the verbose flag when building pyodide wheels would be needed, or maybe a warning or something could be put in place, or maybe just doc updates to warn about it. Not sure what the preferred pattern for this project would be for that though, but if one was recommended I could contribute that

[agriyakhetarpal]

Hmm, I think the place to modify the behaviour for stripping verbose flags would be somewhere around

cibuildwheel/cibuildwheel/frontend.py

Lines 39 to 53 in c0c1dea

	def _get_verbosity_flags(level: int, frontend: BuildFrontendName) -> list[str]:
	if level < 0:
	if frontend == "pip":
	return ["-" + -level * "q"]
	msg = f"build_verbosity {level} is not supported for {frontend} frontend. Ignoring."
	log.warning(msg)
	if level > 0:
	if frontend == "pip":
	return ["-" + level * "v"]
	if level > 1:
	return ["-" + (level - 1) * "v"]
	return []

However, the issue I see here is that we don't immediately know of a way to distinguish the build frontend between pyodide-build and pypa/build. pyodide-build as a build frontend is a thin wrapper over pypa/build APIs with some extra handling for special cases: https://github.com/pyodide/pyodide-build/blob/bdd0b1422064de6d92264a08c599e85001a28567/pyodide_build/pypabuild.py. Perhaps we should check if "PYODIDE" in os.environ is True, and use that as our source of truth, or will that be too hacky? Maybe @ryanking13 has some insight into this.

[joerick]

I think, from cibuildwheel's point of view, we shouldn't be crashing out like this just due to a build-verbosity flag being set. We know the build is pyodide here-

https://github.com/pypa/cibuildwheel/blob/main/cibuildwheel/platforms/pyodide.py#L418

So we could thread through a parameter to those functions (e.g. frontend_variant: Literal["pyodide"] | None) to make sure the output is right.

Of course, if it can be supported in pyodide-build instead, that would be better :)

[agriyakhetarpal]

Thanks, that sounds like a good idea, @joerick! I don't think my path forward in #2609 is right so far, so I'm happy to try yours.

[ryanking13]

Thanks for the ping. Maybe let's support the verbose flag in pyodide-build. I guess it won't be very complex? I think we can override VERBOSITY variable (though it is not a public API).

[agriyakhetarpal]

Yes, I think it might be a more straightforward solution than handling it in cibuildwheel. The question, I think, is more around which parts of our code need to be verbose or would benefit from the verbosity.

[ryanking13]

The question, I think, is more around which parts of our code need to be verbose or would benefit from the verbosity.

Yes, we can think about that too. We do have a logger with loglevel, so maybe we can update the loglevel when the verbose parameter is passed.