Fixed wheels issues with Python 3.14 + improvements

[jakub-kocka]

Description

• Added wheels to the exclude_list that can not be built under Python 3.14
• Added normalize function for wheels before the upload so Espressif's PyPI has the correct naming corresponding with PEP503
• Fixed wheels directory resolving (starts at the beginning and adds every character (even dots) until the version) - fixes, for example, ruamel.yaml.clib-0.2.12-cp313-cp313-linux_armv7l.whl resolution
• Enhanced build with --no-build-isolation tag (now all the packages are built in the Action environment, not its own - speeding the process and resolving, for example, xcffi MacOS issue)
• Switched from macos-13 to macos-15 for Intel CPU runners

Related

• Failing workflow: https://github.com/espressif/idf-python-wheels/actions/runs/18513415864/job/52761135855
• Closes internal trackers IDF-12693, IDF-12706

Testing

• Workflow: https://github.com/espressif/idf-python-wheels/actions/runs/19035772460
• Upload testing in "defined wheels" workflow: https://github.com/espressif/idf-python-wheels/actions/runs/18744046289/job/53471001823

---

Checklist

Before submitting a Pull Request, please ensure the following:

• 🚨 This PR does not introduce breaking changes.
• All CI checks (GH Actions) pass.
• Documentation is updated as needed.
• Tests are updated or added as necessary.
• Code is well-commented, especially in complex areas.
• Git history is clean — commits are squashed to the minimum necessary.

[github-actions]

	Messages
📖	You might consider squashing your 3 commits (simplifying branch history).

👋 Hello jakub-kocka, we appreciate your contribution to this project!

---

Click to see more instructions ...

This automated output is generated by the PR linter DangerJS, which checks if your Pull Request meets the project's requirements and helps you fix potential issues.

DangerJS is triggered with each push event to a Pull Request and modify the contents of this comment.

Please consider the following:
- Danger mainly focuses on the PR structure and formatting and can't understand the meaning behind your code or changes.
- Danger is not a substitute for human code reviews; it's still important to request a code review from your colleagues.
- Addressing info messages (📖) is strongly recommended; they're less critical but valuable.
- To manually retry these Danger checks, please navigate to the Actions tab and re-run last Danger workflow.

Review and merge process you can expect ...

We do welcome contributions in the form of bug reports, feature requests and pull requests.

1. An internal issue has been created for the PR, we assign it to the relevant engineer.
2. They review the PR and either approve it or ask you for changes or clarifications.
3. Once the GitHub PR is approved we do the final review, collect approvals from core owners and make sure all the automated tests are passing.
- At this point we may do some adjustments to the proposed change, or extend it by adding tests or documentation.
4. If the change is approved and passes the tests it is merged into the default branch.

Generated by 🚫 dangerJS against 7130418

[jakub-kocka]

Opened issue about windows-curses not supporting Python 3.14 zephyrproject-rtos/windows-curses#76

[fhrbata]

@jakub-kocka I noticed that the https://dl.espressif.com/pypi/ruamel-yaml-clib/ once again contains packages built with newer versions of setuptools, which use normalized names. For instance ruamel_yaml_clib-0.2.14-cp39-cp39-linux_armv7l.whl. These packages appear to have been uploaded via this action. I believe the presence of these packages may reintroduce issues similar to the one described in espressif/esp-idf#15713. As discussed and explained in espressif/esp-idf@3bad434, these packages will not be recognized by our check_python_dependencies.py script before the fix when running with python versions earlier than 3.10, causing the check to fail. We have backported the fix, but the already released versions do not include it.

I believe this should be handled, probably by not creating such wheels for python < 3.10 or to make sure these are created by setuptools < 75.8.1. It's been some time since I was looking into this, so I may be missing something.

Otherwise LGTM, my only worry is about reintroducing problem with the ruamel.yaml.clib package.

[jakub-kocka]

@jakub-kocka I noticed that the https://dl.espressif.com/pypi/ruamel-yaml-clib/ once again contains packages built with newer versions of setuptools, which use normalized names. For instance ruamel_yaml_clib-0.2.14-cp39-cp39-linux_armv7l.whl. These packages appear to have been uploaded via this action. I believe the presence of these packages may reintroduce issues similar to the one described in espressif/esp-idf#15713. As discussed and explained in espressif/esp-idf@3bad434, these packages will not be recognized by our check_python_dependencies.py script before the fix when running with python versions earlier than 3.10, causing the check to fail. We have backported the fix, but the already released versions do not include it.

I believe this should be handled, probably by not creating such wheels for python < 3.10 or to make sure these are created by setuptools < 75.8.1. It's been some time since I was looking into this, so I may be missing something.

Otherwise LGTM, my only worry is about reintroducing problem with the ruamel.yaml.clib package.

@fhrbata, there is a lot of forms in which is this package uploaded, as it can be seen here https://github.com/espressif/idf-python-wheels/actions/runs/18129873715/job/51597185640#step:5:1

I am not sure if I understand this correctly. The issue is with the normalized name of the package in the binary/wheel name?

• normalized_package_name-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl
• original.package.name-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl

So the first bullet point can't be resolved in the releases before the fix? If there is a variety of packages with and without the name normalization, will it be OK, when pip chooses?

[fhrbata]

@jakub-kocka

@fhrbata, there is a lot of forms in which is this package uploaded, as it can be seen here https://github.com/espressif/idf-python-wheels/actions/runs/18129873715/job/51597185640#step:5:1

These are build with older setuptools, which keep the dot in the distribution name, and since there is a bug in the upload script they ended up in the clib directory. IOW they are not visible to pip. The package I mentioned is in ruamel-yaml-clib and was uploaded by the action I mentioned earlier. You can take a look at the WHEEL medatata. For example:

https://dl.espressif.com/pypi/clib/ruamel.yaml.clib-0.2.14-cp312-cp312-manylinux2014_aarch64.whl

ruamel.yaml.clib-0.2.14.dist-info/WHEEL
Generator: bdist_wheel (0.43.0)

vs

https://dl.espressif.com/pypi/ruamel-yaml-clib/ruamel_yaml_clib-0.2.14-cp39-cp39-linux_armv7l.whl

ruamel_yaml_clib-0.2.14.dist-info/WHEEL
Generator: setuptools (80.9.0)

The problem is IMHO described in detail in espressif/esp-idf@3bad434 which I pointed out earlier.

So the first bullet point can't be resolved in the releases before the fix? If there is a variety of packages with and without the name normalization, will it be OK, when pip chooses?

IMHO it's not about the pip or this fix for normalized project names. The problem is that with this fix you will IMHO reintroduce the problem described in espressif/esp-idf#15713, unless we exclude ruamel.yaml.clib wheel from building or build it with older setuptools. TBH I think with the already uploaded wheels the problem is already there again. The problem is with the check_python_dependencies.py, which uses importlib.metadata to get package version from the wheel metadata. If the wheel is generated with newer setuptools with normalized name and older importlib.metadata<py3.10 is used, the get_version will fail with importlib.metadata.PackageNotFoundError: ruamel.yaml.clib

[jakub-kocka]

@fhrbata,
Ah, okay, hope I am starting to catch up, but still not fully. I will need to think about this a bit more and maybe come up with a solution that will be added in this PR.

TBH I think with the already uploaded wheels the problem is already there again.

IMHO, there is no possible way the action you have mentioned could have uploaded the directory with wheels. The bug in the upload script ended up in the clib directory. Which is still the thing, since it is still using the master branch script for uploading and the fix is being introduced just now.

The packages uploaded are as follows: e.g., for ruamel_yaml_clib-0.2.14-cp310-cp310-linux_armv7l.whl

Metadata-Version: 2.4
Name: ruamel.yaml.clib
Version: 0.2.14

Wheel-Version: 1.0
Generator: setuptools (80.9.0)
Root-Is-Purelib: false
Tag: cp310-cp310-linux_armv7l

[fhrbata]

@jakub-kocka

IMHO, there is no possible way the action you have mentioned could have uploaded the directory with wheels. The bug in the upload script ended up in the clib directory. Which is still the thing, since it is still using the master branch script for uploading and the fix is being introduced just now.

Hm, I may be reading it wrong, but I can see https://github.com/espressif/idf-python-wheels/actions/runs/18129873715. This was started by you 3 weeks ago manually with this branch. And I can see the ruamel_yaml_clib package being uploaded in that action https://github.com/espressif/idf-python-wheels/actions/runs/18129873715/job/51597185640. At least that's my understanding. Moreover we deleted all the ruamel_yaml_clib packages and replace them with manually. So if this action did not upload the packages, then I'm not sure how they possibly got there.

Since the packages were built with newer setuptools, they have the proper distribution name and the bug in the upload script does not apply. Hence the packages were uploaded to https://dl.espressif.com/pypi/ruamel-yaml-clib/ instead of https://dl.espressif.com/pypi/clib/.

If you take a look at the https://dl.espressif.com/pypi/clib/ there are six ruamel_yaml_clib packages uploaded, which matches the action you manually started. Previously there were only two ruamel.yaml.clib packages, which were uploaded manually to overcome espressif/esp-idf#15713

[jakub-kocka]

@fhrbata, @dobairoland, I have addressed all your pointed out misleadings and concerns.

Do you agree to proceed with the test, including the upload of the wheels with the defined wheels action?

[fhrbata]

FWIW LGTM :). I guess we will see how it goes. Thank you

[jakub-kocka]

Guys, @dobairoland, @fhrbata, can you please take a final look before merging?

I have addressed all the issues mentioned in the related internal tasks, as well as all those that occur during the process.
In tests, there are successful builds/uploads.

[fhrbata]

FWIW LGTM, thank you!

[dobairoland]

LGTM