Backport PR #59136 on branch 2.2.x (Upload 3.13 & free-threaded nightly wheels) (#59835)

* Bump pypa/cibuildwheel from 2.19.1 to 2.19.2 (#59208)

Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 2.19.1 to 2.19.2.
- [Release notes](https://github.com/pypa/cibuildwheel/releases)
- [Changelog](https://github.com/pypa/cibuildwheel/blob/main/docs/changelog.md)
- [Commits](pypa/cibuildwheel@v2.19.1...v2.19.2)

---
updated-dependencies:
- dependency-name: pypa/cibuildwheel
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit ad09dc6)

* Upload 3.13 & free-threaded nightly wheels (#59136)

* Upload free-threaded nightly wheels on Linux and macOS

* Consolidate jobs into one

* Install build dependencies in before-build and pass --no-build-isolation

* Fix {project} placeholder in cibuildwheel config

* Correctly quote echo CIBW_BUILD_FRONTEND command

* Run echo -e

* Add {package} to before-build

* Include cibw script in sdist & add matrix value for build frontend

* Change manifest and gitattributes

* Change gitattributes

* Install verioneer in before-build

* Add cibw_before_test to install nightly NumPy

* Expand before-test to musl

* Better comments plus always run before-build/before-test on 3.13

* Add --no-build-isolation in 3.13 as well

* Install nightly numpy before windows tests

* Address feedback; add todo for NumPy nightly and move default outside matrix

* Set build_frontend to 'build' in pyodide build

---------

Co-authored-by: Matthew Roeschke <[email protected]>
(cherry picked from commit 7c0ee27)

* CI: Update to cibuildwheel 2.20.0 (#59401)

cibuildwheel 2.20.0 uses the ABI stable Python 3.13.0rc1 and build Python 3.13 wheels by default, which allows removing the `CIBW_PRERELEASE_PYTHONS` flag.

Co-authored-by: Matthew Roeschke <[email protected]>
(cherry picked from commit 70bb855)

* Update wheels.yml

* BLD/RLS: build wheels with released numpy/cython for Python 3.13 (#59819)

(cherry picked from commit 2237217)

* enable prerelease again

---------

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Lysandros Nikolaou <[email protected]>
Co-authored-by: Ewout ter Hoeven <[email protected]>
Co-authored-by: Thomas Li <[email protected]>

Author: 5 people

.circleci/config.yml

@@ -72,7 +72,7 @@ jobs:
           name: Build aarch64 wheels
           no_output_timeout: 30m # Sometimes the tests won't generate any output, make sure the job doesn't get killed by that
           command: |
-            pip3 install cibuildwheel==2.15.0
+            pip3 install cibuildwheel==2.20.0
             cibuildwheel --prerelease-pythons --output-dir wheelhouse
 
           environment:

.gitattributes

@@ -68,7 +68,7 @@ ci export-ignore
 doc export-ignore
 gitpod export-ignore
 MANIFEST.in export-ignore
-scripts export-ignore
+scripts/** export-ignore
 typings export-ignore
 web export-ignore
 CITATION.cff export-ignore
@@ -82,3 +82,6 @@ setup.py export-ignore
 # csv_dir_path fixture checks the existence of the directory
 # exclude the whole directory to avoid running related tests in sdist
 pandas/tests/io/parser/data export-ignore
+
+# Include cibw script in sdist since it's needed for building wheels
+scripts/cibw_before_build.sh -export-ignore

.github/workflows/wheels.yml

@@ -99,7 +99,17 @@ jobs:
         - [macos-14, macosx_arm64]
         - [windows-2022, win_amd64]
         # TODO: support PyPy?
-        python: [["cp39", "3.9"], ["cp310", "3.10"], ["cp311", "3.11"], ["cp312", "3.12"]]
+        python: [["cp39", "3.9"], ["cp310", "3.10"], ["cp311", "3.11"], ["cp312", "3.12"], ["cp313", "3.13"], ["cp313t", "3.13"]]
+        include:
+        # TODO: Remove this plus installing build deps in cibw_before_build.sh
+        # after pandas can be built with a released NumPy/Cython
+        - python: ["cp313t", "3.13"]
+          cibw_build_frontend: 'pip; args: --no-build-isolation'
+        # TODO: Build free-threaded wheels for Windows
+        exclude:
+        - buildplat: [windows-2022, win_amd64]
+          python: ["cp313t", "3.13"]
+
     env:
       IS_PUSH: ${{ github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v') }}
       IS_SCHEDULE_DISPATCH: ${{ github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' }}
@@ -140,12 +150,13 @@ jobs:
         run: echo "sdist_name=$(cd ./dist && ls -d */)" >> "$GITHUB_ENV"
 
       - name: Build wheels
-        uses: pypa/cibuildwheel@v2.17.0
+        uses: pypa/cibuildwheel@v2.20.0
         with:
          package-dir: ./dist/${{ startsWith(matrix.buildplat[1], 'macosx') && env.sdist_name || needs.build_sdist.outputs.sdist_file }}
         env:
           CIBW_PRERELEASE_PYTHONS: True
           CIBW_BUILD: ${{ matrix.python[0] }}-${{ matrix.buildplat[1] }}
+          CIBW_BUILD_FRONTEND: ${{ matrix.cibw_build_frontend || 'pip' }}
 
       - name: Set up Python
         uses: mamba-org/setup-micromamba@v1

MANIFEST.in

@@ -62,3 +62,6 @@ prune pandas/tests/io/parser/data
 # Selectively re-add *.cxx files that were excluded above
 graft pandas/_libs/src
 graft pandas/_libs/include
+
+# Include cibw script in sdist since it's needed for building wheels
+include scripts/cibw_before_build.sh

pyproject.toml

@@ -160,9 +160,11 @@ test-command = """
   pd.test(extra_args=["-m not clipboard and not single_cpu and not slow and not network and not db", "-n 2", "--no-strict-data-files"]); \
   pd.test(extra_args=["-m not clipboard and single_cpu and not slow and not network and not db", "--no-strict-data-files"]);' \
   """
+free-threaded-support = true
+before-build = "bash {package}/scripts/cibw_before_build.sh"
 
 [tool.cibuildwheel.windows]
-before-build = "pip install delvewheel"
+before-build = "pip install delvewheel && bash {package}/scripts/cibw_before_build.sh"
 repair-wheel-command = "delvewheel repair -w {dest_dir} {wheel}"
 
 [[tool.cibuildwheel.overrides]]

scripts/cibw_before_build.sh

@@ -0,0 +1,7 @@
+# TODO: Delete when there's a PyPI Cython release that supports free-threaded Python 3.13.
+FREE_THREADED_BUILD="$(python -c"import sysconfig; print(bool(sysconfig.get_config_var('Py_GIL_DISABLED')))")"
+if [[ $FREE_THREADED_BUILD == "True"  ]]; then
+    python -m pip install -U pip
+    python -m pip install -i https://pypi.anaconda.org/scientific-python-nightly-wheels/simple numpy cython
+    python -m pip install ninja meson-python versioneer[toml]
+fi