Skip to content

Commit 2a5f3a9

Browse files
indygregindygreg
committed
global: CPython 3.12 support
I cobbled this together over several partial days of work. There's a lot of changes in there and I'm too lazy to split it back up into microcomments. I haven't performed a full audit of the produced archives and their PYTHON.json files. I suspect there's some subtle bugs lingering in here: there usually is for the initial release of a major version. But CI and our Rust validation is passing and that usually means a lot. So I'm going to commit what we have and iterate on it as needed. Closes #186.
1 parent a8c0808 commit 2a5f3a9

18 files changed

+659
-28
lines changed

‎.github/workflows/apple.yml‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,16 @@ jobs:
8383
py: 'cpython-3.11'
8484
optimizations: 'lto'
8585

86+
- target_triple: 'aarch64-apple-darwin'
87+
py: 'cpython-3.12'
88+
optimizations: 'debug'
89+
- target_triple: 'aarch64-apple-darwin'
90+
py: 'cpython-3.12'
91+
optimizations: 'noopt'
92+
- target_triple: 'aarch64-apple-darwin'
93+
py: 'cpython-3.12'
94+
optimizations: 'lto'
95+
8696
# macOS on Intel hardware. This is pretty straightforward. We exclude
8797
# noopt because it doesn't provide any compelling advantages over PGO
8898
# or LTO builds.
@@ -137,6 +147,19 @@ jobs:
137147
- target_triple: 'x86_64-apple-darwin'
138148
py: 'cpython-3.11'
139149
optimizations: 'pgo+lto'
150+
151+
- target_triple: 'x86_64-apple-darwin'
152+
py: 'cpython-3.12'
153+
optimizations: 'debug'
154+
- target_triple: 'x86_64-apple-darwin'
155+
py: 'cpython-3.12'
156+
optimizations: 'lto'
157+
- target_triple: 'x86_64-apple-darwin'
158+
py: 'cpython-3.12'
159+
optimizations: 'pgo'
160+
- target_triple: 'x86_64-apple-darwin'
161+
py: 'cpython-3.12'
162+
optimizations: 'pgo+lto'
140163
needs:
141164
- pythonbuild
142165
runs-on: 'macos-11'

‎.github/workflows/linux.yml‎

Lines changed: 180 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -157,6 +157,16 @@ jobs:
157157
py: 'cpython-3.11'
158158
optimizations: 'lto'
159159

160+
- target_triple: 'aarch64-unknown-linux-gnu'
161+
py: 'cpython-3.12'
162+
optimizations: 'debug'
163+
- target_triple: 'aarch64-unknown-linux-gnu'
164+
py: 'cpython-3.12'
165+
optimizations: 'noopt'
166+
- target_triple: 'aarch64-unknown-linux-gnu'
167+
py: 'cpython-3.12'
168+
optimizations: 'lto'
169+
160170
# Cross-compiles can't do PGO and require Python 3.9.
161171
- target_triple: 'armv7-unknown-linux-gnueabi'
162172
py: 'cpython-3.9'
@@ -188,6 +198,16 @@ jobs:
188198
py: 'cpython-3.11'
189199
optimizations: 'lto'
190200

201+
- target_triple: 'armv7-unknown-linux-gnueabi'
202+
py: 'cpython-3.12'
203+
optimizations: 'debug'
204+
- target_triple: 'armv7-unknown-linux-gnueabi'
205+
py: 'cpython-3.12'
206+
optimizations: 'noopt'
207+
- target_triple: 'armv7-unknown-linux-gnueabi'
208+
py: 'cpython-3.12'
209+
optimizations: 'lto'
210+
191211
# Cross-compiles can't do PGO and require Python 3.9.
192212
- target_triple: 'armv7-unknown-linux-gnueabihf'
193213
py: 'cpython-3.9'
@@ -219,6 +239,16 @@ jobs:
219239
py: 'cpython-3.11'
220240
optimizations: 'lto'
221241

242+
- target_triple: 'armv7-unknown-linux-gnueabihf'
243+
py: 'cpython-3.12'
244+
optimizations: 'debug'
245+
- target_triple: 'armv7-unknown-linux-gnueabihf'
246+
py: 'cpython-3.12'
247+
optimizations: 'noopt'
248+
- target_triple: 'armv7-unknown-linux-gnueabihf'
249+
py: 'cpython-3.12'
250+
optimizations: 'lto'
251+
222252
# Cross-compiles can't do PGO and require Python 3.9.
223253
- target_triple: 'mips-unknown-linux-gnu'
224254
py: 'cpython-3.9'
@@ -250,6 +280,16 @@ jobs:
250280
py: 'cpython-3.11'
251281
optimizations: 'lto'
252282

283+
- target_triple: 'mips-unknown-linux-gnu'
284+
py: 'cpython-3.12'
285+
optimizations: 'debug'
286+
- target_triple: 'mips-unknown-linux-gnu'
287+
py: 'cpython-3.12'
288+
optimizations: 'noopt'
289+
- target_triple: 'mips-unknown-linux-gnu'
290+
py: 'cpython-3.12'
291+
optimizations: 'lto'
292+
253293
# Cross-compiles can't do PGO and require Python 3.9.
254294
- target_triple: 'mipsel-unknown-linux-gnu'
255295
py: 'cpython-3.9'
@@ -281,6 +321,16 @@ jobs:
281321
py: 'cpython-3.11'
282322
optimizations: 'lto'
283323

324+
- target_triple: 'mipsel-unknown-linux-gnu'
325+
py: 'cpython-3.12'
326+
optimizations: 'debug'
327+
- target_triple: 'mipsel-unknown-linux-gnu'
328+
py: 'cpython-3.12'
329+
optimizations: 'noopt'
330+
- target_triple: 'mipsel-unknown-linux-gnu'
331+
py: 'cpython-3.12'
332+
optimizations: 'lto'
333+
284334
# Cross-compiles can't do PGO and require Python 3.9.
285335
- target_triple: 's390x-unknown-linux-gnu'
286336
py: 'cpython-3.9'
@@ -312,6 +362,16 @@ jobs:
312362
py: 'cpython-3.11'
313363
optimizations: 'lto'
314364

365+
- target_triple: 's390x-unknown-linux-gnu'
366+
py: 'cpython-3.12'
367+
optimizations: 'debug'
368+
- target_triple: 's390x-unknown-linux-gnu'
369+
py: 'cpython-3.12'
370+
optimizations: 'noopt'
371+
- target_triple: 's390x-unknown-linux-gnu'
372+
py: 'cpython-3.12'
373+
optimizations: 'lto'
374+
315375
# Cross-compiles can't do PGO and require Python 3.9.
316376
- target_triple: 'ppc64le-unknown-linux-gnu'
317377
py: 'cpython-3.9'
@@ -343,6 +403,16 @@ jobs:
343403
py: 'cpython-3.11'
344404
optimizations: 'lto'
345405

406+
- target_triple: 'ppc64le-unknown-linux-gnu'
407+
py: 'cpython-3.12'
408+
optimizations: 'debug'
409+
- target_triple: 'ppc64le-unknown-linux-gnu'
410+
py: 'cpython-3.12'
411+
optimizations: 'noopt'
412+
- target_triple: 'ppc64le-unknown-linux-gnu'
413+
py: 'cpython-3.12'
414+
optimizations: 'lto'
415+
346416
# We don't publish noopt builds when PGO is available.
347417
- target_triple: 'x86_64-unknown-linux-gnu'
348418
py: 'cpython-3.8'
@@ -412,6 +482,23 @@ jobs:
412482
optimizations: 'pgo+lto'
413483
run: true
414484

485+
- target_triple: 'x86_64-unknown-linux-gnu'
486+
py: 'cpython-3.12'
487+
optimizations: 'debug'
488+
run: true
489+
- target_triple: 'x86_64-unknown-linux-gnu'
490+
py: 'cpython-3.12'
491+
optimizations: 'lto'
492+
run: true
493+
- target_triple: 'x86_64-unknown-linux-gnu'
494+
py: 'cpython-3.12'
495+
optimizations: 'pgo'
496+
run: true
497+
- target_triple: 'x86_64-unknown-linux-gnu'
498+
py: 'cpython-3.12'
499+
optimizations: 'pgo+lto'
500+
run: true
501+
415502
- target_triple: 'x86_64_v2-unknown-linux-gnu'
416503
py: 'cpython-3.9'
417504
optimizations: 'debug'
@@ -463,6 +550,23 @@ jobs:
463550
optimizations: 'pgo+lto'
464551
run: true
465552

553+
- target_triple: 'x86_64_v2-unknown-linux-gnu'
554+
py: 'cpython-3.12'
555+
optimizations: 'debug'
556+
run: true
557+
- target_triple: 'x86_64_v2-unknown-linux-gnu'
558+
py: 'cpython-3.12'
559+
optimizations: 'lto'
560+
run: true
561+
- target_triple: 'x86_64_v2-unknown-linux-gnu'
562+
py: 'cpython-3.12'
563+
optimizations: 'pgo'
564+
run: true
565+
- target_triple: 'x86_64_v2-unknown-linux-gnu'
566+
py: 'cpython-3.12'
567+
optimizations: 'pgo+lto'
568+
run: true
569+
466570
- target_triple: 'x86_64_v3-unknown-linux-gnu'
467571
py: 'cpython-3.9'
468572
optimizations: 'debug'
@@ -514,6 +618,23 @@ jobs:
514618
optimizations: 'pgo+lto'
515619
run: true
516620

621+
- target_triple: 'x86_64_v3-unknown-linux-gnu'
622+
py: 'cpython-3.12'
623+
optimizations: 'debug'
624+
run: true
625+
- target_triple: 'x86_64_v3-unknown-linux-gnu'
626+
py: 'cpython-3.12'
627+
optimizations: 'lto'
628+
run: true
629+
- target_triple: 'x86_64_v3-unknown-linux-gnu'
630+
py: 'cpython-3.12'
631+
optimizations: 'pgo'
632+
run: true
633+
- target_triple: 'x86_64_v3-unknown-linux-gnu'
634+
py: 'cpython-3.12'
635+
optimizations: 'pgo+lto'
636+
run: true
637+
517638
# GitHub Actions runners don't support x86-64-v4 so we can't PGO.
518639
- target_triple: 'x86_64_v4-unknown-linux-gnu'
519640
py: 'cpython-3.9'
@@ -546,6 +667,16 @@ jobs:
546667
py: 'cpython-3.11'
547668
optimizations: 'lto'
548669

670+
- target_triple: 'x86_64_v4-unknown-linux-gnu'
671+
py: 'cpython-3.12'
672+
optimizations: 'debug'
673+
- target_triple: 'x86_64_v4-unknown-linux-gnu'
674+
py: 'cpython-3.12'
675+
optimizations: 'noopt'
676+
- target_triple: 'x86_64_v4-unknown-linux-gnu'
677+
py: 'cpython-3.12'
678+
optimizations: 'lto'
679+
549680
# musl doesn't support PGO.
550681
- target_triple: 'x86_64-unknown-linux-musl'
551682
py: 'cpython-3.8'
@@ -599,6 +730,19 @@ jobs:
599730
optimizations: 'lto'
600731
run: true
601732

733+
- target_triple: 'x86_64-unknown-linux-musl'
734+
py: 'cpython-3.12'
735+
optimizations: 'debug'
736+
run: true
737+
- target_triple: 'x86_64-unknown-linux-musl'
738+
py: 'cpython-3.12'
739+
optimizations: 'noopt'
740+
run: true
741+
- target_triple: 'x86_64-unknown-linux-musl'
742+
py: 'cpython-3.12'
743+
optimizations: 'lto'
744+
run: true
745+
602746
- target_triple: 'x86_64_v2-unknown-linux-musl'
603747
py: 'cpython-3.9'
604748
optimizations: 'debug'
@@ -638,6 +782,19 @@ jobs:
638782
optimizations: 'lto'
639783
run: true
640784

785+
- target_triple: 'x86_64_v2-unknown-linux-musl'
786+
py: 'cpython-3.12'
787+
optimizations: 'debug'
788+
run: true
789+
- target_triple: 'x86_64_v2-unknown-linux-musl'
790+
py: 'cpython-3.12'
791+
optimizations: 'noopt'
792+
run: true
793+
- target_triple: 'x86_64_v2-unknown-linux-musl'
794+
py: 'cpython-3.12'
795+
optimizations: 'lto'
796+
run: true
797+
641798
- target_triple: 'x86_64_v3-unknown-linux-musl'
642799
py: 'cpython-3.9'
643800
optimizations: 'debug'
@@ -677,6 +834,19 @@ jobs:
677834
optimizations: 'lto'
678835
run: true
679836

837+
- target_triple: 'x86_64_v3-unknown-linux-musl'
838+
py: 'cpython-3.12'
839+
optimizations: 'debug'
840+
run: true
841+
- target_triple: 'x86_64_v3-unknown-linux-musl'
842+
py: 'cpython-3.12'
843+
optimizations: 'noopt'
844+
run: true
845+
- target_triple: 'x86_64_v3-unknown-linux-musl'
846+
py: 'cpython-3.12'
847+
optimizations: 'lto'
848+
run: true
849+
680850
- target_triple: 'x86_64_v4-unknown-linux-musl'
681851
py: 'cpython-3.9'
682852
optimizations: 'debug'
@@ -707,6 +877,16 @@ jobs:
707877
py: 'cpython-3.11'
708878
optimizations: 'lto'
709879

880+
- target_triple: 'x86_64_v4-unknown-linux-musl'
881+
py: 'cpython-3.12'
882+
optimizations: 'debug'
883+
- target_triple: 'x86_64_v4-unknown-linux-musl'
884+
py: 'cpython-3.12'
885+
optimizations: 'noopt'
886+
- target_triple: 'x86_64_v4-unknown-linux-musl'
887+
py: 'cpython-3.12'
888+
optimizations: 'lto'
889+
710890
needs:
711891
- pythonbuild
712892
- image

‎.github/workflows/windows.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ jobs:
4545
- 'cpython-3.9'
4646
- 'cpython-3.10'
4747
- 'cpython-3.11'
48+
- 'cpython-3.12'
4849
vcvars:
4950
- 'vcvars32.bat'
5051
- 'vcvars64.bat'

‎cpython-unix/Makefile‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -259,6 +259,10 @@ $(OUTDIR)/cpython-3.10-$(CPYTHON_3.10_VERSION)-$(HOST_PLATFORM).tar: $(PYTHON_HO
259259
$(OUTDIR)/cpython-3.11-$(CPYTHON_3.11_VERSION)-$(HOST_PLATFORM).tar: $(PYTHON_HOST_DEPENDS)
260260
$(RUN_BUILD) --docker-image $(DOCKER_IMAGE_BUILD) cpython-3.11-host
261261

262+
$(OUTDIR)/cpython-3.12-$(CPYTHON_3.12_VERSION)-$(HOST_PLATFORM).tar: $(PYTHON_HOST_DEPENDS)
263+
$(RUN_BUILD) --docker-image $(DOCKER_IMAGE_BUILD) cpython-3.12-host
264+
265+
262266
PYTHON_DEPENDS := \
263267
$(PYTHON_SUPPORT_FILES) \
264268
$(OUTDIR)/versions/VERSION.pip \
@@ -302,3 +306,6 @@ $(OUTDIR)/cpython-$(CPYTHON_3.10_VERSION)-$(PACKAGE_SUFFIX).tar: $(ALL_PYTHON_DE
302306

303307
$(OUTDIR)/cpython-$(CPYTHON_3.11_VERSION)-$(PACKAGE_SUFFIX).tar: $(ALL_PYTHON_DEPENDS)
304308
$(RUN_BUILD) --docker-image $(DOCKER_IMAGE_BUILD) cpython-3.11
309+
310+
$(OUTDIR)/cpython-$(CPYTHON_3.12_VERSION)-$(PACKAGE_SUFFIX).tar: $(ALL_PYTHON_DEPENDS)
311+
$(RUN_BUILD) --docker-image $(DOCKER_IMAGE_BUILD) cpython-3.12

‎cpython-unix/build-cpython-host.sh‎

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -69,10 +69,26 @@ case "${BUILD_TRIPLE}" in
6969
;;
7070
esac
7171

72+
EXTRA_CONFIGURE_FLAGS=
73+
74+
# We may not have a usable libraries to build against. Forcefully disable extensions
75+
# that may not build.
76+
if [ -n "${PYTHON_MEETS_MINIMUM_VERSION_3_12}" ]; then
77+
for m in _hashlib _ssl; do
78+
EXTRA_CONFIGURE_FLAGS="${EXTRA_CONFIGURE_FLAGS} py_cv_module_${m}=n/a"
79+
done
80+
fi
81+
7282
CC="${HOST_CC}" CXX="${HOST_CXX}" CFLAGS="${EXTRA_HOST_CFLAGS}" CPPFLAGS="${EXTRA_HOST_CFLAGS}" LDFLAGS="${EXTRA_HOST_LDFLAGS}" ./configure \
7383
--prefix /tools/host \
74-
--without-ensurepip
84+
--without-ensurepip \
85+
${EXTRA_CONFIGURE_FLAGS}
7586

76-
make -j "${NUM_CPUS}" install DESTDIR=${ROOT}/out
87+
# Ideally we'd do `make install` here and be done with it. But there's a race
88+
# condition in CPython's build system related to directory creation that gets
89+
# tickled when we do this. https://github.com/python/cpython/issues/109796.
90+
make -j "${NUM_CPUS}"
91+
make -j sharedinstall DESTDIR=${ROOT}/out
92+
make -j install DESTDIR=${ROOT}/out
7793

7894
popd

0 commit comments

Comments
 (0)