Skip to content

Commit afd8d0e

Browse files
author
Release Manager
committed
gh-36358: CI: Fix the multi-stage build in ci-linux.yml, fix conda/centos/archlinux system packages <!-- ^^^^^ Please provide a concise, informative and self-explanatory title. Don't put issue numbers in there, do this in the PR body below. For example, instead of "Fixes #1234" use "Introduce new method to calculate 1+1" --> <!-- Describe your changes here in detail --> Fixes breakage from #36338 seen in https://github.com/sagemath/sage/acti ons/runs/6332061118/job/17206153694: The 2nd stage (`...-with-targets`) does not find the container from the 1st stage (`...-with-targets-pre`) because they are keyed to the commit sha. Fixed here by keying it to the sha before merging the CI fixes. Also fixing the following failures: - `conda-forge-minimal` (as seen in https://github.com/sagemath/sage/act ions/runs/6207962004/job/16858998561); failed because the python3 depcheck packages were missing. Fixed now - check with `tox -e docker- conda-forge-minimal -- config.status` - `centos-stream-9-python3.9-minimal` (https://github.com/sagemath/sage/ actions/runs/6207962004/job/16858997494), `almalinux-9-python3.11-minimal` (https://github.com/sagemath/sage/actio ns/runs/6207962004/job/16858997663) and `-standard`; fixes #34786 - `archlinux-standard` - libgiac was missing in at least one run Also adding a few more uses of GH Action output groups for convenience. Tests running at https://github.com/mkoeppe/sage/actions/workflows/ci- linux.yml <!-- Why is this change required? What problem does it solve? --> <!-- If this PR resolves an open issue, please link to it here. For example "Fixes #12345". --> <!-- If your change requires a documentation PR, please link it appropriately. --> ### 📝 Checklist <!-- Put an `x` in all the boxes that apply. --> <!-- If your change requires a documentation PR, please link it appropriately --> <!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! --> <!-- Feel free to remove irrelevant items. --> - [x] The title is concise, informative, and self-explanatory. - [x] The description explains in detail what this PR is about. - [ ] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation accordingly. ### ⌛ Dependencies <!-- List all open PRs that this PR logically depends on - #12345: short description why this is a dependency - #34567: ... --> <!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! --> URL: #36358 Reported by: Matthias Köppe Reviewer(s): Dima Pasechnik
2 parents b30cc4b + def59ae commit afd8d0e

File tree

11 files changed

+49
-26
lines changed

11 files changed

+49
-26
lines changed

‎.github/workflows/docker.yml

Lines changed: 12 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -179,12 +179,6 @@ jobs:
179179
(export PATH=$(pwd)/build/bin:$PATH; (cd upstream && bash -x update-pkgs.sh) && sed -i.bak '/upstream/d' .dockerignore && echo "/:toolchain:/i ADD upstream upstream" | sed -i.bak -f - build/bin/write-dockerfile.sh && git diff)
180180
if: inputs.upstream_artifact
181181

182-
- name: Merge CI fixes from sagemath/sage
183-
run: |
184-
.ci/merge-fixes.sh
185-
env:
186-
GH_TOKEN: ${{ github.token }}
187-
188182
- name: Try to login to ghcr.io
189183
if: inputs.docker_push_repository != ''
190184
# https://docs.github.com/en/actions/reference/workflow-commands-for-github-actions#setting-an-environment-variable
@@ -197,6 +191,9 @@ jobs:
197191
echo "DOCKER_PUSH_REPOSITORY=$(echo ${{ inputs.docker_push_repository }} | tr "[:upper:]" "[:lower:]")" >> $GITHUB_ENV
198192
echo "DOCKER_CONFIG_FILE=$HOME/.docker/config.json" >> $GITHUB_ENV
199193
fi
194+
# This line needs to be run before the step "Merge CI fixes from sagemath/sage".
195+
DOCKER_TAG="$(git describe --dirty --always)"
196+
echo "DOCKER_TAG=$DOCKER_TAG" >> $GITHUB_ENV
200197
# From the docker documentation via .ci/update-env.sh:
201198
# "A tag name must be valid ASCII and may
202199
# contain lowercase and uppercase letters, digits, underscores, periods and
@@ -211,6 +208,15 @@ jobs:
211208
EXTRA_DOCKER_TAGS="dev $EXTRA_DOCKER_TAGS";;
212209
esac
213210
echo "EXTRA_DOCKER_TAGS=$EXTRA_DOCKER_TAGS" >> $GITHUB_ENV
211+
212+
- name: Merge CI fixes from sagemath/sage
213+
# This step needs to happen after the commit sha is put in DOCKER_TAG
214+
# so that multi-stage builds can work correctly.
215+
run: |
216+
.ci/merge-fixes.sh
217+
env:
218+
GH_TOKEN: ${{ github.token }}
219+
214220
- name: Configure and build Sage distribution within a Docker container
215221
run: |
216222
set -o pipefail; EXTRA_DOCKER_BUILD_ARGS="--build-arg NUMPROC=4 --build-arg USE_MAKEFLAGS=\"-k V=0 SAGE_NUM_THREADS=3\"" tox -e $TOX_ENV -- $TARGETS 2>&1 | sed "/^configure: notice:/s|^|::warning file=artifacts/$LOGS_ARTIFACT_NAME/config.log::|;/^configure: warning:/s|^|::warning file=artifacts/$LOGS_ARTIFACT_NAME/config.log::|;/^configure: error:/s|^|::error file=artifacts/$LOGS_ARTIFACT_NAME/config.log::|;"

‎.github/workflows/scan-logs.sh

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,13 @@ LOGS=${1-logs}
55
for a in $(find "$LOGS" -type f -name "*.log"); do
66
if tail -100 "$a" 2>/dev/null | grep "^[A-Za-z]*Error" >/dev/null; then
77
echo :":"error file=$a:":" ==== ERROR IN LOG FILE $a ====
8+
echo "::group::$a"
89
cat "$a"
10+
echo "::endgroup::"
911
elif tail -20 "$a" 2>/dev/null | grep -E "^(Warning: Error testing|^sage .*doctest.*failed)" >/dev/null; then
1012
echo :":"warning file=$a:":" ==== TESTSUITE FAILURE IN LOG FILE $a ====
13+
echo "::group::$a"
1114
cat "$a"
15+
echo "::endgroup::"
1216
fi
1317
done

‎build/bin/write-dockerfile.sh

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -233,11 +233,11 @@ ARG EXTRA_CONFIGURE_ARGS=""
233233
EOF
234234
if [ ${WITH_SYSTEM_SPKG} = "force" ]; then
235235
cat <<EOF
236-
$RUN echo "****** Configuring: ./configure --enable-build-as-root $CONFIGURE_ARGS \${EXTRA_CONFIGURE_ARGS} *******"; ./configure --enable-build-as-root $CONFIGURE_ARGS \${EXTRA_CONFIGURE_ARGS} || (echo "********** configuring without forcing ***********"; cat config.log; ./configure --enable-build-as-root; cat config.log; exit 1) $ENDRUN
236+
$RUN echo "****** Configuring: ./configure --enable-build-as-root $CONFIGURE_ARGS \${EXTRA_CONFIGURE_ARGS} *******"; ./configure --enable-build-as-root $CONFIGURE_ARGS \${EXTRA_CONFIGURE_ARGS} || (echo "********** configuring without forcing ***********"; echo "::group::config.log"; cat config.log; echo "::endgroup::"; ./configure --enable-build-as-root; echo "::group::config.log"; cat config.log; echo "::endgroup::"; exit 1) $ENDRUN
237237
EOF
238238
else
239239
cat <<EOF
240-
$RUN echo "****** Configuring: ./configure --enable-build-as-root $CONFIGURE_ARGS \${EXTRA_CONFIGURE_ARGS} *******"; ./configure --enable-build-as-root $CONFIGURE_ARGS \${EXTRA_CONFIGURE_ARGS} || (cat config.log; exit 1) $ENDRUN
240+
$RUN echo "****** Configuring: ./configure --enable-build-as-root $CONFIGURE_ARGS \${EXTRA_CONFIGURE_ARGS} *******"; ./configure --enable-build-as-root $CONFIGURE_ARGS \${EXTRA_CONFIGURE_ARGS} || (echo "::group::config.log"; cat config.log; echo "::endgroup::"; exit 1) $ENDRUN
241241
EOF
242242
fi
243243
cat <<EOF

‎build/pkgs/_bootstrap/distros/arch.txt

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
11
# Packages needed for ./bootstrap
22
autoconf automake libtool
3-
# Otherwise: Error: could not locate the pkg-config autoconf macros. These are
4-
pkg-config
3+
pkgconf

‎build/pkgs/_python3.10/distros/fedora.txt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
python310
2-
python310-devel
1+
python3.10
2+
python3.10-devel

‎build/pkgs/_python3.11/distros/fedora.txt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
python311
2-
python311-devel
1+
python3.11
2+
python3.11-devel

‎build/pkgs/_python3.12/distros/fedora.txt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
python312
2-
python312-devel
1+
python3.12
2+
python3.12-devel

‎build/pkgs/_python3.9/distros/fedora.txt

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
1-
python39
2-
python39-devel
1+
python3.9
2+
python3.9-devel
3+
# Except on centos-stream-8 and almalinux-8, where it is called python39 and python39-devel; we special-case this in tox.ini

‎build/pkgs/freetype/distros/debian.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
libfreetype6-dev
1+
libfreetype-dev

‎build/pkgs/pkgconf/distros/arch.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
pkgconf

0 commit comments

Comments
 (0)