Skip to content

Commit 8f938b7

Browse files
author
Release Manager
committed
gh-36047: New GitHub workflow to continue our images on Docker Hub <!-- ^^^^^ Please provide a concise, informative and self-explanatory title. Don't put issue numbers in there, do this in the PR body below. For example, instead of "Fixes #1234" use "Introduce new method to calculate 1+1" --> <!-- Describe your changes here in detail --> This PR implements a GitHub workflow which sould establish a continuation of the Docker images in the Sage Docker Hub repositories. Therefore, it aims to replace `$SAGE_ROOT/.gitlab-ci.yml` which has been used formerly for this purpose but isn't working any more (see [sage- devel-thread fom April 5th](https://groups.google.com/g/sage- devel/c/eJ_OicQXM8Y)). The new worklfow uses the same `$SAGE_ROOT/docker/Dockerfile` as `SAGE_ROOT/.gitlab-ci.yml` and is based on [docker/build-push- action@v4](https://github.com/docker/build-push-action). Concerning the images placed in the [sagemath-dev repository](https://hub.docker.com/r/sagemath/sagemath-dev) you should note the following difference to the releases built before 9.8: They are no longer build on target `sagemath-dev` since this causes a "No space left on device" error (see the [log- file](https://github.com/soehms/sage/actions/runs/5694284990) of a correspondig test run). Instead they are build on target `make-build`, now. I think the main difference is the missing build of documentation. But, I don't know if this fits with the idea of this repository. If we really need target `sagemath-dev` then maybe something like the `free disk space` step of job `linux` in `.github/workflows/docker.yml` would help to get it run through. What about to just pull `ghcr.io/sagemath/sage/sage-ubuntu-jammy-standard-with-targets` and push it to `sagemath-dev` on Docker Hub. Please post here what you think would be the preferred way. Until this is not clearified I ommit setting the tags `develop` and `latest` in `sagemath-dev` (note that these tags still have ongoing pulls). The workflow does not use [docker/metadata- action@v4](https://github.com/docker/metadata-action#latest-tag) as we need to distinguish between `latest` and `develop` tags in the Docker Hub repositories. Furthermore, if the workflow is triggered manually, this should result in the same tags as when triggered by pushing a new release tag (`metadata` would take the branch name). The reason why the worklflow is split into two jobs is because the time limit of 6 hours per job was exeeded in a former test with just one job. I've tested the new workflow in my fork repository using the `workflow_dispatch` (see the corresponding [log- file](https://github.com/soehms/sage/actions/runs/5795823327)). The resulting tags on Docker Hub are: * [sagemath/sagemath:10.1.beta9](https://hub.docker.com/layers/sagemath/ sagemath/10.1.beta9/images/sha256- 4c00148d6e6c2134728b0a74d0e778281d64a8f0ff9b49ade0b9af68de758f12?context =repo) * [sagemath/sagemath:develop](https://hub.docker.com/layers/sagemath/sag emath/develop/images/sha256- 4c00148d6e6c2134728b0a74d0e778281d64a8f0ff9b49ade0b9af68de758f12?context =repo) * [sagemath/sagemath- dev:10.1.beta9](https://hub.docker.com/layers/sagemath/sagemath-dev/10.1 .beta9/images/sha256- d5071d0cb683edf9f22d19c2c26d5d580dc4e05823c755bfce85c744b557fd14?context =repo) In order that this worked I had to set the secrets `secrets.DOCKERHUB_USERNAME` and `secrets.DOCKERHUB_TOKEN` in my fork of the GitHub repository. Thus, after this PR is merged a maintainer has to set this secrets on the upstream repository. <!-- Why is this change required? What problem does it solve? --> <!-- If this PR resolves an open issue, please link to it here. For example "Fixes #12345". --> <!-- If your change requires a documentation PR, please link it appropriately. --> ### 📝 Checklist <!-- Put an `x` in all the boxes that apply. --> <!-- If your change requires a documentation PR, please link it appropriately --> <!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! --> <!-- Feel free to remove irrelevant items. --> - [x] The title is concise, informative, and self-explanatory. - [x] The description explains in detail what this PR is about. - [x] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation accordingly. ### ⌛ Dependencies <!-- List all open PRs that this PR logically depends on - #12345: short description why this is a dependency - #34567: ... --> <!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! --> URL: #36047 Reported by: Sebastian Oehms Reviewer(s): Julian Rüth, Sebastian Oehms
2 parents aededc6 + b0d14d2 commit 8f938b7

File tree

2 files changed

+132
-5
lines changed

2 files changed

+132
-5
lines changed

‎.github/workflows/push_to_docker_hub.yml

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
name: Build Docker images and push to DockerHub
2+
3+
on:
4+
workflow_dispatch:
5+
# Allow to run manually
6+
branches:
7+
- 'develop'
8+
- 'docker_hub_gha'
9+
push:
10+
tags:
11+
# Just create image on pushing a tag
12+
- '*'
13+
14+
jobs:
15+
sagemath-dev:
16+
name: Build Docker image on target make-build and push to DockerHub sagemath-dev
17+
runs-on: ubuntu-latest
18+
steps:
19+
- name: Checkout
20+
uses: actions/checkout@v3
21+
22+
- name: Set tag
23+
# docker/metadata-action@v4 is not used since we need to distinguish
24+
# between latest and develop tags
25+
id: set_tag
26+
run: |
27+
git fetch --depth=1 origin +refs/tags/*:refs/tags/*
28+
TAG_NAME=$(git tag --sort=v:refname | tail -1)
29+
TAG="sagemath/sagemath-dev:$TAG_NAME"
30+
TAG_LIST="$TAG, sagemath/sagemath-dev:develop"
31+
TAG_LIST="$TAG" # don't tag develop until meaning of sagemath-dev is clear
32+
echo "TAG_NAME=$TAG_NAME" >> $GITHUB_ENV
33+
echo "TAG=$TAG" >> $GITHUB_ENV
34+
echo "TAG_LIST=$TAG_LIST" >> $GITHUB_ENV
35+
36+
- name: Update Tag List
37+
id: upd_tag_list
38+
run: |
39+
TAG_LIST="${{ env.TAG_LIST }}, sagemath/sagemath-dev:latest"
40+
TAG_LIST="${{ env.TAG_LIST }}" # don't tag latest until meaning of sagemath-dev is clear
41+
echo "TAG_LIST=$TAG_LIST" >> $GITHUB_ENV
42+
if: "!contains(env.TAG_NAME, 'beta') && !contains(env.TAG_NAME, 'rc')"
43+
44+
- name: Check env
45+
run: |
46+
echo ${{ env.TAG_NAME }}
47+
echo ${{ env.TAG }}
48+
echo ${{ env.TAG_LIST }}
49+
50+
- name: Set up QEMU
51+
uses: docker/setup-qemu-action@v2
52+
53+
- name: Set up Docker Buildx
54+
uses: docker/setup-buildx-action@v2
55+
56+
- name: Login to Docker Hub
57+
uses: docker/login-action@v2
58+
with:
59+
username: ${{ secrets.DOCKERHUB_USERNAME }}
60+
password: ${{ secrets.DOCKERHUB_TOKEN }}
61+
62+
- name: Build and push make-build
63+
uses: docker/build-push-action@v4
64+
with:
65+
context: .
66+
file: docker/Dockerfile
67+
target: make-build # see the corresponding header-note
68+
push: true
69+
tags: ${{ env.TAG_LIST }}
70+
cache-from: type=gha
71+
cache-to: type=gha,mode=max
72+
73+
sagemath:
74+
needs: sagemath-dev
75+
name: Build Docker image on target sagemath and push to DockerHub sagemath
76+
runs-on: ubuntu-latest
77+
steps:
78+
- name: Checkout
79+
uses: actions/checkout@v3
80+
81+
- name: Set tag
82+
# docker/metadata-action@v4 is not used since we need to distinguish
83+
# between latest and develop tags
84+
id: set_tag
85+
run: |
86+
git fetch --depth=1 origin +refs/tags/*:refs/tags/*
87+
TAG_NAME=$(git tag --sort=v:refname | tail -1)
88+
TAG="sagemath/sagemath:$TAG_NAME"
89+
TAG_LIST="$TAG, sagemath/sagemath:develop"
90+
BASE="sagemath/sagemath-dev:$TAG_NAME"
91+
echo "TAG_NAME=$TAG_NAME" >> $GITHUB_ENV
92+
echo "TAG=$TAG" >> $GITHUB_ENV
93+
echo "TAG_LIST=$TAG_LIST" >> $GITHUB_ENV
94+
echo "BASE=$BASE" >> $GITHUB_ENV
95+
96+
- name: Update Tag List
97+
id: upd_tag_list
98+
run: |
99+
TAG_LIST="${{ env.TAG_LIST }}, sagemath/sagemath:latest"
100+
echo "TAG_LIST=$TAG_LIST" >> $GITHUB_ENV
101+
if: "!contains(env.TAG_NAME, 'beta') && !contains(env.TAG_NAME, 'rc')"
102+
103+
- name: Set up QEMU
104+
uses: docker/setup-qemu-action@v2
105+
106+
- name: Set up Docker Buildx
107+
uses: docker/setup-buildx-action@v2
108+
109+
- name: Login to Docker Hub
110+
uses: docker/login-action@v2
111+
with:
112+
username: ${{ secrets.DOCKERHUB_USERNAME }}
113+
password: ${{ secrets.DOCKERHUB_TOKEN }}
114+
115+
- name: Build and push sagemath
116+
uses: docker/build-push-action@v4
117+
with:
118+
context: .
119+
file: docker/Dockerfile
120+
build-args: |
121+
MAKE_BUILD=${{ env.BASE }}
122+
target: sagemath
123+
push: true
124+
tags: ${{ env.TAG_LIST }}
125+
cache-from: type=gha
126+
cache-to: type=gha,mode=max

‎docker/Dockerfile

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -70,11 +70,12 @@
7070

7171

7272
ARG ARTIFACT_BASE=source-clean
73+
ARG MAKE_BUILD=make-build
7374

7475
################################################################################
7576
# Image containing the run-time dependencies for Sage #
7677
################################################################################
77-
FROM ubuntu:jammy as run-time-dependencies
78+
FROM ubuntu:latest as run-time-dependencies
7879
LABEL maintainer="Erik M. Bray <[email protected]>, Julian Rüth <[email protected]>"
7980
# Set sane defaults for common environment variables.
8081
ENV LC_ALL C.UTF-8
@@ -122,7 +123,7 @@ ARG SAGE_ROOT=/home/sage/sage
122123
RUN mkdir -p "$SAGE_ROOT"
123124
WORKDIR $SAGE_ROOT
124125
RUN git init
125-
RUN git remote add trac https://gitlab.com/sagemath/dev/tracmirror.git
126+
RUN git remote add upstream https://github.com/sagemath/sage.git
126127

127128
################################################################################
128129
# Image with the build context added, i.e., the directory from which `docker #
@@ -155,10 +156,10 @@ WORKDIR $SAGE_ROOT
155156
# We create a list of all files present in the artifact-base (with a timestamp
156157
# of now) so we can find out later which files were added/changed/removed.
157158
RUN find . \( -type f -or -type l \) > $HOME/artifact-base.manifest
158-
RUN git fetch "$HOME/sage-context" HEAD \
159+
RUN git fetch --update-shallow "$HOME/sage-context" HEAD \
159160
&& if [ -e docker/.commit ]; then \
160161
git reset `cat docker/.commit` \
161-
|| ( echo "Could not find commit `cat docker/.commit` in your local Git history. Please merge in the latest built develop branch to fix this: git fetch trac && git merge `cat docker/.commit`." && exit 1 ) \
162+
|| ( echo "Could not find commit `cat docker/.commit` in your local Git history. Please merge in the latest built develop branch to fix this: git fetch upstream && git merge `cat docker/.commit`." && exit 1 ) \
162163
else \
163164
echo "You are building from $ARTIFACT_BASE which has no docker/.commit file. That's a bug unless you are building from source-clean or something similar." \
164165
&& git reset FETCH_HEAD \
@@ -203,7 +204,7 @@ RUN make build
203204
################################################################################
204205
# Image with a full build of sage and its documentation. #
205206
################################################################################
206-
FROM make-build as make-all
207+
FROM $MAKE_BUILD as make-all
207208
# The docbuild needs quite some RAM (as of May 2018). It sometimes calls
208209
# os.fork() to spawn an external program which then exceeds easily the
209210
# overcommit limit of the system (no RAM is actually used, but this limit is

0 commit comments

Comments
 (0)