Skip to content

Commit 5d643d3

Browse files
jgaucher-csjgaucher-cs
authored
Merge pull request #194 from RS-PYTHON/feat/s1-ard
Replace dask-eopf image by dask-l0 and dask-s1ard
2 parents c9acfa0 + 1c9a042 commit 5d643d3

File tree

6 files changed

+57
-15
lines changed

6 files changed

+57
-15
lines changed

‎.github/common/resources/requirements-dask-eopf.txt‎ renamed to ‎.github/common/resources/requirements-dask-l0.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@
1919
--extra-index-url https://__token__:${GITLAB_EOPF_TOKEN}@gitlab.eopf.copernicus.eu/api/v4/projects/14/packages/pypi/simple # eopf cpm
2020
#--extra-index-url https://__token__:${GITLAB_EOPF_TOKEN}@gitlab.eopf.copernicus.eu/api/v4/projects/102/packages/pypi/simple # l0
2121
#l0==1.0.0
22-
git+https://__token__:${GITLAB_EOPF_TOKEN}@gitlab.eopf.copernicus.eu/vincent.privat/l0@rspy
22+
git+https://__token__:${GITLAB_EOPF_TOKEN}@gitlab.eopf.copernicus.eu/jgaucher/l0@develop
2323

2424
# other python dependencies
2525
graphviz

‎.github/common/resources/requirements-dask-s1ard.txt‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Copyright 2025 CS Group
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
# --extra-index-url https://gitlab.eopf.copernicus.eu/api/v4/projects/94/packages/pypi/simple # orekit jcc
16+
# --extra-index-url https://gitlab.eopf.copernicus.eu/api/v4/projects/67/packages/pypi/simple # sxgeo
17+
# --extra-index-url https://gitlab.eopf.copernicus.eu/api/v4/projects/78/packages/pypi/simple # pyrugged
18+
# --extra-index-url https://gitlab.eopf.copernicus.eu/api/v4/projects/52/packages/pypi/simple # asgard
19+
# --extra-index-url https://gitlab.eopf.copernicus.eu/api/v4/projects/14/packages/pypi/simple # eopf cpm
20+
--extra-index-url https://__token__:${GITLAB_EOPF_TOKEN}@gitlab.eopf.copernicus.eu/api/v4/projects/114/packages/pypi/simple # s1-ard-core
21+
22+
git+https://__token__:${GITLAB_EOPF_TOKEN}@gitlab.eopf.copernicus.eu/vincent.privat/s1-l12-rp@rspy
23+
24+
# other python dependencies
25+
graphviz

‎.github/dask-gateway/Dockerfile‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -21,8 +21,7 @@ ARG DASK_GATEWAY_TAG=2024.1.0
2121
USER root
2222

2323
COPY ./*.whl ./
24-
COPY ./resources/requirements-dask-eopf.txt /opt/
25-
COPY ./resources/requirements-dask-staging.txt /opt/
24+
COPY ./resources/requirements-dask-*.txt /opt/
2625
COPY ./resources/layer-cleanup.sh /usr/local/bin/
2726
RUN chmod 755 /usr/local/bin/layer-cleanup.sh
2827

@@ -44,7 +43,7 @@ RUN --mount=type=secret,id=GITLAB_EOPF_TOKEN \
4443
RS_SERVER_COMMON_WHEEL=$(find . -maxdepth 1 -type f -name 'rs_server_common-*-py3-none-any.whl' | head -n 1) \
4544
RS_SERVER_STAGING_WHEEL=$(find . -maxdepth 1 -type f -name 'rs_server_staging-*-py3-none-any.whl' | head -n 1) \
4645
pip install -U -r /opt/requirements-${IMAGE2BUILD}.txt && \
47-
rm -f /opt/Dockerfile.requirements.txt /opt/requirements-dask-eopf.txt /opt/requirements-dask-staging.txt ./*.whl && \
46+
rm -f /opt/Dockerfile.requirements.txt /opt/requirements-dask-*.txt ./*.whl && \
4847
layer-cleanup.sh
4948

5049
# Install Dask with correct version
@@ -62,7 +61,7 @@ RUN pip install opentelemetry-distro opentelemetry-exporter-otlp opentelemetry-i
6261
opentelemetry-bootstrap -a install && \
6362
layer-cleanup.sh
6463

65-
RUN if [ "${IMAGE2BUILD}" = "dask-eopf" ]; then \
64+
RUN if [ "${IMAGE2BUILD}" != "dask-staging" ]; then \
6665
apt update && apt install -y python3-pydot graphviz && layer-cleanup.sh; \
6766
fi
6867
RUN apt autoremove -y gcc && layer-cleanup.sh
@@ -76,6 +75,11 @@ WORKDIR /home/user
7675
# https://github.com/opencontainers/image-spec/blob/main/annotations.md#pre-defined-annotation-keys
7776
#
7877
LABEL org.opencontainers.image.source="https://github.com/RS-PYTHON/rs-infra-core"
79-
# ghcr.io/rs-python/rs-infra-core-dask-staging or ghcr.io/rs-python/rs-infra-core-dask-eopf
8078
LABEL org.opencontainers.image.ref.name="ghcr.io/rs-python/rs-infra-core-${IMAGE2BUILD}"
8179
LABEL dockerfile.url="https://github.com/RS-PYTHON/rs-infra-core/blob/develop/.github/dask-gateway/Dockerfile"
80+
81+
# This Dockerfile is used to build:
82+
# ghcr.io/rs-python/rs-infra-core-dask-staging
83+
# ghcr.io/rs-python/rs-infra-core-dask-l0
84+
# ghcr.io/rs-python/rs-infra-core-dask-s1ard
85+

‎.github/dask-gateway/Dockerfile.mockup‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,5 @@ WORKDIR /home/user
9898
# https://github.com/opencontainers/image-spec/blob/main/annotations.md#pre-defined-annotation-keys
9999
#
100100
LABEL org.opencontainers.image.source="https://github.com/RS-PYTHON/rs-infra-core"
101-
# ghcr.io/rs-python/rs-infra-core-dask-staging or ghcr.io/rs-python/rs-infra-core-dask-eopf
102-
LABEL org.opencontainers.image.ref.name="ghcr.io/rs-python/rs-infra-core-${IMAGE2BUILD}"
101+
LABEL org.opencontainers.image.ref.name="ghcr.io/rs-python/rs-infra-core-dask-eopf-mockup"
103102
LABEL dockerfile.url="https://github.com/RS-PYTHON/rs-infra-core/blob/develop/.github/dask-gateway/Dockerfile.mockup"

‎.github/workflows/publish-image.yml‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -93,16 +93,19 @@ jobs:
9393
publish-docker-matrix:
9494
strategy:
9595
matrix:
96-
image: [jupyter, dask-staging, dask-eopf]
96+
# path: relative path on disk where to find resources and Dockerfile
97+
# image: used for the name of the docker image to build
9798
include:
9899
- path: jupyter
99100
image: jupyter
100101
- path: dask-gateway
101102
image: dask-staging
102-
- path: dask-gateway
103-
image: dask-eopf
104103
- path: dask-gateway
105104
image: dask-eopf-mockup
105+
- path: dask-gateway
106+
image: dask-l0
107+
- path: dask-gateway
108+
image: dask-s1ard
106109
if: github.actor != 'dependabot[bot]'
107110
runs-on: ubuntu-latest
108111
name: "Docker image for ${{ matrix.image }}"

‎docs/how-to/Dask-gateway.md‎

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -41,11 +41,17 @@ options = gateway.cluster_options()
4141
for key in options.keys():
4242
print(f"{key}: {options[key]}")
4343

44+
# NOTE: use one of the following images:
45+
# - ghcr.io/rs-python/rs-infra-core-dask-staging
46+
# - ghcr.io/rs-python/rs-infra-core-dask-l0
47+
# - ghcr.io/rs-python/rs-infra-core-dask-s1ard
48+
# - ...
49+
4450
cluster = gateway.new_cluster(
4551
worker_cores=1,
4652
worker_memory=2.0,
4753
namespace='dask-gateway',
48-
image='ghcr.io/rs-python/rs-infra-core-dask-eopf:latest'
54+
image='ghcr.io/rs-python/rs-infra-core-dask-l0:latest'
4955
)
5056

5157
print (cluster.name)
@@ -150,11 +156,16 @@ options = gateway.cluster_options()
150156
for key in options.keys():
151157
print(f"{key}: {options[key]}")
152158

159+
# NOTE: use one of the following images and cluster names:
160+
# - ghcr.io/rs-python/rs-infra-core-dask-l0 cluster_name:dask-l0
161+
# - ghcr.io/rs-python/rs-infra-core-dask-s1ard cluster_name:dask-s1ard
162+
# - ...
163+
153164
cluster = gateway.new_cluster(
154165
namespace='dask-gateway',
155-
image='ghcr.io/rs-python/rs-infra-core-dask-eopf:latest',
156-
cluster_name='dask-eopf',
157-
scheduler_extra_pod_labels={'cluster_name': 'dask-eopf'},
166+
image='ghcr.io/rs-python/rs-infra-core-dask-l0:latest',
167+
cluster_name='dask-l0',
168+
scheduler_extra_pod_labels={'cluster_name': 'dask-l0'},
158169
worker_cores=1,
159170
worker_memory=2.0, # In GB
160171
cluster_max_workers=3,

0 commit comments

Comments
 (0)