Skip to content

Commit 199aaeb

Browse files
Malmahrouqi3Malmahrouqi3sbryngelson
authored
MFC Containerization (#971)
Co-authored-by: Malmahrouqi3 <[email protected]> Co-authored-by: Spencer Bryngelson <[email protected]> Co-authored-by: Spencer Bryngelson <[email protected]>
1 parent 4f8fb91 commit 199aaeb

File tree

7 files changed

+504
-29
lines changed

7 files changed

+504
-29
lines changed

‎.devcontainer/devcontainer.json‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"name": "MFC Container",
3+
"image": "sbryngelson/mfc:latest-cpu",
4+
"workspaceFolder": "/opt/MFC",
5+
"settings": {
6+
"terminal.integrated.shell.linux": "/bin/bash",
7+
"editor.formatOnSave": true
8+
},
9+
}

‎.github/.dockerignore‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
node_modules/
2+
package.json
3+
yarn.lock
4+
5+
.venv/
6+
.vscode/
7+
src/*/autogen/
8+
9+
*.swo
10+
*.swp
11+
12+
*:Zone.Identifier
13+
14+
.nfs*
15+
16+
__pycache__
17+
18+
*.egg-info
19+
20+
.DS_Store
21+
22+
# NVIDIA Nsight Compute
23+
*.nsys-rep
24+
*.sqlite
25+
26+
docs/*/initial*
27+
docs/*/result*
28+
docs/documentation/*-example.png
29+
docs/documentation/examples.md
30+
31+
examples/*batch/*/
32+
examples/**/D/*
33+
examples/**/p*
34+
examples/**/D_*
35+
examples/**/*.inf
36+
examples/**/*.inp
37+
examples/**/*.o*
38+
examples/**/silo*
39+
examples/**/restart_data*
40+
examples/**/*.out
41+
examples/**/binary
42+
examples/**/fort.1
43+
examples/**/*.sh
44+
examples/**/*.err
45+
examples/**/viz/
46+
examples/*.jpg
47+
examples/*.png
48+
examples/*/workloads/
49+
examples/*/run-*/
50+
examples/*/logs/
51+
examples/**/*.f90
52+
workloads/
53+
54+
benchmarks/*batch/*/
55+
benchmarks/*/D/*
56+
benchmarks/*/p*
57+
benchmarks/*/D_*
58+
benchmarks/*/*.inf
59+
benchmarks/*/*.inp
60+
benchmarks/*/*.dat
61+
benchmarks/*/*.o*
62+
benchmarks/*/silo*
63+
benchmarks/*/restart_data*
64+
benchmarks/*/*.out
65+
benchmarks/*/binary
66+
benchmarks/*/fort.1
67+
benchmarks/*/*.sh
68+
benchmarks/*/*.err
69+
benchmarks/*/viz/
70+
benchmarks/*.jpg
71+
benchmarks/*.png
72+
73+
*.mod
74+
75+
# Video Files
76+
*.mp4
77+
*.mov
78+
*.mkv
79+
*.avi

‎.github/Dockerfile‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
ARG BASE_IMAGE
2+
FROM ${BASE_IMAGE}
3+
4+
ARG TARGET
5+
ARG CC_COMPILER
6+
ARG CXX_COMPILER
7+
ARG FC_COMPILER
8+
ARG COMPILER_PATH
9+
ARG COMPILER_LD_LIBRARY_PATH
10+
11+
RUN apt-get update -y && \
12+
if [ "$TARGET" != "gpu" ]; then \
13+
apt-get install -y \
14+
build-essential git make cmake gcc g++ gfortran bc\
15+
python3 python3-venv python3-pip \
16+
openmpi-bin libopenmpi-dev libfftw3-dev \
17+
mpich libmpich-dev; \
18+
else \
19+
apt-get install -y \
20+
build-essential git make cmake bc\
21+
python3 python3-venv python3-pip \
22+
libfftw3-dev \
23+
openmpi-bin libopenmpi-dev; \
24+
fi && \
25+
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
26+
27+
ENV OMPI_ALLOW_RUN_AS_ROOT=1
28+
ENV OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1
29+
ENV PATH="/opt/MFC:$PATH"
30+
31+
COPY ../ /opt/MFC
32+
33+
ENV CC=${CC_COMPILER}
34+
ENV CXX=${CXX_COMPILER}
35+
ENV FC=${FC_COMPILER}
36+
ENV PATH="${COMPILER_PATH}:$PATH"
37+
ENV LD_LIBRARY_PATH="${COMPILER_LD_LIBRARY_PATH}:${LD_LIBRARY_PATH:-}"
38+
39+
RUN echo "TARGET=$TARGET CC=$CC_COMPILER FC=$FC_COMPILER" && \
40+
cd /opt/MFC && \
41+
if [ "$TARGET" = "gpu" ]; then \
42+
./mfc.sh build --gpu -j $(nproc); \
43+
else \
44+
./mfc.sh build -j $(nproc); \
45+
fi
46+
47+
RUN cd /opt/MFC && \
48+
if [ "$TARGET" = "gpu" ]; then \
49+
./mfc.sh test -a --dry-run --gpu -j $(nproc); \
50+
else \
51+
./mfc.sh test -a --dry-run -j $(nproc); \
52+
fi
53+
54+
WORKDIR /opt/MFC
55+
ENTRYPOINT ["tail", "-f", "/dev/null"]

‎.github/workflows/docker.yml‎

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
name: Containerization
2+
3+
on:
4+
release:
5+
types: [published]
6+
workflow_dispatch:
7+
inputs:
8+
tag:
9+
description: 'tag to containerize'
10+
required: true
11+
12+
concurrency:
13+
group: Containerization
14+
cancel-in-progress: false
15+
16+
jobs:
17+
Container:
18+
strategy:
19+
matrix:
20+
config:
21+
- { name: 'cpu', runner: 'ubuntu-22.04', base_image: 'ubuntu:22.04' }
22+
- { name: 'gpu', runner: 'ubuntu-22.04', base_image: 'nvcr.io/nvidia/nvhpc:23.11-devel-cuda_multi-ubuntu22.04' }
23+
- { name: 'gpu', runner: 'ubuntu-22.04-arm', base_image: 'nvcr.io/nvidia/nvhpc:23.11-devel-cuda_multi-ubuntu22.04' }
24+
runs-on: ${{ matrix.config.runner }}
25+
outputs:
26+
tag: ${{ steps.clone.outputs.tag }}
27+
steps:
28+
- name: Free Disk Space
29+
uses: jlumbroso/free-disk-space@main
30+
with:
31+
tool-cache: false
32+
android: true
33+
dotnet: true
34+
haskell: true
35+
large-packages: true
36+
docker-images: true
37+
swap-storage: true
38+
39+
- name: Login
40+
uses: docker/login-action@v3
41+
with:
42+
username: ${{ secrets.DOCKERHUB_USERNAME }}
43+
password: ${{ secrets.DOCKERHUB_PASSWORD }}
44+
45+
- name: Setup Buildx
46+
uses: docker/setup-buildx-action@v3
47+
48+
- name: Setup QEMU
49+
uses: docker/setup-qemu-action@v3
50+
51+
- name: Clone
52+
id: clone
53+
run: |
54+
TAG="${{ github.event.inputs.tag || github.ref_name }}"
55+
echo "tag=$TAG" >> $GITHUB_OUTPUT
56+
echo "TAG=$TAG" >> $GITHUB_ENV
57+
git clone --branch "$TAG" --depth 1 https://github.com/MFlowCode/MFC.git mfc
58+
59+
- name: Stage
60+
run: |
61+
sudo fallocate -l 8G /swapfile
62+
sudo chmod 600 /swapfile
63+
sudo mkswap /swapfile
64+
sudo swapon /swapfile
65+
sudo mkdir -p /home/runner/tmp
66+
export TMPDIR=/home/runner/tmp
67+
free -h
68+
sudo mkdir -p /mnt/share
69+
sudo chmod 777 /mnt/share
70+
cp -r mfc/* /mnt/share/
71+
cp -r mfc/.git /mnt/share/.git
72+
cp mfc/.github/Dockerfile /mnt/share/
73+
cp mfc/.github/.dockerignore /mnt/share/
74+
docker buildx create --name mfcbuilder --driver docker-container --use
75+
76+
- name: Build and push image (cpu)
77+
if: ${{ matrix.config.name == 'cpu' }}
78+
uses: docker/build-push-action@v6
79+
with:
80+
builder: mfcbuilder
81+
context: /mnt/share
82+
file: /mnt/share/Dockerfile
83+
platforms: linux/amd64,linux/arm64
84+
build-args: |
85+
BASE_IMAGE=${{ matrix.config.base_image }}
86+
TARGET=${{ matrix.config.name }}
87+
CC_COMPILER=${{ 'gcc' }}
88+
CXX_COMPILER=${{ 'g++' }}
89+
FC_COMPILER=${{ 'gfortran' }}
90+
COMPILER_PATH=${{ '/usr/bin' }}
91+
COMPILER_LD_LIBRARY_PATH=${{ '/usr/lib' }}
92+
tags: ${{ secrets.DOCKERHUB_USERNAME }}/mfc:${{ env.TAG }}-${{ matrix.config.name }}
93+
push: true
94+
95+
- name: Build and push image (gpu)
96+
if: ${{ matrix.config.name == 'gpu' }}
97+
uses: docker/build-push-action@v5
98+
with:
99+
builder: default
100+
context: /mnt/share
101+
file: /mnt/share/Dockerfile
102+
build-args: |
103+
BASE_IMAGE=${{ matrix.config.base_image }}
104+
TARGET=${{ matrix.config.name }}
105+
CC_COMPILER=${{ 'nvc' }}
106+
CXX_COMPILER=${{ 'nvc++' }}
107+
FC_COMPILER=${{ 'nvfortran' }}
108+
COMPILER_PATH=${{ '/opt/nvidia/hpc_sdk/Linux_x86_64/compilers/bin' }}
109+
COMPILER_LD_LIBRARY_PATH=${{ '/opt/nvidia/hpc_sdk/Linux_x86_64/compilers/lib' }}
110+
tags: ${{ secrets.DOCKERHUB_USERNAME }}/mfc:${{ env.TAG }}-${{ matrix.config.name }}-${{ matrix.config.runner}}
111+
push: true
112+
113+
manifests:
114+
runs-on: ubuntu-latest
115+
needs: Container
116+
steps:
117+
- name: Login
118+
uses: docker/login-action@v3
119+
with:
120+
username: ${{ secrets.DOCKERHUB_USERNAME }}
121+
password: ${{ secrets.DOCKERHUB_PASSWORD }}
122+
123+
- name: Create and Push Manifest Lists
124+
env:
125+
TAG: ${{ needs.Container.outputs.tag }}
126+
REGISTRY: ${{ secrets.DOCKERHUB_USERNAME }}/mfc
127+
run: |
128+
docker buildx imagetools create -t $REGISTRY:latest-cpu $REGISTRY:$TAG-cpu
129+
docker manifest create $REGISTRY:$TAG-gpu $REGISTRY:$TAG-gpu-ubuntu-22.04 $REGISTRY:$TAG-gpu-ubuntu-22.04-arm
130+
docker manifest create $REGISTRY:latest-gpu $REGISTRY:$TAG-gpu-ubuntu-22.04 $REGISTRY:$TAG-gpu-ubuntu-22.04-arm
131+
docker manifest push $REGISTRY:$TAG-gpu
132+
docker manifest push $REGISTRY:latest-gpu

‎README.md‎

Lines changed: 24 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,9 @@
2727

2828
**Welcome!**
2929
MFC simulates compressible multi-phase flows, [among other things](#what-else-can-this-thing-do).
30-
It uses metaprogramming to stay short and portable (~20K lines).
31-
MFC conducted the largest known, open CFD simulation at <a href="https://arxiv.org/abs/2505.07392" target="_blank">200 trillion grid points</a>, and 1 quadrillion degrees of freedom (as of September 2025), and is a 2025 Gordon Bell Prize finalist.
30+
It uses metaprogramming and is short (20K lines) and portable.
31+
MFC conducted the largest known CFD simulation at <a href="https://arxiv.org/abs/2505.07392" target="_blank">200 trillion grid points</a>, and 1 quadrillion degrees of freedom (as of September 2025).
32+
MFC is a 2025 Gordon Bell Prize Finalist.
3233

3334
<p align="center">
3435
<a href="https://doi.org/10.48550/arXiv.2503.07953" target="_blank">
@@ -76,7 +77,7 @@ This one simulates high-Mach flow over an airfoil:
7677
<img src="docs/res/airfoil.png" alt="Airfoil Example" width="700"/><br/>
7778
</p>
7879

79-
And here is a high amplitude acoustic wave reflecting and emerging through a circular orifice:
80+
And here is a high-amplitude acoustic wave reflecting and emerging through a circular orifice:
8081

8182
<p align="center">
8283
<img src="docs/res/orifice.png" alt="Orifice Example" width="700"/><br/>
@@ -85,15 +86,23 @@ And here is a high amplitude acoustic wave reflecting and emerging through a cir
8586

8687
## Getting started
8788

88-
You can navigate [to this webpage](https://mflowcode.github.io/documentation/md_getting-started.html) to get started using MFC!
89+
For a _very_ quick start, open a GitHub Codespace to load a pre-configured Docker container and familiarize yourself with MFC commands.
90+
Click <kbd> <> Code</kbd> (green button at top right) → <kbd>Codespaces</kbd> (right tab) → <kbd>+</kbd> (create a codespace).
91+
92+
> ****Note:**** Codespaces is a free service with a monthly quota of compute time and storage usage.
93+
> It is recommended for testing commands, troubleshooting, and running simple case files without installing dependencies or building MFC on your device.
94+
> Don't conduct any critical work here!
95+
> To learn more, please see [how Docker & Containers work](https://mflowcode.github.io/documentation/docker.html).
96+
97+
You can navigate [to this webpage](https://mflowcode.github.io/documentation/md_getting-started.html) to get you get started using MFC on your local machine, cluster, or supercomputer!
8998
It's rather straightforward.
90-
We'll give a brief intro. here for MacOS.
99+
We'll give a brief introdocution for MacOS below.
91100
Using [brew](https://brew.sh), install MFC's dependencies:
92101
```shell
93102
brew install coreutils python cmake fftw hdf5 gcc boost open-mpi lapack
94103
```
95104
You're now ready to build and test MFC!
96-
Put it to a convenient directory via
105+
Put it to a local directory via
97106
```shell
98107
git clone https://github.com/MFlowCode/MFC
99108
cd MFC
@@ -123,17 +132,14 @@ You can visualize the output data in `examples/3d_shockdroplet/silo_hdf5` via Pa
123132
## Is this _really_ exascale?
124133

125134
[OLCF Frontier](https://www.olcf.ornl.gov/frontier/) is the first exascale supercomputer.
126-
The weak scaling of MFC on this machine shows near-ideal utilization.
135+
The weak scaling of MFC on this machine shows near-ideal utilization.
136+
We also scale ideally to >98% of LLNL El Capitan.
127137

128138
<p align="center">
129139
<img src="docs/res/scaling.png" alt="Scaling" width="400"/>
130140
</p>
131141

132-
133-
## What else can this thing do
134-
135-
MFC has many features.
136-
They are organized below.
142+
## What else can this thing do?
137143

138144
### Physics
139145

@@ -209,7 +215,7 @@ They are organized below.
209215

210216
If you use MFC, consider citing it as below.
211217
Ref. 1 includes all modern MFC features, including GPU acceleration and many new physics features.
212-
If referencing MFC's (GPU) performance, consider citing ref. 1 and 2, which describe the solver and how it was crafted.
218+
If referencing MFC's (GPU) performance, consider citing ref. 1 and 2, which describe the solver and its design.
213219
The original open-source release of MFC is ref. 3, which should be cited for provenance as appropriate.
214220

215221
```bibtex
@@ -249,11 +255,11 @@ MFC is under the MIT license (see [LICENSE](LICENSE) for full text).
249255

250256
## Acknowledgements
251257

252-
Federal sponsors have supported MFC development, including the US Department of Defense (DOD), the National Institutes of Health (NIH), the Department of Energy (DOE), and the National Science Foundation (NSF).
258+
Federal sponsors have supported MFC development, including the US Department of Defense (DOD), the National Institutes of Health (NIH), the Department of Energy (DOE) and National Nuclear Security Administration (NNSA), and the National Science Foundation (NSF).
253259

254260
MFC computations have used many supercomputing systems. A partial list is below
255-
* OLCF Frontier and Summit, and testbeds Wombat, Crusher, and Spock (allocation CFD154, PI Bryngelson)
256-
* LLNL El Capitan, Tuolumne, and Lassen; El Capitan early access system Tioga
261+
* OLCF Frontier and Summit, and testbeds Wombat, Crusher, and Spock (allocation CFD154, PI Bryngelson).
262+
* LLNL El Capitan, Tuolumne, and Lassen; El Capitan early access system Tioga.
257263
* NCSA Delta and DeltaAI, PSC Bridges(1/2), SDSC Comet and Expanse, Purdue Anvil, TACC Stampede(1-3), and TAMU ACES via ACCESS-CI allocations from Bryngelson, Colonius, Rodriguez, and more.
258-
* DOD systems Blueback, Onyx, Carpenter, Nautilus, and Narwhal via the DOD HPCMP program
259-
* Sandia National Labs systems Doom and Attaway and testbed systems Weaver and Vortex
264+
* DOD systems Blueback, Onyx, Carpenter, Nautilus, and Narwhal via the DOD HPCMP program.
265+
* Sandia National Labs systems Doom and Attaway, and testbed systems Weaver and Vortex.

0 commit comments

Comments
 (0)