You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: Docker/README.md
+53-11Lines changed: 53 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,19 +8,61 @@ A container is an isolated environment based on a specific operating system (OS)
8
8
In this work the containers are created using software [Docker](https://www.docker.com). A Docker container is built from a dockerfile. After it is created, it is saved as an image that can be copied anywhere is needed or uploaded on the cloud [DockerHub](https://hub.docker.com).
9
9
10
10
## Dockerfile
11
-
In the folder Docker/ there are two subfolders ubuntu20/ and ubuntu22/ containing a dockerfile each. A dockerfile usually begins with an image import of the environment OS, which in this case is Ubuntu-20.04 and Ubuntu-22.04.
12
-
For more details about the dockerfiles created in this work, refer to the DockerHub page[personalDockerHub](https://hub.docker.com/repository/docker/dcodoni/lib/general).
11
+
In the folder Docker/ there are three subfolders solver/, ubuntu20/, ubuntu22/ containing a dockerfile each. A dockerfile usually begins with an image import of the OS, which in this case is Ubuntu-20.04 and Ubuntu-22.04. It could also start by importing an image that contain the desired environment such as in the dockerfile in the solver/ folder.
12
+
For more details about the dockerfiles created in this work, refer to the [simvascular DockerHub page](https://registry.hub.docker.com/u/simvascular).
13
13
14
14
## Build a container
15
-
Once the dockerfile is created, it must be built to oget the final image of the container. This can be done by moving to the directory where dockerfile is and run the command:
16
-
17
-
docker buildx build -t RepositoryName:tagImage .
18
-
19
-
where -t allows the user to set the name for the image created.
20
-
If your file is not named dockerfile, then in order to build a particular file just use:
In this section the steps to build the image with a pre-compiled svFSIplus solver are briefly described. To create the image from the dockerfile provided in Docker/solver, follow the steps below:
16
+
1) build an Ubuntu-based image containing the whole environment in which svFSIplus program can be compiled. The provided dockerfiles are based on Ubuntu-20.04 and Ubuntu-22.04, but they can be easily adapted to use the latest version of Ubuntu, by changing the following line in Docker/ubuntu20/dockerfile or Docker/ubuntu22/dockerfile:
17
+
```
18
+
FROM ubuntu:20.04 AS base / FROM ubuntu:22.04 AS base
19
+
```
20
+
to
21
+
```
22
+
FROM ubuntu:latest AS base
23
+
```
24
+
Build the environmnet Docker image:
25
+
```
26
+
cd Docker/ubuntu20 or cd Docker/ubuntu22
27
+
```
28
+
```
29
+
docker build -t RepositoryName:tagImage .
30
+
```
31
+
where -t allows the user to set the name for the image created. For example:
32
+
```
33
+
docker build -t libraries:latest .
34
+
```
35
+
2) build the image containing the compiled svFSIplus program. This image will be based on the environment created in the previous step (libraries:latest). In order to do this, open the Docker/solver/dockerfile and modify the following lines:
36
+
```
37
+
FROM simvascular/libraries:ubuntu22 AS builder
38
+
```
39
+
to
40
+
```
41
+
FROM libraries:latest AS builder
42
+
```
43
+
and
44
+
```
45
+
FROM simvascular/libraries:ubuntu22 AS final
46
+
```
47
+
to
48
+
```
49
+
FROM libraries:latest AS final
50
+
```
51
+
Build the solver image:
52
+
```
53
+
cd Docker/solver
54
+
```
55
+
```
56
+
docker build -t solver:latest .
57
+
```
58
+
The image include the PETSc-based svFSIplus executable in:
59
+
```
60
+
/build-petsc/svFSIplus-build/bin/svfsiplus
61
+
```
62
+
and the Trilinos-based svFSIplus executable in:
63
+
```
64
+
/build-trilinos/svFSIplus-build/bin/svfsiplus
65
+
```
24
66
## Run a container
25
67
Once the image is created, it can be run interactively by running the following command:
@@ -29,6 +29,120 @@ The [SimVascular svFSIplus Documentation](https://simvascular.github.io/document
29
29
30
30
The [svFSIplus Internal Code Documentation](https://simvascular.github.io/svfsiplus/index.html) provides documentation of the svFSIplus source code. It is automatically generated using [Doxygen](https://www.doxygen.nl).
The preferred way to use svFSIplus, is to take advantage of the provided Docker container, which include the latest version of svFSIplus pre-compiled. To use this option, Docker must be installed first. Please refer to [Docker webpage](https://www.docker.com/products/docker-desktop/) to know more about Docker and how to install it on your machine. The following steps describe how to build a Docker image or pull an existent one from DockerHub, and how to run a Docker container. The last section is a brief guide to perform the same steps but in Singularity, since HPC systems usually use Singularity to handle containers.
38
+
39
+
## Docker image
40
+
A Docker image is a read-only template that may contain dependencies, libraries, and everything needed to run a program. It is like a snapshot of a particular environment.
41
+
A Docker image can be created directly from a [dockerfile](https://docs.docker.com/reference/dockerfile/#:~:text=A%20Dockerfile%20is%20a%20text,can%20use%20in%20a%20Dockerfile.) or an existent image can be pulled from [DockerHub](https://hub.docker.com). For this repository, both options are available.
42
+
The latest version of svFSIplus program is pre-compiled in a Docker image, built from a dockerfile provided in Docker/solver. The Docker image includes two different type of builds, one where the solver is compiled with Trilinos and the other one where the solver is compiled with PETSc.
43
+
This Docker image can be downloaded (pulled) from the dockerhub simvascular repository [simvascular/solver](https://registry.hub.docker.com/u/simvascular). To pull an image, run the command:
44
+
```
45
+
docker pull simvascular/solver:latest
46
+
```
47
+
Note that this image was built for AMD64 (x86) architecture, and it will not work on other architectures such as ARM64 (AArch64, also note that the Apple M-series processors are based on ARM-type architectures). In this case, the image has to be built from the provided dockerfile. Please refer to the README inside Docker/ folder for more information on how to build images from the provided dockerfiles.
48
+
49
+
## Docker container
50
+
A Docker container is a running instance of a Docker image. It is a lightweight, isolated, and executable unit.
51
+
Once the image is created, it can be run interactively by running the following command:
52
+
```
53
+
docker run -it -v FolderToUpload:/NameOfFolder solver:latest
54
+
```
55
+
In this command:
56
+
- -it: means run interactively Docker image
57
+
- -v: mounts a directory 'FolderToUpload' from the host machine in the container where the directory has the name '/NameOfFolder'. For example the folder containing the mesh and the input file necessary to run a simulation should be mounted. Once inside the container we can move into the folder jsut mounted and run the simulation, for example with the following command:
The previous command will run the solver on 4 processors using the input file svFSIplus.xml and the mesh in the folder 'FolderToUpload' mounted inside the container.
62
+
As an example if we want to run the test case in tests/cases/fluid/pipe_RCR_3d we can proceed as follows:
63
+
```
64
+
docker run -it -v ~/full_path_to/tests/cases/fluid/pipe_RCR_3d:/case solver:latest
65
+
```
66
+
Now we are inside the container and we run the simulation:
Most of the HPC systems (if not all) are based on AMD64 architecture and the solver image can be directly pulled from [simvascular/solver](https://hub.docker.com/r/simvascular/solver). First of all, make sure the singularity module is loaded on the HPC system. Then, pull the solver image (it is recommended to run the following command on the compute node for example through an interactive job):
After the pull is complete, you should have a file with extension .sif (solver image). This image contains the two executables of the svFSIplus program build with PETSc and Trilinos support, respectively.
83
+
In the following, we provide two example of job submission's scripts that can be used as a reference to run a simulation using the svFSIplus solver on an HPC cluster.
84
+
1) single-node job script:
85
+
```
86
+
#!/bin/bash
87
+
#SBATCH --job-name
88
+
#SBATCH --output
89
+
#SBATCH --partition
90
+
#SBATCH --nodes=1
91
+
#SBATCH --ntasks-per-node=
92
+
#SBATCH --mem=0
93
+
#SBATCH -t 48:00:00
94
+
95
+
NTASKS = # number of tasks
96
+
FOLDER_TO_BIND1 = # path to folder to bind to the container (it will be accessible to the container)
97
+
FOLDER_TO_BIND2 = # path to folder to bind to the container (it will be accessible to the container)
98
+
PATH_TO_IMAGE = # full path to image, including the image name (*.sif file)
99
+
100
+
# For single node, no modules should be loaded to avoid incongruences between HPC and containers environments
101
+
module purge
102
+
103
+
singularity run --bind $FOLDER_TO_BIND1, $FOLDER_TO_BIND2, # and so on \
Since the multi-node relies on both MPI, the one on the HPC and the one inside the container, there may be some problems. In the following, we give a solution (workaround) for two common problems:
137
+
- if the HPC OpenMPI was built with cuda support, then it may happen that it is expecting that OpenMPI inside the container to be built with cuda support too, which is not the case. Possible solution is to add --mca mpi_cuda_support 0:
@@ -273,15 +387,4 @@ A simulation can be run in parallel on four processors using
273
387
```
274
388
mpiexec -np 4 svfsiplus fluid3.xml
275
389
```
276
-
In this case a directory named `4-procs` containing the simulation results output will be created. Results from different processors will be combined into a single file for a given time step.
In this case a directory named `4-procs` containing the simulation results output will be created. Results from different processors will be combined into a single file for a given time step.
0 commit comments