Merge branch 'master' into feat/repeat-tests

Author: Vasileios Karakasis

docs/config_reference.rst

@@ -472,6 +472,16 @@ ReFrame can launch containerized applications, but you need to configure properl
    - ``Singularity``: The `Singularity <https://sylabs.io/>`__ container runtime.
 
 
+.. js:attribute:: .systems[].partitions[].container_platforms[].default
+
+   :required: No
+
+   If set to ``true``, this is the default container platform of this partition.
+   If not specified, the default container platform is assumed to be the first in the list of :js:attr:`container_platforms`.
+
+   .. versionadded:: 3.12.0
+
+
 .. js:attribute:: .systems[].partitions[].container_platforms[].modules
 
    :required: No

docs/tutorial_advanced.rst

@@ -683,6 +683,9 @@ First, we need to enable the container platform support in ReFrame's configurati
 For each partition, users can define a list of container platforms supported using the :js:attr:`container_platforms` `configuration parameter <config_reference.html#.systems[].partitions[].container_platforms>`__.
 In this case, we define the `Sarus <https://github.com/eth-cscs/sarus>`__ platform for which we set the :js:attr:`modules` parameter in order to instruct ReFrame to load the ``sarus`` module, whenever it needs to run with this container platform.
 Similarly, we add an entry for the `Singularity <https://sylabs.io>`__ platform.
+Optionally, users are allowed to set the ``default`` attribute to :obj:`True` in order to mark a specific container platform as the default of that partition (see below on how this information is being used).
+If no default container platform is specified explicitly, then always the first in the list will be considered as successful.
+
 
 The following parameterized test, will create two tests, one for each of the supported container platforms:
 
@@ -698,14 +701,17 @@ The following parameterized test, will create two tests, one for each of the sup
 
 A container-based test can be written as :class:`~reframe.core.pipeline.RunOnlyRegressionTest` that sets the :attr:`~reframe.core.pipeline.RegressionTest.container_platform` attribute.
 This attribute accepts a string that corresponds to the name of the container platform that will be used to run the container for this test.
-If such a platform is not `configured <config_reference.html#container-platform-configuration>`__ for the current system, the test will fail.
+It is not necessary to specify this attribute, in which case, the default container platform of the current partition will be used.
+You can still differentiate your test based on the actual container platform that is being used by checking the ``self.container_platform.name`` variable.
+
+As soon as the container platform to be used is determined, you need to specify the container image to use by setting the :attr:`~reframe.core.containers.ContainerPlatform.image`.
+If the image is not specified, then the container logic is skipped and the test executes as if the :attr:`~reframe.core.pipeline.RegressionTest.container_platform` was never set.
 
-As soon as the container platform to be used is defined, you need to specify the container image to use by setting the :attr:`~reframe.core.containers.ContainerPlatform.image`.
 In the ``Singularity`` test variant, we add the ``docker://`` prefix to the image name, in order to instruct ``Singularity`` to pull the image from `DockerHub <https://hub.docker.com/>`__.
 The default command that the container runs can be overwritten by setting the :attr:`~reframe.core.containers.ContainerPlatform.command` attribute of the container platform.
 
 The :attr:`~reframe.core.containers.ContainerPlatform.image` is the only mandatory attribute for container-based checks.
-It is important to note that the :attr:`~reframe.core.pipeline.RegressionTest.executable` and :attr:`~reframe.core.pipeline.RegressionTest.executable_opts` attributes of the actual test are ignored in case of container-based tests.
+It is important to note that the :attr:`~reframe.core.pipeline.RegressionTest.executable` and :attr:`~reframe.core.pipeline.RegressionTest.executable_opts` attributes of the actual test are ignored if the containerized code path is taken, i.e., when :attr:`~reframe.core.containers.ContainerPlatform.image` is not :obj:`None`.
 
 ReFrame will run the container according to the given platform as follows:
 
@@ -759,6 +765,117 @@ For a complete list of the available attributes of a specific container platform
 On how to configure ReFrame for running containerized tests, please have a look at the :ref:`container-platform-configuration` section of the :doc:`config_reference`.
 
 
+.. versionchanged:: 3.12.0
+   There is no need any more to explicitly set the :attr:`container_platform` in the test.
+   This is automatically initialized from the default platform of the current partition.
+
+
+
+Combining containerized and native application tests
+====================================================
+
+.. versionadded:: 3.12.0
+
+It is very easy in ReFrame to have a single run-only test to either test the native or the containerized version of an application.
+This is possible, since the framework will only take the "containerized" code path only if the :attr:`~reframe.core.containers.ContainerPlatform.image` attribute of the :attr:`~reframe.core.pipeline.RegressionTest.container_platform` is not :obj:`None`.
+Otherwise, the *bare metal* version of the tested application will be run.
+The following test uses exactly this trick to test a series of GROMACS images as well as the native one provided on the Piz Daint supercomputer.
+It also extends the GROMACS benchmark tests that are provided with ReFrame's test library (see :doc:`hpctestlib`).
+For simplicity, we are assuming a single system here (the hybrid partition of Piz Daint) and we set fixed values for the :attr:`num_cpus_per_task` as well as the ``-ntomp`` option of GROMACS (NB: in a real-world test we would use the auto-detected processor topology information to set these values; see :ref:`proc-autodetection` for more information).
+We also redefine and restrict the benchmark's parameters ``benchmark_info`` and ``nb_impl`` to the values that are of interest for the demonstration of this test.
+Finally, we also reset the executable to use ``gmx`` instead of the ``gmx_mpi`` that is used from the library test.
+
+
+.. literalinclude:: ../tutorials/advanced/containers/gromacs_test.py
+   :start-after: # rfmdocstart: gromacstest
+
+All this test does in addition to the library test it inherits from is to set the :attr:`~reframe.core.containers.ContainerPlatform.image` and the :attr:`~reframe.core.containers.ContainerPlatform.command` attributes of the :attr:`~reframe.core.pipeline.RegressionTest.container_platform`.
+The former is set from the ``gromacs_image`` test parameter whereas the latter from the test's :attr:`~reframe.core.pipeline.RegressionTest.executable` and :attr:`~reframe.core.pipeline.RegressionTest.executable_opts` attributes.
+Remember that these attributes are ignored if the framework takes the path of launching  a container.
+Finally, if the image is :obj:`None` we handle the case of the native run, in which case we load the modules required to run GROMACS natively on the target system.
+
+In the following, we run the GPU version of a single benchmark with a series of images from NVIDIA and natively:
+
+.. code-block:: console
+
+   $ ./bin/reframe -C tutorials/config/settings.py -c tutorials/advanced/containers/gromacs_test.py -r
+
+.. code-block:: console
+
+   [==========] Running 6 check(s)
+   [==========] Started on Fri Jun 17 16:20:16 2022
+
+   [----------] start processing checks
+   [ RUN      ] gromacs_containerized_test %benchmark_info=HECBioSim/hEGFRDimerSmallerPL %nb_impl=gpu %gromacs_image=nvcr.io/hpc/gromacs:2022.1 @daint:gpu+gnu
+   [ RUN      ] gromacs_containerized_test %benchmark_info=HECBioSim/hEGFRDimerSmallerPL %nb_impl=gpu %gromacs_image=nvcr.io/hpc/gromacs:2021.3 @daint:gpu+gnu
+   [ RUN      ] gromacs_containerized_test %benchmark_info=HECBioSim/hEGFRDimerSmallerPL %nb_impl=gpu %gromacs_image=nvcr.io/hpc/gromacs:2021 @daint:gpu+gnu
+   [ RUN      ] gromacs_containerized_test %benchmark_info=HECBioSim/hEGFRDimerSmallerPL %nb_impl=gpu %gromacs_image=nvcr.io/hpc/gromacs:2020.2 @daint:gpu+gnu
+   [ RUN      ] gromacs_containerized_test %benchmark_info=HECBioSim/hEGFRDimerSmallerPL %nb_impl=gpu %gromacs_image=nvcr.io/hpc/gromacs:2020 @daint:gpu+gnu
+   [ RUN      ] gromacs_containerized_test %benchmark_info=HECBioSim/hEGFRDimerSmallerPL %nb_impl=gpu %gromacs_image=None @daint:gpu+gnu
+   [       OK ] (1/6) gromacs_containerized_test %benchmark_info=HECBioSim/hEGFRDimerSmallerPL %nb_impl=gpu %gromacs_image=nvcr.io/hpc/gromacs:2020.2 @daint:gpu+gnu
+   [       OK ] (2/6) gromacs_containerized_test %benchmark_info=HECBioSim/hEGFRDimerSmallerPL %nb_impl=gpu %gromacs_image=nvcr.io/hpc/gromacs:2020 @daint:gpu+gnu
+   [       OK ] (3/6) gromacs_containerized_test %benchmark_info=HECBioSim/hEGFRDimerSmallerPL %nb_impl=gpu %gromacs_image=None @daint:gpu+gnu
+   [       OK ] (4/6) gromacs_containerized_test %benchmark_info=HECBioSim/hEGFRDimerSmallerPL %nb_impl=gpu %gromacs_image=nvcr.io/hpc/gromacs:2022.1 @daint:gpu+gnu
+   [       OK ] (5/6) gromacs_containerized_test %benchmark_info=HECBioSim/hEGFRDimerSmallerPL %nb_impl=gpu %gromacs_image=nvcr.io/hpc/gromacs:2021 @daint:gpu+gnu
+   [       OK ] (6/6) gromacs_containerized_test %benchmark_info=HECBioSim/hEGFRDimerSmallerPL %nb_impl=gpu %gromacs_image=nvcr.io/hpc/gromacs:2021.3 @daint:gpu+gnu
+   [----------] all spawned checks have finished
+
+   [  PASSED  ] Ran 6/6 test case(s) from 6 check(s) (0 failure(s), 0 skipped)
+   [==========] Finished on Fri Jun 17 16:23:47 2022
+
+
+We can also inspect the generated job scripts for the native and a containerized run:
+
+.. code-block:: console
+
+   cat output/daint/gpu/gnu/gromacs_containerized_test_0/rfm_gromacs_containerized_test_0_job.sh
+
+.. code-block:: bash
+
+   #!/bin/bash
+   #SBATCH --job-name="rfm_gromacs_containerized_test_0_job"
+   #SBATCH --ntasks=1
+   #SBATCH --ntasks-per-node=1
+   #SBATCH --cpus-per-task=12
+   #SBATCH --output=rfm_gromacs_containerized_test_0_job.out
+   #SBATCH --error=rfm_gromacs_containerized_test_0_job.err
+   #SBATCH -A csstaff
+   #SBATCH --constraint=gpu
+   #SBATCH --hint=nomultithread
+   module unload PrgEnv-cray
+   module load PrgEnv-gnu
+   module load daint-gpu
+   module load GROMACS
+   curl -LJO https://github.com/victorusu/GROMACS_Benchmark_Suite/raw/1.0.0/HECBioSim/hEGFRDimerSmallerPL/benchmark.tpr
+   srun gmx mdrun -dlb yes -ntomp 12 -npme -1 -v -nb gpu -s benchmark.tpr
+
+And the containerized run:
+
+.. code-block:: console
+
+   cat output/daint/gpu/gnu/gromacs_containerized_test_1/rfm_gromacs_containerized_test_1_job.sh
+
+.. code-block:: bash
+
+   #!/bin/bash
+   #SBATCH --job-name="rfm_gromacs_containerized_test_1_job"
+   #SBATCH --ntasks=1
+   #SBATCH --ntasks-per-node=1
+   #SBATCH --cpus-per-task=12
+   #SBATCH --output=rfm_gromacs_containerized_test_1_job.out
+   #SBATCH --error=rfm_gromacs_containerized_test_1_job.err
+   #SBATCH -A csstaff
+   #SBATCH --constraint=gpu
+   #SBATCH --hint=nomultithread
+   module unload PrgEnv-cray
+   module load PrgEnv-gnu
+   module load sarus
+   curl -LJO https://github.com/victorusu/GROMACS_Benchmark_Suite/raw/1.0.0/HECBioSim/hEGFRDimerSmallerPL/benchmark.tpr
+   sarus pull nvcr.io/hpc/gromacs:2020
+   srun sarus run --mount=type=bind,source="/users/user/Devel/reframe/stage/daint/gpu/gnu/gromacs_containerized_test_43",destination="/rfm_workdir" -w /rfm_workdir nvcr.io/hpc/gromacs:2020 gmx mdrun -dlb yes -ntomp 12 -npme -1 -v -nb gpu -s benchmark.tpr
+
+
+
 Writing reusable tests
 ----------------------
 

reframe/core/containers.py

@@ -6,8 +6,9 @@
 import abc
 
 import reframe.core.fields as fields
+import reframe.core.warnings as warn
+import reframe.utility as util
 import reframe.utility.typecheck as typ
-from reframe.core.exceptions import ContainerError
 
 
 _STAGEDIR_MOUNT = '/rfm_workdir'
@@ -80,24 +81,18 @@ class ContainerPlatform(abc.ABC):
     #: :default: ``[]``
     options = fields.TypedField(typ.List[str])
 
-    _workdir = fields.TypedField(str, type(None))
     #: The working directory of ReFrame inside the container.
     #:
     #: This is the directory where the test's stage directory is mounted inside
     #: the container. This directory is always mounted regardless if
     #: :attr:`mount_points` is set or not.
     #:
-    #: .. deprecated:: 3.5
-    #:    Please use the `options` field to set the working directory.
-    #:
     #: :type: :class:`str`
     #: :default: ``/rfm_workdir``
-    workdir = fields.DeprecatedField(
-        _workdir,
-        'The `workdir` field is deprecated, please use the `options` field to '
-        'set the container working directory',
-        fields.DeprecatedField.OP_SET, from_version='3.5.0'
-    )
+    #:
+    #: .. versionchanged:: 3.12.0
+    #:    This attribute is no more deprecated.
+    workdir = fields.TypedField(str, type(None))
 
     def __init__(self):
         self.image = None
@@ -106,8 +101,8 @@ def __init__(self):
         # NOTE: Here we set the target fields directly to avoid the deprecation
         # warnings
         self._commands = []
-        self._workdir = _STAGEDIR_MOUNT
 
+        self.workdir = _STAGEDIR_MOUNT
         self.mount_points  = []
         self.options = []
         self.pull_image = True
@@ -143,12 +138,37 @@ def launch_command(self, stagedir):
         :arg stagedir: The stage directory of the test.
         '''
 
-    def validate(self):
-        if self.image is None:
-            raise ContainerError('no image specified')
+    @classmethod
+    def create(cls, name):
+        '''Factory method to create a new container by name.'''
+        name = name.capitalize()
+        try:
+            return globals()[name]()
+        except KeyError:
+            raise ValueError(f'unknown container platform: {name}') from None
+
+    @classmethod
+    def create_from(cls, name, other):
+        new = cls.create(name)
+        new.image = other.image
+        new.command = other.command
+        new.mount_points = other.mount_points
+        new.options = other.options
+        new.pull_image = other.pull_image
+        new.workdir = other.workdir
+
+        # Update deprecated fields
+        with warn.suppress_deprecations():
+            new.commands = other.commands
+
+        return new
+
+    @property
+    def name(self):
+        return type(self).__name__
 
     def __str__(self):
-        return type(self).__name__
+        return self.name
 
     def __rfm_json_encode__(self):
         return str(self)
@@ -165,15 +185,17 @@ def launch_command(self, stagedir):
         super().launch_command(stagedir)
         mount_points = self.mount_points + [(stagedir, _STAGEDIR_MOUNT)]
         run_opts = [f'-v "{mp[0]}":"{mp[1]}"' for mp in mount_points]
-        run_opts += self.options
+        if self.workdir:
+            run_opts.append(f'-w {self.workdir}')
 
+        run_opts += self.options
         if self.command:
             return (f'docker run --rm {" ".join(run_opts)} '
                     f'{self.image} {self.command}')
 
         if self.commands:
             return (f"docker run --rm {' '.join(run_opts)} {self.image} "
-                    f"bash -c 'cd {self.workdir}; {'; '.join(self.commands)}'")
+                    f"bash -c '{'; '.join(self.commands)}'")
 
         return f'docker run --rm {" ".join(run_opts)} {self.image}'
 
@@ -197,7 +219,8 @@ def emit_prepare_commands(self, stagedir):
         # The format that Sarus uses to call the images is
         # <reposerver>/<user>/<image>:<tag>. If an image was loaded
         # locally from a tar file, the <reposerver> is 'load'.
-        if not self.pull_image or self.image.startswith('load/'):
+        if (not self.pull_image or not self.image or
+            self.image.startswith('load/')):
             return []
         else:
             return [f'{self._command} pull {self.image}']
@@ -210,15 +233,17 @@ def launch_command(self, stagedir):
         if self.with_mpi:
             run_opts.append('--mpi')
 
-        run_opts += self.options
+        if self.workdir:
+            run_opts.append(f'-w {self.workdir}')
 
+        run_opts += self.options
         if self.command:
             return (f'{self._command} run {" ".join(run_opts)} {self.image} '
                     f'{self.command}')
 
         if self.commands:
             return (f"{self._command} run {' '.join(run_opts)} {self.image} "
-                    f"bash -c 'cd {self.workdir}; {'; '.join(self.commands)}'")
+                    f"bash -c '{'; '.join(self.commands)}'")
 
         return f'{self._command} run {" ".join(run_opts)} {self.image}'
 
@@ -232,6 +257,12 @@ def __init__(self):
         super().__init__()
         self._command = 'shifter'
 
+    def launch_command(self, stagedir):
+        # Temporarily change `workdir`, since Sarus and Shifter have otherwise
+        # the same interface
+        with util.temp_setattr(self, 'workdir', None):
+            return super().launch_command(stagedir)
+
 
 class Singularity(ContainerPlatform):
     '''Container platform backend for running containers with `Singularity
@@ -257,14 +288,17 @@ def launch_command(self, stagedir):
         if self.with_cuda:
             run_opts.append('--nv')
 
+        if self.workdir:
+            run_opts.append(f'-W {self.workdir}')
+
         run_opts += self.options
         if self.command:
             return (f'singularity exec {" ".join(run_opts)} '
                     f'{self.image} {self.command}')
 
         if self.commands:
             return (f"singularity exec {' '.join(run_opts)} {self.image} "
-                    f"bash -c 'cd {self.workdir}; {'; '.join(self.commands)}'")
+                    f"bash -c '{'; '.join(self.commands)}'")
 
         return f'singularity run {" ".join(run_opts)} {self.image}'
 
@@ -275,10 +309,6 @@ def __init__(self, *other_types):
 
     def __set__(self, obj, value):
         if isinstance(value, str):
-            try:
-                value = globals()[value]()
-            except KeyError:
-                raise ValueError(
-                    f'unknown container platform: {value}') from None
+            value = ContainerPlatform.create(value)
 
         super().__set__(obj, value)