Possibility to enter specific options for mpirun to run the ctests on particular machines

[ischeider]

Some machines, particularly HPC systems, often have a specific configuration, which makes it necessary that mpirun gets parameters different from the standard, which are as of now hardcoded in CMakeLists.txt:

set(MPIEXEC_EXTRA_OPTS_FOR_TESTING
    "--bind-to none --use-hwthread-cpus --mca orte_tmpdir_base ${PROJECT_BINARY_DIR}/tmp"
    )

At least @ppraegla and me need some other and/or added parameters. Anybody else, who also would like to have more flexibility in the generation of the CTestTestfile.cmake?

For example, if we introduce an environment variable, which can be set by users (or even system administrators) and maybe has some default, e.g., the variables defined above, we avoid breaking the compliance by hard-coding particular parameters in the repo.

[sebproell]

@ischeider I fully support this. I moved these parameters to their current place a few years ago since they were duplicated in all the user configurations but that is not how we should do it. Our CMake setup is a lot saner these days, so I am sure we can find a better way here.

[ischeider]

For me it would be sufficient to replace the set(MPIEXEC_EXTRA_OPTS_FOR_TESTING ... by a string(APPEND MPIEXEC_EXTRA_OPTS_FOR_TESTING..., which allows then to define the variable in the CMake presets. But I don't have a clue whether the three parameters given so far (--bind-to none --use-hwthread-cpus --mca orte_tmpdir_base ${PROJECT_BINARY_DIR}/tmp) are necessary (or at least valid) for the most common cases. Otherwise I would like to shift this base definition to the various CMakePresets.json files or skip them completely and give the users the opportunity to set them in the CMakeUserPresets.json.

The other option I see is to have an

if(NOT MPIEXEC_EXTRA_OPTS_FOR_TESTING)
  set(MPIEXEC_EXTRA_OPTS_FOR_TESTING "--bind-to none ...")

in the CMakeLists.txt.

What do you prefer?

[ischeider]

AFAICS, the PR #727 changes the behavior, so the discussion can be closed.