Provide project templates

[tristan0x]

Requirements

The following Basalt project should be completed first.

Deliverables

2 project templates: one for pure Python projects and the other for C++/CMake

Pure Python project

• Provide instructions to generate a Python project with pyscaffold.
• Provide guidelines to create cython functions if necessary.

Use case

The main target is to create Python packages (source distribution or wheels)

References

Interesting work made by NSE team to build Python wheels (still work-in-progress)

C++ project

Create a cookiecutter project inspired by basalt.

Input parameters

Parameter	Default Value
team name	HPC
team_email	bbp-ou-hpc@groupes.epfl.ch
project_slug	foo
python_bindings	y

Third parties

The 3rd parties libraries below should be ready to be used. single-header releases are bundled in the project, the other provided as git submodules:

• hpc-coding-guidelines
• From Tooling:
  • Catch2
  • CLI11
  • {fmt}
  • Google benchmark
  • spdlog
  • nlohmann/json
  • gsl-lite

Python bindings

If python_bindings is enabled, then the generated project would be enriched with pyscaffold goodness (tox.ini, setup.py, ...) and pybind11.

[pramodk]

👍

Some important missing items are related to parallel / distributed aspects. For example, for new software components (including coreneuron) what we will additionally need is:

• Logger support for parallel/mpi application (i.e. logging from multiple mpi ranks, old discussion)
• Error handling for parallel/mpi applications (same as above)
• Cmake supporting MPI enabled build (e.g. some applications directly set CC=mpicc and others use CC=gcc and use FindMPI.cmake module to get link libraries)
• GPU enabled build support :
  • Building C++ project together with CUDA code
  • Building C++ project with CUDA as well as OpenACC
• Parallel tests support
• Cross compile build support (not that important)
  • certain components like mod2c/nmodl are built for front-end node
  • rest of the code is built for compute node

Note : above items need considerable effort and could be moved to separate ticket.

[tristan0x]

Thank you for these additions Pramod. You are right, some of these items can be added to the project template afterward. But I think there are already some elements that can be part of the first version:

• Logger support for parallel/mpi application (i.e. logging from multiple mpi ranks, old discussion)

One should investigate whether spdlog is MPI ready, or if a layer of top of spdlog to ease the logging experience in MPI programs already exists or should be developed by us.

• Error handling for parallel/mpi applications (same as above)

IMO this belongs to a GoodPractices document in the C++ coding conventions. Don't you think @pramodk ?

• Cmake supporting MPI enabled build (e.g. some applications directly set CC=mpicc and others use CC=gcc and use FindMPI.cmake module to get link libraries)

I suggest to add an input parameter use_mpi in the project template in the initial version. I like the FindMPI approach as it provides more insights about the MPI setup (version, flags, ...) and runtime environment (like the path to mpiexec/mpirun) that are useful during the test phase.

• GPU enabled build support :

  • Building C++ project together with CUDA code
  • Building C++ project with CUDA as well as OpenACC

Maybe in a future version.

• Parallel tests support

The CMake project that comes with hpc-coding-conventions should provide a bunch of CMake helpers like the add_mpi_test function in zee:

Usage:

add_mpi_test(
    NAME LaplacianPETSc
    NUM_PROCS 16
    COMMAND $<TARGET_FILE:LaplacianPETSc>)

add_mpi_test(NUM_PROC 16 COMMAND

• Cross compile build support (not that important)

  • certain components like mod2c/nmodl are built for front-end node
  • rest of the code is built for compute node

This should be addressed later also.

[pramodk]

One should investigate whether spdlog is MPI ready, or if a layer of top of spdlog to ease the logging experience in MPI programs already exists or should be developed by us.

yes, small layer on spdlog should do the job. (I dont think spdlog is mpi ready).

IMO this belongs to a GoodPractices document in the C++ coding conventions. Don't you think @pramodk ?

What I meant us some kind of common utility classes for logging and error handling. For example, in MPI application, we typically do following throughout the application:

if (rank == 0) {
    logger->error(...);
    abort(); // MPI_Abort();
}

Some common way would be useful for multiple applications.