Optimizing History with the PFIO server

Using the PFIO server

Applications using the MAPL_Cap such as the GEOSgcm, GEOSctm, GEOSldas, GCHP, etc... use the MAPL_History for diagnostic output. All output is routed through the PFIO server. Extra resources beyond the needs of the model in an MPI sense can be allotted for the PFIO Server to exploit asynchronous output and decrease your applications wall time. By default the user does not have to do this and History will work just fine if you don't, although maybe not optimally. This page will explain how and when to configure the PFIO server to run on separate resources. While we can give general recommendations for how to configure the PFIO server we cannot emphasize enough that you should run your application multiple times and tune it to find the optimal configuration for you use case as what may be appropriate in one use case is not in another.

Simple Server?

As was stated in the introduction, the MAPL_History component is always writing the output via the PFIO server. However, in the default case (i.e. the user does nothing other than start the application on the number of MPI tasks needed for the model) the PFIO server is running on the same MPI resources as the application. Each time the run method of History is executed it will not return until all the files that were to be written in that step have completed. All the data aggregation and writing is done on the same MPI tasks as the rest of the application so the it cannot proceed until all output for that step is completed. There is no asynchronicity or overlap between compute and write in this case.

At low resolutions of your model run or cases with little history output this is sufficient. For example if you are running the GESOgcm.x at c24/c48/c90 for development purposes with a modest History output on 2 or 3 nodes, there's no sense in dedicating any extra resources for the PFIO server. As a concrete example, by default the GEOSgcm uses an NX = 4 and NY = 24 at c24 and c48, so you launch your application with 96 MPI tasks.

Multigroup Server

For exploiting asynchronous output when using History we recommend using the multgroup server option for the PFIO server.