[WIP] Tiny Profiler: default report to diags/performance.txt

Docs/source/developers/how_to_profile.rst

@@ -15,7 +15,10 @@ AMReX's Tiny Profiler
 ---------------------
 
 By default, WarpX uses the AMReX baseline tool, the TINYPROFILER, to evaluate the time information for different parts of the code (functions) between the different MPI ranks.
-The results, timers, are stored into four tables in the standard output, stdout, that are located below the simulation steps information and above the warnings regarding unused input file parameters (if there were any).
+The results, timers, are stored into four tables.
+By default, WarpX writes them to ``diags/performance.txt`` (controlled by :pp:param:`tiny_profiler.output_file`).
+Set that parameter to ``stdout`` to print them to the standard output instead.
+When printed to stdout, the tables are located below the simulation steps information and above the warnings regarding unused input file parameters (if there were any).
 
 The timers are displayed in tables for which the columns correspond to:
 

Docs/source/usage/parameters.rst

@@ -5771,5 +5771,37 @@ When developing, testing and :ref:`debugging WarpX <debugging_warpx>`, the follo
     Run all ``FillBoundary`` operations on ``MultiFab`` to force-synchronize shared nodal points.
     This slightly increases communication cost and can help to spot missing ``nodal_sync`` flags in these operations.
 
+.. _running-cpp-parameters-tiny-profiler:
+
+Tiny Profiler
+^^^^^^^^^^^^^
+
+By default, WarpX is compiled with the :ref:`AMReX Tiny Profiler <developers-profiling-tiny-profiler>`, which collects timers for the main functions of the code as well as a device memory profile.
+The profiler is controlled with `AMReX runtime parameters <https://amrex-codes.github.io/amrex/docs_html/RuntimeParameters.html#tiny-profiler>`__, the most commonly used of which are documented below.
+
+.. pp:param:: tiny_profiler.enabled
+    :type: ``bool``
+    :default: ``1`` for true
+
+    Enable or disable tiny profiling (including memory profiling) at runtime.
+    If disabled, no report is written and WarpX does not create a diagnostics directory for it.
+
+.. pp:param:: tiny_profiler.output_file
+    :type: ``string``
+    :default: ``diags/performance.txt``
+
+    File name for the tiny profiler report.
+    By default, WarpX writes the report as ``performance.txt`` into the directory where the
+    diagnostics are written (``diags/performance.txt`` for the default file prefix), instead of
+    stdout.
+    The directory follows a user-set ``<diag_name>.file_prefix`` (full/back-transformed
+    diagnostics) or :pp:param:`reduced_diags.path` (reduced diagnostics).
+    If no diagnostics are configured (no :pp:param:`diagnostics.diags_names` and no
+    :pp:param:`warpx.reduced_diags_names`, or :pp:param:`diagnostics.enable` ``= 0``), the profiler
+    is turned off (``/dev/null``) so that no diagnostics folder is created.
+
+    Set this to a path to choose a specific file, to ``stdout`` or ``stderr`` to print the
+    report to the standard output or error stream, or to ``/dev/null`` to disable the output.
+
 .. bibliography::
     :keyprefix: param-

Source/Diagnostics/Diagnostics.H

@@ -47,6 +47,9 @@ public:
     Diagnostics(Diagnostics&& )                    = default;
     Diagnostics& operator=(Diagnostics&& )         = default;
 
+    /** Return the file prefix (output directory and base name) of this diagnostic */
+    [[nodiscard]] std::string GetFilePrefix () const { return m_file_prefix; }
+
     /** Stores the diag type */
     DiagTypes m_diag_type;
     /** Pack (stack) all fields in the cell-centered output MultiFab m_mf_output.

Source/Initialization/WarpXInitData.cpp

@@ -73,11 +73,13 @@
 #include <AMReX_REAL.H>
 #include <AMReX_RealBox.H>
 #include <AMReX_SPACE.H>
+#include <AMReX_Utility.H>
 #include <AMReX_Vector.H>
 
 #include <algorithm>
 #include <array>
 #include <cctype>
+#include <filesystem>
 #include <iostream>
 #include <memory>
 #include <string>
@@ -815,6 +817,59 @@ WarpX::InitData ()
     /** create object for reduced diagnostics */
     reduced_diags = std::make_unique<MultiReducedDiags>();
 
+    // By default, write the AMReX tiny profiler report into the diagnostics
+    // directory (e.g. diags/performance.txt) instead of stdout. The directory is
+    // derived from the configured diagnostics, so the report follows a user-set
+    // <diag_name>.file_prefix (full/back-transformed diagnostics) or
+    // reduced_diags.path (reduced diagnostics). If no diagnostics are configured,
+    // the profiler is turned off (/dev/null) so that no diagnostics folder is
+    // created. A user-set tiny_profiler.output_file always takes precedence, and
+    // nothing is done if the profiler is disabled via tiny_profiler.enabled.
+    // AMReX reads output_file lazily at Finalize, so setting it here is early enough.
+    {
+        ParmParse pp_tiny_profiler("tiny_profiler");
+        bool profiler_enabled = true;
+        pp_tiny_profiler.query("enabled", profiler_enabled);
+        std::string output_file;
+        if (profiler_enabled && !pp_tiny_profiler.query("output_file", output_file))
+        {
+            const bool any_diags =
+                (multi_diags->GetTotalDiags() > 0) || !reduced_diags->m_rd_names.empty();
+            if (!any_diags) {
+                output_file = "/dev/null";
+            } else {
+                // Determine the directory under which WarpX writes its diagnostics.
+                std::string diags_dir;
+                if (multi_diags->GetTotalDiags() > 0) {
+                    // Full/back-transformed diagnostics: <diag>.file_prefix, e.g. "diags/diag1".
+                    diags_dir = std::filesystem::path(multi_diags->GetDiag(0).GetFilePrefix())
+                        .parent_path().generic_string();
+                } else {
+                    // Reduced diagnostics: reduced_diags.path, e.g. "./diags/reducedfiles/".
+                    std::string reduced_path = "./diags/reducedfiles/";
+                    const ParmParse pp_reduced_diags("reduced_diags");
+                    pp_reduced_diags.query("path", reduced_path);
+                    diags_dir = std::filesystem::path(reduced_path).lexically_normal()
+                        .parent_path().generic_string();
+                }
+
+                if (diags_dir.empty()) {
+                    // The diagnostics resolve to the current working directory.
+                    output_file = "performance.txt";
+                } else {
+                    output_file = diags_dir + "/performance.txt";
+                    // Ensure the directory exists so the report can be written at
+                    // Finalize (full diagnostics create it lazily, only on output steps).
+                    if (ParallelDescriptor::IOProcessor()) {
+                        constexpr int permission_flag_rwxrxrx = 0755;
+                        UtilCreateDirectory(diags_dir, permission_flag_rwxrxrx);
+                    }
+                }
+            }
+            pp_tiny_profiler.add("output_file", output_file);
+        }
+    }
+
     // WarpX::computeMaxStepBoostAccelerator
     // needs to start from the initial zmin_domain_boost,
     // even if restarting from a checkpoint file