Skip to content

Update "benchmark_scrape_gchp_timers.py" to look for timing information in "allPEs.log" if not found in the GCHP log file#388

Merged
yantosca merged 1 commit intodevfrom
bugfix/read-gchp-timers-from-allPEs-log
Dec 19, 2025
Merged

Update "benchmark_scrape_gchp_timers.py" to look for timing information in "allPEs.log" if not found in the GCHP log file#388
yantosca merged 1 commit intodevfrom
bugfix/read-gchp-timers-from-allPEs-log

Conversation

@yantosca
Copy link
Contributor

Name and Institution (Required)

Name: Bob Yantosca
Institution: Harvard + GCST

Describe the update

This PR fixes an issue caused by the update to MAPL v2.59 in GCHP 14.7..0 and later. MAPL v2.59 now sends the GCHP timers output to the allPEs.log file instead of the regular GCHP log file. This had caused the GCHP benchmark timing table to show up as blank when reading logs from simulations using MAPL v2.59:

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%% GCHP Benchmark Timing Information
%%%
%%% Ref = gchp-c24-1Mon-14.7.0-alpha.15
%%% Dev = gchp-c24-1Mon-14.7.0-alpha.16
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


GCHPchem Timer                     Ref [s]             Dev [s]         % Diff
-------------------------------------------------------------------------------


Summary                            Ref [s]             Dev [s]         % Diff
-------------------------------------------------------------------------------

In this PR we have updated module gcpy/benchmark/modules/benchmark_scrape_gchp_timers.py:

  • Added a new function check_file_for_timing_info. This checks if the input file has GCHP timing info (e.g. has a string such as Times for component <GCHPchem>. If it finds timing info in the given input file, it will return the same file name it was given. If not, it will return the path to allPEs.log in the same folder.

  • Modified routine read_one_text_file to call check_file_for_timing_info so as to be sure we are reading the file with GCHP timers output. Also modified the parsing to strip out prefixes (such as 0000: MAPL.profiler: INFO: that are added by MAPL to allPEs.log) as each line is read in.

  • Imported the ENCODING constant from gcpy.constants. This is used in open commands instead of e.g. hardwired encoding="utf-8".

Expected changes

Module gcpy/benchmark/modules/benchmark_scrape_gchp_timers.py will now generate timing comparisons regardless of whether the GCHP timers output is in the log file or is in allPEs.log, such as:

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%% GCHP Benchmark Timing Information
%%%
%%% Ref = gchp-c24-1Mon-14.6.3                <=== GCHP timers in log file
%%% Dev = gchp-c24-1Mon-14.7.0-alpha.17       <=== GCHP timers in allPEs.log
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


GCHPchem Timer                     Ref [s]             Dev [s]         % Diff
-------------------------------------------------------------------------------
GCHPchem                         12024.660           11690.420         -2.780
--SetService                         0.430               0.230        -46.512 *
----generic                          0.000               0.000            nan
--Initialize                        55.680              23.520        -57.759 *
----INITIALIZE                      55.680              23.520        -57.759 *
------generic                       29.830              11.510        -61.415 *
--Record                             0.030               0.010        -66.667 *
----generic                          0.010               0.000       -100.000 *
--Run                            11897.830           11665.460         -1.953
----GenRunMine                   11897.790           11665.440         -1.953
------RUN                        11859.520           11633.760         -1.904
--------CP_BFRE                      3.720               2.160        -41.935 *
--------DO_CHEM                  11806.370           11604.540         -1.710
----------GC_CONV                 1241.590            1611.260         29.774 *
----------GC_DRYDEP                 11.040              11.330          2.627
----------GC_EMIS                 1047.920            1464.010         39.706 *
----------GC_TURB                  769.090             911.000         18.452 *
----------GC_CHEM                 5480.700            3098.400        -43.467 *
----------GC_WETDEP               2342.860            3628.010         54.854 *
----------GC_DIAGN                   4.970               1.370        -72.435 *
--------CP_AFTR                      0.010               0.000       -100.000 *
--Finalize                           3.670               1.170        -68.120 *
----generic                          3.670               1.140        -68.937 *


Summary                            Ref [s]             Dev [s]         % Diff
-------------------------------------------------------------------------------
All                              34461.342           21349.699        -38.047 *
--SetService                         3.234               0.846        -73.840 *
----GCHP                             2.005               0.276        -86.234 *
------GCHPctmEnv                     0.003               0.001        -66.667 *
------GCHPchem                       0.806               0.242        -69.975 *
------DYNAMICS                       1.131               0.012        -98.939 *
----HIST                             0.000               0.000            nan
----EXTDATA                          0.000               0.000            nan
--Initialize                       154.219              37.277        -75.829 *
----GCHP                           129.599              24.066        -81.430 *
------GCHPctmEnv                     0.012               0.013          8.333
------GCHPchem                     123.984              23.522        -81.028 *
------DYNAMICS                       4.008               0.015        -99.626 *
----HIST                             9.309               4.743        -49.049 *
----EXTDATA                         14.054               8.102        -42.351 *
--Run                            34297.774           21309.128        -37.870 *
----EXTDATA                       1077.890             534.291        -50.432 *
----GCHP                         24090.934           17104.509        -29.000 *
------GCHPctmEnv                    46.276              19.990        -56.803 *
------GCHPchem                   17444.564           15095.501        -13.466 *
------DYNAMICS                    6597.700            1987.885        -69.870 *
----HIST                           153.603              97.788        -36.337 *
--Finalize                           4.224               1.210        -71.354 *
----GCHP                             4.198               1.198        -71.463 *
------GCHPctmEnv                     0.011               0.003        -72.727 *
------GCHPchem                       4.170               1.191        -71.439 *
------DYNAMICS                       0.008               0.002        -75.000 *
----HIST                             0.011               0.006        -45.455 *
----EXTDATA                          0.011               0.005        -54.545 *

gcpy/benchmark/modules/benchmark_scrape_gchp_timers.py
- Now import ENCODING from gcpy.constants and use that instead of
  the hardwired "utf-8" in open statements
- Added routine "check_file_for_timing_info", which tests if a text
  file has GCHP timing info.  If not it returns the file path to
  "allPEs.log".  This is necessary due to an update in behavior
  in MAPL 2.59.
- In routine "read_one_text_file"
  - Now call "check_file_for_timing_info"before reading the file.
  - Strip out MAPL text that is usually placed into allPEs.log.
  - Skip all lines between after the GCHPchem section until
    the Summary section.
  - Change the marker for the Summary section to "Report on process:"
    (i.e. without a number).
  - Changed the break-out-of-loop command for cloud benchmark log files
    to "++", which occurs after the timers section.  This is only for
    MAPL versions prior to 2.59.
@yantosca yantosca added this to the 1.7.0 milestone Dec 18, 2025
@yantosca yantosca requested a review from msulprizio December 18, 2025 21:40
@yantosca yantosca self-assigned this Dec 18, 2025
@yantosca yantosca added topic: Benchmark Plots and Tables Issues pertaining to generating plots/tables from benchmark output category: Bug Fix Fixes a bug that was previously reported labels Dec 18, 2025
Copy link
Contributor

@msulprizio msulprizio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Thanks @yantosca!

@yantosca yantosca changed the title Update "benchmark_scrape_gcclassic_timers.py" to look for timing information in "allPEs.log" if not found in the GCHP log file Update "benchmark_scrape_gchp_timers.py" to look for timing information in "allPEs.log" if not found in the GCHP log file Dec 19, 2025
@yantosca yantosca merged commit 600ee3b into dev Dec 19, 2025
20 checks passed
@yantosca yantosca deleted the bugfix/read-gchp-timers-from-allPEs-log branch December 19, 2025 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: Bug Fix Fixes a bug that was previously reported topic: Benchmark Plots and Tables Issues pertaining to generating plots/tables from benchmark output

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants