Add mpas_analysis_subsection parameter to global_time_series (#788)

* Add mpas_analysis_subsection parameter to global_time_series

* Allow multiple mpas_analysis subsections

* Fix mpas_analysis status file names.

We need to use the climatology and time series bounds from that
analysis, rather than from global time series, to construct their
filenames.

To make this easier and better share code, the bulk of this work
happens in the mpas_analysis module, not in global_time_series.

* Add unit tests to ensure correct status filenames

* Add comparison_type subdir

* Put mpas_analysis runs in mvo or mvm subdirs

This requires having a way for a user to specify whether an
external reference run (not defined in a given config file as
a subsection) was `mvo` or `mvm` via a new `reference_comparison_type`
config option.

* Update the docs with reference_comparison_type, etc.

* Update auto-generated files

---------

Co-authored-by: Xylar Asay-Davis <xylarstorm@gmail.com>

Author: forsyth2

docs/source/parameters.rst

@@ -76,6 +76,11 @@ For the ``mpas_analysis`` task:
 * ``reference_data_path`` and ``test_data_path`` are optional and are only used for model-vs-model comparisons.
   If provided, ``zppy`` uses them to locate the MPAS-Analysis config files from a *previous* MPAS-Analysis run and passes those through to MPAS-Analysis as ``controlRunConfigFile`` (reference) and ``mainRunConfigFile`` (test).
 
+  The comparison type of the *current* MPAS-Analysis run is inferred implicitly:
+  if ``reference_data_path`` is set, the run is treated as model-vs-model (``mvm``);
+  otherwise it is treated as model-vs-observations (``mvo``). Users normally do
+  not need to set a comparison type for the current run.
+
   .. note::
      These parameter names are intentionally consistent with the terminology used by ``e3sm_diags`` for model-vs-model runs: in both cases, ``reference_data_path`` identifies the *reference simulation's zppy-generated outputs*.
 
@@ -84,7 +89,16 @@ For the ``mpas_analysis`` task:
      For MPAS-Analysis, ``zppy`` resolves the config file when ``reference_data_path`` points to the prior run's zppy output directory (the one containing ``post/``).
 
    ``reference_data_path`` is intended to point to the prior run's zppy output directory (the one containing ``post/``). ``zppy`` will then use:
-   ``<reference_data_path>/post/analysis/mpas_analysis/cfg/mpas_analysis_<identifier>.cfg`` (or ``mpas_analysis_mvm`` if the referenced run was MVM).
+   ``<reference_data_path>/post/analysis/mpas_analysis/<comparison_type>/cfg/mpas_analysis_<identifier>.cfg``
+   where ``<comparison_type>`` is ``mvo`` or ``mvm``.
+
+   For referenced prior runs, ``reference_comparison_type`` and
+   ``test_comparison_type`` can be set to ``"auto"``, ``"mvo"``, or ``"mvm"``.
+   The default is ``"auto"``. In auto mode:
+
+   * if the path points to ``[[subsection]]``, zppy uses the referenced subsection's actual comparison type
+   * if the path points to an external zppy output directory, zppy looks for the matching cfg under ``mvo`` and ``mvm``
+   * if both exist for the same identifier, zppy raises an error and the user should set ``reference_comparison_type`` or ``test_comparison_type`` explicitly
 
    When ``reference_data_path`` is set to a non-subsection path, ``reference_case`` is required so the MVM output directory can include the reference case name. If ``reference_data_path`` is set to ``[[subsection]]``, ``reference_case`` is inferred to be the same as the current ``case``.
 
@@ -132,4 +146,4 @@ In ``e3sm_diags.py``:
 * ``check_mvm_only_parameters_for_bash``: similar, but these are specifically parameters used for model-vs-model runs. Uses ``check_parameter_defined`` in addition to ``check_set_specific_parameter``.
 * ``check_and_define_parameters``: make sure all parameters are defined, using ``utils.py get_value_from_parameter``, ``utils.py set_value_of_parameter_if_undefined``,  and ``check_mvm_only_parameters_for_bash``.
 
-``check_parameters_for_bash`` can be run immediately for each subtask because it has very few conditions. Other checks are included in ``check_and_define_parameters`` later on in the code.
+``check_parameters_for_bash`` can be run immediately for each subtask because it has very few conditions. Other checks are included in ``check_and_define_parameters`` later on in the code.

docs/source/post.mpas_analysis_model_vs_model.cfg

@@ -22,9 +22,14 @@ enso_years = "1850-2014",
 
 # Point at a *previous zppy run output directory* for the reference simulation.
 # zppy will locate the matching MPAS-Analysis cfg file under:
-#   <reference_data_path>/post/analysis/mpas_analysis/cfg/
+#   <reference_data_path>/post/analysis/mpas_analysis/<comparison_type>/cfg/
 reference_data_path = <reference zppy output directory>
 
+# Optional: override how zppy resolves the referenced prior run.
+# Usually "auto" is sufficient. Set this only if both mvo and mvm outputs exist
+# for the same year range under the referenced run.
+# reference_comparison_type = "auto"
+
 # Required when reference_data_path is not a [[subsection]].
 # Used to build the MVM output directory name.
 reference_case = <reference case name>
@@ -34,6 +39,7 @@ reference_case = <reference case name>
 # If set to [[subsection]], zppy will use that subsection's year ranges
 # when ts_years/climo_years/enso_years are not provided.
 # test_data_path = <test zppy output directory>
+# test_comparison_type = "auto"
 
 # Optional: override the reference year ranges (defaults to the test ranges).
 # If reference_data_path points to a prior [mpas_analysis] subsection using

docs/source/tutorial.rst

@@ -74,6 +74,10 @@ Unlike ``e3sm_diags`` (where ``run_type = "model_vs_model"`` and ``reference_dat
 points directly at reference climatology output), MPAS-Analysis comparisons are driven
 by MPAS-Analysis config files. For model-vs-model mode, ``zppy`` locates the matching
 config file(s) from prior MPAS-Analysis output and passes them to MPAS-Analysis.
+The current run's type is inferred automatically: setting ``reference_data_path``
+makes it an ``mvm`` run, otherwise it is an ``mvo`` run. If a referenced prior
+run could resolve to either ``mvo`` or ``mvm``, use ``reference_comparison_type``
+or ``test_comparison_type`` to disambiguate.
 
 .. literalinclude:: post.mpas_analysis_model_vs_model.cfg
    :language: cfg

tests/integration/generated/test_weekly_comprehensive_v3_chrysalis.cfg

@@ -284,24 +284,36 @@ tc_obs = "/lcrc/group/e3sm/diagnostics/observations/Atm/tc-analysis/"
 [mpas_analysis]
 active = True
 anomalyRefYear = 1985
-climo_years = "1985-1989", "1990-1995",
-enso_years = "1985-1989", "1990-1995",
 environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
 mesh = "IcoswISC30E3r5"
 parallelTaskCount = 6
 partition = "compute"
 qos = "regular"
 shortTermArchive = True
-ts_years = "1985-1989", "1985-1995",
 walltime = "00:30:00"
 
+  [[ reference ]]
+  ts_years = "1985-1989",
+  climo_years = "1985-1989",
+  enso_years = "1985-1989",
+
+  [[ test ]]
+  ts_years = "1985-1995",
+  climo_years = "1990-1995",
+  enso_years = "1990-1995",
+
+  [[ mvm ]]
+  reference_data_path = [[ reference ]]
+  test_data_path = [[ test ]]
+
 [global_time_series]
 active = True
 climo_years = "1985-1989", "1990-1995",
 environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
 experiment_name = "v3.LR.historical_0051"
 figstr = "v3.LR.historical_0051"
 #moc_file=mocTimeSeries_1985-1995.nc
+mpas_analysis_subsections = "reference", "test",
 # plots_lnd = "FSH,RH2M,LAISHA,LAISUN,QINTR,QOVER,QRUNOFF,QSOIL,QVEGE,QVEGT,SOILWATER_10CM,TSA,H2OSNO,TOTLITC,CWDC,SOIL1C,SOIL2C,SOIL3C,SOIL4C,WOOD_HARVESTC,TOTVEGC,NBP,GPP,AR,HR"
 ts_num_years = 5
 ts_years = "1985-1989", "1985-1995",

tests/integration/image_checker.py

@@ -185,10 +185,7 @@ def _check_mismatched_images(
         print(f"Reading expected images file {parameters.expected_images_list}")
         for line in f:
             image_name = line.strip("./").strip("\n")
-            proceed = False
-            if image_name.startswith(prefix):
-                proceed = True
-            if proceed:
+            if image_name.startswith(f"{prefix}/"):
                 counter += 1
                 if counter % 250 == 0:
                     print("On line #", counter)

tests/integration/template_weekly_comprehensive_v3.cfg

@@ -313,6 +313,7 @@ environment_commands = "#expand global_time_series_environment_commands#"
 experiment_name = "#expand case_name#"
 figstr = "#expand case_name#"
 #moc_file=mocTimeSeries_1985-1995.nc
+mpas_analysis_subsections = "reference", "test",
 # plots_lnd = "FSH,RH2M,LAISHA,LAISUN,QINTR,QOVER,QRUNOFF,QSOIL,QVEGE,QVEGT,SOILWATER_10CM,TSA,H2OSNO,TOTLITC,CWDC,SOIL1C,SOIL2C,SOIL3C,SOIL4C,WOOD_HARVESTC,TOTVEGC,NBP,GPP,AR,HR"
 ts_num_years = 5
 ts_years = "1985-1989", "1985-1995",

tests/test_zppy_global_time_series.py

@@ -1,8 +1,16 @@
+import os
 from typing import Any, Dict, List
 
 import pytest
+from configobj import ConfigObj
+from validate import Validator
 
 from zppy.global_time_series import determine_and_add_dependencies, determine_components
+from zppy.mpas_analysis import (
+    _get_referenced_comparison_type,
+    _resolve_mpas_analysis_config_file,
+    get_mpas_analysis_prefixes,
+)
 
 
 def test_determine_components():
@@ -123,6 +131,28 @@ def test_determine_and_add_dependencies():
     expected = ["script_dir/mpas_analysis_ts_1980-1989_climo_1980-1989.status"]
     assert dependencies == expected
 
+    c = {
+        "use_atm": False,
+        "use_lnd": False,
+        "use_ocn": True,
+        "mpas_analysis_subsections": ["reference", "test"],
+    }
+    dependencies = []
+    determine_and_add_dependencies(
+        c,
+        dependencies,
+        "script_dir",
+        mpas_analysis_prefixes={
+            "reference": ["mpas_analysis_reference_ts_1980-1989_climo_1980-1989"],
+            "test": ["mpas_analysis_test_ts_1980-1999_climo_1990-1999"],
+        },
+    )
+    expected = [
+        "script_dir/mpas_analysis_reference_ts_1980-1989_climo_1980-1989.status",
+        "script_dir/mpas_analysis_test_ts_1980-1999_climo_1990-1999.status",
+    ]
+    assert dependencies == expected
+
     c = {
         "use_atm": False,
         "use_lnd": False,
@@ -144,3 +174,157 @@ def test_determine_and_add_dependencies():
     dependencies = []
     with pytest.raises(Exception):
         determine_and_add_dependencies(c, dependencies, "script_dir")
+
+    c = {
+        "use_atm": False,
+        "use_lnd": False,
+        "use_ocn": True,
+        "mpas_analysis_subsections": ["missing"],
+    }
+    dependencies = []
+    with pytest.raises(ValueError):
+        determine_and_add_dependencies(
+            c,
+            dependencies,
+            "script_dir",
+            mpas_analysis_prefixes={
+                "reference": ["mpas_analysis_reference_ts_1980-1989_climo_1980-1989"]
+            },
+        )
+
+
+def test_get_mpas_analysis_prefixes(tmp_path):
+    config_path = tmp_path / "mpas_analysis.cfg"
+    config_path.write_text(
+        """
+[default]
+case = "case_name"
+input = "input_dir"
+output = "output_dir"
+www = "www_dir"
+
+[mpas_analysis]
+active = True
+mesh = "EC30to60E2r2"
+
+  [[ reference ]]
+  ts_years = "1985-1989",
+  climo_years = "1985-1989",
+  enso_years = "1985-1989",
+
+  [[ test ]]
+  ts_years = "1985-1995",
+  climo_years = "1990-1995",
+  enso_years = "1990-1995",
+
+  [[ mvm ]]
+  reference_data_path = [[ reference ]]
+  test_data_path = [[ test ]]
+""".strip()
+    )
+
+    config = ConfigObj(
+        str(config_path), configspec=os.path.join("zppy", "defaults", "default.ini")
+    )
+    validator = Validator()
+    assert config.validate(validator)
+
+    prefixes = get_mpas_analysis_prefixes(config)
+
+    assert prefixes == {
+        "reference": ["mpas_analysis_reference_ts_1985-1989_climo_1985-1989"],
+        "test": ["mpas_analysis_test_ts_1985-1995_climo_1990-1995"],
+        "mvm": [
+            "mpas_analysis_mvm_ts_1985-1995_climo_1990-1995_vs_ref_ts_1985-1989_climo_1985-1989"
+        ],
+    }
+
+
+def test_resolve_mpas_analysis_config_file_uses_comparison_type(tmp_path):
+    config_file = _resolve_mpas_analysis_config_file(
+        str(tmp_path), "ts_1985-1989_climo_1985-1989", comparison_type="mvm"
+    )
+
+    assert config_file == os.path.join(
+        str(tmp_path.resolve()),
+        "post",
+        "analysis",
+        "mpas_analysis",
+        "mvm",
+        "cfg",
+        "mpas_analysis_ts_1985-1989_climo_1985-1989.cfg",
+    )
+
+
+def test_get_referenced_comparison_type_infers_from_subsection():
+    comparison_type = _get_referenced_comparison_type(
+        "auto",
+        "/unused",
+        "reference",
+        "ts_1985-1989_climo_1985-1989",
+        {"reference": "mvm"},
+        "reference_data_path",
+    )
+
+    assert comparison_type == "mvm"
+
+
+def test_get_referenced_comparison_type_raises_on_subsection_mismatch():
+    with pytest.raises(ValueError, match="which is a mvo run, not mvm"):
+        _get_referenced_comparison_type(
+            "mvm",
+            "/unused",
+            "reference",
+            "ts_1985-1989_climo_1985-1989",
+            {"reference": "mvo"},
+            "reference_data_path",
+        )
+
+
+def test_get_referenced_comparison_type_auto_prefers_mvo_if_cfg_missing(tmp_path):
+    comparison_type = _get_referenced_comparison_type(
+        "auto",
+        str(tmp_path),
+        "",
+        "ts_1985-1989_climo_1985-1989",
+        {},
+        "reference_data_path",
+    )
+
+    assert comparison_type == "mvo"
+
+
+def test_get_referenced_comparison_type_auto_detects_mvm(tmp_path):
+    cfg_dir = tmp_path / "post" / "analysis" / "mpas_analysis" / "mvm" / "cfg"
+    cfg_dir.mkdir(parents=True)
+    (cfg_dir / "mpas_analysis_ts_1985-1989_climo_1985-1989.cfg").write_text("")
+
+    comparison_type = _get_referenced_comparison_type(
+        "auto",
+        str(tmp_path),
+        "",
+        "ts_1985-1989_climo_1985-1989",
+        {},
+        "reference_data_path",
+    )
+
+    assert comparison_type == "mvm"
+
+
+def test_get_referenced_comparison_type_auto_raises_if_ambiguous(tmp_path):
+    for comparison_type in ["mvo", "mvm"]:
+        cfg_dir = (
+            tmp_path / "post" / "analysis" / "mpas_analysis" / comparison_type / "cfg"
+        )
+        cfg_dir.mkdir(parents=True, exist_ok=True)
+        (cfg_dir / "mpas_analysis_ts_1985-1989_climo_1985-1989.cfg").write_text("")
+
+    with pytest.raises(ValueError, match="reference_comparison_type"):
+        _get_referenced_comparison_type(
+            "auto",
+            str(tmp_path),
+            "",
+            "ts_1985-1989_climo_1985-1989",
+            {},
+            "reference_data_path",
+        )

zppy/defaults/default.ini

@@ -327,11 +327,16 @@ enso_years = string_list(default=list(""))
 # Set these to the output directory of a prior zppy run (the one containing post/),
 # or to [[ subsection ]] to refer to a previous mpas_analysis subsection in this workflow.
 # zppy will use:
-#   <output>/post/analysis/mpas_analysis/cfg/mpas_analysis_<identifier>.cfg
-#   (or <output>/post/analysis/mpas_analysis_mvm/cfg/ if the referenced run was MVM)
+#   <output>/post/analysis/mpas_analysis/<comparison_type>/cfg/mpas_analysis_<identifier>.cfg
 # where <identifier> matches each MPAS-Analysis sub-run (e.g. ts_1850-2014_climo_1985-2014).
+# comparison_type for the current run is inferred implicitly:
+# if reference_data_path is set, it is an mvm run; otherwise it is an mvo run.
+# These options are only for resolving referenced prior runs. In auto mode, zppy
+# prefers mvo unless it finds only mvm; if both exist, it raises an error.
 reference_data_path = string(default="")
 test_data_path = string(default="")
+reference_comparison_type = option("auto", "mvo", "mvm", default="auto")
+test_comparison_type = option("auto", "mvo", "mvm", default="auto")
 # Required when reference_data_path is set to a non-subsection path.
 # If reference_data_path is set to [[subsection]], reference_case is inferred
 # to be the same as the current case.
@@ -370,6 +375,8 @@ walltime = string(default="06:00:00")
   enso_years = string_list(default=None)
   reference_data_path = string(default=None)
   test_data_path = string(default=None)
+  reference_comparison_type = option("auto", "mvo", "mvm", default=None)
+  test_comparison_type = option("auto", "mvo", "mvm", default=None)
   reference_case = string(default=None)
   ref_ts_years = string_list(default=None)
   ref_climo_years = string_list(default=None)
@@ -408,6 +415,9 @@ ts_years = string_list(default=list(""))
 # NOTE: always overrides value in [default]
 input_subdir = string(default="archive/ocn/hist")
 moc_file = string(default="")
+# Name of the `[mpas_analysis]` subtasks to depend on
+# Leave empty if no subtasks are defined for mpas_analysis
+mpas_analysis_subsections = string_list(default=list(""))
 # nrows, ncols are DEPRECATED. Specifying them will have no effect.
 # Number of columns per page
 ncols = integer(default=2)
@@ -441,6 +451,7 @@ plots_ocn = string(default="")
   input_subdir = string(default=None)
   make_viewer = boolean(default=None)
   moc_file = string(default=None)
+  mpas_analysis_subsections = string_list(default=None)
   ncols = integer(default=None)
   nrows = integer(default=None)
   plots_original = string(default=None)