Update compiler flags

[minghangli-uni]

Same flags as in #82 (comment)

---

🚀 The latest prerelease access-om3/pr148-21 at d61b197 is here: #148 (comment) 🚀

[github-actions]

🚀 Attempted to deploy access-om3 Prerelease pr148-1 with commit 241b5bd

🖥️ Gadi Deployment ✔️

Usage Instructions

access-om3, defined in ./spack.yaml, will be deployed to Gadi as:

• 2025.08.001 as a Release (when merged).
• pr148-1 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om3/pr148-1

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-om3-pr148-1 environment.
Due to inode-saving measures, one will have to manually untar the environment metadata before environment activation with tar -xf .spack-env .spack-env.tar. It will require one to have write privileges.

Configuration Information

This Prerelease is deployed using:

• access-nri/spack on branch 0.22
• access-nri/spack-packages version 2025.08.003
• access-nri/spack-config version 2025.08.000

If the above was not what was expected, commit changes to config/versions.json in this PR.

[minghangli-uni]

It takes ~2 hours to build the code... I don't think this is what we expect? Or there is something I'm missing?

[anton-seaice]

It’s possibly just a transient Gadi thing ? That is a bit long , I don’t think it’s going to be related to changing the compiler flags.

[github-actions]

🚀 Attempted to deploy access-om3 Prerelease pr148-2 with commit c8adb30

🖥️ Gadi Deployment ❌

[minghangli-uni]

!redeploy

[minghangli-uni]

I thought the long esmf build issue had already been resolved - is it still a problem, or is there a workaround to speed it up?

[github-actions]

🚀 Attempted to deploy access-om3 Prerelease pr148-3 with commit c8adb30

🖥️ Gadi Deployment ✔️

Usage Instructions

access-om3, defined in ./spack.yaml, will be deployed to Gadi as:

• 2025.08.001 as a Release (when merged).
• pr148-3 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om3/pr148-3

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-om3-pr148-3 environment.
Due to inode-saving measures, one will have to manually untar the environment metadata before environment activation with tar -xf .spack-env .spack-env.tar. It will require one to have write privileges.

Configuration Information

This Prerelease is deployed using:

• access-nri/spack on branch 0.22
• access-nri/spack-packages version 2025.08.003
• access-nri/spack-config version 2025.08.000

If the above was not what was expected, commit changes to config/versions.json in this PR.

[github-actions]

🚀 Attempted to deploy access-om3 Prerelease pr148-4 with commit 2c12228

🖥️ Gadi Deployment ❌

[minghangli-uni]

!redeploy

[github-actions]

🚀 Attempted to deploy access-om3 Prerelease pr148-5 with commit 2c12228

🖥️ Gadi Deployment ❌

[github-actions]

🚀 Attempted to deploy access-om3 Prerelease pr148-6 with commit 07df87a

🖥️ Gadi Deployment ❌

[minghangli-uni]

!redeploy

[github-actions]

🚀 Attempted to deploy access-om3 Prerelease pr148-7 with commit 07df87a

🖥️ Gadi Deployment ✔️

Usage Instructions

access-om3, defined in ./spack.yaml, will be deployed to Gadi as:

• 2025.08.001 as a Release (when merged).
• pr148-7 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om3/pr148-7

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-om3-pr148-7 environment.
Due to inode-saving measures, one will have to manually untar the environment metadata before environment activation with tar -xf .spack-env .spack-env.tar. It will require one to have write privileges.

Configuration Information

This Prerelease is deployed using:

• access-nri/spack on branch 0.22
• access-nri/spack-packages version 2025.08.003
• access-nri/spack-config version 2025.08.000

If the above was not what was expected, commit changes to config/versions.json in this PR.

[minghangli-uni]

With the help of https://github.com/ACCESS-NRI/esmf-trace, I was able to extract timestep-level performance data, rather than relying on coarse wall-clock runtime. This allows each test case to be much shorter (only 2 days of simulation for below tests), while still producing reliable performance statistics.

I tested three compiler configurations, each repeated five times (repeat in config.yaml).
Each 2-day run produces 192 timesteps -> 192 * 5 samples per setup.

1. compiler_flag_1 - existing
2. compiler_flag_2 - pr148-1
3. compiler_flag_3 - pr148-7

In each run, the first timestep appears very very fast (not shown in below plot). I’m not entirely certain of the cause, but likely reflects nuopc bookkeeping or something I am not sure (happy to take comments from others on this).

The second timestep is then the first timestep where the full dynamics is active and it is consistently the slowest “real” step. This is supported where fms timing in access-om3.out only reports 191 timesteps, so implying that the model does not treat the first timestep as part of the main timing window.

Similarly, the final timestep is noticeably slower due to I/O things.

So to avoid biasing the statistics with these startup/shutdown artifacts, I exclude steps 1–2 and the final timestep, leaving 189 consistent timesteps per run for the statistical comparison.

From the box plot, compiler_flag_2 consistently achieves the lowest mean and median timestep runtime while also showing a tighter variance and no strong outliers, in contrast to the other two compiler setups.

[minghangli-uni]

Below shows the setup for the three configurations using experiment-generator

model_type: access-om3
repository_url: git@github.com:ACCESS-NRI/access-om3-configs.git
start_point: "7851c1e" # Control commit hash for new branches
test_path: "." # All control and perturbation experiment repositories will be created here; can be relative, absolute or ~ (user-defined)
repository_directory: mom6-cice6-compiler_flags_25km_ryf # Local directory name for the central repository (user-defined)
control_branch_name: ctrl
Control_Experiment:

Perturbation_Experiment:
  Parameter_block_compiler_flags_2days_from_scratch:
    branches:
      - "compiler_flag_1" # default
      - "compiler_flag_2" # pr148-1
      - "compiler_flag_3" # pr148-7

    config.yaml:
      manifest:
        reproduce:
          exe: False

      modules:
        use:
          - - PRESERVE
          - - /g/data/vk83/prerelease/modules
            - /g/data/vk83/modules 
          - - /g/data/vk83/prerelease/modules
            - /g/data/vk83/modules
        load:
          - - PRESERVE
          - - access-om3/pr148-1 # not adding model-tools/mppnccombine-fast because the positional indexing
          - - access-om3/pr148-7 # not adding model-tools/mppnccombine-fast because the positional indexing

      env:
        ESMF_RUNTIME_PROFILE: "on"
        ESMF_RUNTIME_TRACE: "on"
        ESMF_RUNTIME_TRACE_PETLIST: "0 1 291-292 1975"
        ESMF_RUNTIME_PROFILE_OUTPUT: "SUMMARY"

      repeat: True

    nuopc.runconfig:
      CLOCK_attributes:
        restart_n: 2
        restart_option: ndays
        stop_n: 2
        stop_option: ndays

And this is the yaml plan for the experiment-runner

test_path: /g/data/tm70/ml0072/COMMON/git_repos/access-experiment-generator/performance_runnings_ncmas/om3 # All control and perturbation experiment repositories.
repository_directory: mom6-cice6-compiler_flags_25km_ryf # Local directory name for the central repository, where the running_branches are forked from.
keep_uuid: True

running_branches: # List of experiment branches to run.
  - compiler_flag_1 # default
  - compiler_flag_2 # pr148-1
  - compiler_flag_3 # pr148-7

nruns: # Number of runs for each branch; must match the order of running_branches.
  - 5
  - 5
  - 5

startfrom_restart:
  - cold
  - cold
  - cold

[github-actions]

🚀 Attempted to deploy access-om3 Prerelease pr148-8 with commit 97ae9f0

🖥️ Gadi Deployment ❌

[minghangli-uni]

!redeploy

[github-actions]

🚀 Attempted to deploy access-om3 Prerelease pr148-9 with commit 97ae9f0

🖥️ Gadi Deployment ❌

[github-actions]

🚀 Attempted to deploy access-om3 Prerelease pr148-17 with commit 2c87c1f

🖥️ Gadi Deployment ❌

[github-actions]

🚀 Attempted to deploy access-om3 Prerelease pr148-18 with commit 927e963

🖥️ Gadi Deployment ✔️

Usage Instructions

access-om3, defined in ./spack.yaml, will be deployed to Gadi as:

• 2025.08.002/x4lh6v3ng as a Release (when merged).
• pr148-18 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om3/pr148-18

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-om3-pr148-18 environment.
Due to inode-saving measures, one will have to manually untar the environment metadata before environment activation with tar -xf .spack-env .spack-env.tar. It will require one to have write privileges.

Configuration Information

This Prerelease is deployed using:

• access-nri/spack on branch 0.22
• access-nri/spack-packages version 2025.08.003
• access-nri/spack-config version 2025.08.000

If the above was not what was expected, commit changes to config/versions.json in this PR.

[minghangli-uni]

!redeploy

[github-actions]

The model version in the ./spack.yaml has not been updated.
You can update it manually.

[github-actions]

The model version in the ./spack.yaml has not been updated.
You can update it manually.

[github-actions]

🚀 Attempted to deploy access-om3 Prerelease pr148-19 with commit d61b197

🖥️ Gadi Deployment ❌

[github-actions]

🚀 Attempted to deploy access-om3 Prerelease pr148-19 with commit d61b197

🖥️ Gadi Deployment ❌

[minghangli-uni]

!redeploy

[github-actions]

🚀 Attempted to deploy access-om3 Prerelease pr148-20 with commit d61b197

🖥️ Gadi Deployment ❌

[github-actions]

🚀 Attempted to deploy access-om3 Prerelease pr148-21 with commit d61b197

🖥️ Gadi Deployment ❌

[access-hive-bot]

This pull request has been mentioned on ACCESS Hive Community Forum. There might be relevant details there:

https://forum.access-hive.org.au/t/cosima-twg-meeting-minutes-2026/5906/7