Add regridding from low resolution ML (.npy) prediction to high resolution restart

[ma595]

Initial implementation of upscaling. Pipeline consists of

• loading predictions (and selecting some time index - doesn't have to be the last one)
• creating a coarse restart file
• regridding to fine resolution using xESMF.

It has been kept largely separate from the restart.py work even thought there is potential redundancy (for instance computation of density is done here with a separate EOS).

This is a functional re-implementation of David Kamm's Dinostics OOP repo and bears little resemblance to his work. I decided to do this as the Dinostics package is overkill for upscaling / regridding.

TODOs before merge:

• xESMF regridding is outputting nans.
• Check correspondence between the C2 file generated and the file created by Etienne and co. Compute metrics.
• Simple set of integration tests.

Set of instructions to reproduce entire pipeline.

[ma595]

File 1: generated_mine/generated_restart_C2_fine.nc
/Users/matt/work/nemo/nemo-spinup-restart/analysis/compare_fields.py:51: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  print(f"  Dimensions: {dict(ds1.dims)}")
  Dimensions: {'time_counter': 1, 'y': 797, 'x': 242, 'nav_lev': 36}
  Shape: (1, 36, 797, 242)

File 2: reference_data/diffusion_states/generated_restart_chamon_C2_clean_1_4deg.nc
/Users/matt/work/nemo/nemo-spinup-restart/analysis/compare_fields.py:55: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  print(f"  Dimensions: {dict(ds2.dims)}")
  Dimensions: {'time_counter': 1, 'y': 797, 'x': 242, 'nav_lev': 36}
  Shape: (1, 36, 797, 242)

============================================================
TEMPERATURE (tn) STATISTICS
============================================================
File 1 - Min: -3.0802°C, Max: 27.8069°C, Mean: 8.8126°C
File 2 - Min: -3.0802°C, Max: 27.8069°C, Mean: 8.8126°C

Difference (File2 - File1):
  Min: -0.000052°C
  Max: 0.000046°C
  Mean: 0.000000°C
  Std: 0.000002°C
  RMS: 0.000002°C

============================================================
SALINITY (sn) STATISTICS
============================================================
File 1 - Min: 0.0000 PSU, Max: 37.1324 PSU, Mean: 34.0153 PSU
File 2 - Min: 0.0000 PSU, Max: 37.1324 PSU, Mean: 34.0153 PSU

Difference (File2 - File1):
  Min: -0.000011 PSU
  Max: 0.000011 PSU
  Mean: -0.000000 PSU
  Std: 0.000001 PSU
  RMS: 0.000001 PSU

============================================================
CREATING COMPARISON PLOTS
============================================================
Saved comparison plot: field_comparison.png
Saved vertical profile plot: vertical_profiles.png

generated using compare_fields.py

[ma595]

Code runs as follows:

nemo-upscale upscale \
  --predictions-dir ./reference_data/diffusion_states/chamon_C2_clean/ \
  --coarse-template ./reference_data/1deg_restart/DINO_00000002_restart.nc \
  --coarse-mask ./reference_data/1deg_restart/mesh_mask.nc \
  --coarse-namelist ./reference_data/1deg_restart/namelist_cfg \
  --fine-template ./reference_data/025deg_restart_new/restart25_arch/DINO_10800000_restart.nc \  --fine-mask ./reference_data/025deg_restart_new/restart25_arch/mesh_mask.nc \
  --name C2 \
  --time-index 4 \
  --output-dir ./generated_mine