The docker images with the compiled CROCO binaries should be configurable

[zachsa]

Algoa Bay Model
Here is the Dockerfile for compiling the Algoa Bay CROCO image: https://github.com/SAEON/somisana/blob/stable/models/algoa-bay-forecast/croco.Dockerfile

Some of the settings in param.h , cppdefs.h, and the launch script (run_croco.bash) should be configurable. To achieve this:

• The Dockerfile needs to be updated to accept build args
• Then after copying the param.h, cppdefs.h, and run_croco.bash files into the Docker build context (see these lines), those files need to be updated depending on what build args are passed (using string replacement).

What about if the defaults were suitable for local computer execution @GilesFearon? Then in production the docker build gets bespoke commands. I think these commands have to be passed at build time, as once the image is built it contains the compiled Fortran executable.

So far as I can remember from the CROCO workshop, there are two places to update when configuring how many cores to use (please check these @GilesFearon / @mattcarr03):

• https://github.com/SAEON/somisana/blob/stable/models/algoa-bay-forecast/croco/overwrites/param.h#L229
• https://github.com/SAEON/somisana/blob/stable/models/algoa-bay-forecast/croco/run-model/run_croco.bash#L15

Are there any other easily configurable params?

False Bay Model
Same as above, just in the False Bay folder

[GilesFearon]

I think it would be really cool to be able run the system on a regular 4 core laptop (or a 32 core desktop for that matter). It would make it a lot more generic.

Yup, as far as I can remember (it's been so long since I ran CROCO!), param.h and run_croco.bash would be the only places where you need to configure the number of cores. I guess we could just define $NP_X and $NP_ETA as environment variables (then you don't even need to do string replacement right?).

My understanding is that to run the model locally, we currently need to run the docker image with the CROCO binary already baked in from the latest iteration of the forecast system? So I'm still not sure how it becomes easily configurable from the local users point of view if we keep it like that? Unless the user has to re-build their own docker image, which is not ideal.

Would it be an idea to not compile the code when creating the docker image (all the other steps would remain), but we do that as a first step in run_croco.bash? We would need one line to run jobcomp and another to move the executable?

I can't think of other params which we would want to make easily configurable. Any changes to cppdefs.h would constitute a new configuration, which I think would mean you're changing the source code of the system

[zachsa]

Okay. Compiling at runtime makes sense.

So, two environment variables:

• NP_X
• NP_ETA

And then NUN_CORES = NP_X * NP_ETA

[zachsa]

Next steps

1. Move the param.h configuration code from the Dockerfile into the run_croco.bash script
2. Move the compilation code from the Dockerfile into the run_croco.bash script
3. Apply the same changes to False Bay model

[GilesFearon]

Documenting the progress like this is really cool! Would it be possible to do the compilation step as a separate "job" in the algoa and false bay workflows? Rather than all in the run execution step. This way the user could test the code compilation separately from running the model? Or is this creating unnecessary work?

[zachsa]

That's currently how it's done. The croco executable is baked into the croco image. So there are two jobs

1. Compile the croco exe. https://github.com/SAEON/somisana/blob/stable/.github/workflows/run_algoa-bay-forecast.yml#L63
2. Run the croco exe. https://github.com/SAEON/somisana/blob/stable/.github/workflows/run_algoa-bay-forecast.yml#L300

[zachsa]

If you build the same image twice it's usually quick the second time because all the image layers are cached locally

At the moment the compile croco step is run on a shared Microsoft runner so there's no caching. Still quick though.

There are also other options. Like an image with all the libs for running the fortran compiler, that you then use to compile various models. This might be a good idea for supporting lots and lots of models

[GilesFearon]

I guess what I mean is the Compile the croco exe step would be split into two steps - one would be to create a docker image with all the packages installed to do the compilation, and the other would be the compilation itself which just runs the docker image while providing the (configurable) param.h and cppdefs.h inputs. Does that make sense?

Hey you shouldn't be looking at this today!