Allow for model-differentiated tasks

[mranst]

Description

Simple pr that would allow for tasks to be differentiated by model. Inspired by #625 and recent discussions about decoupling some of the structure that might be burdened by trying to support both models at the same time.

When runningswell task <task_name> -m <model_component> config, Swell will try to import <task_name> from src/swell/tasks/<model>. If it can't find it, it will just try to find one under src/swell/tasks/. This means that any tasks that would benefit from having a separate model-specific version can do so, and any tasks that don't can continue to use the general version.

I haven't added any of these tasks yet, but they just need to be put under src/swell/tasks/<model> (for example)

[Dooruk]

Hmm, I started writing my tasks with different names (e.g., RunCoupledGeosExecutable) but I'm not sure about how to group them yet, I'm open to discussion.

Ultimately, some tasks might be more application specific rather than model specific (GEOS-FP, MERRA3, or GEOS-IT would all use geos_atmosphere). I do like the concept of reverting to default if a task not available though.

[Dooruk]

We briefly touched on this today but this PR seems to be a good first step in the right direction. I began doing this in an inelegant way in my convoluted PR #643

I guess I see two potential issues here:

1. 3-4 tasks/scripts doing different things will have the same name with a generic task at the high /tasks level. This is not a problem for seasoned SWELL user but might be confusing for new users. This is already how it is handled in flow.cylc after the jinja template is filled with proper values (e.g., EvaJediLog-geos_marine or EvaJediLog-geos_atmosphere).

2. The tasks are not necessarily only model-specific but could be (or should be) application specific (coupled reanalysis, coupled seasonal forecast, OSSE etc. I'm just throwing some of them out there). This is where things get out of control but allowing certain tasks/applications to manipulate the gcm_run.j script (or it's next-gen equivalent) would allow people to transfer their workflows to ours easier.

I'm open to further discussion @rtodling or Skylab insight @jeromebarre @mer-a-o

[mranst]

Since this was brought up again in #677, I've updated this and added messages to the task factory method. Hopefully this will help make it clear which task module is being used, but as noted before this does have the potential to be confusing to newer users

[Dooruk]

Revisiting my comment above:

1. I'm wondering if we want task names in /model folders would this be a simple change via:

task_class = getattr(importlib.import_module(f'swell.tasks.{model}.{task_lower}-{model}'), task)
factory_logger.info(f'Using model-specific version of {task} in <swell.tasks.{model}.{task_lower}-{model}>')

So tasks can have different names with -model. Not sure if this would run into any Swell specific problems.

2. I thought about this further and I'm not sure if it is practical to have suite specific tasks (and folders) and we don't need to implement now. However GMAO tasks/scripts are more specific to the product type, rather than model, MERRA3, NWP, OSSE, S2S etc.

   However, to facilitate it we could create a hierarchy:

   • Check suite specific tasks
   • Check model specific tasks
   • Resort back to generic task

   but this is more code in terms of Swell wrapper code, task factory etc. So more confusion!

[mranst]

   However, to facilitate it we could create a hierarchy:
   
   * Check suite specific tasks
   * Check model specific tasks
   * Resort back to generic task
   
   but this is more code in terms of Swell wrapper code, task factory etc. So more confusion!

Yeah it's a lot to keep track of. If application-specific tasks would provide more utility, it may just be better to ditch the model-specific idea entirely and just have differently named tasks and bake more logic into flow.cylc that selects the appropriate task. If gcm_run.j needs different parameters depending on the suite/application etc, we can just add keys to experiment.yaml and do something like the following:

{% if application == 'example1' %}
Precursor => Example1
{% if application == 'example2' %}
Precursor => Example2
{% endif %}

[Dooruk]

Yeah it's a lot to keep track of. If application-specific tasks would provide more utility, it may just be better to ditch the model-specific idea entirely and just have differently named tasks and bake more logic into flow.cylc that selects the appropriate task. If gcm_run.j needs different parameters depending on the suite/application etc, we can just add keys to experiment.yaml and do something like the following:

{% if application == 'example1' %}
Precursor => Example1
{% if application == 'example2' %}
Precursor => Example2
{% endif %}

Let's stick with your changes for now and keep things simple. It is a good step towards model specific tasks and we can just expect different applications to name particular tasks under the model folder.

[mranst]

I changed the file naming as suggested above so they follow swell.tasks.{model}.{task_name}-{model}, otherwise this should be ready