Agreeing on terminology and naming conventions

[savente93]

There are a few names and conventions that have been part of hydromt for a while, and which it would be good to agree on whether we want to keep them, and if not, what they should be. Below I will detail the ones that I am aware of, but please add other topics if you think they are relevant here.

Small disclaimer: with this I don't mean to imply that the current names are mistakes, should not have been made, or were not thought out well. I think these were useful at the times they emerged, but as hydromt keeps growing I think it is a good opportunity to reevaluate. I am not dead set on the names, this is just a suggestion but as an initial proposal.

Terminology

Use of the term "Workflow"

Currently we have a submodule called workflows. However, in common parlance a workflow would be closer to what we now call the "The yaml file" containing hydromt steps, which is actually rather vague.

• Workflow (file): what we now informally refer to as "the yaml interface" or "hydromt config". I think this is the closest thing to what is commonly called a "workflow" so it makes sense to adopt that language here.
• A step: an entry of a workflow that corresponds to some function within hydromt. This is also how it's called in the code now. Sometimes we also call this a "hydromt method" which is sort of acceptable but it makes it harder to distinguish between something that you can call from the yaml and the programming definition of a method (a function associated with a particular instance of an object). I think agreeing to call it a "step" solves both at the same time.
• Routine/transformation/helper: @DirkEilander remarked that what is now called the workflows submodule actually contains rather low level things, so I would suggest we rename that to something like "transformations", "routines", "helpers" or something similar. Here I'm happy to defer to others if they have a preference.

Use of the term "config"

As far as I can tell there are two types of configuration that hydromt has to deal with: the config in the "workflow file" as proposed in the previous item and the configuration that hydromt has to produce for the kernel that is going to consume the model we are building (e.g. SFINCS or Wflow), both of which are refereed to as "configs", which I personally find rather confusing (please correct me if I am mistaken here). The workflow terminology above should already help, but to make it also very clear that we are producing configs that we do not have to consume, I would suggest that we also adopt the following terminology:

• Kernel/simulation config: I'm not really sure whether those terms are appropriate, so I'll defer there to the hydrologists in our team, but I would like to urge that we call it something besides just "config". While the configuration of hydromt itself mostly happens in the plugins which doesn't have a file, I still think it is a helpful distinction. So this would be any kind of config that hydromt should produce but does not change the behaviour of hydromt.
• Model/hydromt config While this doesn't really correspond to a file as such since most of it comes from either a plugin or the CLI, there is still a lot that can influence the behaviour of hydromt itself, so I would propose to refer to this as just a config, or call it a model config.

Naming conventions

While there is also a separate issue to harmonise conventions w.r.t. things like units and hydrological concepts, I also think there are two things that could be improved in hydromt.

Use of the abbreviation fn to mean "filename"

• This one is used a lot, but it has two problems.
  a) fn is a very common abbreviation for "function" not "filename" in almost all programming contexts, making this confusing to developers coming from other places.
  b) very often a parameter called fn (or has that as an affix) is not actually a filename. Usually it is a path relative to the model root, which is close enough, but it can also be an absolute path, a data source name that the data should be read from, an actual data object itself or a dictionary containing some collection of the aforementioned.

I appreciate that it is something cultural but I still find it difficult at times. I would like to use something that doesn't have another commonly understood meaning and is a bit more suggestive of all the things it could me like data_like or data_specifier. Honestly what the actual name becomes is less important to me than us agreeing on something more descriptive.

Use of the name time_tuple

This is not a big one, but in a lot of places a time range is refereed to as a time_tuple which is not bad, but refers more to an implementation detail than is necessary in my opinion. I'd suggest renaming this to time_range

Honourable mentions

There are also a few name conventions that we've already started using which I thought it was good to still document and agree on for clarities sake:

• Data source: Anything that can provide a data set, includes
• Source spec/specification: all the information that is necessary to uniquely identify a source, currently this consists of a source name, a provider and a version. (does not mean that all have to be specified, but it is enough to uniquely determine one)
• data variable: a variable in the hydrological/mathematical sense like precipitation, location, temperature etc.
• Program/digital variable: a variable in the programming sense like i, total or path
• Region: anything either specifies a geo-spatial area or can be turned into one.
• parsing: interpreting raw data and putting it into some known sensible format. (ensuring syntax is correct)
• validating: making sure the semantics of data in some format are as expected/allowed. Usually happens either after or at the same time as parsing. (i.e. ensuring semantics are correct)

[savente93]

@Jaapel

[Jaapel]

Nice overview.

I think HydroMT itself should also be configurable, if not now, then in the future. So I would indeed make config more spefic like you listed: KernelConfig sounds great at least for Deltares terminology, as many people call the simulation software containing different hydrological/hydrodynamic/etc. logic a kernel. If you need a yaml file detailing steps HydroMT has to make, I would call this a WorkflowDefinition or something similar. I would keep HydroMTConfig or simply Config for configuration of HydroMT itself. Let's say in the future you can configure some defauls (like a default data catalog), or the number of processors HydroMT uses, or the maximum memory available to HydroMT, that is where those would go.

[Tjalling-dejong]

Good points on workflow and config. I can imagine that this can be confusing. I am in favor of referring to the "hydromt config" as workflow.
And I agree on changing time_tuple to time_range

[savente93]

@hboisgon @DirkEilander what do you think?

[DirkEilander]

Good discussion @savente93

• Workflow: I agree on the workflow file and steps. I suggest renaming the workflows submodule to routines, _processes or data_processing
• config: With the workflow file (instead of hydromt config) I think most confusion is solved. The model/kernel/simulation config I would call model config to stay consistent with our definition of model. Simulation or kernel would be yet another term.
• filename: data_like sounds good for general cases. In most ModelComponent.add_* methods it would be best to refer with what data is expected, e.g.: precip_data_like, but as this will be quite lengthy I'm also fine with just precip in this case.
• time_range: agreed

[hboisgon]

Good points indeed. I agree with @DirkEilander. A few suggestions:

• Workflow: yes to rename "hydromt config" to hydromt "workflow file". Especially since we added "steps" in it, it makes sense. For the workflows module maybe indeed processing or data_processing would be better names.
• config: if we rename the hydromt config to workflow file, then the model config component can be just config and ConfigComponent (eg ForcingComponent is not called KernelForcingComponent so it is more consistent).
• filename: data_like is a good idea but maybe we could use data_source? In most cases like Dirk mentions it was something_fn to help the user know which type of data to provide. So then it would be precip_source, rasterdataset_source, geodataframe_source etc. Does that also maybe make sense with our definition of "data source" for the data catalog? After all the first thing we do with the data is call data_catalog.get_data on it even if python objects are directly passed.
• time_range: also agree

[savente93]

I like data_source though, I think that also has become a class in the implementations of @Jaapel, so we should make sure those concepts don't clash. What do you think Jaap?

[Jaapel]

The DataSource is the python representation of a parsed entry in the DataCatalog. The DataSource is responsable for validating the DataCatalog entry. It also carries the DataAdapter and DataDriver (more info below) and serves as an entrypoint to the data. Per HydroMT data type (e.g. RasterDataset, GeoDataFrame), HydroMT has one DataSource, e.g. RasterDatasetSource, GeoDataFrameSource.

So I would not suggest data source. data_like sounds fine to me if the possible types are: Path | str | xr.DataSet | xr.DataArray. If you just mean a path then just call it pathlike?

[Tjalling-dejong]

Going through the code for the GeoDataFrameSource I spotted the use of the word mask for specifying an area of interest. I think the geom or geometry would be more appropriate. I associate mask with raster data and to be used to exclude data from your raster for a certain area. What do you think @Jaapel?

[Jaapel]

I think I started with the GeoDataFrame data_type when refactoring the code and saw mask mentioned in their code vs. geom. I also know mask from the EO world, but indeed that is often raster data. Our geom argument is used to filter (or mask) the data. Therefore I like using mask instead of geom. However, I think that our users should also agree that this is more clear and I am open for whatever is most clear.