Input Data Handling

[maximilian-gelbrecht]

How do we want to handle input data?

At the moment, there's e.g. the data missing to reproduce the global soil script just from the repo. For the Pluto notebook in #77 we also need more input data.

Short term we could just put all in inputs folder. Do we want that for now?

Long term we could use the artefact system in combination with a separate repository just for the data, as done now for Speedy in SpeedyWeather/SpeedyWeather.jl#983
When we do a new repository for input data, we could/should probably also wait if we still move to NumericalEarth org.

[bgroenks96]

Short term, I would just put it in the inputs folder, but yes I would like to use the artifact system. ClimaLand also does this.

The bigger question is which data, in what format, at what resolution, and where to actually store it. You mentioned creating a GitHub repository for input data, but that's not going to work for larger files. Also, are we going to automatically interpolate the reoslution to the user's choice of grid?

[maximilian-gelbrecht]

I would definitely automatically interpolate. We can just use RingGrids for that, it's just a few lines of code. Because of that I'd save the data at the highest reasonable resolution (for now).

So far I was mostly thinking about climatology files, putting those in a repo would still fine imo. If we want more input data, yes, it won't work. I guess we could either ask our IT, if we can put them on a server where it's downloadable for externals as well, or just use Google cloud storage.

[maximilian-gelbrecht]

Ok, new RingGrids released. I'll apply that directly in #77

Then, we could also slightly revise the other scripts that use input data.