Add "GDAL" to list of CF-aware software

[Armin-RS]

GDAL is a translator library for raster and vector geospatial data formats (https://gdal.org/en/latest/index.html) under an MIT style Open Source license (https://gdal.org/en/latest/license.html).

It supports a wide list of file formats (https://gdal.org/en/latest/drivers/raster/index.html), including netCDF (https://gdal.org/en/latest/drivers/raster/netcdf.html#raster-netcdf).

GDAL can be used with various programming languages like C/C++, Fortran (via the FortranGIS wrapper), Python and Java (https://gdal.org/en/latest/api/index.html) and in addition offers a number of useful command-line tools for working with geo-referenced data (https://gdal.org/en/latest/programs/index.html#programs).

Considering these points, I think it would be a useful addition to the list of CF-aware software on the CF-Website at https://cfconventions.org/software.html

[ChrisBarker-NOAA]

GDAL supports netcdf -- but I don't know that it has any support for CF itself?

As a rule, GDAL doesn't have any opinion about what the data mean.

Also, it attempts to interpret all files as a standard data model I.e GIS raster format. Many CF constructs don't fit into that.

Please correct me if GDAL has grown CF support since I last looked, which was quite some time ago :-).

[DocOtak]

@ChrisBarker-NOAA GDAL has support for Section 7.5 Geometries. I use GDAL and the features in section 7.5 via ogr2ogr to create geojson geometries of where my data are located for searching/visualization on in the browser.

I think GDAL is a worthy inclusion in the list of CF aware software.

[ChrisBarker-NOAA]

Sounds good then. Let's do it.

I'm a bit confused about GDAL vs OGR, but they are maintained together, so sure.

[DocOtak]

ogr2ogr is just one of the command line tools that comes with gdal: https://gdal.org/en/latest/programs/ogr2ogr.html sorry this confusion.

[ChrisBarker-NOAA]

Yes, it looks like it's all under the GDAL umbrella.

Now someone just needs to write a PR.

:-)

[Armin-RS]

I was also thinking about the "gdal_translate" utility which can copy only subwindows of the input file. So it must somehow understand the geo-referencing of the input file. However I don't know whether this is done via the single CF-attributes or the WKT string.

[Armin-RS]

Now someone just needs to write a PR.

I would be willing to create a draft for a PR. However, I've never done that.
Is there a guide somewhere how exactly this is done in the CF repo ?

Thanks!

[JonathanGregory]

Dear @Armin-RS

Thanks for your offer! A while ago I wrote a recipe for doing a PR in CF GitHub for someone who wasn't at all familiar with git or GitHub. That was my situation not long ago, but probably you're better-informed than I was. Anyway, in case it is useful, here it is. If that's not what you meant, please ask again.

If you or anyone else notice mistakes in the below, please say so.

Best wishes

Jonathan

---

Create a GitHub account for yourself on the web

You have to authenticate yourself from Linux to GitHub for all operations. This can be done with username and password and two-factor authentication, but I think it's easier to create a GitHub personal access token. The Linux command git config --global credential.helper store causes your username and token to be stored in ~/.git-credentials the next time you are asked for them. Thereafter you don't get asked again.

"Fork your own copy" of the GitHub repo. There's a button for this on the web page for each repo e.g. https://github.com/cf-convention/cf-convention.github.io for the website, https://github.com/cf-convention/cf-conventions for the conventions. This creates a copy in GitHub e.g. https://github.com/JonathanGregory/cf-conventions. You only have to do this once for each repo, since any repo contains all the branches. Your GitHub repo doesn't automatically stay up to date with the cf-convention's GitHub repo, but that doesn't matter.

Create a local clone on Linux of your GitHub repo, with the command e.g. git clone https://github.com/JonathanGregory/cf-convention.github.io which will be downloaded to a directory named cf-convention.github.io in your current working directory. Then git remote add upstream https://github.com/cf-convention/cf-convention.github.io to designate the CF GitHub repo as "upstream", meaning it's the repo you forked.

You don't have to clone your repo more than once, because you can reuse it next time. However, once you've committed your change to the GitHub repo you can rm -rf the local clone if you like and clone it again again next time.

When you make the clone, it will be up-to-date. However, on future occasions, you have to bring it up to date before starting to work on it. To do this, change directory to the local repo e.g. cd cf-convention. Then

git fetch origin # updates the local repo from your own GitHub repo 
git fetch upstream # fetches updates from the CF GitHub repo
git checkout main # change to main branch (in case it's not already)

You can ignore 'Your branch is ahead of origin/main'. This means that your main branch on GitHub is not up-to-date, but it doesn't matter because you aren't going to use it. We just need an up-to-date local version of main:

git merge upstream/main # fast-forward your main to agree with CF main

Now it's up-to-date and ready for you to prepare changes. The fetch and merge aren't needed if you've just forked your GitHub repo from CF (meaning your fork is up-to-date) and then made a local clone of it.

An important feature of git is that the local directory changes its contents when you change branches in the repo. The repo knows all the contents of all branches (stored in hidden files in the directory), but it only shows you one at a time. The branch which is eventually published on the web is the "main" branch. Modifications are prepared by copying the main branch within the repo to a new branch, and modifying the new branch. To make a new branch, first cd to the local directory of the repo, then git checkout -b NAME copies the main branch to a new branch, where NAME is some word that you choose to name the new branch. For instance, you could name it after the GitHub issue it refers to. The name is arbitrary.

Now you can modify the local files. You change and add files using any methods. In order to tell it which changes should be committed to the repo git add FILE [FILE ...] for any new or modified files. To remove files git rm FILE [FILE ...].

While you're working and when you've prepared what you want, you can do

git status # to see which files you've changed
git diff # to see the changes in files you haven't declared to git add
git diff --cached # to see changes in files declared to git add

Then git commit -a -m "COMMENT" where COMMENT is a text string to describe what you've done. This comment will appear in GitHub. The commit command modifies your local repo. After commit, the changes are no longer "cached". Then git push origin NAME to send the changes to your corresponding GitHub repo (i.e. the "origin", where you cloned it from). NAME is the name of the branch on the GitHub repo. I suppose it doesn't have to be the same as in the local repo, but I've always made it the same. The branch will be created on your GitHub repo as a copy of your local repo. That ends the business with your local repo.

Go back to your GitHub repo on the web. Click "Pull requests" at the top left of the window, then "New pull request". Set base repo on the left to the CF one e.g. cf-convention/cf-conventions, and the base branch to main. Set the head repo on the right to yours e.g. JonathanGregory/cf-conventions, and the branch to the NAME you've just created. It will compare the two of them and show you all the changes. For each file, you have a choice between the "source diff" <> line-by-line comparison, the "rich diff" with the page icon, which shows you text with insertions and deletions, and "view file" under the ... menu.

If it's all OK, click "Create pull request". The pull request is created in the CF repo, not your own copy. It will be automatically numbered and appear in the list of pull requests. The pull request is an instruction to import the modifications from your new branch of your repo into the main branch of the CF repo. It is not enacted until someone clicks "Merge" on its web page. You should keep the branch in your repo until the merge is done. If you need to make any amendments, you can do them locally as before and push them to GitHub, and they will get included in the pull request automatically. Once it's merged, you can delete the branch in your repo if you like. You can delete the whole repo if you want to, and fork again next time, but then you lose the history, which might perhaps be useful.

[DocOtak]

@JonathanGregory Should we get that added to the CONTRIBUTING.md file perhaps under at "git usage" type heading?

[Armin-RS]

Thanks @JonathanGregory ! That was exactly what I was looking for :-)

[JonathanGregory]

@Armin-RS. I am glad that my recipe looks useful. If you find mistakes or omissions, please say. In order to avoid confusing this issue about GDAL, please could you, or anyone else, make any comments about that in conventions issue 369.

@DocOtak. Yes, if the instructions are correct and likely to be useful to others, they should go somewhere for reference. Since the instructions are the same for the conventions and website repos (and any other, but those are the two most people would need to change), it might be better to put the instructions in a web page, and refer to it from CONTRIBUTING.md in both repos. The open conventions issue 369 is about the conventions CONTRIBUTING.md file. I'll make a note there about this point.