Best way to upload local version without losing custom changes

[Llannelongue]

Each installation requires custom values in cluster_info.yaml and GreenAlgorithms_global.py, but when pulling the latest release of the code, these get overwritten...

[TomSmithCGAT]

I'm not sure what custom values are required in GreenAlgorithms_global.py. From the first few lines, it sounds like it shouldn't be edited:

GreenAlgorithms4HPC/GreenAlgorithms_global.py

Lines 2 to 5 in 6d33be2

	## ~~~ TO NOT EDIT ~~~
	##
	## This script is common to all clusters.
	##

For cluster_info.yaml, I suggest making this line use a command-line input.

GreenAlgorithms4HPC/GreenAlgorithms_global.py

Line 367 in 6d33be2

with open("cluster_info.yaml", "r") as stream:

Then the user can keep their own .yaml file separate to this repository. You could also have a subdirectory in this repository to collect .yaml files for any HPC on which the script has been run, starting with the Cam CDS3.

[jfrost-mo]

Given you have an installer script, you could just have the file named something different (cluster_info.yaml.example?) and then copy it to the correct filename on first run if it doesn't already exist. Then don't track cluster_info.yaml in git, and it will all work.

[rubenhorn]

Given you have an installer script, you could just have the file named something different (cluster_info.yaml.example?) and then copy it to the correct filename on first run if it doesn't already exist. Then don't track cluster_info,yaml in git, and it will all work.

I was also going to propose this solution, but I do not think that cluster_info.yaml should be created automatically on first run, since the default values are not useable and this might cause confusion.
Simply displaying “Please first create data/cluster_info.yaml from data/cluster_info.yaml.example!” should be enough.

Perhaps it would be possible to create an automatic setup script which used scontrol show partition, lscpu, etc. to set some sensible defaults.
There are probably too many edge cases, so some manual editing will likely always be necessary.
However, it would be nice to at least set up all partitions available on the cluster.

As for setting up python in myCarbonFootprint.sh…
It would be best to use $PYTHON_BIN and $PIP_BIN and set them up by including the line source ./setup-python.sh.
This (untracked) file should not be tracked and can be used to load the required module and set the two variables.
If the file does not exist or these variables are not set, a fallback should be used and a warning displayed.

With these changes, one could just use git pull to update the code without loosing local configurations.