Reproducibility Checklist: Documentation

[LilacHo]

Extra resources:

• OHW23 Joseph Gum's presentation
• Software Best Practices, Valentina Staneva's presentation

• Do you provide instructions on how to run/install the software? Are the versions of the dependencies provided?

  
  - Export the existing environment and store it on the GitHub repo (conda export or pixi.toml)
  - Which version of Python are you running on?
  - Provide a minimal list of packages needed to start from scratch
  
• Do you provide examples of how to use the software?

  
  - Add instructions on the order in which to run the software pieces.
  - Provide an example dataset.
  - In function docstrings you can provide example calls.
  
• Can a user run the examples?

  
  - If not, can the user see the outputs (such as plots), so that they do not have to run the example to understand it. Execute and save your notebooks before committing to GitHub.
  
• Do you describe how the data was o?

  
  - If you have Python notebooks, you can weave them with markdown files with more instructions into a Jupyter Book and publish them on Github Pages. (OHW example)
  - If you use Rmd, you can weave them into Quarto (OHW example)
  
• If so, does that document have an associated copyright license?

  
  - Creative Commons Attribution License is a good choice for documentation.
  
• Is it going to be available in 1 year?

  
  - You can create a GitHub Release, and generate a DOI for it through Zenodo
  
• Can a user regenerate the results?

  
  - If not all of them, maybe a subset of them?
  - If you know there is a step that cannot be run by the user, indicate that so they do not try.
  
• Is the procedure for generating all of the results automated?

  
  - If you are using scripts, you can make one script to rerun all commands.
  - If you are using notebooks you can use nbconvert to execute them and you can do that automatically with Github Actions (Example).
  
• Are the results stochastic? Is it indicated somewhere?

  
  - If the algorithms use the random number generators, set he seed to be able to reproduce the result.
  
• Are some of the steps requiring manual input?

  
  - Provide a guide on how it was done.
  - Can some steps be automated? With GitHub Actions, Docker?