Criteria to evaluate reproducibility

[dleehr]

I suggest building a grading system to quantify how reproducible a project is, in terms of

• availability of dependencies (software and hardware)
• lifespan/maintenance/inclusion of dependencies
• is the code black box only or reusable/modifiable?
• are the data formats portable/accessible

[benmarwick]

In the UW Reproducible research group we've developed this form to learn more about what the members of our group do, and allow researchers all over campus to self-assess the reproducibility of a publication and obtain further information about key elements such as

• data (location, availability, licence)
• code and software (availability, licensing and location)
• version control and workflow management

We're thinking of a simple bronze-silver-gold ranking to recognize no-partial-full reproducibiltiy of research publications based on the information provided in the form. (p.s. I'm not attending the Dec 8 event in person, but it sounds really interesting, so I hope it's ok to lurk)