Skip to content

Commit 694519c

Browse files
committed
move funding to project description and add project links
1 parent a218cd3 commit 694519c

File tree

2 files changed

+30
-19
lines changed

2 files changed

+30
-19
lines changed

partners.md

Lines changed: 0 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -114,15 +114,3 @@ Take a look at the CODECHECK register for 🗂️ [all codechecks conducted as p
114114
CODECHECK is a member in the [German Reproducibility Network](https://reproducibilitynetwork.de/).
115115

116116
[![GRN logo]({{ 'img/RN_German_kurz.png' | relative_url }}){:width="200"}](https://reproducibilitynetwork.de/)
117-
118-
## Funding
119-
120-
[![Mozilla logo]({{ 'img/mozilla.png' | relative_url }})](https://foundation.mozilla.org)
121-
122-
The project was supported by a Mozilla Open Science Mini-Grant (see [official announcement](https://medium.com/read-write-participate/meet-mozillas-latest-open-science-awardees-cfa45348e5d5)) from February 2019 to May 2020.
123-
124-
--------
125-
126-
![NWO Logo](https://www.nwo.nl/themes/custom/nwo/assets/images/logo.svg?v=24)
127-
128-
The project was supported by the NWO Open Science Fund from March 2024 to February 2025, see [NWO project description page](https://www.nwo.nl/en/projects/osf232063) and details of [CHECK-NL](/nl).

project.md

Lines changed: 30 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,9 @@ layout: page
33
title: Project
44
permalink: /project/
55
style: principles
6+
redirect_from:
7+
- /about
8+
- /about/
69
---
710

811
## About
@@ -14,10 +17,10 @@ The fact that such a check is conducted and the communication between codechecke
1417

1518
## The CODECHECK principles
1619

17-
1. <span class="principle">Codecheckers record but dont investigate or fix.</span>
20+
1. <span class="principle">Codecheckers record but don't investigate or fix.</span>
1821
<details>
1922
<summary>More about this principle...</summary>
20-
The codechecker follows the authors instructions to run the code. If instructions are unclear, or if code does not run, the codechecker tells the author. We believe that the job of the codechecker is not to fix these problems but simply to report them to the author and await a fix. The level of documentation required for third parties to reproduce a workflow is hard to get right, and too often this uncertainty leads researchers to give up and not document it at all. The conversation with a codechecker fixes this problem.
23+
The codechecker follows the author's instructions to run the code. If instructions are unclear, or if code does not run, the codechecker tells the author. We believe that the job of the codechecker is not to fix these problems but simply to report them to the author and await a fix. The level of documentation required for third parties to reproduce a workflow is hard to get right, and too often this uncertainty leads researchers to give up and not document it at all. The conversation with a codechecker fixes this problem.
2124
<em>Codecheckers take the pictures at a crime scene, they do not hunt the criminal.</em>
2225
</details>
2326
1. <span class="principle">Communication between humans is key.</span>
@@ -28,24 +31,23 @@ The fact that such a check is conducted and the communication between codechecke
2831
1. <span class="principle">Credit is given to codecheckers.</span>
2932
<details>
3033
<summary>More about this principle...</summary>
31-
The value of performing a CODECHECK is comparable to that of a peer review, and it may require a similar amount of time. Therefore, the codecheckers activity should be recorded, ideally in the published paper. The public record can be realised by publishing the certificate in a citable form (i.e., with a DOI), by listing codecheckers on the journals website or, ideally, by publishing the checks alongside peer review activities in public databases.
34+
The value of performing a CODECHECK is comparable to that of a peer review, and it may require a similar amount of time. Therefore, the codechecker's activity should be recorded, ideally in the published paper. The public record can be realised by publishing the certificate in a citable form (i.e., with a DOI), by listing codecheckers on the journal's website or, ideally, by publishing the checks alongside peer review activities in public databases.
3235
<em>Codechecks are an excellent opportunity to involve early career researchers (ECRs) or research software engineers (RSEs) in peer review.</em>
3336
</details>
3437
1. <span class="principle">Workflows must be auditable.</span>
3538
<details>
3639
<summary>More about this principle...</summary>
37-
The codechecker should have sufficient material to validate the workflow outputs submitted by the authors. <a href="https://doi.org/10.1038/d41586-018-05256-0" title="Stark PB: Before reproducibility must come preproducibility. Nature. 2018; 557(7707): 613.">Stark</a> calls this "preproducibility" and the <a href="https://icerm.brown.edu/topical_workshops/tw12-5-rcem/icerm_report.pdf" title="Stodden V, Bailey DH, Borwein J, et al.: Setting the Default to Reproducible: Reproducibility in Computational and Experimental Mathematics. Technical report, The Institute for Computational and Experimental Research in Mathematics, 2013.">ICERM report</a> defines the level "Auditable Research" similarly. Communities can establish their own good practices or adapt generic concepts and practical tools, such as publishing all building blocks of science in a research compendium (cf. <a href="https://research-compendium.science/">https://research-compendium.science/</a>) or <a href="https://doi.org/10.22541/au.153922477.77361922" title="Barba LA: Praxis of Reproducible Computational Science. 2018.">repro-pack</a>. A completed check means that code could be executed at least once using the provided instructions, and, therefore, all code and data was given and could be investigated more deeply or extended in the future. Ideally, this is a “one click” step, but achieving this requires particular skills and a sufficient level of documentation for third parties. Furthermore, automation may lead to people gaming the system or reliance on technology, which can often hide important details. All such aspects can reduce the understandability of the material, so we estimate our approach to codechecking, done without automation and with open human communication, to be a simple way to ensure long-term transparency and usefulness. We acknowledge that <a href="https://twitter.com/khinsen/status/1242842759733665799" title="Konrad Hinsen (@khinsen) on Twitter: 'My crystal ball tells me that in the long run, bit-for-bit reproducibility will become the norm. Not because people realize it matters, but because it can be automatized. ´Good-enough´ reproducibility requires scientific judgment, so it's more expensive to ensure/check.'">others have argued</a> in favour of bitwise reproducibility because, in the long run, it can be automated, but until then we need CODECHECK’s approach.
40+
The codechecker should have sufficient material to validate the workflow outputs submitted by the authors. <a href="https://doi.org/10.1038/d41586-018-05256-0" title="Stark PB: Before reproducibility must come preproducibility. Nature. 2018; 557(7707): 613.">Stark</a> calls this "preproducibility" and the <a href="https://icerm.brown.edu/topical_workshops/tw12-5-rcem/icerm_report.pdf" title="Stodden V, Bailey DH, Borwein J, et al.: Setting the Default to Reproducible: Reproducibility in Computational and Experimental Mathematics. Technical report, The Institute for Computational and Experimental Research in Mathematics, 2013.">ICERM report</a> defines the level "Auditable Research" similarly. Communities can establish their own good practices or adapt generic concepts and practical tools, such as publishing all building blocks of science in a research compendium (cf. <a href="https://research-compendium.science/">https://research-compendium.science/</a>) or <a href="https://doi.org/10.22541/au.153922477.77361922" title="Barba LA: Praxis of Reproducible Computational Science. 2018.">repro-pack</a>. A completed check means that code could be executed at least once using the provided instructions, and, therefore, all code and data was given and could be investigated more deeply or extended in the future. Ideally, this is a “one click” step, but achieving this requires particular skills and a sufficient level of documentation for third parties. Furthermore, automation may lead to people gaming the system or reliance on technology, which can often hide important details. All such aspects can reduce the understandability of the material, so we estimate our approach to codechecking, done without automation and with open human communication, to be a simple way to ensure long-term transparency and usefulness. We acknowledge that <a href="https://twitter.com/khinsen/status/1242842759733665799" title="Konrad Hinsen (@khinsen) on Twitter: 'My crystal ball tells me that in the long run, bit-for-bit reproducibility will become the norm. Not because people realize it matters, but because it can be automatized. ´Good-enough´ reproducibility requires scientific judgment, so it's more expensive to ensure/check.'">others have argued</a> in favour of bitwise reproducibility because, in the long run, it can be automated, but until then we need CODECHECK's approach.
3841
</details>
3942
1. <span class="principle">Open by default and transitional by disposition.</span>
4043
<details>
4144
<summary>More about this principle...</summary>
4245
Unless there are strong reasons to the contrary (e.g., sensitive data on human subjects), all code and data, both from author and codechecker, will be made freely available when the certificate is published. Openness is not required for the paper itself, to accommodate journals in their transition to Open Access models. The code and data publication should follow community good practices. Ultimately we may find that CODECHECK activities are subsumed within peer review.
4346
</details>
4447

45-
4648
These basic principles ensure they are feasible to add in a scholarly communication process but still have a huge positive impact on the transparency and usefulness of the published material.
4749
They strike a **balance** between the ideals of auditable high-quality research software and the reality of publication pressure and only slowly changing academic evaluation practices.
48-
Of course, numerous requirements on openness/transparency (e.g. depositing the CODECHECK report publicly with a DOI), about software quality (tests, releases, documentation), on copyright/licensing, and regarding best practices for computer-based analyses (e.g. workflow management, data/software citation) are thinkable, but intentionally remain to be defined by implementations of the principles in each community of practice.
50+
Of course, numerous requirements on openness/transparency (e.g. depositing the CODECHECK certificate publicly with a DOI), about software quality (tests, releases, documentation), on copyright/licensing, and regarding best practices for computer-based analyses (e.g. workflow management, data/software citation) are thinkable, but intentionally remain to be defined by implementations of the principles in each community of practice.
4951
While the CODECHECK initiators strongly support of Open Science, a CODECHECK does not exclude research not falling into your definition of Open Science.
5052

5153
These principles can be **implemented** in different ways.
@@ -60,7 +62,13 @@ In the future, we hope to update these principles and to work together with rese
6062
Our main objective is to increase the number of communities and publication platforms that include codechecking in their review processes: preprint servers, journals, or conferences, for example.
6163
We especially target the future of scholarly communication: independent scholar-led diamond Open Access journals.
6264
To make codechecking feasible for these journals and others, we are looking for funding for _CODECHECK editors_, for developing _training material and mentoring programmes_ together with [friendly projects from the Open Science community](/partners/#projects), and for building and providing _open and free supporting infrastructure_ for conducting checks.
63-
With these steps we will scale CODECHECK beyond the [existing list of awesome volunteer codecheckers](https://github.com/codecheckers/codecheckers/blob/master/codecheckers.csv) and [current partner journals and conferences](/partners).
65+
With these steps, we will scale CODECHECK beyond the [existing list of awesome volunteer codecheckers](https://github.com/codecheckers/codecheckers/blob/master/codecheckers.csv) and [current partner journals and conferences](/partners).
66+
67+
Ongoing and past **projects** that are related to CODECHECK include:
68+
69+
- [Mozilla project](/mozilla-project) - an early demonstrator for codechecking
70+
- [CHECK-NL](/nl) - establish a codechecking community in the Netherlands
71+
- [CHECK-PUB](/pub) - support codechecking in independent journals based on Open Journal Systems (OJS)
6472

6573
## Project history
6674

@@ -85,3 +93,18 @@ To cite CODECHECK in scientific publications, please use the following citation/
8593
> _Eglen, S., & Nüst, D. (2019). CODECHECK: An open-science initiative to facilitate the sharing of computer programs and results presented in scientific publications. Septentrio Conference Series, (1). [https://doi.org/10.7557/5.4910](https://doi.org/10.7557/5.4910)_
8694
8795
To get or give a quick overview of the project, feel free to use or extend the [existing slide decks](https://github.com/codecheckers/slides).
96+
97+
## Funding
98+
99+
[![Mozilla logo]({{ 'img/mozilla.png' | relative_url }})](https://foundation.mozilla.org)
100+
101+
The project was supported by a Mozilla Open Science Mini-Grant (see [official announcement](https://medium.com/read-write-participate/meet-mozillas-latest-open-science-awardees-cfa45348e5d5)) from February 2019 to May 2020.
102+
103+
------
104+
105+
![NWO Logo](https://www.nwo.nl/themes/custom/nwo/assets/images/logo.svg?v=24)
106+
107+
The project was supported by the NWO Open Science Fund from March 2024 to February 2025, see [NWO project description page](https://www.nwo.nl/en/projects/osf232063) and details of [CHECK-NL](/nl).
108+
109+
------
110+

0 commit comments

Comments
 (0)