Skip to content

Commit 3aad5d1

Browse files
committed
fix links and mention new page on homepage
1 parent caaea63 commit 3aad5d1

File tree

4 files changed

+56
-54
lines changed

4 files changed

+56
-54
lines changed

guide/community-workflow-author.md

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ See also the [CODECHECK community workflow overview](/guide/community-workflow-o
1313

1414
<span style="font-size: 60%; color: grey;">Inspired by Greg Wilson's first rule of <a href="http://teachtogether.tech/" title="Teaching Tech Together">Teaching Tech Together</a>.</span>
1515

16-
### Background
16+
### 1. Background
1717

1818
The author must provide a _[preproducible](https://doi.org/10.1038/d41586-018-05256-0) workflow_:
1919
all data and code files necessary to reproduce the results are provided in a way that allows fellow researchers to follow and execute the steps.
@@ -32,7 +32,7 @@ Structured information about the computing environment, such as a _colophon_ or
3232
Common sense shall be applied to decide about the suitable amount of data and to handle big datasets, sensitive datasets with privacy concerns, and long execution times.
3333
For example, data may be deposited depending on community practices in remote repositories, synthetic data may be used, subsets or preprocessed data may be included, or protected access to information may be provided (e.g. cloud-based data enclaves).
3434

35-
### Requirements
35+
### 2. Requirements
3636

3737
The minimal set of files, besides all required data and code, to implement a CODECHECK process are the following ones (`.` is the project root directory, which could be for example, `/home/username/research-project/2020/great-paper`).
3838
Ideally, the author supplies first versions of these three files, though they might also be jointly created in collaboration with the codechecker.
@@ -43,14 +43,14 @@ In case of the metadata, parts of the metadata will be provided by the codechec
4343
1. **`./codecheck.yml` file** with basic metadata and a list of output files created by the workflow, the so called manifest; these files must be created by the workflow and are the basis for validating a successful CODECHECK; see the [latest CODECHECK configuration file specification]({{ 'spec/config/latest' | absolute_url }}) for the required and optional contents and start with the [_minimal example for authors_]({{ 'spec/config/1.0/#tldr-for-authors' | absolute_url }})
4444
1. **`./LICENSE` file** with information about licenses for all submitted material, e.g. code license for scripts and data licenses for used datasets; see [The Turing Way](https://book.the-turing-way.org/reproducible-research/licensing.html) on licensing for guidance
4545

46-
### Publication
46+
### 3. Publication of workflow
4747

4848
The required files and the workflow code and data are published in a dedicated self-contained repository in the [codecheckers organisation on GitHub](https://github.com/codecheckers/).
4949
This happens by [forking](https://help.github.com/en/github/getting-started-with-github/fork-a-repo) the authors repository, if one already exist.
5050
If the repository is on GitLab(.com), the [cdchck organisation on GitLab](https://gitlab.com/cdchck) can be used.
5151
After the CODECHECK, authors may transfer the improvements and certificate back to their own repository.
5252

53-
### Extras
53+
### 4. Extras
5454

5555
Beyond the minimally required files to run a workflow, any additional configuration and information is extremely helpful, of course, and can greatly improve the smoothness of the CODECHECK process.
5656
Some hints as to what this can entail are given by the codechecker's tasks below - it is worth taking the checker's perspective to improve your submission.
@@ -69,11 +69,11 @@ Feel free to inquire in the [CODECHECK discussion forum](https://github.com/orgs
6969

7070
A great way to learn what a good way to meaningfully package your research for others to inspect, understand, and reproduce is to participate in a [ReproHack](https://reprohack.github.io/reprohack-hq/) and take on the reader/codechecker perspective.
7171

72-
### Submission
72+
### 5. Submission to CODECHECK
7373

74-
When your workflow is ready to be CODECHECKed, open an issue on the [CODECHECK register](https://github.com/codecheckers/register/issues/new/choose).
74+
When your workflow is ready to be codechecked, open an issue on the [CODECHECK register](https://github.com/codecheckers/register/issues/new/choose).
7575

76-
### During submission/preprint stage
76+
### 6. During submission/preprint stage
7777

7878
After the publication of the CODECHECK certificate, add a reference to the certificate in your paper, e.g., in a section describing your workflow or in the acknowledgements:
7979

@@ -85,5 +85,6 @@ After the publication of the CODECHECK certificate, add a reference to the certi
8585

8686
See the [CODECHECK register](/register) for a full list of codechecks, including direct links to the certificates.
8787
Take a look at existing checks for your discipline or community to get an impression on how CODECHECKs work.
88+
Selected examples are on the [workflow overview page](/guide/community-workflow-overview).
8889

8990
_Is your scientific dispipline missing?_ Time to **[get involved!](/get-involved)**.
Lines changed: 46 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
---
22
layout: page
33
title: CODECHECK community workflow for codecheckers
4-
permalink: guide/community-workflow
54
---
65

76
See also the [CODECHECK community workflow overview](/guide/community-workflow-overview) and [discuss your issues](https://github.com/orgs/codecheckers/discussions).
@@ -18,10 +17,10 @@ Now it's your turn.
1817
1. Create a repository in the CODECHECK GitHub organization, either by forking existing repository or creating new one and uploading materials
1918
1. Create a new directory in that repository where all new files will go
2019
1. Create a new document to write the CODECHECK certificate and *start documenting the ongoing codecheck now*
21-
The exact form of a codechecking procedure and form of documentation vary greatly, but there are some tools, such as [an R package](https://github.com/codecheckers/codecheck) to automate some steps, including [an Rmd template](https://github.com/codecheckers/codecheck/blob/master/inst/extdata/templates/codecheck/codecheck.Rmd); all of that is optional, as long as the final report contains the mandatory information
20+
The exact form of a codechecking procedure and form of documentation vary greatly, but there are some tools, such as [an R package](https://github.com/codecheckers/codecheck) to automate some steps, including [an Rmd template](https://github.com/codecheckers/codecheck/blob/master/inst/extdata/templates/codecheck/codecheck.Rmd); all of that is optional, as long as the final certificate contains the mandatory information
2221
1. Open the manuscript and follow the instructions to reproduce a workflow
2322
1. During the CODECHECK, contact the authors in case of problems; keep in mind the general [CODECHECK principles](/project/#the-codecheck-principles), especially “the codechecker records but does not fix” – unless it is a very trivial bug like pathnames; the authors can provide updated versions of code and documentation; however, the entire procedure should not be much more time-intensive than a normal peer review of a paper and not involve more than a few code revisions; the codechecker can always stop the process after a reasonable effort and close the issue as not successfully reproduced.
24-
1. Summarize the process and outcomes in your certificate and upload it as PDF to [Zenodo](https://zenodo.org/) or [OSF](https://osf.io/); add edited files and intemediary as well as output files as you see fit; the certifiate must at least contain the information on who checked what and how; the ambition should be to document for future self and other researchers; have a look at the available reports.
23+
1. Summarize the process and outcomes in your certificate and upload it as PDF to [Zenodo](https://zenodo.org/) or [OSF](https://osf.io/); add edited files and intemediary as well as output files as you see fit; the certifiate must at least contain the information on who checked what and how; the ambition should be to document for future self and other researchers; have a look at the available certificates.
2524
1. Adds a pull request to original repository for the CODECHECK badge (optional)
2625
1. Notify the editor in the GitHub issue about the completion
2726

@@ -37,7 +36,7 @@ Codecheckers should give feedback to the author and definetely allow workflows t
3736
It is very hard to put a precise number on the amount of work you should put into a CODECHECK.
3837
You're not expected to spend more time on a CODECHECK than you would on peer-reviewing a manuscript.
3938
You should take advantage of the fact that you can _talk to the author_ and feel free to reach out early and often, when you think that issues can be resolved quickly.
40-
Depending on your familiarity with the used programming language and specific tools at hand, a very rough experience value could be 30 minutes of reading documentation, downloading and installing software, and another 30 minutes to write up the CODECHECK report.
39+
Depending on your familiarity with the used programming language and specific tools at hand, a very rough experience value could be 30 minutes of reading documentation, downloading and installing software, and another 30 minutes to write up the CODECHECK certificate.
4140
The time in between for running the actual workflow will vary greatly, from minutes to hours, and hopefully can be run in the background.
4241
In case the computations run longer than your regular working day, consider asking the author to prepare a subset of the workflow.
4342

@@ -59,48 +58,49 @@ Some further tips:
5958
You can use `.codecheck` if `codecheck` exists in submission for some reason.
6059
_All files created by you go into this directory._
6160
The exception are files that the author could have used and which you suggest the author transfers into her own repository after the check (see "leave in a better condition" above).
62-
1. **Write a `Makefile`** to re-run the workflow based on provided documentation, i.e., recreate all files listed in the manifest by runnign the command `make codecheck`.
63-
This target should run the whole or most suitable subset of the workflow and create the report.
64-
1. **Optional contents** of the check directory.
65-
- Document the used computing environment, see [CODECHECK bundle guide](/guide/bundle).
66-
- Create a notebook as the basis for the report (see below), e.g. `codecheck.Rmd`.
67-
- Make the repository [Binder-ready](https://mybinder.readthedocs.io/); put all Binder-related files into `.binder/` directory to separate them from the author's content.
68-
1. **Write the CODECHECK report** and save it as a PDF file named **`codecheck.pdf`** in the check directory.
69-
The report should cover at least _WHO_ checked _WHAT_, and _HOW_.
70-
Imagine the report as a reminder for future you so you will be to re-check the workflow in two years time - what help would you need to do that?
71-
Take a look at the [example CODECHECKs](#examples) for existing reports to serve as templates.
72-
1. **Optional report sections** depending on interest, time, and skills:
73-
- _How to cite the report?_ Your CODECHECK is a valuable contribution to science, and you should add a short note on how to cite your report (see below for reserving the DOI).
74-
- Do the generated outputs match the ones in the original manuscript? Are the differences relevant or not?
75-
- Are there any parts of the workflow where the author could improve the code?
76-
- How long did it take you to conduct the CHECK, and where did you struggle?
77-
- Are used pieces of software and data properly CITED and publicly DEPOSITED und suitable LICENSES?
78-
- Are open formats (text-based etc.) used for input and output data?
79-
- Is the data and [software](https://content.iospress.com/articles/data-science/ds190026) FAIR?
80-
1. Add **mandatory codechecker-contributed information** to the **`codecheck.yml`** file, see [spec](/spec/config/latest)
81-
1. Wait for the article DOI.
82-
1. **Deposit the CODECHECK report on _Zenodo_** using your own Zenodo account
61+
1. **Write a `Makefile`** to re-run the workflow based on provided documentation, i.e., recreate all files listed in the manifest by runnign the command `make codecheck`.
62+
This target should run the whole or most suitable subset of the workflow and create the certificate.
63+
1. **Optional contents** of the check directory.
64+
- Document the used computing environment, see [CODECHECK bundle guide](/guide/bundle).
65+
- Create a notebook as the basis for the certificate (see below), e.g. `codecheck.Rmd`.
66+
- Make the repository [Binder-ready](https://mybinder.readthedocs.io/); put all Binder-related files into `.binder/` directory to separate them from the author's content.
67+
1. **Write the CODECHECK certificate** and save it as a PDF file named **`codecheck.pdf`** in the check directory.
68+
The certificate should cover at least _WHO_ checked _WHAT_, and _HOW_.
69+
Imagine the certificate as a reminder for future you so you will be to re-check the workflow in two years time - what help would you need to do that?
70+
Include a _full citation of the certificate_, because your CODECHECK is a valuable contribution to science (see below for reserving the DOI).
71+
Take a look at the [example CODECHECKs](/guide/community-workflow-overview#examples) for existing certificates to serve as templates.
72+
1. **Optional certificate sections** depending on interest, time, and skills:
73+
- Do the generated outputs match the ones in the original manuscript? Are the differences relevant or not?
74+
- Are there any parts of the workflow where the author could improve the code?
75+
- How long did it take you to conduct the CHECK, and where did you struggle?
76+
- Are used pieces of software and data properly CITED and publicly DEPOSITED und suitable LICENSES?
77+
- Are open formats (text-based etc.) used for input and output data?
78+
- Is the data and [software](https://content.iospress.com/articles/data-science/ds190026) FAIR?
79+
1. Add **mandatory codechecker-contributed information** to the **`codecheck.yml`** file, see [spec](/spec/config/latest)
80+
1. Wait for the article DOI.
81+
1. **Deposit the CODECHECK report on _Zenodo_** using your own Zenodo account.
82+
- _Reserve a DOI_
83+
- Add the DOI to the `codecheck.yml` file.
84+
- Add the DOI to the `codecheck.pdf` CODECHECK report, which should include a full citation of itself.
8385
- _Files_
8486
- `codecheck.pdf` (mandatory)
85-
- Optional: You can add any material to this record that you see fit, especially things that helped you with your reproduction, i.e., the [CODECHECK bundle](/guide/bundle)
86-
- _Communities_: Search for "codecheck" to add the record to the [CODECHECK community on Zenodo](https://zenodo.org/communities/codecheck)
87-
- _Authors_: Add all codecheckers as authors
88-
- _Title_: `"CODECHECK Certificate YYYY-NNN"` (certificate number issued via the register ticket above)
89-
- _License_: Use `Creative Commons Attribution 4.0 International` if you only upload the CODECHECK report, otherwise use `Other (Open)` or `Other (Attribution)` and document the licensing in the "Additional notes" field.
90-
- _Reserve a DOI_
91-
- Add the DOI to the `codecheck.yml` file
92-
- Mention the DOI to the `codecheck.pdf` CODECHECK report, e.g. as a subtitle
93-
- Use _other fields_ as you see fit (Description, Version, Language, Keywords)
94-
- _Contributors_: Add the original authors as contributors (see Zendo Metadata form section "Contributors (optional)") with a suitable role (e.g., "Researcher")
95-
- _Optionally_, add extra metadata
96-
- connect the Zenodo record to the GitHub repository with a "Relate/alternate identifier"
97-
- connect the Zenodo record to the article/preprint with a "Relate/alternate identifier"
98-
1. If the check was conducted for a piece of software for the first time or resulted in important lessons learned, consider adding it to the list of examples below.
99-
1. If possible, add the [CODE WORKS badge](https://github.com/codecheckers/website/tree/master/badges) <img src="/img/codeworks-badge.svg" alt="CODECHECK badge" height="16" style="margin-top: -4px;" /> to the article or the original software repository, e.g., by sending a pull request. The badge should link directly to the Zenodo record _via the DOI_.
100-
The following snippet should work in Markdown:
101-
102-
```md
103-
[![CODECHECK](https://codecheck.org.uk/img/codeworks-badge.svg)](https://doi.org/<DOI HERE>)
104-
```
105-
87+
- Optional: You can add any material to this record that you see fit, especially things that helped you with your reproduction, i.e., the [CODECHECK bundle](/guide/bundle).
88+
- _Communities_: Search for "codecheck" to add the record to the [CODECHECK community on Zenodo](https://zenodo.org/communities/codecheck).
89+
- _Authors_: Add all codecheckers as authors.
90+
- _Title_: `"CODECHECK Certificate YYYY-NNN"` (certificate number issued via the register ticket above).
91+
- _License_: Use `Creative Commons Attribution 4.0 International` if you only upload the CODECHECK report, otherwise use `Other (Open)` or `Other (Attribution)` and document the licensing of the different parts in an _Additional notes_ field.
92+
- _Description_: Copy the summary of the check here.
93+
- _Contributors_: Add the original authors as contributors (see Zendo Metadata form section "Contributors (optional)") with a suitable role (e.g., "Researcher").
94+
- _Optionally_, add extra metadata as you see fit (fields such as _Version_, _Language_, _Keywords_).
95+
- Connect the Zenodo record to the GitHub repository with a _Relate/alternate identifier_.
96+
- Connect the Zenodo record to the article/preprint with a _Related/alternate identifier_.
97+
1. If the check was conducted for a piece of software for the first time or resulted in important lessons learned, please suggest it to the editor in the checks's GitHub issue.
10698
1. If the check material is published on `github.com/codecheckers`, add the [`codecheck` topic](https://github.com/search?q=topic%3Acodecheck+fork%3Atrue+org%3Acodecheckers&type=Repositories) to the project.
99+
1. If possible, add the [CODE WORKS badge](https://github.com/codecheckers/website/tree/master/badges) <img src="/img/codeworks-badge.svg" alt="CODECHECK badge" height="16" style="margin-top: -4px;" /> to the software repository, e.g., by sending a pull request.
100+
The badge should link directly to the Zenodo record _via the DOI_.
101+
The following snippet should work in Markdown:
102+
103+
```md
104+
[![CODECHECK](https://codecheck.org.uk/img/codeworks-badge.svg)](https://doi.org/<DOI HERE>)
105+
```
106+

guide/community-workflow-overview.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ The community workflow is a concrete implementation of the abstract [CODECHECK p
1818
The community workflow documentation is split into three parts:
1919

2020
- [CODECHECK community workflow for authors](/guide/community-workflow-author)
21-
- [CODECHECK community workflow for codecheckers](/guide/community-workflow-author)
21+
- [CODECHECK community workflow for codecheckers](/guide/community-workflow-codechecker)
2222
This perspective also has variants: a focused version, a more detailed version, and a version for [in-person workshop with CHECK-NL](/nl/workflow)
2323
- [CODECHECK community workflow for editors](/guide/community-workflow-editor)
2424

index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@ CODECHECK is based on **five principles** which are described in detail in the [
3535
1. Workflows must be auditable.
3636
1. Open by default and transitional by disposition.
3737

38+
The principles can be implemented in different [processes](/process/), one of which is the [CODECHECK community workflow](/guide/community-workflow-overview).
3839
**If you want to get involved as a codechecker in the community, or if you want to apply the CODECHECK principles in your journal or conference, please take a look at the [Get Involved](/get-involved) page.**
3940

4041
To **stay in touch** with the project, follow us on social media at **<https://fediscience.org/@codecheck>**.

0 commit comments

Comments
 (0)