Campaign proposal: Open Code Pledge

[rmib200]

• Campaign co-creator/s: Manuel Illanes @rmib200

• Rationale: Many areas of Neuroscience are critically dependent on computational tools for things like analysis of large volumes of data, tasks, visualization, etc. Right now, we can say that sharing code is the exception when new research is published. Only certain journals require papers to include a statement on whether programs are available.
  We belive that all jounals should adopt policies that strongly encourage or mandate the sharing of code. Sharing all code and data needed to allow others to reproduce our work should be considered a regular practice. At least the key parts when this is not possible or practical.

• Action: (1) Whenever you publish an article, also publish the code you used for the analysis and implementation via an open, freely available platform (e.g., Github, OSF, papers with code). (2) Generate a DOI for the code and link to this DOI in the article. (Optional)
  * Make sure that at least one or more figures from the manuscript are reproducible.
  * If you think the work is too computationally intensive, one small "toy" example could be used to demostrate it.
  * Also, you can upload a config file when possible.
  
  We can demonstrate, leading by example, the obvious benefits of adopting these practices like more transparency, reliability and faster adoption of new methods. This will improve the future work produced on Neuroscience, even if the code doesn't fully work in 20 years. Because it will still be more precise and complete than text in a paper.
  
  Open-source code can be used for promotion, reference letters, and mentions. Take into account that right now, the limitation is cultural rather than technical so let's take a part in changing that. Let's free our knowledge.

If you can provide feedback on anything that could be done differently to benefit the field or get more people motivated to do anything regarding science, please don't doubt to comment. For Example, we could start by listing all the possible best practices for this.

• Eligibility criteria: All fields (but we will target neuroscience researchers initially).

• Optional anonymity: Yes.

• Threshold: 100 500 (calculated separately for each field)

• Pledge duration: 18 months. Lifetime

[CooperSmout]

Great idea! Thanks for proposing this campaign! Some initial thoughts:

• Action: publish ALL your code. ALWAYS.

I think we could make this more specific, e.g. ask people to publish code in an open (indexed?) repository (e.g., OSF, Github). Also, I wonder if will scare people off by asking them to publish ALL of their code (as great as this would be!). What do you think about instead asking people to publish all code related to any papers/preprints they publish?

• Eligibility criteria: Mainly for researchers on BCI and Neuroscience but could apply for all fields.

In previous campaigns we've included a drop-down box for people to indicate their field. This way the campaign can be open to all researchers, but pledges only activate when there is a critical mass of support in your field. But I'm also happy to target a campaign specifically at Neuroscience researchers, if you prefer. This is all experimentation after all, and we are finding out what works as we go :)

• Pledge duration: ? Not sure. Should be permanent.

Totally agree that permanent would be best, but wonder if we might get more people to sign if we only ask for a fixed period (e.g. 1 or 2 years)? I can imagine some people thinking "Sure, but what if I change labs and my new PI doesn't support open code?". Again, it's just about lowering barriers to signing, and we can of course follow up with another (bigger) campaign in a years time in which we ask for a longer/permanent duration.

[sina-mansour]

That's a great idea!

I'm thinking, a good way to achieve this goal is to create a GitHub repo for this purpose. Let me elaborate on what I mean.

I'm thinking if a template github repo is made with github pages web integration. We can ask researchers to fork that repository and add their code on top of that. This way a template for data and code sharing comes along with the open code proposal/pledge.

This way, ideally after a while, there will be quite a few forks of the repositories that have a similar format and are all linked to a separate paper.

This may require a bit of development to prepare the template (it should contain a proposed directory structure, and a landing page which loads through github pages and is linked to the published article).

[CooperSmout]

Hi and welcome to the forum @sina-mansour :) that's a cool idea! You mentioned using a similar format -- did you have something in mind? I'm aware of standards for neuroimaging data (e.g., BIDS), but not for neuroimaging code.

One thing I'm wary of is making these campaigns too complex, because various people have advised me to keep campaigns as simple as possible to reduce barriers to entry. Basically, every requirement you add gives people an opportunity to misunderstand/pause for thought/disagree, and can ultimately mean they don't take the pledge in the first place. E.g. for this idea, I can imagine some people might be happy uploading their code to OSF, but as soon as you start talking about 'forking' and 'Github pages' they will just tune out and we lose them. But I wonder if this could work as a suggestion (e.g. linking to a template as your suggest), rather than a requirement? That way people would still be free to use other methods, if they prefer, and we don't lose them to confusion.

[rmib200]

Thanks for all the feedback, I edited the details of the original a little. I guess the main goal is to encourage people who is about to publish their research to also add the code they wrote for greater chance of replicability. Report the version of the libraries, OS, docker containers, etc. Still looking for more feedback on this.

[I-Khormi]

I support this idea of publishing the code.
To keep this idea simple, I think if the researchers/authors put a link in the article paper to take you to their code either in GitHub or the institutional website or personal page.
It will also help new PhD students in the Neuroscience field to learn and practice to reproduce their work.

[CooperSmout]

Hi @I-Khormi, thanks for showing your support (helps us know which campaigns will have traction when we launch!)

I think if the researchers/authors put a link in the article paper to take you to their code either in GitHub or the institutional website or personal page.

Absolutely, best practices is for authors to link to their code in the paper. I wonder if this should also be part of the pledge, i.e. to provide a link in their paper?

The cool part about some repositories (e.g., OSF) is that you can also get a DOI for your data, so if people were to upload their data AND code, they could get a DOI, which they link to in their paper, and gives people a chance to cite their data/code

[rmib200]

@CooperSmout great!! And OSF asks you for anything on particular to let you get a DOI? A link for the paper would be a great idea.

[rmib200]

@I-Khormi yes! Authors can put a link of the repo on the same paper to facilitate access to the code.

[nsunami]

Great campaign! My thought is whether it would be better to frame the message positively, like "publish all research with code". Framing it as "stop doing" definitely catches attention, but it may be easier for people to put their names with a positive message (since we generally do not want to be told to stop something 😄). Just a thought!

[rmib200]

@nsunami Great idea! Yes, people will definetly will be more likely to put their names with a more positive tone. 😄 Thanks!

[rmib200]

Description updated!