diff --git a/appendices/editor-in-chief-checks.md b/appendices/editor-in-chief-checks.md index d08232fd..c8333c34 100644 --- a/appendices/editor-in-chief-checks.md +++ b/appendices/editor-in-chief-checks.md @@ -9,12 +9,12 @@ to work on them before the review process begins. Please check our [Python packaging guide](https://www.pyopensci.org/python-package-guide) for more information on the elements below. -- [ ] **Installation** The package can be installed from a community repository such as PyPI (preferred), and/or a community channel on conda (e.g., conda-forge, bioconda). +- [ ] **Installation** The package can be installed from a community repository such as PyPI (preferred), and/or a community channel on conda (for example, conda-forge, bioconda). - [ ] The package imports properly into a standard Python environment `import package`. - [ ] **Fit** The package meets criteria for [fit](https://www.pyopensci.org/software-peer-review/about/package-scope.html#what-types-of-packages-does-pyopensci-review) and [overlap](https://www.pyopensci.org/software-peer-review/about/package-scope.html#package-overlap). - [ ] **Documentation** The package has sufficient online documentation to allow us to evaluate the package's function and scope *without installing the package*. This includes: - [ ] User-facing documentation that overviews how to install and start using the package. - - [ ] Short quickstart tutorials that help a user understand how to use the package and what it can do for them. + - [ ] Short, quickstart tutorials that help a user understand how to use the package and what it can do for them. - [ ] API documentation (documentation for your code's functions, classes, methods, and attributes): this includes clearly written docstrings with variables defined using a standard docstring format. - [ ] Core GitHub repository Files - [ ] **README** The package has a `README.md` file with a clear explanation of what the package does, instructions on how to install it, and a link to development instructions. @@ -28,13 +28,14 @@ NOTE: We prefer that you have development instructions in your documentation too - [ ] **Package overlap** The package doesn't entirely overlap with the functionality of other packages that have already been submitted to pyOpenSci. - [ ] **Archive** (JOSS only, may be post-review): The repository DOI resolves correctly. - [ ] **Version** (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)? +- [ ] The package authors have disclosed the use of Generative AI tools in the development and/or maintenance of their package. **Optional:** Let projects know that it's a great idea for projects to have a .github repository for the project organization where they can host a commonly used LICENSE, Code of Conduct, and even a YAML file with label definitions. These items will then be automatically applied to every repository in the organization to ensure consistency (but can be customized within repos too). The [SunPy project](https://github.com/sunpy/.github/) has a great example of this. --- - [ ] [Initial onboarding survey was filled out ](https://forms.gle/F9mou7S3jhe8DMJ16) We appreciate each maintainer of the package filling out this survey individually. :raised_hands: -Thank you, authors, in advance for setting aside five to ten minutes to do this. It truly helps our organization. :raised_hands: +Thank you, authors, in advance, for setting aside five to ten minutes to do this. It truly helps our organization. :raised_hands: --- ******* diff --git a/appendices/gen-ai-checklist.md b/appendices/gen-ai-checklist.md new file mode 100644 index 00000000..b2b1b2cd --- /dev/null +++ b/appendices/gen-ai-checklist.md @@ -0,0 +1,10 @@ +```markdown +- [ ] Some parts of the package were created using LLMs in some way. + * Please check any of the boxes below to clarify which parts of the package were impacted by LLM use + - [ ] LLMs were used to develop code + - [ ] LLMs were used to develop documentation + - [ ] LLMs were used to develop tests + - [ ] LLMs were used to develop infrastructure (CI, automation) + +- [ ] I acknowledge that a human (at least one member of our maintainer team) has reviewed, edited, and modified all LLM-generated code and text submitted in this review to ensure it’s not biased (if text) and the code runs (if code). +``` diff --git a/how-to/author-guide.md b/how-to/author-guide.md index f5820967..3248bae5 100644 --- a/how-to/author-guide.md +++ b/how-to/author-guide.md @@ -65,7 +65,7 @@ Curious about the general timeline for pyOpenSci reviews? :link-type: doc :class-card: left-aligned -Read about our peer review policies. +Read through our peer review policies before submitting a package to us. ::: :::: @@ -86,9 +86,10 @@ Before you begin this process, [please be sure to read the review process guidel 1. Please be sure that you have time to devote to making changes to your package. During review, you will receive feedback from an editor and two reviewers. Changes could -take time. Please consider this before submitting to us. You can read more about the timeline to make changes in our [peer review policies page](../our-process/policies). +take time. Please consider this before submitting to us. You can read more about the timeline for making changes on our [peer review policies page](../our-process/policies). 2. A diverse group of volunteer editors and reviewers leads peer review. Please be considerate when engaging with everyone online. +3. All participants in our peer review process need to follow the [pyOpenSci Code of Conduct](https://www.pyopensci.org/handbook/CODE_OF_CONDUCT.html). Please review it before submitting to us. ``` ## 1. Do you plan to continue to maintain your package? @@ -128,7 +129,7 @@ emphasis on long-term software maintenance and focuses more on publication quality and citation/credit. ``` -## 2. Does Your Package Meet Packaging Requirements? +## 2. Does your package meet packaging requirements? Before submitting your project for review with pyOpenSci, make sure that your package meets all of the requirements listed in the editor checks (see below). @@ -147,6 +148,11 @@ for Python packaging, including discussions of: ``` +:::{important} +Given the increased use of Generative AI tools (LLMs), we have developed a disclosure policy for all packages submitted to pyOpenSci (generative-ai-policy). Please review this before submitting to us. +::: + + ```{hint} **Do you have questions about Python packaging or our peer review process?** @@ -169,7 +175,7 @@ that you can use to create a high-quality Python package.](https://www.pyopensci.org/python-package-guide/package-structure-code/python-package-build-tools.html) ``` -## 3. Is Your Package in Scope for pyOpenSci? +## 3. Is your package in scope for pyOpenSci? Next, check to see if your package falls within the topical and technical scope of pyOpenSci. If you aren't sure about whether your package fits within pyOpenSci's scope (below), submit @@ -187,13 +193,13 @@ Our current categories for determining package scope are below: Click here to view our technical and domain scope requirements. ``` -## 4. Submit Your Package for Peer Review +## 4. Submit your package for peer review To submit your package for peer review, you can open an issue in our [pyopensci/software-review repo](https://github.com/pyOpenSci/software-review/issues/new/choose/) repository and fill out the [Submit Software for Review](https://github.com/pyOpenSci/software-submission/issues/new?template=submit-software-for-review.md) issue template. -## 5. Editor-in-Chief Reviews Package for Scope and Minimal Infrastructure Criteria +## 5. Editor-in-Chief reviews package for scope and minimal infrastructure criteria Once the issue is opened, our editor-in-chief and an editor from our editorial board will review your submission within **2 weeks** and respond with next steps. The editor may request that you make updates @@ -206,7 +212,7 @@ package if it does not fall within our scope. Click here to view the editor checks that will be used to evaluate your package. ``` -## 6. The Review Begins +## 6. The review begins If your package meets the minimal criteria for being reviewed, it may then be given to an editor with appropriate domain experience @@ -216,7 +222,7 @@ issue within **3 weeks**. Reviewers can also open issues in your package reposit We prefer issues that link back to the review as they document changes made to your package that were triggered by our review process. -## 7. Response to Reviews +## 7. Response to reviews You should respond to reviewers’ comments within **2 weeks** of the last-submitted review. You can make updates to your package at any time. We @@ -230,7 +236,7 @@ Once the reviewers are happy with the changes that you've made to the package, t editor will review everything and accept your package into the pyOpenSci ecosystem. Congratulations! You are almost done! -## My Package is Approved, Now What? +## My package is approved, now what? Congratulations on being accepted into the pyOpenSci community of maintainers! Once your package is approved, a few things will happen: @@ -245,7 +251,7 @@ Once your package is approved, a few things will happen: If you'd like to submit your package to JOSS, you can do so now. Remember that JOSS will accept our review as theirs, so **you DO NOT need to go through another review**. Read more below. -### Journal of Open Source Software (JOSS) Submission +### Journal of Open Source Software (JOSS) submission pyOpenSci has a [partnership with JOSS](JOSS), where our review is accepted by JOSS by default if the package fits into the JOSS scope. diff --git a/our-process/policies.md b/our-process/policies.md index f31a2b97..c4117be7 100644 --- a/our-process/policies.md +++ b/our-process/policies.md @@ -3,15 +3,15 @@ ## Review process guidelines pyOpenSci packages are reviewed for quality, fit, scope, documentation, and -usability. The review process is similar to a manuscript review, however, it +usability. The review process is similar to a manuscript review; however, it has a stronger focus on Python packaging best practices. Unlike a manuscript review, our peer review process is an ongoing conversation. -Once all major issues and questions are addressed, the review editor will make -a decision to accept, hold, or reject the package. +Once all major issues and questions are addressed, the review editor will decide +to accept, hold, or reject the package. Rejections are usually done early in the process, before the review process -begins. In rare cases, a package may also not be on-boarded into the pyOpenSci +begins. In rare cases, a package may also not be onboarded into the pyOpenSci ecosystem after review & revision. It is ultimately the editor’s decision on whether or not to reject the package @@ -20,26 +20,27 @@ based on how the reviews are addressed. ## Review communication approach Communication between authors, reviewers, and editors takes -place on GitHub. You can, however choose to contact the editor by email if +place on GitHub. You can, however, choose to contact the editor by email if needed. When submitting a package, please make sure that your GitHub notification -settings are setup to notify you when you receive feedback on the review issue. +settings are turned on for the software-submission repository to notify you +when you receive feedback on a review issue. ## Submitting your package for review in other venues We recommend submitting your package for review with pyOpenSci before submitting a software paper describing the package to a journal. -Review feedback may result in major improvements and updates to your package, -including changes that could be break package functionality. +Review feedback may result in significant improvements and updates to your package, +including changes that could break package functionality. Applying reviewer or editor recommendations to your package can improve your -users' experience with future versions of your package even if your package is +users' experience with future versions of your package, even if your package is already published on `PyPI` or `conda-forge`. > Please do not submit your package for review while it or an associated -> manuscript is also under review at another venue, as this may result on +> manuscript is also under review at another venue, as this may result in > conflicting requests for changes from two sets of reviewers. ### Publication with Journal of Open Source Software (JOSS) @@ -77,6 +78,33 @@ a conflict of interest if: In the case where none of the associate editors can serve as editor, an external guest editor will be recruited to lead the package review. +(generative-ai-policy)= +## Policy for use of generative AI / LLMs + +:::{admonition} How this policy was developed +:class: important + +The policy below was co-developed by the pyOpenSci community. Its goals are: + +* Acknowledgment of and transparency around the widespread use of Generative AI tools with a focus on Large Language Models (LLMs) in Open Source. +* Protect peer review efficiency: Ensure human review of any LLM-generated contributions to a package to protect editor and reviewer volunteer time in our peer review process. +* Raise awareness of challenges that Generative AI tools present to the scientific (and broader) open source community. +[Please see this GitHub issue for a discussion of the topic.](https://github.com/pyOpenSci/software-peer-review/issues/331) +::: + +### Disclosure of generative AI use in pyOpenSci reviewed packages + +* When you submit a package to pyOpenSci, please disclose any use of LLMs (Large Language Models) in your package’s generation by checking the appropriate boxes on our software submission form. Disclosure should generally include what parts of your package were developed using LLM tools. + * Please also disclose this use of Generative AI tools in your package's `README.md` file and in any modules where generative AI contributions have been implemented. +* We require that all aspects of your package have been reviewed carefully by a human on your maintainer team. Please ensure all text and code have been carefully checked for bias, bugs, and issues before submitting to pyOpenSci. +* Your acknowledgment of using Generative AI will not impact the success of your submission unless you have blindly copied text and code into your package without careful review and evaluation of its accuracy, and for any systemic bias. +* If the review team (comprised of the editor and reviewers) determines that the code and text in the package are too challenging to review, they can decide to pause and/or discontinue the review following this policy’s guidelines. + +Below is the checklist that you will need to respond to in our submission form: + +```{include} ../appendices/gen-ai-checklist.md +``` + ## Review timelines and on-hold reviews At any time, an author can choose to have their submission put on hold @@ -114,9 +142,9 @@ including issues, pull requests, and dates of the last release and last commit to the package repository. Activity is defined as a repository commit, pull request, or release. -We will flag packages that haven't been updated within a 1 year/ 12 month time +We will flag packages that haven't been updated within a 1-year/12-month time period based on activity. Packages with no activity after 12 months will be -flagged. At that time, pyOpenSci editorial team member will contact the package +flagged. At that time, a pyOpenSci editorial team member will contact the package maintainers to evaluate the maintenance status of their package. (archive-process)= @@ -130,15 +158,15 @@ ecosystem. In cases where the community heavily uses a package, we may collaborate with the community to identify reasonable next steps, such as assisting in finding a new maintainer. If a solution for the ongoing package -maintenance is not found, the package will be archived within the pyOpenSci -ecosystem. +maintenance of your package is not found, the package will be archived within +the pyOpenSci ecosystem. [See section on archiving below.](package-archive) If a sub-community decides to fork and maintain the package, we are open to working with the new maintainers to register the newly forked package within our ecosystem. The original package will be archived with a link to the new fork. -We will also add a note to any blogs written that highlight your tool that +We will also add a note to any blogs written that highlight your tool, that the package is no longer maintained. ### Quality commitment @@ -156,19 +184,20 @@ in touch with us if they do need to step down from maintaining a tool. ### Requesting Package Removal from the pyOpenSci Ecosystem In the unlikely scenario that a contributor of a package requests removal of -their package from our ecosystem, we retain the right offer the last / most +their package from our ecosystem, we retain the right to offer the last/most recently released version of that package in our ecosystem for archival purposes only. +(package-archive)= ### Archiving a package If a package appears to be no longer maintained, we will mark it as -archived which moves the package from our +archived, which moves the package from our [main package listing](https://www.pyopensci.org/python-packages.html#all-packages) to our [archived packaging](https://www.pyopensci.org/python-packages.html#archived-packages) listing section. -To archive a pyOpenSci approved package, add the +To archive a pyOpenSci-approved package, add the [archive label](https://github.com/pyOpenSci/software-submission/issues?q=label%3Aarchived) to the original review issue. Once this label is applied to the issue, the website will automatically update to reflect this status. If at any point