You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: _posts/2025-11-18-generative-ai-peer-review.md
+36-36Lines changed: 36 additions & 36 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -31,12 +31,12 @@ Our goal is transparency and fostering reproducible research. For scientific rig
31
31
32
32
LLMs are perceived as helping developers:
33
33
34
-
* Explain complex codebases
35
-
* Generate unit tests and docstrings
36
-
* In some cases, simplifying language barriers for participants in open source around the world
37
-
* Speeding up everyday workflows
34
+
- Explain complex codebases
35
+
- Generate unit tests and docstrings
36
+
- In some cases, simplifying language barriers for participants in open source around the world
37
+
- Speeding up everyday workflows
38
38
39
-
Some contributors also believe these products open source more accessible. And for some, maybe they do. However, LLMs also present
39
+
Some contributors perceive these products as making open source more accessible. And for some, maybe they do. However, LLMs also present
40
40
unprecedented social and environmental challenges.
41
41
42
42
### Incorrectness of LLMs and misleading time benefits
@@ -51,7 +51,7 @@ Training and running LLMs [requires massive energy consumption](https://www.unep
51
51
52
52
Heavy reliance on LLMs risks producing developers who can prompt, but not debug, maintain, or secure production code. This risk undermines long-term project sustainability and growth. In the long run, it will make it [harder for young developers to learn how to code and troubleshoot independently](https://knowledge.wharton.upenn.edu/article/without-guardrails-generative-ai-can-harm-education/).
53
53
54
-
> We’re really worried that if humans don’t learn, if they start using these tools as a crutch and rely on it, then they won’t actually build those fundamental skills to be able to use these tools effectively in the future. *Hamsa Bastani*
54
+
> We’re really worried that if humans don’t learn, if they start using these tools as a crutch and rely on it, then they won’t actually build those fundamental skills to be able to use these tools effectively in the future. _Hamsa Bastani_
55
55
56
56
### Ethics and inclusion
57
57
@@ -63,75 +63,75 @@ We acknowledge that social and ethical norms, as well as concerns about environm
63
63
64
64
Our community’s expectation is simple: **be open and disclose any Generative AI use in your package** when you submit it to our open software review process.
65
65
66
-
* Disclose LLM use in your README and at the top of relevant modules.
67
-
* Describe how the Generative AI tools were used in your package's development.
68
-
* Be clear about what human review you performed on Generative AI outputs before submitting the package to our open peer review process.
66
+
- Disclose LLM use in your README and at the top of relevant modules.
67
+
- Describe how the Generative AI tools were used in your package's development.
68
+
- Be clear about what human review you performed on Generative AI outputs before submitting the package to our open peer review process.
69
69
70
70
Transparency helps reviewers understand context, trace decisions, and focus their time where it matters most.
71
71
72
72
### Human oversight
73
73
74
74
LLM-assisted code must be **reviewed, edited, and tested by humans** before submission.
75
75
76
-
* Run your tests and confirm the correctness of the code that you submitted.
77
-
* Check for security and quality issues.
78
-
* Ensure style, readability, and concise docstrings.
79
-
* Explain your review process in your software submission to pyOpenSci.
76
+
- Run your tests and confirm the correctness of the code that you submitted.
77
+
- Check for security and quality issues.
78
+
- Ensure style, readability, and concise docstrings. Depending on the AI tool, generated docstrings can sometimes be overly verbose without adding meaningful understanding.
79
+
- Explain your review process in your software submission to pyOpenSci.
80
80
81
-
Please **don’t offload vetting of generative AI content to volunteer reviewers**. Arrive with human-reviewed code that you understand, have tested, and can maintain.
81
+
Please **don't offload vetting of generative AI content to volunteer reviewers**. Arrive with human-reviewed code that you understand, have tested, and can maintain. As the submitter, you are accountable for your submission: you take responsibility for the quality, correctness, and provenance of all code in your package, regardless of how it was generated.
82
82
83
83
### Watch out for licensing issues.
84
84
85
-
LLMs are trained on large amounts of open source code; most of that code has licenses that require attribution.
86
-
The problem? LLMs sometimes spit out near-exact copies of that training data, but without any attribution or copyright notices.
85
+
LLMs are trained on large amounts of open source code, and most of that code has licenses that require attribution (including permissive licenses like MIT and BSD-3).
86
+
The problem? LLMs sometimes produce near-exact copies of that training data, but without any attribution or copyright notices. **LLM output does not comply with the license requirements of the input code, even when the input is permissively licensed**, because it fails to provide the required attribution.
87
87
88
88
Why this matters:
89
89
90
-
* LLM-generated code may be *substantially similar* to copyrighted training data; sometimes it is identical. Copyright law focuses on how similar your content is compared to the original.
91
-
* You can't trace what content the LLM learned from (the black box problem); this makes due diligence impossible on your part. You might accidentally commit plagiarism or copyright infringement by using LLM output in your code even if you modify it.
92
-
* License conflicts can occur because of both items above. Read on...
90
+
- LLM-generated code may be _substantially similar_ to copyrighted training data; sometimes it is identical. Copyright law focuses on how similar your content is compared to the original.
91
+
- You can't trace what content the LLM learned from (the black box problem); this makes due diligence impossible on your part. You might accidentally commit plagiarism or copyright infringement by using LLM output in your code even if you modify it.
92
+
- License conflicts occur because of both items above. Read on...
93
93
94
-
When licenses clash, it gets messy. Say your package uses an MIT license (common in scientific Python), but an LLM outputs Apache-2.0 or GPL code—those licenses aren't compatible. You can't just add attribution to fix it. Technically, you'd have to delete everything and rewrite it from scratch to comply with the licensing requirements.
94
+
When licenses clash, it gets particularly messy. Even when licenses are compatible (e.g., MIT-licensed training data and MIT-licensed output), you still have a violation because attribution is missing. With incompatible licenses (say, an LLM outputs GPL code and your package uses MIT), you can't just add attribution to fix it—you'd technically have to delete everything and rewrite it from scratch using clean-room methods to comply with licensing requirements.
95
95
96
96
The reality of all of this is that you can't eliminate this risk of license infringement or plagiarism with current LLM technology. But you can be more thoughtful about how you use the technology.
97
97
98
98
**What you can do now:**
99
99
100
-
* Be aware that when you directly use content developed by an LLM, there will be inherent license conflicts.
101
-
* Understand and transform code that is returned from a LLM: Don't paste LLM outputs directly. Review, edit, and ensure you fully understand what you're using. You can ask the LLM questions to better understand it's outputs. This approach also helps you learn which addresses the education concerns that we raised earlier.
102
-
***Use LLMs as learning tools**: Ask questions, review outputs critically, then write your own implementation based on understanding. Often the outputs of LLMs are messy or inefficient. Use them to learn, not to copy.
103
-
* Consider [clean-room techniques](https://en.wikipedia.org/wiki/Clean-room_design): Have one person review LLM suggestions for approach; have another person implement from that high-level description
104
-
***Document your process**: If you plan to submit a Python package for pyOpenSci review, we will ask you about your use of LLM's in your work. Document the use of LLMs in your project's README file and in any modules with LLM outputs have been applied. Confirm that it has been reviewed by a human prior to submitting it to us, or any other volunteer lead peer review process.
100
+
- Be aware that when you directly use content from an LLM, there will be inherent license conflicts and attribution issues.
101
+
- Understand and transform code returned from an LLM: Don't paste LLM outputs directly. Review, edit, and ensure you fully understand what you're using. You can ask the LLM questions to better understand its outputs. This approach also helps you learn, which addresses the education concerns that we raised earlier.
102
+
-**Use LLMs as learning tools**: Ask questions, review outputs critically, then write your own implementation based on understanding. Often the outputs of LLMs are messy or inefficient. Use them to learn, not to copy.
103
+
- Consider [clean-room techniques](https://en.wikipedia.org/wiki/Clean-room_design): Have one person review LLM suggestions for approach; have another person implement from that high-level description
104
+
-**Document your process**: If you plan to submit a Python package for pyOpenSci review, we will ask you about your use of LLM's in your work. Document the use of LLMs in your project's README file and in any modules with LLM outputs have been applied. Confirm that it has been reviewed by a human prior to submitting it to us, or any other volunteer lead peer review process.
105
105
106
106
You can't control what's in training data, but you can be thoughtful about how you use these tools.
107
107
108
108
<divclass="notice"markdown="1">
109
109
Examples of how these licensing issues are impacting and stressing our legal systems:
0 commit comments