docs(various): tweaks to images (white space, labels), protocol (organisation) and references.bib

Author: amyheather

images/ispor_criteria.png

@@ -218,6 +218,11 @@ stress_chart = eval_chart(criteria_wide)
 # Display figure
 stress_chart.show(config={'displayModeBar': False})
 
+# Reduce white space
+stress_chart.update_layout(
+    margin=dict(l=0, r=0, t=10, b=0)
+)
+
 # Save figure as 300DPI static image for use in article
 pio.write_image(stress_chart, '../images/stress_criteria.png',
                 width=2.8*300, height=2.5*300, scale=1.97)
@@ -396,10 +401,10 @@ criteria_dict = {
   '8 Inputs': [6, 2, 0, 0],
   '9 Uncertainty': [3, 1, 3, 1],
   '10 Sensitivity': [4, 0, 3, 1],
-  '11 Face val.': [1, 0, 7, 0],
-  '12 Cross val.': [2, 0, 5, 1],
-  '13 External val.': [2, 0, 3, 3],
-  '14 Predictive val.': [0, 0, 1, 7],
+  '11 Face validity': [1, 0, 7, 0],
+  '12 Cross validity': [2, 0, 5, 1],
+  '13 External validity': [2, 0, 3, 3],
+  '14 Predictive validity': [0, 0, 1, 7],
   '15 Generalisability': [5, 1, 2, 0],
   '16 Stakeholders': [1, 0, 7, 0],
   '17 Funding': [6, 0, 2, 0],
@@ -413,6 +418,11 @@ ispor_chart = eval_chart(criteria_wide)
 # Display figure
 ispor_chart.show(config={'displayModeBar': False})
 
+# Reduce white space
+ispor_chart.update_layout(
+    margin=dict(l=0, r=0, t=10, b=0)
+)
+
 # Save figure as 300DPI static image for use in article
 pio.write_image(ispor_chart, '../images/ispor_criteria.png',
                 width=2.5*300, height=2*300, scale=2.2)

images/stars_criteria.png

@@ -33,7 +33,7 @@ For this work, six published healthcare DES models were selected. These were mod
         * Association for Computing Machinery (ACM) (@association_for_computing_machinery_acm_artifact_2020)
         * Center for Open Science (COS) (@blohowiak_badges_2023)
         * Institute of Electrical and Electronics Engineers (IEEE) (@institute_of_electrical_and_electronics_engineers_ieee_about_2024)
-        * Psychological Science (@hardwicke_transparency_2023 and @association_for_psychological_science_aps_psychological_2024)
+        * Psychological Science (@hardwicke_transparency_2024 and @association_for_psychological_science_aps_psychological_2024)
     * Recommendations from the pilot STARS framework for the sharing of code and associated materials from discrete-event simulation models (@monks_towards_2024).
 * This is timed
 
@@ -76,17 +76,17 @@ It was also informed by:
 
 ## Modifications from the protocol
 
-There was one major change from the protocol...
+There were two main changes from the protocol...
 
 | Deviation | Description and reason for the change |
 | --- | --- |
 | Expanding from 6 to 8 studies | Whilst partway through the 6th study, we reflected on what we had found so far, and felt it would be beneficial to do a few more studies. The primary motivation for this was to try and include a selection of studies that reflect the range of studies in the literature. So far, we had a few studies where the model code/run times were very large or very small, and we felt it beneficial to try and include some more "medium-sized" studies, as these are more typical of the literature. We also thought it would be good to include an older study (for example, about 10 years ago, if we can find one), as all those included so far were within the last few years, but working with more outdated code will be/can be a problem people encounter. |
+| Using the latest software packages | In the protocol, I had planned that - if no versions were provided  we select a version of the software and each package that is closest to but still prior to the date of the code archive or paper publication. I kept to this for the Python models (easily set using a conda/mamba environment). However, I had great difficulties attempting to do this in R, and could not successfully backdate both. As such, I used the latest versions of R and each package for those studies |
 
-There were also some minor changes, which are explained below...
+There were also some very minor changes, which are explained below...
 
 | Deviation | Description and reason for the change |
 | --- | --- |
-| Using the latest software packages | In the protocol, I had planned that - if no versions were provided  we select a version of the software and each package that is closest to but still prior to the date of the code archive or paper publication. I kept to this for the Python models (easily set using a conda/mamba environment). However, I had great difficulties attempting to do this in R, and could not successfully backdate both. As such, I used the latest versions of R and each package for those studies |
 | Using percentage difference in results to help decided reproduction success | This is not particularly deviation, as I did explore this, but I ultimately found it very unhelpful, as the percentage difference could be greatly impacted by scale (for example, 0.1 vs 0.2 would appear much greater than 3 vs 4, but the actual meaning of these differences might be similar (e.g. both might be considered a small difference) depending on the scale used and what is being compared - whilst in another context with a different scale, 0.1 vs 0.2 might actually reflect a huge difference!). |
 | Moving onto evaluation stage before receive author response regarding reproduction discrepancy or before getting consensus on reproduction | In the protocol, we required that authors are contacted if there are any remaining difficulties in running the code or items in the scope that were not reproduced. The authors were given a total of four weeks to respond if they chose to. We had implied that we must wait for this time to pass before continuing to the evaluation stage (since the three stages were presented as being completed one after another). The rationale for this was that the timings for the reproduction would be influenced by whether the evaluation had been completed or not, and vice versa. However, given the many possible influences on the timings, this was considered negligble.<br><br>We also required consensus on whether items had been succesfully reproduced or not before moving on. In some cases, this was not done, as other team members were not available (e.g. busy or on annual leave) and so could not yet give a second opinion, and so I progressed with the evaluation and got consensus on reproductions afterwards, once they were available. |
 | Organisation of the repository for the research compendium | In the protocol, we had planned that seperate folders were created for data, methods and outputs. This was generally followed but, if an alternative structure seemed more suitable. For example, if the original study already divided items well, but perhaps with different folder names or with multiple scripts folders or so on, we might have used that original structure, as it still served the purpose of being clear and easy to re-run, whilst reducing the number of differences compared with the original study. |

images/stress_criteria.png

@@ -518,6 +518,11 @@ stars_chart = eval_chart(stars_wide)
 # Display figure
 stars_chart.show(config={'displayModeBar': False})
 
+# Reduce white space
+stars_chart.update_layout(
+    margin=dict(l=0, r=0, t=10, b=0)
+)
+
 # Save figure as 300DPI static image for use in article
 pio.write_image(stars_chart, '../images/stars_criteria.png',
                 width=2.5*300, height=1.8*300, scale=2.2)

pages/paper_evaluation.qmd

@@ -2398,7 +2398,99 @@ @article{harris_array_2020
 	month = sep,
 	year = {2020},
 	note = {Publisher: Nature Publishing Group},
-	keywords = {Computational neuroscience, Computational science, Computer science, Software, Solar physics},
+	keywords = {Software, Computer science, Computational science, Computational neuroscience, Solar physics},
 	pages = {357--362},
 	file = {Full Text PDF:/home/amy/Zotero/storage/H3CV27FC/Harris et al. - 2020 - Array programming with NumPy.pdf:application/pdf},
 }
+
+@misc{plotly_technologies_inc_collaborative_2015,
+	address = {Montreal, QC},
+	title = {Collaborative data science},
+	url = {https://plot.ly},
+	publisher = {Plotly Technologies Inc.},
+	author = {{Plotly Technologies Inc.}},
+	year = {2015},
+}
+
+@article{hunter_matplotlib_2007,
+	title = {Matplotlib: {A} {2D} {Graphics} {Environment}},
+	volume = {9},
+	issn = {1558-366X},
+	shorttitle = {Matplotlib},
+	url = {https://ieeexplore.ieee.org/document/4160265},
+	doi = {10.1109/MCSE.2007.55},
+	abstract = {Matplotlib is a 2D graphics package used for Python for application development, interactive scripting,and publication-quality image generation across user interfaces and operating systems},
+	number = {3},
+	urldate = {2024-11-28},
+	journal = {Computing in Science \& Engineering},
+	author = {Hunter, John D.},
+	month = may,
+	year = {2007},
+	note = {Conference Name: Computing in Science \& Engineering},
+	keywords = {Python, application development, Computer languages, Equations, Graphical user interfaces, Graphics, Image generation, Interpolation, Operating systems, Packaging, Programming profession, scientific programming, scripting languages, User interfaces},
+	pages = {90--95},
+	file = {Full Text PDF:/home/amy/Zotero/storage/MLPM8FY4/Hunter - 2007 - Matplotlib A 2D Graphics Environment.pdf:application/pdf;IEEE Xplore Abstract Record:/home/amy/Zotero/storage/K8BZJFZV/4160265.html:text/html},
+}
+
+@misc{the_pandas_development_team_pandas-devpandas_2024,
+	title = {pandas-dev/pandas: {Pandas}},
+	url = {https://doi.org/10.5281/zenodo.3509134},
+	publisher = {Zenodo},
+	author = {{The pandas development team}},
+	month = sep,
+	year = {2024},
+}
+
+@inproceedings{mckinney_data_2010,
+	address = {Austin, Texas},
+	title = {Data {Structures} for {Statistical} {Computing} in {Python}},
+	url = {https://doi.curvenote.com/10.25080/Majora-92bf1922-00a},
+	doi = {10.25080/Majora-92bf1922-00a},
+	urldate = {2024-11-28},
+	author = {McKinney, Wes},
+	year = {2010},
+	pages = {56--61},
+	file = {Full Text:/home/amy/Zotero/storage/TKSWRQVZ/McKinney - 2010 - Data Structures for Statistical Computing in Pytho.pdf:application/pdf},
+}
+
+@article{van_gulick_resources_2024,
+	title = {Resources for {Selecting} and {Using} {Generalist} {Repositories} to {Support} {NIH} {Data} {Sharing}},
+	copyright = {Creative Commons Attribution 4.0 International},
+	url = {https://doi.org/10.5281/zenodo.13987727},
+	doi = {10.5281/ZENODO.13987727},
+	abstract = {This brief presentation for the NCI Office of Data Sharing Annual Data Sharing Symposium: Driving Cancer Advances through Impactful Research shared information about the Generalist Repository Ecosystem Inititative (GREI) and highlighted two GREI resources, the Generalist Repository Comparison Chart and the Generalist Repository Selection Flowchart.},
+	language = {en},
+	urldate = {2024-11-28},
+	author = {Van Gulick, Ana and Curtin, Lisa},
+	month = oct,
+	year = {2024},
+	note = {Publisher: Zenodo},
+}
+
+@misc{analysis_function_central_team_dependency_2024,
+	title = {Dependency and reproducibility},
+	url = {https://analysisfunction.civilservice.gov.uk/support/reproducible-analytical-pipelines/dependency-and-reproducibility/},
+	urldate = {2024-12-02},
+	journal = {Government Analysis Function},
+	author = {{Analysis Function Central Team}},
+	year = {2024},
+	file = {Dependency and reproducibility – Government Analysis Function:/home/amy/Zotero/storage/3RXF264Y/dependency-and-reproducibility.html:text/html},
+}
+
+@article{alston_beginners_2021,
+	title = {A {Beginner}'s {Guide} to {Conducting} {Reproducible} {Research}},
+	volume = {102},
+	copyright = {© 2020 The Authors. The Bulletin of the Ecological Society of America, published by Wiley Periodicals LLC on behalf of the Ecological Society of America},
+	issn = {2327-6096},
+	url = {https://onlinelibrary.wiley.com/doi/abs/10.1002/bes2.1801},
+	doi = {10.1002/bes2.1801},
+	language = {en},
+	number = {2},
+	urldate = {2024-12-02},
+	journal = {The Bulletin of the Ecological Society of America},
+	author = {Alston, Jesse M. and Rick, Jessica A.},
+	year = {2021},
+	note = {\_eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/bes2.1801},
+	pages = {e01801},
+	file = {Full Text PDF:/home/amy/Zotero/storage/5JPQLYHX/Alston and Rick - 2021 - A Beginner's Guide to Conducting Reproducible Rese.pdf:application/pdf;Snapshot:/home/amy/Zotero/storage/3PNWT4JP/bes2.html:text/html},
+}