Skip to content

SurveyPreview.md

Latest commit

 

History

History
121 lines (91 loc) · 3.8 KB

SurveyPreview.md

File metadata and controls

121 lines (91 loc) · 3.8 KB

AI-Assisted Marine Ecosystem Modelling Survey

Thank you for participating in this experiment. This survey should take around 10–15 minutes.

Section 1: Participant Background

Q1. How would you describe your level of experience with ecosystem modelling?

  • < 1 year
  • 1 – 3 years
  • 3 – 10 years
  • > 10 years

Q2. Before this exercise, how often did you use Generative AI tools for your work?

  • Every day
  • Once or twice a week
  • Once or twice a month
  • Never

Section 2: AI Tool Use and Experience

Q3. How many hours were you able to spend on this exercise?
(Slider: 0–20 hours)

Q4. How many different AI tools did you attempt to use for this exercise?
(Select from 1 to 10)

Q5. For each tool, include the wrapper/IDE and the underlying LLM (e.g., Cursor with GPT-4, LMNotebook).
(Text Entry)

Q6. For each of the following tasks, rate the AI tool’s performance:
(Likert scale: Excellent → Not Attempted)

  • Model development
  • Parameterisation
  • Simulation setup
  • Diagnostic plotting
  • Documentation generation

Q7. Did the AI refuse or resist any tasks? If yes, please describe the task and the AI’s response.
(Text Entry)

Q8. Where did you have to intervene manually? Please describe any tasks where human input was necessary.
(Text Entry)

Section 3: Modelling Techniques and Processes

Q9. Please provide a brief overview of the techniques and processes you used in your modelling workflow.
(Text Entry)

Q10. What aspects of the modelling process did the AI excel at?
(Text Entry)

Q11. What aspects did the AI struggle with or fail to complete?
(Text Entry)

Q12. What insights did you gain about what works well in AI-assisted modelling?
(Text Entry)

Q13. Do you have any advice or best practices for others using AI in ecosystem modelling?
(Text Entry)

Q14. Please provide a detailed walk-through of the specific steps and strategies you used when applying AI tools in this exercise.
(Text Entry)

Section 4: Reflections on AI Use

Q15. To what extent do you agree that each of the following is a potential benefit or risk of AI-assisted modelling?
(Likert scale: Strongly Disagree → Strongly Agree)

  • Time savings
  • Improved reproducibility
  • Enhanced model complexity or scope
  • Reduced manual coding effort
  • Novel insights or emergent patterns
  • Better documentation or traceability
  • Increased accessibility for non-experts
  • Over-reliance on automation
  • Reduced transparency or interpretability
  • Misinterpretation of model outputs
  • Ethical concerns (e.g., bias, accountability)
  • Difficulty tracing errors or debugging
  • Loss of domain expertise in decision-making
  • Inappropriate generalisation or extrapolation

Q16. How did AI use affect your confidence in the model outputs?

  • Much less confident
  • Somewhat less confident
  • No change
  • Somewhat more confident
  • Much more confident

Q17. Please share any reflections on using AI in your modelling workflow, including any surprises, future integration ideas, or safeguards you think are important.
(Text Entry)

Section 0: Modelling File Upload

Q18. Which modelling framework did you attempt to translate the EwE model into?

  • Mizer
  • Other (please specify)

Q19. Please create a zip file of all of your model files and relevant AI conversations, including:

  1. All R scripts
  2. Output files in CSV format
  3. Diagnostic plots and visualizations
  4. Transcripts of AI conversations (optional)
    (File Upload)

Q20. How far were you able to get in that amount of time?
(Likert scale: No Progress → Completed)

  • Model Development and Coding
  • Parameterisation
  • Documentation
  • Calibration
  • Validation