README.md

HDR-ML-Hackathon-Virtual-2026

This repo documents resources for the HDR ML Hackathon 2026 event at CU Boulder

REGISTER HERE

Learn more about the NSF HDR ML Challenge here and prize info here.

Event Info

• When: Thursday, Feb 12, 1000-1400 MST
• Where: Virtual, Zoom link https://cuboulder.zoom.us/j/94963837906

This event is being organized by:

• The National Ecological Observatory Network (NEON), Battelle
• The Environmental Data Science Innovation & Impact Lab (ESIIL), CU Boulder
• The Imageomics Institute
• CyVerse, University of Arizona

Resources

• Shared google drive https://drive.google.com/drive/folders/1goGOlKTISdQmjlCtUiSvp1Plmau4zArh?usp=sharing
• Imageomics GitHub repo for example beetle challenge code: https://github.com/Imageomics/HDR-SMood-Challenge-sample
• Our fork of the above repo with GitHub actions to run code on a JS2 GPU: https://github.com/jeffgillan/HDR-SMood-Challenge-sample

Agenda

• 0950-1000 - Log on, make sure audio works
• 1000-1020 - Welcome from organizers, NEON, and ESIIL
• 1020-1040 - Community Care, Introductions, and Ice breaker
• 1040-1110 - Overview of challenge themes - SLIDES
  • Imageomics - Beetles as sentinel taxa
  • A3D3 - Neurobiology
  • iHarp - Predicting coastal flooding
• 1115-1130 - Getting started with compute resources
  • CyVerse DE overview
• 1130-1200 - How to create a submission
  • Overview of Codabench pages
    • Beetles as Sentinel Taxa (Imageomics): https://www.codabench.org/competitions/9854/
    • Predicting Coastal Flooding (iHARP): https://www.codabench.org/competitions/10801/
    • Neural Forecasting (A3D3): https://www.codabench.org/competitions/9806/
  • Overview of baseline models/submissions
    • Beetles example: https://github.com/jeffgillan/HDR-SMood-Challenge-sample
  • How to submit - Imageomics submission example
• 1200-1215 - Break!
• 1215-1245 - Breakout discussions by theme over lunch - options for how to proceed will be up to participants:
  • Option 1 - open discussion of themes, then breakout into teams - if participants are unsure of which theme/how to approach the challenge
    • Round 1 - 10 min discussion by theme (3 rooms, random assign)
    • Round 2 - 10 min discussion by theme (3 rooms, random assign)
    • Round 3 - 10 min discussion by theme (3 rooms, random assign)
  • Option 2 - pitch for team - if there are participants with ideas and they want to get to work, but are looking for teams
  • Option 3 - get to work in breakout rooms - if participants already have teams and ideas how to proceed.
• 1245-1300 - Regroup, summarize discussions
• 1300-1345 - Breakout by theme (3 rooms, self select) Folks have the option to choose smaller breakouts during this time if they form a team
• 1345-1400 - Report outs and wrap up.

Getting started with baseline models/code

• Challenge 1: Beetles as Sentinel Taxa
• Challenge 2: Predicting Coastal Flooding
• Challenge 3: Neural Forecasting (NOTE: no GitHub repo, the link goes to the codabench page, see the "Starting Kit and Sample Submission" tab there for example code)

Computing Infrastructure

Housed at the University of Arizona, CyVerse is a one-of-a-kind cloud computing system for the academic and research communities. It's mission is to design, deploy and expand a national cyberinfrastructure for scientific research and to train scientists how to use it.

• Sign Up for Cyverse

• Regular Login

• CyVerse Learning Materials

Data Storage

Cyverse Data Store is the ideal cloud storage to host your large datasets, share data with colleagues, and meet publication/grant archival requirements.

• Pro account has a 3TB limit
• Data I/O is accessible through website GUI and command line tools such as gocommands
• Share your data with your colleagues and world with a URL
• Data can be public/private, shared with anyone, set permission levels

Sharing Folders

Shared persistent storage for this hackathon:

• Path (e.g., for GoCommands): /iplant/home/shared/esiil/HDR_ML_challenge_2025
• URL: https://de.cyverse.org/data/ds/iplant/home/shared/esiil/HDR_ML_challenge_2025?type=folder&resourceId=f506b3ce-b8db-11f0-b119-90e2ba675364

Apps Tab

Cyverse Apps are containerized software applications that include Jupyter Notebooks, Rstudio, Cloud Shell, QGIS, and VScode. These can be launched from the 'Apps' tab. When launching the apps, you can choose the size of the computing resouces you want to use (CPUs, RAM, Disk storage).

Launching ESIIL ML Challenge 2025

Apps are ephemeral. Once they are shutdown, they disappear forever along with any data or code you have on it. Make sure to move any important data over to your personal datastore.

Analysis Tab

The 'Analysis' Tab is a dashboard showing the all the running and past app launches. Here you can see how much time is remaining on the app. If you need more time, you can extend the time.

If you are finished with the app, please shut it down!! Closing browser tabs will not shutdown an app.

Cloud Reproducible Triangle

• Compute = your scratch pad: fast but disposable

• GitHub = your lab notebook: versioned, accountable, and collaborative

• Data Store = your filing cabinet: heavy, permanent, and backed up

App Command line navigation

When you first launch a Cyverse App, you are likely to be located in the directory /home/jovyan/data-store. This is a good place to have your workspace. You should clone repos here, bring data here, or put any folder or file here you are working with. However, anything you want to save SHOULD NOT BE STORED HERE! For permanent storage, put items in your personal datastore directory /home/jovyan/data-store/home/<cyverse-user-name>.

If you want to move data from workspace to permanent storage, you could type: cp file.txt ~/data-store/home/jgillan. Or you can copy and paste using the graphical file explorer.

Authenticate with Github in App

Authenticating with Github allows you to push changes back up to a repository.

In the terminal of jupyterlab, authenticate to github by typing:

gh auth login

Follow the prompts to connect the DE app with your Github account

Clone repository to App

Best practices would be to clone git repositories to the directory ~/data-store within the Cyverse App.

Clone using the Terminal:

git clone --branch <branch-name> https://github.com/jeffgillan/HDR-SMood-Challenge-sample.git

Clone using the Git Widget:

GPU Machine for Beetle Challenge

If you are working on the Beetle Challenge, you can leverage more powerful GPUs from the cloud computer Jetstream2. Go here to see how.