Skip to content

Meeting Notes April 2020

Jump to bottom
Tim Randles edited this page Apr 13, 2020 · 13 revisions

Monday, April 13, 2020

Attendees - Pat Grubel (pagrubel), Al McPherson (mcpherson), Tim Randles (trandles-lanl), Jake Tronge (jtronge)

Agenda

PR Review

  1. NONE

Issue Review

  1. NONE

Discussion

  1. Thoughts on FY21 cloud milestone
    • using ORNL or Chameleon cloud for target platform

Around the room

  • Pat
    • working to get pyslurm tests running
    • using DockerRequirement from CWL
  • Al
    • getting on darwin and fog
  • Qiang & Jake
    • got examples running that were in milestone documentation
    • Jake will document his scripts and dockerfiles for setting up their test environment
    • Jake will get things running on group server
    • Qiang to send thoughts on FY21 cloud milestone
  • Tim
    • continue working on BEEStart script

Wednesday, April 8, 2020

Attendees - Rusty Davis (rstyd), Pat Grubel (pagrubel), Al McPherson (mcpherson), Tim Randles (trandles-lanl), Jake Tronge (jtronge)

Agenda

  1. discuss CWL and container support
  2. TaskManager design for modular support of container runtimes and resource managers
  3. discuss proposed FY21 ECP P6 Activites
    • BEE- FY21 P6-1 Develop the ability to archive, clone, and re-run workflows (start 10/01/20, due 3/31/21)
    • BEE- FY21 P6-2 Run BEE jobs on private cloud infrastructure (due 9/31/21)

PR Review

  1. NONE

Issue Review

  1. NONE

Discussion

  1. APPROVED April 6, 2020 meeting notes
  2. TaskManager discussion mostly shelved for now, revisit next week
  3. CWL support for containers
  4. FY21 ECP Activities are documented at
    • Tim starting on design document for the activities

Around the room

  • Jake
    • neo4j issues (Task already exists)
    • Rusty knows how to fix itj
    • close to being able to run test workflows
  • Rusty
    • starting test work
    • looking at PyTest for integration testing
    • maybe pexpect for client testing
    • Flask has some testing framework (Jake)
    • BEE should start a document of what CWL is supported by project
  • Pat
    • question for Rusty about passing Task object to worker from TaskManager
    • will need to think about how to pass things around when there's more data (requirements and hints)
  • Al
    • refactoring database and building new API to it
      • no way to version python APIs
      • API changes only affect WorkflowManager
    • next use case CWL example
      • maybe BLAST workflow again
      • keep scope of parsing to HPC use cases, not "generic everything CWL"
      • Do srudin VASP workflow (parameter study) #66

Action Items

  1. Tim - get VASP containers that work with Charliecloud (Power9, x86_64)

Monday, April 6, 2020

Attendees - Rusty Davis (rstyd), Pat Grubel (pagrubel), Qiang Guan (guanxyz), Tim Randles (trandles-lanl), Jake Tronge (jtronge)

PR Review

  1. #138 APPROVED (trandles-lanl) - Use bee.conf to configure listen ports for BEEWorkflowManager and BEETaskManager
    • Pat approves of merging this PR, but into master instead of develop. The rationale is the functionality is simple and enables everyone to do development work at the same time on the same system.

Issue Review

  1. #137 (pagrubel) - Slurm worker to properly check DockerRequirment

Discussion

  1. extending CWL for other container runtimes (rstyd)
    • discuss on Wednesday
    • guanxyz had some ideas
  2. next ECP milestones up on wiki

Around the room

  • Jake
    • got a test environment set up at KSU
    • initial problems with PySlurm due to having a too-new Slurm installed
  • Rusty
    • working on unittest and CI tests for client/WorkflowManager
    • not a lot of time for BEE this week (very understandable, everyone prioritized BEE the past 2 week (trandles-lanl))
  • Pat
    • unittest for TaskManager
    • issue #137 above
    • not much time for BEE this week
  • Tim
    • issue #139 planning to discuss on Wednesday
    • ECP milestone housekeeping

Clone this wiki locally