-
Notifications
You must be signed in to change notification settings - Fork 0
7. Data management
- ""
- ""
- ""
FAIR data principles (https://www.go-fair.org/fair-principles/)
Findable: Metadata and data should be easy to find for both humans and computers.
Accessible: Once a user finds the required data, she/he/they need to know how they can be accessed.
Interoperable: The data usually need to be integrated with other data. In addition, the data need to interoperate with applications or workflows for analysis, storage, and processing.
Reusable: Metadata and data should be well-described so that they can be replicated and/or combined in different settings.
https://faircookbook.elixir-europe.org/content/home.html
"The principles emphasise machine-actionability (i.e., the capacity of computational systems to find, access, interoperate, and reuse data with none or minimal human intervention) because humans increasingly rely on computational support to deal with data as a result of the increase in volume, complexity, and creation speed of data."
DMPs
https://docs.google.com/document/d/1d27-q1Tozdh1MDoEl6uMKD0uH2v_fz7__RHoPwx61Ik/edit?usp=sharing
- Induction
- HPC Best practice
- Job Arrays - RC documentation
- Methods to Improve I/O Performance - RC documentation
- Customising your bash profile for ease and efficiency
- Customise bash profile: Logging Your Command History Automatically
- Using the ei-gpu partition on the Earlham Institute computing cluster
- Using the GPUs at EI
- HPC Job Summary Tool
- EI Cloud (CyVerse)
- Git and GitHub
- Worked examples
- Job Arrays
- Using Parabricks on the GPUs
- dependencies
- Software installations
- Workflow management system
- Transfers
- Local (mounting HPC storage)
- Remote - <1gb (ood)
- Remote - <50gb (nbi drop off)
- Remote - No limit (globus)
- mv command