|
| 1 | +=============================================================== |
| 2 | +February 2025 / Intro to Triton and HPC / HPC Winter Kickstart |
| 3 | +=============================================================== |
| 4 | + |
| 5 | +.. admonition:: Quick links |
| 6 | + :class: important |
| 7 | + |
| 8 | + * News and important links (20/02/2025): |
| 9 | + |
| 10 | + * Link to register: https://link.webropol.com/ep/hpcwinter25 |
| 11 | + * See the time-line below |
| 12 | + * Livestream (morning): https://twitch.tv/coderefinery |
| 13 | + * Exercises (afternoon): *Link sent to registered participants* |
| 14 | + * Notes document: *Link sent to registered participants* |
| 15 | + |
| 16 | + |
| 17 | + |
| 18 | +For winter 2025, we are running an intensive version (one-day-only) of our **Kickstart course**. |
| 19 | +The longer version of the course will be held on the first week of June and will last for 3 half days |
| 20 | + |
| 21 | +Learning goals |
| 22 | +-------------- |
| 23 | + |
| 24 | +* Learn the basics of High Performance Computing with slurm |
| 25 | +* Watch a step-by-step example of the typical data analysis workflow with Aalto Triton HPC cluster |
| 26 | +* Engage with hands-on exercises to make sure you are able to run your analysis on Triton |
| 27 | + |
| 28 | +This course is part of :doc:`Scientific Computing in Practice <index>` lecture series |
| 29 | +at Aalto University, supported by many others outside Aalto, and offered to others as part of `CodeRefinery <https://coderefinery.org>`__. |
| 30 | + |
| 31 | + |
| 32 | + |
| 33 | +Practical information |
| 34 | +--------------------- |
| 35 | + |
| 36 | +The course happens on Wed 26 February 2025 and is divided in two parts: |
| 37 | + |
| 38 | +**Morning lectures** (9:45 - 12:00 EET): This is the **livestream demo** part of the course. Everyone may attend the **livestream** at |
| 39 | +https://twitch.tv/coderefinery, no registration needed. This is done so that we get a higher quality recording without any personal data from course participants |
| 40 | + |
| 41 | +**Lunch** (12:00 - 13:00 EET): Lunch on your own |
| 42 | + |
| 43 | +**Afternoon hands-on session** (13:00 - 16:00): This is the practical part of the course. We will be connected to the same zoom room and do the exercises together. |
| 44 | + |
| 45 | +**Cost:** Free! |
| 46 | + |
| 47 | +**Language:** English |
| 48 | + |
| 49 | +**Additional course info at:** scip@aalto.fi |
| 50 | + |
| 51 | + |
| 52 | + |
| 53 | + |
| 54 | +Schedule |
| 55 | +-------- |
| 56 | + |
| 57 | +**All times are EEST (Europe/Helsinki time)!** |
| 58 | + |
| 59 | +The daily schedule will be adjusted based on the audience's questions. |
| 60 | +There will be frequent breaks and continuous questions time going on, |
| 61 | +this is the mass equivalent of an informal help session to get you |
| 62 | +started with the computing resources. |
| 63 | + |
| 64 | + |
| 65 | +.. admonition:: Subject to change |
| 66 | + |
| 67 | + Schedule may still have minor updates as it happens. |
| 68 | + |
| 69 | + * 09:45--10:00: Joining time/icebreaker |
| 70 | + |
| 71 | + * **10:00--10:15 Introduction, about the course** *Enrico Glerean and |
| 72 | + other staff* Materials: :doc:`../../training/kickstart/intro` |
| 73 | + |
| 74 | + * **10:15--11:45: A day in the life of an HPC user** *Richard Darst and Simo Tuomisto* |
| 75 | + |
| 76 | + - 1. Check your connection to the cluster [[ref.](https://scicomp.aalto.fi/triton/tut/connecting/)] |
| 77 | + - 2. Moving data to the cluster [[ref.](https://scicomp.aalto.fi/triton/tut/storage/)] |
| 78 | + - 3. Datasets/projects that could be used for the demo: |
| 79 | + - ["Ngrams example"](https://github.com/AaltoSciComp/hpc-examples/tree/master/ngrams) |
| 80 | + - 4. Loading an application? Not needed but good to remeber (e.g. python env) [[ref.]( |
| 81 | +https://scicomp.aalto.fi/triton/tut/modules/)] |
| 82 | + - 5. Analysing data with slurm [[ref.](https://scicomp.aalto.fi/triton/#running-calculations)] |
| 83 | + - 5.1 Getting an interactive session [[ref.](https://scicomp.aalto.fi/triton/tut/interactive/)] |
| 84 | + - 5.2 Non-interactive serial job [[ref.](https://scicomp.aalto.fi/triton/tut/interactive/)] |
| 85 | + - 5.3 Parralelisation and array jobs [[ref1](https://scicomp.aalto.fi/triton/tut/parallel/), [ref2](https://scicomp.aalto.fi/triton/tut/array/)] |
| 86 | + - 5.4 More advanced parallelisation - discussion only - ([multithreading/multiprocessing](https://scicomp.aalto.fi/triton/tut/parallel-shared/), [MPI](https://scicomp.aalto.fi/triton/tut/parallel-mpi/), [GPUs](https://scicomp.aalto.fi/triton/tut/gpu/)) |
| 87 | + - 6. Visualising the results (e.g OOD) |
| 88 | + - 7. Moving the data away from the cluster |
| 89 | + |
| 90 | + |
| 91 | + |
| 92 | + * **11:45--12:00: Where to go from here and how to ask for help (Susanne Merz and Enrico Glerean) |
| 93 | +
|
| 94 | + * **12:00--13:00: Lunch break (on your own)** |
| 95 | + |
| 96 | + * **13:00--16:00: Hands-on exercises with Triton HPC cluster |
| 97 | +
|
| 98 | + - Main room: Lobby and Generic questions (SM) |
| 99 | + - Room 1: Hands-on with exercises from the morning, Slurm and Triton basics (RD) |
| 100 | + - Room 2: Connecting questions? (TP) |
| 101 | + - Room 3: GPUs (ST) |
| 102 | + - Room 4: Advanced parallelization |
| 103 | + - Room 5: AI / LLMs (YT) |
| 104 | + - Room 6: Speech2Text (TR) |
| 105 | + |
| 106 | +Preparation |
| 107 | +----------- |
| 108 | + |
| 109 | +We strongly recommend you are familiar with the Linux command line. |
| 110 | +Browsing the following material is sufficient: |
| 111 | + |
| 112 | +* Command line/shell basics [[ref](https://scicomp.aalto.fi/triton/tut/cluster-shell/)] [[video](https://youtu.be/bJMmz5-svJo?t=7&list=PLZLVmS9rf3nMKR2jMglaN4su3ojWtWMVw&index=8)] |
| 113 | + - **Important background knowledge which we won't go over again.** |
| 114 | + |
| 115 | + - A more detailed version of the above, for those who automate a lot of analysis, is [Basic Linux shell and scripting](https://www.youtube.com/watch?v=ESXLbtaxpdI&list=PLZLVmS9rf3nN_tMPgqoUQac9bTjZw8JYc&index=3) |
| 116 | + - Or read/watch the shorter :doc:`crash course |
| 117 | + </scicomp/shell>` / `video <https://youtu.be/56p6xX0aToI>`__. |
| 118 | + |
| 119 | + |
| 120 | +* Watch `this background info about why we use computer clusters <https://www.youtube.com/playlist?list=PLZLVmS9rf3nNDHRo1Baz_JVQWDI0mTYyB>`__. This is important information for *why* we are in this course, which we *won't cover directly*. The most important videos are the `intro (what is a cluster and why?) <https://www.youtube.com/watch?v=yqGtnA7CUtU&list=PLZLVmS9rf3nNDHRo1Baz_JVQWDI0mTYyB&index=1&pp=gAQBiAQB>`, `storage hierarchy (how the data looks) <https://www.youtube.com/watch?v=JAR9xyy5rcE&list=PLZLVmS9rf3nNDHRo1Baz_JVQWDI0mTYyB&index=2&pp=gAQBiAQB>`, and `the Slurm job scheduler (how the cluster runs things) <https://www.youtube.com/watch?v=Y73A7lXISxU&list=PLZLVmS9rf3nNDHRo1Baz_JVQWDI0mTYyB&index=5&pp=gAQBiAQB>`. |
| 121 | + |
| 122 | + |
| 123 | + |
| 124 | +**How to attend:** Online workshops can be a productive format, but it |
| 125 | +takes some effort to get ready. Browse these resources: |
| 126 | + |
| 127 | +* `Attending a livestream workshop |
| 128 | + <https://coderefinery.github.io/manuals/how-to-attend-stream/>`__, |
| 129 | + good to read in detail (ignore the CodeRefinery-specific parts). |
| 130 | +* `How to use HackMD to take answer questions and hold discussions <https://coderefinery.github.io/manuals/hackmd-mechanics/>`__. |
| 131 | + |
| 132 | + |
| 133 | +Technical prerequisites |
| 134 | +----------------------- |
| 135 | + |
| 136 | +**Software installation** |
| 137 | + |
| 138 | +* SSH client to connect to the cluster (+ be able to connect, see next |
| 139 | + point) |
| 140 | +* `Zoom <https://coderefinery.github.io/installation/zoom/>`__ (if |
| 141 | + attending breakout rooms) |
| 142 | + |
| 143 | + |
| 144 | +**Cluster account and connection verification:** |
| 145 | + |
| 146 | +* Access to your computer cluster. |
| 147 | + |
| 148 | + * Aalto: if you do not yet have access to Triton, :doc:`request an account |
| 149 | + </triton/accounts>` in advance. |
| 150 | + |
| 151 | +* Then, connect and get it working |
| 152 | + |
| 153 | + * Aalto (and possibly useful to others): try to :doc:`connect to |
| 154 | + Triton </triton/tut/connecting>` to be ready. Come to the |
| 155 | + Wednesday session for help connecting (required). |
| 156 | + |
| 157 | + |
| 158 | + |
| 159 | +Next steps / follow-up courses |
| 160 | +------------------------------ |
| 161 | + |
| 162 | +Keep the :doc:`Triton quick reference </triton/ref/index>` close (or |
| 163 | +equivalent for your cluster), or print `this cheatsheet |
| 164 | +<https://aaltoscicomp.github.io/cheatsheets/triton-cheatsheet.pdf>`__ |
| 165 | +if that's your thing. |
| 166 | + |
| 167 | +Each year the first day has varying topics presented. We don't repeat |
| 168 | +these every year, but we strongly recommend that you watch some of |
| 169 | +these videos yourself as preparation. |
| 170 | + |
| 171 | +Very strongly recommended: |
| 172 | + |
| 173 | +* `Installing Python packages with Conda |
| 174 | + <https://youtu.be/dmTlNh3MWx8>`__ (Note that conda on new-Triton has changed. See :doc:`/triton/apps/python-conda` for details) |
| 175 | +* `Git intro |
| 176 | + <https://www.youtube.com/watch?v=r9AT7MqmLrU&list=PLZLVmS9rf3nOaNzGrzPwLtkvFLu35kVF4&index=5>`__ (useful) |
| 177 | + |
| 178 | +Other useful material in previous versions of this course: |
| 179 | + |
| 180 | +* Scientific Computing workflows at Aalto - concepts apply to other |
| 181 | + sites, too (optional): `lecture notes |
| 182 | + <https://hackmd.io/@AaltoSciComp/SciCompIntro>`__ and `video |
| 183 | + <https://www.youtube.com/watch?v=Oz37XAzWFhk>`__, :doc:`reference |
| 184 | + material </triton/usage/workflows>`. |
| 185 | +* Tools of scientific computing (optional): `lecture notes |
| 186 | + <https://hackmd.io/@AaltoSciComp/ToolsOfScientificComputing>`__ and |
| 187 | + `video <https://www.youtube.com/watch?v=kXYfxXEb0Go>`__ |
| 188 | + |
| 189 | +While not an official part of this course, we suggest these videos |
| 190 | +(co-produced by our staff) as a follow-up perspective: |
| 191 | + |
| 192 | +* Attend a `CodeRefinery workshop <https://coderefinery.org>`__, |
| 193 | + which teaches more useful tools for scientific software |
| 194 | + development. |
| 195 | +* Look at `Hands-on Scientific Computing |
| 196 | + <https://hands-on.coderefinery.org>`__ for an online course to |
| 197 | + either browse or take for credits. |
| 198 | +* `Cluster Etiquette (in Research Software Hour) |
| 199 | + <https://www.youtube.com/watch?v=NIW9mqDwnJE&list=PLpLblYHCzJAB6blBBa0O2BEYadVZV3JYf>`__: |
| 200 | + The Summer Kickstart teaches what you *can* do from this course, |
| 201 | + but what *should* you do to be a good user. |
| 202 | +* `How to tame the cluster (in Research Software Hour) |
| 203 | + <https://www.youtube.com/watch?v=5HN9-MW7Tw8&list=PLpLblYHCzJAB6blBBa0O2BEYadVZV3JYf>`__. |
| 204 | + This mostly repeats the contents of this course, with a bit more |
| 205 | + discussion, and working one example from start to parallel. |
| 206 | + |
| 207 | + |
| 208 | + |
| 209 | +Community standards |
| 210 | +------------------- |
| 211 | + |
| 212 | +We hope to make a good learning environment for everyone, and expect |
| 213 | +everyone to do their part for this. If there is anything we can do to |
| 214 | +support that, let us know. |
| 215 | + |
| 216 | +If there is anything wrong, *tell us right away* - if you need to |
| 217 | +contact us privately, you can message the host on Zoom or |
| 218 | +:doc:`contact us outside the course </help/index>`. This could be as |
| 219 | +simple as "speak louder / text on screen is unreadable / go slower" or |
| 220 | +as complex as "someone is distracting our group by discussing too |
| 221 | +advanced things". |
| 222 | + |
| 223 | + |
| 224 | + |
| 225 | +Material |
| 226 | +-------- |
| 227 | + |
| 228 | +See the schedule |
0 commit comments