Merge pull request #284 from coding-for-reproducible-research/iss283_fix_parallel_computing

Fix issues with parallel computing course

Author: liamjberrisford

_toc.yml

@@ -258,7 +258,7 @@ parts:
                 sections:
                   - file: individual_modules/parallel_computing/architecture_and_concurrency
                   - file: individual_modules/parallel_computing/multithreading_io
-                  - file: individual_modules/parallel_computing/multiprocessing_fractal
+                  - file: individual_modules/parallel_computing/multiprocessing_cpu
                   - file: individual_modules/parallel_computing/mpi_hello_world
                   - file: individual_modules/parallel_computing/mpi_simple_communication
                   - file: individual_modules/parallel_computing/mpi_collective_comms

individual_modules/parallel_computing/complete_files/cpu_bound_complete.py

@@ -0,0 +1,15 @@
+import time
+
+def main():
+    start_time = time.perf_counter()
+    for _ in range(100):
+        fibonacci(30)
+    run_time = time.perf_counter() - start_time
+    print(f"Run time was {run_time} seconds")
+
+def fibonacci(n):
+    return n if n < 2 else fibonacci(n - 1) + fibonacci(n - 2)
+
+if __name__ == "__main__":
+    main()
+

individual_modules/parallel_computing/complete_files/multiprocessing_cpu_bound_complete.py

@@ -0,0 +1,16 @@
+import time
+from concurrent.futures import ProcessPoolExecutor
+
+def main():
+    start_time = time.perf_counter()
+    with ProcessPoolExecutor(max_workers=5) as executor:
+        executor.map(fibonacci, [30]*100)
+    run_time = time.perf_counter() - start_time
+    print(f"Run time was {run_time} seconds")
+
+def fibonacci(n):
+    return n if n < 2 else fibonacci(n - 1) + fibonacci(n - 2)
+
+if __name__ == "__main__":
+    main()
+

individual_modules/parallel_computing/complete_files/multiprocessing_fractal_complete.py

@@ -7,9 +7,8 @@ def get_cpu_info():
         # Get CPU count
         cpu_count = psutil.cpu_count(logical=False) # Get physical cores
         cores = psutil.cpu_count(logical=True) # Get logical cores
-        cpu_usage = psutil.cpu_percent() # Get overall CPU usage
 
-        return f"CPU: {cpu_count} physical cores, {cores} logical cores\nCPU Usage: {cpu_usage}%"
+        return f"CPU: {cpu_count} physical cores, {cores} logical cores"
 
     except Exception as e:
         print(f"Error getting CPU info: {e}")
@@ -21,10 +20,8 @@ def get_ram_info():
     try:
         memory_info = psutil.virtual_memory() # Get virtual memory usage
         total_memory_gb = memory_info.total / (1024**3) # Convert to GB
-        available_memory_gb = memory_info.available / (1024**3) # Convert to GB
-        used_memory_gb = memory_info.used / (1024**3) # Convert to GB
 
-        return f"RAM: Total {total_memory_gb:.2f} GB, Available {available_memory_gb:.2f} GB, Used {used_memory_gb:.2f} GB"
+        return f"RAM: Total {total_memory_gb:.2f} GB"
 
     except Exception as e:
         print(f"Error getting RAM info: {e}")

individual_modules/parallel_computing/complete_files/resources_report_complete.py

@@ -18,9 +18,89 @@
     "  \n",
     "## MPI real world example problem\n",
     "\n",
-    "In a previous lesson we have seen *multi-processing* being used to solve the generation of the Julia set. An alternative approach is to use *message passing*.\n",
+    "The problem we will attempt to solve is constructing a fractal. This kind of problem is often known as \"embarrassingly parallel\" meaning that each element of the result has no dependency on any of the other elements, meaning that we can solve this problem in parallel without too much difficulty. \\Let's get started by creating a new script - `mpi_fractal.py`:\n",
     "\n",
-    "As mentioned earlier, this is a relatively simple problem to parallelise. If we consider running the program with multiple processes, all we need to do to divide the work is to divide the complex grid up between the processes. Thinking back to previous sections, we covered an MPI function that can achieve this - the `scatter` method of the MPI communicator.\n",
+    "### Setting up our serial problem\n",
+    "\n",
+    "Let's first think about our problem in serial - we want to construct the [Julia set](https://en.wikipedia.org/wiki/Julia_set) fractal, so we need to create a grid of complex numbers to operate over. We can create a simple function to do this:\n",
+    "\n",
+    "```python\n",
+    "# fractal.py\n",
+    "import numpy as np\n",
+    "\n",
+    "def complex_grid(extent, n_cells, grid_range):\n",
+    "    mesh_range = np.arange(-extent, extent, extent/ncells)\n",
+    "    x, y = np.meshgrid(grid_range * 1j, grid_range)\n",
+    "    z = x + y\n",
+    "\n",
+    "    return z\n",
+    "```\n",
+    "\n",
+    "Now, we can create a function that will calculate the Julia set convergence for each element in the complex grid:\n",
+    "\n",
+    "```python\n",
+    "import warnings\n",
+    "\n",
+    "...\n",
+    "\n",
+    "def julia_set(grid):\n",
+    "\n",
+    "    fractal = np.zeros(np.shape(grid))\n",
+    "\n",
+    "    # Iterate through the operation z := z**2 + c.\n",
+    "    for j in range(num_iter):\n",
+    "        grid = grid ** 2 + c\n",
+    "        # Catch the overflow warning because it's annoying\n",
+    "        with warnings.catch_warnings():\n",
+    "            warnings.simplefilter(\"ignore\")\n",
+    "            index = np.abs(grid) < np.inf\n",
+    "        fractal[index] = fractal[index] + 1\n",
+    "\n",
+    "    return fractal\n",
+    "```\n",
+    "\n",
+    "This function calculates how many iterations it takes for each element in the complex grid to reach infinity (if ever) when operated on with the equation `x = x**2 + c`. The function itself is not the focus of this exercise as much as it is a way to make the computer perform some work! Let's use these functions to set up our problem in serial, without any parallelism:\n",
+    "\n",
+    "```python\n",
+    "\n",
+    "...\n",
+    "\n",
+    "c = -0.8 - 0.22 * 1j\n",
+    "extent = 2\n",
+    "cells = 2000\n",
+    "\n",
+    "grid = complex_grid(extent, cells)\n",
+    "fractal = julia_set(grid, 80, c)\n",
+    "```\n",
+    "\n",
+    "If we run the python script (`python fractal.py`) it takes a few seconds to complete (this will vary depending on your machine), so we can already see that we are making our computer work reasonably hard with just a few lines of code. If we use the `time` command we can get a simple overview of how much time and resource are being used:\n",
+    "\n",
+    "```\n",
+    "$ time python parallel_fractal_complete.py\n",
+    "python parallel_fractal_complete.py  5.96s user 3.37s system 123% cpu 7.558 total\n",
+    "```\n",
+    "\n",
+    "\n",
+    "\n",
+    "```{note}\n",
+    " We can also visualise the Julia set with the code snippet:\n",
+    "`\n",
+    "import matplotlib.pyplot as plt\n",
+    "\n",
+    "...\n",
+    "\n",
+    "plt.imshow(fractal, extent=[-extent, extent, -extent, extent], aspect='equal')\n",
+    "plt.show()\n",
+    "`\n",
+    "but doing so will impact the numbers returned when we time our function, so it's important to remember this before trying to measure how long the function takes.\n",
+    "```\n",
+    "\n",
+    "### Download Complete Serial File \n",
+    "[Download complete serial fractal example file](complete_files/fractal_complete.py)\n",
+    "\n",
+    "### Parallelising our serial problem\n",
+    "\n",
+    "Next we are going to sovle the Julia set problem in parallel using *message passing*. As mentioned earlier, this is a relatively simple problem to parallelise. If we consider running the program with multiple processes, all we need to do to divide the work is to divide the complex grid up between the processes. Thinking back to previous sections, we covered an MPI function that can achieve this - the `scatter` method of the MPI communicator.\n",
     "\n",
     "We can directly take the example from the previous chapter and apply it to the complex mesh creation function:\n",
     "\n",

individual_modules/parallel_computing/mpi_parallel_fractal.ipynb

@@ -0,0 +1,76 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "0d031ccb-54d9-44a8-8282-63064ec52ba0",
+   "metadata": {},
+   "source": [
+    "# Python Multiprocessing\n",
+    "\n",
+    "## Learning Objectives\n",
+    "\n",
+    "By the end of this lesson, learners will be able to:\n",
+    "\n",
+    "- Use Python's `multiprocessing` library to parallelize a CPU bound problem.\n",
+    "- Set up a pool of workers and delegate tasks to different processes to run concurrently, using the `ProcessPoolExecutor` class.\n",
+    "\n",
+    "\n",
+    "## IO bound example with Python multithreading\n",
+    "\n",
+    "In this simple example we will create an expensive CPU bound recursive function that generates the n-th Fibonacci number several times, and report the amount of time it took to run for benchmarking purposes. Note there are much more efficient ways to implement this function, but this expensive impplimentation is deliberate. First we will do it serially, and then use the multiprocessing library to delegate blocks of webpages to different threads. This problem is CPU bound, as the time it takes for process to complete is dependent on the speed of the CPU."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b45c5b07-4d05-4705-93fe-fd841171e4cc",
+   "metadata": {},
+   "source": [
+    "### The serial example\n",
+    "\n",
+    "[Download complete serial cpu bound example file](complete_files/cpu_bound_complete.py)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "56a0bb9c",
+   "metadata": {},
+   "source": [
+    "### The multithreading example\n",
+    "\n",
+    "[Download complete threaded cpu_bound example file](complete_files/multithreading_cpu_bound_complete.py)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b2758e5a-7cc5-43e9-a417-78cc9dabc07c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from jupyterquiz import display_quiz\n",
+    "display_quiz(\"questions/summary_multithreading.json\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.13.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}