Merge branch 'main' into feature/a-100-analysis

Author: naryasomayaj

README.md

@@ -166,6 +166,7 @@ contains tools to add a number of useful derived columns for plotting and analys
 | UUID   | VARCHAR | Unique identifier | 
 | JobID  | INTEGER | Slurm job ID |
 | ArrayID | INTEGER | Position in job array |
+|ArrayJobID| INTEGER | Slurm job ID within array|
 | JobName |  VARCHAR | Name of job |
 | IsArray |  BOOLEAN | Indicator if job is part of an array |
 | Interactive |  VARCHAR | Indicator if job was interactive
@@ -184,10 +185,10 @@ contains tools to add a number of useful derived columns for plotting and analys
 | Partition |  VARCHAR | Job partition |
 | Nodes |  VARCHAR | Job nodes as compact string |
 | NodeList |  VARCHAR[] | List of job nodes |
-| CPUs |  SMALLINT | Number of CPUs |
+| CPUs |  SMALLINT | Number of CPU cores |
 | Memory |  INTEGER | Job allocated memory (bytes) |
 | GPUs |  SMALLINT | Number of GPUs requested |
-| GPUType |  VARCHAR[] | List of GPU types |
+| GPUType |  DICT | Dictionary with keys as type of GPU (str) and the values as number of GPUs corresponding to that type (int) |
 | GPUMemUsage |  FLOAT | GPU memory usage (bytes) |
 | GPUComputeUsage |  FLOAT | GPU compute usage (pct) |
 | CPUMemUsage |  FLOAT | GPU memory usage (bytes) |

dev-requirements.txt

@@ -1,17 +1,15 @@
+-r requirements.txt
 ruff
 pytest
+pytest-env
 mkdocs
 mkdocs-material
 mkdocstrings-python
-ipykernel
 build
 black
 mypy
 coverage
-pandas-stubs
 types-seaborn
-jupyterlab
 nbconvert
 nbdime
-jq
-pydantic
+types-requests

docs/mvp-scripts/cpu_metrics.md

@@ -0,0 +1,3 @@
+# CPU Metrics
+
+::: mvp_scripts.cpu_metrics

docs/mvp-scripts/gpu_metrics.md

@@ -0,0 +1,3 @@
+# GPU Metrics
+
+::: mvp_scripts.gpu_metrics

docs/mvp_scripts/cpu_metrics.md

@@ -7,4 +7,4 @@ email bodies with user-specific resource usage. This script will only run on Uni
 of the ```pi_bpachev_umass_edu``` group. It is included as an example of the sort of tool that 
 might be useful to the Unity team as a final deliverable of this project.
 
-::: mvp-scripts.zero_gpu_usage_list
+::: mvp_scripts.zero_gpu_usage_list

docs/mvp_scripts/gpu_metrics.md

@@ -34,7 +34,7 @@ nav:
     - 'visualization/efficiency_metrics.md'
     - 'visualization/models.md'
   - MVP Scripts:
-    - 'mvp-scripts/cpu_metrics.md'
-    - 'mvp-scripts/gpu_metrics.md'
-    - 'mvp-scripts/zero_gpu_usage.md'
+    - 'mvp_scripts/cpu_metrics.md'
+    - 'mvp_scripts/gpu_metrics.md'
+    - 'mvp_scripts/zero_gpu_usage.md'
 

docs/mvp-scripts/zero_gpu_usage.md docs/mvp_scripts/zero_gpu_usage.mddocs/mvp-scripts/zero_gpu_usage.md renamed to docs/mvp_scripts/zero_gpu_usage.md

@@ -15,7 +15,7 @@ Please feel free to use the scripts and notebook in this repository as a templat
 The following guidelines may prove helpful in maximizing the utility of this repository:
 
 - Please avoid committing code unless it is meant to be used by the rest of the team.
-- New code should first be comitted in a dedicated branch (```feature/newanalysis``` or ```bugfix/typo```), and later merged into ```main``` following a code
+- New code should first be committed in a dedicated branch (```feature/newanalysis``` or ```bugfix/typo```), and later merged into ```main``` following a code
 review.
 - Shared datasets should usually be managed with a shared folder on Unity, not committed to Git.
 - Prefer committing Python modules with plotting routines like ```gpu_metrics.py``` instead of Jupyter notebooks, when possible.