Skip to content
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
ecbaf64
Merge branch 'fix/remove-cpu-partition-jobs' of github.com:UnityHPC/d…
MisterArdavan Aug 5, 2025
371198c
Add method to get memory of node from node_info.json alongside testing
MisterArdavan Aug 7, 2025
f35250a
Replace string with enum
MisterArdavan Aug 7, 2025
c68f1d4
Implement calculations for memory hoarding metrics including total ra…
MisterArdavan Aug 12, 2025
6588452
Add tests for new methods in remote_config.py and refactor test direc…
MisterArdavan Aug 12, 2025
ad6880d
Fix ANN errors in test_remote_config.py
MisterArdavan Aug 12, 2025
ef1b3ba
Add metric for sorting jobs based on RAM hoarding. Visualize the metr…
MisterArdavan Aug 12, 2025
7664948
Add metrics for sorting users hoarding too many CPU cores by compari…
MisterArdavan Aug 12, 2025
43f8802
Add static and dynamic type checking for enum types passed to Efficie…
MisterArdavan Aug 12, 2025
b702092
Refactor ResourceHoardingDataFrameNameEnum
MisterArdavan Aug 12, 2025
22abd89
Add tests for new MetricsDataFrameName enums
MisterArdavan Aug 12, 2025
3ae4aa2
Add metrics for the users DataFrame
MisterArdavan Aug 13, 2025
6242b23
Polish ResourceHoarding.ipynb
MisterArdavan Aug 13, 2025
50ac674
Merge branch 'main' of github.com:UnityHPC/ds4cg-job-analytics into f…
MisterArdavan Aug 13, 2025
1652079
Resolve merge conflict
MisterArdavan Aug 17, 2025
0d20681
Refactor preprocessing code
MisterArdavan Aug 17, 2025
de1c659
Refactor preprocessing error
MisterArdavan Aug 17, 2025
61d7257
Resolved merge conflicts.
Sep 3, 2025
829025c
Fix formatting errors.
Sep 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 14 additions & 9 deletions notebooks/Efficiency Analysis.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -90,18 +90,17 @@
"metadata": {},
"outputs": [],
"source": [
"# Automatically reload modules before executing code (set this up BEFORE imports)\n",
"%load_ext autoreload\n",
"%autoreload 2\n",
"\n",
"# Add project root to sys.path for module imports\n",
"if project_root not in sys.path:\n",
" sys.path.insert(0, project_root)\n",
"\n",
"from src.analysis import efficiency_analysis as ea\n",
"from src.visualization import JobsWithMetricsVisualizer, UsersWithMetricsVisualizer\n",
"\n",
"# Automatically reload modules before executing code\n",
"# This is useful for development to see changes without restarting the kernel.\n",
"%load_ext autoreload\n",
"# Reload all modules imported with %aimport every time before executing the Python code typed.\n",
"%autoreload 2"
"from src.config.enum_constants import MetricsDataFrameNameEnum, ResourceHoardingDataFrameNameEnum"
]
},
{
Expand Down Expand Up @@ -135,7 +134,9 @@
"metadata": {},
"outputs": [],
"source": [
"efficiency_analysis = ea.EfficiencyAnalysis(jobs_df=preprocessed_jobs_df)"
"efficiency_analysis = ea.EfficiencyAnalysis(\n",
" jobs_df=preprocessed_jobs_df, metrics_df_name_enum=ResourceHoardingDataFrameNameEnum\n",
")"
]
},
{
Expand Down Expand Up @@ -215,7 +216,7 @@
"outputs": [],
"source": [
"inefficient_jobs_vram_hours = efficiency_analysis.sort_and_filter_records_with_metrics(\n",
" metrics_df_name_enum=ea.MetricsDataFrameNameEnum.JOBS,\n",
" metrics_df_name_enum=MetricsDataFrameNameEnum.JOBS,\n",
" sorting_key=\"vram_hours\",\n",
" ascending=False, # Sort by vram_hours in descending order\n",
" filter_criteria={\n",
Expand Down Expand Up @@ -608,7 +609,11 @@
]
}
],
"metadata": {},
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Loading