Skip to content

Commit 0b71ca0

Browse files
div-tygdiv-tyg
authored
Merge pull request #34 from BAMresearch/33-rectify-artefacts-storage-and-modify-folder-structure-to-include-software-tool
artefacts storage & Docs for adding a benchmark
2 parents e9ecbfb + 0966b08 commit 0b71ca0

22 files changed

+389
-40
lines changed

‎.github/workflows/run-benchmark.yml‎

Lines changed: 7 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -51,21 +51,22 @@ jobs:
5151
snakemake --use-conda --force --cores 'all'
5252
snakemake --use-conda --force --cores all \
5353
--reporter metadata4ing \
54-
--report-metadata4ing-paramscript parameter_extractor.py \
55-
--report-metadata4ing-filename metadata4ing_provenance
54+
--report-metadata4ing-paramscript ../common/parameter_extractor.py \
55+
--report-metadata4ing-filename snakemake_provenance
56+
unzip snakemake_provenance -d snakemake_provenance
5657
5758
- name: run_linear-elastic-plate-with-hole-benchmarks_nextflow
5859
shell: bash -l {0}
5960
run: |
6061
cd $GITHUB_WORKSPACE/benchmarks/linear-elastic-plate-with-hole/
61-
nextflow run main.nf -params-file workflow_config.json -plugins [email protected]
62+
nextflow run main.nf -params-file workflow_config.json -c ../common/nextflow.config -plugins [email protected]
6263
6364
- name: Archive Linear Elastic plate with a hole benchmark data for snakemake
6465
uses: actions/upload-artifact@v4
6566
with:
6667
name: snakemake_results_linear-elastic-plate-with-hole
6768
path: |
68-
benchmarks/linear-elastic-plate-with-hole/metadata4ing_provenance.zip
69+
benchmarks/linear-elastic-plate-with-hole/snakemake_provenance/
6970
7071
- name: Archive Linear Elastic plate with a hole benchmark data for nextflow
7172
uses: actions/upload-artifact@v4
@@ -85,12 +86,7 @@ jobs:
8586
uses: actions/download-artifact@v4
8687
with:
8788
name: snakemake_results_linear-elastic-plate-with-hole
88-
path: ./artifact_files
89-
90-
- name: Unzip metadata4ing_provenance.zip
91-
run: |
92-
mkdir -p ./metadata4ing_provenance
93-
unzip -o ./artifact_files/metadata4ing_provenance.zip -d ./metadata4ing_provenance
89+
path: ./snakemake_provenance
9490

9591
- name: Setup Mambaforge with postprocessing env
9692
uses: conda-incubator/setup-miniconda@v3
@@ -103,7 +99,7 @@ jobs:
10399
- name: Run plotting script
104100
shell: bash -l {0}
105101
run: |
106-
python benchmarks/linear-elastic-plate-with-hole/plot_provenance.py ./metadata4ing_provenance
102+
python benchmarks/linear-elastic-plate-with-hole/plot_metrics.py ./snakemake_provenance
107103
108104
- name: Upload PDF plot as artifact
109105
uses: actions/upload-artifact@v4

benchmarks/linear-elastic-plate-with-hole/nextflow.config renamed to benchmarks/common/nextflow.config

File renamed without changes.

benchmarks/linear-elastic-plate-with-hole/parameter_extractor.py renamed to benchmarks/common/parameter_extractor.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,12 @@
44
ParameterExtractorInterface,
55
)
66

7+
"""
8+
Parses the parameter configuration files and their corresponding output files. Returns a dictionary.
9+
https://github.com/izus-fokus/snakemake-report-plugin-metadata4ing
10+
11+
"""
12+
713
class ParameterExtractor(ParameterExtractorInterface):
814
def extract_params(self, rule_name: str, file_path: str) -> dict:
915
results = {}

benchmarks/linear-elastic-plate-with-hole/summarise_results.py renamed to benchmarks/common/summarize_results.py

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,14 +7,29 @@ def create_summary(configurations: list[str],
77
solution_metrics: list[str],
88
solution_field_data: list[str],
99
benchmark: str,
10+
benchmark_uri: str,
1011
summary_json: str) -> None:
1112

13+
"""
14+
Create a summary JSON file containing simulation results and metadata.
15+
16+
Args:
17+
configurations: List of configuration names
18+
parameter_files: List of paths to parameter JSON files
19+
mesh_files: List of paths to mesh files
20+
solution_metrics: List of paths to metrics JSON files
21+
solution_field_data: List of paths to solution field data files
22+
benchmark: Name of the benchmark
23+
benchmark_uri: URI of the benchmark
24+
summary_json: Output path for the summary JSON file
25+
"""
26+
27+
1228
all_summaries = []
1329
for idx, config in enumerate(configurations):
14-
print(idx, config)
1530
summary = {}
1631
summary["benchmark"] = benchmark
17-
print(solution_metrics[idx])
32+
summary["benchmark_uri"] = benchmark_uri
1833
with open(parameter_files[idx], "r") as param_file:
1934
summary["parameters"] = json.load(param_file)
2035
summary["mesh"] = f"{config}/mesh"
@@ -38,6 +53,7 @@ def create_summary(configurations: list[str],
3853
parser.add_argument("--input_solution_metrics", nargs="+", type=str, required=True, help="Path to the metrics JSON file (input)")
3954
parser.add_argument("--input_solution_field_data", nargs="+", type=str, required=True, help="Path to the zipped solution files (input)")
4055
parser.add_argument("--input_benchmark", required=True, type=str, help="Name of the benchmark (input)")
56+
parser.add_argument("--input_benchmark_uri", required=True, type=str, help="URI of the benchmark (input)")
4157
parser.add_argument("--output_summary_json", required=True, type=str, help="Path to the summary JSON file (output)")
4258
args = parser.parse_args()
4359
create_summary(
@@ -47,5 +63,6 @@ def create_summary(configurations: list[str],
4763
args.input_solution_metrics,
4864
args.input_solution_field_data,
4965
args.input_benchmark,
66+
args.input_benchmark_uri,
5067
args.output_summary_json
5168
)

‎benchmarks/linear-elastic-plate-with-hole/Snakefile‎

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ configuration_to_parameter_file = config["configuration_to_parameter_file"]
66
configurations = config["configurations"]
77
tools = config["tools"]
88
benchmark = config["benchmark"]
9+
benchmark_uri = config["benchmark_uri"]
910

1011

1112
rule all:
@@ -41,7 +42,7 @@ rule summary:
4142
input:
4243
# the summary is performed for all configurations saved into a single file
4344
# (snakemake_results/linear-elastic-plate-with-hole/fenics/summary.json)
44-
script = "summarise_results.py",
45+
script = "../common/summarize_results.py",
4546
parameters = expand("{param}", param=[configuration_to_parameter_file[c] for c in configurations]),
4647
mesh = expand(f"{result_dir}/mesh/mesh_{{configuration}}.msh", configuration=configurations),
4748
metrics = lambda wildcards: expand(
@@ -64,5 +65,22 @@ rule summary:
6465
--input_solution_metrics {input.metrics} \
6566
--input_solution_field_data {input.solution_field_data} \
6667
--input_benchmark {benchmark} \
68+
--input_benchmark_uri {benchmark_uri} \
6769
--output_summary_json {output.summary_json}
68-
"""
70+
"""
71+
72+
"""
73+
Steps to add a new simulation tool to the workflow:
74+
75+
1. Write the tool-specific workflow, scripts, environment file and store them in the benchmarks/linear-elastic-plate-with-hole/tool_name/.
76+
2. Add the tool name to "tools" workflow_config.json (generated here using generate_config.py)
77+
78+
------------------------------------------------------------------------------------------------------------------------
79+
"rule all" defines the final target of the workflow. Knowing the final target, the snakemake determines
80+
the dependency chain automatically.
81+
82+
Wildcards in the rule definitions allow to generalize the rules for multiple configurations and tools.
83+
They act like placeholders (variables) in filenames or paths that get automatically filled in by Snakemake.
84+
85+
Information on snakemake rules: https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html
86+
"""

benchmarks/linear-elastic-plate-with-hole/plateWithHoleSolution.py renamed to benchmarks/linear-elastic-plate-with-hole/analytical_solution.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
import numpy as np
22
import sympy as sp
33

4-
class PlateWithHoleSolution:
4+
class AnalyticalSolution:
55
def __init__(self, E: float, nu: float, radius: float, L:float, load:float) -> None:
66
self.radius = radius
77
self.L = L

‎benchmarks/linear-elastic-plate-with-hole/environment_mesh.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
11
name: mesh-generation
2+
# Environment file for create_mesh.py script. Called by the main workflow.
3+
24
channels:
35
- conda-forge
46

‎benchmarks/linear-elastic-plate-with-hole/environment_postprocessing.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
11
name: postprocessing
2+
# Environment file for summarize_results.py script. Called by the main workflow.
3+
24
channels:
35
- conda-forge
46
- defaults

‎benchmarks/linear-elastic-plate-with-hole/fenics/environment_simulation.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
11
name: fenics_simulation
2+
# Environment file for fenics simulation scripts. Called by fenics tool workflow.
3+
24
channels:
35
- conda-forge
46

‎benchmarks/linear-elastic-plate-with-hole/fenics/run_fenics_simulation.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414

1515
# Add parent directory to sys.path
1616
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
17-
from plateWithHoleSolution import PlateWithHoleSolution
17+
from analytical_solution import AnalyticalSolution
1818

1919

2020
def run_fenics_simulation(
@@ -76,7 +76,7 @@ def run_fenics_simulation(
7676
.magnitude
7777
)
7878

79-
analytical_solution = PlateWithHoleSolution(
79+
analytical_solution = AnalyticalSolution(
8080
E=E,
8181
nu=nu,
8282
radius=radius,

0 commit comments

Comments
 (0)