@@ -5,8 +5,13 @@ A structure for managing parameter studies with multiple peridynamic simulations
55`Study` type coordinates the execution of multiple simulation jobs with different parameter
66configurations, tracks their status, and logs all results to a central logfile.
77
8+ If a logfile already exists at the study root (from a previous interrupted run), the
9+ constructor automatically reads it and initializes the `sim_success` field based on the
10+ recorded completion status. This enables seamless resumption of interrupted studies.
11+
812# Arguments
9- - `jobcreator::Function`: A function with signature `jobcreator(setup::NamedTuple, root::String)`
13+ - `jobcreator::Function`: A function with signature
14+ `jobcreator(setup::NamedTuple, root::String)`
1015 that creates and returns a [`Job`](@ref) object. The function receives:
1116 - `setup`: A `NamedTuple` containing the parameters for one simulation
1217 - `root`: The root directory path for the study (to construct individual job paths)
@@ -137,6 +142,18 @@ This function creates the study root directory and logfile, then submits each jo
137142respective job directories, while a central study logfile tracks the overall progress and
138143status of all simulations.
139144
145+ ## Resuming Interrupted Runs
146+
147+ If the study logfile already exists (from a previous run), `submit!` will:
148+ - Read the logfile and refresh `study.sim_success` to detect previously completed jobs
149+ - Skip jobs that already completed successfully (marked with "skipped" in the logfile)
150+ - Only execute jobs that failed or were not yet started
151+ - Append a "RESUMED" marker with timestamp to the logfile
152+
153+ This allows you to safely resume a study after an interruption (crash, timeout, manual
154+ stop) without re-running successful simulations. Simply call `submit!(study)` again with
155+ the same study configuration.
156+
140157# Arguments
141158- `study::Study`: The study containing all jobs to execute
142159
@@ -160,6 +177,11 @@ status of all simulations.
160177study = Study(create_job, setups; root="my_study")
161178submit!(study; quiet=true)
162179
180+ # If the process was interrupted, simply call submit! again to resume:
181+ # It will skip completed jobs and only run remaining ones
182+ study = Study(create_job, setups; root="my_study") # Reads existing logfile
183+ submit!(study; quiet=true) # Resumes from where it left off
184+
163185# Check which simulations succeeded
164186successful_indices = findall(study.sim_success)
165187println("Successful simulations: ", successful_indices)
@@ -231,9 +253,9 @@ function submit!(study::Study; kwargs...)
231253 study. sim_success[i] = success
232254 open (study. logfile, " a" ) do io
233255 if success
234- msg * = @sprintf (" status: completed ✓ (%.2f seconds)\n\n " , simtime)
256+ msg = @sprintf (" status: completed ✓ (%.2f seconds)\n\n " , simtime)
235257 else
236- msg * = " status: failed ✗\n\n "
258+ msg = " status: failed ✗\n\n "
237259 end
238260 write (io, msg)
239261 end
@@ -308,6 +330,11 @@ This function iterates through all jobs in the study and applies the user-define
308330processing function `f` to jobs that completed successfully. Failed jobs or jobs where
309331the processing function errors will use the `default_result` instead.
310332
333+ Before processing, this function automatically refreshes `study.sim_success` from the
334+ logfile if it exists. This ensures that jobs completed in a previous interrupted run
335+ are properly detected and processed, even if the current Julia session doesn't reflect
336+ their completion status yet.
337+
311338# Arguments
312339- `f::Function`: A processing function with signature `f(job::Job, setup::NamedTuple)`
313340 that returns a `NamedTuple` containing the processed results. The function receives:
@@ -323,17 +350,19 @@ the processing function errors will use the `default_result` instead.
323350 `default_result` for failed/errored cases.
324351
325352# Behavior
353+ - Refreshes job completion status from logfile (if exists) before processing
326354- Only processes jobs where `study.sim_success[i] == true`
327355- Failed jobs automatically receive `default_result` (warning logged)
328- - If processing function `f` throws an error, that job receives `default_result` (error logged)
356+ - If processing function `f` throws an error, that job receives `default_result` (error
357+ logged)
329358- Processing is sequential (one job at a time)
330359- All jobs in the study will have a corresponding entry in the results vector
331360
332361# Example
333362```julia
334- # After running a parameter study
363+ # After running a parameter study (possibly interrupted and resumed)
335364study = Study(create_job, setups; root="my_study")
336- submit!(study)
365+ # Status is automatically refreshed from logfile in constructor
337366
338367# Define a function to extract maximum displacement from results
339368function extract_max_displacement(job::Job, setup::NamedTuple)
@@ -343,6 +372,7 @@ function extract_max_displacement(job::Job, setup::NamedTuple)
343372end
344373
345374default = (; E=0.0, velocity=0.0, max_displacement=NaN)
375+ # This will process all successfully completed jobs, even if from a previous run
346376results = process_each_job(extract_max_displacement, study, default)
347377
348378# Filter successful results
0 commit comments