Clear runner state of previous executions before warm starts (#1905)

* attach all run controller logs to the run

* make controller-level pre-suspend wait duration configurable

* snapshot status should remain EXECUTING for short retry delays

* add suspended process error

* ensure clean slate before waiting for next run

* treat immediate retries as warm starts

* fix for finished runs waiting forever

Author: nicktrn