the job submission PENDING status update has this principle so that scaling strategy will see submitted jobs immediately, before a provider status refresh happens. this makes the scaling code immediately aware of what just happened, rather than for one poll period acting as if nothing had happened.

When making changes that will later be reflected in the _status table, then those changes should be immediately also be made in the cached _status table eplciitly.

before this PR, this code path does not happen in the case of a failed submission, where a failure status will appear when a refresh happens, but not before. in that case the scaling code will act as if the failed submission did not happen and will continue to submit repeatedly until a refresh happens much later.  this PR makes _status be updated in this case too.

fixes 3235

Author: benclifford

parsl/executors/status_handling.py

@@ -202,7 +202,9 @@ def scale_out_facade(self, n: int) -> List[str]:
                 block_ids.append(block_id)
 
             except Exception as ex:
-                self._simulated_status[block_id] = JobStatus(JobState.FAILED, "Failed to start block {}: {}".format(block_id, ex))
+                failed_status = JobStatus(JobState.FAILED, "Failed to start block {}: {}".format(block_id, ex))
+                self._simulated_status[block_id] = failed_status
+                self._status[block_id] = failed_status
 
         self.send_monitoring_info(monitoring_status_changes)
         return block_ids