Skip to content

Commit c14b729

Browse files
authored
Make dependency failure handling more consistent (#4004)
# Description This moves code around a little bit to make error handling in the dependency failure case more consistent with other failures. This unlocks potential for refactoring of the error handling code. Rough principles for error handling that are true for other errors and that this PR makes true for dependency failures: * `_launch_if_ready_async` creates a Future for the task, and does not modify the `TaskRecord` state or send monitoring information * `handle_exec_update` decides if a task has failed by whether the Future contains an exception or not. It updates the `TaskRecord` state and sends monitoring information. Removing one call of `_send_task_log_info` (in `_launch_if_ready_async`) and instead relying on the existing call in `handle_exec_update` removes a duplicate `dep_fail` entry in the monitoring database, that was previously unnoticed. ```console $ pytest parsl/tests/test_python_apps/test_depfail_propagation.py --config parsl/tests/configs/htex_local_alternate.py ``` Before this PR: ```console sqlite> select * from status where task_status_name = 'dep_fail'; 1|dep_fail|2025-10-16 09:29:20.171145|1d65bd94-8639-4ebf-90f9-2b61cca36763|0 1|dep_fail|2025-10-16 09:29:20.171472|1d65bd94-8639-4ebf-90f9-2b61cca36763|0 3|dep_fail|2025-10-16 09:29:20.255851|1d65bd94-8639-4ebf-90f9-2b61cca36763|0 3|dep_fail|2025-10-16 09:29:20.256067|1d65bd94-8639-4ebf-90f9-2b61cca36763|0 4|dep_fail|2025-10-16 09:29:20.256353|1d65bd94-8639-4ebf-90f9-2b61cca36763|0 4|dep_fail|2025-10-16 09:29:20.256452|1d65bd94-8639-4ebf-90f9-2b61cca36763|0 5|dep_fail|2025-10-16 09:29:20.256732|1d65bd94-8639-4ebf-90f9-2b61cca36763|0 5|dep_fail|2025-10-16 09:29:20.256856|1d65bd94-8639-4ebf-90f9-2b61cca36763|0 7|dep_fail|2025-10-16 09:29:20.335936|1d65bd94-8639-4ebf-90f9-2b61cca36763|0 7|dep_fail|2025-10-16 09:29:20.336110|1d65bd94-8639-4ebf-90f9-2b61cca36763|0 8|dep_fail|2025-10-16 09:29:20.336398|1d65bd94-8639-4ebf-90f9-2b61cca36763|0 8|dep_fail|2025-10-16 09:29:20.336522|1d65bd94-8639-4ebf-90f9-2b61cca36763|0 ``` After this PR: ```console sqlite> select * from status where task_status_name = 'dep_fail'; 1|dep_fail|2025-10-16 09:31:35.782225|be2cd195-b13b-4b03-ac2f-95b73cc9a200|0 3|dep_fail|2025-10-16 09:31:35.865129|be2cd195-b13b-4b03-ac2f-95b73cc9a200|0 4|dep_fail|2025-10-16 09:31:35.865509|be2cd195-b13b-4b03-ac2f-95b73cc9a200|0 5|dep_fail|2025-10-16 09:31:35.865873|be2cd195-b13b-4b03-ac2f-95b73cc9a200|0 7|dep_fail|2025-10-16 09:31:35.945858|be2cd195-b13b-4b03-ac2f-95b73cc9a200|0 8|dep_fail|2025-10-16 09:31:35.946355|be2cd195-b13b-4b03-ac2f-95b73cc9a200|0 ``` # Changed Behaviour monitoring database contents reduced as reported above ## Type of change - Bug fix - Code maintenance/cleanup
1 parent cb9556e commit c14b729

File tree

1 file changed

+3
-5
lines changed

1 file changed

+3
-5
lines changed

parsl/dataflow/dflow.py

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -351,7 +351,9 @@ def handle_exec_update(self, task_record: TaskRecord, future: Future) -> None:
351351
else:
352352
task_record['fail_cost'] += 1
353353

354-
if task_record['status'] == States.dep_fail:
354+
if isinstance(e, DependencyError):
355+
# was this sending two task log infos? if so would I see the row twice in the monitoring db?
356+
self.update_task_state(task_record, States.dep_fail)
355357
logger.info("Task {} failed due to dependency failure so skipping retries".format(task_id))
356358
task_record['time_returned'] = datetime.datetime.now()
357359
self._send_task_log_info(task_record)
@@ -654,10 +656,6 @@ def _launch_if_ready_async(self, task_record: TaskRecord) -> None:
654656
else:
655657
logger.info(
656658
"Task {} failed due to dependency failure".format(task_id))
657-
# Raise a dependency exception
658-
self.update_task_state(task_record, States.dep_fail)
659-
660-
self._send_task_log_info(task_record)
661659

662660
exec_fu = Future()
663661
exec_fu.set_exception(DependencyError(exceptions_tids,

0 commit comments

Comments
 (0)