Skip to content

Commit 975adcf

Browse files
authored
Remove connected workers call in strategy (#3991)
This isn't used for strategizing. It is an RPC into the htex interchange that is at elevated risk of thread-related ZMQ misuse failures. For other executors, it is ignored. It is an abstraction leakage in general: nothing in the job scaling code needs to know about connected workers, and executors in general don't have a per-task-slot concept of connected workers. Even the MPI variant of HTEX doesn't have that. This removes connected workers use entirely, but leaves the call in place for interested users (and tests). The interchange log already contains details of workers, and that is where users should now go for worker information. This makes debugging a little different: this information was useful to see periodically in Parsl logs to see if workers are connected, and now that information lives elsewhere: in interchange.log. However the strategy polling thread is not a centralised status/logging aggregator system! # Changed Behaviour Less logging information in main parsl.log about connected workers - none at all, in fact. Balancing that, hopefully less ZMQ-misuse-related misbehaviour. ## Type of change - Code maintenance/cleanup
1 parent 0685765 commit 975adcf

File tree

1 file changed

+2
-6
lines changed

1 file changed

+2
-6
lines changed

parsl/jobs/strategy.py

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -218,12 +218,8 @@ def _general_strategy(self, executors: List[BlockProviderExecutor], *, strategy_
218218

219219
logger.debug(f"Slot ratio calculation: active_slots = {active_slots}, active_tasks = {active_tasks}")
220220

221-
if hasattr(executor, 'connected_workers'):
222-
logger.debug('Executor {} has {} active tasks, {}/{} running/pending blocks, and {} connected workers'.format(
223-
label, active_tasks, running, pending, executor.connected_workers()))
224-
else:
225-
logger.debug('Executor {} has {} active tasks and {}/{} running/pending blocks'.format(
226-
label, active_tasks, running, pending))
221+
logger.debug('Executor {} has {} active tasks and {}/{} running/pending blocks'.format(
222+
label, active_tasks, running, pending))
227223

228224
# reset idle timer if executor has active tasks
229225

0 commit comments

Comments
 (0)