Skip to content

Commit 84ec5cf

Browse files
authored
Move DB manager exit check to happen in main thread (#3922)
Prior to this PR, self.close() was called inside the message migration thread. But self.close() does database stuff, which should all happen in one thread: the main thread. The historical reason is that self.close was originally triggered by a special stop message in what was previously many threads all running this migrate code. That stop message was removed in PR #3807 and replaced by a multiprocessing Event. This addresses some exceptions at shutdown when both the main thread and the message migration close() attempt to write to the database at the same time. This PR does not address the possibility of outstanding messages which have not reached the database manager at the time of shutdown. The message migration thread now does only message routing. ## Type of change - Bug fix
1 parent df56a01 commit 84ec5cf

File tree

1 file changed

+4
-3
lines changed

1 file changed

+4
-3
lines changed

parsl/monitoring/db_manager.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -547,6 +547,10 @@ def start(self,
547547
"or some other error. monitoring data may have been lost"
548548
)
549549
exception_happened = True
550+
551+
if self.external_exit_event.is_set():
552+
self.close()
553+
550554
if exception_happened:
551555
raise RuntimeError("An exception happened sometime during database processing and should have been logged in database_manager.log")
552556

@@ -558,9 +562,6 @@ def _migrate_logs_to_internal(self, logs_queue: mpq.Queue, kill_event: threading
558562
logger.debug("Checking STOP conditions: kill event: %s, queue has entries: %s",
559563
kill_event.is_set(), logs_queue.qsize() != 0)
560564

561-
if self.external_exit_event.is_set():
562-
self.close()
563-
564565
try:
565566
x = logs_queue.get(timeout=0.1)
566567
except queue.Empty:

0 commit comments

Comments
 (0)