Adds additional lock cleanup to worker cleanup

As another layer of security to guard against lock cleanup not occurring
due to Redis not delivering the _release_resource task, ensure all locks
are also cleaned up even for tasks that are in their final states.

backports #7907

fixes #9547

(cherry picked from commit 516df31)

Author: bmbouter

CHANGES/9547.bugfix

@@ -0,0 +1,3 @@
+Prevented a Redis failure scenario from causing the tasking system to back up due to "tasking system
+locks" not being released, even on worker restart.
+(backported from #7907)

pulpcore/tasking/services/worker_watcher.py

@@ -160,8 +160,12 @@ def mark_worker_offline(worker_name, normal_shutdown=False):
         pass
     else:
         # Cancel all of the tasks that were assigned to this worker's queue
-        for task_status in worker.tasks.filter(state__in=TASK_INCOMPLETE_STATES):
-            cancel(task_status.pk)
+        for task in worker.tasks.filter(state__in=TASK_INCOMPLETE_STATES):
+            cancel(task.pk)
+
+        # Ensure all locks are released for those tasks that are in final states also
+        for task in worker.tasks.exclude(state__in=TASK_INCOMPLETE_STATES):
+            task.release_resources()
 
         if normal_shutdown:
             worker.gracefully_stopped = True