Is there a way to figure out metadata config for dead jobs?

[znavidi]

I ran DC on several batches of images and about 21 out of 2000 jobs moved to dead messages and the rest completed without error. Is there a way that I can find which set of metadata batches failed and re-run them?
Thanks in advance!

[ErinWeisbart]

Here's a Python script that you can use to describe your Dead Messages.
https://github.com/broadinstitute/ImagingPlatformHelpfulScripts/blob/main/describe_messages_in_SQS_queue.py

I also regularly use this code to just move all the dead messages back to an active queue.

# Move DEAD_MESSAGES to active queue
import boto3
session = boto3.Session(profile_name='PROFILENAME') # in ~/.aws/config, section named [profile PROFILENAME], must have key, secret key, region, output
client = session.client('sqs')

client.start_message_move_task(
SourceArn='arn:aws:sqs:us-east-1:123456789123:Erin_DeadMessages',
DestinationArn='arn:aws:sqs:us-east-1:123456789123:ActiveProject_AnalysisQueue',
)

[znavidi]

Thanks a lot Erin! I ran the script you shared and can see that dead messages are back to my other queue. Should I next run the last command, which is python run.py startCluster files/{YourFleetFile}.json
? could you please confirm?

[ErinWeisbart]

What you do next depends on what infrastructure you have in place from your last run.
If you had monitor clean up and shut everything down, you'll need to run setup and startCluster.
If you only cancelled your spot fleet request but left everything else in place you'll just need to run startCluster.