Skip to content

Conversation

@farski
Copy link
Member

@farski farski commented Jan 6, 2026

Previously we would do something in synchronous Fargate tasks, which have no capacity for passing data back to the state machine.

This changes to using a async waitForTaskToken config, which allows the Fargate task to pass arbitrary data back to the state machine upon completion. These negates the previous need to write similar data to S3, and then immediately read that data back from S3 in a Lambda function, just to make it available within the state machine.

The main thing to review here is ensuring that the output of send_task_success that's now happening within the Fargate code matches the previous result of the now-removed -task-output Lambda functions. And that send_task_failure is being used in a way that makes sense.

The FtpCopyTaskOutputLambdaErrorAlarm alarm has also been removed, since it was based on the now-removed function, and we've decided there's no need to find an alternative.

Closes #183

@farski farski requested review from cavis and kookster January 6, 2026 17:42
raise StandardError, "FFmpeg probe failed" unless system ffprobe_cmd

# Write the probe output to S3
puts "Writing probe output to S3 artifact bucket"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This probe output file is what was previously being read back by the subsequent Lambda function, so the relevant data (Size, Duration, etc) is now being passed back to the state machine directly here, (compare to this)

ErrorMessage: e.message
})
)
sf.send_task_failure({
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Compare to the old way, which caused Lambda executions to fail to force the FTP issue to be treated as a task failure. Now we do that directly.

@farski farski marked this pull request as ready for review January 6, 2026 18:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Switch to waitForTaskToken for Fargate tasks

2 participants