TimeoutHandlerWrapper for graceful timeout management in handlers

[fredericgboutin-yapla]

Hi,
I just want to share this little handler wrapper that I created like... 2 years ago. We are using it in production since then and it basically make it so that we get a specialized TimeoutException that our scripts can try/catch on.

In our situation, it is valuable so we can send logs (read: Sentry) or suspend a specific processing so it can be picked-up later on.

I'm sharing it because I'm pretty convinced there is real value in there for people processing SQS messages a lot with lambda functions in PHP, like we do.

Dependency

composer require -W phpunit/php-invoker

So yes, it uses SebastianBergmann\Invoker - even though it is from phpunit, it basically just make use of the pcntl extension in a convenient way. The code is simple, well isolated and straight forward so, I don't see any problem having this dependency in the non-dev section of my composer file but yeah it is a bit awkward.

Handler

<?php

use Bref\Event\Handler;

class TimeoutHandlerWrapper implements Handler
{
    private Handler $handler;
    private int $desiredTimeLeftBeforeLambdaTimeout;

    public function __construct(Handler $handler, int $desiredTimeLeftBeforeLambdaTimeout = 5)
    {
        $this->handler = $handler;
        $this->desiredTimeLeftBeforeLambdaTimeout = $desiredTimeLeftBeforeLambdaTimeout;
    }

    /**
     * @inheritdoc
     */
    public function handle($event, Bref\Context\Context $context)
    {
        $invoke = new SebastianBergmann\Invoker\Invoker();

        $timeout = \Hpjorg\round($context->getRemainingTimeInMillis() / 1000 - $this->desiredTimeLeftBeforeLambdaTimeout);

        $invoke->invoke([$this->handler, 'handle'], [$event, $context], max($timeout, 1));
    }
}

Unit-test

<?php

class HandlerMock implements Bref\Event\Handler
{
	public bool $wasCalled = false;
	public float $timeLeft = -1;

	public function handle($event, Bref\Context\Context $context)
	{
		try {
			sleep(15);
		} catch (\SebastianBergmann\Invoker\TimeoutException) {
			$this->wasCalled = true;
			$this->timeLeft = $context->getRemainingTimeInMillis() / 1000;
		}
	}
}

class TimeoutHandlerWrapperTest extends TestCase
{
	public function testHandlerWrapperShouldTriggerATimeoutWithEnoughTimeLeft()
	{
		$handler = new HandlerMock();
		$handlerWrapper = new TimeoutHandlerWrapper($handler, 8);

		$deadlineInTheFuture = intval(microtime(true) * 1000) + 10 * 1000; // 10 seconds in the future
		$handlerWrapper->handle([], new Bref\Context\Context('', $deadlineInTheFuture, 0, '', '', ''));

		$this->assertTrue($handler->wasCalled);

		$this->assertLessThanOrEqual(10, $handler->timeLeft);
		$this->assertGreaterThanOrEqual(7, $handler->timeLeft);
	}
}

Usage

<?php

use Bref\Context\Context;
use Bref\Event\Sqs\SqsEvent;
use Bref\Event\Sqs\SqsHandler;
use SebastianBergmann\Invoker\TimeoutException;

class MySqsMessagesHandler extends SqsHandler
{
	public function handleSqs(SqsEvent $event, Context $context): void
	{
		$records = $event->getRecords();

		foreach ($records as $key => $record) {
			try {
				// Do my processing here 
			} catch (TimeoutException) {
				Log::warning('Lambda timeout reached');

				// Gracefully shutdown
				 return;
			} catch (\Throwable $e) {
				Log::error($e);
			}
		}
	}
}

return new TimeoutHandlerWrapper(new MySqsMessagesHandler(), desiredTimeLeftBeforeLambdaTimeout: 5);

The real kicker here is that we created an abstract class to help us better manage SQS message handling, taking into account to properly markAsFailed remaining records on timeout and to markAsFailed the current and all remaining messages on any other exception when dealing with a FIFO queue - as per AWS recommandation. - but I'm getting a bit out of my way here, that last subject would be better off to keep for another thread 😅

Caveat

The big caveat here ❗ is that since the timeout exception is raised asynchronously (really, that's kind of rare in PHP, I know) it means the original code processing something was interrupted. From a probabilistic point of view, chances are the script stopped on a lengthy call to the DB or an external API.

But yeah, it forces the programmer to think about its processing being stop in the middle. At least, the catch section offers an opportunity to set a status, to cleanup some data, etc. compared to a brutal lambda timeout.