->duration() method on TestResponse

[Gummibeer]

Sometimes you want to test the performance of your application. For sure it's not real performance testing - but getting the duration of a request in your tests could be the first start.

The idea is to add a method ->duration() on the \Illuminate\Testing\TestResponse class which returns the duration of the request in microseconds.

This would allow tests like the following - which checks that a heavy API endpoint is done within 3 seconds.

$response = $this->getJson(route('heavy.api.endpoint'));
$this->assertLessThanOrEqual(
    CarbonInterval::seconds(3)->totalMicroseconds,
    $response->duration()
);

This could also be added as a general test for all your endpoints to be for example faster than half a second.

This would be a convenient shorthand done by the framework for the current option to do it:

$start = microtime(true);
$response = $this->getJson(route('heavy.api.endpoint'));
$end = microtime(true);
$duration = ($end - $start) * 1000000;

The unit the duration is returned in would be to be defined - I just think that an integer is easier to test and read than a float.

A more limited solution but likely easier for most cases would be to add a assertMaxDuration(CarbonInterval::seconds(3)) method. Possibly it would be even worth adding both as it's for the status right now for example.

---

More advancements could be to add similar methods to test memory usage and query count as well.

$response->memory();
$response->assertMaxMemory($m);

$response->queries();
$response->assertQueryCount($x);
$response->assertMaxQueryCount($y);

[Wulfheart]

I like the idea conceptually. However, I see an issue with the reproducibility of a test which uses this method as the setup might be different and the computer it runs on might not be as powerful as another dev‘s machine.

[Gummibeer]

This is something for the people using it. We have such tests defined as only on CI. This way we have very good reproducible results and also have always a little rounding buffer.
So if we assert for half a second it means that the request should be done in a third second. If we assert for three seconds it should be more like two or two and a half a second.
Performance-Testing is never fully reproducible so independent of your setup you will always have to add some buffer and have a defined setup.

This test is for us to prevent side-effects increasing the duration dramatically. Like someone adds $with to any model and this leads to tons more queries and models to be loaded and rendered in the API resource.

[Wulfheart]

How can a test be marked CI only?

[Gummibeer]

You can use $this->markTestSkipped() and wrap it in any condition you want.
With a bit more work you can use doc-block annotations or even PHP8 attributes to declare these things outside of test logic and check&skip in the setUp method.

[thannaske]

I see @Wulfheart's point and would suggest adding only those assertions that are really deterministic. I e.g. really like the idea of using assertMaxQueryCount() and assertQueryCount().

[Gummibeer]

The breakiness is based on the expected value and different test setups. Which isn't related to the assertion itself. Changing things in hard or software can break a lot of things, not only a duration assertion.
It's up to the user how to use it and ensure a consistent setup.

[djaiss]

I agree with @Wulfheart above, but I would say that having the assertQueryCount is more important that testing the duration of the response. The duration depends on many different components that are often out of control of the CI environment it runs on. But knowing if you have too many queries is not. And I would love to know if there are any side effects to new concepts in my code that would increase the number of executed queries.