Report memory-usage while coverage report generation

[staabm]

closes #6419

before this PR:

XDEBUG_MODE=coverage php phpunit tests/unit --coverage-html tmp/html
PHPUnit 13.0-g39f926a92 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.4.15 with Xdebug 3.4.7
Configuration: /Users/staabm/workspace/phpunit/phpunit.xml
.............................................................   61 / 3834 (  1%)
.............................................................  122 / 3834 (  3%)
...
............................................................. 3782 / 3834 ( 98%)
....................................................          3834 / 3834 (100%)

Time: 00:08.935, Memory: 102.50 MB

OK (3834 tests, 11391 assertions)

Generating code coverage report in HTML format ... done [00:01.557]

after this PR:

XDEBUG_MODE=coverage php phpunit tests/unit --coverage-html tmp/html
PHPUnit 13.0-g39f926a92 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.4.15 with Xdebug 3.4.7
Configuration: /Users/staabm/workspace/phpunit/phpunit.xml
.............................................................   61 / 3834 (  1%)
.............................................................  122 / 3834 (  3%)
...
............................................................. 3782 / 3834 ( 98%)
....................................................          3834 / 3834 (100%)

Time: 00:08.935, Memory: 102.50 MB

OK (3834 tests, 11391 assertions)

Generating code coverage report in HTML format ... done [Time: 00:01.468, Memory: 36.98 MB]

see the difference in the last line of the output

[sebastianbergmann]

That is not really the amount of memory spent for the report generation, but memory_get_peak_usage() for the entire run.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.87%. Comparing base (39f926a) to head (db06bf3).
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #6421   +/-   ##
=========================================
  Coverage     95.87%   95.87%           
  Complexity     7272     7272           
=========================================
  Files           789      789           
  Lines         22378    22378           
=========================================
  Hits          21456    21456           
  Misses          922      922           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[staabm]

hmm you are right. I had it more precise before but then fell in love with ResourceUsageFormatter, without realizing the results changed ;)

adjusted accordingly

[sebastianbergmann]

Would this change in phpunit/php-timer help?

❯ git diff
src/ResourceUsageFormatter.php --- PHP
24            'MB' => 1048576,
25            'KB' => 1024,
26        ];
27    
28        public function resourceUsage(Duration $duration): string
32                $duration->asString(),
33                $this->bytesToString(memory_get_peak_usage(true)),
   28     public function resourceUsage(Result $result): string
   32             $result->duration()->asString(),
   33             $this->bytesToString($result->peakMemoryUsage()),
   34         );
   35     }
   36 
   37     /**

src/Timer.php --- 1/2 --- PHP
10    namespace SebastianBergmann\Timer;
11    
12    use function array_pop;
13    use function hrtime;
   14 use function memory_get_peak_usage;
   15 
   16 final class Timer
   17 {
   18     /**

src/Timer.php --- 2/2 --- PHP
18         * @var list<float>
19         */
20        private array $startTimes = [];
30        public function stop(): Duration
38            return Duration::fromNanoseconds((float) hrtime(true) - array_pop($this->startTimes));
   23     /**
   24      * @var list<int>
   25      */
   26     private array $peakMemoryUsages = [];
   31         $this->peakMemoryUsages[] = memory_get_peak_usage(true);
   37     public function stop(): Result
   45         return Result::from(
   46             Duration::fromNanoseconds((float) hrtime(true) - array_pop($this->startTimes)),
   47             array_pop($this->peakMemoryUsages),
   48         );
   49     }
   50 }

This would be a BC break as-is, but that would be okay if we wait with this until PHPUnit 13. Basically I am brainstorming future API ideas here. Result would be a new immutable value object that combines time and memory.

[staabm]

I think the question needs to be: will the user of phpunit/php-timer always need time and memory togehter.

or do use cases exist, where you either only need time, or only need memory but not both.
(from a output format point of view)