Skip to content

Conversation

ondrejmirtes
Copy link
Member

/cc @staabm PHPUnit crashes with:

PHP Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 146082352 bytes) in /Users/ondrej/Development/phpstan/vendor/sebastian/exporter/src/Exporter.php on line 440
Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 146082352 bytes) in /Users/ondrej/Development/phpstan/vendor/sebastian/exporter/src/Exporter.php on line 440

Maybe there's an opportunity for optimization. Could you look into it and maybe contribute a fix to PHPUnit? Thanks!

@staabm
Copy link
Contributor

staabm commented Jun 10, 2025

I might have some time later this week to look into it

@ondrejmirtes
Copy link
Member Author

These kinds of changes work: 7389a6d

So PHPUnit is once again not handling passing huge objects from data providers to test methods. Scope contains class reflection and that will be hard to serialize.

@ondrejmirtes
Copy link
Member Author

If you're going to try to improve PHPUnit, do it on one commit before that one :)

@ondrejmirtes
Copy link
Member Author

Even if I update/remove tests that cause memory exhaustion, the whole test suite is still several times slower than before 🙁 This really needs to be solved in PHPUnit.

@staabm
Copy link
Contributor

staabm commented Jun 12, 2025

is this the same problem we discussed once in phpstan/phpstan#9914 ?

Blackfire Profile of PHPUnit 11 with blackfire run vendor/bin/phpunit tests/PHPStan/Type/ObjectTypeTest.php
https://blackfire.io/profiles/38ce70d3-f409-4081-b846-8b8bd017ff14/graph

@staabm
Copy link
Contributor

staabm commented Jun 12, 2025

btw: did you try whether phpunit 12 has the same problem?

@staabm
Copy link
Contributor

staabm commented Jun 12, 2025

Changing vendor/phpunit/phpunit/src/Event/Value/Test/TestMethodBuilder.php

            $testData[] = DataFromDataProvider::from(
                $dataSetName,
-               Exporter::export($testCase->providedData()),
+               Exporter::shortenedRecursiveExport($testCase->providedData()),
                $testCase->dataSetAsStringWithData(),
            );

turns vendor/bin/phpunit tests/PHPStan/Type/ObjectTypeTest.php from

➜  phpstan-src git:(phpunit-slow) ✗ vendor/bin/phpunit tests/PHPStan/Type/ObjectTypeTest.php
PHPUnit 11.5.22 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.22
Configuration: /Users/m.staab/Documents/GitHub/phpstan-src/phpunit.xml
Random Seed:   1749737382

...............................................................  63 / 102 ( 61%)
.......................................                         102 / 102 (100%)

Time: 00:23.674, Memory: 675.19 MB

OK (102 tests, 114 assertions)

into

➜  phpstan-src git:(phpunit-slow) vendor/bin/phpunit tests/PHPStan/Type/ObjectTypeTest.php
PHPUnit 11.5.22 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.22
Configuration: /Users/m.staab/Documents/GitHub/phpstan-src/phpunit.xml
Random Seed:   1749737437

...............................................................  63 / 102 ( 61%)
.......................................                         102 / 102 (100%)

Time: 00:00.256, Memory: 88.00 MB

OK (102 tests, 114 assertions)

.. still looking into it..

@staabm
Copy link
Contributor

staabm commented Jun 12, 2025

with the above patch the test-suite is working for me again:

ddde40e with PHPUnit 11.x

$ make tests

XDEBUG_MODE=off php tests/vendor/bin/paratest --runner WrapperRunner --no-coverage
ParaTest v7.8.3 upon PHPUnit 11.5.22 by Sebastian Bergmann and contributors.

Processes:     14
Runtime:       PHP 8.3.22
Configuration: /Users/m.staab/Documents/GitHub/phpstan-src/phpunit.xml
Random Seed:   1749738159

......................SS...................................    59 / 12839 (  0%)
...........................................................   118 / 12839 (  0%)
...........................................................   177 / 12839 (  1%)
....................S..S...................................   236 / 12839 (  1%)
.................................S.........................   295 / 12839 (  2%)
..........S.......S..................SS....................   354 / 12839 (  2%)
.............S.............................S...............   413 / 12839 (  3%)
...............................................SS..........   472 / 12839 (  3%)
...........................................................   531 / 12839 (  4%)
...........................................................   590 / 12839 (  4%)
...S.......................................................   649 / 12839 (  5%)
.........................SS................................   708 / 12839 (  5%)
...........................SS..............................   767 / 12839 (  5%)
.............................S.................S...........   826 / 12839 (  6%)
......................SSS..............SSSSSSSSSSSSSSSSS...   885 / 12839 (  6%)
...................S.............I.........................   944 / 12839 (  7%)
...........................................................  1003 / 12839 (  7%)
...........................................................  1062 / 12839 (  8%)
.......................................SS..S...............  1121 / 12839 (  8%)
................................................SSSSSSSSS..  1180 / 12839 (  9%)
...........................................................  1239 / 12839 (  9%)
.......................S...................................  1298 / 12839 ( 10%)
...........................................................  1357 / 12839 ( 10%)
...........................................................  1416 / 12839 ( 11%)
..........................................................S  1475 / 12839 ( 11%)
........S..................................................  1534 / 12839 ( 11%)
...........................................................  1593 / 12839 ( 12%)
...........................................................  1652 / 12839 ( 12%)
..........................................................S  1711 / 12839 ( 13%)
...........................................................  1770 / 12839 ( 13%)
...........................................................  1829 / 12839 ( 14%)
...........................................................  1888 / 12839 ( 14%)
.....R......S..............................................  1947 / 12839 ( 15%)
..........................S................................  2006 / 12839 ( 15%)
...........................................................  2065 / 12839 ( 16%)
....S..............SSSS....................................  2124 / 12839 ( 16%)
...........................................................  2183 / 12839 ( 17%)
..................................SS.......................  2242 / 12839 ( 17%)
...........................................................  2301 / 12839 ( 17%)
...........................................................  2360 / 12839 ( 18%)
...........................................................  2419 / 12839 ( 18%)
...........................................................  2478 / 12839 ( 19%)
...........................................................  2537 / 12839 ( 19%)
...........................................................  2596 / 12839 ( 20%)
...........................................................  2655 / 12839 ( 20%)
...........................................................  2714 / 12839 ( 21%)
...........................................................  2773 / 12839 ( 21%)
....................................................S......  2832 / 12839 ( 22%)
...........................................................  2891 / 12839 ( 22%)
...............................................S...........  2950 / 12839 ( 22%)
...........................................................  3009 / 12839 ( 23%)
................................S.S........................  3068 / 12839 ( 23%)
...........................................................  3127 / 12839 ( 24%)
...........................................................  3186 / 12839 ( 24%)
...........................................................  3245 / 12839 ( 25%)
...............................................S...........  3304 / 12839 ( 25%)
..............S............................................  3363 / 12839 ( 26%)
...........................................................  3422 / 12839 ( 26%)
...........................................................  3481 / 12839 ( 27%)
...........................................................  3540 / 12839 ( 27%)
...........................................................  3599 / 12839 ( 28%)
...........................................................  3658 / 12839 ( 28%)
...........................................................  3717 / 12839 ( 28%)
...........................................................  3776 / 12839 ( 29%)
...........................................................  3835 / 12839 ( 29%)
...........................................................  3894 / 12839 ( 30%)
...........................................................  3953 / 12839 ( 30%)
...........................................................  4012 / 12839 ( 31%)
...........................................................  4071 / 12839 ( 31%)
...........................................................  4130 / 12839 ( 32%)
...........................................................  4189 / 12839 ( 32%)
...........................................................  4248 / 12839 ( 33%)
...........................................................  4307 / 12839 ( 33%)
...........................................................  4366 / 12839 ( 34%)
...........................................................  4425 / 12839 ( 34%)
...........................................................  4484 / 12839 ( 34%)
...........................................................  4543 / 12839 ( 35%)
...........................................................  4602 / 12839 ( 35%)
...........................................................  4661 / 12839 ( 36%)
...........................................................  4720 / 12839 ( 36%)
...........................................................  4779 / 12839 ( 37%)
...........................................................  4838 / 12839 ( 37%)
...........................................................  4897 / 12839 ( 38%)
...........................................................  4956 / 12839 ( 38%)
...........................................................  5015 / 12839 ( 39%)
...........................................................  5074 / 12839 ( 39%)
...S.......................................................  5133 / 12839 ( 39%)
...........................................................  5192 / 12839 ( 40%)
...........................................................  5251 / 12839 ( 40%)
...........................................................  5310 / 12839 ( 41%)
...........................................................  5369 / 12839 ( 41%)
...........................................................  5428 / 12839 ( 42%)
...........................................................  5487 / 12839 ( 42%)
...........................................................  5546 / 12839 ( 43%)
...........................................................  5605 / 12839 ( 43%)
...........................................................  5664 / 12839 ( 44%)
...........................................................  5723 / 12839 ( 44%)
...........................................................  5782 / 12839 ( 45%)
...........................................................  5841 / 12839 ( 45%)
...........................................................  5900 / 12839 ( 45%)
.........................................................S.  5959 / 12839 ( 46%)
.......................................SS..................  6018 / 12839 ( 46%)
...........................................................  6077 / 12839 ( 47%)
...........................................................  6136 / 12839 ( 47%)
..............................................SS...........  6195 / 12839 ( 48%)
...........................................................  6254 / 12839 ( 48%)
........S..................................................  6313 / 12839 ( 49%)
.................................S...S.......S.............  6372 / 12839 ( 49%)
...........................................................  6431 / 12839 ( 50%)
...........................................................  6490 / 12839 ( 50%)
......................................S..............SSSSSS  6549 / 12839 ( 51%)
SSSSSSSS.S..............................................S..  6608 / 12839 ( 51%)
...........................S...............................  6667 / 12839 ( 51%)
S.................................S........................  6726 / 12839 ( 52%)
.........................S.................................  6785 / 12839 ( 52%)
.....................................S.....................  6844 / 12839 ( 53%)
.............................S.....S.......................  6903 / 12839 ( 53%)
.........................................................SS  6962 / 12839 ( 54%)
...........................................................  7021 / 12839 ( 54%)
...........................................................  7080 / 12839 ( 55%)
S..........................................................  7139 / 12839 ( 55%)
.................SS........................................  7198 / 12839 ( 56%)
...........................................SSS.............  7257 / 12839 ( 56%)
.......................................SSSSSSSSS..........S  7316 / 12839 ( 56%)
SSSSSSS....................................................  7375 / 12839 ( 57%)
......................S....................................  7434 / 12839 ( 57%)
...........................................................  7493 / 12839 ( 58%)
...........................................................  7552 / 12839 ( 58%)
...........................................................  7611 / 12839 ( 59%)
...........................................................  7670 / 12839 ( 59%)
...........................................................  7729 / 12839 ( 60%)
...........................................................  7788 / 12839 ( 60%)
...........................................................  7847 / 12839 ( 61%)
...........................................................  7906 / 12839 ( 61%)
...........................................................  7965 / 12839 ( 62%)
...........................................................  8024 / 12839 ( 62%)
...........................................................  8083 / 12839 ( 62%)
...........................................................  8142 / 12839 ( 63%)
...........................................................  8201 / 12839 ( 63%)
...........................................................  8260 / 12839 ( 64%)
...........................................................  8319 / 12839 ( 64%)
...........................................................  8378 / 12839 ( 65%)
...........................................................  8437 / 12839 ( 65%)
...........................................................  8496 / 12839 ( 66%)
...........................................................  8555 / 12839 ( 66%)
...........................................................  8614 / 12839 ( 67%)
...........................................................  8673 / 12839 ( 67%)
...........................................................  8732 / 12839 ( 68%)
...........................................................  8791 / 12839 ( 68%)
...........................................................  8850 / 12839 ( 68%)
...........................................................  8909 / 12839 ( 69%)
...........................................................  8968 / 12839 ( 69%)
...........................................................  9027 / 12839 ( 70%)
...........................................................  9086 / 12839 ( 70%)
...........................................................  9145 / 12839 ( 71%)
...........................................................  9204 / 12839 ( 71%)
...........................................................  9263 / 12839 ( 72%)
...........................................................  9322 / 12839 ( 72%)
...........................................................  9381 / 12839 ( 73%)
...........................................................  9440 / 12839 ( 73%)
...........................................................  9499 / 12839 ( 73%)
...........................................................  9558 / 12839 ( 74%)
...........................................................  9617 / 12839 ( 74%)
...........................................................  9676 / 12839 ( 75%)
...........................................................  9735 / 12839 ( 75%)
...........................................................  9794 / 12839 ( 76%)
...........................................................  9853 / 12839 ( 76%)
...........................................................  9912 / 12839 ( 77%)
...........................................................  9971 / 12839 ( 77%)
........................................................... 10030 / 12839 ( 78%)
........................................................... 10089 / 12839 ( 78%)
........................................................... 10148 / 12839 ( 79%)
........................................................... 10207 / 12839 ( 79%)
........................................................... 10266 / 12839 ( 79%)
........................................................... 10325 / 12839 ( 80%)
........................................................... 10384 / 12839 ( 80%)
........................................................... 10443 / 12839 ( 81%)
........................................................... 10502 / 12839 ( 81%)
........................................................... 10561 / 12839 ( 82%)
........................................................... 10620 / 12839 ( 82%)
........................................................... 10679 / 12839 ( 83%)
........................................................... 10738 / 12839 ( 83%)
........................................................... 10797 / 12839 ( 84%)
........................................................... 10856 / 12839 ( 84%)
........................................................... 10915 / 12839 ( 85%)
........................................................... 10974 / 12839 ( 85%)
........................................................... 11033 / 12839 ( 85%)
........................................................... 11092 / 12839 ( 86%)
........................................................... 11151 / 12839 ( 86%)
........................................................... 11210 / 12839 ( 87%)
........................................................... 11269 / 12839 ( 87%)
........................................................... 11328 / 12839 ( 88%)
........................................................... 11387 / 12839 ( 88%)
........................................................... 11446 / 12839 ( 89%)
........................................................... 11505 / 12839 ( 89%)
........................................................... 11564 / 12839 ( 90%)
........................................................... 11623 / 12839 ( 90%)
........................................................... 11682 / 12839 ( 90%)
........................................................... 11741 / 12839 ( 91%)
........................................................... 11800 / 12839 ( 91%)
........................................................... 11859 / 12839 ( 92%)
........................................................... 11918 / 12839 ( 92%)
........................................................... 11977 / 12839 ( 93%)
........................................................... 12036 / 12839 ( 93%)
........................................................... 12095 / 12839 ( 94%)
........................................................... 12154 / 12839 ( 94%)
........................................................... 12213 / 12839 ( 95%)
........................................................... 12272 / 12839 ( 95%)
........................................................... 12331 / 12839 ( 96%)
........................................................... 12390 / 12839 ( 96%)
........................................................... 12449 / 12839 ( 96%)
........................................................... 12508 / 12839 ( 97%)
........................................................... 12567 / 12839 ( 97%)
........................................................... 12626 / 12839 ( 98%)
...........................................R............... 12685 / 12839 ( 98%)
........................................................... 12744 / 12839 ( 99%)
........................................................... 12803 / 12839 ( 99%)
....................................                        12839 / 12839 (100%)

Time: 00:44.635, Memory: 538.50 MB

There were 2 risky tests:

1) PHPStan\Analyser\AnalyserIntegrationTest::testCollectWarnings
Test code or tested code removed error handlers other than its own

/Users/m.staab/Documents/GitHub/phpstan-src/tests/PHPStan/Analyser/AnalyserIntegrationTest.php:185

2) PHPStan\Command\AnalyseApplicationIntegrationTest::testExecuteOnANonExistentPath
Test code or tested code removed error handlers other than its own

/Users/m.staab/Documents/GitHub/phpstan-src/tests/PHPStan/Command/AnalyseApplicationIntegrationTest.php:34

OK, but there were issues!
Tests: 12839, Assertions: 84526, Skipped: 132, Incomplete: 1, Risky: 2.
make: *** [tests] Error 1

vs

5b4bf80 with PHPUnit 10.x

make tests

XDEBUG_MODE=off php tests/vendor/bin/paratest --runner WrapperRunner --no-coverage
ParaTest v7.3.1 upon PHPUnit 10.5.31 by Sebastian Bergmann and contributors.

Processes:     14
Runtime:       PHP 8.3.22
Configuration: /Users/m.staab/Documents/GitHub/phpstan-src/phpunit.xml
Random Seed:   1749738298

............................................SSS............    59 / 12845 (  0%)
..............................................SS...........   118 / 12845 (  0%)
.....................SSS...................................   177 / 12845 (  1%)
..S........................................................   236 / 12845 (  1%)
.......................................................SSSS   295 / 12845 (  2%)
SSSSS..........SSSSSSSS....................................   354 / 12845 (  2%)
.......S...................................................   413 / 12845 (  3%)
...........................................................   472 / 12845 (  3%)
.........................................S.................   531 / 12845 (  4%)
....................................................SS.....   590 / 12845 (  4%)
...........................................................   649 / 12845 (  5%)
....................S......................................   708 / 12845 (  5%)
...................S.......................................   767 / 12845 (  5%)
..S..............................S.........................   826 / 12845 (  6%)
...........................................................   885 / 12845 (  6%)
..........................S...........S...S................   944 / 12845 (  7%)
...........................................................  1003 / 12845 (  7%)
...........................................................  1062 / 12845 (  8%)
..........I................................................  1121 / 12845 (  8%)
...........................................................  1180 / 12845 (  9%)
.......S...................................................  1239 / 12845 (  9%)
...........................................................  1298 / 12845 ( 10%)
......................................................SS...  1357 / 12845 ( 10%)
...........................................................  1416 / 12845 ( 11%)
...........................................................  1475 / 12845 ( 11%)
...........................................................  1534 / 12845 ( 11%)
...........................................................  1593 / 12845 ( 12%)
.........................................................S.  1652 / 12845 ( 12%)
...........................................................  1711 / 12845 ( 13%)
...........................................................  1770 / 12845 ( 13%)
...........................................................  1829 / 12845 ( 14%)
...........................................................  1888 / 12845 ( 14%)
...........................................................  1947 / 12845 ( 15%)
...........................................................  2006 / 12845 ( 15%)
...........................................................  2065 / 12845 ( 16%)
...........................................................  2124 / 12845 ( 16%)
...........................................................  2183 / 12845 ( 16%)
...........................................................  2242 / 12845 ( 17%)
...........................................................  2301 / 12845 ( 17%)
...........................................................  2360 / 12845 ( 18%)
...........................................................  2419 / 12845 ( 18%)
...........................................................  2478 / 12845 ( 19%)
...........................................................  2537 / 12845 ( 19%)
...........................................................  2596 / 12845 ( 20%)
...........................................................  2655 / 12845 ( 20%)
...........................................................  2714 / 12845 ( 21%)
...........................................................  2773 / 12845 ( 21%)
...........................................................  2832 / 12845 ( 22%)
......................S................S...................  2891 / 12845 ( 22%)
........S...............................S..................  2950 / 12845 ( 22%)
...............................S...........................  3009 / 12845 ( 23%)
......S....................................................  3068 / 12845 ( 23%)
...........................................................  3127 / 12845 ( 24%)
...........................................................  3186 / 12845 ( 24%)
...........................................................  3245 / 12845 ( 25%)
...........................................................  3304 / 12845 ( 25%)
...........................................................  3363 / 12845 ( 26%)
.........................S.................................  3422 / 12845 ( 26%)
...........................................................  3481 / 12845 ( 27%)
.........................................................SS  3540 / 12845 ( 27%)
....SSSSSS.................................................  3599 / 12845 ( 28%)
.....................S...............S.....................  3658 / 12845 ( 28%)
..........S.........S......................................  3717 / 12845 ( 28%)
...........................................................  3776 / 12845 ( 29%)
...........................................................  3835 / 12845 ( 29%)
.....................................  101 / 2705 (  3%)  202 / 2705 (  7%)  303 / 2705 ( 11%)  404 / 2705 ( 14%)  505 / 2705 ( 18%)  606 / 2705 ( 22%)  707 / 2705 ( 26%)  808 / 2705 ( 29%)  909 / 2705 ( 33%)......................  3894 / 12845 ( 30%)
...........................................................  3953 / 12845 ( 30%)
...........................................................  4012 / 12845 ( 31%)
...........................................................  4071 / 12845 ( 31%)
...........................................................  4130 / 12845 ( 32%)
...........................................................  4189 / 12845 ( 32%)
...........................................................  4248 / 12845 ( 33%)
...........................................................  4307 / 12845 ( 33%)
...........................................................  4366 / 12845 ( 33%)
...........................................................  4425 / 12845 ( 34%)
...........................................................  4484 / 12845 ( 34%)
...........................................................  4543 / 12845 ( 35%)
...........................................................  4602 / 12845 ( 35%)
...........................................................  4661 / 12845 ( 36%)
...........................................................  4720 / 12845 ( 36%)
...........................................................  4779 / 12845 ( 37%)
...........................................................  4838 / 12845 ( 37%)
...........................................................  4897 / 12845 ( 38%)
...........................................................  4956 / 12845 ( 38%)
...........................................................  5015 / 12845 ( 39%)
...........................................................  5074 / 12845 ( 39%)
...........................................................  5133 / 12845 ( 39%)
...........................................................  5192 / 12845 ( 40%)
...........................................................  5251 / 12845 ( 40%)
...........................................................  5310 / 12845 ( 41%)
...........................................................  5369 / 12845 ( 41%)
...........................................................  5428 / 12845 ( 42%)
...........................................................  5487 / 12845 ( 42%)
...........................................................  5546 / 12845 ( 43%)
...........................................................  5605 / 12845 ( 43%)
...........................................................  5664 / 12845 ( 44%)
...........................................................  5723 / 12845 ( 44%)
...........................................................  5782 / 12845 ( 45%)
...........................................................  5841 / 12845 ( 45%)
...........................................................  5900 / 12845 ( 45%)
...........................................................  5959 / 12845 ( 46%)
...........................................................  6018 / 12845 ( 46%)
...........................................................  6077 / 12845 ( 47%)
...........................................................  6136 / 12845 ( 47%)
...........................................................  6195 / 12845 ( 48%)
...........................................................  6254 / 12845 ( 48%)
...........................................................  6313 / 12845 ( 49%)
...........................................................  6372 / 12845 ( 49%)
...........................................................  6431 / 12845 ( 50%)
...........................................................  6490 / 12845 ( 50%)
...........................................................  6549 / 12845 ( 50%)
...........................................................  6608 / 12845 ( 51%)
...........................................................  6667 / 12845 ( 51%)
...........................................................  6726 / 12845 ( 52%)
...........................................................  6785 / 12845 ( 52%)
...............................................S...........  6844 / 12845 ( 53%)
..................................S............SSSSSSSSSSSS  6903 / 12845 ( 53%)
SSSSS......................................................  6962 / 12845 ( 54%)
...........................................................  7021 / 12845 ( 54%)
...........................................................  7080 / 12845 ( 55%)
...........................................................  7139 / 12845 ( 55%)
...........................................................  7198 / 12845 ( 56%)
...........................................................  7257 / 12845 ( 56%)
...........................................................  7316 / 12845 ( 56%)
...SSS..........SS.........................................  7375 / 12845 ( 57%)
.................  101 / 2565 (  3%)  202 / 2565 (  7%)  303 / 2565 ( 11%)  404 / 2565 ( 15%)  505 / 2565 ( 19%)  606 / 2565 ( 23%)  707 / 2565 ( 27%)  808 / 2565 ( 31%)  909 / 2565 ( 35%)..........................................  7434 / 12845 ( 57%)
...........................................................  7493 / 12845 ( 58%)
...........................................................  7552 / 12845 ( 58%)
...........................................................  7611 / 12845 ( 59%)
...........................................................  7670 / 12845 ( 59%)
...........................................................  7729 / 12845 ( 60%)
...........................................................  7788 / 12845 ( 60%)
...........................................................  7847 / 12845 ( 61%)
...........................................................  7906 / 12845 ( 61%)
...........................................................  7965 / 12845 ( 62%)
...........................................................  8024 / 12845 ( 62%)
...........................................................  8083 / 12845 ( 62%)
...........................................................  8142 / 12845 ( 63%)
...........................................................  8201 / 12845 ( 63%)
...........................................................  8260 / 12845 ( 64%)
...........................................................  8319 / 12845 ( 64%)
...........................................................  8378 / 12845 ( 65%)
...........................................................  8437 / 12845 ( 65%)
...........................................................  8496 / 12845 ( 66%)
...........................................................  8555 / 12845 ( 66%)
...........................................................  8614 / 12845 ( 67%)
...........................................................  8673 / 12845 ( 67%)
...........................................................  8732 / 12845 ( 67%)
...........................................................  8791 / 12845 ( 68%)
...........................................................  8850 / 12845 ( 68%)
...........................................................  8909 / 12845 ( 69%)
...........................................................  8968 / 12845 ( 69%)
...........................................................  9027 / 12845 ( 70%)
...........................................................  9086 / 12845 ( 70%)
.......................................................S...  9145 / 12845 ( 71%)
...........................................................  9204 / 12845 ( 71%)
...........................................................  9263 / 12845 ( 72%)
...........................................................  9322 / 12845 ( 72%)
...........................................................  9381 / 12845 ( 73%)
...........................................................  9440 / 12845 ( 73%)
...........................................................  9499 / 12845 ( 73%)
...........................................................  9558 / 12845 ( 74%)
...........................................................  9617 / 12845 ( 74%)
...........................................................  9676 / 12845 ( 75%)
...........................................................  9735 / 12845 ( 75%)
...........................................................  9794 / 12845 ( 76%)
...........................................................  9853 / 12845 ( 76%)
...........................................................  9912 / 12845 ( 77%)
...........................................................  9971 / 12845 ( 77%)
........................................................... 10030 / 12845 ( 78%)
.....................................SS.................... 10089 / 12845 ( 78%)
........................................................... 10148 / 12845 ( 79%)
........................................................... 10207 / 12845 ( 79%)
........................................................... 10266 / 12845 ( 79%)
..SSSSSSS.................................................. 10325 / 12845 ( 80%)
................SS........................................S 10384 / 12845 ( 80%)
..........................................................S 10443 / 12845 ( 81%)
SSSSSS.....SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS............... 10502 / 12845 ( 81%)
...........S............................................... 10561 / 12845 ( 82%)
..S........................................................ 10620 / 12845 ( 82%)
..............S............................................ 10679 / 12845 ( 83%)
........................................................... 10738 / 12845 ( 83%)
.................................S......................... 10797 / 12845 ( 84%)
...............................................S.S......... 10856 / 12845 ( 84%)
........................................................... 10915 / 12845 ( 84%)
...................................S....................... 10974 / 12845 ( 85%)
........................................................... 11033 / 12845 ( 85%)
........................................................... 11092 / 12845 ( 86%)
......................................S......S............. 11151 / 12845 ( 86%)
........................SSSSSSSSSSSSSS.S......S............ 11210 / 12845 ( 87%)
............S......................SSSS.........SS....S.... 11269 / 12845 ( 87%)
........................................................... 11328 / 12845 ( 88%)
........................................................... 11387 / 12845 ( 88%)
..........................SS............................... 11446 / 12845 ( 89%)
........................................................... 11505 / 12845 ( 89%)
.................................  101 / 1118 (  9%)  202 / 1118 ( 18%)  303 / 1118 ( 27%)  404 / 1118 ( 36%)  505 / 1118 ( 45%)  606 / 1118 ( 54%)  707 / 1118 ( 63%)  808 / 1118 ( 72%)  909 / 1118 ( 81%).......................... 11564 / 12845 ( 90%)
........................................................... 11623 / 12845 ( 90%)
........................................................... 11682 / 12845 ( 90%)
........................................................... 11741 / 12845 ( 91%)
........................................................... 11800 / 12845 ( 91%)
........................................................... 11859 / 12845 ( 92%)
........................................................... 11918 / 12845 ( 92%)
........................................................... 11977 / 12845 ( 93%)
........................................................... 12036 / 12845 ( 93%)
........................................................... 12095 / 12845 ( 94%)
........................................................... 12154 / 12845 ( 94%)
........................................................... 12213 / 12845 ( 95%)
........................................................... 12272 / 12845 ( 95%)
........................................................... 12331 / 12845 ( 95%)
........................................................... 12390 / 12845 ( 96%)
........................................................... 12449 / 12845 ( 96%)
........................................................... 12508 / 12845 ( 97%)
........................................................... 12567 / 12845 ( 97%)
........................................................... 12626 / 12845 ( 98%)
........................................................... 12685 / 12845 ( 98%)
........................................................... 12744 / 12845 ( 99%)
........................................................... 12803 / 12845 ( 99%)
..........................................                  12845 / 12845 (100%)

Time: 00:45.158, Memory: 534.50 MB

OK, but there were issues!
Tests: 12845, Assertions: 84492, Skipped: 172, Incomplete: 1.

wdyt?

@staabm
Copy link
Contributor

staabm commented Jun 13, 2025

Fix landed in sebastianbergmann/phpunit@7ff0969

For PHPUnit 11.5.x and newer

released https://github.com/sebastianbergmann/phpunit/releases/tag/11.5.23


tested locally with 11.5.23 and can confirm the fix

@ondrejmirtes
Copy link
Member Author

Thank you!

@ondrejmirtes
Copy link
Member Author

Maybe this test matrix generation needs updating? https://github.com/phpstan/phpstan-src/actions/runs/15630379293/job/44033085940?pr=4057

@staabm
Copy link
Contributor

staabm commented Jun 13, 2025

Can have a look later today

@staabm
Copy link
Contributor

staabm commented Jun 13, 2025

the xml format generated by vendor/bin/phpunit --list-tests-xml changed with sebastianbergmann/phpunit@09f8ddf#diff-001d0bba0217b0f5f5ead3d7620704c4a23e35f2b86b579ebb90f3a76004edfe

I am looking into adjusting the script


drive by observation: running blackfire run php vendor/bin/phpunit --list-tests-xml test-list.xml

has a bottleneck on AutowiredAttributeServicesExtension::processParameters

grafik

https://blackfire.io/profiles/2f4cb6e1-e2a5-4455-881f-e2eeff149dbe/graph

not sure this is expected

@staabm
Copy link
Contributor

staabm commented Jun 13, 2025

please try this fix: staabm@d024500

@ondrejmirtes
Copy link
Member Author

Trying it now. BTW you can submit PRs against (based on) other branches too, which would cause it being merged into this PR 😊

@ondrejmirtes ondrejmirtes marked this pull request as ready for review June 13, 2025 20:07
@phpstan-bot
Copy link
Collaborator

This pull request has been marked as ready for review.

@ondrejmirtes
Copy link
Member Author

On my computer:

PHPUnit 11: Time: 00:52.399, Memory: 432.00 MB
PHPUnit 10: Time: 00:51.174, Memory: 462.00 MB

Awesome! 👏

@ondrejmirtes ondrejmirtes merged commit a421da1 into 2.1.x Jun 13, 2025
418 of 421 checks passed
@ondrejmirtes ondrejmirtes deleted the phpunit-11 branch June 13, 2025 20:09
@ondrejmirtes
Copy link
Member Author

And as expected, we're not the only ones benefiting from that: https://x.com/spudley99/status/1933485562025128390

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants