[BUGFIX] Respect composer mode only extension in FunctionalTestCase

[DanielSiepmann]

The typo3/testing-framework creates functional test instances
as "classic" mode instances, writing a PackageStates.php file
composed using composer information, provided and handled by the
internal ComposerPackageManager class. Extensions in the state
file are not sorted based on their dependencies and suggestions.

To mitigate this and having a correctly sorted extension state
file, the PackageCollection service has been introduced with
the goal to resort the extension state file after the initial
write to provide the correct sorted extension state.

The extension sorting within the state file is important, as the
TYPO3 Core uses this sorting to loop over extensions to collect
information from the extensions, for example TCA, TCAOverrides,
ext_localconf.php and other things. Package sorting is here very
important to allow extensions to reconfigure or change the stuff
from other extensions, which is guaranteed in normal "composer"
and "classic" mode instances.

For "classic" mode instances, only the ext_emconf.php file is
taken into account and "composer.json" as the source of thruth
for "composer" mode instances since TYPO3 v12, which made the
ext_emconf.php file obsolete for extensions only installed
with composer.

Many agencies removed the optional ext_emconf.php file for
project local path extensions like a sitepackage to remove
the maintenance burden, which is a valid case.

Sadly, the PackageCollection adopted from the TYPO3 Core
PackageManager did not reflected this case and failed for
extensions to properly sort only on extension dependencies
and keys due the mix of extension key and composer package
name handling. Extension depending on another extension
failed to be sorted correctly with following exception:

UnexpectedValueException: The package "extension_key"
depends on "composer/package-key" which is not present
in the system.

This change modifies the PackageCollection implementation
to take only TYPO3 extension and system extension into account
for dependency resolving and sorting, using ext_emconf.php
depends/suggest information as first source and falling back
to composer require and suggestion information.

Resolves: #541
Releases: main, 8

[oliverklee]

Should this only go into the main branch, or would it make sense to backport it to 7.x as well?

[DanielSiepmann]

I'm fine with both. I personally need this in one project where we apply this as a patch.
I'd leave this up to the maintainer to decide.

[DanielSiepmann]

Looks like there is somehow an issue with loading order within acceptance tests … I'll investiage.

[liayn]

Hm... Feature?
I would rather call this a serious bugfix...

All of our internal extensions skip emconf (obviously) so this hits us on every upgrade from v11 to v12 of each extension. :-(

[sbuerk]

on it ..

[sbuerk]

I guess there is more work to do, and need more time to investigate this further. So no quick merge of this (not saying that it is a bad change !).

Thanks @DanielSiepmann for a first solution suggestion and the tests !!!! That helps to investigate that more in detail.

Continue on this the next days. Stay tuned !

[liayn]

@sbuerk Thanks a bunch. I just wonder what we changed in Core v12 that this hits so hard? I mean this worked flawlessly - at least for us - with Core v11 and TF 7

[sbuerk]

@sbuerk Thanks a bunch. I just wonder what we changed in Core v12 that this hits so hard? I mean this worked flawlessly - at least for us - with Core v11 and TF 7

TF8 added the sorting of packages to ensure that extension in functional instances has the correct sorting, normally TYPO3 would take care of in a legacy instance. As the package state is written by tf after symlinking to it, the state file must be sorted. For that, a custom sorting thing has been implemented in TF to take care of that.

With TF7 / v11 or v12 you have to take care to mention extension in the correct order yourself.

For example, of you have a extension which requries to be executed after another extension. For example having suggest defined so your extension runs after for example EXT:dashboard to register widgets etc.

The PackageCollection just assums (use code which assums) that extension still have ext_emconf.php in the extension.

There are lot more details to this, but we need to address some things in this PackageCollection now.

[sbuerk]

@DanielSiepmann Modified your commit/pull-request.

Note, testest the very same commit with #634 against TYPO3 Core with https://review.typo3.org/c/Packages/TYPO3.CMS/+/87253 and also
in a nightly.

Is the change okay for you ?

[d-s-codappix]

Sure. I've replaced all ext_emconf.php files of our test fixture extensions with corresponding composer.json files and tested that state with the latest version of this PR. This version works fine for us in our v12 project with TF 8.