Update packagist to API p2

[elrayle]

The composer API in use for importing from the package manager is deprecated as of Sept 1. It now returns a 403 for all requests.

This PR updates packagist calls to the p2 endpoint. The format of that endpoint has changed. The mock data is the result of the actual call to that endpoint.

curl "https://repo.packagist.org/p2/symfony/polyfill-mbstring.json"

Added a test to directly test that _getRegistryData processes data in the p2 format.

[elrayle]

Deployed to dev and ran a test with cd:/composer/packagist/symfony/clock/v6.4.24. I confirmed that it was processed and extracted the expected license.

[qtomlinson]

Great catch, and thanks for your contribution! This will definitely improve Composer package harvesting.

Just a note: ClearlyDefined also tries to harvest source (e.g., GitHub) components during source discovery, using registry metadata. Composer v2 metadata (p2 API) is compressed—only the latest version has full metadata, while older versions are diffs and may miss key fields like homepage, dist, and source. These fields are used during source discovery in composerExtract.js.

To restore full metadata for any version, you can use logic similar to Composer\MetadataMinifier\MetadataMinifier::expand() (see "Getting the Package Data, Using the Composer v2 metadata" at https://packagist.org/apidoc). Alternatively, the package API provides complete metadata for all versions (see "Getting the Package Data, Using the API" at https://packagist.org/apidoc).

This pull request addresses most of the cases. Some cases that rely on the full registry data can be addressed in a separate pull request.

[elrayle]

Adding in the expand process is more involved and better to push off to another PR.

[qtomlinson]

@elrayle In the integration test run, Failure Case 1 (expected to return the same definition as production for composer/packagist/symfony/polyfill-mbstring/v1.28.0) and Failure Case 3 (expected to return the same notice as production for the same package and version) are likely caused by changes in the registry data. In both cases, the discrepancies are related to the project website information retrieved from the registry. Expanding the registry data for older versions likely will resolve these issues.