Skip to content

Conversation

@HannesWell
Copy link
Member

Since 4.28 respectively 2023-06 the property 'p2.mirrorsURL' is not added to release repositories anymore, which effectively disables the use of mirrors.
This happened accidentally because the JDKs installed at the Jenkins agents running the release process became insufficient then.

In order to simplify the release process, the P2-repositories produced by I-builds alredy have their final name set.

Fixes #3201

This is currently a draft as it has to be tested first.

@HannesWell
Copy link
Member Author

HannesWell commented Jul 25, 2025

An alternative would be to just fix the old approach and use a suitable JDK (for Java-21) and maybe simplify it's setup.
But I noticed that the org.eclipse.wtp.releng.tools.addRepoProperties application used before, doesn't update the artifact.xml.xz as well:
https://gitlab.eclipse.org/eclipse/webtools/releng/webtools-releng/-/blob/2c01e14765f8d298c73196c1364d5cd632d7855a/plugins/org.eclipse.wtp.releng.tools/src/org/eclipse/wtp/releng/tools/AddRepoProperties.java

This would render this fix useless as the xz file is preferred by P2 for quite some time.
Furthermore this different approach should be faster and is simpler to implement.

<includeAllSources>true</includeAllSources>
<repositoryName>Eclipse ${releaseVersion} repository (for ${releaseYear}-${releaseMonth})</repositoryName>
<extraArtifactRepositoryProperties>
<p2.statsURI>https://download.eclipse.org/stats</p2.statsURI>
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to add this p2.statsURI?

@laeubi
Copy link
Contributor

laeubi commented Jul 26, 2025

The less tools, the better, so if we can use plain Tycho it would be preferable... If anything is missing there we can add it.

To account for different types of builds, one can have a profile that only enable the settings if a given ENV variable is set.

@HannesWell
Copy link
Member Author

The less tools, the better, so if we can use plain Tycho it would be preferable... If anything is missing there we can add it.

It would be great if this could be simplified, but currently I didn't found a better solution to add or just alter a property in an existing P2 repository. Neither with Tycho nor with the P2 ant tasks.
But if you know better, please let me know.
I'm not sure if it's worth just for this case to add an extra mojo to add a property to a repository.

To account for different types of builds, one can have a profile that only enable the settings if a given ENV variable is set.

The problem is that we don't have a final release build but just promote a selected I-build repository (which is basically copying it and adding that property).

@laeubi
Copy link
Contributor

laeubi commented Jul 26, 2025

We already have some mojos that cope with modify a repository:

https://tycho.eclipseprojects.io/doc/latest/tycho-p2-repository-plugin/plugin-info.html

tycho-p2-repository:remap-artifacts-to-m2-repo for example modify an artifact repository and tycho-p2-repository:fix-artifacts-metadata modify the metadata repo (and can recompress it as well afterwards).

These mojos can usually also be called as a CLI invocation so you do not need a real "build" to do so, a good example is the tycho-p2-director:director that allows to run the director application without the need to download a full eclipse.

@HannesWell
Copy link
Member Author

I know all of them and tried to use them for this task, but didn't find a solution.

and tycho-p2-repository:fix-artifacts-metadata modify the metadata repo (and can recompress it as well afterwards).

If you think it's suitable I can look into creating a modify-artifacts-metadata or alike mojo for the tycho-p2-repository-plugin plugin. Which would then support modifying the name adding or removing properties. Later more modifications could be implemented, if desired (thinking about the extra IU of the equinox.executable feature were about to eliminate).
One question would be if it should just be for artifacts or also for content repos? Or the mojo could be generalized to be modify-metadata and have an option to select artifacts, content or both (by default).
And in order to save bandwidth, the source could be any remote and only the resulting artifact/content.jar/xml.xz would be outputted locally (but could also overwrite a local repository).
WDYT?

@laeubi
Copy link
Contributor

laeubi commented Jul 26, 2025

WDYT?

As mentioned before, if something is missing we should add it :-)

As mojos are quite "cheap" we can have many specialized or even a more generic modify-repository, it is often more a matter of how much one would do in one step, usually one does not really care about the difference between metadata / artifact repo as they both being colocated is the norm.

Also some repos are maybe a bit more complicated than needed since they date back to where we have a split between tycho-maven and tycho-osgi so usually it can be as simple as read the data (maybe even as MetadataIO/ArtifactIO) change some little bit and write back.

@HannesWell
Copy link
Member Author

HannesWell commented Aug 14, 2025

Now this change is reworked to use Maven and the new Tycho Mojo (eclipse-tycho/tycho#5209) to add the p2.mirrorsURL property. All my testing looked good so far.
In contrast to previous values of p2.mirrorsURL like https://www.eclipse.org/downloads/download.php?format=xml&file=/eclipse/updates/4.25/R-4.25-202208311800, I have now stream-lined it a bit to be just
https://www.eclipse.org/downloads/download.php?file=/eclipse/updates/4.37/R-4.37-202508121800. I.e. the format=xml parameter is removed. This parameter is added by P2 already, when mirrors are queried:
https://github.com/eclipse-equinox/p2/blob/0814abdb5e02e75c943956301ddfacbfd97dec10/bundles/org.eclipse.equinox.p2.artifact.repository/src/org/eclipse/equinox/internal/p2/artifact/repository/MirrorSelector.java#L338

Furthermore it causes difficulties with the ampersand, which is actually illegal as XML-attribute value in this form, but Tycho seems to read it without considering the correct escaping. Having just one parameter solves this riddle (actually I have verified that the previous form is treated as desired).

The other noticeable change in the results is that the name of the p2-repository is now set already when it's build to
Eclipse ${releaseVersion}. This affects I-build, milestone, RC and release repositories.
Before just the release repo was named:
Eclipse Project Repository for ${TRAIN_NAME}
Note this is the 'eclipse' repository, not the SimRel repo.

@HannesWell HannesWell marked this pull request as ready for review August 14, 2025 00:17
Since 4.28 respectively 2023-06 the property 'p2.mirrorsURL' is not
added to release repositories anymore, which effectively disables the
use of mirrors.
This happened accidentally because the JDKs installed at the Jenkins
agents running the release process became insufficient then.
Instead of just providing the JDK, the whole approach to add this
property is reworked:
- The property is added using Maven and the new tycho mojo
'tycho-p2-repository:modify-repository-properties'
- The repo is copied diretly on the download-server from the source to
its target location. Only the artifacts.jar/xml.xz files are downloaded
to the build workspace, modified as described above and then copied back
to the target location.

In order to simplify the release process, the P2-repositories produced
by I-builds already have their final name set.

Fixes eclipse-platform#3201
@HannesWell HannesWell force-pushed the ensure-p2.mirrorsURL branch from 383ae3a to d196968 Compare August 14, 2025 15:17
@HannesWell HannesWell merged commit 25f8540 into eclipse-platform:master Aug 14, 2025
6 checks passed
@HannesWell HannesWell deleted the ensure-p2.mirrorsURL branch August 14, 2025 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Release P2 repositories miss p2.mirrorsURL property since 4.28

2 participants