Feed the next streams update site repository with the contents of M1 / M2 / RC1 / RC2 prior to release

[laeubi]

With the discussion here:

• [Proposal] Switch master to next release after M3 #863

and the first steps to automate most of the "opening" process e.g. here

• Add a reusable workflow to update the sub-project to the next release #883
• Add a workflow to prepare the next release cycle #876

I noticed that there is actual a little gap in the process regarding the update repository. As noted by @sravanlakkimsetti here, it is usually filled only 5 days before the release and the stream is opened before that.

As this is used to decide what is the "base", this means our builds and verifications running for a while against an possibly outdated baseline if versions are already increased for the next stream.

My proposal therefore would be to fill the repo (e.g in this case https://download.eclipse.org/eclipse/updates/4.27/ ) with the content of the M+RC builds as soon as they are available and not open the stream before the RC1 is published. Then as suggested by @iloveeclipse it would later on even be possible to open the stream after M3.

[laeubi]

Technically the https://download.eclipse.org/eclipse/updates/4.27/ already exits (it is just quite empty) and is a composite repo, so one could think of having simply the following sub-directories:

• M1
• M2
• M3
• RC1
• RC2
• release (what might be a composite with versioned folders like today directly in the root)

The composite repo will then always only point to the most recent one (and finally to the release folder), that way one can either reference to a fixed item, or to the "most recent" in buildscripts or updatesites, and because it has a fixed structure (no timestamps!) could be easily derived by automation workflows from a release number.

[sravanlakkimsetti]

@akurtakov what do you think? Right now we are not creating milestones repositories. This would require us to create milestone repos.

[akurtakov]

AFAICS, it's about using the release repo not milestones. I'm fine with doing it that way as long as manual work gets less.

[sravanlakkimsetti]

AFAICS, it's about using the release repo not milestones. I'm fine with doing it that way as long as manual work gets less.

yes We will need to generate milestone repos and add them to the composite of the release repo. I think this can be done.

I don't think we should revive milestones repo. but we should generate individual miletone repositories and add them to release repo.

[merks]

That's much like what happens for https://download.eclipse.org/releases/2023-03/ where we accumulate each M*, RC*, and R repository, we point only to the most recent one from the composite, and then we delete all but the release repository at the end of the release cycle.

[laeubi]

@merks can you maybe give some advice or help to archive this? I think M3 is probably to late but maybe we can start this with RC1 (should be this week?).

[merks]

So many of these "maintain a composite" are a bunch shell script glue and everyone has variants thereof...

For SimRel the following effectively copies the staging repo to a repo nested in the appropriate releases folder:

https://ci.eclipse.org/simrel/view/Releng%20jobs/job/simrel.releng.promoteToReleases/

It also prepares the composites jars but with a different name which is then used to make the new repository visible at the correct date and time:

https://ci.eclipse.org/simrel/view/Releng%20jobs/job/simrel.releng.makeVisible/

I think all the job configurations are visible in you're logged in.

Lots of scripts here:

https://git.eclipse.org/c/simrel/org.eclipse.simrel.tools.git/tree/

I make no claims that these are pretty or ideal... I'm much happier using the reusable JustJ tools mechanism but I don't think that would be easy to apply to the platform's existing and more complex update site structure...

[sravanlakkimsetti]

@merks can you maybe give some advice or help to archive this? I think M3 is probably to late but maybe we can start this with RC1 (should be this week?).

Please take a look at

1. promoteSites.sh This has code to generate renamed repository. We are not generating separate milestone repositories now. but we used to do that please take a look. as far as I know, we are still using that part of the code for final repository
2. makeVisible.sh this has the composite creation code.

[HannesWell]

Is this still relevant since we now initialize the new I-build repos with the previous cycle's RC2 upon creation and the I-build repo is never empty?

eclipse.platform.releng.aggregator/JenkinsJobs/Releng/prepareNextDevCycle.jenkinsfile

Lines 182 to 186 in b7dc3cb

	build job: 'Releng/modifyP2CompositeRepository', wait: true, propagate: true, parameters: [
	string(name: 'repositoryPath', value: "eclipse/updates/${NEXT_RELEASE_VERSION}-I-builds"),
	string(name: 'add', value: "https://download.eclipse.org/eclipse/updates/${PREVIOUS_RELEASE_VERSION}-I-builds/${PREVIOUS_RELEASE_CANDIDATE_I_BUILD}/")
	// Size-limit is not relevant, the repository is initially empty.
	]

[HannesWell]

Is this still relevant since we now initialize the new I-build repos with the previous cycle's RC2 upon creation and the I-build repo is never empty?

Closing this now, as I believe there is no gap anymore. Please reopen if I missed something.