Storage restore problems

[iloveeclipse]

Similar to eclipse-jdt/eclipse.jdt.core#3960.

Due the Eclipse.org storage outage we are asked what do we want to do with Jenkins: https://www.eclipse.org/lists/eclipse.org-committers/msg01496.html

To avoid prolonging the outage any further, we have two options:

We can provision a Jenkins instance with the last known version of the Jenkins data. 
This data is the last snapshot prior to migration to the storage cluster, and, depending on when your instance was migrated, may be weeks, or up to several months old.

We can provision a new, bare Jenkins instance for your project. 
If your build is defined in pipelines alongside your code, recovery may be more straightforward.

You can also choose to wait for the storage cluster to recover, so that we can either continue its usage, or extract data from it to provision a new instance. 
We do not, however, have an ETA for this recovery, nor do we know with any level of certainty that it actually will recover.

I'm for option 1 (restore from backup) because if it fails, we can always use 2.

I do not want to wait anymore, as we have lot of merged but not "SDK tested" PR's, one unfinished aggregator problem with pending compiler PR, and a big pending JDT PR which would also need to be merged / verified soon before M3.

Please comment how do we want to proceed.

[merks]

The backups are quite recent for us. Restore now seems best.

[MohananRahul]

I still have some doubts regarding the other Jenkins jobs in releng, as they don’t seem to be available in the repositories. Are they included in the mentioned backups?

[iloveeclipse]

Are they included in the mentioned backups?

I guess we will see what is inside only after we try to restore?

[merks]

The backup for releng is extremely recent:

https://docs.google.com/spreadsheets/d/13zNUbwAHkYacC193HFwgZOd1fqn-N9cyzXf_lHke07s/edit?gid=0#gid=0

[akurtakov]

I start to think that such accidents might be beneficial in the long run to make projects and people not take "shortcuts" and actively work towards standardizing and improving so recoveries can be faster and simpler.

[HannesWell]

Are they included in the mentioned backups?

I expect that, yes.

[HannesWell]

In the RelEng JIPP (https://ci.eclipse.org/releng/ currently not reachable), we definitively still have some jobs that are not yet recorded in code (Jenkins pipeline or GH workflow where suitable). However these jobs are not frequently used so an older state would be fine. In fact I removed a few of them recently (not sure if before or after the back-up), but that can be redone easily. Furthermore we should propably rerun the seed-job once to ensure that all recent jobs are properly created (but that's also trivial).

So for the RelEng JIPP I'm strongly in favor option 1 (restore from back-up)!

For the other JIPPs part of the SDK, I'm not aware of any relevant jobs not defined as code, at least for platform, equinox and PDE (I cannot speak with certainty for JDT). So at least for the former three a new JIPP (option 2) would be ok as well, option 1 (restore from backup) would also be fine and would probably some work to even recreate the (simple) job configurations to use the Jenkinsfiles.

[iloveeclipse]

So should I create a ticket to restore releng & platform instances from backup?
For PDE @HannesWell could do that, and JDT team seem to be undecided yet but I count 3 voices for 1) too and no other proposals yet.

[HannesWell]

Yes, please.
Just did it for PDE: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/5972

[iloveeclipse]

Created https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/5973 for platform & platform releng

[iloveeclipse]

@tjwatson : what is about equinox?

[tjwatson]

Apologize for the slow response, I opened https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/6020

[iloveeclipse]

FYI: see problems we observe in jdt jenkins because of https://repo.eclipse.org/content/repositories incomplete backup state.

I have no idea what that mean, how many artifacts were hosted there, and where all the things that were hosted there should be retrieved, but it sounds like we will not have SDK build soon :-(

[iloveeclipse]

PS: I'm going mostly offline till the end of the week because of the Holiday in Germany.

[iloveeclipse]

Jenkins is back, going over to: #3006