mergify: support backports automation with labels

[v1v]

What does this PR do?

• Enable https://docs.mergify.com/workflow for:
  • Creating backports automatically using GitHub backport labels when the original PR gets merged.
  • Notify when backports are stalled.
  • Notify the backports policy
  • Create PRs even with conflicts.
• Enable copy GitHub labels from the original PR to the backported PRs. Mergify does not support this feature out of the box, see github-actions: support for copying the backport labels apm-server#15190
• Enable merge conflict detection. See github-actions(pre-commit): detect merge conflicts and shebangs apm-server#15319

Why is it important/What is the impact to the user?

Use the same backport mechanism as done in some other projects. While still support for the former approach.

Tasks

• Wait for Mergify to be installed
• Create GitHub labels

Follow-ups

• Delete https://github.com/elastic/logstash-filter-elastic_integration/blob/main/.github/workflows/pr_backporter.yml

Related issues

Relates elastic/logstash#16933

[mashhurs]

Can we leverage the branches defined in the repo for CI purpose?

[v1v]

Can we leverage the branches defined in the repo for CI purpose?

In GitHub actions, the branches section accept static values or wildcards , if you mean being explicit, we can, or if you mean to add 7.17, I can include it. However, I cannot think of any issues for running the pre-commit workflow with any push commit in any of those wildcard branches. Or do you have any other concerns?

[v1v]

BTW, if you want to avoid maintaining branches.json, we have a public bucket to help with those details too, see https://storage.googleapis.com/artifacts-api/snapshots/branches.json

[mashhurs]

My main concern is maintenance:

• do we need to update pre-defined valid .mergify.yml on major & minor releases?.
• if so how other teams are modifying to avoid manual effort?

[v1v]

do we need to update pre-defined valid .mergify.yml on major & minor releases?.
if so how other teams are modifying to avoid manual effort?

I understand, and thanks for pointing those concerns! This PR is still draft and gathering those questions can help us to have something workable and fully functional.

We have automated that as part of the FF scripts teams use. For instance:

1. APM Server uses this process
2. Beats uses an internal process to do the same.

In those cases, there are PRs with those changes created automatically. See elastic/apm-server#11760 or elastic/beats#41706

That's something we can help with, for instance, I added the support for creating the backport labels for the logstash in elastic/logstash@f2ccbce (#16937). I'll add the support to modify the mergify file in logstash and will review how those bumps work for this project to do the same

[mashhurs]

Nice, thank you for sharing valuable info.
For this case, we need to also include a manual (in the Logstash release) and mention both logstash and elastic_integration repos.

[mashhurs]

LGTM with minor comment to remove 7.17.

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: 97ae7ab

Failed CI Steps

• Unit test for 8.19.0-SNAPSHOT, snapshot: true
• Integration test for 8.19.0-SNAPSHOT, snapshot: true
• Unit test for 9.0.0-SNAPSHOT, snapshot: true
• Integration test for 9.0.0-SNAPSHOT, snapshot: true
• Unit test for 9.1.0-SNAPSHOT, snapshot: true
• Integration test for 9.1.0-SNAPSHOT, snapshot: true

History

• 💔 Build #523 failed f583456
• 💚 Build #493 succeeded 4b75a84

[mashhurs]

LGTM!

[mergify]

This pull request does not have a backport label. Could you fix it @v1v? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit.
• backport-8.x is the label to automatically backport to the 8.x branch.
• If no backport is necessary, please add the backport-skip label