Conditionally auto-approve dependabot PRs

[cadmiumcat]

What problem does this pull request solve?

We want to set up a flow to to auto-approve some dependabot PRs. PRs will only
be approved if:

• they are not npm updates, and
• the update is only a version patch

Trello card

Things to consider when reviewing

It would be great to sense check the workflow against what we're trying to achieve.

The workflow is configured to approve and merge a PR only if:

• The PR was raised by Dependabot
• The PR is a patch version and not part of an NPM package
• All checks have passed

Otherwise, those steps will be skipped and the PR goes through the regular review process. You can see this in action in this PR, where all there a bunch of skipped checks relating to the auto-approving and auto-merging.

General things to consider

• Is there enough debug logging in the workflow?
• Are there any name changes you'd suggest?
• Can the workflow be optimised in any way?
• Ensure you consider the wider context.
• Is it clear what the code is doing?
• Do the commit messages explain why the changes were made?
• Has all relevant documentation been updated?

Local testing

The only way to test if this really works will be to merge it into the main branch unfortunately. Below are the two testing strategies Cat and I took.

Cat

I wanted to try this workflow locally using act but it keep getting an error at the step where we ask it to get the dependabot metadata.

[Dependabot auto-approve/dependabot]   💬  ::debug::Verifying the job is for an authentic Dependabot Pull Request
[Dependabot auto-approve/dependabot]   ❗  ::error::Api Error: (404) Not Found
[Dependabot auto-approve/dependabot]   ❌  Failure - Main dependabot/fetch-metadata@v2
[Dependabot auto-approve/dependabot] exitcode '1': failure
[Dependabot auto-approve/dependabot] 🏁  Job failed
<!-- If this section isn't relevant for your PR feel free to edit or remove it -->

Sarah

I ended up forking the forms-runner repo so I could test changes on a safe main branch (see fork here: https://github.com/sarahseewhy/forms-runner). I created a new file review_apps_on_pr_change.yml in the forked repo and copied the contents from this branch.

This was a useful approach because it allowed me to close and reopen Dependabot PRs to test the workflow and learn quite a bit.

Here's an example of the Actions at work on the forked repo: https://github.com/sarahseewhy/forms-runner/actions/runs/15110638823/job/42469149737?pr=39

However, forked repos don't have the Github permission to actually auto-merge so I'm returning to this branch and pull request.

You can take a similar approach by:

1. Forking the repo
2. Creating a new file review_apps_on_pr_change.yml
3. Copy and pasting the contents of review_apps_on_pr_change.yml from this branch into the forked repo
4. Configure the fork to have the same Dependabot and branch settings as this repo
5. Find a Dependabot PR which matches our criteria
6. Close the dependabot PR and then reopen it by typing @dependabot reopen in the comment section

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[chao-xian]

🚀

[sarahseewhy]

I'm going to merge this pull request in an hour (after a meeting). Then either update a relevant Dependabot PR or close/reopen a Dependabot PR (which ever works to test). I'll also check the other Dependabot PRs, especially ones which we don't want merged (like npm or minor bumps).

If anything goes wrong I'll revert the merge and we can figure out next steps.

[lfdebrux]

Another thought (sorry)... We should probably add branch protections to all dependabot/** branches preventing pushes... Otherwise I think an attacker could add malicious commits to the PR and get them auto-approved and merged (if they are quick).

I got the idea from https://boostsecurity.io/blog/weaponizing-dependabot-pwn-request-at-its-finest#:~:text=To%20prevent%20branch%20protection%20bypasses.

[cadmiumcat]

Another thought (sorry)... We should probably add branch protections to all dependabot/** branches preventing pushes... Otherwise I think an attacker could add malicious commits to the PR and get them auto-approved and merged (if they are quick).

I got the idea from https://boostsecurity.io/blog/weaponizing-dependabot-pwn-request-at-its-finest#:~:text=To%20prevent%20branch%20protection%20bypasses.

It's a good point. My concern with branch protections is that we'd then have to open a PR to commit changes to dependabot PRs, which is not a great flow for us.
I was wondering if we could check the contributors to the PR: if any of them is not dependabot, then we don't autoapprove/merge. But I can only find references to the author of the PR

AP-Hunt is gone :(

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[github-actions]

🎉 A review copy of this PR has been deployed! It is made of up two components

1. A review copy of forms-runner
2. A production copy of forms-admin

Important

Not all of the functionality of forms-runner is present in review apps.
Functionality such as sending emails, file upload, and S3 submission types are
deliberately disabled for the sake of simplifying review apps.

You should use the full dev environment to test the functionality which is disabled here.

It may take 5 minutes or so for the application to be fully deployed and working. If it still isn't ready
after 5 minutes, there may be something wrong with the ECS task. You will need to go to the integration AWS account
to debug, or otherwise ask an infrastructure person.

For the sign in details and more information, see the review apps wiki page.