Create a bot account

[ringods]

For a set of Pulumiverse providers I maintain, I activated pulumi/ci-mgmt to keep the workflows up to date via automation. I'm using a Github Personal Access Token (PAT) to allow some workflows to create branches, push commits to it and create a linked pull request.

Working via a PAT is that any action is taking as if it was me. The downside of this is that I don't get any Github notifications for things I do "myself". Due to that, I missed that the Grafana provider couldn't be upgraded automatically:

pulumiverse/pulumi-grafana#240
pulumiverse/pulumi-grafana#242

Would it be possible to set up a separate "bot" user for such automation purposes? If the bot user performs these actions, and I assign myself as issue & PR assignee, then I will receive proper notifications on Github.

[omercnet]

I concur

leveraging the wonderful automations in ci-mgmt that depend on pulumi-bot, like auto merge, release tagging, etc. would increase velocity of bridged providers that are currently hand-managed

[tmeckel]

@ringods I did a quick research at GitHub and their documentation about those "bot accounts" is ... let's simply call it confusing. What type of "bot account" is needed here? As far as I understood the "bot account" shall be used to run the scripts (worklow) from ci-mgmt to create PRs that will (auto-)update the build infrastructure and even the (bridged) provider itself.

Main issue from my side is right now, aside I don't know what "bot user" we need here, how do we do this in a secure manner, i.e. we need a controlled way to "onboard" Pulumiverse provider repos onto the "bot account" and to ci-mgmt.

Oh and the documentation in ci-mgmt is written for Pulumi itself, because I don't see any possibility to specify that the provider is hosted in the pulumiverse org and NOT in the pulumi org.

[ringods]

@tmeckel I think the preferred way these days is to create Github App. I know it sounds confusing as we are not implementing an app, but a Github App acts as a separate identity, which is the core of my request here.

https://docs.github.com/en/apps/creating-github-apps/about-creating-github-apps/deciding-when-to-build-a-github-app#choosing-between-a-github-app-or-a-personal-access-token

[tmeckel]

@ringods I was thinking about this as well. Issue: the workflow must run as that app or be more precisely the PR must be created as the app. There is a documentation about how to do this https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/authenticating-as-a-github-app-installation#generating-an-installation-access-token but I don't want to expose the app installation id, which is required to create the JWT, simply for every repository. Seems not very secure to me.

https://github.com/actions/create-github-app-token
https://github.com/marketplace/actions/create-github-app-token
https://stackoverflow.com/a/74892482/874043

[tmeckel]

@ringods thought about to create a separate app for every provider repo that wants to automate things using an app, like the ci-mgmnt workflow (scripts). But the GitHub provider does not support this and I think this might be overkill.

https://docs.github.com/en/apps/sharing-github-apps/registering-a-github-app-from-a-manifest

And there is a limitation of 100 registered apps per org/user. Not that we would reach that number soon 🤣 , but the limitation exists.

https://docs.github.com/en/apps/creating-github-apps/registering-a-github-app/registering-a-github-app#about-registering-github-apps

[tmeckel]

Right now I think a meaningful solution would be to have a GitHub workflow like the https://github.com/pulumi/ci-mgmt/blob/master/.github/workflows/update-bridge-ecosystem-providers.yml in the ci-mgmt repository in the Pulumiverse infra repository, which iterates over all provider repos and creates update PRs if a .ci-mgmt.yaml is found. We could use a GitHub App for this or simply stick to the GitHub Admin token authentication, which should work as well.

[omercnet]

How can I help further this?

[ringods]

@tmeckel @simenandre it seems a Github App is not an easy setup.

Would it be possible that one of you creates a separate Github account which will act as the bot and store the login credentials in 1Password pulumiverse?

This is the same approach Pulumi uses (they have @pulumi-bot).

If this account is created, it needs to be added to the pulumiverse org. Then create a personal access token for this user with org wide permissions to read & write, execute workflows, create issues and PRs. Once you have this token, please configure it organization wide as a Github Action secret under the name PULUMI_PROVIDER_AUTOMATION_TOKEN.

[tmeckel]

@ringods so my plan (if my current customers would leave some room to breath for me) is actually to use an GitHub App. I configured an app in our own company GitHub organization successfully which is used to trigger other workflows implicitly (e.g. by pushing a tag) or explicitly by starting a workflow via the gh cli.

So I'd utilize this (new) GitHub app on pulumiverse/infra with a new workflow that would do the following:

1. load the defined list of provider repos from 02-repositories
2. for every repo definition it checks for property like auto_update: true.
   it will push the list of repo-names to an task output variable
3. A matrix build iterates over the repo-names output variable and does:
   3.1. checkout the provider repo
   3.2. check for an existing .ci-mgmt.yaml if not -> END
   3.3. execute pulumi/pulumi-upgrade-provider-action

Any comments on that?

[ringods]

@tmeckel in all honesty, I don't mind how it gets solved to have an identity that has permissions to run the currently generated workflows in an authenticated way. I dropped my previous message because I seemed to understand from the comments even earlier that a Github App is very difficult.

I'm not fond of the push model you propose because not all providers have adopted the pulumi/ci-mgmt and I don't want to introduce yet another config option. Can the Github App be configured so the current workflows work authenticated and from that identity?

[tmeckel]

@ringods The pulumi/pulumi-upgrade-provider-action can be configured to use a token of the GitHub App (https://github.com/pulumi/pulumi-upgrade-provider-action/blob/e247104aede3eb4641f48c8ad0ea9de9346f2457/action.yml#L142C25-L142C34), which in turn will list the created Pull Request from this particular app and will this will eventually send you (as a maintainer) a notification email that a new Pull Request exists.