Skip to content

Flakey tests due to GitHub API rate limits #468

@swissspidy

Description

@swissspidy

Sometimes tests involving the GitHub REST API fail due to rate limits. Example:

001 Example: | one-time-login | 0.4.0   | https://github.com/danielbachhuber/one-time-login/releases/latest | # features/upgradables.feature:244
      When I run `wp plugin install [https://github.com/danielbachhuber/one-time-login/releases/latest`](https://github.com/danielbachhuber/one-time-login/releases/latest%60)        # features/upgradables.feature:221
        $ wp plugin install https://github.com/danielbachhuber/one-time-login/releases/latest
        
        Error: API rate limit exceeded for 145.132.102.176. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)
        https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting
        In order to pass the token to WP-CLI, you need to use the GITHUB_TOKEN environment variable.
        cwd: /tmp/wp-cli-test-run--68dde0e15dd076.39224548/
        run time: 0.36239576339722
        exit status: 1 (RuntimeException)

The command already uses the GITHUB_TOKEN env var if available. But we don't really pass the one from the workflow to Behat.

We'd need to do that here:

https://github.com/wp-cli/.github/blob/f25b21f651e0d8d90956289b88eb7bb21c9785dd/.github/workflows/reusable-testing.yml#L460-L480

However, I think we actually need to pass the secret in the caller workflow:

https://docs.github.com/en/actions/how-tos/reuse-automations/reuse-workflows#passing-inputs-and-secrets-to-a-reusable-workflow

Makes me wonder whether this usage here actually even works:

https://github.com/wp-cli/.github/blob/f25b21f651e0d8d90956289b88eb7bb21c9785dd/.github/workflows/reusable-testing.yml#L277-L285

Not to mention that we should then limit the permissions drastically to be read-only:

https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#defining-access-for-the-github_token-scopes

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions