Skip to content

Maestro (local) always fails on PRs from forks: secrets are unavailable, so checkEnv.js throws after a full APK build and emulator boot #7653

Description

@andreBurnt

Steps to reproduce

  1. Open a pull request from a fork (any change; this week's examples: Make NotificationResultProcessor.emit wait until the batch is processed #7640, Let the user give a reason when removing a message #7538, Let the user cancel a message forward that never completes #7526, Keep the push foreground service alive while notifications are fetched #7639, Fix crash due to JSON.stringify on a cirular structure in injectes js. #7600, Stop rescanning message bodies for emoji only on every recomposition #7531, Fall back to the message body when the formatted body has nothing to render #7523).
  2. Let the Maestro (local) workflow run (for first-time contributors a maintainer has to approve it first).
  3. Watch the run: Build APK succeeds (4-12 min), then in Maestro test suite the emulator boots and the very first Maestro step fails:
Run ./scripts/checkEnv.js...maestro.js.JsEvaluationException: Fatal: missing env variable MAESTRO_PASSWORD

This is deterministic. MAESTRO_PASSWORD / MAESTRO_RECOVERY_KEY are injected from repository secrets (.github/workflows/maestro-local.yml L104-106), GitHub never exposes secrets to pull_request runs from forks, and .maestro/scripts/checkEnv.js aborts on the first missing variable. Every fork-PR run that reached the Maestro job in the last week failed exactly this way, 11 of 11 (2026-08-29 to 2026-09-04): runs 33846962079, 33794480195, 33701035985, 33280438838, 33377021972, 33309328779, 33308370076, 33307026619, 33280438464, 33280437567, 33793877313 (each at https://github.com/element-hq/element-x-android/actions/runs/<id>). A further 27 fork-PR runs in the same 100-run window are sitting in action_required; approving any of them produces the same result.

Outcome

What did you expect?

The lane is skipped on PRs that cannot receive the secrets (or fails in a few seconds with a message saying why), so external contributors do not get a red check they cannot act on, maintainers do not approve runs that are guaranteed to fail, and the shared maestro-test concurrency slot is not consumed.

What happened instead?

  • A red Maestro test suite check on every external PR, indistinguishable from a real test failure.
  • Those 11 runs cost ~111 runner-minutes (mean 10.1 min/run: 6.6 min APK build + 3.5 min emulator boot) to reach the exception.
  • Each one also holds the global maestro-test concurrency group for ~3.5 min while internal PRs wait behind it (e.g. run 33280438464 held it 10:05:53-10:09:19 UTC on 2026-08-31 while 33380732765 was queued).
  • The guard that appears intended to handle forks is dead code. L82:
    if: (github.event_name == 'pull_request' && github.event.pull_request.fork == null) || github.event_name == 'workflow_dispatch'
    The pull_request object has no fork property (it lives at pull_request.head.repo.fork; gh api repos/element-hq/element-x-android/pulls/7647 --jq 'has("fork")' prints false), so the expression is null == null and always true. It has been that way since the workflow was introduced in 271f032 (Test using Maestro CLI + emulator instead of Cloud #4092).

Line numbers are at develop 66c5028 (2026-09-04).

Proposed fix

Skip both jobs for fork PRs at job level (the APK is only consumed by the Maestro job, so building it for a fork PR has no value either) and remove the step-level if: on the checkout at L81-82:

jobs:
  build-apk:
    if: github.event_name == 'workflow_dispatch' || github.event.pull_request.head.repo.full_name == github.repository
  maestro-cloud:
    if: github.event_name == 'workflow_dispatch' || github.event.pull_request.head.repo.full_name == github.repository

(needs: [build-apk] already skips maestro-cloud when build-apk is skipped, but the explicit condition keeps the intent readable.) If a visible signal on fork PRs is wanted instead of a skip, a tiny first job that fails fast before the APK build does that at near-zero cost.

Your phone model

n/a (CI: reactivecircus/android-emulator-runner, pixel_7_pro, API 33, x86_64)

Operating system version

n/a

Application version and app store

develop @ 66c5028, :app:assembleGplayDebug (CI build)

Homeserver

matrix.org (the CI test account)

Will you send logs?

No (the CI run logs linked above contain everything)

Will you submit a Pull Request?

No

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions