GitLab CI: No need to wait for gitlab pipeline to finish (#4797)

We have switched to using amrex-gitlab-ci-reporter to report back the
status.

Author: WeiqunZhang

.github/workflows/trigger-hpsf-gitlab-ci.yml

@@ -61,7 +61,6 @@ jobs:
           PIPELINE_URL="https://gitlab.spack.io/${{ env.GITLAB_PROJECT_PATH }}/-/pipelines/$PIPELINE_ID"
           echo "Pipeline URL: $PIPELINE_URL"
           echo "pipeline_url=$PIPELINE_URL" >> $GITHUB_OUTPUT
-          echo "pipeline_id=$PIPELINE_ID" >> $GITHUB_OUTPUT
 
       - name: Post status to GitHub PR
         shell: bash
@@ -74,44 +73,3 @@ jobs:
             -f body="$COMMENT" || echo "Failed to post comment to GitHub"
         env:
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-
-      - name: Wait for GitLab pipeline to finish
-        id: wait
-        shell: bash
-        run: |
-          PIPELINE_ID=${{ steps.trigger.outputs.pipeline_id }}
-
-          echo "Waiting on GitLab pipeline $PIPELINE_ID..."
-
-          STATUS="running"
-          while [[ "$STATUS" == "running" || "$STATUS" == "pending" ]]; do
-            sleep 900  # 15 minutes
-
-            STATUS=$(curl -s \
-              "https://gitlab.spack.io/api/v4/projects/${{ env.GITLAB_PROJECT_ID }}/pipelines/$PIPELINE_ID" \
-              | jq -r '.status')
-
-            if [ "$STATUS" == "null" ] || [ -z "$STATUS" ]; then
-              echo "Failed to get pipeline status. Response: $RESPONSE"
-              echo "final_status=error" >> $GITHUB_OUTPUT
-              exit 1
-            fi
-
-            echo "Status: $STATUS"
-          done
-
-          echo "final_status=$STATUS" >> $GITHUB_OUTPUT
-
-      - name: Post result back to GitHub PR
-        shell: bash
-        run: |
-          STATUS=${{ steps.wait.outputs.final_status }}
-          PIPELINE_ID=${{ steps.trigger.outputs.pipeline_id }}
-          PIPELINE_URL=${{ steps.trigger.outputs.pipeline_url }}
-          COMMENT="GitLab CI ${PIPELINE_ID} finished with status: **$STATUS**. See details at ${PIPELINE_URL}."
-
-          gh api \
-            repos/${{ github.repository }}/issues/${{ github.event.issue.number }}/comments \
-            -f body="$COMMENT" || echo "Failed to post comment to GitHub"
-        env:
-          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.gitlab/README.md

@@ -36,6 +36,24 @@ pipeline: `.gitlab/hpsf-gitlab-ci.yml`. The PR comment triggered job pulls
 the PR branch from GitHub first before running tests. For this approach to
 work, we store a pipeline trigger (obtained from GitLab's `Settings -> CI/CD
 -> Pipeline trigger tokens`) as a secret at GitHub's `Settings -> Secrets
-and variables -> Actions -> Repository secrets`. The GitHub workflow waits
-for the result of the GitLab pipeline result and posts the final status and
-a link to the result as a comment.
+and variables -> Actions -> Repository secrets`.
+
+After the GitLab pipeline finishes, its `.post` stage will post the final
+status and a link to the results back to the GitHub PR as a comment. This is
+done through a GitHub App that we built and installed in the AMReX
+repository. The App was created via
+https://github.com/organizations/AMReX-Codes/settings/apps. It does not need
+"Webhook" access. For repository permissions, it only needs read & write to
+pull requests so it can create PR comments. The app requires a private key,
+which you generate during the setup. After the app was installed in the
+amrex repository, we got an installation ID. We then stored the app ID,
+installation ID and the private key in GitLab's `Settings -> CI/CD ->
+Variables`. The app ID isn't a secret. So you can store it as clear text. In
+fact, GitLab does not allow 7-digit masked variables anyway. The
+installation ID is also not senstive, but nevertheless we stored it as
+protected and masked. The private key is a secret that must be protected and
+masked. We also diabled "Expand" for all of these variables because the CI
+script doesn't need variable expansion. GitLab seems to have a bug that
+prevents saving the private key as a multi-line value, and saving it as a
+file didn't work either. So we encoded it with `base64 -w0` to turn it into
+a single line. That's why in the GitLab CI script we have to decode it.

.gitlab/hpsf-gitlab-ci.yml

@@ -136,7 +136,7 @@ post_github_comment:
               if job['name'] == current_job_name:
                   continue
 
-              if job['status'] in ['failed', 'canceld']:
+              if job['status'] in ['failed', 'canceled']:
                   final_status = 'failed'
                   break # Found a failure, no need to check further
 
@@ -160,13 +160,12 @@ post_github_comment:
 
       # Get installation access token
       installation_id = os.environ['GITHUB_APP_INSTALLATION_ID']
-      headers = {
-          'Authorization': f'Bearer {token}',
-          'Accept': 'application/vnd.github+json'
-      }
       token_response = requests.post(
           f'https://api.github.com/app/installations/{installation_id}/access_tokens',
-          headers=headers
+          headers = {
+              'Authorization': f'Bearer {token}',
+              'Accept': 'application/vnd.github+json'
+          }
       )
       if token_response.status_code != 201:
          print(f"Failed to get access token: HTTP {token_response.status_code}")
@@ -186,11 +185,9 @@ post_github_comment:
           },
           json={'body': comment_body}
       )
-      del token_response
       if comment_response.status_code != 201:
           print(f"Failed to post comment: HTTP {comment_response.status_code}")
           sys.exit(1)
-      del comment_response
       print("Successfully posted comment to GitHub PR")
       EOF
   when: always