[CI] Make CI Jobs Upload Failures to Premerge Advisor

[boomanaiden154]

This allows the premerge advisor infrastructure to start ingesting data
so we can get to work on making failure explanations work.

[llvmbot]

@llvm/pr-subscribers-tablegen

@llvm/pr-subscribers-infrastructure

Author: Aiden Grossman (boomanaiden154)

Changes

This allows the premerge advisor infrastructure to start ingesting data
so we can get to work on making failure explanations work.

---

Full diff: https://github.com/llvm/llvm-project/pull/163276.diff

2 Files Affected:

• (added) .ci/premerge_advisor_upload.py (+48)
• (modified) .ci/utils.sh (+6)

diff --git a/.ci/premerge_advisor_upload.py b/.ci/premerge_advisor_upload.py
new file mode 100644
index 0000000000000..27e45bdd7f589
--- /dev/null
+++ b/.ci/premerge_advisor_upload.py
@@ -0,0 +1,48 @@
+# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+"""Script for uploading results to the premerge advisor."""
+
+import argparse
+import os
+
+import requests
+
+import generate_test_report_lib
+
+PREMERGE_ADVISOR_URL = "http://premerge-advisor:5000/upload"
+
+
+def main(commit_sha, workflow_run_number, build_log_files):
+    junit_objects, ninja_logs = generate_test_report_lib.load_info_from_files(
+        build_log_files
+    )
+    test_failures = generate_test_report_lib.get_failures(junit_objects)
+    source = "pull_request" if "GITHUB_ACTIONS" in os.environ else "postcommit"
+    failure_info = {
+        "source_type": source,
+        "base_commit_sha": commit_sha,
+        "source_id": workflow_run_number,
+        "failures": [],
+    }
+    if test_failures:
+        for name, failure_message in test_failures:
+            failure_info["failures"].append({"name": name, "message": failure_message})
+    else:
+        ninja_failures = generate_test_report_lib.find_failure_in_ninja_logs(ninja_logs)
+        for name, failure_message in ninja_failures:
+            failure_info["failures"].append({"name": name, "message": failure_message})
+    requests.post(PREMERGE_ADVISOR_URL, json=failure_info)
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("commit_sha", help="The base commit SHA for the test.")
+    parser.add_argument("workflow_run_number", help="The run number from GHA.")
+    parser.add_argument(
+        "build_log_files", help="Paths to JUnit report files and ninja logs.", nargs="*"
+    )
+    args = parser.parse_args()
+
+    main(args.commit_sha, args.workflow_run_number, args.build_log_files)
+    main()
diff --git a/.ci/utils.sh b/.ci/utils.sh
index 5d32968babb39..9aefcf212e457 100644
--- a/.ci/utils.sh
+++ b/.ci/utils.sh
@@ -38,6 +38,12 @@ function at-exit {
       $retcode "${BUILD_DIR}"/test-results.*.xml "${MONOREPO_ROOT}"/ninja*.log \
       >> $GITHUB_STEP_SUMMARY
   fi
+
+  if [[ "$retcode" != "0" ]]; then
+    python "${MONOREPO_ROOT}"/.ci/premerge_advisor_upload.py \
+      $(git rev-parse HEAD~1) $GITHUB_RUN_NUMBER \
+      "${BUILD_DIR}"/test-results.*.xml "${MONOREPO_ROOT}"/ninja*.log
+  fi
 }
 trap at-exit EXIT
 

[github-actions]

✅ With the latest revision this PR passed the Python code formatter.

[statham-arm]

Hi @boomanaiden154 ,

I had pre-commit CI fail in #163941 just now, and looking at the logs, it appears to be a Python issue related to this newly added script:

+ python /home/gha/actions-runner/_work/llvm-project/llvm-project/.ci/premerge_advisor_upload.py a43c0cf77ff52c844a841bab00cd0b54063709ef 141046 /home/gha/actions-runner/_work/llvm-project/llvm-project/build/test-results.7uydifj0.xml /home/gha/actions-runner/_work/llvm-project/llvm-project/build/test-results._8ln509k.xml /home/gha/actions-runner/_work/llvm-project/llvm-project/build/test-results.b7807bka.xml /home/gha/actions-runner/_work/llvm-project/llvm-project/build/test-results.cyqiqhj4.xml /home/gha/actions-runner/_work/llvm-project/llvm-project/build/test-results.gtqbnlys.xml /home/gha/actions-runner/_work/llvm-project/llvm-project/build/test-results.j1puvxhy.xml /home/gha/actions-runner/_work/llvm-project/llvm-project/build/test-results.o45gylf5.xml /home/gha/actions-runner/_work/llvm-project/llvm-project/build/test-results.tqr_iadk.xml /home/gha/actions-runner/_work/llvm-project/llvm-project/build/test-results.yt8a5zth.xml /home/gha/actions-runner/_work/llvm-project/llvm-project/ninja.log
Traceback (most recent call last):
  File "/home/gha/actions-runner/_work/llvm-project/llvm-project/.ci/premerge_advisor_upload.py", line 58, in <module>
    main(args.commit_sha, args.workflow_run_number, args.build_log_files)
  File "/home/gha/actions-runner/_work/llvm-project/llvm-project/.ci/premerge_advisor_upload.py", line 35, in main
    for name, failure_message in test_failures:
        ^^^^^^^^^^^^^^^^^^^^^
ValueError: too many values to unpack (expected 2)
##[error]Process completed with exit code 1.

Do you know anything about this? (Or have I totally misunderstood, and am asking in the wrong place?)

[statham-arm]

Ah, I see that 38ce641 got there at about the same time and has fixed it. Sorry for the noise.

(Also, this wouldn't have gone wrong in my CI run without an actual test failure to mis-unpack, so I should go and look at that 🙂)