Skip to content

Conversation

@DavidSpickett
Copy link
Collaborator

No description provided.

@DavidSpickett DavidSpickett force-pushed the ci-report-bash branch 9 times, most recently from 615958c to 0c7d1ad Compare October 25, 2024 12:22
The CI builds now send the results of every lit run to a unique file.
This means we can read them all to make a combined report for all
tests.

This report will be shown as an "annotation" in the build results:
https://buildkite.com/docs/agent/v3/cli-annotate#creating-an-annotation

Here is an example: https://buildkite.com/llvm-project/github-pull-requests/builds/112546
(make sure it is showing "All" instead of "Failures")

This is an alternative to using the existing Buildkite plugin:
https://github.com/buildkite-plugins/junit-annotate-buildkite-plugin

As the plugin is:
* Specific to Buildkite, and we may move away from Buildkite.
* Requires docker, unless we were to fork it ourselves.
* Does not let you customise the report format unless again,
  we make our own fork.

Annotations use GitHub's flavour of Markdown so the main code in the
script generates that text. There is an extra "style" argument generated
to make the formatting nicer in Buildkite.

"context" is the name of the annotation that will be created. By using
different context names for Linux and Windows results we get 2 separate
annotations.

The script also handles calling the buildkite-agent. This makes passing
extra arguments to the agent easier, rather than piping the output of
this script into the agent.

In the future we can remove the agent part of it and simply use
the report content. Either printed to stdout or as a comment on
the GitHub PR.
@DavidSpickett
Copy link
Collaborator Author

DavidSpickett commented Oct 25, 2024

Example intentionally failing build: https://buildkite.com/llvm-project/github-pull-requests/builds/113045#_
Example unintentionally failing build: https://buildkite.com/llvm-project/github-pull-requests/builds/113055#_

The second example should not be failing but this modulemap test is for whatever reason, probably a mistake from me. If this approach is chosen I'll fix that and expand this to Windows and cleanup everything else.

But it's at a point where you can look at the changes to .ci/monolithic-linux.sh and compare to #113447 which uses the lit feature to get unique file names.

The complexity is caused mostly by our choice to pass multiple check- targets to ninja, which was pointed out on #113447 as well.

@DavidSpickett
Copy link
Collaborator Author

Abandoning in favour of other methods.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant