Skip to content

htmlreport: Add new command that creates HTML reports of SIG test case failures#92

Merged
dhiller merged 1 commit intomainfrom
html-report
Sep 11, 2025
Merged

htmlreport: Add new command that creates HTML reports of SIG test case failures#92
dhiller merged 1 commit intomainfrom
html-report

Conversation

@brianmcarey
Copy link
Member

What this PR does / why we need it:

This new command will primarily used by periodic prow jobs to generate daily reports on each specific SIG's tescase failures. These reports will help SIG members to identify and tackle the faiures.

The reports will be stored on the kubevirt GCS bucket and can be linked to from the ci-health README where we track the number of SIG e2e failures

Here is an example report for SIG Network:
screenshot-2025-08-19-14:41:23

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

/cc @dhiller @xpivarc

Checklist

This checklist is not enforcing, but it's a reminder of items that could be relevant to every PR.
Approvers are expected to review this list.

Release note:


@kubevirt-bot kubevirt-bot added dco-signoff: yes Indicates the PR's author has DCO signed all their commits. size/L labels Aug 19, 2025
Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes - here's some feedback:

  • Avoid using global variables (SigFailures and jobRegex); instead declare them inside Generate so the function is reentrant and testable.
  • On critical failures (e.g. reading results.json or creating the output file), return an error or exit rather than just logging, and defer closing any opened files to prevent resource leaks.
  • When printing the CLI success message, include the full path to the generated report file (including its filename) instead of just the directory for clearer feedback.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Avoid using global variables (SigFailures and jobRegex); instead declare them inside Generate so the function is reentrant and testable.
- On critical failures (e.g. reading results.json or creating the output file), return an error or exit rather than just logging, and defer closing any opened files to prevent resource leaks.
- When printing the CLI success message, include the full path to the generated report file (including its filename) instead of just the directory for clearer feedback.

## Individual Comments

### Comment 1
<location> `README.md:171` </location>
<code_context>
+
+### html-report command
+
+A command that generates html reports of test case failures per SIG. To create a report for SIG Compute failures:
+```
+go run ./cmd/html-report --sig compute --results-path ./output/kubevirt/kubevirt/results.json --path /tmp/
</code_context>

<issue_to_address>
Capitalize 'html' to 'HTML' for correctness.

Use 'HTML' instead of 'html' to align with standard terminology.
</issue_to_address>

### Comment 2
<location> `README.md:176` </location>
<code_context>
+go run ./cmd/html-report --sig compute --results-path ./output/kubevirt/kubevirt/results.json --path /tmp/
+```
+
+This should create a html report called sig-compute-failure-report.html under /tmp/.
</code_context>

<issue_to_address>
Use 'an HTML report' instead of 'a html report'.

Use 'an' before 'HTML' because it starts with a vowel sound, and ensure 'HTML' is capitalized.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

README.md Outdated

### html-report command

A command that generates html reports of test case failures per SIG. To create a report for SIG Compute failures:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (typo): Capitalize 'html' to 'HTML' for correctness.

Use 'HTML' instead of 'html' to align with standard terminology.

README.md Outdated
go run ./cmd/html-report --sig compute --results-path ./output/kubevirt/kubevirt/results.json --path /tmp/
```

This should create a html report called sig-compute-failure-report.html under /tmp/.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (typo): Use 'an HTML report' instead of 'a html report'.

Use 'an' before 'HTML' because it starts with a vowel sound, and ensure 'HTML' is capitalized.

@brianmcarey brianmcarey force-pushed the html-report branch 2 times, most recently from 374e0b7 to 6e4fd0f Compare August 19, 2025 14:31
brianmcarey added a commit to brianmcarey/project-infra that referenced this pull request Sep 4, 2025
…rom ci-health

This requires kubevirt/ci-health#92

Signed-off-by: Brian Carey <bcarey@redhat.com>
Copy link
Contributor

@dhiller dhiller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@brianmcarey great idea, and a very useful report. Some questions inline.


*/ -}}

{{- /* sig-failure-report.gohtml */ -}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this comment necessary?

@brianmcarey
Copy link
Member Author

@dhiller I've cleaned up the duplication - can you please take a look?

Copy link
Contributor

@dhiller dhiller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks much better, thanks!

One last comment though - I believe it would be beneficial to use the junit library as mentioned here: #92 (comment)

…e failures

This new command will primarily used by periodic prow jobs to generate
daily reports on each specific SIG's tescase failures. These reports
will help SIG members to identify and tackle the faiures.

The reports will be stored on the kubevirt GCS bucket and can be linked
to from the ci-health README where we track the number of SIG e2e
failures

Signed-off-by: Brian Carey <bcarey@redhat.com>
@brianmcarey
Copy link
Member Author

Looks much better, thanks!

One last comment though - I believe it would be beneficial to use the junit library as mentioned here: #92 (comment)

thanks for the great suggestion @dhiller - applied the changes and it helped to simplify things.

Copy link
Contributor

@dhiller dhiller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @brianmcarey! 🎉

@dhiller dhiller merged commit d8b511e into main Sep 11, 2025
2 checks passed
brianmcarey added a commit to brianmcarey/project-infra that referenced this pull request Sep 12, 2025
…rom ci-health

This requires kubevirt/ci-health#92

Signed-off-by: Brian Carey <bcarey@redhat.com>
kubevirt-bot pushed a commit to kubevirt/project-infra that referenced this pull request Sep 12, 2025
…rom ci-health (#4345)

This requires kubevirt/ci-health#92

Signed-off-by: Brian Carey <bcarey@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has DCO signed all their commits. size/XXL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants