Skip to content

Commit 2419317

Browse files
authored
Convert flake report from Python to Go (#9334)
## What changed? * Convert the flake report we use to monitor functional test reliability from invoking the `tringa` cli tool and a Python script to only a Go script that fetches the artifacts, parses the JUnit, and creates the report. * This also fixes an issue where `tringa` only found the last 20 tests, the Go script will iterate through all failures in the last `N` days ## Why? Everything is in a single executable making it easier to test and extend. ## How did you test it? - [X] built - [X] run locally and tested manually - [ ] covered by existing tests - [X] added new unit test(s) - [ ] added new functional test(s) ## Potential risks Minimal, this is only a tooling change.
1 parent a283511 commit 2419317

File tree

16 files changed

+1792
-833
lines changed

16 files changed

+1792
-833
lines changed

.github/workflows/flaky-tests-report.yml

Lines changed: 14 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -40,65 +40,33 @@ jobs:
4040
token: ${{ steps.generate_token.outputs.token }}
4141
fetch-depth: 0
4242

43-
- name: Set up Python
44-
uses: actions/setup-python@v5
43+
- name: Set up Go
44+
uses: actions/setup-go@v5
4545
with:
46-
python-version: '3.13'
46+
go-version-file: 'go.mod'
4747

48-
- name: Install uv
49-
uses: astral-sh/setup-uv@v4
50-
with:
51-
version: ">=0.8.0"
52-
53-
- name: Install Python dependencies
54-
run: |
55-
cd tools/flakes
56-
uv sync
57-
58-
- name: Install tringa
59-
run: |
60-
uv tool install git+https://github.com/dandavison/tringa@main --force
61-
62-
- name: Run tringa
63-
id: generate-output-file
64-
env:
65-
GH_TOKEN: ${{ steps.generate_token.outputs.token }}
66-
DAYS_PARAM: ${{ github.event.inputs.days || '7' }}
67-
run: |
68-
set -euo pipefail
69-
70-
# Create output directory
71-
mkdir -p tools/flakes/out
72-
73-
tringa --json --since-days "$DAYS_PARAM" repo sql \
74-
'select classname, name, artifact from test where passed = false and skipped = false order by classname, name, artifact desc' \
75-
--branch main \
76-
--workflow-id 80591745 \
77-
https://github.com/temporalio/temporal > tools/flakes/out/out.json
78-
79-
echo "✅ Tringa command completed"
80-
echo "📊 Output file size: $(wc -c < tools/flakes/out/out.json) bytes"
81-
echo "📄 Full output file:"
82-
cat tools/flakes/out/out.json
83-
84-
- name: Run Python script to process flaky tests
48+
- name: Generate flaky test report
8549
id: process-flaky-tests
8650
env:
51+
GH_TOKEN: ${{ steps.generate_token.outputs.token }}
8752
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}
53+
DAYS_PARAM: ${{ github.event.inputs.days || '7' }}
54+
MAX_LINKS_PARAM: ${{ github.event.inputs.max_links || '3' }}
8855
RUN_ID: ${{ github.run_id }}
8956
REF_NAME: ${{ github.ref_name }}
9057
SHA: ${{ github.sha }}
91-
MAX_LINKS_PARAM: ${{ github.event.inputs.max_links || '3' }}
9258
run: |
93-
set -x
94-
cd tools/flakes && uv run main.py \
95-
--file out/out.json \
59+
set -euo pipefail
60+
61+
go run ./cmd/tools/flakereport generate \
62+
--days "$DAYS_PARAM" \
63+
--max-links "$MAX_LINKS_PARAM" \
64+
--output-dir tools/flakes/out \
9665
--github-summary \
9766
--slack-webhook "$SLACK_WEBHOOK" \
9867
--run-id "$RUN_ID" \
9968
--ref-name "$REF_NAME" \
100-
--sha "$SHA" \
101-
--max-links "$MAX_LINKS_PARAM"
69+
--sha "$SHA"
10270
10371
- name: Upload generated reports
10472
uses: actions/upload-artifact@v4

cmd/tools/flakereport/main.go

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
package main
2+
3+
import (
4+
"fmt"
5+
"os"
6+
7+
"go.temporal.io/server/tools/flakereport"
8+
)
9+
10+
func main() {
11+
if err := flakereport.NewCliApp().Run(os.Args); err != nil {
12+
fmt.Fprintf(os.Stderr, "Error running flakereport: %v\n", err)
13+
os.Exit(1)
14+
}
15+
}

0 commit comments

Comments
 (0)