Skip to content

Fix Flakeguard race attribution #17119

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kalverra merged 2 commits into from
Apr 4, 2025
Merged

Fix Flakeguard race attribution #17119

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6a84581
Fix flakeguard race attribution
kalverra Apr 4, 2025
27b031e
Update flakeguard version
kalverra Apr 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 46 additions & 43 deletions .github/workflows/flakeguard.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -487,7 +487,9 @@ jobs:

# Read the summary from the generated report
summary=$(jq -c '.summary_data' ./flakeguard-report/all-test-report.json)
unable_to_attribute_test=$(jq 'any(.results.[]; .test_name | startswith("UnableToAttribute"))')
echo "summary=$summary" >> $GITHUB_OUTPUT
echo "unable_to_attribute_test=$unable_to_attribute_test" >> $GITHUB_OUTPUT

- name: Upload All Test Report as Artifact
if: ${{ (success() || failure()) && fromJSON(steps.results.outputs.summary).total_runs > 0 }}
Expand Down Expand Up @@ -685,45 +687,46 @@ jobs:
]
}

- name: Send general Slack message
uses: slackapi/slack-github-action@6c661ce58804a1a20f6dc5fbee7f0381b469e001 # v1.25.0
if: ${{ (success() || failure()) && inputs.slackNotificationAfterTestsChannelId != '' && fromJSON(steps.results.outputs.summary).flaky_tests == 0 && fromJSON(steps.results.outputs.summary).total_tests > 0 }}
id: slack
env:
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
with:
channel-id: ${{ inputs.slackNotificationAfterTestsChannelId }}
payload: |
{
"attachments": [
{
"color": "${{ contains(join(needs.*.result, ','), 'failure') && '#C62828' || contains(join(needs.*.result, ','), 'cancelled') && '#FFA000' || '2E7D32' }}",
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "Flaky Test Detector for `${{ steps.set_project_path_pretty.outputs.path }}` project - ${{ contains(join(needs.*.result, ','), 'failure') && 'Failed :x:' || contains(join(needs.*.result, ','), 'cancelled') && 'Was cancelled :warning:' || 'Passed :white_check_mark:' }}"
}
},
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "${{ inputs.runAllTests == true && format('Ran all tests for `{0}` branch.', env.GIT_HEAD_REF) || format('Ran changed tests between `{0}` and `{1}` (`{2}`).', env.GIT_BASE_REF, needs.get-tests.outputs.git_head_short_sha, env.GIT_HEAD_REF) }}"
}
},
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "${{ inputs.runAllTests == true && format('<{0}/{1}/actions/runs/{2}|View Flaky Detector Details>', github.server_url, github.repository, github.run_id) || format('<{0}/{1}/actions/runs/{2}|View Flaky Detector Details> | <{3}/compare/{4}...{5}#files_bucket|Compare Changes>{6}', github.server_url, github.repository, github.run_id, inputs.repoUrl, inputs.baseRef, needs.get-tests.outputs.git_head_sha, github.event_name == 'pull_request' && format(' | <{0}|View PR>', github.event.pull_request.html_url) || '') }}"
}
}
]
}
]
}
# This has mostly been noise. We can re-enable it if we find it necessary
Copy link
Contributor

@sebawo sebawo Apr 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kalverra please add TODO and JIRA ticket

# - name: Send general Slack message
# uses: slackapi/slack-github-action@6c661ce58804a1a20f6dc5fbee7f0381b469e001 # v1.25.0
# if: ${{ (success() || failure()) && inputs.slackNotificationAfterTestsChannelId != '' && fromJSON(steps.results.outputs.summary).flaky_tests == 0 && fromJSON(steps.results.outputs.summary).total_tests > 0 }}
# id: slack
# env:
# SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
# with:
# channel-id: ${{ inputs.slackNotificationAfterTestsChannelId }}
# payload: |
# {
# "attachments": [
# {
# "color": "${{ contains(join(needs.*.result, ','), 'failure') && '#C62828' || contains(join(needs.*.result, ','), 'cancelled') && '#FFA000' || '2E7D32' }}",
# "blocks": [
# {
# "type": "section",
# "text": {
# "type": "mrkdwn",
# "text": "Flaky Test Detector for `${{ steps.set_project_path_pretty.outputs.path }}` project - ${{ contains(join(needs.*.result, ','), 'failure') && 'Failed :x:' || contains(join(needs.*.result, ','), 'cancelled') && 'Was cancelled :warning:' || 'Passed :white_check_mark:' }}"
# }
# },
# {
# "type": "section",
# "text": {
# "type": "mrkdwn",
# "text": "${{ inputs.runAllTests == true && format('Ran all tests for `{0}` branch.', env.GIT_HEAD_REF) || format('Ran changed tests between `{0}` and `{1}` (`{2}`).', env.GIT_BASE_REF, needs.get-tests.outputs.git_head_short_sha, env.GIT_HEAD_REF) }}"
# }
# },
# {
# "type": "section",
# "text": {
# "type": "mrkdwn",
# "text": "${{ inputs.runAllTests == true && format('<{0}/{1}/actions/runs/{2}|View Flaky Detector Details>', github.server_url, github.repository, github.run_id) || format('<{0}/{1}/actions/runs/{2}|View Flaky Detector Details> | <{3}/compare/{4}...{5}#files_bucket|Compare Changes>{6}', github.server_url, github.repository, github.run_id, inputs.repoUrl, inputs.baseRef, needs.get-tests.outputs.git_head_sha, github.event_name == 'pull_request' && format(' | <{0}|View PR>', github.event.pull_request.html_url) || '') }}"
# }
# }
# ]
# }
# ]
# }

- name: Send Slack message for Flakeguard Errors
if: ${{ (success() || failure()) && inputs.slackNotificationAfterTestsChannelId != '' && (steps.check-errors.conclusion == 'failure' || steps.results.conclusion == 'failure' || steps.generate-report.conclusion == 'failure') }}
Expand Down Expand Up @@ -757,8 +760,8 @@ jobs:
]
}

- name: Send Slack message for Found Panics
if: ${{ (success() || failure()) && inputs.slackNotificationAfterTestsChannelId != '' && fromJSON(steps.results.outputs.summary).panicked_tests > 0 }}
- name: Send Slack message for UnableToAttribute tests
if: ${{ (success() || failure()) && inputs.slackNotificationAfterTestsChannelId != '' && steps.results.outputs.summary == 'true' }}
uses: slackapi/slack-github-action@6c661ce58804a1a20f6dc5fbee7f0381b469e001 # v1.25.0
env:
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
Expand All @@ -774,14 +777,14 @@ jobs:
"type": "header",
"text": {
"type": "plain_text",
"text": "Flakeguard Found a Panic :warning:"
"text": "Flakeguard Unable to Attribute a Test Due to Panic or Race :warning:"
}
},
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "<@U01Q4N37KFG> ${{ format('<{0}/{1}/actions/runs/{2}|See details>', github.server_url, github.repository, github.run_id) }} and add to panic detection scenarios."
"text": "<@U01Q4N37KFG> <@U04DYU1KLGJ> ${{ format('<{0}/{1}/actions/runs/{2}|See details>', github.server_url, github.repository, github.run_id) }} and add to panic detection scenarios."
}
}
]
Expand Down
Loading