-
Notifications
You must be signed in to change notification settings - Fork 1
Description
Describe the bug
The codecov github bot will leave its usual coverage report comment when tests pass, and edit that comment to report test failures, but will not edit that comment again once tests pass (to include the coverage report) - it gets 'stuck.'
To Reproduce
I've experienced this in a github monorepo (python/typescript) which uploads multiple coverage reports using codecov/[email protected], and pytest reports using codecov/test-results-action@v1.
Steps to reproduce the behavior:
- Open a PR with passing tests
- See the codecov bot leave a PR comment/review with coverage results
- Push a change to the PR that results in broken test(s)
- See the bot edit the PR comment reporting the failing tests
- Push a change so all tests pass
- Check the comment again - it'll be 'stale' (not edited) and still reflect the broken test(s)
Expected behavior
If the bot edits its own comment from its usual coverage report, to instead reflect broken tests, I expect it to edit that comment again once tests are passing, to reflect the change in coverage.
Screenshots
Example part of the initial 'normal' coverage report comment:
Which then gets edited in-place like this:
But does not get re-edited once all tests are passing.
Additional context
This is our current config:
comment:
layout: "header, diff, flags, files, feedback, components, footer"
behavior: default
require_changes: false
require_base: false
require_head: false
hide_project_coverage: false
codecov:
branch: develop
require_ci_to_pass: false
notify:
wait_for_ci: true
notify_error: true
coverage:
status:
project: trueThere is one scenario I've found where it actually will edit the comment again: If codecov.notify.wait_for_ci is set to false. But, since we have multiple jobs uploading coverage reports, that results in the bot editing the comment multiple times, each time only reflecting one of the coverage reports (which is not surprising, but only creates a separate problem for our case)