Codecov bot

[daniellepintz]

I find codecov bot can be pretty noisy on PRs- do people get value from it? I wonder if it’s possible to make it only comment if code coverage regresses by a significant amount?

[justusschock]

Personally I ignore this most of the time since it often reports wrong results (not accumulated between all CI runs or not all CI finished yet). This means that even when we set it to comment only for a coverage decrease it would comment as soon as the first CI run finishes.

[daniellepintz]

thanks for your input @justusschock! I sent out a PR to hopefully ameliorate the situation #10199

[justusschock]

@daniellepitz it's worth a try but probably that won't really help. E.g if our CPU CI finishes early and GPU CI takes a long time (due to long queue), it will still spam those messages since > 5% of the code (everything GPU related) was not yet covered.