Skip send-report notifications if all tests passed via --only-on-failures #1900
Description
Is your feature request related to a problem? Please describe.
Currently, the send-report command always sends a Slack notification even when all tests have passed, which creates noise—reminders pile up even though nothing is actually failing.
Describe the solution you'd like
Add a new option --only-on-failures (or an equivalent name) to send-report so that, when specified, Slack notifications are skipped only if all tests have passed. Here’s an example implementation:
def send_test_results_summary(
self,
days_back: int,
test_runs_amount: int,
disable_passed_test_metrics: bool = False,
bucket_website_url: Optional[str] = None,
include_description: bool = False,
only_on_failures: bool = False
) -> bool:
tests_api = TestsAPI(
dbt_runner=self.internal_dbt_runner,
days_back=days_back,
invocations_per_test=test_runs_amount,
disable_passed_test_metrics=disable_passed_test_metrics,
)
invocations_api = InvocationsAPI(
dbt_runner=self.internal_dbt_runner,
)
invocation = invocations_api.get_test_invocation_from_filter(
self.selector_filter.to_selector_filter_schema()
)
summary_test_results = tests_api.get_test_results_summary(
filter=self.selector_filter.to_selector_filter_schema(),
dbt_invocation=invocation,
)
all_passed = all(t.status == "passed" for t in summary_test_results)
if self.slack_client:
if only_on_failures and all_passed:
send_succeeded = True
else:
send_succeeded = self.slack_client.send_message(
channel_name=self.config.slack_channel_name,
message=SlackReportSummaryMessageBuilder().get_slack_message(
test_results=summary_test_results,
bucket_website_url=bucket_website_url,
include_description=include_description,
filter=self.selector_filter.to_selector_filter_schema(),
days_back=days_back,
env=self.config.env,
project_name=self.config.project_name,
),
)
else:
send_succeeded = False
self.execution_properties[
"sent_test_results_summary_successfully"
] = send_succeeded
self.execution_properties[
"test_results_all_passed"
] = all_passed
self.success = send_succeeded
if send_succeeded:
logger.info("Sent test results summary to Slack")
self.execution_properties["success"] = self.success
return self.success
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
- When the flag is not specified, the behavior remains unchanged (notifications are always sent).
- Do not break the existing updates to execution_properties or the return-value logic; when skipping, the method should still be treated as successful (return True).
Would you be willing to contribute this feature?
Yes. I can implement the change and add corresponding tests. With your guidance, I will submit a PR.