Job Notifications

[IamSandeepGunda]

We've been exploring the Databricks Notification Destinations and set up a Slack destination to notify on errors and when the job is over it's runtime thresholds.

Is there a way to set these up in the deployment.yml?

Currently, if I set it up on the UI, it's wiped away next time i perform a dbx deploy... to that job.

[SamarpanRai-TomTom]

Any updates on this?

[IamSandeepGunda]

We made it work by adding those webhook notifications and duration thresholds for the job on the UI then comparing with the past job to identify the new keys. Then added them to the deployment.yml

Basically adding these keys to a job definition.

  webhook_notifications:
    on_failure:
      - id: "webhook-destination-id"
    on_duration_warning_threshold_exceeded:
      - id: "webhook-destination-id"
  email_notifications:
    no_alert_for_skipped_runs: false
  notification_settings:
    no_alert_for_skipped_runs: false
    no_alert_for_canceled_runs: false
  health:
    rules:
      - metric: "RUN_DURATION_SECONDS"
        op: "GREATER_THAN"
        value: 1800

Hope this helps.

[renardeinside]

hi @IamSandeepGunda ,
Sorry for the late reply. Unfortunately I rarely check the GH discussions page - could you please create a GH issue out of it?
I assume the problem is with strict model validation that cuts off the provided payload.