edr monitor --teams-webhook giving error

[JyotsnaMalode]

Describe the bug
When I execute below command:
edr monitor --teams-webhook giving error

Getting below error in Power Automate workflow
Oops, we failed to format the alert ! -_-' Please share this with the Elementary team via https://elementary-data.com/community or a https://github.com/elementary-data/elementary/issues/new|GitHub issue."

To Reproduce
Steps to reproduce the behavior:

1. Run dbt tests
2. Execute command "edr monitor --teams-webhook "
3. Getting error as below, even though i have ?api-version=2016-06-01& in webhook url
   

Also getting below error in PowerAutomate

Expected behavior
I was expecting to see alerts in Teams channel

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

• Elementary CLI (edr) version: [e.g. 0.5.3], can be found by running pip show elementary-data
  Version: 0.15.1
• Elementary dbt package version: [e.g. 0.4.1], can be found in packages.yml file
  version: 0.14.1
• dbt version you're using [e.g. 1.8.1]
  1.7.10
• Data warehouse [e.g. snowflake]
  Redshift
• Infrastructure details (e.g. operating system, prod / dev / staging, deployment infra, CI system, etc)

Additional context
Add any other context about the problem here.

Would you be willing to contribute a fix for this issue?
Let us know if you're willing to contribute so that we'll be able to provide guidance.

[dapollak]

Hey @JyotsnaMalode,
How do you pass the webhook url to the CLI ?

[JyotsnaMalode]

hi @dapollak , I am using below command
edr monitor --teams-webhook "url"

I am using Teams workflow and not Teams webhook as it will soon be not supported

When I check the raw output for "When a Teams webhook request is received" in Wrokflow, I get below data
"title": "Oops, we failed to format the alert ! --' Please share this with the Elementary team via https://elementary-data.com/community or a https://github.com/elementary-data/elementary/issues/new|GitHub issue.",_

[mriveraFacephi]

Hi, same error here. pymsteams appears not to have a solution yet.

A workaround could be sent alerts in plain text and allowing ok status code between 200 <= x < 300.
Not the best solution but at least it allows to continue receive alerts.

[JyotsnaMalode]

Thanks @mriveraFacephi , will try your suggestion.

[Jonathan-MW]

I also tried to setup MS Teams alerts with the new Power Automate workflow method, because the connector already does not work anymore for us.
Getting this error: "ERROR — Unable to send alert via Teams: 'Response' object has no attribute 'body'"

[JyotsnaMalode]

Update - workaround suggested by @mriveraFacephi worked for me to receive the text message.

[FredRodrigues]

Hey @Jonathan-MW , I am having the same issue. Did you try the solution provided by @mriveraFacephi ?

I tried but does not seem to work :/

[FredRodrigues]

In short, we will need for an update from pymsteams in order to solve this issue or is there any other option we can use to solve this problem?

[FrankTub]

@FredRodrigues I think we could also suggest to the elementary team to start using https://github.com/AeroFlorian/MsTeamsAdaptiveCard/tree/master instead of pymsteams to switch to the new MS Teams way of doing things.

[FrankTub]

@ellakz I just looked into this issue and verified that moving the existing webhooks we have to the new webhooks as defined by MS Teams, but then I also get an error:

ERROR — Unable to send alert fallback via Teams: 'Response' object has no attribute 'body'

When looking at the documentation of MS Teams we will have to migrate to a new solution before 31 januari 2025:

URLs for all webhook-based connectors must be updated by January 31, 2025 for the connectors to continue posting messages into Teams

What do you think about my above suggestion to start using another package to send messages to MS Teams?

[MikaKerman]

Hi @JyotsnaMalode,
Thanks for raising the issue!

As the deprecation deadline for Office 365 Connectors approaches, we're working on adding support for sending alerts to Teams via workflows.
Workflows only support the new Adaptive Cards format (see the note here), whereas elementary currently uses the older message card format supported by Office 365 Connectors.

Currently, when we can't format the message, we send a fallback message with JSON data to ensure you're notified about the alert. However, this fallback message is also not in the Adaptive Cards format, so it fails in workflows, which is why you see the error in Power Automate. You might also see another error related to the original alert.

Until we roll out support for the Adaptive Cards format, I suggest using the Incoming Webhook application as a temporary solution, if possible. You can pass the URL generated by the Incoming Webhook to edr monitor, and alerts will route correctly.

Regarding the ERROR log, @FrankTub, this was a minor bug in logging the actual error when sending the alert. I've deployed a quick fix for this today, so it should now print a more accurate message.

[FrankTub]

@MikaKerman , can I test this by pinning the version of elementary to use as the master branch?

[FrankTub]

@MikaKerman , I tried to test it out for you but this results in worse behavior in my opinion, see explanation below.

I pinned Elementary to the last commit on master in my requirements.txt by adding it to:

elementary-data @ git+https://github.com/elementary-data/elementary@master

Next I ran something like:

edr monitor --profiles-dir $DBT_PROFILES_DIR --teams-webhook "$MS_TEAMS_DATA_QUALITY_WEBHOOK_URL" --report-url "$DATA_QUALITY_REPORT_URL"

This does not result in an error, but no messages are sent to the Teams webhook:

    ________                          __                  
   / ____/ /__  ____ ___  ___  ____  / /_____ ________  __
  / __/ / / _ \/ __ `__ \/ _ \/ __ \/ __/ __ `/ ___/ / / /
 / /___/ /  __/ / / / / /  __/ / / / /_/ /_/ / /  / /_/ / 
/_____/_/\___/_/ /_/ /_/\___/_/ /_/\__/\__,_/_/   \__, /  
                                                 /____/   

Any feedback and suggestions are welcomed! join our community here - https://bit.ly/slack-elementary

2024-11-25 11:32:52 — INFO — Running with edr=0.16.1
2024-11-25 11:32:54 — INFO — edr (0.16.1) and Elementary's dbt package (0.16.2) are compatible.
2024-11-25 11:32:55 — INFO — Elementary's database and schema: '"staging.elementary"'
2024-11-25 11:32:55 — INFO — Running internal dbt run to populate alerts
2024-11-25 11:32:55 — INFO — Running dbt command run -m elementary_cli.alerts.alerts_v2 --project-dir /Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/elementary/monitor/dbt_project --profiles-dir /Users/frank.tubbing/.dbt --vars {"days_back": 1}
2024-11-25 11:32:58 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_pending_alerts", "macro_args": {"days_back": 1, "type": null}} --project-dir /Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/elementary/monitor/dbt_project --profiles-dir /Users/frank.tubbing/.dbt
2024-11-25 11:33:00 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_last_alert_sent_times", "macro_args": {"days_back": 1, "type": null}} --project-dir /Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/elementary/monitor/dbt_project --profiles-dir /Users/frank.tubbing/.dbt
2024-11-25 11:33:01 — INFO — Update skipped alerts
Sending alerts |████████████████████████████████████████| 20/20 [100%] in 0.0s (872.94/s)
2024-11-25 11:33:01 — INFO — Update sent alerts