Elementary-data 0.18.3 report errors when using Databricks

[balexander]

Describe the bug
A fresh install of elementary-data[databricks]==0.18.3 fails when running edr report. The error message is below. After failure, the report is generated but shows a "No Test Results" message.

The failure seems to be because elementary-data[databricks]==0.18.3 installed the latest version of dbt-databricks, version 1.10.3. The problem is fixed by manually downgrading dbt-databricks to version 1.10.1.

Error Message:

ERROR:elementary.monitor.data_monitoring.report.data_monitoring_report:Could not generate the report - Error: Failed to run dbt command.
{"info": {"name": "RunningOperationCaughtError", "code": "Q001", "msg": "Encountered an error while running operation: Database Error\n  'Macro' object has no attribute 'config'", "level": "error", "invocationId": "2e9f2a82-02cf-4190-9002-1b6ab4f033b0", "pid": 2143, "thread": "MainThread", "ts": "2025-06-11T16:21:13.556338Z"}, "data": {"exc": "Database Error\n  'Macro' object has no attribute 'config'"}}
Please reach out to our community for help with this issue.
Traceback (most recent call last):
  File "/Users/my_user_name/Code/data-team/edr/.venv/lib/python3.11/site-packages/elementary/monitor/api/report/report.py", line 84, in get_report_data
    tests_api = TestsAPI(
                ^^^^^^^^^
  File "/Users/my_user_name/Code/data-team/edr/.venv/lib/python3.11/site-packages/elementary/monitor/api/tests/tests.py", line 52, in __init__
    self.test_results_db_rows = self._get_test_results_db_rows(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/my_user_name/Code/data-team/edr/.venv/lib/python3.11/site-packages/elementary/monitor/api/tests/tests.py", line 64, in _get_test_results_db_rows
    return self.tests_fetcher.get_all_test_results_db_rows(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/my_user_name/Code/data-team/edr/.venv/lib/python3.11/site-packages/elementary/monitor/fetchers/tests/tests.py", line 26, in get_all_test_results_db_rows
    run_operation_response = self.dbt_runner.run_operation(
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/my_user_name/Code/data-team/edr/.venv/lib/python3.11/site-packages/elementary/clients/dbt/command_line_dbt_runner.py", line 177, in run_operation
    result = self._run_command(
             ^^^^^^^^^^^^^^^^^^
  File "/Users/my_user_name/Code/data-team/edr/.venv/lib/python3.11/site-packages/elementary/clients/dbt/command_line_dbt_runner.py", line 114, in _run_command
    result = self._inner_run_command(
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/my_user_name/Code/data-team/edr/.venv/lib/python3.11/site-packages/elementary/clients/dbt/api_dbt_runner.py", line 54, in _inner_run_command
    raise DbtCommandError(
elementary.exceptions.exceptions.DbtCommandError: Failed to run dbt command.
{"info": {"name": "RunningOperationCaughtError", "code": "Q001", "msg": "Encountered an error while running operation: Database Error\n  'Macro' object has no attribute 'config'", "level": "error", "invocationId": "2e9f2a82-02cf-4190-9002-1b6ab4f033b0", "pid": 2143, "thread": "MainThread", "ts": "2025-06-11T16:21:13.556338Z"}, "data": {"exc": "Database Error\n  'Macro' object has no attribute 'config'"}}

To Reproduce
Steps to reproduce the behavior:

1. pip install "elementary-data[databricks]
2. run edr report <- the failure happens here

Expected behavior
The report should be built successfully.

Environment (please complete the following information):

• Elementary CLI (edr) version: 0.18.3
• Elementary dbt package version: 0.18.3
• dbt version you're using: 1.9.8
• Data warehouse: Databricks
• Infrastructure details: Running the elementary cli locally on a mac. Dbt is hosted in dbt cloud.

Additional context
None

Would you be willing to contribute a fix for this issue?
Yes.

[Jeremynadalmirakl]

I just had the same issue. It is also there using dbt-databricks==1.10.2.

[arbiv]

@balexander, @Jeremynadalmirakl thanks for reporting this issue!
We're looking into it

[arbiv]

The parse_model module, introduced in dbt-databricks 1.1.0.2, contains functions that can crash when encountering:

1. Macro objects - When the adapter tries to parse Macro objects (which are not actual models), it fails because these objects don't have the expected model attributes
2. Missing configurations - When model configs are incomplete or missing specific attributes, the parsing functions throw exceptions instead of gracefully handling the missing data
3. Unsupported object types - The adapter doesn't correctly handle edge cases where objects passed to it aren't the expected model types

Changing these to a safer implementation solves the issue, as can be seen in this draft PR.

We will check if this is a bug in dbt-databricks or our usage of it, and continue accordingly.

In the meantime, we will use dbt-databricks<1.10.2 as a workaround.

[mi-volodin]

Just to bump up the issue - my company have a critical need to go to dbt-databricks>=1.10.5 or higher, and this issue prevents us from doing so.

I see that PR with a draft fix was already proposed... but since it's been 2 month since it was updated last time - is there any chance that it will be fixed anytime soon? Is there a way to help?

[Jeremynadalmirakl]

Just to bump up the issue - my company have a critical need to go to dbt-databricks>=1.10.5 or higher, and this issue prevents us from doing so.

I see that PR with a draft fix was already proposed... but since it's been 2 month since it was updated last time - is there any chance that it will be fixed anytime soon? Is there a way to help?

Hi @mi-volodin
I don't know your setup but as a "patch" on my side, I made the process that generates the elementary report run with an appropriated dbt-databricks while my workflows run with a newer version.... :/

Hope that helps...

[mi-volodin]

Yep, using different venvs might help indeed. Just it's then not easy to deliver this, but could be an option. I was just curious is that would be fixed anytime soon.

[asos-dipeshbhundia]

@arbiv Is there a timeline on when this issue likely to be fixed? dbt-databricks is now on 1.12 and we need the later versions for our project. I don't really want to have to remove elementary

[haritamar]

Hey all, we were a bit cautious about the fix mentioned above because it patches internal dbt-databricks functions.
That being said - it looks like the issue is solved in newer versions!
So we're removing the restriction and it will be in a version we intend to release today.