Skip to content

Commit 7239276

Browse files
committed
add clarifying comment on dremio logic
1 parent 0a05536 commit 7239276

File tree

1 file changed

+13
-4
lines changed

1 file changed

+13
-4
lines changed

macros/utils/cross_db_utils/get_user_creation_query.sql

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,18 @@ GRANT VIEW REFLECTION ON {{ db_type }} "{{ db_name }}" TO USER "{{ parameters["u
131131
{% endmacro %}
132132

133133
{% macro get_dremio_databases() %}
134+
{# Dremio has a distinction between "databases" that contain views, and ones that contain writable tables:
135+
1. Tables exist come from sources (e.g. datalake Iceberg tables).
136+
2. Views exist in catalogs / spaces (naming changes depending on Dremio cloud vs software).
137+
138+
So we return in this macro both the db name and type so we can understand what kind of object to give permission on.
139+
We deduce the db type acco
140+
141+
NOTE: Despite of the above, I saw in our dev env that some "catalogs" might contain tables (not views).
142+
Our logic below deduces the db type by the actual table types we're seeing - so in order to handle this edge case we'll
143+
consider a db as a "catalog" if it has at least one view.
144+
#}
145+
134146
{% set dremio_databases_query %}
135147
select distinct
136148
case when lower(table_type) = 'view' then 'CATALOG' else 'SOURCE' end database_type,
@@ -144,14 +156,11 @@ GRANT VIEW REFLECTION ON {{ db_type }} "{{ db_name }}" TO USER "{{ parameters["u
144156
{% for row in elementary.agate_to_dicts(elementary.run_query(dremio_databases_query)) %}
145157
{% set db_name_lower = row["database_name"] | lower %}
146158

147-
{# Only include dbs configured in the dbt project #}
148159
{% if db_name_lower not in configured_dbs %}
149160
{% continue %}
150161
{% endif %}
151162

152-
{# It seems that in some cases there can be tables in catalogs (spaces), even though the docs clain they
153-
should only contain views.
154-
So anyway, to be safe - if we see at least one view on the db we'll categorize it as a catalog. #}
163+
{# This condition guarantees that if there's at least one view in the DB we'll consider it as a catalog (see explanation above) %}
155164
{% if db_name_lower in db_name_to_type and row["database_type"] != "CATALOG" %}
156165
{% continue %}
157166
{% endif %}

0 commit comments

Comments
 (0)