Skip to content

Commit bec9908

Browse files
arosychukarosychuk
authored
Refactor of the Code Base (#18)
* Refactor of the Code Base * Non-implemented functions return an exception
1 parent d673287 commit bec9908

27 files changed

+600
-510
lines changed

‎README.md‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,8 @@ Uses [vertica-python](https://github.com/vertica/vertica-python) to connect to V
88

99
### dbt Core Features
1010

11-
Below is a table for what features the current Vertica adapter supports for dbt. This is constantly improving and changing as both dbt adds new functionality, as well as the dbt-vertica driver improves.
11+
Below is a table for what features the current Vertica adapter supports for dbt. This is constantly improving and changing as both dbt adds new functionality, as well as the dbt-vertica driver improves. This list is based upon dbt 1.0.3.
12+
1213

1314
| dbt Core Features | Supported |
1415
| ------------------------------------------------- | ----------- |
@@ -45,6 +46,10 @@ Below is a table for what features the current Vertica adapter supports for Vert
4546

4647
## Changes
4748

49+
### 1.0.3
50+
- Refactored the adapter to model after dbt's global_project macros
51+
- Unimplemented functions should throw an exception that it's not implemented. If you stumble across this, please open an Issue or PR so we can investigate.
52+
4853
### 1.0.2
4954
- Added support for snapshot timestamp with passing tests
5055
- Added support for snapshot check cols with passing tests

‎dbt/include/vertica/macros/README.md‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
# dbt-vertica Macro Implementations
2+
3+
Below is a table for what macros the current Vertica adapter supports for dbt. This is constantly improving and changing as both dbt adds new functionality, as well as the dbt-vertica driver improves. This list is based upon dbt 1.0.3, with the list found under https://github.com/dbt-labs/dbt-core/tree/main/core/dbt/include/global_project/macros.
4+
5+
## dbt Macros - adapters
6+
7+
| dbt Macros - adapters | Function | Implemented |
8+
| ------------------------- | ----------------------------------- | ----------- |
9+
| adapters/columns.sql | get_columns_in_relation() | Yes |
10+
| adapters/columns.sql | sql_convert_columns_in_relation() | No |
11+
| adapters/columns.sql | get_columns_in_query() | Yes |
12+
| adapters/columns.sql | alter_column_type() | No |
13+
| adapters/columns.sql | alter_relation_add_remove_columns() | No |
14+
| adapters/freshness.sql | collect_freshness() | No |
15+
| adapters/freshness.sql | current_timestamp() | Yes |
16+
| adapters/indexes.sql | create_indexes() | Yes |
17+
| adapters/indexes.sql | get_create_index_sql() | No |
18+
| adapters/metadata.sql | get_catalog() | Yes |
19+
| adapters/metadata.sql | information_schema_name() | Yes |
20+
| adapters/metadata.sql | list_schemas() | Yes |
21+
| adapters/metadata.sql | list_relations_without_caching() | Yes |
22+
| adapters/persist_docs.sql | alter_column_comment() | No |
23+
| adapters/persist_docs.sql | alter_relation_comment() | No |
24+
| adapters/persist_docs.sql | persist_docs() | Yes |
25+
| adapters/relation.sql | drop_relation() | Yes |
26+
| adapters/relation.sql | drop_relation_if_exists() | Yes |
27+
| adapters/relation.sql | get_or_create_relation() | No |
28+
| adapters/relation.sql | load_relation() | No |
29+
| adapters/relation.sql | make_temp_relation() | Yes |
30+
| adapters/relation.sql | rename_relation() | Yes |
31+
| adapters/relation.sql | truncate_relation() | Yes |
32+
| adapters/schema.sql | create_schema() | Yes |
33+
| adapters/schema.sql | drop_schema() | Yes |
34+
35+
## dbt Macros - materializations
36+
| dbt Macros - materializations | Function | Implemented |
37+
| -------------------------------------------------------- | -------------------------------- | ----------- |
38+
| materializations/models/incremental/merge.sql | get_merge_sql() | Yes |
39+
| materializations/models/incremental/merge.sql | get_delete_insert_merge_sql() | Yes |
40+
| materializations/models/incremental/merge.sql | get_insert_overwrite_merge_sql() | No |
41+
| materializations/models/table/create_table_as.sql | create_table_as() | Yes |
42+
| materializations/models/view/create_or_replace_view.sql | create_or_replace_view() | Yes |
43+
| materializations/models/view/create_view_as.sql | create_view_as() | Yes |
44+
| materializations/models/view/create_view_as.sql | get_create_view_as_sql() | Yes |
45+
| materializations/seeds/helpers.sql | create_csv_table() | Yes |
46+
| materializations/seeds/helpers.sql | reset_csv_table() | Yes |
47+
| materializations/seeds/helpers.sql | get_binding_char() | Yes |
48+
| materializations/seeds/helpers.sql | get_seed_column_quoted_csv() | Yes |
49+
| materializations/seeds/helpers.sql | load_csv_rows() | Yes |
50+
| materializations/snapshots/helper.sql | build_snapshot_table() | Yes |
51+
| materializations/snapshots/helper.sql | create_columns() | Yes |
52+
| materializations/snapshots/helper.sql | post_snapshot() | Yes |
53+
| materializations/snapshots/helper.sql | snapshot_staging_table() | Yes |
54+
| materializations/snapshots/snapshot_merge.sql | snapshot_merge_sql() | Yes |
55+
| materializations/snapshots/strategies.sql | snapshot_get_time() | Yes |
56+
| materializations/snapshots/strategies.sql | snapshot_hash_arguments() | Yes |
57+
| materializations/snapshots/strategies.sql | snapshot_string_as_time() | Yes |
58+
| materializations/configs.sql | set_sql_header() | Yes |
59+
| materializations/configs.sql | should_full_refresh() | Yes |
60+
| materializations/configs.sql | should_store_failures() | Yes |
61+
| materializations/hooks.sql | run_hooks() | Yes |
62+
| materializations/hooks.sql | make_hook_config() | Yes |
63+
| materializations/hooks.sql | before_begin() | Yes |
64+
| materializations/hooks.sql | in_transaction() | Yes |
65+
| materializations/hooks.sql | after_commit() | Yes |

‎dbt/include/vertica/macros/adapters/columns.sql‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
{% macro vertica__get_columns_in_relation(relation) -%}
2+
{% call statement('get_columns_in_relation', fetch_result=True) %}
3+
select
4+
column_name
5+
, data_type
6+
, character_maximum_length
7+
, numeric_precision
8+
, numeric_scale
9+
from (
10+
select
11+
column_name
12+
, data_type
13+
, character_maximum_length
14+
, numeric_precision
15+
, numeric_scale
16+
, ordinal_position
17+
from v_catalog.columns
18+
where table_schema = '{{ relation.schema }}'
19+
and table_name = '{{ relation.identifier }}'
20+
union all
21+
select
22+
column_name
23+
, data_type
24+
, character_maximum_length
25+
, numeric_precision
26+
, numeric_scale
27+
, ordinal_position
28+
from v_catalog.view_columns
29+
where table_schema = '{{ relation.schema }}'
30+
and table_name = '{{ relation.identifier }}'
31+
) t
32+
order by ordinal_position
33+
{% endcall %}
34+
{% set table = load_result('get_columns_in_relation').table %}
35+
{{ return(sql_convert_columns_in_relation(table)) }}
36+
{% endmacro %}
37+
38+
39+
{% macro vertica__sql_convert_columns_in_relation(table) -%}
40+
{{ exceptions.raise_not_implemented(
41+
'sql_convert_columns_in_relation macro not implemented for adapter '+adapter.type()) }}
42+
{%- endmacro %}
43+
44+
45+
{% macro vertica__alter_column_type(relation, column_name, new_column_type) -%}
46+
{{ exceptions.raise_not_implemented(
47+
'alter_column_type macro not implemented for adapter '+adapter.type()) }}
48+
{%- endmacro %}
49+
50+
51+
{% macro vertica__alter_relation_add_remove_columns(relation, column_name, new_column_type) -%}
52+
{{ exceptions.raise_not_implemented(
53+
'alter_relation_add_remove_columns macro not implemented for adapter '+adapter.type()) }}
54+
{%- endmacro %}
55+
56+
57+
{#
58+
No need to implement get_columns_in_query(). Syntax supported by default.
59+
#}

‎dbt/include/vertica/macros/adapters/freshness.sql‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{% macro vertica__current_timestamp() -%}
2+
current_timestamp
3+
{%- endmacro %}
4+
5+
6+
{% macro vertica__collect_freshness() -%}
7+
{{ exceptions.raise_not_implemented(
8+
'collect_freshness macro not implemented for adapter '+adapter.type()) }}
9+
{%- endmacro %}

‎dbt/include/vertica/macros/adapters/indexes.sql‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{% macro vertica__get_create_index_sql(relation, index_dict) -%}
2+
{{ exceptions.raise_not_implemented(
3+
'get_create_index_sql macro not implemented for adapter '+adapter.type()) }}
4+
{%- endmacro %}
5+
6+
7+
{#
8+
No need to implement create_indexes(). Syntax supported by default.
9+
#}

dbt/include/vertica/macros/adapters.sql renamed to dbt/include/vertica/macros/adapters/metadata.sql

Lines changed: 48 additions & 139 deletions
Original file line numberDiff line numberDiff line change
@@ -1,140 +1,3 @@
1-
2-
{% macro vertica__information_schema_name(database) -%}
3-
{%- if database -%}
4-
{{ adapter.quote_as_configured(database, 'database') }}.v_catalog
5-
{%- else -%}
6-
v_catalog
7-
{%- endif -%}
8-
{%- endmacro %}
9-
10-
{% macro vertica__list_schemas(database) %}
11-
{% call statement('list_schemas', fetch_result=True, auto_begin=False) %}
12-
select schema_name
13-
from v_catalog.schemata
14-
{% endcall %}
15-
{{ return(load_result('list_schemas').table) }}
16-
{% endmacro %}
17-
18-
{% macro vertica__check_schema_exists(database, schema) -%}
19-
{% call statement('check_schema_exists', fetch_result=True, auto_begin=False) -%}
20-
select count(*)
21-
from v_catalog.schemata
22-
where schema_name='{{ schema }}'
23-
{%- endcall %}
24-
{{ return(load_result('check_schema_exists').table) }}
25-
{% endmacro %}
26-
27-
{% macro vertica__drop_schema(relation) -%}
28-
{% call statement('drop_schema') -%}
29-
drop schema if exists {{ relation.without_identifier().include(database=False) }} cascade
30-
{% endcall %}
31-
{% endmacro %}
32-
33-
{% macro vertica__create_schema(relation) -%}
34-
{%- call statement('create_schema') -%}
35-
create schema if not exists {{ relation.without_identifier().include(database=False) }}
36-
{% endcall %}
37-
{% endmacro %}
38-
39-
{% macro vertica__list_relations_without_caching(schema_relation) %}
40-
{% call statement('list_relations_without_caching', fetch_result=True) -%}
41-
select
42-
'{{ schema_relation.database }}' as database,
43-
table_name as name,
44-
table_schema as schema,
45-
'table' as type
46-
from v_catalog.tables
47-
where table_schema ilike '{{ schema_relation.schema }}'
48-
union all
49-
select
50-
'{{ schema_relation.database }}' as database,
51-
table_name as name,
52-
table_schema as schema,
53-
'view' as type
54-
from v_catalog.views
55-
where table_schema ilike '{{ schema_relation.schema }}'
56-
{% endcall %}
57-
{{ return(load_result('list_relations_without_caching').table) }}
58-
{% endmacro %}
59-
60-
{% macro vertica__rename_relation(from_relation, to_relation) %}
61-
{% set target_name = adapter.quote_as_configured(to_relation.identifier, 'identifier') %}
62-
{% call statement('rename_relation') -%}
63-
alter {{ from_relation.type }} {{ from_relation }} rename to {{ target_name }}
64-
{%- endcall %}
65-
{% endmacro %}
66-
67-
68-
{% macro vertica__get_columns_in_relation(relation) -%}
69-
{% call statement('get_columns_in_relation', fetch_result=True) %}
70-
select
71-
column_name
72-
, data_type
73-
, character_maximum_length
74-
, numeric_precision
75-
, numeric_scale
76-
from (
77-
select
78-
column_name
79-
, data_type
80-
, character_maximum_length
81-
, numeric_precision
82-
, numeric_scale
83-
, ordinal_position
84-
from v_catalog.columns
85-
where table_schema = '{{ relation.schema }}'
86-
and table_name = '{{ relation.identifier }}'
87-
union all
88-
select
89-
column_name
90-
, data_type
91-
, character_maximum_length
92-
, numeric_precision
93-
, numeric_scale
94-
, ordinal_position
95-
from v_catalog.view_columns
96-
where table_schema = '{{ relation.schema }}'
97-
and table_name = '{{ relation.identifier }}'
98-
) t
99-
order by ordinal_position
100-
{% endcall %}
101-
{% set table = load_result('get_columns_in_relation').table %}
102-
{{ return(sql_convert_columns_in_relation(table)) }}
103-
{% endmacro %}
104-
105-
{% macro vertica__create_view_as(relation, sql) %}
106-
{% set sql_header = config.get('sql_header', none) %}
107-
108-
{{ sql_header if sql_header is not none }}
109-
create or replace view {{ relation }} as (
110-
{{ sql }}
111-
);
112-
113-
{% endmacro %}
114-
115-
{% macro vertica__create_table_as(temporary, relation, sql) -%}
116-
{%- set sql_header = config.get('sql_header', none) -%}
117-
118-
{{ sql_header if sql_header is not none }}
119-
120-
create {% if temporary: -%}local temporary{%- endif %} table
121-
{{ relation.include(database=(not temporary), schema=(not temporary)) }}
122-
{% if temporary: -%}on commit preserve rows{%- endif %}
123-
as (
124-
{{ sql }}
125-
);
126-
{% endmacro %}
127-
128-
{% macro vertica__make_temp_relation(base_relation, suffix) %}
129-
{% set tmp_identifier = base_relation.identifier ~ suffix %}
130-
{% do return(base_relation.incorporate(
131-
path={
132-
"identifier": tmp_identifier,
133-
"schema": none,
134-
"database": none
135-
})) -%}
136-
{% endmacro %}
137-
1381
{% macro vertica__get_catalog(information_schema, schemas) -%}
1392
{% call statement('get_catalog', fetch_result=True) %}
1403

@@ -185,6 +48,52 @@
18548
{{ return(load_result('get_catalog').table) }}
18649
{% endmacro %}
18750

188-
{% macro vertica__current_timestamp() -%}
189-
current_timestamp
51+
52+
{% macro vertica__information_schema_name(database) -%}
53+
{%- if database -%}
54+
{{ adapter.quote_as_configured(database, 'database') }}.v_catalog
55+
{%- else -%}
56+
v_catalog
57+
{%- endif -%}
19058
{%- endmacro %}
59+
60+
61+
{% macro vertica__list_schemas(database) %}
62+
{% call statement('list_schemas', fetch_result=True, auto_begin=False) %}
63+
select schema_name
64+
from v_catalog.schemata
65+
{% endcall %}
66+
{{ return(load_result('list_schemas').table) }}
67+
{% endmacro %}
68+
69+
70+
{% macro vertica__check_schema_exists(database, schema) -%}
71+
{% call statement('check_schema_exists', fetch_result=True, auto_begin=False) -%}
72+
select count(*)
73+
from v_catalog.schemata
74+
where schema_name='{{ schema }}'
75+
{%- endcall %}
76+
{{ return(load_result('check_schema_exists').table) }}
77+
{% endmacro %}
78+
79+
80+
{% macro vertica__list_relations_without_caching(schema_relation) %}
81+
{% call statement('list_relations_without_caching', fetch_result=True) -%}
82+
select
83+
'{{ schema_relation.database }}' as database,
84+
table_name as name,
85+
table_schema as schema,
86+
'table' as type
87+
from v_catalog.tables
88+
where table_schema ilike '{{ schema_relation.schema }}'
89+
union all
90+
select
91+
'{{ schema_relation.database }}' as database,
92+
table_name as name,
93+
table_schema as schema,
94+
'view' as type
95+
from v_catalog.views
96+
where table_schema ilike '{{ schema_relation.schema }}'
97+
{% endcall %}
98+
{{ return(load_result('list_relations_without_caching').table) }}
99+
{% endmacro %}

‎dbt/include/vertica/macros/adapters/persist_docs.sql‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
{% macro vertica__alter_column_comment(relation, column_dict) -%}
2+
{{ exceptions.raise_not_implemented(
3+
'alter_column_comment macro not implemented for adapter '+adapter.type()) }}
4+
{%- endmacro %}
5+
6+
7+
{% macro vertica__alter_relation_comment(relation, relation_comment) -%}
8+
{{ exceptions.raise_not_implemented(
9+
'alter_relation_comment macro not implemented for adapter '+adapter.type()) }}
10+
{%- endmacro %}
11+
12+
13+
{#
14+
No need to implement persist_docs(). Syntax supported by default.
15+
#}

0 commit comments

Comments
 (0)