-
Notifications
You must be signed in to change notification settings - Fork 71
Description
Describe the feature
In some cases, it is not desirable for the properties defined in a model to be used when creating tables in a situation where the config context variable is defined by a given model. An example would be the case where a model specifies a location in its properties dictionary, but the running of the dbt model necessitates a temporary table to be created. As currently implemented, this temporary table is written to the same location as specified in the model. This is problematic when a location can only support one table being written there as is the case in Trino running with an Iceberg connector, for example. My proposed solution would allow for the trino__create_table_as macro to accept an optional argument named "properties_update_dict" that is defaulted to None where a dictionary could be supplied to modify what values are saved to the _properties variable through a call of the update method. Allowing this optional argument would allow a user to override the trino__create_table_as macro by defining a macro of the same name in his or her project that can serve as a wrapper around dbt.trino__create_table_as and supply the properties_update_dict parameter if desired. This would allow the overriden macro to stay in line with the built-in macro as updates are made while allowing the flexibility to adjust the behavior, which is presently not possible without copying the built-in macro and sidestepping the revision control of the built-in macro.
Describe alternatives you've considered
The present design does not allow for alternatives aside from overriding the macro in one's own project.
Who will benefit?
This will certainly benefit Trino/Iceberg users who are presently unable to use some incremental materializations and snapshots on account of writing multiple tables to a shared location not being allowed. As a general practice, others will benefit by not unnecessarily writing temporary files to the same location as their persisted table. The change is very simple and self-contained. I will create a PR.
Are you willing to submit PR?
- Yes I am willing to submit a PR!