Skip to content

Accept Dictionary to Allow Updating Properties in trino__create_table_as #430

@james-long-mx

Description

@james-long-mx

Describe the feature

In some cases, it is not desirable for the properties defined in a model to be used when creating tables in a situation where the config context variable is defined by a given model. An example would be the case where a model specifies a location in its properties dictionary, but the running of the dbt model necessitates a temporary table to be created. As currently implemented, this temporary table is written to the same location as specified in the model. This is problematic when a location can only support one table being written there as is the case in Trino running with an Iceberg connector, for example. My proposed solution would allow for the trino__create_table_as macro to accept an optional argument named "properties_update_dict" that is defaulted to None where a dictionary could be supplied to modify what values are saved to the _properties variable through a call of the update method. Allowing this optional argument would allow a user to override the trino__create_table_as macro by defining a macro of the same name in his or her project that can serve as a wrapper around dbt.trino__create_table_as and supply the properties_update_dict parameter if desired. This would allow the overriden macro to stay in line with the built-in macro as updates are made while allowing the flexibility to adjust the behavior, which is presently not possible without copying the built-in macro and sidestepping the revision control of the built-in macro.

Describe alternatives you've considered

The present design does not allow for alternatives aside from overriding the macro in one's own project.

Who will benefit?

This will certainly benefit Trino/Iceberg users who are presently unable to use some incremental materializations and snapshots on account of writing multiple tables to a shared location not being allowed. As a general practice, others will benefit by not unnecessarily writing temporary files to the same location as their persisted table. The change is very simple and self-contained. I will create a PR.

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions