-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Description
Jinja vs. Model Code; Compilation vs. Execution
I am not sure how to frame this but it has costed me a great deal of time in debugging so I am sharing it for other developers' visibility.
When you call a macro in a dbt model, if the macro is not wrapped in the {% if execute %}{% endif %} clause, the macro will run in the COMPILATION phase, which can lead to some very confusing logs and potential* errors. If someone were creating an orchestrated DBT Pipeline with Airflow for example, calling a macro within a model would enable parallelization of a given task. In that case, the call to the macro would need to be wrapped in the {% if execute %}{% endif %} clause though to avoid having that code be executed whenever the dbt project code is compiled. This is a very common use-case for data pipelines and I think it could save a lot of people a lot of time if an example of this were available in the docs.