case insensitive check on partition matching #888
Conversation
| table_field.lower() == conf_table_field.lower() | ||
| or (conf_partition.time_ingestion_partitioning and table_field is not None) | ||
| ) and table_granularity == conf_partition.granularity | ||
| ) and table_granularity.lower() == conf_partition.granularity.lower() |
There was a problem hiding this comment.
Op's table actually didn't have the granularity key set:
-- models/foo_sans_granularity.sql
{{
config(
partition_by={
"field": "updated_at",
"data_type": "date",
}
)
}}
select id, updated_at, updated_at_utc from sources.rawDo we need to account for that?
There was a problem hiding this comment.
Daily partitioning is the default for all column types.
My understanding from the dbt docs was that the default granularity is day-partitioned.
Is this where the default is set?
https://github.com/dbt-labs/dbt-bigquery/blob/main/dbt/adapters/bigquery/impl.py#L77
There was a problem hiding this comment.
yes, there is a default so it should be safe
dbeatty10
added
the
triage:ready-for-review
|
Thanks for the triage @dbeatty10 ! Could you enable the OK to test to run the ITs too? |
|
Re-opening to make the |
|
@mikealfare it looks like both this PR & #866 are failing over |
|
@github-christophe-oudar rerunning tests to see if it was ci/cd. |
|
It's still failing on those tests and I don't know why 🤔 |
github-christophe-oudar
force-pushed
the
ADAP-834
branch
from
892e28f to
02b19bd
Compare
|
@github-christophe-oudar great work on this. |
* case insensitive check on partition matching * Review change --------- Co-authored-by: Christophe Oudar <[email protected]>
|
Great! Thanks for the swift turnaround on this bugfix 🙌 |
* case insensitive check on partition matching * Review change --------- Co-authored-by: Christophe Oudar <[email protected]> Co-authored-by: Christophe Oudar <[email protected]>
7 participants
resolves #886
Problem
The API returns sometimes upper cased fields such
timePartitioning.typeand in case the user config has also a different casing, we should do a case insensitive check on partition matching.Solution
It introduce a proper case insensitive check on partition field matching
Checklist