-
Notifications
You must be signed in to change notification settings - Fork 3.5k
Description
Problem
When using the Glue catalog, Trino doesn't push date partition predicates to Glue's GetPartitions API. This causes Trino to fetch all partitions and filter them locally, which is inefficient for large tables partitioned by date.
Proposed Solution
Currently, the GlueExpressionUtil.canConvertSqlTypeToStringForGlue() function sets DateType as unconvertible even when hive.metastore.glue.assume-canonical-partition-keys=true is set.
Thrift, on the other hand, already allows DateType when assumeCanonicalPartitionKeys=true, therefore the infrastructure to implement this feature for Glue already exists:
MetastoreUtil.sqlScalarToString()converts epoch days to ISO strings (e.g.2019-04-14)QUOTED_TYPESinGlueExpressionUtilalready includes"date", so values are properly quoted as'2019-04-14'
The proposal aims to align how Glue and Thrift behave by allowing Trino to push DateType predicates to Glue when assumeCanonicalPartitionKeys=true.
Impact
Improve the planning time of queries on large tables that partition data by date when using Glue.