Skip to content

Trino doesn't push date partition predicates to Glue #28817

@rafaelbenvenuti

Description

@rafaelbenvenuti

Problem

When using the Glue catalog, Trino doesn't push date partition predicates to Glue's GetPartitions API. This causes Trino to fetch all partitions and filter them locally, which is inefficient for large tables partitioned by date.

Proposed Solution

Currently, the GlueExpressionUtil.canConvertSqlTypeToStringForGlue() function sets DateType as unconvertible even when hive.metastore.glue.assume-canonical-partition-keys=true is set.

Thrift, on the other hand, already allows DateType when assumeCanonicalPartitionKeys=true, therefore the infrastructure to implement this feature for Glue already exists:

  • MetastoreUtil.sqlScalarToString() converts epoch days to ISO strings (e.g. 2019-04-14)
  • QUOTED_TYPES in GlueExpressionUtil already includes "date", so values are properly quoted as '2019-04-14'

The proposal aims to align how Glue and Thrift behave by allowing Trino to push DateType predicates to Glue when assumeCanonicalPartitionKeys=true.

Impact

Improve the planning time of queries on large tables that partition data by date when using Glue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions