Skip to content

Extend PartitionedAssetTimetable to allow per asset partition#60966

Open
Lee-W wants to merge 9 commits intoapache:mainfrom
astronomer:per-asset-partition-mapper
Open

Extend PartitionedAssetTimetable to allow per asset partition#60966
Lee-W wants to merge 9 commits intoapache:mainfrom
astronomer:per-asset-partition-mapper

Conversation

@Lee-W
Copy link
Member

@Lee-W Lee-W commented Jan 23, 2026

Why

closes: #57694

What

  • add default_partition_mapper and partition_mapper_mapping to PartitionedAssetTimetable.

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@Lee-W Lee-W changed the title feat: add partition_mapper to base asset class Per Asset Partition Jan 23, 2026
@Lee-W Lee-W force-pushed the per-asset-partition-mapper branch 6 times, most recently from 4f7d47c to 463e306 Compare January 30, 2026 10:13
@Lee-W Lee-W force-pushed the per-asset-partition-mapper branch 6 times, most recently from b516821 to ce928f8 Compare February 5, 2026 14:01
@Lee-W Lee-W changed the title Per Asset Partition Extend PartitionedAssetTimetable to allow per asset partition Feb 5, 2026
@Lee-W Lee-W marked this pull request as ready for review February 5, 2026 14:05
@Lee-W Lee-W force-pushed the per-asset-partition-mapper branch 2 times, most recently from dee403b to 587fd5b Compare February 9, 2026 09:00
@Lee-W Lee-W force-pushed the per-asset-partition-mapper branch 2 times, most recently from 58253a2 to 568528a Compare February 10, 2026 01:44
@Lee-W Lee-W force-pushed the per-asset-partition-mapper branch 2 times, most recently from 8ec2ee0 to bd8c91d Compare February 12, 2026 02:35
@Lee-W Lee-W force-pushed the per-asset-partition-mapper branch from 4c80db9 to 8c9d2ed Compare February 16, 2026 05:30
self,
*,
assets: SerializedAssetBase,
partition_mapper_mapping: dict[SerializedAssetBase, PartitionMapper] | None = None,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just brainstorming naming ideas
we could possibly call it partition_mapper_config or mapping_config or something ... just a thought though i'm not sure what's better

Copy link
Member

@guan404ming guan404ming Feb 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a difficult one. I think the partition_mapper_mapping makes sense to me but is kind of hard to understand at first glance. partition_mapper_config sounds like a great option but open to any better option!

Copy link
Member

@guan404ming guan404ming left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

self,
*,
assets: SerializedAssetBase,
partition_mapper_mapping: dict[SerializedAssetBase, PartitionMapper] | None = None,
Copy link
Member

@guan404ming guan404ming Feb 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a difficult one. I think the partition_mapper_mapping makes sense to me but is kind of hard to understand at first glance. partition_mapper_config sounds like a great option but open to any better option!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Interface for per-asset-dep partition mapping?

4 participants