Skip to content

Add map_index_template support for mapped task group (#40799)#61975

Open
anishgirianish wants to merge 2 commits intoapache:mainfrom
anishgirianish:support-map-index-template-for-mapped-task-groups
Open

Add map_index_template support for mapped task group (#40799)#61975
anishgirianish wants to merge 2 commits intoapache:mainfrom
anishgirianish:support-map-index-template-for-mapped-task-groups

Conversation

@anishgirianish
Copy link
Contributor

@anishgirianish anishgirianish commented Feb 16, 2026


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Summary

This adds map_index_template support to @task_group, so all child tasks in a mapped task group show meaningful labels instead of numeric indexes.

image
  @task_group(map_index_template="{{ filename }}")
  def file_transforms(filename):
      extracted = extract(filename)
      load(extracted)

Testing

  • Unit tests across all layers (decorator, serialization, API, task runner)
  • Manual E2E in breeze with three scenarios: basic chained tasks, task-level precedence, and partial() + expand()

closes #40799.

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

XComModel.task_id == operator.task_id,
XComModel.run_id == run_id,
XComModel.key == key,
XComModel.map_index == -1,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean the group's expand input can't depend on mapped upstream task outputs? If so, this should be documented as a limitation.

Copy link
Contributor Author

@anishgirianish anishgirianish Feb 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great question! From what I can tell, the upstream producing the expand input is unmapped; it returns a list that drives the mapping. If it were mapped, it would go through expand_kwargs instead, so it wouldn't be a SchedulerXComArg here. But I'd love to hear if you've seen a case where this breaks,

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not that familiar with xcom intricacies - perhaps it's worth adding a test that ensures this scenario works correctly.

Copy link
Collaborator

@Dev-iL Dev-iL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall! Could benefit from some (docstring-level) clarifications.

@anishgirianish anishgirianish force-pushed the support-map-index-template-for-mapped-task-groups branch from d9355cb to 048094d Compare February 16, 2026 07:28
@anishgirianish
Copy link
Contributor Author

Thank you so much for the thorough review, @Dev-iL. I've addressed all the feedback, applied your match/case refactor, simplified the schema, added the iteration order comment, and added debug logging. Would really appreciate a re-review when you get a chance!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support named mapping for task group

2 participants