Skip to content

Conversation

@gabotechs
Copy link
Collaborator

@gabotechs gabotechs commented Aug 4, 2025

In preparation of the upcoming work, removes the stage delegation mechanism in favor of something simpler: assign the StageContexts to the ArrowFlightReadExec nodes as an extra planning step instead of dynamically during the query.

This PR ships several simplifications:

  • Removes the stage delegation mechanism in favor of leaving an empty slot in the ArrowFlightReadExec that an extra assing_stages planning task will fill. All the StageContexts and the node urls are now known at planning-time.
  • Removes the delegate-actor terminology in favor of just tasks
  • Updates the physical plan assertions in the tests to also account for the new information known during planning time.

@gabotechs gabotechs marked this pull request as ready for review August 4, 2025 06:32
@gabotechs gabotechs force-pushed the remove-stage-delegation branch from 6c7b21f to 57216d9 Compare August 4, 2025 07:04
@robtandy
Copy link
Collaborator

robtandy commented Aug 4, 2025

This looks like the right step before the ExecutionStage work comes in a subsequent PR

Base automatically changed from add-error-serde to main August 4, 2025 13:16
@gabotechs gabotechs force-pushed the remove-stage-delegation branch from 57216d9 to 7936ba9 Compare August 4, 2025 13:23
@gabotechs gabotechs merged commit a25907e into main Aug 4, 2025
3 checks passed
@gabotechs gabotechs deleted the remove-stage-delegation branch August 4, 2025 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants