Commit 6377254
authored
[Chore](shuffle) adjust some local shuffle rules (#59366)
1. make distinct streaming agg always shuffle(Improve parallelism)
2. make broadcast join probe do not shuffle
This pull request updates the logic for determining required data
distributions in the aggregation and join pipeline operators. The main
focus is on improving the handling of exchange types, especially for
passthrough and broadcast scenarios.
Key changes include:
**Data distribution logic updates:**
* In `DistinctStreamingAggOperatorX`, the method now always returns
`ExchangeType::PASSTHROUGH` instead of delegating to the base class,
simplifying the distribution requirement when colocation is not needed.
* In `HashJoinProbeOperatorX`, the logic for broadcast joins is refined:
if the child is a serial operator, it returns
`ExchangeType::PASSTHROUGH`; otherwise, it returns `ExchangeType::NOOP`.
The handling of bucket shuffle and colocate join distributions is also
clarified.1 parent d49d620 commit 6377254
File tree
2 files changed
+10
-8
lines changed- be/src/pipeline/exec
2 files changed
+10
-8
lines changedLines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
120 | 120 | | |
121 | 121 | | |
122 | 122 | | |
123 | | - | |
| 123 | + | |
124 | 124 | | |
125 | 125 | | |
126 | 126 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
133 | 133 | | |
134 | 134 | | |
135 | 135 | | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
136 | 140 | | |
137 | | - | |
138 | | - | |
139 | | - | |
140 | | - | |
141 | | - | |
142 | | - | |
143 | | - | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
144 | 146 | | |
145 | 147 | | |
146 | 148 | | |
| |||
0 commit comments