Commit b3f6a23
* MSQ: Fix composing channels losing partitionless frames.
The main change is that single-partition ComposingWritableFrameChannels (i.e.
those created by `ChannelOutputFactory#openChannel`) now associate all incoming
frames with that partition. Previously, frames might have come in with
partition set to `NO_PARTITION`, which would cause them to get "lost" by
the composing channel.
Fixes a bug introduced in #18144 when composed intermediate stage-internal
channels started being used for the output of hash partitioning. Prior to
#18144, they were only used for internal channels of the SuperSorter. This
bug could cause frames to go missing during sortMerge joins.
This patch also adds an embedded test for various durable storage scenarios,
including sortMerge join tests that would have caught the original bug.
Finally, this patch adjusts the way that Calcites escapes string literals,
to use the actual characters more often when possible. This helps format
the test SQLs generated by the embedded test more nicely.
* dependency analyze.
---------
Co-authored-by: Gian Merlino <gianmerlino@gmail.com>
Co-authored-by: Karan Kumar <karankumar1100@gmail.com>
1 parent cc93015 commit b3f6a23
File tree
13 files changed
+602
-8
lines changed- embedded-tests
- src/test/java/org/apache/druid/testing/embedded/msq
- processing/src
- main/java/org/apache/druid/frame
- channel
- processor
- test/java/org/apache/druid/frame
- channel
- processor
- services/src/test/java/org/apache/druid/testing/embedded
- sql/src
- main/java/org/apache/druid/sql/calcite/planner
- test/java/org/apache/druid/sql/calcite/planner
13 files changed
+602
-8
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
182 | 182 | | |
183 | 183 | | |
184 | 184 | | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
185 | 195 | | |
186 | 196 | | |
187 | 197 | | |
| |||
0 commit comments