Commit a6b6003
[fix](load) fix quorum success invalid in move-memtable-on-sink load path (#60681)
## Description
In `VTabletWriterV2`, the `_quorum_success()` function always returns
`false` even when
all streams have finished successfully, making the quorum success write
feature effectively
non-functional.
## Root Cause
In both `_build_tablet_node_mapping()` and
`_incremental_open_streams()`,
`_tablets_by_node[node].emplace(tablet_id)` is guarded by the
`known_indexes` check:
```cpp
if (known_indexes.contains(index.index_id)) [[likely]] {
continue;
}
_indexes_from_node[node].emplace_back(tablet);
_tablets_by_node[node].emplace(tablet_id);
known_indexes.insert(index.index_id);
```
The known_indexes set is shared across all partitions, tablets, and
nodes. Once an
index_id is inserted after processing the first tablet's first node, all
subsequent
tablets (and all other nodes of the same tablet) with the same index_id
skip the
_tablets_by_node update.
For example, with 1 index, 3 tablets [T1, T2, T3], and 3 replicas [N1,
N2, N3]:
Only _tablets_by_node[N1] = {T1} gets populated
T1 on N2/N3, and T2/T3 on all nodes are skipped
This causes _quorum_success() to compute finished_tablets_replica
incorrectly:
finished_tablets_replica[T1] = 1 (only counted from N1)
finished_tablets_replica[T2] = 0, finished_tablets_replica[T3] = 0
With a quorum requirement of 2 (for 3 replicas), the check always fails.
The known_indexes optimization was intended only for _indexes_from_node
(to avoid
sending duplicate schema info per index), but it incorrectly also
blocked the
_tablets_by_node population.
Note: vtablet_writer.cpp does NOT have this issue — its
_tablets_by_channel is
populated without the known_indexes guard.
## Fix
Move _tablets_by_node[node].emplace(tablet_id) before the known_indexes
check in
both _build_tablet_node_mapping() and _incremental_open_streams(), so
that every
tablet on every node is correctly recorded.1 parent a133581 commit a6b6003
File tree
3 files changed
+42
-2
lines changed- be/src
- runtime
- vec/sink/writer
- regression-test/suites/load_p0/stream_load
3 files changed
+42
-2
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
707 | 707 | | |
708 | 708 | | |
709 | 709 | | |
| 710 | + | |
710 | 711 | | |
711 | 712 | | |
712 | 713 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
102 | 102 | | |
103 | 103 | | |
104 | 104 | | |
| 105 | + | |
105 | 106 | | |
106 | 107 | | |
107 | 108 | | |
108 | 109 | | |
109 | | - | |
110 | 110 | | |
111 | 111 | | |
112 | 112 | | |
| |||
343 | 343 | | |
344 | 344 | | |
345 | 345 | | |
| 346 | + | |
346 | 347 | | |
347 | 348 | | |
348 | 349 | | |
349 | 350 | | |
350 | | - | |
351 | 351 | | |
352 | 352 | | |
353 | 353 | | |
| |||
Lines changed: 39 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
117 | 117 | | |
118 | 118 | | |
119 | 119 | | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
120 | 159 | | |
121 | 160 | | |
0 commit comments