Commit 022af98
authored
#5417 made several changes to
`SparseGlobalOrderReader::merge_result_cell_slabs`. One of these changes
erroneously caused coordinates which do not pass the query condition to
be emitted. This pull request restores the correct behavior.
The crux of the change is
```
length = 1;
```
to
```
length = to_process.max_slab_length(1);
```
The affected line of code occurs during coordinate de-duplication.
`merge_result_cell_slabs` puts coordinates from the leading tile of each
fragment into a priority queue and pops them off. If the coordinate at
the head of the queue matches what we just pulled off, then we continue
popping until there is no longer a match.
This de-duplication can cause a fragment to run out of loaded tiles, in
which case it is necessary to end the loop to get more data so that we
do not emit coordinates out of order. That's how we get to this `length
= 1` code, which signals that we should emit a result slab of length 1.
In contrast, `max_slab_length(1)` checks the query condition bitmap to
see if the duplicated coordinate actually belongs in the query result.
This was found by a customer data set which has many fragments which are
essentially copies of the whole coordinate domain. This scenario
de-duplicates many coordinates and is likely to hit the condition
described. The new test `Sparse global order reader: fragment full copy
1d` simulates this set-up.
A large fraction of the changes in this PR are the test scaffolding
changes to support evaluating query conditions upon our `FxRun1D` and
`FxRun2D` structures, so we can continue to check whether the expected
and actual results match in the presence of a query condition.
---
TYPE: BUG
DESC: Fix query condition regression introduced by #5417
1 parent d0348dc commit 022af98
File tree
10 files changed
+784
-166
lines changed- test
- src
- support
- rapidcheck
- src
- stdx
- tiledb/sm/query/readers
10 files changed
+784
-166
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
169 | 169 | | |
170 | 170 | | |
171 | 171 | | |
172 | | - | |
173 | | - | |
| 172 | + | |
174 | 173 | | |
175 | 174 | | |
176 | 175 | | |
177 | 176 | | |
| 177 | + | |
| 178 | + | |
178 | 179 | | |
179 | 180 | | |
180 | 181 | | |
| |||
0 commit comments