Commit 41f10f1
committed
fix(concurrent): use last slice for client-side incremental filtering
The _get_concurrent_state() method was using slices[0] (the earliest slice)
instead of slices[-1] (the most recent slice) when extracting the cursor
value from partitioned state. Since merge_intervals() sorts slices in
ascending order by (START_KEY, END_KEY), using the first slice caused
should_be_synced() to use the earliest timestamp as the lower bound,
admitting ALL records instead of only new records.
This bug affected all low-code connectors using DatetimeBasedCursor with
is_client_side_incremental: true, including Notion's pages, databases,
comments, and blocks streams.
Fixes: airbytehq/oncall#8854
Co-Authored-By: unknown <>1 parent daf7d48 commit 41f10f1
1 file changed
+7
-5
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
233 | 233 | | |
234 | 234 | | |
235 | 235 | | |
236 | | - | |
237 | | - | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
238 | 240 | | |
239 | | - | |
240 | | - | |
241 | | - | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
242 | 244 | | |
243 | 245 | | |
244 | 246 | | |
| |||
0 commit comments