Skip to content

Commit d572e50

Browse files
authored
Avoid snapshot in ALTER SINK if the sink has made progress (#34588)
In the case of an `ALTER SINK`, don't snapshot if the sink upper has progressed beyond the as_of. ### Motivation fixes: MaterializeInc/database-issues#9998 ### Checklist - [ ] This PR has adequate test coverage / QA involvement has been duly considered. ([trigger-ci for additional test/nightly runs](https://trigger-ci.dev.materialize.com/)) - [ ] This PR has an associated up-to-date [design doc](https://github.com/MaterializeInc/materialize/blob/main/doc/developer/design/README.md), is a design doc ([template](https://github.com/MaterializeInc/materialize/blob/main/doc/developer/design/00000000_template.md)), or is sufficiently small to not require a design. <!-- Reference the design in the description. --> - [ ] If this PR evolves [an existing `$T ⇔ Proto$T` mapping](https://github.com/MaterializeInc/materialize/blob/main/doc/developer/command-and-response-binary-encoding.md) (possibly in a backwards-incompatible way), then it is tagged with a `T-proto` label. - [ ] If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label ([example](MaterializeInc/cloud#5021)). <!-- Ask in #team-cloud on Slack if you need help preparing the cloud PR. --> - [ ] If this PR includes major [user-facing behavior changes](https://github.com/MaterializeInc/materialize/blob/main/doc/developer/guide-changes.md#what-changes-require-a-release-note), I have pinged the relevant PM to schedule a changelog post.
1 parent d1c4043 commit d572e50

File tree

1 file changed

+10
-1
lines changed
  • src/storage-controller/src

1 file changed

+10
-1
lines changed

src/storage-controller/src/lib.rs

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1577,6 +1577,15 @@ where
15771577
};
15781578
*cur_export = new_export;
15791579

1580+
// For `ALTER SINK`, the snapshot should only occur if the sink has not made any progress.
1581+
// This prevents unnecessary decoding in the sink.
1582+
// If the write frontier of the sink is strictly larger than its read hold, it must have at
1583+
// least written out its snapshot, and we can skip reading it; otherwise assume we may have
1584+
// to replay from the beginning.
1585+
// TODO(database-issues#10002): unify this with run_export, if possible
1586+
let with_snapshot = new_description.sink.with_snapshot
1587+
&& !PartialOrder::less_than(&new_description.sink.as_of, &cur_export.write_frontier);
1588+
15801589
let cmd = RunSinkCommand {
15811590
id,
15821591
description: StorageSinkDesc {
@@ -1587,7 +1596,7 @@ where
15871596
as_of: new_description.sink.as_of,
15881597
version: new_description.sink.version,
15891598
from_storage_metadata,
1590-
with_snapshot: new_description.sink.with_snapshot,
1599+
with_snapshot,
15911600
to_storage_metadata,
15921601
commit_interval: new_description.sink.commit_interval,
15931602
},

0 commit comments

Comments
 (0)