Refactor: separate SplitAndSizeRestriction from FnApiDoFnRunner#34919
Conversation
96111bc to
544e5c0
Compare
|
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment |
...s/src/main/java/org/apache/beam/fn/harness/SplittableSplitAndSizeRestrictionsDoFnRunner.java
Outdated
Show resolved
Hide resolved
...s/src/main/java/org/apache/beam/fn/harness/SplittableSplitAndSizeRestrictionsDoFnRunner.java
Show resolved
Hide resolved
...s/src/main/java/org/apache/beam/fn/harness/SplittableSplitAndSizeRestrictionsDoFnRunner.java
Outdated
Show resolved
Hide resolved
sdks/java/harness/src/main/java/org/apache/beam/fn/harness/WindowedSplitResult.java
Show resolved
Hide resolved
sdks/java/harness/src/main/java/org/apache/beam/fn/harness/SplitResultsWithStopIndex.java
Show resolved
Hide resolved
939637a to
f6d9878
Compare
...s/src/main/java/org/apache/beam/fn/harness/SplittableSplitAndSizeRestrictionsDoFnRunner.java
Outdated
Show resolved
Hide resolved
f6d9878 to
712d29d
Compare
...arness/src/main/java/org/apache/beam/fn/harness/SplittablePairWithRestrictionDoFnRunner.java
Outdated
Show resolved
Hide resolved
...s/src/main/java/org/apache/beam/fn/harness/SplittableSplitAndSizeRestrictionsDoFnRunner.java
Outdated
Show resolved
Hide resolved
138b7b2 to
82555e7
Compare
|
Assigning reviewers: R: @m-trieu for label java. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
scwhittle
left a comment
There was a problem hiding this comment.
LGTM, with some small cleanups
...s/src/main/java/org/apache/beam/fn/harness/SplittableSplitAndSizeRestrictionsDoFnRunner.java
Outdated
Show resolved
Hide resolved
| PaneInfo.NO_FIRING); | ||
| } | ||
|
|
||
| @Test |
There was a problem hiding this comment.
nit: move down so grouped with other tests?
...c/test/java/org/apache/beam/fn/harness/SplittableSplitAndSizeRestrictionsDoFnRunnerTest.java
Outdated
Show resolved
Hide resolved
...c/test/java/org/apache/beam/fn/harness/SplittableSplitAndSizeRestrictionsDoFnRunnerTest.java
Show resolved
Hide resolved
82555e7 to
9765f7c
Compare
|
Thanks for bearing with me and paying close attention to this! I know the diffs are not particularly helpful in seeing what the real change is. I do think the positive delta is almost all boilerplate and things are shrinking in spirit and cyclomatic complexity is reducing. Just a couple more and it'll be more like straight line code based on the URN, which should lend itself to (1) pulling out more shared and meaningful abstractions and (2) optimizing and further simplifying specific paths by finding more stuff that was in there just because it was hard to tell which operation we were performing. |
This is a further step in untangling FnApiDoFnRunner which implements a variety of loosely-related transforms, which mostly have in common that they call a user's DoFn but otherwise are disjoint.
9765f7c to
3b09453
Compare
No problem, thanks for making this more maintainable. I think that the fact that I added the currentTrackerClaimed creation to this method when it was part of the huge file shows how difficult it was to see what exactly was going on there :) |
This builds on #34883 and #34907 which are steps 1 and 2 of untangling FnApiDoFnRunner. This splits off the still-very-simple transform that calls
SplitRestrictionon each restriction andSizeRestrictionon each of the sub-parts that is emitted.Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.