Conversation
smatzana
commented
Nov 30, 2021
smatzana
commented
- Adding some necessary attributes to a Flow in order to enable transforms
- Only affect Batch and OnlineFlows
smatzana
force-pushed
the
feature/AIP-5284-integrate-txf-main
branch
2 times, most recently
from
65d5efd to
ce564d9
Compare
smatzana
force-pushed
the
feature/AIP-5284-integrate-txf-main
branch
from
ce564d9 to
c269a75
Compare
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| TxTestFlow() |
There was a problem hiding this comment.
Could you please add a step for how the Transform is expected to be used? This would be used as an example and test of expected customer usage for us to review. thanks!
|
|
||
| if len(self._txf_callbacks): | ||
| for _, callback in self._txf_callbacks.items(): | ||
| callback() |
There was a problem hiding this comment.
Is this callback actually calling the transform? better named txf_callback.
It'll help to see an example transform in unit tests and/or tutorials.
|
|
||
| add_txf_attributes(self) | ||
| if not self._txf_registered_datasets.get(flow_name, None): | ||
| self._txf_registered_datasets[flow_name] = [dataset] |
There was a problem hiding this comment.
Why do we maintain a dictionary of _txf_registered_datasets[flow_name] where flow_name is always the name of the flow?
I think you mean _txf_registered_datasets[dataset.name]. However, that may not work because one could reference the multiple Flow dataset variable/parameter to the same logically named dataset.name.
What is _txf_registered_datasets used for? Could you please reference a PR for what it is used for?
3 participants
