-
Notifications
You must be signed in to change notification settings - Fork 42
Description
Is your feature request related to a problem? Please describe.
Right now processors only run after each batch is generated. It should also be possible for the execution to happen pre/post-generation.
Also, users should be able to implement custom processors easily via callbacks (same as e.g., the LocalCallableValidator).
Describe the solution you'd like
There are a couple of ways of going about this. Following the original plan, we would allow the user to pick different stages for any given processor.
However, it might be more interesting to have processors implementing all different stages via different methods - instead of process, preprocess/postprocess/process_after_batch/etc.
That because a processor could actually require different procedures to happen at different times (e.g., save artifacts after batch is generated, and collect all of them at the end).
Describe alternatives you've considered
n/a
Additional context
n/a