-
Notifications
You must be signed in to change notification settings - Fork 205
Description
Is your feature request related to a problem? Please describe.
Workflows that use the ray actor pool executor (KMeans, CC, Shuffle etc) currently don't show any progress bars. For long running large jobs it becomes hard to track the progress especially as the ray dashboard also starts lagging and showing slower progress for more than 10000 tasks (from my experience). It would be good to monitor progress in some way.
Describe the solution you'd like
Since most operations there use the map_unordered method that returns an iterable as results are available it should be straightforward to wrap that with something like tqdm to monitor progress for those stages.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.