Add cell execution tracking during notebook execution #587
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
References
Resolves #586
Code changes
This PR adds a
completed_cells
field to the Job model to track the number of cells executed during notebook execution in real-time. The implementation includes:Model Changes:
completed_cells
column to the Job table in ORM (jupyter_scheduler/orm.py
)jupyter_scheduler/models.py
)Execution Tracking:
JobFeature.track_cell_execution
feature flag to enable/disable cell trackingDefaultExecutionManager
to use nbconvert's nativeon_cell_executed
hook. Note that this was found to be cleaner than the subclassing approach defined in Track completed cell progress during notebook execution #586.User-facing changes
/jobs/{job_id}
now returns the current count of completed cells/jobs/{job_id}
accepts completed_cells updates for manual correctionsBackwards-incompatible changes
None