Skip to content

Add cell execution tracking during notebook execution #587

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

agupta01
Copy link

References

Resolves #586

Code changes

This PR adds a completed_cells field to the Job model to track the number of cells executed during notebook execution in real-time. The implementation includes:

Model Changes:

  • Added nullable completed_cells column to the Job table in ORM (jupyter_scheduler/orm.py)
  • Added completed_cells field to DescribeJob and UpdateJob models (jupyter_scheduler/models.py)

Execution Tracking:

  • Added JobFeature.track_cell_execution feature flag to enable/disable cell tracking
  • Modified DefaultExecutionManager to use nbconvert's native on_cell_executed hook. Note that this was found to be cleaner than the subclassing approach defined in Track completed cell progress during notebook execution #586.
  • Hook updates the database with ep.code_cells_executed after each cell execution

User-facing changes

  • GET /jobs/{job_id} now returns the current count of completed cells
  • PATCH /jobs/{job_id} accepts completed_cells updates for manual corrections

Backwards-incompatible changes

None

@agupta01 agupta01 changed the title All cell execution tracking during notebook execution Add cell execution tracking during notebook execution Jul 23, 2025
@andrii-i andrii-i added the enhancement New feature or request label Jul 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Track completed cell progress during notebook execution
2 participants