Marking a worker offline for updates or swapping workers in kubernetes

[michaelpporter]

I'm wondering if there are steps/best practices for cordoning workers. to ensure jobs are done before upgrading or swapping nodes pools.

I am aware of fly land-worker. The issue I'm seeing is the health checks fail and the pod is restarted, which brings it back online.

When upgrading, I would like to mark workers offline so jobs move to new workers. I can make a new worker base as I'm applying the helm chart separately from the web pods.