Skip to content

Update status display for terminating pods#4179

Open
cuza wants to merge 5 commits intomasterfrom
u/cuza/PAASTA-18678-Make-paasta-status-show-terminating-pods
Open

Update status display for terminating pods#4179
cuza wants to merge 5 commits intomasterfrom
u/cuza/PAASTA-18678-Make-paasta-status-show-terminating-pods

Conversation

@cuza
Copy link
Member

@cuza cuza commented Dec 16, 2025

This change ensures that terminating pods are still represented in the status output until they are fully removed.

image

@cuza cuza changed the base branch from u/cuza/upgrade-paasta-playground-to-python-3.10 to master December 16, 2025 04:45
cuza added 3 commits December 15, 2025 20:52
…w-terminating-pods

* master:
  Released 1.36.3 via make release
  Moved the scs conf from --conf to env var (#4177)
  Released 1.36.2 via make release
  Revert "remove some unused code."
  Released 1.36.1 via make release
  Revert "Merge pull request #4151 from Yelp/timma/upgrade_spark_configuration_lib_to_3.3.7" (#4174)
  Released 1.36.0 via make release
  Released 1.35.8 via make release
  remove some mesos-isms from local-run
  remove some unused code.
  Changed the expected_scs_conf to json instead of raw string
  Changed the user to null in scs_conf which fixes the failing tron_tools test
  Upgraded service configuration lib to 3.3.7 and updated the test accordingly
@cuza cuza marked this pull request as ready for review January 5, 2026 14:19
@cuza cuza requested a review from a team as a code owner January 5, 2026 14:19
ilkinmammadzada
ilkinmammadzada previously approved these changes Jan 5, 2026
Copy link
Member

@ilkinmammadzada ilkinmammadzada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, with a small comment

Comment on lines +462 to +463
# Check if pod is terminating (has deletion timestamp)
delete_timestamp = getattr(pod, "delete_timestamp", None)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think we can swap this to just pod.delete_timestamp once this is fully rolled out, right?

type: array
items:
$ref: '#/components/schemas/KubernetesPodEvent'
delete_timestamp:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imo, it'd be nice to keep the k8s name here (i.e., s/delete_timestamp/deletion_timestamp)


A replicaset is considered "actually running" if:
- It has non-zero desired replicas OR non-zero ready replicas, OR
- It has pods associated with it (e.g., terminating pods)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- It has pods associated with it (e.g., terminating pods)
- It has ANY pods associated with it (e.g., terminating pods are still "running")

Comment on lines +518 to +520
and (
pod_status_by_replicaset is None
or not pod_status_by_replicaset.get(rs.metadata.name)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, i think that once we're at a newer k8s version, we can replace this with looking at rs.status.terminating_replicas

...but for now, i think this is fine?

(that said, i think this function is either at or getting to the point of being too clever and we might want to refactor this at some point :p)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(although, maybe just some comments on the new conditions would be good enough for a bit longer? mostly so that folks don't need to figure out why pod_status_by_replicaset being None or a RS not being in there means there's only terminating pods for that rs)

mock_naturaltime,
mock_kubernetes_pod,
):
from datetime import datetime as real_datetime
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can just use datetime.datetime, right? (since we have a top-level import datetime import)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants