Deferred ResourceQuota Enforcement for pods with SchedulingReadinessGates

[ichekrygin]

Enhancement Description

Current ResourceQuota enforcement is strict: if a pod's resource request exceeds the remaining quota, it is rejected during admission. This is inconsistent with other Kubernetes paradigms, such as node resource availability, where unschedulable pods are allowed to remain in a Pending state.

In addition, there is a notable misalignment between quota usage accounting and pod readiness state. Pods that are gated by mechanisms like PodSchedulingReadiness cannot yet be scheduled or consume actual cluster resources, yet they are still counted against quota usage. This leads to inefficiencies and unnecessary admission rejections, particularly for systems managing asynchronous or staged workload execution.

The strict admission approach:

• Breaks compatibility with asynchronous or deferred scheduling patterns.
• Prevents advanced workload orchestrators (e.g., Kueue) from managing workloads that temporarily exceed quota.
• Impedes resource overcommit strategies even if final scheduling may be feasible after other pods complete.

This proposal introduces a more natural, flexible alternative.

• Kubernetes Enhancement Proposal:
• Discussion Link:
  • sig-apps: June 7, 2025:
    • meeting agenda
    • meeting recording
  • sig-scheduling: July 24, 2025
    • meeting agenda
    • meeting recording
• PRs by stage and milestone:
  • Alpha - v1.34
    • KEP (k/enhancements) update PR(s):
    • Code (k/k) update PR(s):
    • Docs (k/website) update PR(s):

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

[ichekrygin]

/sig scheduling

/sig apps

[iholder101]

@ichekrygin thanks for raising this important issue.

I'd like to point out this discussion: kubernetes/kubernetes#131835. It is another severe problem that will only get worse with in-place resize. Perhaps we should be discussing these together?

[ichekrygin]

Hello @iholder101 , thank you for calling this out.

I agree, these are related issues. Both are about bridging the gap between Kubernetes’ current strict ResourceQuota enforcement model and the more flexible, deferred scheduling patterns that PodSchedulingReadiness was designed to support.

While this issue focuses on initial pod admission, the KEP-3521 discussion focuses on post-admission resource changes. Both share the same root cause: Kubernetes enforces quota without considering deferred scheduling states.

How Deferred ResourceQuota Enforement helps

Fixes creation-time misalignment: If we defer CPU, memory, etc. enforcement for schedule-gated pods until the gate is cleared, external quota managers get a clean interception point at scheduling time.

Where it falls short for the resize case

In‑place resize does not reenter scheduling, so a “deferred at scheduling time” check will never run.
Without a way to gate the resize or route it through a scheduler path, deferred RQ alone cannot protect quota on post‑admission resource increases.

Bottom line

• Deferred RQ enforcement is (or could be viewed) a solid precursor that solves the admission-time half of the problem and provides the machinery, state, and scheduler hooks we need.
• To cover KEP‑3521’s resize gap, we must additionally gate resizes or otherwise force resizes to pass through the same deferred validation point.