DRA: Pair a requeued Workload with quota charges computed for its current generation

[Anjali-Chauhan1]

What type of PR is this?

/kind bug
/area dra

What this PR does / why we need it:

When a DRA-backed Workload is updated while inflight in the scheduler, it can be requeued with its newest object paired with quota charges preprocessed for an older generation.

PushOrUpdate drops updates for the workload the scheduler is processing, assuming requeue re-reads the object anyway. That holds for requests rebuilt from the object, but not for DRA: those charges are resolved by the workload controller and the workload.Info is the only place they exist, so the dropped Info is the only copy pairing the newer object with charges computed for it. RequeueWorkload then fetches the newest object but keeps the charges it already holds, and the scheduling-equivalence hash is derived from that mix — one cycle of over- or under-admission, plus a hash describing neither shape.

This PR sources the charges from a computation made for the generation being requeued:

1. PushOrUpdate captures the Info it would have dropped while a workload is inflight. It is set and cleared together with inflight, so no later cycle can be charged for a workload shape nobody asked for. The capture is only taken while KueueDRAIntegration is enabled; with the gate off, the path is unchanged.
2. RequeueWorkload prefers that capture when its generation matches the object; otherwise, it preserves its own charges while the object has not moved on.
3. If neither describes the newest object, the charges are kept but the hash is dropped to SchedulingHashUnknown, so the mismatch cannot bulk-move unrelated workloads.

Rebuilding from the object is deliberately not an option: the spec carries resourceClaims, not the quota keys they translate to, so a rebuild would requeue the workload asking for almost nothing. Checking the capture before falling back to preservation also covers footprint changes that don't bump the generation, such as a LimitRange default surfacing or a DeviceClass mutating while inflight.

Which issue(s) this PR fixes:

Fixes #14535

Open design question. Case 3 is mitigated, not eliminated: the workload is requeued with one-generation-stale charges and no hash, and self-heals when the reconcile for the current generation re-pushes it. Closing the window fully would mean either firing draReconcileChannel from the requeue path (needs RequeueWorkload restructured to send outside m.Lock(), as addLocalQueueLocked already does) or requeuing to inadmissibleWorkloads under a new requeue reason. Both are left out to keep the diff reviewable — happy to follow up with either.

Re-running DRA preprocessing at requeue was rejected: it needs the client, ResourceSlice cache and DeviceClass mapper, and RequeueWorkload holds m.Lock().

Distinct from #13930 and #14035, which are about the reconciler calling AddOrUpdateWorkload; builds on the preserve-path added in #11927.

Testing. Unit coverage for the new WithTotalRequestsFrom option (including that adopted requests are deep-copied), the capture lifecycle, and all three cases through the real Manager.RequeueWorkload — reverting the fix fails 3 of those 4 cases. The integration test covers the requeue branch but not the capture path: there is no hook to hold a workload inflight deterministically in envtest, and a timing-based spec would just become a flake. Commits are split so each builds and passes on its own.

Does this PR introduce a user-facing change?

DRA: Fixed a bug where a Workload updated while the scheduler was processing it could be requeued with its newest spec paired with quota charges preprocessed for an older generation, causing one scheduling cycle of over- or under-admission and an incorrect scheduling-equivalence hash.

Summary by CodeRabbit

• Bug Fixes

  • Improved handling of workload updates received while scheduling is in progress.
  • Ensured requeued workloads use quota information from the latest workload generation.
  • Fixed stale quota data handling to trigger accurate rescheduling.
  • Improved admission of workloads with translated resource requests after quota becomes available.

• Improvements

  • Added support for preserving and safely reusing precomputed workload resource requests.

[netlify]

✅ Deploy Preview for kubernetes-sigs-kueue ready!

Name	Link
🔨 Latest commit	70b5cbc
🔍 Latest deploy log	https://app.netlify.com/projects/kubernetes-sigs-kueue/deploys/6a83637fe3b77500088c1fe5
😎 Deploy Preview	https://deploy-preview-14562--kubernetes-sigs-kueue.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[kubernetes-prow]

Hi @Anjali-Chauhan1. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Tip

We noticed you've done this a few times! Consider joining the org to skip this step and gain /lgtm and other bot rights. We recommend asking approvers on your previous PRs to sponsor you.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[kubernetes-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Anjali-Chauhan1
Once this PR has been reviewed and has the lgtm label, please assign sohankunkerkar for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

Important

Review skipped

Auto reviews are limited based on label configuration.

🚫 Excluded labels (none allowed) (3)

• needs-ok-to-test
• do-not-merge/work-in-progress
• cncf-cla: no

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: f52b6a65-df2e-4984-832c-a61a72fd0cc9

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 5599e5d1-953f-4dce-a9c1-e537b7afc035

📥 Commits

Reviewing files that changed from the base of the PR and between e5084fe and 1833558.

📒 Files selected for processing (7)

• pkg/cache/queue/cluster_queue.go
• pkg/cache/queue/cluster_queue_test.go
• pkg/cache/queue/manager.go
• pkg/cache/queue/manager_test.go
• pkg/workload/workload.go
• pkg/workload/workload_test.go
• test/integration/singlecluster/controller/dra/dra_test.go

Included review availability: Your plan includes up to 8 reviews per rolling hour; 7 remain after this review.

---

📝 Walkthrough

Walkthrough

The queue now retains DRA workload updates received during inflight scheduling. Requeue logic selects total requests that match the workload generation, while workload rebuilding adopts and clones externally computed requests. Unit and integration tests cover lifecycle cleanup, generation handling, and DRA admission.

Changes

DRA requeue consistency

Layer / File(s)	Summary
Adopted total requests and cloning pkg/workload/workload.go, pkg/workload/workload_test.go	Info accepts precomputed total requests, gives them rebuild precedence, and deep-copies mutable request data. Tests validate precedence and isolation.
Inflight update lifecycle pkg/cache/queue/cluster_queue.go, pkg/cache/queue/cluster_queue_test.go	ClusterQueue captures DRA updates for inflight workloads and clears or consumes them across pop, deletion, requeue, and replacement paths.
Generation-matched requeue pkg/cache/queue/manager.go, pkg/cache/queue/manager_test.go, test/integration/singlecluster/controller/dra/dra_test.go	DRA requeueing adopts matching-generation charges, preserves matching existing charges, and marks stale or unavailable charges as unknown for scheduling equivalence. Tests cover unit cases, queue integration, and logical-resource admission.

Estimated code review effort: 4 (Complex) | ~45 minutes

Merge Risk: ⚪ Minimal · up to 18335

The PR addresses stale DRA quota-charge pairing without any identified current-head merge-blocking risk; it is merge-ready after normal checks and review.

Suggested reviewers: sohankunkerkar, j-skiba

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	The changes capture inflight updates and pair requeued Workloads with current-generation DRA charges, addressing issue #14535.
Out of Scope Changes check	✅ Passed	The production changes and tests directly support the DRA requeue consistency fix described in issue #14535.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the DRA requeue fix and the pairing of Workloads with quota charges for the current generation.

✨ Finishing Touches 💡 2

⚔️ Resolve merge conflicts 💡

• Resolve merge conflict in branch fix/dra-requeue-stale-total-requests

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[tenzen-y]

/ok-to-test