Skip to content

Binding preemption Design for Karmada scheduler#7327

Draft
seanlaii wants to merge 1 commit intokarmada-io:masterfrom
seanlaii:binding-preemption
Draft

Binding preemption Design for Karmada scheduler#7327
seanlaii wants to merge 1 commit intokarmada-io:masterfrom
seanlaii:binding-preemption

Conversation

@seanlaii
Copy link
Copy Markdown
Contributor

What type of PR is this?
/kind documentation
/kind feature

What this PR does / why we need it:
This PR adds a design proposal for binding-level preemption in the Karmada scheduler.
Currently, priority-based scheduling only orders the queue — a high-priority binding that arrives after cluster resources are consumed by lower-priority bindings remains pending indefinitely. This proposal introduces preemption so the scheduler can evict lower-priority bindings to make room, scoped to single-cluster Divided scheduling in Phase 1 with estimator-based node-level simulation planned for Phase 2.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:


@karmada-bot karmada-bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/documentation Categorizes issue or PR as related to documentation. kind/feature Categorizes issue or PR as related to a new feature. labels Mar 25, 2026
@karmada-bot karmada-bot requested a review from Tingtal March 25, 2026 03:57
@karmada-bot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign seanlaii for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@karmada-bot karmada-bot requested a review from zach593 March 25, 2026 03:57
@karmada-bot karmada-bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Mar 25, 2026
@gemini-code-assist
Copy link
Copy Markdown

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a comprehensive design proposal for binding-level preemption within the Karmada scheduler. The core purpose is to resolve resource contention by allowing higher-priority workloads to preempt lower-priority ones, preventing high-priority tasks from being indefinitely blocked. The proposal outlines a strategic, phased approach, beginning with a foundational summary-based preemption for single-cluster environments and planning for a more advanced estimator-based system in the future. This enhancement significantly improves resource utilization and fairness for critical workloads.

Highlights

  • Binding-Level Preemption Introduced: Introduced binding-level preemption to the Karmada scheduler, enabling high-priority ResourceBindings to evict lower-priority ones when cluster resources are insufficient, initially scoped to single-cluster scenarios.
  • Phased Implementation Strategy: Designed a two-phase implementation, starting with summary-based preemption (Phase 1) and evolving to estimator-based node-level simulation for more precise victim selection (Phase 2).
  • API Extension for Preemption Policy: Added a PreemptionPolicy field to ResourceBindingSpec.SchedulePriority, which defaults to Never and requires explicit opt-in for PreemptLowerPriority.
  • In-Memory Preemption Claims Mechanism: Implemented an in-memory preemption claims store to coordinate preemption, prevent repeated preemption attempts, and reserve cluster capacity for the preemptor.
  • Integration with Existing Karmada Features: Reused the existing SchedulePriority and PriorityClass mechanisms for preemption ordering and integrated the eviction process with GracefulEviction.
  • Feature Gating: Gated the new preemption functionality behind the PriorityBasedPreemptiveScheduling alpha feature gate, ensuring it is opt-in and can be safely rolled out.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a comprehensive proposal for binding-level preemption within the Karmada scheduler, specifically for single-cluster scheduling. The feature aims to allow high-priority ResourceBindings to evict lower-priority ones when cluster resources are insufficient. The proposal details a two-phase implementation, starting with summary-based preemption and progressing to estimator-based preemption for more precise victim selection. A review comment suggests enhancing the clarity of eviction messages for victim bindings by including the workload's kind, namespace, and name for better user understanding.

Comment on lines +456 to +458
workv1alpha2.WithMessage(fmt.Sprintf("Preempted by %s/%s (priority=%d)",
preemptor.Namespace, preemptor.Name,
preemptor.Spec.SchedulePriority.Priority)),
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The eviction message for the victim binding could be more informative. Currently, it uses preemptor.Namespace and preemptor.Name, which refer to the ResourceBinding's metadata. This might be confusing for users, as the ResourceBinding name is often generated (e.g., my-app-deployment) and doesn't directly reflect the workload's identity.

To improve clarity, I suggest using the workload's kind, namespace, and name from preemptor.Spec.Resource. This would generate a more user-friendly message like Preempted by Deployment default/my-app (priority=100).

This change should also be reflected in the Observability section's event table for the Preempted event.

Suggested change
workv1alpha2.WithMessage(fmt.Sprintf("Preempted by %s/%s (priority=%d)",
preemptor.Namespace, preemptor.Name,
preemptor.Spec.SchedulePriority.Priority)),
workv1alpha2.WithMessage(fmt.Sprintf("Preempted by %s %s/%s (priority=%d)",
preemptor.Spec.Resource.Kind, preemptor.Spec.Resource.Namespace, preemptor.Spec.Resource.Name,
preemptor.Spec.SchedulePriority.Priority)),

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Mar 25, 2026

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 42.03%. Comparing base (5ba1066) to head (16f153a).
⚠️ Report is 2 commits behind head on master.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #7327   +/-   ##
=======================================
  Coverage   42.02%   42.03%           
=======================================
  Files         874      874           
  Lines       53551    53551           
=======================================
+ Hits        22505    22509    +4     
+ Misses      29354    29351    -3     
+ Partials     1692     1691    -1     
Flag Coverage Δ
unittests 42.03% <ø> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: seanlaii <qazwsx0939059006@gmail.com>
@seanlaii seanlaii force-pushed the binding-preemption branch from a5c9fa3 to 16f153a Compare March 26, 2026 21:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/documentation Categorizes issue or PR as related to documentation. kind/feature Categorizes issue or PR as related to a new feature. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants