fix(kep-3633): redesign matchLabelKeys as matchLabelSelectors to introduce Operator field

Author: sanposhiho

keps/sig-scheduling/3633-matchlabelkeys-to-podaffinity/README.md renamed to keps/sig-scheduling/3633-matchlabelselectors-to-podaffinity/README.md

@@ -58,7 +58,7 @@ If none of those approvers are still appropriate, then changes to that list
 should be approved by the remaining approvers and/or the owning SIG (or
 SIG Architecture for cross-cutting KEPs).
 -->
-# KEP-3633: Introduce MatchLabelKeys to PodAffinity and PodAntiAffinity
+# KEP-3633: Introduce MatchLabelSelectors to PodAffinity and PodAntiAffinity
 
 <!--
 This is the title of your KEP. Keep it short, simple, and descriptive. A good
@@ -85,6 +85,7 @@ tags, and then generate with `hack/update-toc.sh`.
 - [Proposal](#proposal)
   - [User Stories (Optional)](#user-stories-optional)
     - [Story 1](#story-1)
+    - [Story 2](#story-2)
   - [Notes/Constraints/Caveats (Optional)](#notesconstraintscaveats-optional)
   - [Risks and Mitigations](#risks-and-mitigations)
 - [Design Details](#design-details)
@@ -175,7 +176,7 @@ updates.
 [documentation style guide]: https://github.com/kubernetes/community/blob/master/contributors/guide/style-guide.md
 -->
 
-This KEP proposes introducing a complementary field `MatchLabelKeys` to `PodAffinityTerm`.
+This KEP proposes introducing a complementary field `MatchLabelSelectors` to `PodAffinityTerm`.
 This enables users to finely control the scope where Pods are expected to co-exist (PodAffinity)
 or not (PodAntiAffinity), on top of the existing `LabelSelector`.
 
@@ -207,7 +208,7 @@ The same issue applies to other scheduling directives as well. For example, Matc
 
 ### Goals
 
-- Introduce `MatchLabelKeys` in `PodAffinityTerm` to let users define the scope where Pods are evaluated in required and preferred Pod(Anti)Affinity.
+- Introduce `MatchLabelSelectors` in `PodAffinityTerm` to let users define the scope where Pods are evaluated in required and preferred Pod(Anti)Affinity.
 
 ### Non-Goals
 
@@ -216,7 +217,7 @@ What is out of scope for this KEP? Listing non-goals helps to focus discussion
 and make progress.
 -->
 
-- Apply additional internal labels when evaluating `MatchLabelKeys`
+- Apply additional internal labels when evaluating `MatchLabelSelectors`
 
 ## Proposal
 
@@ -245,7 +246,7 @@ and they want only replicas from the same replicaset to be evaluated.
 
 The deployment controller adds [pod-template-hash](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#pod-template-hash-label) to underlying ReplicaSet and thus every Pod created from Deployment carries the hash string.
 
-Therefore, users can use `pod-template-hash` in `MatchlabelKeys` to inform the scheduler to only evaluate Pods with the same `pod-template-hash` value.
+Therefore, users can use `pod-template-hash` in `matchLabelSelector.matchLabelKeys` to inform the scheduler to only evaluate Pods with the same `pod-template-hash` value.
 
 ```yaml
 apiVersion: apps/v1
@@ -263,10 +264,51 @@ metadata:
             values:
             - database
         topologyKey: topology.kubernetes.io/zone
-        matchlabelKeys: # ADDED
-        - pod-template-hash
+        matchLabelSelectors: # ADDED
+        - matchLabelKeys: pod-template-hash
+          operator: In
 ```
 
+#### Story 2
+
+When users want to achieve exclusive 1:1 job to domain placement in [kubernetes-sigs/jobset](https://github.com/kubernetes-sigs/jobset).
+Noone knows the job's name until it's created from JobSet, but each Job and Pod will get [`job-name`](https://github.com/kubernetes-sigs/jobset/blob/749a77c2647ac46b309b8227f66293e4116b97ed/api/v1alpha1/jobset_types.go#L26) label on it.
+
+Users can use `job-name` label to ensure that the Pods with the same `job-name` will land on the same rack 
+and Pods with other `job-name` won't labd on the same rack via `matchLabelSelectors`.
+
+```yaml
+apiVersion: jobset.x-k8s.io/v1alpha1
+kind: JobSet
+metadata:
+  name: pytorch
+  annotations:
+    alpha.jobset.sigs.k8s.io/exclusive-topology: rack
+spec:
+  replicatedJobs:
+    - name: workers
+      template:
+      ...
+        affinity:
+          podAffinity:      # ensures the pods of this job land on the same rack
+              requiredDuringSchedulingIgnoredDuringExecution:
+              - matchLabelSelectors:
+                  - matchLabelKey: job-name
+                    operator: In
+                topologyKey: rack
+            podAntiAffinity: # ensures only this job lands on the rack
+              requiredDuringSchedulingIgnoredDuringExecution:
+              - matchLabelSelectors:
+                  - matchLabelKey: job-name
+                    operator: NotIn
+              - labelSelector:
+                  matchExpressions:
+                  - key: job-name
+                    operator: Exists
+                topologyKey: rack
+```
+
+
 ### Notes/Constraints/Caveats (Optional)
 
 <!--
@@ -297,7 +339,7 @@ Consider including folks who also work outside the SIG or subproject.
 -->
 
 In addition to using `pod-template-hash` added by the Deployment controller, 
-users can also provide the customized key in  `MatchLabelKeys` to identify 
+users can also provide the customized key in  `MatchLabelKey` to identify 
 which pods should be grouped. If so, the user needs to ensure that it is 
 correct and not duplicated with other unrelated workloads.
 
@@ -310,30 +352,44 @@ required) or even code snippets. If there's any ambiguity about HOW your
 proposal will be implemented, this is the place to discuss them.
 -->
 
-A new optional field `MatchLabelKeys` is introduced to `PodAffinityTerm`.
+A new optional field `MatchLabelSelectors` is introduced to `PodAffinityTerm`.
 
 ```go
+type LabelSelectorOperator string
+
+type MatchLabelSelector struct {
+  // MatchLabelKey is used to lookup value from the incoming pod labels, 
+  // and that key-value label is merged with `LabelSelector`.
+  // Key that doesn't exist in the incoming pod labels will be ignored. 
+  MatchLabelKey  string
+  // Operator defines how key-value, fetched via the above `MatchLabelKeys`, is merged into LabelSelector.
+  // If Operator is `In`, `key in (value)` is merged with LabelSelector. 
+  // If Operator is `NotIn`, `key notin (value)` is merged with LabelSelector. 
+  //
+  // +optional
+  Operator       LabelSelectorOperator
+}
+
 type PodAffinityTerm struct {
 	LabelSelector *metav1.LabelSelector
 	Namespaces []string
 	TopologyKey string
 	NamespaceSelector *metav1.LabelSelector
 
-	// MatchLabelKeys is a set of pod label keys to select which pods will
-	// be taken into consideration. The keys are used to lookup values from the
-	// incoming pod labels, those key-value labels are ANDed with `LabelSelector`
-	// to select the group of existing pods which pods will be taken into consideration 
-	// for the incoming pod's pod (anti) affinity. Keys that don't exist in the incoming 
-	// pod labels will be ignored. The default value is empty.
+	// MatchLabelSelectors is a set of pod label keys to select the group of existing pods 
+  // which pods will be taken into consideration for the incoming pod's pod (anti) affinity. 
+  // The default value is empty.
 	// +optional
-	MatchLabelKeys []string
+	MatchLabelSelectors []strinMatchLabelSelectorg
 }
 ```
 
 The inter-Pod Affinity plugin will obtain the labels from the pod 
-labels by the keys in `MatchLabelKeys`. The obtained labels will be merged 
-to `LabelSelector` of `PodAffinityTerm` to filter and group pods. 
-The pods belonging to the same group will be evaluated.
+labels by the key in `MatchLabelKey`. 
+
+The obtained labels will be merged to `LabelSelector` of `PodAffinityTerm` depending on `Operator`. 
+- If Operator is `In`, `key in (value)` is merged with LabelSelector. 
+- If Operator is `NotIn`, `key notin (value)` is merged with LabelSelector. 
 
 ### Test Plan
 
@@ -394,9 +450,9 @@ https://storage.googleapis.com/k8s-triage/index.html
 -->
 
 - These tests will be added.
-  - `MatchLabelKeys` in `PodAffinity` (both in Filter and Score) works as expected.
-  - `MatchLabelKeys` in `PodAntiAffinity` (both in Filter and Score) works as expected.
-  - `MatchLabelKeys` with the feature gate enabled/disabled.
+  - `MatchLabelSelectors` in `PodAffinity` (both in Filter and Score) works as expected.
+  - `MatchLabelSelectors` in `PodAntiAffinity` (both in Filter and Score) works as expected.
+  - `MatchLabelSelectors` with the feature gate enabled/disabled.
 
 **Filter**
 - `k8s.io/kubernetes/test/integration/scheduler/filters/filters_test.go`: https://storage.googleapis.com/k8s-triage/index.html?test=TestPodTopologySpreadFilter
@@ -420,9 +476,12 @@ https://storage.googleapis.com/k8s-triage/index.html
 We expect no non-infra related flakes in the last month as a GA graduation criteria.
 -->
 
-- These e2e tests will be added.
-  - `MatchLabelKeys: ["pod-template-hash"]` in `PodAffinity` (both in Filter and Score) works as expected with a rolling upgrade scenario.
-  - `MatchLabelKeys: ["pod-template-hash"]` in `PodAntiAffinity` (both in Filter and Score) works as expected with a rolling upgrade scenario.
+N/A
+
+--
+
+This feature doesn't introduce any new API endpoints and doesn't interact with other components. 
+So, E2E tests doesn't add extra value to integration tests.
 
 ### Graduation Criteria
 
@@ -491,7 +550,7 @@ in back-to-back releases.
 #### Alpha
 
 - Feature implemented behind a feature flag
-- Unit tests and e2e tests are implemented 
+- Unit tests and integration tests are implemented 
 - No significant performance degradation is observed from the benchmark test
 
 #### Beta
@@ -523,11 +582,11 @@ The previous PodAffinity/PodAntiAffinity behavior will not be broken. Users can
 their Pod specs as it is.
 
 To use this enhancement, users need to enable the feature gate (during this feature is in the alpha.),
-and add `MatchLabelKeys` on their PodAffinity/PodAntiAffinity.
+and add `MatchLabelSelectors` on their PodAffinity/PodAntiAffinity.
 
 **Downgrade**
 
-kube-apiserver will ignore `MatchLabelKeys` in PodAffinity/PodAntiAffinity,
+kube-apiserver will ignore `MatchLabelSelectors` in PodAffinity/PodAntiAffinity,
 and thus, kube-scheduler will also do nothing with it.
 
 ### Version Skew Strategy
@@ -590,7 +649,7 @@ well as the [existing list] of feature gates.
 -->
 
 - [x] Feature gate (also fill in values in `kep.yaml`)
-  - Feature gate name: `MatchLabelKeysInPodAffinityAndPodAntiAffinity`
+  - Feature gate name: `MatchLabelSelectorsInPodAffinity`
   - Components depending on the feature gate: `kube-scheduler`, `kube-apiserver`
 - [ ] Other
 
@@ -619,7 +678,7 @@ NOTE: Also set `disable-supported` to `true` or `false` in `kep.yaml`.
 The feature can be disabled in Alpha and Beta versions
 by restarting kube-apiserver and kube-scheduler with the feature-gate off.
 In terms of Stable versions, users can choose to opt-out by not setting the
-`MatchLabelKeys` field.
+`MatchLabelSelectors` field.
 
 
 ###### What happens if we reenable the feature if it was previously rolled back?

keps/sig-scheduling/3633-matchlabelkeys-to-podaffinity/kep.yaml renamed to keps/sig-scheduling/3633-matchlabelselectors-to-podaffinity/kep.yaml

@@ -15,15 +15,15 @@ see-also:
 
 stage: alpha
 
-latest-milestone: "v1.27"
+latest-milestone: "v1.28"
 
 milestone:
-  alpha: "v1.27"
-  beta: "v1.28"
-  stable: "v1.30"
+  alpha: "v1.28"
+  beta: "v1.29"
+  stable: "v1.31"
 
 feature-gates:
-  - name: MatchLabelKeysInPodAffinity
+  - name: MatchLabelSelectorsInPodAffinity
     components:
       - kube-scheduler
 disable-supported: true