Propose KEP 1860 as implementable

Signed-off-by: Patrik Cyvoct <[email protected]>

Author: Sh4d1

keps/sig-network/1860-kube-proxy-IP-node-binding/README.md

@@ -16,17 +16,20 @@
   - [Risks and Mitigations](#risks-and-mitigations)
 - [Drawbacks](#drawbacks)
 - [Alternatives](#alternatives)
+- [Design Details](#design-details)
+- [Test Plan](#test-plan)
+- [Graduation Criteria](#graduation-criteria)
 <!-- /toc -->
 
 ## Release Signoff Checklist
 
 Items marked with (R) are required *prior to targeting to a milestone / release*.
 
-- [ ] (R) Enhancement issue in release milestone, which links to KEP dir in [kubernetes/enhancements] (not the initial KEP PR)
-- [ ] (R) KEP approvers have approved the KEP status as `implementable`
-- [ ] (R) Design details are appropriately documented
-- [ ] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input
-- [ ] (R) Graduation criteria is in place
+- [x] (R) Enhancement issue in release milestone, which links to KEP dir in [kubernetes/enhancements] (not the initial KEP PR)
+- [x] (R) KEP approvers have approved the KEP status as `implementable`
+- [x] (R) Design details are appropriately documented
+- [x] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input
+- [x] (R) Graduation criteria is in place
 - [ ] (R) Production readiness review completed
 - [ ] Production readiness review approved
 - [ ] "Implementation History" section is up-to-date for milestone
@@ -66,31 +69,31 @@ Currently there is some hacky workaround that set the `Hostname` on the `Service
 
 ## Proposal
 
-The solution would be to add a new field in the `loadBalancer` field of a `Service`'s `status`, like `ingressMode`. This new field will be used by kube-proxy in order to not bind the Load Balancer's External IP to the node (in both IPVS and iptables mode). The value `VIP` would be the default one (if not set for instance), keeping the current behaviour. The value `Proxy` would be used in order to disable the the shortcut.
+The solution would be to add a new field in the `loadBalancer` field of a `Service`'s `status`, like `ipMode`. This new field will be used by kube-proxy in order to not bind the Load Balancer's External IP to the node (in both IPVS and iptables mode). The value `VIP` would be the default one (if not set for instance), keeping the current behaviour. The value `Proxy` would be used in order to disable the the shortcut.
 
-Since the `EnsureLoadBalancer` returns a `LoadBalancerStatus`, the Cloud Controller Manager can optionally set the `ingressMode` field before returning the status.
+Since the `EnsureLoadBalancer` returns a `LoadBalancerStatus`, the Cloud Controller Manager can optionally set the `ipMode` field before returning the status.
 
 ### User Stories
 
 #### Story 1
 
-User 1 is a Managed Kubernetes user. There cluster is running on a cloud provider where the LB's behaviour matches the `VIP` `ingressMode`.
+User 1 is a Managed Kubernetes user. There cluster is running on a cloud provider where the LB's behaviour matches the `VIP` `ipMode`.
 Nothing changes for them since the cloud provider manages the Cloud Controller Manager.
 
 #### Story 2
 
-User 2 is a Managed Kubernetes user. There cluster is running on a cloud provider where the LB's behaviour matches the `Proxy` `ingressMode`.
+User 2 is a Managed Kubernetes user. There cluster is running on a cloud provider where the LB's behaviour matches the `Proxy` `ipMode`.
 Almost nothing changes for them since the cloud provider manages the Cloud Controller Manager.
 The only difference is that pods using the load balancer IP may observe a higher response time since the datapath will go through the load balancer. 
 
 #### Story 3
 
 User 3 is a developer working on a cloud provider Kubernetes offering. 
-On the next version of `k8s.io/cloud-provider`, the cloud provider can optionally set the `ingressMode` field as part of `LoadBalancerStatus`, and reflect the cloud provider load balancer behaviour.
+On the next version of `k8s.io/cloud-provider`, the cloud provider can optionally set the `ipMode` field as part of `LoadBalancerStatus`, and reflect the cloud provider load balancer behaviour.
 
 ### Risks and Mitigations
 
-1. The first risk is when using the `Proxy` `ingressMode` for pod using the load balancer IP. In this case the packets will not bypass the load balancer anymore.
+1. The first risk is when using the `Proxy` `ipMode` for pod using the load balancer IP. In this case the packets will not bypass the load balancer anymore.
 However if a user wants to to keep using the in cluster datapath, he can still use the ClusterIP of the service.
 
 ## Drawbacks
@@ -102,3 +105,43 @@ However if a user wants to to keep using the in cluster datapath, he can still u
 A viable alternative may be to use a flag directly on `kube-proxy`.
 When running on a cloud provider managed Kubernetes this solution is viable since the cloud provider will be able to set the right value on `kube-proxy`.
 When running a self hosted cluster, the user needs to be aware of how the cloud's load balancers works and need to set the flag on `kube-proxy` accordingly.
+
+## Design Details
+
+API changes to Service:
+- Add a new field `status.loadBalancer.ingress[].ipMode: VIP | Proxy`.
+- `ipMode` defaults to VIP if the feature gate is enabled, `nil` otherwise, preserving existing behaviour for Service Type=LoadBalancer.
+- On create and update, it fails if `ipMode` is set without the `ip` field.
+
+## Test Plan
+
+Unit tests:
+- unit tests for the ipvs and iptables rules
+- unit tests for the validation
+- unit tests for a new util in pkg/proxy
+
+E2E tests:
+- The default behavior for `ipMode` does not break any existing e2e tests
+- The default `VIP` value is already tested
+
+## Graduation Criteria
+
+### Alpha
+
+Adds new field `ipMode` to Service, which is used when `LoadBalancerIPMode` feature gate is enabled, allowing for rollback.
+
+### Beta/GA
+
+`LoadBalancerIPMode` is enabled by default.
+
+### Upgrade / Downgrade Strategy
+
+On upgrade, while the feature gate is disabled, nothing will change. Once the feature gate is enabled, all the previous LoadBalancer service will get an `ipMode` of `VIP`.
+If `kube-proxy` was not yet upgraded: the field will simply be ignored.
+If `kube-proxy` was upgraded, and the feature gate enabled, it will stil behave as before if the `ipMode` is `VIP`, and will behave accordingly if the `ipMode` is `Proxy`.
+
+On downgrade, the feature gate will simply be disabled, and as long as `kube-proxy` was downgraded before, nothing should be impacted.
+
+### Version Skew Strategy
+
+Version skew from the control plane to `kube-proxy` should be trivial since `kube-proxy` will simply ignore the `ipMode` field.

keps/sig-network/1860-kube-proxy-IP-node-binding/kep.yaml

@@ -5,17 +5,27 @@ authors:
 owning-sig: sig-network
 participating-sigs:
   - sig-cloud-provider
+status: implementable
 reviewers:
   - "@thockin"
   - "@cheftako"
   - "@MorrisLaw"
 approvers:
   - "@thockin"
   - "@andrewsykim"
-editor: "@Sh4d1"
-creation-date: 2019-12-05
-last-updated: 2020-06-18
-status: provisional
-replaces:
-superseded-by:
-see-also:
+
+stage: "alpha"
+
+latest-milestone: "v1.21"
+
+milestone:
+  alpha: "v1.21"
+  beta: "v1.22"
+  stable: "v1.22"
+
+feature-gates:
+  - name: LoadBalancerIPMode
+    components:
+      - kube-apiserver
+      - kube-proxy
+disable-supported: true