Merge pull request #7583 from thockin/master

Update feature-gates doc re. 'Deprecated'

Author: k8s-ci-robot

contributors/devel/sig-architecture/feature-gates.md

@@ -6,13 +6,13 @@ Feature gates are intended to cover the development life cycle of a feature - th
 
 ## Lifecycle
 
-Feature progress through `Alpha` -> `Beta` -> `GA`. Sometimes we end up deciding that a feature is not going to be supported and we end up marking them as `Deprecated`.
+Features generally progress through `Alpha` -> `Beta` -> `GA`. Sometimes we end up deciding that a feature is not going to be supported and we end up marking them as `Deprecated`.
 
 The majority of features will go through all three stages, but occasionally there are features which may skip stages.
 
 When we add a feature flag, we basically add if/else conditions to ensure that a feature is ONLY activated when either the default is on or if the deployer has switched it on explicitly. When a feature gate is disabled, the system should behave as if the feature doesn't exist. The only exception to this is [API input validation](https://kubernetes.io/docs/reference/using-api/deprecation-policy/#deprecating-parts-of-the-api) on updates, which should preserve and validate data if and only if it was present before the update (which could occur in case of a version rollback).
 
-There is no supported way to trigger a feature gate at runtime for production Kubernetes use. A feature gate is typically toggled by a component restart.
+There is no supported way to change a feature gate at runtime for production Kubernetes use. A feature gate is typically toggled by a component restart.
 
 Unless an exception is granted for a particular feature, as documented and approved as part of [Production Readiness Review], it is expected that:
 
@@ -39,37 +39,49 @@ Unless an exception is granted for a particular feature, as documented and appro
 * `Default` is always set to `false`
 * `LockToDefault` is not set. Defaults to `false`
 
-By default it is not switched on. This enables folks to switch on the feature using the command line. All API changes must start with an Alpha gate, which makes it possible to rollback from future versions.
+By default Alpha features are not switched on. This enables folks to switch on the feature using the command line. All API changes must start with an Alpha gate, which makes it possible to rollback from future versions.
 
 ## Beta Features
 
 * `PreRelease` is set to `featuregate.Beta`
 * `Default` is usually set to `true` (see below)
 * `LockToDefault` is not set. Defaults to `false`
 
-This enables the feature to be on and available in the default installation of Kubernetes. 
+Beta features are usually on by default. This enables the feature to be available in the default installation of Kubernetes.
 
-Sometimes (rarely) the `Default` is set to `false`. This tells folks that while this feature is in Beta, they will still need to do some work to switch it on and use it and potentially take some other explicit action outside of Kubernetes. For example, see the [CSIMigration feature gates](https://github.com/kubernetes/kubernetes/blob/5b0a2c3a29f6b5392e0f8f94ba5669bdc9eb73f6/pkg/features/kube_features.go#L792).
+Sometimes (rarely) the `Default` is set to `false` in Beta. This tells folks that while this feature is Beta, they will still need to do some work to switch it on and use it, and potentially take some other explicit action outside of Kubernetes. For example, the CSIMigration feature gates looked like this:
+
+```
+	CSIMigration:                                   {Default: true, PreRelease: featuregate.Beta},
+	CSIMigrationGCE:                                {Default: false, PreRelease: featuregate.Beta}, // Off by default (requires GCE PD CSI Driver)
+	CSIMigrationAWS:                                {Default: false, PreRelease: featuregate.Beta}, // Off by default (requires AWS EBS CSI driver)
+	CSIMigrationAzureDisk:                          {Default: false, PreRelease: featuregate.Beta}, // Off by default (requires Azure Disk CSI driver)
+	CSIMigrationAzureFile:                          {Default: false, PreRelease: featuregate.Beta}, // Off by default (requires Azure File CSI driver)
+	CSIMigrationvSphere:                            {Default: false, PreRelease: featuregate.Beta}, // Off by default (requires vSphere CSI driver)
+```
 
 ## GA Features
 
 * `PreRelease` is set to `featuregate.GA`
 * `Default` is always set to `true`
 * `LockToDefault` is set to `true`
 
-GA features are always on. 
+GA features are always on by default, and usually cannot be disabled.
 
-Sometimes (rarely) we do not set `LockToDefault` and let it default to `false`. This enables folks to switch off the GA feature. We do this to tell folks that while the feature (for example `coredns`) is GA, they need to move off say `kubedns` to `coredns` in their infrastructures pretty soon and if they want to continue `kubedns` for a short time, they will have to switch off the GA flag. When we do remove the support for `kubedns` entirely we would set `LockToDefault` to `true` with some grace period for the transition.
+Sometimes (rarely) we do not set `LockToDefault` (thus defaulting to `false`, meaning "not locked"). This enables folks to switch off the GA feature. We do this to indicate that while the feature (for example `coredns`) is GA, they need to move off `kubedns` to `coredns` in their infrastructure. If they want to continue using `kubedns` for a short time, they can choose to switch off the GA flag. When we eventually remove the support for `kubedns` entirely we would set `LockToDefault` to `true` with some grace period for the transition.
 
-[After at least two releases post-GA and deprecation](https://kubernetes.io/docs/reference/using-api/deprecation-policy/#deprecation), the feature gate is removed. Typically, we add a comment in [kubefeatures.go](https://github.com/kubernetes/kubernetes/blob/master/pkg/features/kube_features.go) such as: `// remove in 1.23` to signal when we plan to remove the feature gate. Remember when the feature gate is removed and the deployer has forgotten to drop the reference to the feature in the CLI flags (say the `kube-apiserver`), then they will see a hard failure. 
+[After at least two releases post-GA and deprecation](https://kubernetes.io/docs/reference/using-api/deprecation-policy/#deprecation), the feature gate is removed. Typically, we add a comment in [kubefeatures.go](https://github.com/kubernetes/kubernetes/blob/master/pkg/features/kube_features.go) such as: `// remove in 1.23` to signal when we plan to remove the feature gate. Remember when the feature gate is removed and the deployer has forgotten to drop the reference to the feature in the CLI flags (say the `kube-apiserver`), then they will see a hard failure.
 
-Also note that when we set `LockToDefault` to `true`, we remove all references (if/then conditions) to the feature gate from the codebase. 
+Also note that when we set `LockToDefault` to `true`, we remove all references (if/then conditions) to the feature gate from the codebase.
 
 ## Deprecation
 
 * `PreRelease` is set to `featuregate.Deprecated`
+* `Default` is set to `false`
 * See [Kubernetes Deprecation Policy](https://kubernetes.io/docs/reference/using-api/deprecation-policy/#deprecation) for more details
 
+Very rarely we will deprecate some aspect of Kubernetes (almost always something that has no actual impact).  When we do that, the pattern is to name the gate so it describes the functionality being deprecated and to default the value to `false`.  If some user is impacted by the deprecation, they can set that gate to `true` to unbreak themselves (and then file a bug).  If this happens, we must reconsider the deprecation and may choose to abandon it entirely by changing the gate back to `true` for a release or two and eventually removing it.
+
 ## Other scenarios
 
 Sometimes we use`{Default: true, PreRelease: featuregate.Beta}` for keeping legacy behavior on while a new alternative is being implemented. When the new default behavior is transitioning to GA and it is time to drop the old legacy behavior, we will end up with `{Default: false, PreRelease: featuregate.GA, LockToDefault: true}`. For an example, see: [LegacyNodeRoleBehavior & ServiceNodeExclusion](https://github.com/kubernetes/kubernetes/pull/97543/files).