Merge pull request #27466 from chrisnegus/prod-env-text

Adding new Production Environment section

Author: k8s-ci-robot

content/en/docs/setup/production-environment/_index.md

@@ -1,4 +1,293 @@
 ---
-title: Production environment
+title: "Production environment"
+description: Create a production-quality Kubernetes cluster
 weight: 30
+no_list: true
 ---
+<!-- overview -->
+
+A production-quality Kubernetes cluster requires planning and preparation.
+If your Kubernetes cluster is to run critical workloads, it must be configured to be resilient.
+This page explains steps you can take to set up a production-ready cluster,
+or to uprate an existing cluster for production use.
+If you're already familiar with production setup and want the links, skip to
+[What's next](#what-s-next).
+
+<!-- body -->
+
+## Production considerations
+
+Typically, a production Kubernetes cluster environment has more requirements than a
+personal learning, development, or test environment Kubernetes. A production environment may require
+secure access by many users, consistent availability, and the resources to adapt
+to changing demands.
+
+As you decide where you want your production Kubernetes environment to live
+(on premises or in a cloud) and the amount of management you want to take
+on or hand to others, consider how your requirements for a Kubernetes cluster
+are influenced by the following issues:
+
+- *Availability*: A single-machine Kubernetes [learning environment](/docs/setup/#learning-environment)
+has a single point of failure. Creating a highly available cluster means considering:
+  - Separating the control plane from the worker nodes.
+  - Replicating the control plane components on multiple nodes.
+  - Load balancing traffic to the cluster’s {{< glossary_tooltip term_id="kube-apiserver" text="API server" >}}.
+  - Having enough worker nodes available, or able to quickly become available, as changing workloads warrant it.
+
+- *Scale*: If you expect your production Kubernetes environment to receive a stable amount of
+demand, you might be able to set up for the capacity you need and be done. However,
+if you expect demand to grow over time or change dramatically based on things like
+season or special events, you need to plan how to scale to relieve increased
+pressure from more requests to the control plane and worker nodes or scale down to reduce unused
+resources.
+
+- *Security and access management*: You have full admin privileges on your own
+Kubernetes learning cluster. But shared clusters with important workloads, and
+more than one or two users, require a more refined approach to who and what can
+access cluster resources. You can use role-based access control
+([RBAC](/docs/reference/access-authn-authz/rbac/)) and other
+security mechanisms to make sure that users and workloads can get access to the
+resources they need, while keeping workloads, and the cluster itself, secure.
+You can set limits on the resources that users and workloads can access
+by managing [policies](https://kubernetes.io/docs/concepts/policy/) and
+[container resources](/docs/concepts/configuration/manage-resources-containers/).
+
+Before building a Kubernetes production environment on your own, consider
+handing off some or all of this job to 
+[Turnkey Cloud Solutions](/docs/setup/production-environment/turnkey-solutions/) 
+providers or other [Kubernetes Partners](https://kubernetes.io/partners/).
+Options include:
+
+- *Serverless*: Just run workloads on third-party equipment without managing
+a cluster at all. You will be charged for things like CPU usage, memory, and
+disk requests.
+- *Managed control plane*: Let the provider manage the scale and availability
+of the cluster's control plane, as well as handle patches and upgrades.
+- *Managed worker nodes*: Configure pools of nodes to meet your needs,
+then the provider makes sure those nodes are available and ready to implement
+upgrades when needed.
+- *Integration*: There are providers that integrate Kubernetes with other
+services you may need, such as storage, container registries, authentication
+methods, and development tools.
+
+Whether you build a production Kubernetes cluster yourself or work with
+partners, review the following sections to evaluate your needs as they relate
+to your cluster’s *control plane*, *worker nodes*, *user access*, and
+*workload resources*.
+
+## Production cluster setup
+
+In a production-quality Kubernetes cluster, the control plane manages the
+cluster from services that can be spread across multiple computers
+in different ways. Each worker node, however, represents a single entity that
+is configured to run Kubernetes pods.
+
+### Production control plane
+
+The simplest Kubernetes cluster has the entire control plane and worker node
+services running on the same machine. You can grow that environment by adding
+worker nodes, as reflected in the diagram illustrated in
+[Kubernetes Components](/docs/concepts/overview/components/).
+If the cluster is meant to be available for a short period of time, or can be
+discarded if something goes seriously wrong, this might meet your needs.
+
+If you need a more permanent, highly available cluster, however, you should
+consider ways of extending the control plane. By design, one-machine control
+plane services running on a single machine are not highly available.
+If keeping the cluster up and running
+and ensuring that it can be repaired if something goes wrong is important,
+consider these steps:
+
+- *Choose deployment tools*: You can deploy a control plane using tools such
+as kubeadm, kops, and kubespray. See
+[Installing Kubernetes with deployment tools](/docs/setup/production-environment/tools/)
+to learn tips for production-quality deployments using each of those deployment
+methods. Different [Container Runtimes](/docs/setup/production-environment/container-runtimes/)
+are available to use with your deployments.
+- *Manage certificates*: Secure communications between control plane services
+are implemented using certificates. Certificates are automatically generated
+during deployment or you can generate them using your own certificate authority.
+See [PKI certificates and requirements](/docs/setup/best-practices/certificates/) for details.
+- *Configure load balancer for apiserver*: Configure a load balancer
+to distribute external API requests to the apiserver service instances running on different nodes. See 
+[Create an External Load Balancer](/docs/tasks/access-application-cluster/create-external-load-balancer/)
+for details.
+- *Separate and backup etcd service*: The etcd services can either run on the
+same machines as other control plane services or run on separate machines, for
+extra security and availability. Because etcd stores cluster configuration data,
+backing up the etcd database should be done regularly to ensure that you can
+repair that database if needed.
+See the [etcd FAQ](https://etcd.io/docs/v3.4/faq/) for details on configuring and using etcd.
+See [Operating etcd clusters for Kubernetes](/docs/tasks/administer-cluster/configure-upgrade-etcd/)
+and [Set up a High Availability etcd cluster with kubeadm](/docs/setup/production-environment/tools/kubeadm/setup-ha-etcd-with-kubeadm/)
+for details.
+- *Create multiple control plane systems*: For high availability, the
+control plane should not be limited to a single machine. If the control plane
+services are run by an init service (such as systemd), each service should run on at
+least three machines. However, running control plane services as pods in
+Kubernetes ensures that the replicated number of services that you request
+will always be available.
+The scheduler should be fault tolerant,
+but not highly available. Some deployment tools set up [Raft](https://raft.github.io/)
+consensus algorithm to do leader election of Kubernetes services. If the
+primary goes away, another service elects itself and take over. 
+- *Span multiple zones*: If keeping your cluster available at all times is
+critical, consider creating a cluster that runs across multiple data centers,
+referred to as zones in cloud environments. Groups of zones are referred to as regions.
+By spreading a cluster across
+multiple zones in the same region, it can improve the chances that your
+cluster will continue to function even if one zone becomes unavailable.
+See [Running in multiple zones](/docs/setup/best-practices/multiple-zones/) for details.
+- *Manage on-going features*: If you plan to keep your cluster over time,
+there are tasks you need to do to maintain its health and security. For example,
+if you installed with kubeadm, there are instructions to help you with
+[Certificate Management](/docs/tasks/administer-cluster/kubeadm/kubeadm-certs/)
+and [Upgrading kubeadm clusters](/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/).
+See [Administer a Cluster](/docs/tasks/administer-cluster/)
+for a longer list of Kubernetes administrative tasks.
+
+To learn about available options when you run control plane services, see
+[kube-apiserver](/docs/reference/command-line-tools-reference/kube-apiserver/),
+[kube-controller-manager](/docs/reference/command-line-tools-reference/kube-controller-manager/),
+and [kube-scheduler](/docs/reference/command-line-tools-reference/kube-scheduler/)
+component pages. For highly available control plane examples, see
+[Options for Highly Available topology](/docs/setup/production-environment/tools/kubeadm/ha-topology/),
+[Creating Highly Available clusters with kubeadm](/docs/setup/production-environment/tools/kubeadm/high-availability/),
+and [Operating etcd clusters for Kubernetes](/docs/tasks/administer-cluster/configure-upgrade-etcd/).
+See [Backing up an etcd cluster](/docs/tasks/administer-cluster/configure-upgrade-etcd/#backing-up-an-etcd-cluster)
+for information on making an etcd backup plan.
+
+### Production worker nodes
+
+Production-quality workloads need to be resilient and anything they rely
+on needs to be resilient (such as CoreDNS). Whether you manage your own
+control plane or have a cloud provider do it for you, you still need to
+consider how you want to manage your worker nodes (also referred to
+simply as *nodes*).  
+
+- *Configure nodes*: Nodes can be physical or virtual machines. If you want to
+create and manage your own nodes, you can install a supported operating system,
+then add and run the appropriate
+[Node services](/docs/concepts/overview/components/#node-components). Consider:
+  - The demands of your workloads when you set up nodes by having appropriate memory, CPU, and disk speed and storage capacity available.
+  - Whether generic computer systems will do or you have workloads that need GPU processors, Windows nodes, or VM isolation.
+- *Validate nodes*: See [Valid node setup](/docs/setup/best-practices/node-conformance/)
+for information on how to ensure that a node meets the requirements to join
+a Kubernetes cluster.
+- *Add nodes to the cluster*: If you are managing your own cluster you can
+add nodes by setting up your own machines and either adding them manually or
+having them register themselves to the cluster’s apiserver. See the
+[Nodes](/docs/concepts/architecture/nodes/) section for information on how to set up Kubernetes to add nodes in these ways.
+- *Add Windows nodes to the cluster*: Kubernetes offers support for Windows
+worker nodes, allowing you to run workloads implemented in Windows containers. See
+[Windows in Kubernetes](/docs/setup/production-environment/windows/) for details.
+- *Scale nodes*: Have a plan for expanding the capacity your cluster will
+eventually need. See [Considerations for large clusters](/docs/setup/best-practices/cluster-large/)
+to help determine how many nodes you need, based on the number of pods and
+containers you need to run. If you are managing nodes yourself, this can mean
+purchasing and installing your own physical equipment.
+- *Autoscale nodes*: Most cloud providers support
+[Cluster Autoscaler](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler#readme)
+to replace unhealthy nodes or grow and shrink the number of nodes as demand requires. See the
+[Frequently Asked Questions](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md)
+for how the autoscaler works and
+[Deployment](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler#deployment)
+for how it is implemented by different cloud providers. For on-premises, there
+are some virtualization platforms that can be scripted to spin up new nodes
+based on demand.
+- *Set up node health checks*: For important workloads, you want to make sure
+that the nodes and pods running on those nodes are healthy. Using the
+[Node Problem Detector](/docs/tasks/debug-application-cluster/monitor-node-health/)
+daemon, you can ensure your nodes are healthy.
+
+## Production user management
+
+In production, you may be moving from a model where you or a small group of
+people are accessing the cluster to where there may potentially be dozens or
+hundreds of people. In a learning environment or platform prototype, you might have a single
+administrative account for everything you do. In production, you will want
+more accounts with different levels of access to different namespaces.
+
+Taking on a production-quality cluster means deciding how you
+want to selectively allow access by other users. In particular, you need to
+select strategies for validating the identities of those who try to access your
+cluster (authentication) and deciding if they have permissions to do what they
+are asking (authorization):
+
+- *Authentication*: The apiserver can authenticate users using client
+certificates, bearer tokens, an authenticating proxy, or HTTP basic auth.
+You can choose which authentication methods you want to use.
+Using plugins, the apiserver can leverage your organization’s existing
+authentication methods, such as LDAP or Kerberos. See
+[Authentication](/docs/reference/access-authn-authz/authentication/)
+for a description of these different methods of authenticating Kubernetes users.
+- *Authorization*: When you set out to authorize your regular users, you will probably choose between RBAC and ABAC authorization. See [Authorization Overview](/docs/reference/access-authn-authz/authorization/) to review different modes for authorizing user accounts (as well as service account access to your cluster):
+  - *Role-based access control* ([RBAC](/docs/reference/access-authn-authz/rbac/)): Lets you assign access to your cluster by allowing specific sets of permissions to authenticated users. Permissions can be assigned for a specific namespace (Role) or across the entire cluster (ClusterRole). Then using RoleBindings and ClusterRoleBindings, those permissions can be attached to particular users.
+  - *Attribute-based access control* ([ABAC](/docs/reference/access-authn-authz/abac/)): Lets you create policies based on resource attributes in the cluster and will allow or deny access based on those attributes. Each line of a policy file identifies versioning properties (apiVersion and kind) and a map of spec properties to match the subject (user or group), resource property, non-resource property (/version or /apis), and readonly. See [Examples](/docs/reference/access-authn-authz/abac/#examples) for details.
+
+As someone setting up authentication and authorization on your production Kubernetes cluster, here are some things to consider:
+
+- *Set the authorization mode*: When the Kubernetes API server
+([kube-apiserver](/docs/reference/command-line-tools-reference/kube-apiserver/))
+starts, the supported authentication modes must be set using the *--authorization-mode*
+flag. For example, that flag in the *kube-adminserver.yaml* file (in */etc/kubernetes/manifests*)
+could be set to Node,RBAC. This would allow Node and RBAC authorization for authenticated requests.
+- *Create user certificates and role bindings (RBAC)*: If you are using RBAC
+authorization, users can create a CertificateSigningRequest (CSR) that can be
+signed by the cluster CA. Then you can bind Roles and ClusterRoles to each user.
+See [Certificate Signing Requests](/docs/reference/access-authn-authz/certificate-signing-requests/)
+for details.
+- *Create policies that combine attributes (ABAC)*: If you are using ABAC
+authorization, you can assign combinations of attributes to form policies to
+authorize selected users or groups to access particular resources (such as a
+pod), namespace, or apiGroup. For more information, see
+[Examples](/docs/reference/access-authn-authz/abac/#examples).
+- *Consider Admission Controllers*: Additional forms of authorization for
+requests that can come in through the API server include
+[Webhook Token Authentication](/docs/reference/access-authn-authz/authentication/#webhook-token-authentication).
+Webhooks and other special authorization types need to be enabled by adding
+[Admission Controllers](/docs/reference/access-authn-authz/admission-controllers/)
+to the API server.
+
+## Set limits on workload resources
+
+Demands from production workloads can cause pressure both inside and outside
+of the Kubernetes control plane. Consider these items when setting up for the
+needs of your cluster's workloads:
+
+- *Set namespace limits*: Set per-namespace quotas on things like memory and CPU. See
+[Manage Memory, CPU, and API Resources](/docs/tasks/administer-cluster/manage-resources/)
+for details. You can also set
+[Hierarchical Namespaces](/blog/2020/08/14/introducing-hierarchical-namespaces/)
+for inheriting limits.
+- *Prepare for DNS demand*: If you expect workloads to massively scale up,
+your DNS service must be ready to scale up as well. See
+[Autoscale the DNS service in a Cluster](/docs/tasks/administer-cluster/dns-horizontal-autoscaling/).
+- *Create additional service accounts*: User accounts determine what users can
+do on a cluster, while a service account defines pod access within a particular
+namespace. By default, a pod takes on the default service account from its namespace.
+See [Managing Service Accounts](/docs/reference/access-authn-authz/service-accounts-admin/)
+for information on creating a new service account. For example, you might want to:
+  - Add secrets that a pod could use to pull images from a particular container registry. See [Configure Service Accounts for Pods](/docs/tasks/configure-pod-container/configure-service-account/) for an example.
+  - Assign RBAC permissions to a service account. See [ServiceAccount permissions](/docs/reference/access-authn-authz/rbac/#service-account-permissions) for details.
+
+## What's next {#what-s-next}
+
+- Decide if you want to build your own production Kubernetes or obtain one from
+available [Turnkey Cloud Solutions](/docs/setup/production-environment/turnkey-solutions/)
+or [Kubernetes Partners](https://kubernetes.io/partners/).
+- If you choose to build your own cluster, plan how you want to
+handle [certificates](/docs/setup/best-practices/certificates/)
+and set up high availability for features such as
+[etcd](/docs/setup/production-environment/tools/kubeadm/setup-ha-etcd-with-kubeadm/)
+and the
+[API server](/docs/setup/production-environment/tools/kubeadm/ha-topology/).
+- Choose from [kubeadm](/docs/setup/production-environment/tools/kubeadm/), [kops](/docs/setup/production-environment/tools/kops/) or [Kubespray](/docs/setup/production-environment/tools/kubespray/)
+deployment methods.
+- Configure user management by determining your
+[Authentication](/docs/reference/access-authn-authz/authentication/) and
+[Authorization](docs/reference/access-authn-authz/authorization/) methods.
+- Prepare for application workloads by setting up
+[resource limits](docs/tasks/administer-cluster/manage-resources/),
+[DNS autoscaling](/docs/tasks/administer-cluster/dns-horizontal-autoscaling/)
+and [service accounts](/docs/reference/access-authn-authz/service-accounts-admin/).