|
| 1 | +--- |
| 2 | +layout: blog |
| 3 | +title: "Introducing Hierarchical Namespaces" |
| 4 | +date: 2020-08-14 |
| 5 | +--- |
| 6 | + |
| 7 | +**Author**: Adrian Ludwin (Google) |
| 8 | + |
| 9 | +Safely hosting large numbers of users on a single Kubernetes cluster has always |
| 10 | +been a troublesome task. One key reason for this is that different organizations |
| 11 | +use Kubernetes in different ways, and so no one tenancy model is likely to suit |
| 12 | +everyone. Instead, Kubernetes offers you building blocks to create your own |
| 13 | +tenancy solution, such as Role Based Access Control (RBAC) and NetworkPolicies; |
| 14 | +the better these building blocks, the easier it is to safely build a multitenant |
| 15 | +cluster. |
| 16 | + |
| 17 | +# Namespaces for tenancy |
| 18 | + |
| 19 | +By far the most important of these building blocks is the namespace, which forms |
| 20 | +the backbone of almost all Kubernetes control plane security and sharing |
| 21 | +policies. For example, RBAC, NetworkPolicies and ResourceQuotas all respect |
| 22 | +namespaces by default, and objects such as Secrets, ServiceAccounts and |
| 23 | +Ingresses are freely usable _within_ any one namespace, but fully segregated |
| 24 | +from _other_ namespaces. |
| 25 | + |
| 26 | +Namespaces have two key properties that make them ideal for policy enforcement. |
| 27 | +Firstly, they can be used to **represent ownership**. Most Kubernetes objects |
| 28 | +_must_ be in a namespace, so if you use namespaces to represent ownership, you |
| 29 | +can always count on there being an owner. |
| 30 | + |
| 31 | +Secondly, namespaces have **authorized creation and use**. Only |
| 32 | +highly-privileged users can create namespaces, and other users require explicit |
| 33 | +permission to use those namespaces - that is, create, view or modify objects in |
| 34 | +those namespaces. This allows them to be carefully created with appropriate |
| 35 | +policies, before unprivileged users can create “regular” objects like pods and |
| 36 | +services. |
| 37 | + |
| 38 | +# The limits of namespaces |
| 39 | + |
| 40 | +However, in practice, namespaces are not flexible enough to meet some common use |
| 41 | +cases. For example, let’s say that one team owns several microservices with |
| 42 | +different secrets and quotas. Ideally, they should place these services into |
| 43 | +different namespaces in order to isolate them from each other, but this presents |
| 44 | +two problems. |
| 45 | + |
| 46 | +Firstly, these namespaces have no common concept of ownership, even though |
| 47 | +they’re both owned by the same team. This means that if the team controls |
| 48 | +multiple namespaces, not only does Kubernetes not have any record of their |
| 49 | +common owner, but namespaced-scoped policies cannot be applied uniformly across |
| 50 | +them. |
| 51 | + |
| 52 | +Secondly, teams generally work best if they can operate autonomously, but since |
| 53 | +namespace creation is highly privileged, it’s unlikely that any member of the |
| 54 | +dev team is allowed to create namespaces. This means that whenever a team wants |
| 55 | +a new namespace, they must raise a ticket to the cluster administrator. While |
| 56 | +this is probably acceptable for small organizations, it generates unnecessary |
| 57 | +toil as the organization grows. |
| 58 | + |
| 59 | +# Introducing hierarchical namespaces |
| 60 | + |
| 61 | +[Hierarchical |
| 62 | +namespaces](https://github.com/kubernetes-sigs/multi-tenancy/blob/master/incubator/hnc/docs/user-guide/concepts.md#basic) |
| 63 | +are a new concept developed by the [Kubernetes Working Group for Multi-Tenancy |
| 64 | +(wg-multitenancy)](https://github.com/kubernetes-sigs/multi-tenancy) in order to |
| 65 | +solve these problems. In its simplest form, a hierarchical namespace is a |
| 66 | +regular Kubernetes namespace that contains a small custom resource that |
| 67 | +identifies a single, optional, parent namespace. This establishes the concept of |
| 68 | +ownership _across_ namespaces, not just _within_ them. |
| 69 | + |
| 70 | +This concept of ownership enables two additional types of behaviours: |
| 71 | + |
| 72 | +* **Policy inheritance:** if one namespace is a child of another, policy objects |
| 73 | + such as RBAC RoleBindings are [copied from the parent to the |
| 74 | + child](https://github.com/kubernetes-sigs/multi-tenancy/blob/master/incubator/hnc/docs/user-guide/concepts.md#basic-propagation). |
| 75 | +* **Delegated creation:** you usually need cluster-level privileges to create a |
| 76 | + namespace, but hierarchical namespaces adds an alternative: |
| 77 | + [_subnamespaces_](https://github.com/kubernetes-sigs/multi-tenancy/blob/master/incubator/hnc/docs/user-guide/concepts.md#basic-subns), |
| 78 | + which can be manipulated using only limited permissions in the parent |
| 79 | + namespace. |
| 80 | + |
| 81 | +This solves both of the problems for our dev team. The cluster administrator can |
| 82 | +create a single “root” namespace for the team, along with all necessary |
| 83 | +policies, and then delegate permission to create subnamespaces to members of |
| 84 | +that team. Those team members can then create subnamespaces for their own use, |
| 85 | +without violating the policies that were imposed by the cluster administrators. |
| 86 | + |
| 87 | +# Hands-on with hierarchical namespaces |
| 88 | + |
| 89 | +Hierarchical namespaces are provided by a Kubernetes extension known as the |
| 90 | +[**Hierarchical Namespace |
| 91 | +Controller**](https://github.com/kubernetes-sigs/multi-tenancy/tree/master/incubator/hnc), |
| 92 | +or **HNC**. The HNC consists of two components: |
| 93 | + |
| 94 | +* The **manager** runs on your cluster, manages subnamespaces, propagates policy |
| 95 | + objects, ensures that your hierarchies are legal and manages extension points. |
| 96 | +* The **kubectl plugin**, called `kubectl-hns`, makes it easy for users to |
| 97 | + interact with the manager. |
| 98 | + |
| 99 | +Both can be easily installed from the [releases page of our |
| 100 | +repo](https://github.com/kubernetes-sigs/multi-tenancy/releases). |
| 101 | + |
| 102 | +Let’s see HNC in action. Imagine that I do not have namespace creation |
| 103 | +privileges, but I can view the namespace `team-a` and create subnamespaces |
| 104 | +within it<sup>[1](#note-1)</sup>. Using the plugin, I can now say: |
| 105 | + |
| 106 | +```bash |
| 107 | +$ kubectl hns create svc1-team-a -n team-a |
| 108 | +``` |
| 109 | + |
| 110 | +This creates a subnamespace called `svc1-team-a`. Note that since subnamespaces |
| 111 | +are just regular Kubernetes namespaces, all subnamespace names must still be |
| 112 | +unique. |
| 113 | + |
| 114 | +I can view the structure of these namespaces by asking for a tree view: |
| 115 | + |
| 116 | +```bash |
| 117 | +$ kubectl hns tree team-a |
| 118 | +# Output: |
| 119 | +team-a |
| 120 | +└── svc1-team-a |
| 121 | +``` |
| 122 | + |
| 123 | +And if there were any policies in the parent namespace, these now appear in the |
| 124 | +child as well<sup>[2](#note-2)</sup>. For example, let’s say that `team-a` had |
| 125 | +an RBAC RoleBinding called `sres`. This rolebinding will also be present in the |
| 126 | +subnamespace: |
| 127 | + |
| 128 | +```bash |
| 129 | +$ kubectl describe rolebinding sres -n svc1-team-a |
| 130 | +# Output: |
| 131 | +Name: sres |
| 132 | +Labels: hnc.x-k8s.io/inheritedFrom=team-a # inserted by HNC |
| 133 | +Annotations: <none> |
| 134 | +Role: |
| 135 | + Kind: ClusterRole |
| 136 | + Name: admin |
| 137 | +Subjects: ... |
| 138 | +``` |
| 139 | + |
| 140 | +Finally, HNC adds labels to these namespaces with useful information about the |
| 141 | +hierarchy which you can use to apply other policies. For example, you can create |
| 142 | +the following NetworkPolicy: |
| 143 | + |
| 144 | +```yaml |
| 145 | +kind: NetworkPolicy |
| 146 | +apiVersion: networking.k8s.io/v1 |
| 147 | +metadata: |
| 148 | + name: allow-team-a |
| 149 | + namespace: team-a |
| 150 | +spec: |
| 151 | + ingress: |
| 152 | + - from: |
| 153 | + - namespaceSelector: |
| 154 | + matchExpressions: |
| 155 | + - key: 'team-a.tree.hnc.x-k8s.io/depth' # Label created by HNC |
| 156 | + operator: Exists |
| 157 | +``` |
| 158 | +
|
| 159 | +This policy will both be propagated to all descendants of `team-a`, and will |
| 160 | +_also_ allow ingress traffic between all of those namespaces. The “tree” label |
| 161 | +can only be applied by HNC, and is guaranteed to reflect the latest hierarchy. |
| 162 | + |
| 163 | +You can learn all about the features of HNC from the [user |
| 164 | +guide](https://github.com/kubernetes-sigs/multi-tenancy/tree/master/incubator/hnc/docs/user-guide). |
| 165 | + |
| 166 | +# Next steps and getting involved |
| 167 | + |
| 168 | +If you think that hierarchical namespaces can work for your organization, [HNC |
| 169 | +v0.5.1 is available on |
| 170 | +GitHub](https://github.com/kubernetes-sigs/multi-tenancy/releases/tag/hnc-v0.5.1). |
| 171 | +We’d love to know what you think of it, what problems you’re using it to solve |
| 172 | +and what features you’d most like to see added. As with all early software, you |
| 173 | +should be cautious about using HNC in production environments, but the more |
| 174 | +feedback we get, the sooner we’ll be able to drive to HNC 1.0. |
| 175 | + |
| 176 | +We’re also open to additional contributors, whether it’s to fix or report bugs, |
| 177 | +or help prototype new features such as exceptions, improved monitoring, |
| 178 | +hierarchical resource quotas or fine-grained configuration. |
| 179 | + |
| 180 | +Please get in touch with us via our |
| 181 | +[repo](https://github.com/kubernetes-sigs/multi-tenancy), [mailing |
| 182 | +list](https://groups.google.com/g/kubernetes-wg-multitenancy) or on |
| 183 | +[Slack](https://kubernetes.slack.com/messages/wg-multitenancy) - we look forward |
| 184 | +to hearing from you! |
| 185 | + |
| 186 | +--- |
| 187 | + |
| 188 | +_[Adrian Ludwin](https://twitter.com/aludwin) is a software engineer and the |
| 189 | +tech lead for the Hierarchical Namespace Controller._ |
| 190 | + |
| 191 | +<a name="note-1"/> |
| 192 | + |
| 193 | +_Note 1: technically, you create a small object called a "subnamespace anchor" |
| 194 | +in the parent namespace, and then HNC creates the subnamespace for you._ |
| 195 | + |
| 196 | +<a name="note-2"/> |
| 197 | + |
| 198 | +_Note 2: By default, only RBAC Roles and RoleBindings are propagated, but you |
| 199 | +can configure HNC to propagate any namespaced Kubernetes object._ |
0 commit comments