Skip to content

Commit fdc53ff

Browse files
authored
Avoid launching datanodes on labeld nodes (#9)
1 parent ae81469 commit fdc53ff

File tree

2 files changed

+31
-3
lines changed

2 files changed

+31
-3
lines changed

charts/hdfs-datanode-k8s/README.md

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,18 +12,24 @@ HDFS `datanodes` running inside a kubernetes cluster. See the other chart for
1212

1313
### Usage
1414

15-
1. Optionally, find the domain name of your k8s cluster that becomes part of
15+
1. In some setup, the master node may launch a datanode. To prevent this,
16+
label the master node with `hdfs-datanode-exclude`.
17+
```
18+
$ kubectl label node YOUR-MASTER-NAME hdfs-datanode-exclude=yes
19+
```
20+
21+
2. Optionally, find the domain name of your k8s cluster that become part of
1622
pod and service host names. Default is `cluster.local`. See `values.yaml`
1723
for additional parameters to change. You can add them below in `--set`,
1824
as comma-separated entries.
1925

20-
2. Launch this helm chart, `hdfs-datanode-k8s`.
26+
3. Launch this helm chart, `hdfs-datanode-k8s`.
2127

2228
```
2329
$ helm install -n my-hdfs-datanode hdfs-datanode-k8s
2430
```
2531

26-
3. Confirm the daemons are launched.
32+
4. Confirm the daemons are launched.
2733

2834
```
2935
$ kubectl get pods | grep hdfs-datanode-

charts/hdfs-datanode-k8s/templates/datanode-daemonset.yaml

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,28 @@ spec:
2424
metadata:
2525
labels:
2626
name: hdfs-datanode
27+
annotations:
28+
# TODO: The following uses annotation to express affinity
29+
# because Kubernetes 1.5 supports affinity only in annotation as alpha
30+
# feature. In Kubernetes 1.6, this is now beta and supports regular
31+
# YAML construct. Switch to the official syntax when we support 1.6.
32+
scheduler.alpha.kubernetes.io/affinity: >
33+
{
34+
"nodeAffinity": {
35+
"requiredDuringSchedulingIgnoredDuringExecution": {
36+
"nodeSelectorTerms": [
37+
{
38+
"matchExpressions": [
39+
{
40+
"key": "hdfs-datanode-exclude",
41+
"operator": "DoesNotExist"
42+
}
43+
]
44+
}
45+
]
46+
}
47+
}
48+
}
2749
spec:
2850
hostNetwork: true
2951
hostPID: true

0 commit comments

Comments
 (0)