Skip to content

Commit 87b7748

Browse files
NuruNuru
authored
[fluentd] Simplified chart for AWS elasticsearch (#205)
1 parent dc984c1 commit 87b7748

File tree

13 files changed

+427
-0
lines changed

13 files changed

+427
-0
lines changed

‎incubator/fluentd-kubernetes-aws/.helmignore‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# Patterns to ignore when building packages.
2+
# This supports shell glob matching, relative path matching, and
3+
# negation (prefixed with !). Only one pattern per line.
4+
.DS_Store
5+
# Common VCS dirs
6+
.git/
7+
.gitignore
8+
.bzr/
9+
.bzrignore
10+
.hg/
11+
.hgignore
12+
.svn/
13+
# Common backup files
14+
*.swp
15+
*.bak
16+
*.tmp
17+
*~
18+
# Various IDEs
19+
.project
20+
.idea/
21+
*.tmproj

‎incubator/fluentd-kubernetes-aws/Chart.yaml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
apiVersion: v1
2+
description: Collect Kubernetes logs with Fluentd and forward to AWS-hosted Elasticsearch.
3+
icon: https://raw.githubusercontent.com/fluent/fluentd-docs/master/public/logo/Fluentd_square.png
4+
name: fluentd-kubernetes-aws
5+
version: 0.1.0
6+
appVersion: 1.4.2
7+
home: https://www.fluentd.org/
8+
sources:
9+
- https://hub.docker.com/r/fluent/fluentd-kubernetes-daemonset
10+
- https://github.com/fluent/fluentd-kubernetes-daemonset
11+
maintainers:
12+
- name: cloudposse
13+
email: hello@cloudposse.com

‎incubator/fluentd-kubernetes-aws/README.md‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# Fluentd AWS
2+
3+
Helm chart to run [fluentd](https://www.fluentd.org/) on kubernetes and connect to
4+
an AWS Elasticsearch domain protected by IAM.
5+
6+
## Use cases
7+
8+
This specialized chart covers the case where:
9+
- your Kubernetes cluster has RBAC enabled
10+
- you are using [`kiam`](https://github.com/uswitch/kiam) to assign IAM roles to pods
11+
- you have an [AWS Elasticsearch](https://aws.amazon.com/elasticsearch-service/)
12+
- you have created an IAM role that has access to Elasticsearch
13+
- you want Fluentd to collect logs from your Kubernetes cluster and forward them to Elasticsearch.
14+
15+
## Credit
16+
17+
This chart is based on [fluentd-daemonset-elasticsearch-rbac.yaml](https://github.com/fluent/fluentd-kubernetes-daemonset/blob/8c76f51/fluentd-daemonset-elasticsearch-rbac.yaml)
18+
19+
#### Quick start
20+
```
21+
helm install incubator/fluentd-kubernetes-aws \
22+
--set elasticsearch.endpoint=<elasticsearch_domain_endpoint> \
23+
--set role=<IAM role>
24+
```
25+
26+
#### Full config
27+
28+
This chart installs the [fluentd-kubernetes-daemonset](https://github.com/fluent/fluentd-kubernetes-daemonset)
29+
that is specialied to forward logs to Elasticsearch. That installation is entirely configured
30+
with environment variables which are not specifcially documented, but are well named
31+
and can be found by inspecting the templates at https://github.com/fluent/fluentd-kubernetes-daemonset/tree/8c76f51/templates
32+
33+
Those values can be set using `env.NAME=value`
34+
35+
Example `values.yaml` file:
36+
```yaml
37+
image:
38+
repository: fluent/fluentd-kubernetes-daemonset
39+
tag: v1.3.3-debian-elasticsearch-1.8
40+
41+
role: elasticsearch-user
42+
43+
elasticsearch:
44+
endpoint: my-elasticsearch-jivhavxbcd5dvcbjzrac7j42rm.us-west-2.es.amazonaws.com
45+
46+
env:
47+
FLUENT_ELASTICSEARCH_RELOAD_CONNECTIONS: false
48+
FLUENT_ELASTICSEARCH_BUFFER_FLUSH_INTERVAL: 10s
49+
```

‎incubator/fluentd-kubernetes-aws/templates/NOTES.txt‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
To verify that Fluentd has started, run:
2+
3+
kubectl --namespace={{ .Release.Namespace }} get all -l "app={{ template "fluentd_kubernetes.name" . }},release={{ .Release.Name }}"
4+
5+
THIS APPLICATION CAPTURES ALL CONSOLE OUTPUT AND FORWARDS IT TO configured backend storage. Anything that might be identifying,
6+
including things like IP addresses, container images, and object names will NOT be anonymized.

‎incubator/fluentd-kubernetes-aws/templates/_helpers.tpl‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
{{/* vim: set filetype=mustache: */}}
2+
{{/*
3+
Expand the name of the chart.
4+
*/}}
5+
{{- define "fluentd_kubernetes.name" -}}
6+
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
7+
{{- end -}}
8+
9+
{{/*
10+
Create a default fully qualified app name.
11+
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
12+
If release name contains chart name it will be used as a full name.
13+
*/}}
14+
{{- define "fluentd_kubernetes.fullname" -}}
15+
{{- if .Values.fullnameOverride -}}
16+
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
17+
{{- else -}}
18+
{{- $name := default .Chart.Name .Values.nameOverride -}}
19+
{{- if contains $name .Release.Name -}}
20+
{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
21+
{{- else -}}
22+
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
23+
{{- end -}}
24+
{{- end -}}
25+
{{- end -}}
26+
27+
{{/*
28+
Create chart name and version as used by the chart label.
29+
*/}}
30+
{{- define "fluentd_kubernetes.chart" -}}
31+
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
32+
{{- end -}}

‎incubator/fluentd-kubernetes-aws/templates/clusterrole.yaml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
apiVersion: rbac.authorization.k8s.io/v1
2+
kind: ClusterRole
3+
metadata:
4+
name: {{ template "fluentd_kubernetes.fullname" . }}
5+
labels:
6+
app: {{ template "fluentd_kubernetes.name" . }}
7+
chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
8+
release: {{ .Release.Name }}
9+
heritage: {{ .Release.Service }}
10+
rules:
11+
- apiGroups:
12+
- ""
13+
resources:
14+
- pods
15+
- namespaces
16+
verbs:
17+
- get
18+
- list
19+
- watch

‎incubator/fluentd-kubernetes-aws/templates/clusterrolebinding.yaml‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
kind: ClusterRoleBinding
2+
apiVersion: rbac.authorization.k8s.io/v1
3+
metadata:
4+
name: {{ template "fluentd_kubernetes.fullname" . }}
5+
labels:
6+
app: {{ template "fluentd_kubernetes.name" . }}
7+
chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
8+
release: {{ .Release.Name }}
9+
heritage: {{ .Release.Service }}
10+
roleRef:
11+
kind: ClusterRole
12+
name: {{ template "fluentd_kubernetes.fullname" . }}
13+
apiGroup: rbac.authorization.k8s.io
14+
subjects:
15+
- kind: ServiceAccount
16+
name: {{ template "fluentd_kubernetes.fullname" . }}
17+
namespace: {{ .Release.Namespace }}

‎incubator/fluentd-kubernetes-aws/templates/daemonset.yaml‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
apiVersion: apps/v1
2+
kind: DaemonSet
3+
metadata:
4+
name: {{ template "fluentd_kubernetes.fullname" . }}
5+
labels:
6+
k8s-app: fluentd-logging
7+
version: v1
8+
kubernetes.io/cluster-service: "true"
9+
app: {{ template "fluentd_kubernetes.name" . }}
10+
chart: {{ template "fluentd_kubernetes.chart" . }}
11+
release: {{ .Release.Name }}
12+
heritage: {{ .Release.Service }}
13+
spec:
14+
selector:
15+
matchLabels:
16+
k8s-app: fluentd-logging
17+
app: {{ template "fluentd_kubernetes.name" . }}
18+
release: {{ .Release.Name }}
19+
template:
20+
metadata:
21+
labels:
22+
k8s-app: fluentd-logging
23+
version: v1
24+
kubernetes.io/cluster-service: "true"
25+
app: {{ template "fluentd_kubernetes.name" . }}
26+
release: {{ .Release.Name }}
27+
{{- if .Values.role }}
28+
annotations:
29+
iam.amazonaws.com/role: {{ .Values.role }}
30+
{{- end }}
31+
spec:
32+
serviceAccountName: {{ template "fluentd_kubernetes.fullname" . }}
33+
tolerations:
34+
- key: node-role.kubernetes.io/master
35+
effect: NoSchedule
36+
containers:
37+
- name: fluentd
38+
image: {{ .Values.image.repository}}:{{ .Values.image.tag }}
39+
imagePullPolicy: {{ .Values.image.pullPolicy | default "IfNotPresent" }}
40+
env:
41+
{{- if .Values.role }}
42+
- name: FLUENT_ELASTICSEARCH_HOST
43+
value: "localhost"
44+
- name: FLUENT_ELASTICSEARCH_PORT
45+
value: "9200"
46+
- name: FLUENT_ELASTICSEARCH_SCHEME
47+
value: "http"
48+
{{- else }}
49+
- name: FLUENT_ELASTICSEARCH_HOST
50+
value: "{{ .Values.elasticsearch.endpoint }}"
51+
- name: FLUENT_ELASTICSEARCH_PORT
52+
value: "443"
53+
- name: FLUENT_ELASTICSEARCH_SCHEME
54+
value: "https"
55+
{{- end }}
56+
{{- range $name, $value := .Values.env }}
57+
{{- if (not (empty $value)) and (not (eq $name "FLUENT_ELASTICSEARCH_HOST" "FLUENT_ELASTICSEARCH_SCHEME")) }}
58+
- name: {{ $name | quote }}
59+
value: {{ $value | quote }}
60+
{{- end }}
61+
{{- end }}
62+
resources:
63+
{{ toYaml .Values.resources | indent 10 }}
64+
volumeMounts:
65+
- name: varlog
66+
mountPath: /var/log
67+
- name: varlibdockercontainers
68+
mountPath: /var/lib/docker/containers
69+
readOnly: true
70+
{{- if .Values.role }}
71+
- name: signing-proxy
72+
# This image, abutaha/aws-es-proxy:0.9, still has issues, but the Fluentd plugin seems not to be affected by them.
73+
# Still, the image should be updated when possible, but once we find a good image it should not need to be
74+
# updated further until AWS changes their signing algorithm.
75+
# https://github.com/abutaha/aws-es-proxy/issues/27
76+
# https://github.com/abutaha/aws-es-proxy/issues/29
77+
# https://github.com/abutaha/aws-es-proxy/issues/35
78+
# An alternative is mozilla/aws-signing-proxy but as of version 1.0.3 it did not work
79+
# https://github.com/mozilla-services/aws-signing-proxy/issues/9
80+
image: abutaha/aws-es-proxy:0.9
81+
imagePullPolicy: IfNotPresent
82+
args:
83+
- "-endpoint"
84+
- "https://{{ .Values.elasticsearch.endpoint }}"
85+
- "-listen"
86+
- "127.0.0.1:9200"
87+
{{- if .Values.debug.signer }}
88+
- "-pretty"
89+
- "-verbose"
90+
- "-log-to-file"
91+
{{- end }}
92+
resources:
93+
requests:
94+
cpu: 5m
95+
memory: 10Mi
96+
{{- end }}
97+
terminationGracePeriodSeconds: 30
98+
volumes:
99+
- name: varlog
100+
hostPath:
101+
path: /var/log
102+
- name: varlibdockercontainers
103+
hostPath:
104+
path: /var/lib/docker/containers

‎incubator/fluentd-kubernetes-aws/templates/prometheusrule.yaml‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
{{- if .Values.prometheus.createRule }}
2+
# Copied from https://github.com/kiwigrid/helm-charts/blob/416e9ef84ad865846d263e2fccdf0c32ed9fee81/charts/fluentd-elasticsearch/templates/prometheusrule.yaml
3+
apiVersion: monitoring.coreos.com/v1
4+
kind: PrometheusRule
5+
metadata:
6+
name: {{ template "fluentd_kubernetes.fullname" . }}
7+
labels:
8+
app.kubernetes.io/name: {{ include "fluentd_kubernetes.name" . }}
9+
helm.sh/chart: {{ include "fluentd_kubernetes.chart" . }}
10+
app.kubernetes.io/instance: {{ .Release.Name }}
11+
app.kubernetes.io/managed-by: {{ .Release.Service }}
12+
{{- if .Values.prometheus.labels }}
13+
{{- toYaml .Values.prometheus.labels | nindent 4 }}
14+
{{- end }}
15+
spec:
16+
groups:
17+
- name: fluentd
18+
rules:
19+
- alert: FluentdNodeDown
20+
expr: up{job="{{ .Release.Name }}"} == 0
21+
for: 10m
22+
labels:
23+
service: fluentd
24+
severity: warning
25+
annotations:
26+
summary: fluentd cannot be scraped
27+
description: Prometheus could not scrape {{ "{{ $labels.job }}" }} for more than 10 minutes
28+
29+
- alert: FluentdNodeDown
30+
expr: up{job="{{ .Release.Name }}"} == 0
31+
for: 30m
32+
labels:
33+
service: fluentd
34+
severity: critical
35+
annotations:
36+
summary: fluentd cannot be scraped
37+
description: Prometheus could not scrape {{ "{{ $labels.job }}" }} for more than 30 minutes
38+
39+
- alert: FluentdQueueLength
40+
expr: rate(fluentd_status_buffer_queue_length[5m]) > 0.3
41+
for: 1m
42+
labels:
43+
service: fluentd
44+
severity: warning
45+
annotations:
46+
summary: fluentd node are failing
47+
description: In the last 5 minutes, fluentd queues increased 30%. Current value is {{ "{{ $value }}" }}
48+
49+
- alert: FluentdQueueLength
50+
expr: rate(fluentd_status_buffer_queue_length[5m]) > 0.5
51+
for: 1m
52+
labels:
53+
service: fluentd
54+
severity: critical
55+
annotations:
56+
summary: fluentd node are critical
57+
description: In the last 5 minutes, fluentd queues increased 50%. Current value is {{ "{{ $value }}" }}
58+
59+
- alert: FluentdRecordsCountsHigh
60+
expr: sum(rate(fluentd_record_counts{job="{{ .Release.Name }}"}[5m])) BY (instance) > (3 * sum(rate(fluentd_record_counts{job="{{ .Release.Name }}"}[15m])) BY (instance))
61+
for: 1m
62+
labels:
63+
service: fluentd
64+
severity: critical
65+
annotations:
66+
summary: fluentd records count are critical
67+
description: In the last 5m, records counts increased 3 times, comparing to the latest 15 min.
68+
69+
{{- end }}

‎incubator/fluentd-kubernetes-aws/templates/service.yaml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
{{- if .Values.prometheus.createService }}
2+
apiVersion: v1
3+
kind: Service
4+
metadata:
5+
name: {{ template "fluentd_kubernetes.fullname" . }}-prometheus
6+
labels:
7+
app: {{ template "fluentd_kubernetes.name" . }}-prometheus
8+
chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
9+
release: {{ .Release.Name }}
10+
heritage: {{ .Release.Service }}
11+
12+
spec:
13+
type: ClusterIP
14+
ports:
15+
- name: prometheus
16+
port: 9224
17+
protocol: TCP
18+
targetPort: {{ index .Values.env "FLUENTD_PROMETHEUS_PORT" | default "24231" }}
19+
selector:
20+
app: {{ template "fluentd_kubernetes.name" . }}
21+
release: {{ .Release.Name }}
22+
{{- end }}

0 commit comments

Comments
 (0)