Skip to content

Commit 81d9d1f

Browse files
rogercolltheletterf
rogercoll
and
theletterf
authored
docs: add host and k8s metrics collection examples (#211)
* docs: add host and k8s metrics collection examples * docs: add eimp pipeline example * fix: typos bad spelling * Update configure-metrics-collection.md * Update configure-metrics-collection.md --------- Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
1 parent 583dac3 commit 81d9d1f

File tree

1 file changed

+198
-1
lines changed

1 file changed

+198
-1
lines changed

‎docs/_edot-collector/config/configure-metrics-collection.md‎

Lines changed: 198 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,4 +7,201 @@ nav_order: 4
77

88
# Configure Metrics Collection
99

10-
🚧 Coming soon
10+
This page contains examples and references for customizing metrics collection.
11+
12+
{: .note}
13+
As of Elastic Stack version {{ site.edot_versions.collector }} Elasticsearch Ingest Pipelines are not (yet) applicable to OTel-native data; see [corresponding limitation documentation](../../compatibility/limitations#centralized-parsing-and-processing-of-data).
14+
Hence, we recommend using OTel collector processing pipelines for pre-processing metrics.
15+
{:toc}
16+
17+
18+
## OTLP metrics
19+
20+
Any application emitting metrics through OTLP (OpenTelemetry Protocol) can forward them to the EDOT Collector using the OTLP receiver. This is the recommended method for collecting application-level telemetry.
21+
22+
The following OTLP receiver configuration turns on both gRPC and HTTP protocols for incoming OTLP traffic:
23+
24+
```yaml
25+
# [OTLP Receiver](https://github.com/open-telemetry/opentelemetry-collector/tree/main/receiver/otlpreceiver)
26+
receivers:
27+
otlp:
28+
protocols:
29+
grpc:
30+
endpoint: 0.0.0.0:4317
31+
http:
32+
endpoint: 0.0.0.0:4318
33+
```
34+
35+
Configure your application to export metrics using the OTLP protocol, targeting the endpoints provided in the previous example.
36+
37+
## Host metrics
38+
39+
The [hostmetrics receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/hostmetricsreceiver) turns on the collection of host-level metrics such as CPU use, memory use, and filesystem stats.
40+
41+
The following configuration collects a standard set of host metrics that aligns with Elastic's Infrastructure dashboards in Kibana:
42+
43+
```yaml
44+
hostmetrics:
45+
collection_interval: 10s
46+
root_path: /proc # Mounted node's root file system
47+
scrapers:
48+
cpu:
49+
metrics:
50+
system.cpu.utilization:
51+
enabled: true
52+
system.cpu.logical.count:
53+
enabled: true
54+
memory:
55+
metrics:
56+
system.memory.utilization:
57+
enabled: true
58+
network: {}
59+
processes: {}
60+
load: {}
61+
disk: {}
62+
filesystem:
63+
exclude_mount_points:
64+
mount_points:
65+
- /dev/*
66+
- /proc/*
67+
- /sys/*
68+
- /run/k3s/containerd/*
69+
- /var/lib/docker/*
70+
- /var/lib/kubelet/*
71+
- /snap/*
72+
match_type: regexp
73+
exclude_fs_types:
74+
fs_types:
75+
- autofs
76+
- binfmt_misc
77+
- bpf
78+
- cgroup2
79+
- configfs
80+
- debugfs
81+
- devpts
82+
- devtmpfs
83+
- fusectl
84+
- hugetlbfs
85+
- iso9660
86+
- mqueue
87+
- nsfs
88+
- overlay
89+
- proc
90+
- procfs
91+
- pstore
92+
- rpc_pipefs
93+
- securityfs
94+
- selinuxfs
95+
- squashfs
96+
- sysfs
97+
- tracefs
98+
match_type: strict
99+
```
100+
101+
You must grant access to the /proc filesystem to the receiver by running the Collector with privileged access and mounting /proc and /sys appropriately. Refer to the hostmetrics container use [guide](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/hostmetricsreceiver#collecting-host-metrics-from-inside-a-container-linux-only) (Linux only).
102+
103+
Enabling the process scraper might significantly increase the volume of scraped metrics, potentially impacting performance. Refer to the upstream issue [#39423](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/39423) for discussion.
104+
105+
To ensure compatibility with Kibana's Infrastructure dashboards, include the [elasticinframetrics processor](https://github.com/elastic/opentelemetry-collector-components/tree/main/processor/elasticinframetricsprocessor) in your pipeline:
106+
107+
```yaml
108+
service:
109+
pipelines:
110+
metrics/infra:
111+
receivers:
112+
- hostmetrics
113+
processors:
114+
- elasticinframetrics
115+
```
116+
117+
## Kubernetes metrics
118+
119+
You can collect Kubernetes metrics using multiple receivers depending on the type and source of the metrics. Each receiver might require specific Kubernetes permissions and require a deployment as DaemonSets or singletons.
120+
121+
As with host metrics, use the [elasticinframetrics processor](https://github.com/elastic/opentelemetry-collector-components/tree/main/processor/elasticinframetricsprocessor) to ensure metrics align with the Kibana Infrastructure inventory.
122+
123+
### Kubelet metrics
124+
125+
The [kubeletstats](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/kubeletstatsreceiver) receiver collects resource usage stats directly from the Kubelet's /stats/summary endpoint. Stats include pod-level and node-level metrics.
126+
127+
```yaml
128+
kubeletstats:
129+
auth_type: serviceAccount # Authentication mechanism with the Kubelet endpoint, refer to: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/kubeletstatsreceiver#configuration
130+
collection_interval: 20s
131+
endpoint: ${env:OTEL_K8S_NODE_NAME}:10250
132+
node: '${env:OTEL_K8S_NODE_NAME}'
133+
# Required to work for all CSPs without an issue
134+
insecure_skip_verify: true
135+
k8s_api_config:
136+
auth_type: serviceAccount
137+
metrics:
138+
k8s.pod.memory.node.utilization:
139+
enabled: true
140+
k8s.pod.cpu.node.utilization:
141+
enabled: true
142+
k8s.container.cpu_limit_utilization:
143+
enabled: true
144+
k8s.pod.cpu_limit_utilization:
145+
enabled: true
146+
k8s.container.cpu_request_utilization:
147+
enabled: true
148+
k8s.container.memory_limit_utilization:
149+
enabled: true
150+
k8s.pod.memory_limit_utilization:
151+
enabled: true
152+
k8s.container.memory_request_utilization:
153+
enabled: true
154+
k8s.node.uptime:
155+
enabled: true
156+
k8s.node.cpu.usage:
157+
enabled: true
158+
k8s.pod.cpu.usage:
159+
enabled: true
160+
extra_metadata_labels:
161+
- container.id
162+
```
163+
164+
To capture stats from every node in the cluster, deploy the Collector with the kubeletstats receiver as a DaemonSet.
165+
166+
### Cluster metrics
167+
168+
The [k8sclusterreceiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/k8sclusterreceiver) gathers metrics and entity events directly from the Kubernetes API server. It captures cluster-wide resources like nodes, deployments, pods, and more.
169+
170+
```yaml
171+
k8s_cluster:
172+
auth_type: serviceAccount # Determines how to authenticate to the K8s API server. This can be one of none (for no auth), serviceAccount (to use the standard service account token provided to the agent pod), or kubeConfig to use credentials from ~/.kube/config.
173+
node_conditions_to_report:
174+
- Ready
175+
- MemoryPressure
176+
allocatable_types_to_report:
177+
- cpu
178+
- memory
179+
metrics:
180+
k8s.pod.status_reason:
181+
enabled: true
182+
resource_attributes:
183+
k8s.kubelet.version:
184+
enabled: true
185+
os.description:
186+
enabled: true
187+
os.type:
188+
enabled: true
189+
k8s.container.status.last_terminated_reason:
190+
enabled: true
191+
```
192+
193+
Run a single instance of this receiver, for example as a Deployment, with sufficient permissions to access the K8s API server.
194+
195+
## Other metrics
196+
197+
The EDOT Collector supports a wide range of metrics receivers for popular software systems, including:
198+
199+
- Redis ([redisreceiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/redisreceiver)): Retrieve Redis INFO data from a single Redis instance.
200+
201+
- JMX-based applications ([jmxreceiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/jmxreceiver)): Open a child Java process running the JMX Metric Gatherer configured with your specified JMX connection information and target Groovy script. It then reports metrics to an implicitly created OTLP receiver.
202+
203+
- Prometheus scrape targets ([prometheusreceiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/prometheusreceiver)): Receives metric data in [Prometheus](https://prometheus.io/) format.
204+
205+
- Kafka ([kafkareceiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/kafkareceiver)): Receives telemetry data from Kafka, with configurable topics and encodings.
206+
207+
For a full list of supported receivers, see the EDOT Collector components [reference](https://elastic.github.io/opentelemetry/edot-collector/components).

0 commit comments

Comments
 (0)