Skip to content

Commit bd5a6b4

Browse files
tmalovetmalove
committed
add new file microshift-telemetry-service
1 parent 6d558b7 commit bd5a6b4

11 files changed

+463
-0
lines changed

‎_topic_maps/_topic_map_ms.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -211,6 +211,8 @@ Topics:
211211
File: microshift-applications
212212
- Name: Deleting or updating Kustomize manifest resources
213213
File: microshift-deleting-resource-manifests
214+
- Name: Configuring Observability
215+
File: microshift-observability-service
214216
- Name: Embedding applications on RHEL for Edge
215217
File: microshift-embedded-apps-on-rhel-edge
216218
- Name: Embedding applications for offline use

‎microshift_install_rpm_opt/microshift-install-optional-rpms.adoc‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,13 @@ include::modules/microshift-install-rpms-olm.adoc[leveloffset=+2]
3434
.Additional resources
3535
* xref:../microshift_running_apps/microshift_operators/microshift-operators-olm.adoc#microshift-operators-olm[Using Operator Lifecycle Manager with {microshift-short}]
3636

37+
include::modules/microshift-otel-install.adoc[leveloffset=+2]
38+
39+
//additional resources for installing Observability
40+
[role="_additional-resources"]
41+
.Additional resources
42+
* xref:../microshift_running_apps/microshift-observability-service.adoc#microshift-observability-service[Using {microshift-short} Observability]
43+
3744
include::modules/microshift-rhoai-install.adoc[leveloffset=+2]
3845

3946
//additional resources for installing RHOAI

‎microshift_running_apps/microshift-observability-service.adoc‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
:_mod-docs-content-type: ASSEMBLY
2+
[id="microshift-observability-service"]
3+
= Using {microshift-short} Observability
4+
include::_attributes/attributes-microshift.adoc[]
5+
:context: microshift-observability-service
6+
7+
toc::[]
8+
9+
{microshift-short} Observability collects and transmits system data for monitoring and analysis. The data includes performance and usage metrics, and error reporting.
10+
11+
include::modules/microshift-otel-install.adoc[leveloffset=+1]
12+
[role="_additional-resources"]
13+
.Additional resources
14+
* xref:../microshift_install_rpm_opt/microshift-install-optional-rpms.adoc#microshift-install-optional-rpms[Installing optional RPM packages]
15+
16+
include::modules/microshift-otel-config.adoc[leveloffset=+1]
17+
18+
// include::modules/microshift-otel-start-service.adoc[leveloffset=+1]
19+
20+
include::modules/microshift-otel-config-examples.adoc[leveloffset=+1]
21+
22+
include::modules/microshift-otel-config-small.adoc[leveloffset=+1]
23+
24+
include::modules/microshift-otel-config-medium.adoc[leveloffset=+1]
25+
26+
include::modules/microshift-otel-config-large.adoc[leveloffset=+1]
27+
28+
include::modules/microshift-otel-verify.adoc[leveloffset=+1]
29+
30+
31+

‎modules/microshift-otel-config-examples.adoc‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
// Module included in the following assemblies:
2+
//
3+
// microshift_running_apps/microshift-observability-service.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="microshift-otel-config-examples_{context}"]
7+
= Selecting a {microshift-short} Observability configuration
8+
9+
The amount and complexity of the data depends on predefined configurations. These configurations determine the number of data sources and the amount of collected data that is transmitted. These configurations are defined as small, medium, and large (default).
10+
11+
The `opentelemetry-collector.yaml` file includes specific parameters that are used to collect data for monitoring the system resources. All warnings for cluster events are included in the collected data. {microshift-short} Observability collects and transmits data for the following resources:
12+
13+
* CPU, memory, disk, and network metrics of containers, pods, and nodes
14+
* Kubernetes events
15+
* Host CPU, memory, disk, and network metrics
16+
* System journals for certain {microshift-short} services, and dependencies
17+
* Metrics exposed by pods that have the `prometheus.io/scrape`: `true` annotation
18+
19+
Replace the values of the `exporters.otlp.endpoint` and `services.telemetry.metrics.readers[0].endpoint` fields with the IP address or host name of the remote back end. This IP address resolves to the local node's host name. Any unreachable endpoint is reported in the {microshift-short} observability service logs.

‎modules/microshift-otel-config-large.adoc‎

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
// Module included in the following assemblies:
2+
//
3+
// microshift_running_apps/microshift-observability-service.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="microshift-otel-config-large_{context}"]
7+
= Selecting a large configuration
8+
9+
You can configure {microshift-short} Observability to collect the maximum amount of performance and resource information, from the maximum number of sources, by updating the YAML file.
10+
11+
. Select a large configuration by adding the following information to the `/etc/microshift/observability/opentelemetry-collector.yaml` file. `Large` is the default configuration.
12+
+
13+
[source,yaml]
14+
----
15+
receivers:
16+
kubeletstats:
17+
auth_type: tls
18+
ca_file: /var/lib/microshift/certs/ca-bundle/client-ca.crt
19+
key_file: /var/lib/microshift/certs/admin-kubeconfig-signer/openshift-observability-client/client.key
20+
cert_file: /var/lib/microshift/certs/admin-kubeconfig-signer/openshift-observability-client/client.crt
21+
insecure_skip_verify: true
22+
collection_interval: 10s
23+
endpoint: "${env:K8S_NODE_NAME}:10250"
24+
node: ${env:K8S_NODE_NAME}
25+
k8s_api_config:
26+
auth_type: kubeConfig
27+
k8s_events:
28+
auth_type: kubeConfig
29+
hostmetrics:
30+
root_path: /
31+
collection_interval: 10s
32+
scrapers:
33+
cpu:
34+
memory:
35+
network:
36+
disk:
37+
filesystem:
38+
journald:
39+
units:
40+
- microshift
41+
- microshift-observability
42+
- microshift-etcd
43+
- crio
44+
- openvswitch.service
45+
- ovsdb-server.service
46+
- ovs-vswitchd.service
47+
priority: info
48+
prometheus:
49+
config:
50+
scrape_configs:
51+
- job_name: k8s
52+
scrape_interval: 10s
53+
kubernetes_sd_configs:
54+
- kubeconfig_file: /var/lib/microshift/resources/observability-client/kubeconfig
55+
role: pod
56+
relabel_configs:
57+
# Only scrape Pods with annotation "prometheus.io/scrape": "true"
58+
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
59+
action: keep
60+
regex: true
61+
# Use value of "prometheus.io/path" annotation for scraping
62+
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
63+
action: replace
64+
target_label: __metrics_path__
65+
regex: (.+)
66+
# Use value of "prometheus.io/port" annotation for scraping
67+
- source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
68+
action: replace
69+
regex: ([^:]+)(?::\d+)?;(\d+)
70+
replacement: $1:$2
71+
target_label: __address__
72+
processors:
73+
batch:
74+
resourcedetection/system:
75+
detectors: [ "system" ]
76+
system:
77+
hostname_sources: [ "os" ]
78+
exporters:
79+
otlp:
80+
sending_queue:
81+
storage: file_storage
82+
endpoint: ${env:OTEL_BACKEND}:4317 <1>
83+
tls:
84+
insecure: true
85+
extensions:
86+
file_storage:
87+
directory: /var/lib/microshift-observability
88+
service:
89+
extensions: [ file_storage ]
90+
pipelines:
91+
metrics/kubeletstats:
92+
receivers: [ kubeletstats ]
93+
processors: [ batch ]
94+
exporters: [ otlp ]
95+
metrics/hostmetrics:
96+
receivers: [ hostmetrics ]
97+
processors: [ resourcedetection/system, batch ]
98+
exporters: [ otlp ]
99+
logs/kube_events:
100+
receivers: [ k8s_events ]
101+
processors: [ resourcedetection/system, batch ]
102+
exporters: [ otlp ]
103+
logs/host:
104+
receivers: [ hostmetrics ]
105+
processors: [ resourcedetection/system ]
106+
exporters: [ otlp ]
107+
logs/journald:
108+
receivers: [ journald ]
109+
processors: [ resourcedetection/system ]
110+
exporters: [ otlp ]
111+
metrics/pods:
112+
receivers: [ prometheus ]
113+
processors: [ batch ]
114+
exporters: [ otlp ]
115+
telemetry:
116+
metrics:
117+
readers:
118+
- periodic:
119+
exporter:
120+
otlp:
121+
protocol: http/protobuf
122+
endpoint: http://${env:OTEL_BACKEND}:4318 <1>
123+
----
124+
<1> Replace the variable `${env:OTEL_BACKEND}` with the IP address or host name of the remote back end. This IP address resolves to the local node's host name. Any unreachable endpoint is reported in the `microshift-observability` service logs.
125+
126+
. Restart {microshift-short} Observability to complete the configuration selection.

‎modules/microshift-otel-config-medium.adoc‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
// Module included in the following assemblies:
2+
//
3+
// microshift_running_apps/microshift-observability-service.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="microshift-otel-config-medium_{context}"]
7+
= Selecting a medium configuration
8+
9+
You can configure {microshift-short} Observability to collect performance and resource information from various sources by updating the YAML file.
10+
11+
. Select a medium configuration by adding the following information to the `/etc/microshift/observability/opentelemetry-collector.yaml` file:
12+
+
13+
[source,yaml]
14+
----
15+
receivers:
16+
kubeletstats:
17+
auth_type: tls
18+
ca_file: /var/lib/microshift/certs/ca-bundle/client-ca.crt
19+
key_file: /var/lib/microshift/certs/admin-kubeconfig-signer/openshift-observability-client/client.key
20+
cert_file: /var/lib/microshift/certs/admin-kubeconfig-signer/openshift-observability-client/client.crt
21+
insecure_skip_verify: true
22+
collection_interval: 10s
23+
endpoint: "${env:K8S_NODE_NAME}:10250"
24+
node: ${env:K8S_NODE_NAME}
25+
k8s_api_config:
26+
auth_type: kubeConfig
27+
k8s_events:
28+
auth_type: kubeConfig
29+
journald:
30+
units:
31+
- microshift
32+
priority: info
33+
processors:
34+
batch:
35+
resourcedetection/system:
36+
detectors: [ "system" ]
37+
system:
38+
hostname_sources: [ "os" ]
39+
exporters:
40+
otlp:
41+
sending_queue:
42+
storage: file_storage
43+
endpoint: ${env:OTEL_BACKEND}:4317 <1>
44+
tls:
45+
insecure: true
46+
extensions:
47+
file_storage:
48+
directory: /var/lib/microshift-observability
49+
service:
50+
extensions: [ file_storage ]
51+
pipelines:
52+
metrics/kubeletstats:
53+
receivers: [ kubeletstats ]
54+
processors: [ batch ]
55+
exporters: [ otlp ]
56+
logs/kube_events:
57+
receivers: [ k8s_events ]
58+
processors: [ resourcedetection/system, batch ]
59+
exporters: [ otlp ]
60+
logs/journald:
61+
receivers: [ journald ]
62+
processors: [ resourcedetection/system ]
63+
exporters: [ otlp ]
64+
telemetry:
65+
metrics:
66+
readers:
67+
- periodic:
68+
exporter:
69+
otlp:
70+
protocol: http/protobuf
71+
endpoint: http://${env:OTEL_BACKEND}:4318 <1>
72+
----
73+
<1> Replace the variable `${env:OTEL_BACKEND}` with the IP address or host name of the remote back end. This IP address resolves to the local node's host name. Any unreachable endpoint is reported in the `microshift-observability` service logs.
74+
75+
. Restart {microshift-short} Observability to complete the configuration selection.

‎modules/microshift-otel-config-small.adoc‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
// Module included in the following assemblies:
2+
//
3+
// microshift_running_apps/microshift-observability-service.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="microshift-otel-config-small_{context}"]
7+
= Selecting a small configuration
8+
9+
You can configure {microshift-short} Observability to collect the smallest amount of performance and resource information from various sources by updating the YAML file.
10+
11+
. Select a small configuration by adding the following information to the `/etc/microshift/observability/opentelemetry-collector.yaml` file:
12+
+
13+
[source,yaml]
14+
----
15+
receivers:
16+
kubeletstats:
17+
auth_type: tls
18+
ca_file: /var/lib/microshift/certs/ca-bundle/client-ca.crt
19+
key_file: /var/lib/microshift/certs/admin-kubeconfig-signer/openshift-observability-client/client.key
20+
cert_file: /var/lib/microshift/certs/admin-kubeconfig-signer/openshift-observability-client/client.crt
21+
insecure_skip_verify: true
22+
collection_interval: 10s
23+
endpoint: "${env:K8S_NODE_NAME}:10250"
24+
node: ${env:K8S_NODE_NAME}
25+
k8s_api_config:
26+
auth_type: kubeConfig
27+
k8s_events:
28+
auth_type: kubeConfig
29+
processors:
30+
batch:
31+
resourcedetection/system:
32+
detectors: [ "system" ]
33+
system:
34+
hostname_sources: [ "os" ]
35+
exporters:
36+
otlp:
37+
sending_queue:
38+
storage: file_storage
39+
endpoint: ${env:OTEL_BACKEND}:4317 <1>
40+
tls:
41+
insecure: true
42+
extensions:
43+
file_storage:
44+
directory: /var/lib/microshift-observability
45+
service:
46+
extensions: [ file_storage ]
47+
pipelines:
48+
metrics/kubeletstats:
49+
receivers: [ kubeletstats ]
50+
processors: [ batch ]
51+
exporters: [ otlp ]
52+
logs/kube_events:
53+
receivers: [ k8s_events ]
54+
processors: [ resourcedetection/system, batch ]
55+
exporters: [ otlp ]
56+
telemetry:
57+
metrics:
58+
readers:
59+
- periodic:
60+
exporter:
61+
otlp:
62+
protocol: http/protobuf
63+
endpoint: http://${env:OTEL_BACKEND}:4318 <1>
64+
----
65+
<1> Replace the variable `${env:OTEL_BACKEND}` with the IP address or host name of the remote back end. This IP address resolves to the local node's host name. Any unreachable endpoint is reported in the {microshift-short} service logs.
66+
67+
. Restart {microshift-short} Observability to complete the configuration selection.

0 commit comments

Comments
 (0)