Skip to content

Commit 349eb44

Browse files
JV0812jpipkin1
andauthored
OTRM Troubleshooting doc (#4984)
* OYRM Troubleshooting doc * minor fix * Update troubleshooting.md * Update docs/send-data/opentelemetry-collector/remote-management/troubleshooting.md Co-authored-by: John Pipkin (Sumo Logic) <[email protected]> * Update docs/send-data/opentelemetry-collector/remote-management/troubleshooting.md Co-authored-by: John Pipkin (Sumo Logic) <[email protected]> * Update docs/send-data/opentelemetry-collector/remote-management/troubleshooting.md Co-authored-by: John Pipkin (Sumo Logic) <[email protected]> * Update docs/send-data/opentelemetry-collector/remote-management/troubleshooting.md Co-authored-by: John Pipkin (Sumo Logic) <[email protected]> --------- Co-authored-by: John Pipkin (Sumo Logic) <[email protected]>
1 parent 79ebfde commit 349eb44

File tree

3 files changed

+129
-0
lines changed

3 files changed

+129
-0
lines changed

docs/send-data/opentelemetry-collector/remote-management/index.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,4 +44,10 @@ In this section, we'll introduce the following concepts:
4444
<p>Discover how to apply processing rules for an OpenTelemetry agent using remote management source templates to enhance data handling.</p>
4545
</div>
4646
</div>
47+
<div className="box smallbox card">
48+
<div className="container">
49+
<a href="/docs/send-data/opentelemetry-collector/remote-management/troubleshooting"><img src={useBaseUrl('img/send-data/otel-color.svg')} alt="OTel thumbnail icon" width="25"/><h4>Troubleshooting</h4></a>
50+
<p>Find solutions to common issues with the OpenTelemetry collector remote management.</p>
51+
</div>
52+
</div>
4753
</div>
Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
---
2+
id: troubleshooting
3+
title: Troubleshooting
4+
description: Learn about common troubleshooting scenarios of Sumo Logic OpenTelemetry collector remote management.
5+
---
6+
7+
This document contains common troubleshooting scenarios about the Sumo Logic OpenTelemetry collector remote management.
8+
9+
## Source template specific issues
10+
11+
### Docker: Path permission denied
12+
13+
#### Error message
14+
15+
`
16+
Error: failed to setup configuration components: cannot start pipelines: permission denied while trying to connect to the Docker daemon socket at unix:///cav/run/docker.sock: Get "http://****>
17+
collector server run finished with error: failed to setup configuration components: cannot start pipelines: permission denied while trying to connect to the Docker daemon soc>
18+
`
19+
20+
#### Solution
21+
22+
1. Ensure that you have satisfied the [Docker source template pre-requisites](/docs/send-data/opentelemetry-collector/remote-management/source-templates/docker/#prerequisites).
23+
1. Restart the agent.
24+
25+
### Windows: Failed to apply OpAMP agent remote config
26+
27+
#### Error message
28+
29+
`
30+
Failed to start service: invalid configuration: no receiver configuration specified in config
31+
`
32+
33+
#### Solution
34+
35+
1. Make sure that you have selected the **Event Channels/Scrappers** sub-options for the source template config.
36+
1. Delete the invalid configuration from machine path `/etc/otelcol-sumo/opamp.d` for Linux/Mac operating system or `C:\ProgramData\Sumo Logic\OpenTelemetry Collector\config\opamp.d` for Windows operating system.
37+
1. Update the invalid inputs, delete invalid configuration source templates, or unlink the agent from UI.
38+
1. Restart the agent.
39+
40+
### Apache: Missing hostname / Invalid hostname
41+
42+
#### Error message
43+
44+
`
45+
Failed to apply OpAMP agent remote config {"kind": "extension", "name": "opamp", "error": "cannot save the OpAMP effective config to /etc/otelcol-sumo/opamp.d: cannot validate config: receivers::apache/ce121f50-***nha-aidn: missing hostname: 'localhost'"}
46+
`
47+
48+
#### Solution
49+
50+
1. Ensure that you have satisfied the [Apache source template pre-requisites](/docs/send-data/opentelemetry-collector/remote-management/source-templates/apache/#prerequisites).
51+
1. Delete the invalid configuration from machine path `/etc/otelcol-sumo/opamp.d` for Linux/Mac operating system or `C:\ProgramData\Sumo Logic\OpenTelemetry Collector\config\opamp.d` for Windows operating system.
52+
1. Update the invalid inputs, delete invalid configuration source templates, or unlink the agent from UI.
53+
1. Restart the agent.
54+
55+
### Syslog: Failed to listen on interface
56+
57+
#### Error message
58+
59+
`
60+
Jan 17 11:34:14 ip-***-***-***-*** otelcol-sumo.sh[1234]: Error: failed to setup configuration components: cannot start pipelines: start stanza: failed to listen on interface: failed to configure tcp listener: listen tcp 0.0.0.0:***: bind: permission denied
61+
Jan 17 11:34:14 ip-***-***-***-*** otelcol-sumo.sh[1234]: 2025/01/17 11:34:14 collector server run finished with error: failed to setup configuration components: cannot start pipelines: start stanza: failed to listen on interface: failed to configure tcp listener: listen tcp 0.0.0.0:***: bind: permission denied
62+
`
63+
64+
#### Solution
65+
66+
1. Delete the invalid configuration from machine path `/etc/otelcol-sumo/opamp.d` for Linux/Mac operating system or `C:\ProgramData\Sumo Logic\OpenTelemetry Collector\config\opamp.d` for Windows operating system.
67+
1. Update the invalid inputs (ensure that valid ports have the right permissions), delete invalid configuration source templates, or unlink the agent from UI.
68+
1. Restart the agent.
69+
70+
## Agent disconnected from opamp server
71+
72+
#### Error message
73+
74+
`
75+
Agent Health Warning on UI : agent disconnected from opamp server.
76+
`
77+
78+
#### Solution
79+
80+
Restart the agent.
81+
82+
## Failed to apply OpAMP agent remote config
83+
84+
#### Error messages
85+
86+
Error message 1:
87+
88+
`
89+
Failed to apply OpAMP agent remote config {"kind": "extension", "name": "opamp", "error": "cannot save the OpAMP effective config to /etc/otelcol-sumo/opamp.d: cannot validate config: processors::filter/logs/5c4f1d86-7946-4174-952f-bb52f016d2cd: unable to parse OTTL condition \"IsMatch(body, \\\"~!@#$%^&*()_+{}|:\\\\\\\"<>?/.,';\\\\\\\\][=-`\\\")\": couldn't create function: the pattern supplied to IsMatch is not a valid regexp pattern: error parsing regexp: missing closing ]: [=-""}
90+
`
91+
92+
Error message 2:
93+
94+
`
95+
collector server run finished with error: failed to setup configuration components: failed to build pipelines: failed to create "filelog/8ee0d9e8-92cf-46c8-97be-764c67e59a83" receiver for data type "logs": compile line end regex: error parsing regexp: unexpected
96+
`
97+
98+
#### Solution
99+
100+
1. Ensure the rules comply with [RE2 syntax](https://github.com/google/re2/wiki/Syntax).
101+
1. Delete the invalid configuration from machine path `/etc/otelcol-sumo/opamp.d` for Linux/Mac operating system or `C:\ProgramData\Sumo Logic\OpenTelemetry Collector\config\opamp.d` for Windows operating system.
102+
1. Update the invalid inputs, delete invalid configuration source templates, or unlink the agent from UI. For example, replace the invalid regex with valid regex.
103+
104+
## Agent crashes completely
105+
106+
If there is a runtime error that leads the agent crash unexpectedly, follow the below steps to rectify the issue:
107+
108+
1. Delete the invalid configuration from machine path `/etc/otelcol-sumo/opamp.d` for Linux/Mac operating system or `C:\ProgramData\Sumo Logic\OpenTelemetry Collector\config\opamp.d` for Windows operating system.
109+
1. Update the invalid inputs, delete invalid configuration source templates, or unlink the agent from UI.
110+
1. Restart the agent.
111+
112+
## Agent stuck in the config processing loop
113+
114+
If the agent repeatedly fails to process the effective configuration, it will be unable to perform any other operations, including sending heartbeat signals to the server. The server may interpret the absence of these heartbeats as an indication that the agent is offline, which can complicate recovery efforts. To resolve this issue, follow the below-mentioned steps:
115+
116+
1. Delete the invalid configuration from machine path `/etc/otelcol-sumo/opamp.d` for Linux/Mac operating system or `C:\ProgramData\Sumo Logic\OpenTelemetry Collector\config\opamp.d` for Windows operating system.
117+
1. Update the invalid inputs, delete invalid configuration source templates, or unlink the agent from the user interface.
118+
1. Restart the agent.
119+
120+
## Agent sends failure response to the server
121+
122+
If the agent attempts to apply the effective configuration but encounters errors, such as incompatible or invalid configuration parameters, it will send a failure response to the Opamp server to indicate the problem. This issue, however, disrupts data ingestion and leads to service interruptions. To resolve this, you can update the invalid inputs, delete any invalid configuration source templates, or unlink the agent from the UI.

sidebars.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -257,6 +257,7 @@ module.exports = {
257257
'send-data/opentelemetry-collector/remote-management/processing-rules/mask-rules-windows',
258258
],
259259
},
260+
'send-data/opentelemetry-collector/remote-management/troubleshooting',
260261
],
261262
},
262263
'send-data/opentelemetry-collector/auto-discovery',

0 commit comments

Comments
 (0)