Skip to content

Commit edccc6e

Browse files
committed
draft gateway complete
1 parent afacf1a commit edccc6e

File tree

4 files changed

+339
-33
lines changed

4 files changed

+339
-33
lines changed

content/en/ninja-workshops/10-advanced-otel/10-agent-setup/1-fileexporter.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ Let's run our second exercise:
1212
{{% notice title="Exercise" style="green" icon="running" %}}
1313

1414
- Add `file:` under `exporters:` key
15-
- Add the `path:` key with a value of `"./agent.out"`
15+
- Add `path:` key with a value of `"./agent.out"`
1616
- Add `rotation:` key
1717
- Add `max_megabytes:` key and set a value of `2` # This set the max size for the file exporter output file
1818
- Add `max_backups:` key and set a value of `2` # This will set the max number rotational backup it creates

content/en/ninja-workshops/10-advanced-otel/20-gateway-setup/1-gateway-test.md

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
2-
title: Test your gateway
3-
linkTitle: 2.1 Test your Gateway
2+
title: Test the gateway and prepare the agent
3+
linkTitle: 2.1 Test Gateway, Setup Agent
44
weight: 1
55
---
66

@@ -24,29 +24,29 @@ If you have done everything correctly, the first and the last line of the output
2424

2525
### Change agent config
2626

27-
Open our agent.yaml in your editor and make the following changes:
27+
Open our agent.yaml in your editor and let's add the `OTLP/HTTP` exporter to the agent.yaml - *This is the new preferred exporter for our backend*:
2828

2929
{{% notice title="Exercise" style="green" icon="running" %}}
3030

31-
* Add the following exporter *This is the new preferred exporter for our backend*
31+
- Add `otlphttp:` under the `exporter:` key
32+
- Add `endpoint:` key and set it to a value of `"http://localhost:5318"` * note that we use the port of the gateway
33+
- Add `headers:` key
34+
- Add `X-SF-Token:` key and set it with a fake access token like `"FAKE_SPLUNK_ACCESS_TOKEN"`
3235

33-
```text
34-
otlphttp: exporter
35-
endpoint: entry, with a value of "http://localhost:5318" * using the port of the gateway
36-
headers: entry,
37-
X-SF-Token: entry, with a fake access token like "FAKE_SPLUNK_ACCESS_TOKEN"
38-
```
39-
40-
* Add this as the first exporter to all the sections of the pipelines. (Remove file and leave debug in place)
36+
- Add this as the first exporter in all the `exporter` arrays of the pipelines. (Replace `file` and leave debug in place)
4137

4238
{{% /notice %}}
4339
Again validate the configuration using **[otelbin.io](https://www.otelbin.io/)**, the results should look like this:
4440

4541
![otelbin-g-2-2-w](../../images/gateway-2-2w.png)
4642

4743
{{% notice title="Tip" style="primary" icon="lightbulb" %}}
48-
Use the **otlphttp** exporter as the default method to send traces to Splunk Observability Cloud.
49-
This component is now included in the default configuration of the Splunk Distribution of the OpenTelemetry Collector to send traces and metrics to Splunk Observability Cloud when deploying in host monitoring (agent) mode
44+
The use of the **otlphttp** exporter is now the default method to send metric and traces to Splunk Observability Cloud.
45+
This component is included in the default configuration of the Splunk Distribution of the OpenTelemetry Collector to send traces and metrics to Splunk Observability Cloud when deploying in host monitoring (agent) mode
5046

5147
The older *apm* and *signalfx* exporters you may be familiar with, will be phased out over time
48+
49+
The `headers:` key with its sub key `X-SF-Token:` is the OpenTelemetry way to pass a access token.
50+
51+
To enable the passthrough mode, we did set `include_metadata:` to `true` on the `otlp` receiver in the gateway. It’ll ensure that headers passed to the collector are passed along with the data down in the collector’s pipeline.
5252
{{% /notice %}}

content/en/ninja-workshops/10-advanced-otel/20-gateway-setup/2-test-agent-gateway.md

Lines changed: 315 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ Verify the gateway is running in its own shell and is ready to receive data, the
1212
[WORKSHOP]/otelcol --config=agent.yaml
1313
```
1414

15-
The agent should start to send *cpu* metrics again and both the agent and the gateway should reflect that in their debug output:
15+
The agent should start to send *cpu* metrics again and both the agent and the gateway should reflect that in their debug output similar like the following snippet from the console's:
1616

1717
```text
1818
NumberDataPoints #37
@@ -39,14 +39,325 @@ Value: 0.000000
3939
{"kind": "exporter", "data_type": "metrics", "name": "debug"}
4040
```
4141

42-
Check to see if `gateway-metrics.out` has been created.
42+
At the the moment you started the agent, it did collect the above metrics and send it along via the gateway.
43+
The Gateway should have created a file called `./gateway-metrics.out` as part of the exporting phase of the pipeline service.
4344

44-
Next, run the curl command to send a trace:
45+
Check to see if `gateway-metrics.out` has been created. It should contain at leasts the following metrics for `cpu1`
46+
47+
{{% tabs %}}
48+
{{% tab title="Compact JSON" %}}
49+
50+
```json
51+
{"resourceMetrics":[{"resource":{"attributes":[{"key":"host.name","value":{"stringValue":"YOUR_HOST_NAME"}},{"key":"os.type","value":{"stringValue":"YOUR_OS"}},{"key":"otelcol.service.mode","value":{"stringValue":"agent"}}]},"scopeMetrics":[{"scope":{"name":"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/cpuscraper","version":"v0.116.0"},"metrics":[{"name":"system.cpu.time","description":"Total seconds each logical CPU spent on each mode.","unit":"s","sum":{"dataPoints":[{"attributes":[{"key":"cpu","value":{"stringValue":"cpu0"}},{"key":"state","value":{"stringValue":"user"}}],"startTimeUnixNano":"1733753908000000000","timeUnixNano":"1737133726158376000","asDouble":1168005.59},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu0"}},{"key":"state","value":{"stringValue":"system"}}],"startTimeUnixNano":"1733753908000000000","timeUnixNano":"1737133726158376000","asDouble":504260.9},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu0"}},{"key":"state","value":{"stringValue":"idle"}}],"startTimeUnixNano":"1733753908000000000","timeUnixNano":"1737133726158376000","asDouble":615648.75},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu0"}},{"key":"state","value":{"stringValue":"interrupt"}}],"startTimeUnixNano":"1733753908000000000","timeUnixNano":"1737133726158376000","asDouble":0},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu1"}},{"key":"state","value":{"stringValue":"user"}}],"startTimeUnixNano":"1733753908000000000","timeUnixNano":"1737133726158376000","asDouble":1168999.03},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu1"}},{"key":"state","value":{"stringValue":"system"}}],"startTimeUnixNano":"1733753908000000000","timeUnixNano":"1737133726158376000","asDouble":497785.47},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu1"}},{"key":"state","value":{"stringValue":"idle"}}],"startTimeUnixNano":"1733753908000000000","timeUnixNano":"1737133726158376000","asDouble":621140.39},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu1"}},{"key":"state","value":{"stringValue":"interrupt"}}]}]}}]}]}]}
52+
```
53+
54+
{{% /tab %}}
55+
{{% tab title="Formatted JSON" %}}
56+
57+
```json
58+
{
59+
"resourceMetrics": [
60+
{
61+
"resource": {
62+
"attributes": [
63+
{
64+
"key": "host.name",
65+
"value": {
66+
"stringValue": "YOUR_HOST_NAME"
67+
}
68+
},
69+
{
70+
"key": "os.type",
71+
"value": {
72+
"stringValue": "YOUR_OS"
73+
}
74+
},
75+
{
76+
"key": "otelcol.service.mode",
77+
"value": {
78+
"stringValue": "agent"
79+
}
80+
}
81+
]
82+
},
83+
"scopeMetrics": [
84+
{
85+
"scope": {
86+
"name": "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/cpuscraper",
87+
"version": "v0.116.0"
88+
},
89+
"metrics": [
90+
{
91+
"name": "system.cpu.time",
92+
"description": "Total seconds each logical CPU spent on each mode.",
93+
"unit": "s",
94+
"sum": {
95+
"dataPoints": [
96+
{
97+
"attributes": [
98+
{
99+
"key": "cpu",
100+
"value": {
101+
"stringValue": "cpu0"
102+
}
103+
},
104+
{
105+
"key": "state",
106+
"value": {
107+
"stringValue": "user"
108+
}
109+
}
110+
],
111+
"startTimeUnixNano": "1733753908000000000",
112+
"timeUnixNano": "1737133726158376000",
113+
"asDouble": 1168005.59
114+
},
115+
{
116+
"attributes": [
117+
{
118+
"key": "cpu",
119+
"value": {
120+
"stringValue": "cpu0"
121+
}
122+
},
123+
{
124+
"key": "state",
125+
"value": {
126+
"stringValue": "system"
127+
}
128+
}
129+
],
130+
"startTimeUnixNano": "1733753908000000000",
131+
"timeUnixNano": "1737133726158376000",
132+
"asDouble": 504260.9
133+
},
134+
{
135+
"attributes": [
136+
{
137+
"key": "cpu",
138+
"value": {
139+
"stringValue": "cpu0"
140+
}
141+
},
142+
{
143+
"key": "state",
144+
"value": {
145+
"stringValue": "idle"
146+
}
147+
}
148+
],
149+
"startTimeUnixNano": "1733753908000000000",
150+
"timeUnixNano": "1737133726158376000",
151+
"asDouble": 615648.75
152+
},
153+
{
154+
"attributes": [
155+
{
156+
"key": "cpu",
157+
"value": {
158+
"stringValue": "cpu0"
159+
}
160+
},
161+
{
162+
"key": "state",
163+
"value": {
164+
"stringValue": "interrupt"
165+
}
166+
}
167+
],
168+
"startTimeUnixNano": "1733753908000000000",
169+
"timeUnixNano": "1737133726158376000",
170+
"asDouble": 0
171+
},
172+
{
173+
"attributes": [
174+
{
175+
"key": "cpu",
176+
"value": {
177+
"stringValue": "cpu1"
178+
}
179+
},
180+
{
181+
"key": "state",
182+
"value": {
183+
"stringValue": "user"
184+
}
185+
}
186+
],
187+
"startTimeUnixNano": "1733753908000000000",
188+
"timeUnixNano": "1737133726158376000",
189+
"asDouble": 1168999.03
190+
},
191+
{
192+
"attributes": [
193+
{
194+
"key": "cpu",
195+
"value": {
196+
"stringValue": "cpu1"
197+
}
198+
},
199+
{
200+
"key": "state",
201+
"value": {
202+
"stringValue": "system"
203+
}
204+
}
205+
],
206+
"startTimeUnixNano": "1733753908000000000",
207+
"timeUnixNano": "1737133726158376000",
208+
"asDouble": 497785.47
209+
},
210+
{
211+
"attributes": [
212+
{
213+
"key": "cpu",
214+
"value": {
215+
"stringValue": "cpu1"
216+
}
217+
},
218+
{
219+
"key": "state",
220+
"value": {
221+
"stringValue": "idle"
222+
}
223+
}
224+
],
225+
"startTimeUnixNano": "1733753908000000000",
226+
"timeUnixNano": "1737133726158376000",
227+
"asDouble": 621140.39
228+
},
229+
{
230+
"attributes": [
231+
{
232+
"key": "cpu",
233+
"value": {
234+
"stringValue": "cpu1"
235+
}
236+
},
237+
{
238+
"key": "state",
239+
"value": {
240+
"stringValue": "interrupt"
241+
}
242+
}
243+
]
244+
}
245+
]
246+
}
247+
}
248+
]
249+
}
250+
]
251+
}
252+
]
253+
}
254+
```
255+
256+
{{% /tab %}}
257+
{{% /tabs %}}
258+
Next, make sure both the gateway and the agent are running in their own shell, then in a 3rd shell run the curl command to send a trace:
45259

46260
```sh
47261
curl -X POST -i http://localhost:4318/v1/traces \
48262
-H "Content-Type: application/json" \
49263
-d @trace.json
50264
```
51265

52-
Check for the `gateway-traces.out`
266+
Now the gateway should have generated a new file called `./gateway-traces.out` and it should contain the following trace:
267+
268+
269+
{{% tabs %}}
270+
{{% tab title="Compacted JSON" %}}
271+
272+
```json
273+
{"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"my.service"}},{"key":"deployment.environment","value":{"stringValue":"my.environment"}},{"key":"host.name","value":{"stringValue":"YOUR_HOST_NAME"}},{"key":"os.type","value":{"stringValue":"YOUR_OS"}},{"key":"otelcol.service.mode","value":{"stringValue":"agent"}}]},"scopeSpans":[{"scope":{"name":"my.library","version":"1.0.0","attributes":[{"key":"my.scope.attribute","value":{"stringValue":"some scope attribute"}}]},"spans":[{"traceId":"5b8efff798038103d269b633813fc60c","spanId":"eee19b7ec3c1b174","parentSpanId":"eee19b7ec3c1b173","name":"I'm a server span","kind":2,"startTimeUnixNano":"1544712660000000000","endTimeUnixNano":"1544712661000000000","attributes":[{"value":{"stringValue":"some value"}}],"status":{}}]}],"schemaUrl":"https://opentelemetry.io/schemas/1.6.1"}]}
274+
```
275+
276+
{{% /tab %}}
277+
{{% tab title="Formatted JSON" %}}
278+
279+
```json
280+
{
281+
"resourceSpans": [
282+
{
283+
"resource": {
284+
"attributes": [
285+
{
286+
"key": "service.name",
287+
"value": {
288+
"stringValue": "my.service"
289+
}
290+
},
291+
{
292+
"key": "deployment.environment",
293+
"value": {
294+
"stringValue": "my.environment"
295+
}
296+
},
297+
{
298+
"key": "host.name",
299+
"value": {
300+
"stringValue": "PIHAGEN-M-2QPQ.hagen-ict.nl"
301+
}
302+
},
303+
{
304+
"key": "os.type",
305+
"value": {
306+
"stringValue": "darwin"
307+
}
308+
},
309+
{
310+
"key": "otelcol.service.mode",
311+
"value": {
312+
"stringValue": "agent"
313+
}
314+
}
315+
]
316+
},
317+
"scopeSpans": [
318+
{
319+
"scope": {
320+
"name": "my.library",
321+
"version": "1.0.0",
322+
"attributes": [
323+
{
324+
"key": "my.scope.attribute",
325+
"value": {
326+
"stringValue": "some scope attribute"
327+
}
328+
}
329+
]
330+
},
331+
"spans": [
332+
{
333+
"traceId": "5b8efff798038103d269b633813fc60c",
334+
"spanId": "eee19b7ec3c1b174",
335+
"parentSpanId": "eee19b7ec3c1b173",
336+
"name": "I'm a server span",
337+
"kind": 2,
338+
"startTimeUnixNano": "1544712660000000000",
339+
"endTimeUnixNano": "1544712661000000000",
340+
"attributes": [
341+
{
342+
"value": {
343+
"stringValue": "some value"
344+
}
345+
}
346+
],
347+
"status": {}
348+
}
349+
]
350+
}
351+
],
352+
"schemaUrl": "https://opentelemetry.io/schemas/1.6.1"
353+
}
354+
]
355+
}
356+
```
357+
358+
{{% /tab %}}
359+
{{% /tabs %}}
360+
361+
Note that both the `./gateway-metrics.out` and the `./gateway-traces.out` have a resource metric/attribute key `"otelcol.service.mode"` with a value of `"agent"`
362+
363+
We will correct that in a later section.

0 commit comments

Comments
 (0)