Skip to content

Commit 9348e8d

Browse files
committed
added a wrap-up and completed work on routing
1 parent c253385 commit 9348e8d

File tree

4 files changed

+123
-192
lines changed

4 files changed

+123
-192
lines changed

content/en/ninja-workshops/10-advanced-otel/80-routing/8-1-routing-test.md

Lines changed: 110 additions & 113 deletions
Original file line numberDiff line numberDiff line change
@@ -4,137 +4,134 @@ linkTitle: 8.1 Testing Routing
44
weight: 1
55
---
66

7-
## WORK IN PROGRESS
7+
### Setup Test
88

9-
Step 1: Setting Up the Directory Structure
10-
First, ensure you have the required directory structure for the checkpoint folder and log files:
9+
In this section, we will test the `routing` rule configured for the `gateway`. The expected result is that the`span` from the `security.json` file will be sent to the `gateway-traces-security.out` file.
1110

12-
```bash
13-
your_project/
14-
├── checkpoint-folder/ # Directory for checkpoint files
15-
├── agent-standard.out # Output file for standard traces
16-
├── agent-security.out # Output file for security traces
17-
└── otel-collector-config.yaml # OpenTelemetry Collector config file
18-
```
19-
20-
Also, configure a Filelog receiver to read log data from files, but this is currently commented out.
11+
{{% notice title="Exercise" style="green" icon="running" %}}
2112

22-
2.3 Exporters
23-
There are multiple exporters configured:
13+
- **Run the Gateway**:
14+
Find your `Gateway` terminal window, and navigate to the `[WORKSHOP]/8-routing` folder and restart the gateway with the routing configurations specified in the gateway.yaml.
15+
It should start up normally and state : `Everything is ready. Begin running and processing data.`
2416

25-
Debug Exporter: Outputs trace data in detailed verbosity for debugging.
26-
File Exporters: Writes trace data to files with rotation policies.
27-
OTLP Gateway Exporter: Sends trace data to a remote OTLP endpoint with queuing and retry options.
17+
- **Run the Agent**:
18+
Find your `Agent` terminal window and navigate to the `[WORKSHOP]/8-routing` folder and restart the agent with the regular `agent.yaml`.
19+
It should also start up normally and state : `Everything is ready. Begin running and processing data.`
2820

29-
2.4 Connectors: Routing
21+
- **Create a trace for different Environments**
22+
Find your `Test` terminal window and navigate to the `[WORKSHOP]/8-routing`.
23+
To test your configuration, you need to generate span data with the correct `ResourceSpan` attributes to trigger the routing rule. Copy the following JSON and save it as `security.json` in the `[WORKSHOP]\8-routing` directory.
3024

31-
The routing connector routes traces based on attributes. Here, traces with the attribute `deployment.environment == "security_applications"` are routed to a separate pipeline, `traces/security`:
25+
{{% tabs %}}
26+
{{% tab title="Compacted JSON" %}}
3227

33-
```yaml
34-
connectors:
35-
routing:
36-
default_pipelines: [traces/standard] # Default pipeline for traces
37-
error_mode: ignore # Ignore errors during routing
38-
table:
39-
- statement: route() where attributes["deployment.environment"] == "security_applications"
40-
pipelines: [traces/security]
28+
```json
29+
{"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"password_check"}},{"key":"deployment.environment","value":{"stringValue":"security_applications"}}]},"scopeSpans":[{"scope":{"name":"my.library","version":"1.0.0","attributes":[{"key":"my.scope.attribute","value":{"stringValue":"some scope attribute"}}]},"spans":[{"traceId":"5B8EFFF798038103D269B633813FC60C","spanId":"EEE19B7EC3C1B174","parentSpanId":"EEE19B7EC3C1B173","name":"I'm a server span","startTimeUnixNano":"1544712660000000000","endTimeUnixNano":"1544712661000000000","kind":2,"attributes":[{"keytest":"my.span.attr","value":{"stringValue":"some value"}}]}]}]}]}
4130
```
4231

43-
You can add additional routing rules by specifying different attribute conditions. The second routing rule is commented out but could be used to delete a specific key from the attributes before routing the data.
44-
45-
2.5 Processors
46-
Processors allow you to manipulate and enrich data before exporting it:
47-
48-
Batch Processor: Batches data to improve performance.
49-
Memory Limiter: Ensures that the collector doesn’t consume too much memory.
50-
Resource Detection: Adds system resource attributes to the traces.
51-
Resource Mode: Adds custom attributes to traces.
52-
53-
```yaml
54-
processors:
55-
batch:
56-
metadata_keys:
57-
- X-SF-Token # Include metadata in batches
58-
memory_limiter:
59-
check_interval: 2s
60-
limit_mib: 512 # Limits memory usage to 512 MB
61-
resourcedetection:
62-
detectors: [system]
63-
override: true # Overrides existing resource detection
64-
resource/add_mode:
65-
attributes:
66-
- action: insert
67-
value: "agent"
68-
key: otelcol.service.mode
32+
{{% /tab %}}
33+
{{% tab title="Formatted JSON" %}}
34+
35+
```json
36+
{
37+
"resourceSpans": [
38+
{
39+
"resource": {
40+
"attributes": [
41+
{
42+
"key": "service.name",
43+
"value": {
44+
"stringValue": "password_check"
45+
}
46+
},
47+
{
48+
"key": "deployment.environment",
49+
"value": {
50+
"stringValue": "security_applications"
51+
}
52+
}
53+
]
54+
},
55+
"scopeSpans": [
56+
{
57+
"scope": {
58+
"name": "my.library",
59+
"version": "1.0.0",
60+
"attributes": [
61+
{
62+
"key": "my.scope.attribute",
63+
"value": {
64+
"stringValue": "some scope attribute"
65+
}
66+
}
67+
]
68+
},
69+
"spans": [
70+
{
71+
"traceId": "5B8EFFF798038103D269B633813FC60C",
72+
"spanId": "EEE19B7EC3C1B174",
73+
"parentSpanId": "EEE19B7EC3C1B173",
74+
"name": "I'm a server span",
75+
"startTimeUnixNano": "1544712660000000000",
76+
"endTimeUnixNano": "1544712661000000000",
77+
"kind": 2,
78+
"attributes": [
79+
{
80+
"keytest": "my.span.attr",
81+
"value": {
82+
"stringValue": "some value"
83+
}
84+
}
85+
]
86+
}
87+
]
88+
}
89+
]
90+
}
91+
]
92+
}
6993
```
7094

71-
2.6 Service Pipelines
72-
The service section defines how the traces, logs, and metrics will flow through the system, from receivers to processors and exporters.
73-
74-
```yaml
75-
service:
76-
extensions: [file_storage/checkpoint]
77-
pipelines:
78-
traces:
79-
receivers: [otlp] # Receives traces via OTLP
80-
exporters: [routing, debug] # Routes and debugs traces
81-
traces/standard:
82-
receivers: [routing] # Receives routed traces
83-
processors:
84-
- memory_limiter
85-
- batch
86-
- resourcedetection
87-
- resource/add_mode
88-
exporters: [file/standard] # Exports to standard file
89-
traces/security:
90-
receivers: [routing] # Receives routed security traces
91-
processors:
92-
- memory_limiter
93-
- batch
94-
- resourcedetection
95-
- resource/add_mode
96-
exporters: [file/security] # Exports to security file
97-
metrics:
98-
receivers: [otlp]
99-
processors:
100-
- memory_limiter
101-
- batch
102-
- resourcedetection
103-
exporters: [file/standard, debug]
104-
logs:
105-
receivers: [otlp]
106-
processors:
107-
- memory_limiter
108-
- batch
109-
- resourcedetection
110-
exporters: [file/standard, debug]
111-
traces/standard: Default pipeline for traces that do not match the routing condition.
112-
traces/security: Pipeline for security-related traces routed by the connector.
113-
```
95+
{{% /tab %}}
96+
{{% /tabs %}}
11497

115-
Step 3: Running OpenTelemetry Collector
98+
{{% /notice %}}
11699

117-
Create the configuration file: Copy the YAML configuration into a file named otel-collector-config.yaml.
100+
### Testing the routing scenario
118101

119-
Start the OpenTelemetry Collector:
102+
{{% notice title="Exercise" style="green" icon="running" %}}
103+
Make sure the are **NO** `*.out` files in the `[WORKSHOP]/8-routing` folder.
120104

121-
```bash
122-
otelcol --config=otel-collector-config.yaml
123-
```
105+
- **Send a Regular Span**:
106+
Find your `Test` terminal window and navigate to the `[WORKSHOP]/8-routing` folder. From there, send a regular span using the `trace.json` file to confirm that communication is functioning as expected.
107+
Both the `agent` and `gateway` should display debug information, including the span you just sent. Additionally, the gateway should generate a new `./gateway-traces-default.out` file, as this is now the destination for regular spans.
108+
109+
{{% notice title="Tip" style="primary" icon="lightbulb" %}}
110+
If you verify the `./gateway-traces-default.out` is should contain the span we send with the cURL command.
111+
{{% /notice %}}
112+
113+
- **Send a Security Span**:
114+
Make sure both the `agent` and `gateway` are running, then send a security span using the `security.json` file to test the routing rule in the gateway.
115+
Both the `agent` and `gateway` should display debug information, including the span you just sent. Additionally, the gateway should generate a new `./gateway-traces-security.out` file, as this is the destination for spans where the `deployment.environment` resourceSpan attribute matches `"security_applications"`.
116+
117+
{{% notice title="Tip" style="primary" icon="lightbulb" %}}
118+
If you verify the `./gateway-traces-security.out` is should only contain the spans from the `"security_applications"` deployment.environment.
119+
{{% /notice %}}
120+
121+
{{% /notice %}}
122+
123+
### Conclusion
124+
125+
In this section, we successfully tested the routing connector in the gateway by sending different spans and verifying their destinations.
124126

125-
Verify that the Collector is running: Check the logs for any errors. If there are no errors, traces should be flowing through the collector.
127+
- **Regular spans** were correctly routed to `gateway-traces-default.out`, confirming that spans without a matching `deployment.environment` attribute follow the default pipeline.
126128

127-
Step 4: Testing the Routing
128-
To test the routing:
129+
- **Security-related spans** from `security.json` were routed to `gateway-traces-security.out`, demonstrating that the routing rule based on `"deployment.environment": "security_applications"` works as expected.
129130

130-
Generate or simulate trace data with the attribute `deployment.environment = security_applications`.
131-
Verify that the traces are routed to the traces/security pipeline by checking the `agent-security.out` file.
132-
You can also check `agent-standard.out` to ensure that other traces are routed to the standard pipeline.
131+
By inspecting the output files, we confirmed that the OpenTelemetry Collector *correctly evaluates span attributes and routes them to the appropriate destinations*. This validates that routing rules can effectively separate and direct telemetry data for different use cases.
133132

134-
Step 5: Checkpoint Management
133+
You can now extend this approach by defining additional routing rules to further categorize spans, metrics, and logs based on different attributes.
135134

136-
The checkpoint extension ensures that data is reliably stored and managed. Check the checkpoint-folder directory for checkpoint files created during operation.
137-
Compaction: Large transactions will be compacted into smaller chunks, which helps in managing large data volumes.
135+
If you want to know more about the `routing` connector , you can find it [here](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/connector/routingconnector)
138136

139-
Conclusion
140-
In this section, you've learned how to configure the OpenTelemetry Collector to route traces based on attributes. You’ve also explored checkpointing, batching, resource detection, and the export of trace data to different destinations. You can extend this setup by adding more routing rules, processing options, and exporters as needed for your use case.
137+
Stop the `agent` and `gateway` using Command-c/Ctrl-c.

content/en/ninja-workshops/10-advanced-otel/80-routing/_index.md

Lines changed: 5 additions & 79 deletions
Original file line numberDiff line numberDiff line change
@@ -70,9 +70,9 @@ This same approach can also be applied to `metrics` and `logs`, allowing you to
7070
The `routing connector` requires different targets for routing. To achieve this, update the default `file/traces:` exporter and name it `file/traces/default` and add a second file exporter called `file/traces/security:`. This will allow the routing connector to direct data to the appropriate target based on the rules you define.
7171

7272
```yaml
73-
file/traces/default: # Exporter Type/Name (For regular traces)
73+
file/traces/standard: # Exporter Type/Name (For regular traces)
7474
# Path where trace data will be saved in OTLP json format
75-
path: "./gateway-traces-default.out"
75+
path: "./gateway-traces-standard.out"
7676
append: false # Overwrite the file each time
7777
file/traces/security: # Exporter Type/Name (For security traces)
7878
# Path where trace data will be saved in OTLP json format
@@ -95,7 +95,7 @@ To enable routing we need to define two pipelines for traces:
9595
- memory_limiter
9696
- batch
9797
- resource/add_mode
98-
exporters: [file/traces/default] # Location for spans not matching rule
98+
exporters: [file/traces/standard] # Location for spans not matching rule
9999
```
100100

101101
- The Target pipeline, that will handle all spans that match the routing rule.
@@ -129,84 +129,10 @@ To enable `routing`, you need to update the original `traces` pipeline by adding
129129
Keep in mind that any existing processors have been removed from this pipeline. They are now handled by either the standard pipeline or the target pipelines, depending on the routing rules.
130130
{{% /notice %}}
131131

132-
- **Create a trace for different Environments**
133-
To test your configuration, you need to generate span data with the correct `ResourceSpan` attributes to trigger the routing rule. Copy the following JSON and save it as security.json in the `[WORKSHOP]\8-routing` directory.
134-
135-
{{% tabs %}}
136-
{{% tab title="Compacted JSON" %}}
137-
138-
```json
139-
{"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"password_check"}},{"key":"deployment.environment","value":{"stringValue":"security_applications"}}]},"scopeSpans":[{"scope":{"name":"my.library","version":"1.0.0","attributes":[{"key":"my.scope.attribute","value":{"stringValue":"some scope attribute"}}]},"spans":[{"traceId":"5B8EFFF798038103D269B633813FC60C","spanId":"EEE19B7EC3C1B174","parentSpanId":"EEE19B7EC3C1B173","name":"I'm a server span","startTimeUnixNano":"1544712660000000000","endTimeUnixNano":"1544712661000000000","kind":2,"attributes":[{"keytest":"my.span.attr","value":{"stringValue":"some value"}}]}]}]}]}
140-
```
141-
142-
{{% /tab %}}
143-
{{% tab title="Formatted JSON" %}}
144-
145-
```json
146-
{
147-
"resourceSpans": [
148-
{
149-
"resource": {
150-
"attributes": [
151-
{
152-
"key": "service.name",
153-
"value": {
154-
"stringValue": "password_check"
155-
}
156-
},
157-
{
158-
"key": "deployment.environment",
159-
"value": {
160-
"stringValue": "security_applications"
161-
}
162-
}
163-
]
164-
},
165-
"scopeSpans": [
166-
{
167-
"scope": {
168-
"name": "my.library",
169-
"version": "1.0.0",
170-
"attributes": [
171-
{
172-
"key": "my.scope.attribute",
173-
"value": {
174-
"stringValue": "some scope attribute"
175-
}
176-
}
177-
]
178-
},
179-
"spans": [
180-
{
181-
"traceId": "5B8EFFF798038103D269B633813FC60C",
182-
"spanId": "EEE19B7EC3C1B174",
183-
"parentSpanId": "EEE19B7EC3C1B173",
184-
"name": "I'm a server span",
185-
"startTimeUnixNano": "1544712660000000000",
186-
"endTimeUnixNano": "1544712661000000000",
187-
"kind": 2,
188-
"attributes": [
189-
{
190-
"keytest": "my.span.attr",
191-
"value": {
192-
"stringValue": "some value"
193-
}
194-
}
195-
]
196-
}
197-
]
198-
}
199-
]
200-
}
201-
]
202-
}
203-
```
204-
205-
{{% /tab %}}
206-
{{% /tabs %}}
207-
208132
{{% /notice %}}
209133

134+
Again, validate the `gateway` configuration using `otelbin.io` for spelling mistakes etc. Your `Traces:` pipeline should like this:
135+
210136
![Routing Processor](../images/routing.png)
211137

212138
Lets' test our configuration!
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
---
2+
title: Wrap-up
3+
linkTitle: 90. wrap-up
4+
time: 1 minutes
5+
weight: 9
6+
---
7+
8+
![Well done](../images/welldone.png)
1.19 MB
Loading

0 commit comments

Comments
 (0)