You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
First, ensure you have the required directory structure for the checkpoint folder and log files:
9
+
In this section, we will test the `routing` rule configured for the `gateway`. The expected result is that the`span` from the `security.json` file will be sent to the `gateway-traces-security.out` file.
11
10
12
-
```bash
13
-
your_project/
14
-
├── checkpoint-folder/ # Directory for checkpoint files
15
-
├── agent-standard.out # Output file for standard traces
16
-
├── agent-security.out # Output file for security traces
Find your `Gateway` terminal window, and navigate to the `[WORKSHOP]/8-routing` folder and restart the gateway with the routing configurations specified in the gateway.yaml.
15
+
It should start up normally and state : `Everything is ready. Begin running and processing data.`
24
16
25
-
Debug Exporter: Outputs trace data in detailed verbosity for debugging.
26
-
File Exporters: Writes trace data to files with rotation policies.
27
-
OTLP Gateway Exporter: Sends trace data to a remote OTLP endpoint with queuing and retry options.
17
+
-**Run the Agent**:
18
+
Find your `Agent` terminal window and navigate to the `[WORKSHOP]/8-routing` folder and restart the agent with the regular `agent.yaml`.
19
+
It should also start up normally and state : `Everything is ready. Begin running and processing data.`
28
20
29
-
2.4 Connectors: Routing
21
+
-**Create a trace for different Environments**
22
+
Find your `Test` terminal window and navigate to the `[WORKSHOP]/8-routing`.
23
+
To test your configuration, you need to generate span data with the correct `ResourceSpan` attributes to trigger the routing rule. Copy the following JSON and save it as `security.json` in the `[WORKSHOP]\8-routing` directory.
30
24
31
-
The routing connector routes traces based on attributes. Here, traces with the attribute `deployment.environment == "security_applications"` are routed to a separate pipeline, `traces/security`:
25
+
{{% tabs %}}
26
+
{{% tab title="Compacted JSON" %}}
32
27
33
-
```yaml
34
-
connectors:
35
-
routing:
36
-
default_pipelines: [traces/standard] # Default pipeline for traces
37
-
error_mode: ignore # Ignore errors during routing
38
-
table:
39
-
- statement: route() where attributes["deployment.environment"] == "security_applications"
40
-
pipelines: [traces/security]
28
+
```json
29
+
{"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"password_check"}},{"key":"deployment.environment","value":{"stringValue":"security_applications"}}]},"scopeSpans":[{"scope":{"name":"my.library","version":"1.0.0","attributes":[{"key":"my.scope.attribute","value":{"stringValue":"some scope attribute"}}]},"spans":[{"traceId":"5B8EFFF798038103D269B633813FC60C","spanId":"EEE19B7EC3C1B174","parentSpanId":"EEE19B7EC3C1B173","name":"I'm a server span","startTimeUnixNano":"1544712660000000000","endTimeUnixNano":"1544712661000000000","kind":2,"attributes":[{"keytest":"my.span.attr","value":{"stringValue":"some value"}}]}]}]}]}
41
30
```
42
31
43
-
You can add additional routing rules by specifying different attribute conditions. The second routing rule is commented out but could be used to delete a specific key from the attributes before routing the data.
44
-
45
-
2.5 Processors
46
-
Processors allow you to manipulate and enrich data before exporting it:
47
-
48
-
Batch Processor: Batches data to improve performance.
49
-
Memory Limiter: Ensures that the collector doesn’t consume too much memory.
50
-
Resource Detection: Adds system resource attributes to the traces.
Make sure the are **NO**`*.out` files in the `[WORKSHOP]/8-routing` folder.
120
104
121
-
```bash
122
-
otelcol --config=otel-collector-config.yaml
123
-
```
105
+
-**Send a Regular Span**:
106
+
Find your `Test` terminal window and navigate to the `[WORKSHOP]/8-routing` folder. From there, send a regular span using the `trace.json` file to confirm that communication is functioning as expected.
107
+
Both the `agent` and `gateway` should display debug information, including the span you just sent. Additionally, the gateway should generate a new `./gateway-traces-default.out` file, as this is now the destination for regular spans.
If you verify the `./gateway-traces-default.out` is should contain the span we send with the cURL command.
111
+
{{% /notice %}}
112
+
113
+
-**Send a Security Span**:
114
+
Make sure both the `agent` and `gateway` are running, then send a security span using the `security.json` file to test the routing rule in the gateway.
115
+
Both the `agent` and `gateway` should display debug information, including the span you just sent. Additionally, the gateway should generate a new `./gateway-traces-security.out` file, as this is the destination for spans where the `deployment.environment` resourceSpan attribute matches `"security_applications"`.
If you verify the `./gateway-traces-security.out` is should only contain the spans from the `"security_applications"` deployment.environment.
119
+
{{% /notice %}}
120
+
121
+
{{% /notice %}}
122
+
123
+
### Conclusion
124
+
125
+
In this section, we successfully tested the routing connector in the gateway by sending different spans and verifying their destinations.
124
126
125
-
Verify that the Collector is running: Check the logs for any errors. If there are no errors, traces should be flowing through the collector.
127
+
-**Regular spans** were correctly routed to `gateway-traces-default.out`, confirming that spans without a matching `deployment.environment` attribute follow the default pipeline.
126
128
127
-
Step 4: Testing the Routing
128
-
To test the routing:
129
+
-**Security-related spans** from `security.json` were routed to `gateway-traces-security.out`, demonstrating that the routing rule based on `"deployment.environment": "security_applications"` works as expected.
129
130
130
-
Generate or simulate trace data with the attribute `deployment.environment = security_applications`.
131
-
Verify that the traces are routed to the traces/security pipeline by checking the `agent-security.out` file.
132
-
You can also check `agent-standard.out` to ensure that other traces are routed to the standard pipeline.
131
+
By inspecting the output files, we confirmed that the OpenTelemetry Collector *correctly evaluates span attributes and routes them to the appropriate destinations*. This validates that routing rules can effectively separate and direct telemetry data for different use cases.
133
132
134
-
Step 5: Checkpoint Management
133
+
You can now extend this approach by defining additional routing rules to further categorize spans, metrics, and logs based on different attributes.
135
134
136
-
The checkpoint extension ensures that data is reliably stored and managed. Check the checkpoint-folder directory for checkpoint files created during operation.
137
-
Compaction: Large transactions will be compacted into smaller chunks, which helps in managing large data volumes.
135
+
If you want to know more about the `routing` connector , you can find it [here](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/connector/routingconnector)
138
136
139
-
Conclusion
140
-
In this section, you've learned how to configure the OpenTelemetry Collector to route traces based on attributes. You’ve also explored checkpointing, batching, resource detection, and the export of trace data to different destinations. You can extend this setup by adding more routing rules, processing options, and exporters as needed for your use case.
137
+
Stop the `agent` and `gateway` using Command-c/Ctrl-c.
Copy file name to clipboardExpand all lines: content/en/ninja-workshops/10-advanced-otel/80-routing/_index.md
+5-79Lines changed: 5 additions & 79 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -70,9 +70,9 @@ This same approach can also be applied to `metrics` and `logs`, allowing you to
70
70
The `routing connector` requires different targets for routing. To achieve this, update the default `file/traces:` exporter and name it `file/traces/default` and add a second file exporter called `file/traces/security:`. This will allow the routing connector to direct data to the appropriate target based on the rules you define.
# Path where trace data will be saved in OTLP json format
@@ -95,7 +95,7 @@ To enable routing we need to define two pipelines for traces:
95
95
- memory_limiter
96
96
- batch
97
97
- resource/add_mode
98
-
exporters: [file/traces/default] # Location for spans not matching rule
98
+
exporters: [file/traces/standard] # Location for spans not matching rule
99
99
```
100
100
101
101
- The Target pipeline, that will handle all spans that match the routing rule.
@@ -129,84 +129,10 @@ To enable `routing`, you need to update the original `traces` pipeline by adding
129
129
Keep in mind that any existing processors have been removed from this pipeline. They are now handled by either the standard pipeline or the target pipelines, depending on the routing rules.
130
130
{{% /notice %}}
131
131
132
-
- **Create a trace for different Environments**
133
-
To test your configuration, you need to generate span data with the correct `ResourceSpan` attributes to trigger the routing rule. Copy the following JSON and save it as security.json in the `[WORKSHOP]\8-routing` directory.
134
-
135
-
{{% tabs %}}
136
-
{{% tab title="Compacted JSON" %}}
137
-
138
-
```json
139
-
{"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"password_check"}},{"key":"deployment.environment","value":{"stringValue":"security_applications"}}]},"scopeSpans":[{"scope":{"name":"my.library","version":"1.0.0","attributes":[{"key":"my.scope.attribute","value":{"stringValue":"some scope attribute"}}]},"spans":[{"traceId":"5B8EFFF798038103D269B633813FC60C","spanId":"EEE19B7EC3C1B174","parentSpanId":"EEE19B7EC3C1B173","name":"I'm a server span","startTimeUnixNano":"1544712660000000000","endTimeUnixNano":"1544712661000000000","kind":2,"attributes":[{"keytest":"my.span.attr","value":{"stringValue":"some value"}}]}]}]}]}
140
-
```
141
-
142
-
{{% /tab %}}
143
-
{{% tab title="Formatted JSON" %}}
144
-
145
-
```json
146
-
{
147
-
"resourceSpans": [
148
-
{
149
-
"resource": {
150
-
"attributes": [
151
-
{
152
-
"key": "service.name",
153
-
"value": {
154
-
"stringValue": "password_check"
155
-
}
156
-
},
157
-
{
158
-
"key": "deployment.environment",
159
-
"value": {
160
-
"stringValue": "security_applications"
161
-
}
162
-
}
163
-
]
164
-
},
165
-
"scopeSpans": [
166
-
{
167
-
"scope": {
168
-
"name": "my.library",
169
-
"version": "1.0.0",
170
-
"attributes": [
171
-
{
172
-
"key": "my.scope.attribute",
173
-
"value": {
174
-
"stringValue": "some scope attribute"
175
-
}
176
-
}
177
-
]
178
-
},
179
-
"spans": [
180
-
{
181
-
"traceId": "5B8EFFF798038103D269B633813FC60C",
182
-
"spanId": "EEE19B7EC3C1B174",
183
-
"parentSpanId": "EEE19B7EC3C1B173",
184
-
"name": "I'm a server span",
185
-
"startTimeUnixNano": "1544712660000000000",
186
-
"endTimeUnixNano": "1544712661000000000",
187
-
"kind": 2,
188
-
"attributes": [
189
-
{
190
-
"keytest": "my.span.attr",
191
-
"value": {
192
-
"stringValue": "some value"
193
-
}
194
-
}
195
-
]
196
-
}
197
-
]
198
-
}
199
-
]
200
-
}
201
-
]
202
-
}
203
-
```
204
-
205
-
{{% /tab %}}
206
-
{{% /tabs %}}
207
-
208
132
{{% /notice %}}
209
133
134
+
Again, validate the `gateway` configuration using `otelbin.io` for spelling mistakes etc. Your `Traces:` pipeline should like this:
0 commit comments