Skip to content

Commit 0e49ef7

Browse files
committed
add some documentation
1 parent 9f3caed commit 0e49ef7

File tree

1 file changed

+74
-0
lines changed

1 file changed

+74
-0
lines changed

README.md

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -285,6 +285,80 @@ interval="10s"
285285
[[inputs.kernel]]
286286
```
287287

288+
#### TransformSource
289+
Sometimes you'll want to transform the incoming data before aggregating.
290+
As shown in the example collectd source, it is possible to wrap a source with
291+
another source that will transform the data. The TransformingSource specifies
292+
sets of transformations to apply to the input records, allowing you to add dimensions
293+
to all metrics, remove dimensions from all metrics, and modify metrics based on their
294+
name.
295+
296+
ex:
297+
```hocon
298+
{
299+
type="com.arpnetworking.metrics.mad.sources.TransformingSource"
300+
name="transforming_source"
301+
transformations = [
302+
{
303+
inject = {
304+
foo {
305+
value = bar
306+
overwriteExisting = false
307+
}
308+
}
309+
remove = [
310+
baz
311+
]
312+
findAndReplace = {
313+
"this" = ["that"]
314+
"extract/([^/]*)/thing" = ["extract/thing/${other_dimension};my_dimension=${1}"]
315+
}
316+
}
317+
]
318+
source {
319+
// your wrapped source goes here
320+
}
321+
}
322+
```
323+
324+
tranformations is a list of TransformationSet objects. Each TransformationSet has an inject (Map\<String, DimensionInjection\>),
325+
remove (List\<String\>) and a findAndReplace (Map\<String, List\<String\>\>).
326+
327+
A DimensionInjection is just a value and a boolean of whether or not to overwrite existing values.
328+
329+
The keys in findAndReplace are regular expressions used to match metrics. If matched, the list of replacements is
330+
executed, allowing for a single input metric to be recorded multiple times with different names or dimensions.
331+
The format for the values in the list are similar to a standard java regex replacement. Variables are enclosed in ${},
332+
for example:
333+
```bash
334+
${my_variable}
335+
```
336+
Variables will be matched in the following order (first match wins):
337+
1. capture group
338+
2. dimension
339+
3. environment variable
340+
341+
342+
Variable names may also have a prefix that specifies where the value should originate. For example:
343+
```bash
344+
${capture:my_var}
345+
```
346+
```bash
347+
${env:MYVAR}
348+
```
349+
```bash
350+
${dimension:some_dimension}
351+
```
352+
353+
This namespacing prevents the built-in precendence search for a matching variable.
354+
355+
__Note: Any dimensions matched as part of a replacement will be removed from the resulting metric.__
356+
357+
Dimensions can also be injected into a metric. To do this, add a ';' after the replacement name of the metric
358+
and proceed to specify key=value pairs (where both the key and value can use variables). Basically, the
359+
replacement is processed, the string split on ';' with the first part being the metric name and anything
360+
after the ';' being parsed as key=value pairs to be added to the metric's dimensions.
361+
288362
Development
289363
-----------
290364

0 commit comments

Comments
 (0)