You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Getting the statsD metrics data from airflow collected by our opentelemetry collector, I want to process the data by key-value pair attributes. But all metadata are in the name, such as "ti.finish.<dag_id>.<task_id>.", we want to add "dag_id" and "task_id" as the key-value pair attributes to datapoint.
I have tried to use the metrics transform processor, transform processor and attributes processor.
The metrics transform processor's combine function can process the counter and gauge data, but can not process summary and histogram data. It will encounter some problems when processing histogram data it will report errors in the terminal, and for summary data, the datapoint part of the result data is null.
For the transform processor, I could use functions like replace_pattern to make the metrics name concise, such as "ti.finish.<dag_id>.<task_id>." to "ti.finish", but I have no idea how to add the <dag_id> and <task_id> into the attributes, maybe the Splitfunction to split name by "." and set function, but it does not work.
Do you have any solution or suggestion to my problems? Thank you very much.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
Getting the statsD metrics data from airflow collected by our opentelemetry collector, I want to process the data by key-value pair attributes. But all metadata are in the name, such as "ti.finish.<dag_id>.<task_id>.", we want to add "dag_id" and "task_id" as the key-value pair attributes to datapoint.
I have tried to use the metrics transform processor, transform processor and attributes processor.
The metrics transform processor's combine function can process the counter and gauge data, but can not process summary and histogram data. It will encounter some problems when processing histogram data it will report errors in the terminal, and for summary data, the datapoint part of the result data is null.
For the transform processor, I could use functions like
replace_pattern
to make the metrics name concise, such as "ti.finish.<dag_id>.<task_id>." to "ti.finish", but I have no idea how to add the <dag_id> and <task_id> into the attributes, maybe theSplit
function to split name by "." and set function, but it does not work.Do you have any solution or suggestion to my problems? Thank you very much.
Beta Was this translation helpful? Give feedback.
All reactions