Unable to Filter Metrics Using Resource Attributes - resource_attributes #8106
-
|
Hi Everyone, This works fine and I see filtered metrics against my apps in the Container Insights. The problem is as soon as I add AWS ADOT Collector stops publishing any metrics to Container Insights. Can anyone please help with this? I am using this guide and sample config file: Sample config file: Thanks |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 7 replies
-
Beta Was this translation helpful? Give feedback.
-
|
@mskhan-devops I was not able to reproduce the problem with neither the current upstream of receivers:
prometheus:
config:
scrape_configs:
- job_name: 'otel-collector'
scrape_interval: 5s
static_configs:
- targets: ['0.0.0.0:8888']
processors:
filter/1:
metrics:
include:
match_type: regexp
metric_names:
- otelcol_process.*
resource_attributes:
- Key: job
Value: otel.*
exporters:
logging/debug:
loglevel: debug
service:
pipelines:
metrics:
receivers: [prometheus]
processors: [filter/1]
exporters: [logging/debug]This self-scrapes the OpenTelemetry Collector and puts output on the console. The filter seems to be working as intended. When |
Beta Was this translation helpful? Give feedback.
-
|
Hi @pmm-sumo , Sure, let me add debug level logging to the exporters in my configuration. That will really help to identify the Resources available in the environment. I will also share the actual configuration file that I am using for AWS ADOT collector here for review. Thank you once again. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @pmm-sumo, I added a debug level to the exporter and went through the logs looking for string attributes but did not see any string like this though I see Resource labels. Following are starting of Can you please help with what parameters/values can I use to export out metrics to Container Insights for a single POD in the EKS cluster using the |
Beta Was this translation helpful? Give feedback.
-
Link to original complete manifest file: I am currently using the following configuration. As you can see, I had to comment out the lines under resource_attributes because it doesn't work when I uncomment: apiVersion: v1 |
Beta Was this translation helpful? Give feedback.
-
@mskhan-devops Yes, this is anticipated. Resource typically contains attributes that are relevant for a given source of data. Record-level attributes (if present) are specific only to a given record Looking at the example you provided, it seems that In some cases it might be easier to actually use exclude filter and remove unwanted things rather than explicitly specify what is allowed, like: |
Beta Was this translation helpful? Give feedback.
-
|
Hi @pmm-sumo, Now that it has been tested and validated, it is working properly. Please close this discussion. Thank you for your help. Kind Regards, |
Beta Was this translation helpful? Give feedback.
@mskhan-devops I was not able to reproduce the problem with neither the current upstream of
opentelemetry-collector-contribnor v0.16 ofadot. I might have missed something, but my understanding of the problem description is that "when both metric_names and resource_attributes are specified, no metrics are emitted". Is that correct? I tried following config: