Skip to content

Commit 3e3efce

Browse files
buraizuolivielpeau
andauthored
Docs11702/agent metrics filtering (#30892)
* [DOCS-11702] Add custom metrics filtering guide * [DOCS-11702] Add custom metrics filtering guide images * [DOCS-11702] Remove placeholder text * [DOCS-11702] Update preview banner * [DOCS-11702] Incorporate feedback Co-authored-by: Olivier Vielpeau <[email protected]> * [DOCS-11702] Update copy, visual resources * [DOCS-11702] Add visual resources * [DOCS-11702] Unify tab name * [DOCS-11702] Fix list numbering * [DOCS-11702] Incorporate feedback --------- Co-authored-by: Olivier Vielpeau <[email protected]>
1 parent ecf8cb2 commit 3e3efce

10 files changed

+189
-0
lines changed
Lines changed: 189 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,189 @@
1+
2+
---
3+
title: Agent-Side Filtering for DogStatsD Custom Metrics
4+
private: true
5+
further_reading:
6+
- link: "/metrics/custom_metrics/"
7+
tag: "Documentation"
8+
text: "Learn more about Custom Metrics"
9+
- link: "/account_management/billing/custom_metrics/?tab=countrate"
10+
tag: "Documentation"
11+
text: "Custom Metrics Billing"
12+
- link: "/metrics/metrics-without-limits/"
13+
tag: "Documentation"
14+
text: "Metrics without Limits™"
15+
- link: "/metrics/volume/"
16+
tag: "Documentation"
17+
text: "Metrics Volume Management"
18+
- link: "https://www.datadoghq.com/blog/custom-metrics-governance/"
19+
tag: "Blog"
20+
text: "Best practices for end-to-end custom metrics governance"
21+
---
22+
23+
{{< callout url="https://www.datadoghq.com/" btn_hidden="true" header="false" >}}
24+
Agent-side filtering for DogStatsD custom metrics is in Preview.
25+
{{< /callout >}}
26+
27+
## Overview
28+
29+
Agent-side filtering enables you to filter out unused or unwanted DogStatsD custom metrics directly at the Datadog Agent, before sending them to Datadog. This can significantly reduce both indexed and ingested custom metric volume.
30+
31+
Filtering is performed at the Agent level but centrally managed through the Datadog UI, giving teams full visibility and control. You can create, update, and manage filtering policies in Datadog, streamlining metric governance while maintaining transparency.
32+
33+
## Create a metric filtering policy
34+
35+
You can create a metric filtering policy from the [Metrics Settings page][1] or the [Metrics Summary page][2].
36+
37+
### From the Metrics Settings page
38+
39+
1. Click **+ Create Policy**.
40+
2. Click **Filter metrics**.
41+
3. Provide a description for the new policy.
42+
4. Select the metrics to filter from the **Metrics to Filter** dropdown, or click **Upload CSV**.
43+
- If you choose to upload a CSV, select the file and click **Open**. You can use multiple CSVs to create the policy.
44+
5. When you're satisfied with the list of metrics to filter, click **Save and Filter**.
45+
46+
### From the Metrics Summary page
47+
48+
Create a metric filtering policy from the Metrics Summary page using any of the following methods:
49+
50+
{{< tabs >}}
51+
{{% tab "From a metric query" %}}
52+
53+
1. Enter a metric query in the search bar.
54+
2. Click the three vertical dots button on the right-hand side of the screen.
55+
3. Click **Filter metrics**.
56+
4. In the **Choose policy** dropdown, click **New Policy**. Provide a description for the policy.
57+
5. Review the **Metrics to Filter**. Click `X` on the right side of any row to remove a metric from the list, or click `+ Include More Metrics` to add metrics to the list.
58+
6. Click **Save and Filter**.
59+
60+
{{< img src="metrics/guide/agent_filtering_for_dogstatsd_custom_metrics/create_policy_from_metric_query.mp4" alt="Creating a metric filtering policy from a metric query" video="true" >}}
61+
62+
{{% /tab %}}
63+
{{% tab "From the policy editor" %}}
64+
65+
1. Click the three vertical dots button on the right-hand side of the screen.
66+
2. Click **Filter metrics**.
67+
3. In the **Choose policy** dropdown, click **New Policy**. Provide a description for the policy.
68+
4. Enter a metric query in the **Metrics to Filter** field, or select metrics individually from the dropdown. Click `X` on the right side of any row to remove a metric from the list.
69+
5. Click **Save and Filter**.
70+
71+
{{< img src="metrics/guide/agent_filtering_for_dogstatsd_custom_metrics/create_policy_with_policy_editor.mp4" alt="Creating a metric filtering policy from the policy editor" video="true" >}}
72+
73+
{{% /tab %}}
74+
{{% tab "From CSV upload" %}}
75+
76+
1. Click the three vertical dots button on the right-hand side of the screen.
77+
2. Click **Filter metrics**.
78+
3. In the **Choose policy** dropdown, click **New Policy**. Provide a description for the policy.
79+
4. Click **Upload CSV** to the right of the **Metrics to Filter** field.
80+
5. Select the CSV file, and click **Open**.
81+
6. Review the metrics listed. Click `X` on the right side of any row to remove a metric from the list. If needed, upload additional CSV files, or add metrics through the **Metrics to Filter** field.
82+
7. Click **Save and Filter**.
83+
84+
{{< img src="metrics/guide/agent_filtering_for_dogstatsd_custom_metrics/create_policy_with_csv_upload.mp4" alt="Creating a metric filtering policy with a CSV file upload" video="true" >}}
85+
86+
{{% /tab %}}
87+
{{< /tabs >}}
88+
89+
## Edit a metric filtering policy
90+
91+
You can edit a metric filtering policy from the [Metrics Settings page][1] or the [Metrics Summary page][2].
92+
93+
### From the Metrics Settings page
94+
95+
1. Click the policy to edit.
96+
2. Click **Edit**.
97+
3. Select the metrics to filter from the **Metrics to Filter** dropdown, or click **Upload CSV**.
98+
- If you choose to upload a CSV, select the file and click **Open**.
99+
4. Click **Save and Filter**.
100+
101+
{{< img src="metrics/guide/agent_filtering_for_dogstatsd_custom_metrics/edit_policy_from_metrics_settings.mp4" alt="Editing a metric filtering policy from the Metrics Settings page" video="true" >}}
102+
103+
### From the Metrics Summary page
104+
105+
Edit a metric filtering policy from the Metrics Summary page using any of the following methods:
106+
107+
{{< tabs >}}
108+
{{% tab "From a metric query" %}}
109+
110+
1. Enter a metric query in the search bar.
111+
2. Click the three vertical dots button on the right-hand side of the screen.
112+
3. Click **Filter metrics**.
113+
4. In the **Choose policy** dropdown, select the policy to edit.
114+
5. Review the **Metrics to Filter** and **Existing metrics in policy** lists. Click `X` on the right side of any row to remove a metric from the list, or click `+ Include More Metrics` to add metrics to the list.
115+
6. Click **Save and Filter**.
116+
117+
{{< img src="metrics/guide/agent_filtering_for_dogstatsd_custom_metrics/edit_policy_with_metric_query.mp4" alt="Editing a metric filtering policy with a metric query" video="true" >}}
118+
119+
{{% /tab %}}
120+
{{% tab "From the policy editor" %}}
121+
122+
1. Click the three vertical dots button on the right-hand side of the screen.
123+
2. Click **Filter metrics**.
124+
3. In the **Choose policy** dropdown, select the policy to edit.
125+
4. Select metrics individually from the **Metrics to Filter** dropdown. Click `X` on the right side of any row to remove a metric from the list.
126+
5. Click **Save and Filter**.
127+
128+
{{% /tab %}}
129+
{{% tab "From CSV upload" %}}
130+
131+
1. Click the three vertical dots button on the right-hand side of the screen.
132+
2. Click **Filter metrics**.
133+
3. In the **Choose policy** dropdown, select the policy to edit.
134+
4. Click **Upload CSV** to the right of the **Metrics to Filter** field.
135+
5. Select the CSV file, and click **Open**.
136+
6. Review the **Metrics to Filter** and **Existing metrics in policy** lists. Click `X` on the right side of any row to remove a metric from the list, or click `+ Include More Metrics` to add metrics to the list.
137+
7. Click **Save and Filter**.
138+
139+
{{% /tab %}}
140+
{{< /tabs >}}
141+
142+
## View all policies and filtered metrics
143+
144+
You can view all your policies and filtered metrics from the [Metrics Settings page][1].
145+
146+
**Click on the [settings button][1]**:
147+
148+
{{< img src="metrics/guide/agent_filtering_for_dogstatsd_custom_metrics/settings_from_summary.png" alt="The settings button on the metric summary page" style="width:100%;" >}}
149+
150+
**Click on Metrics in the navigation bar** and go straight to settings:
151+
152+
{{< img src="metrics/guide/agent_filtering_for_dogstatsd_custom_metrics/settings_from_nav.png" alt="The settings option from the expanded Metrics panel in Datadog" style="width:100%;" >}}
153+
154+
### View all policies
155+
156+
Select the **Policies** tab from the sidebar to see a list of all your policies. If you don't see the sidebar, click the **Show Sidebar** button {{< img src="metrics/guide/agent_filtering_for_dogstatsd_custom_metrics/show_sidebar.png" alt="The Show Sidebar button" inline="true" >}}.
157+
158+
Click on any metric filtering policy to open its detail view for edits or deletion.
159+
160+
### View all filtered metrics
161+
162+
Select the **Filtered Metrics** tab from the sidebar to see a list of all your filtered. If you don't see the sidebar, click the **Show Sidebar** button {{< img src="metrics/guide/agent_filtering_for_dogstatsd_custom_metrics/show_sidebar.png" alt="The Show Sidebar button" inline="true" >}}.
163+
164+
Click on a filtered metric's attached policies in the **ATTACHED POLICIES** column for edits or deletion.
165+
166+
## Delete policies
167+
168+
You can delete metric filtering policies from the [Metrics Settings page][1].
169+
170+
1. Click on the metric filtering policy to delete.
171+
2. Select **Delete** in the top-right corner of the page.
172+
173+
{{< img src="metrics/guide/agent_filtering_for_dogstatsd_custom_metrics/delete_policy.png" alt="The delete policy button on a metric filtering policy detail view" style="width:100%;" >}}
174+
175+
## Preview limitations
176+
177+
This initial preview release includes the following limitations:
178+
179+
- A maximum of 10,000 metric names can be filtered out.
180+
- Resource usage impact on the Agent is limited to up to 10MB [RSS][3]. There is no increase in CPU usage.
181+
- Only DogStatsD metrics are supported.
182+
183+
## Further reading
184+
185+
{{< partial name="whats-next/whats-next.html" >}}
186+
187+
[1]: https://app.datadoghq.com/metric/settings/policies
188+
[2]: https://app.datadoghq.com/metric/summary
189+
[3]: https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/6/html/performance_tuning_guide/network-rss
299 KB
Loading
298 KB
Loading
190 KB
Loading
913 Bytes
Loading

0 commit comments

Comments
 (0)