-
Notifications
You must be signed in to change notification settings - Fork 540
in_mem_metrics: initial documentation. #1170
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
pwhelan
wants to merge
9
commits into
master
Choose a base branch
from
pwhelan-doc-in_mem_metrics
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
4b7e3dd
in_mem_metrics: initial documentation.
pwhelan 234a29c
Update pipeline/inputs/mem-metrics.md
pwhelan 816f19e
Update pipeline/inputs/mem-metrics.md
pwhelan c9b624b
Update pipeline/inputs/mem-metrics.md
pwhelan 978770a
Update pipeline/inputs/mem-metrics.md
pwhelan 656fbd3
Update pipeline/inputs/mem-metrics.md
pwhelan 8f2ee91
Update pipeline/inputs/mem-metrics.md
pwhelan d4c50a5
Update pipeline/inputs/mem-metrics.md
pwhelan 1384ad2
Update pipeline/inputs/mem-metrics.md
pwhelan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,110 @@ | ||
| # Memory Metrics (mem_metrics) | ||
|
|
||
| The **mem_metrics**, or memory metrics plugin, generates metrics for memory statistics for processes using the `smaps_rollup` file for each process in /proc. At the moment this plugin works exclusively on Linux. | ||
pwhelan marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| # Configuration Parameters | ||
|
|
||
| | **Key*** | Description | Default | | ||
| | :------------ | :------------------------------------------------- | :----------- | | ||
| | proc_path | The path of the proc pseudo filesystem | /proc | | ||
| | filter_exec | Filter for a single executable by path | **inactive** | | ||
| | filter_cmd | Filter by command line | **inactive** | | ||
| | filter_pid | Filter by comma delimited list of PIDs | **inactive** | | ||
| | interval_sec | Set the interval seconds between events generation | 5 | | ||
| | interval_nsec | Set the nanoseconds interval (sub seconds) | 0 | | ||
pwhelan marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| The `filter_pid` can include or be set to either `0` or `self` to refer to the fluent-bit process itself. | ||
|
|
||
| ## Getting Started | ||
pwhelan marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| You can run the plugin from the command line or through a configuration file. By default metrics will be generated for all processes the current user can analyze. | ||
|
|
||
| ### Command Line | ||
pwhelan marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ```bash | ||
| $ fluent-bit -i mem_metrics -o stdout | ||
| Fluent Bit v2.1.8 | ||
| * Copyright (C) 2015-2022 The Fluent Bit Authors | ||
| * Fluent Bit is a CNCF sub-project under the umbrella of Fluentd | ||
| * https://fluentbit.io | ||
|
|
||
| 2023-07-26T14:37:49.919778443Z node_smaps_rollup_rss{pid="4540"} = 21064 | ||
| 2023-07-26T14:37:49.919778443Z node_smaps_rollup_pss{pid="4540",type="clean"} = 9598 | ||
| 2023-07-26T14:37:49.919778443Z node_smaps_rollup_pss{pid="4540",type="dirty"} = 5998 | ||
| 2023-07-26T14:37:49.919778443Z node_smaps_rollup_pss{pid="4540",type="anon"} = 5996 | ||
| 2023-07-26T14:37:49.919778443Z node_smaps_rollup_pss{pid="4540",type="file"} = 3602 | ||
| 2023-07-26T14:37:49.919778443Z node_smaps_rollup_pss{pid="4540",type="shmem"} = 0 | ||
| 2023-07-26T14:37:49.919778443Z node_smaps_rollup_shared{pid="4540",type="clean"} = 15008 | ||
| 2023-07-26T14:37:49.919778443Z node_smaps_rollup_shared{pid="4540",type="dirty"} = 4 | ||
| 2023-07-26T14:37:49.919778443Z node_smaps_rollup_private{pid="4540",type="clean"} = 56 | ||
| 2023-07-26T14:37:49.919778443Z node_smaps_rollup_private{pid="4540",type="dirty"} = 5996 | ||
| 2023-07-26T14:37:49.919778443Z node_smaps_rollup_referenced{pid="4540"} = 21064 | ||
| 2023-07-26T14:37:49.919778443Z node_smaps_rollup_anonymous{pid="4540"} = 5996 | ||
| 2023-07-26T14:37:49.919778443Z node_smaps_rollup_lazy_free{pid="4540"} = 0 | ||
| 2023-07-26T14:37:49.919778443Z node_smaps_rollup_anon_huge_pages{pid="4540"} = 0 | ||
| 2023-07-26T14:37:49.919778443Z node_smaps_rollup_shmem_pmd_mapped{pid="4540"} = 0 | ||
| 2023-07-26T14:37:49.919778443Z node_smaps_rollup_file_pmd_mapped{pid="4540"} = 0 | ||
| 2023-07-26T14:37:49.919778443Z node_smaps_rollup_shared_hugetlb{pid="4540"} = 0 | ||
| 2023-07-26T14:37:49.919778443Z node_smaps_rollup_private_hugetlb{pid="4540"} = 0 | ||
| 2023-07-26T14:37:49.919778443Z node_smaps_rollup_swap{pid="4540"} = 0 | ||
| 2023-07-26T14:37:49.919778443Z node_smaps_rollup_swap_pss{pid="4540"} = 0 | ||
| 2023-07-26T14:37:49.919778443Z node_smaps_rollup_locked{pid="4540"} = 0 | ||
| ``` | ||
|
|
||
| ### Configuration File | ||
pwhelan marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| Minimal pipeline for logging memory metrics to the console: | ||
|
|
||
| ```ini | ||
| [INPUT] | ||
| Name mem_metrics | ||
| Tag mem | ||
|
|
||
| [OUTPUT] | ||
| Name stdout | ||
| Match mem | ||
| ``` | ||
|
|
||
| Log all instances of fluent-bit to prometheus: | ||
pwhelan marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ```ini | ||
| [INPUT] | ||
| Name mem_metrics | ||
| Filter_cmd fluent-bit | ||
| Tag mem | ||
|
|
||
| [OUTPUT] | ||
| Name prometheus_remote_write | ||
| Match mem | ||
| Host localhost | ||
| Port 9090 | ||
| Uri /prometheus/v1/write?prometheus_server=mem | ||
| ``` | ||
|
|
||
| ### Exposed Metrics | ||
|
|
||
| All metrics are logged as gauges since they can both increase and decrease. Currenrtly supported gauges are: | ||
| * node_smaps_rollup_rss | ||
| * node_smaps_rollup_pss | ||
| * type=clean | ||
| * type=dirty | ||
| * type=anon | ||
| * type=file | ||
| * type=shmem | ||
| * node_smaps_rollup_shared | ||
| * type=clean | ||
| * type=dirty | ||
| * node_smaps_rollup_private | ||
| * type=dirty | ||
| * type=clean | ||
| * node_smaps_rollup_referenced | ||
| * node_smaps_rollup_anonymous | ||
| * node_smaps_rollup_lazy_free | ||
| * node_smaps_rollup_anon_huge_pages | ||
| * node_smaps_rollup_shmem_pmd_mapped | ||
| * node_smaps_rollup_file_pmd_mapped | ||
| * node_smaps_rollup_shared_hugetlb | ||
| * node_smaps_rollup_private_hugetlb | ||
| * node_smaps_rollup_swap | ||
| * node_smaps_rollup_swap_pss | ||
| * node_smaps_rollup_locked | ||
pwhelan marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.