Skip to content

Commit b797803

Browse files
johnleslieatoulmebraydonk
authored and
ChrsMark
committed
[receiver/hostmetrics] add nfsscraper metrics, Linux scraper, and tests (open-telemetry#40134)
#### Description Add metrics, Linux scraper, and tests to hostmetricsreceiver's nfsscraper Metric names were approved in open-telemetry/semantic-conventions#2287 #### Link to tracking issue Fixes open-telemetry#39978 #### Testing All make targets in /CONTRIBUTING.md : hostmetricsreceiver has system tests that read from /proc and validate the presence of metrics, and nfsscraper has unit tests. In addition, I used the nfs_scraper_linux_test.go fixtures for end-to-end tests: https://gist.github.com/johnleslie/de6e53abe10bdef29186552b07bbcd99 #### Documentation Generated per-metric documentation: receiver/hostmetricsreceiver/internal/scraper/nfsscraper/documentation.md <!--Please delete paragraphs that you did not use before submitting.--> --------- Co-authored-by: Antoine Toulme <[email protected]> Co-authored-by: Braydon Kains <[email protected]>
1 parent 2315e02 commit b797803

24 files changed

+3561
-77
lines changed

‎.chloggen/hostmetricsreceiver-add-nfsscraper.yaml‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Use this changelog template to create an entry for release notes.
2+
3+
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
4+
change_type: enhancement
5+
6+
# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
7+
component: receiver/hostmetrics
8+
9+
# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
10+
note: Add metrics, Linux scraper, and tests to hostmetricsreceiver's nfsscraper
11+
12+
# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
13+
issues: [40134]
14+
15+
# (Optional) One or more lines of additional information to render under the primary note.
16+
# These lines will be padded with 2 spaces and then inserted directly into the document.
17+
# Use pipe (|) for multiline entries.
18+
subtext:
19+
20+
# If your change doesn't affect end users or the exported elements of any package,
21+
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
22+
# Optional: The change log or logs in which this entry should be included.
23+
# e.g. '[user]' or '[user, api]'
24+
# Include 'user' if the change is relevant to end users.
25+
# Include 'api' if there is a change to a library API.
26+
# Default: '[user]'
27+
change_logs: [user, api]

‎receiver/hostmetricsreceiver/README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@ The available scrapers are:
5353
| [filesystem] | All | File System utilization metrics |
5454
| [memory] | All | Memory utilization metrics |
5555
| [network] | All | Network interface I/O metrics & TCP connection metrics |
56+
| [nfs] | Linux | NFS server and client metrics |
5657
| [paging] | All | Paging/Swap space utilization and I/O metrics |
5758
| [processes] | Linux, Mac, FreeBSD, OpenBSD | Process count metrics |
5859
| [process] | Linux, Windows, Mac, FreeBSD | Per process CPU, Memory, and Disk I/O metrics |
@@ -64,6 +65,7 @@ The available scrapers are:
6465
[load]: ./internal/scraper/loadscraper/documentation.md
6566
[memory]: ./internal/scraper/memoryscraper/documentation.md
6667
[network]: ./internal/scraper/networkscraper/documentation.md
68+
[nfs]: ./internal/scraper/nfsscraper/documentation.md
6769
[paging]: ./internal/scraper/pagingscraper/documentation.md
6870
[processes]: ./internal/scraper/processesscraper/documentation.md
6971
[process]: ./internal/scraper/processscraper/documentation.md

‎receiver/hostmetricsreceiver/config_test.go‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ import (
2222
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/loadscraper"
2323
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/memoryscraper"
2424
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/networkscraper"
25+
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/nfsscraper"
2526
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/pagingscraper"
2627
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/processesscraper"
2728
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/processscraper"
@@ -77,6 +78,7 @@ func TestLoadConfig(t *testing.T) {
7778
}
7879
return cfg
7980
})(),
81+
component.MustNewType("nfs"): nfsscraper.NewFactory().CreateDefaultConfig(),
8082
component.MustNewType("processes"): processesscraper.NewFactory().CreateDefaultConfig(),
8183
component.MustNewType("paging"): pagingscraper.NewFactory().CreateDefaultConfig(),
8284
component.MustNewType("process"): (func() component.Config {

‎receiver/hostmetricsreceiver/example_config.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ receivers:
1212
disk:
1313
filesystem:
1414
network:
15+
nfs:
1516
paging:
1617
processes:
1718
uptime:

‎receiver/hostmetricsreceiver/factory.go‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ import (
2525
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/loadscraper"
2626
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/memoryscraper"
2727
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/networkscraper"
28+
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/nfsscraper"
2829
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/pagingscraper"
2930
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/processesscraper"
3031
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/processscraper"
@@ -44,6 +45,7 @@ var (
4445
loadscraper.NewFactory(),
4546
memoryscraper.NewFactory(),
4647
networkscraper.NewFactory(),
48+
nfsscraper.NewFactory(),
4749
pagingscraper.NewFactory(),
4850
processesscraper.NewFactory(),
4951
processscraper.NewFactory(),

‎receiver/hostmetricsreceiver/hostmetrics_receiver_test.go‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ import (
3030
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/loadscraper"
3131
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/memoryscraper"
3232
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/networkscraper"
33+
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/nfsscraper"
3334
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/pagingscraper"
3435
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/processesscraper"
3536
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/processscraper"
@@ -73,6 +74,10 @@ var systemSpecificMetrics = map[string][]string{
7374
"solaris": {"system.filesystem.inodes.usage"},
7475
}
7576

77+
var systemSpecificMetricsNFS = map[string][]string{
78+
"linux": {"nfs.client.net.count", "nfs.client.net.tcp.connection.accepted", "nfs.client.rpc.count", "nfs.client.rpc.retransmit.count", "nfs.client.rpc.authrefresh.count", "nfs.client.procedure.count", "nfs.client.operation.count", "nfs.server.repcache.requests", "nfs.server.fh.stale.count", "nfs.server.io", "nfs.server.thread.count", "nfs.server.net.count", "nfs.server.net.tcp.connection.accepted", "nfs.server.rpc.count", "nfs.server.procedure.count", "nfs.server.operation.count"},
79+
}
80+
7681
func TestGatherMetrics_EndToEnd(t *testing.T) {
7782
sink := new(consumertest.MetricsSink)
7883

@@ -92,6 +97,11 @@ func TestGatherMetrics_EndToEnd(t *testing.T) {
9297
),
9398
}
9499

100+
if runtime.GOOS == "linux" && nfsscraper.CanScrapeAll() {
101+
f := nfsscraper.NewFactory()
102+
cfg.Scrapers[f.Type()] = f.CreateDefaultConfig()
103+
}
104+
95105
if runtime.GOOS == "linux" || runtime.GOOS == "windows" {
96106
f := processscraper.NewFactory()
97107
cfg.Scrapers[f.Type()] = f.CreateDefaultConfig()
@@ -143,6 +153,10 @@ func assertIncludesExpectedMetrics(t *testing.T, got pmetric.Metrics) {
143153
expectedMetrics := allMetrics
144154

145155
expectedMetrics = append(expectedMetrics, systemSpecificMetrics[runtime.GOOS]...)
156+
if nfsscraper.CanScrapeAll() {
157+
expectedMetrics = append(expectedMetrics, systemSpecificMetricsNFS[runtime.GOOS]...)
158+
}
159+
146160
assert.Len(t, returnedMetrics, len(expectedMetrics))
147161
for _, expected := range expectedMetrics {
148162
assert.Contains(t, returnedMetrics, expected)
@@ -325,6 +339,19 @@ func Benchmark_ScrapePagingMetrics(b *testing.B) {
325339
benchmarkScrapeMetrics(b, cfg)
326340
}
327341

342+
func Benchmark_ScrapeNFSMetrics(b *testing.B) {
343+
if !nfsscraper.CanScrapeAll() || runtime.GOOS != "linux" {
344+
b.Skip("skipping test on unsupported platform")
345+
}
346+
347+
cfg := &Config{
348+
ControllerConfig: scraperhelper.NewDefaultControllerConfig(),
349+
Scrapers: newScrapersConfigs(nfsscraper.NewFactory()),
350+
}
351+
352+
benchmarkScrapeMetrics(b, cfg)
353+
}
354+
328355
func Benchmark_ScrapeProcessMetrics(b *testing.B) {
329356
if runtime.GOOS != "linux" && runtime.GOOS != "windows" {
330357
b.Skip("skipping test on non linux/windows")

‎receiver/hostmetricsreceiver/internal/scraper/nfsscraper/documentation.md‎

Lines changed: 183 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,188 @@ metrics:
1212
enabled: false
1313
```
1414
15-
### system.nfs.net.count
15+
### nfs.client.net.count
1616
17-
Reports the count of kernel NFS client network requests handled
17+
Reports the count of kernel NFS client TCP segments and UDP datagrams handled.
1818
19-
| Unit | Metric Type | Value Type | Stability |
20-
| ---- | ----------- | ---------- | --------- |
21-
| {request} | Gauge | Int | development |
19+
| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic | Stability |
20+
| ---- | ----------- | ---------- | ----------------------- | --------- | --------- |
21+
| {record} | Sum | Int | Cumulative | true | development |
22+
23+
#### Attributes
24+
25+
| Name | Description | Values | Optional |
26+
| ---- | ----------- | ------ | -------- |
27+
| network.transport | OSI transport layer or inter-process communication method. | Str: ``udp``, ``tcp`` | false |
28+
29+
### nfs.client.net.tcp.connection.accepted
30+
31+
Reports the count of kernel NFS client TCP connections accepted
32+
33+
| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic | Stability |
34+
| ---- | ----------- | ---------- | ----------------------- | --------- | --------- |
35+
| {connection} | Sum | Int | Cumulative | true | development |
36+
37+
### nfs.client.operation.count
38+
39+
Reports the count of kernel NFSv4+ client operations
40+
41+
| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic | Stability |
42+
| ---- | ----------- | ---------- | ----------------------- | --------- | --------- |
43+
| {operation} | Sum | Int | Cumulative | true | development |
44+
45+
#### Attributes
46+
47+
| Name | Description | Values | Optional |
48+
| ---- | ----------- | ------ | -------- |
49+
| onc_rpc.version | ONC/Sun RPC program version. | Any Int | false |
50+
| nfs.operation.name | NFSv4+ operation name. | Any Str | false |
51+
52+
### nfs.client.procedure.count
53+
54+
Reports the count of kernel NFS client procedures
55+
56+
| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic | Stability |
57+
| ---- | ----------- | ---------- | ----------------------- | --------- | --------- |
58+
| {procedure} | Sum | Int | Cumulative | true | development |
59+
60+
#### Attributes
61+
62+
| Name | Description | Values | Optional |
63+
| ---- | ----------- | ------ | -------- |
64+
| onc_rpc.version | ONC/Sun RPC program version. | Any Int | false |
65+
| onc_rpc.procedure.name | ONC/Sun RPC procedure name. | Any Str | false |
66+
67+
### nfs.client.rpc.authrefresh.count
68+
69+
Reports the count of kernel NFS client RPC authentication refreshes
70+
71+
| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic | Stability |
72+
| ---- | ----------- | ---------- | ----------------------- | --------- | --------- |
73+
| {authrefresh} | Sum | Int | Cumulative | true | development |
74+
75+
### nfs.client.rpc.count
76+
77+
Reports the count of kernel NFS client RPCs sent, regardless of whether they're accepted/rejected by the server.
78+
79+
| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic | Stability |
80+
| ---- | ----------- | ---------- | ----------------------- | --------- | --------- |
81+
| {request} | Sum | Int | Cumulative | true | development |
82+
83+
### nfs.client.rpc.retransmit.count
84+
85+
Reports the count of kernel NFS client RPC retransmits
86+
87+
| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic | Stability |
88+
| ---- | ----------- | ---------- | ----------------------- | --------- | --------- |
89+
| {retransmit} | Sum | Int | Cumulative | true | development |
90+
91+
### nfs.server.fh.stale.count
92+
93+
Reports the cumulative count of kernel NFS server stale file handles.
94+
95+
| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic | Stability |
96+
| ---- | ----------- | ---------- | ----------------------- | --------- | --------- |
97+
| {fh} | Sum | Int | Cumulative | true | development |
98+
99+
### nfs.server.io
100+
101+
Reports the count of kernel NFS server bytes returned to receive and transmit (read and write) requests.
102+
103+
| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic | Stability |
104+
| ---- | ----------- | ---------- | ----------------------- | --------- | --------- |
105+
| By | Sum | Int | Cumulative | true | development |
106+
107+
#### Attributes
108+
109+
| Name | Description | Values | Optional |
110+
| ---- | ----------- | ------ | -------- |
111+
| network.io.direction | The network IO operation direction. | Str: ``transmit``, ``receive`` | false |
112+
113+
### nfs.server.net.count
114+
115+
Reports the count of kernel NFS server TCP segments and UDP datagrams handled.
116+
117+
| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic | Stability |
118+
| ---- | ----------- | ---------- | ----------------------- | --------- | --------- |
119+
| {request} | Sum | Int | Cumulative | true | development |
120+
121+
#### Attributes
122+
123+
| Name | Description | Values | Optional |
124+
| ---- | ----------- | ------ | -------- |
125+
| network.transport | OSI transport layer or inter-process communication method. | Str: ``udp``, ``tcp`` | false |
126+
127+
### nfs.server.net.tcp.connection.accepted
128+
129+
Reports the count of kernel NFS server TCP connections accepted
130+
131+
| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic | Stability |
132+
| ---- | ----------- | ---------- | ----------------------- | --------- | --------- |
133+
| {connection} | Sum | Int | Cumulative | true | development |
134+
135+
### nfs.server.operation.count
136+
137+
Reports the count of kernel NFSv4+ server operations
138+
139+
| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic | Stability |
140+
| ---- | ----------- | ---------- | ----------------------- | --------- | --------- |
141+
| {operation} | Sum | Int | Cumulative | true | development |
142+
143+
#### Attributes
144+
145+
| Name | Description | Values | Optional |
146+
| ---- | ----------- | ------ | -------- |
147+
| onc_rpc.version | ONC/Sun RPC program version. | Any Int | false |
148+
| nfs.operation.name | NFSv4+ operation name. | Any Str | false |
149+
150+
### nfs.server.procedure.count
151+
152+
Reports the count of kernel NFS server procedures
153+
154+
| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic | Stability |
155+
| ---- | ----------- | ---------- | ----------------------- | --------- | --------- |
156+
| {procedure} | Sum | Int | Cumulative | true | development |
157+
158+
#### Attributes
159+
160+
| Name | Description | Values | Optional |
161+
| ---- | ----------- | ------ | -------- |
162+
| onc_rpc.version | ONC/Sun RPC program version. | Any Int | false |
163+
| onc_rpc.procedure.name | ONC/Sun RPC procedure name. | Any Str | false |
164+
165+
### nfs.server.repcache.requests
166+
167+
Reports the kernel NFS server reply cache request count by cache hit status.
168+
169+
| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic | Stability |
170+
| ---- | ----------- | ---------- | ----------------------- | --------- | --------- |
171+
| {request} | Sum | Int | Cumulative | true | development |
172+
173+
#### Attributes
174+
175+
| Name | Description | Values | Optional |
176+
| ---- | ----------- | ------ | -------- |
177+
| nfs.server.repcache.status | NFS Server replies check a Reply Cache (repcache), which can have one of 3 result states: 'hit', 'miss', or 'nocache'. | Str: ``hit``, ``miss``, ``nocache`` | false |
178+
179+
### nfs.server.rpc.count
180+
181+
Reports the count of kernel NFS server RPCs handled.
182+
183+
| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic | Stability |
184+
| ---- | ----------- | ---------- | ----------------------- | --------- | --------- |
185+
| {request} | Sum | Int | Cumulative | true | development |
186+
187+
#### Attributes
188+
189+
| Name | Description | Values | Optional |
190+
| ---- | ----------- | ------ | -------- |
191+
| error.type | Describes a class of error the operation ended with. | Str: ``format``, ``auth``, ``client`` | false |
192+
193+
### nfs.server.thread.count
194+
195+
Reports the count of kernel NFS server available threads
196+
197+
| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic | Stability |
198+
| ---- | ----------- | ---------- | ----------------------- | --------- | --------- |
199+
| {thread} | Sum | Int | Cumulative | false | development |

‎receiver/hostmetricsreceiver/internal/scraper/nfsscraper/factory.go‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,15 +33,15 @@ func createDefaultConfig() component.Config {
3333

3434
// createMetricsScraper creates a resource scraper based on provided config.
3535
func createMetricsScraper(
36-
ctx context.Context,
36+
_ context.Context,
3737
settings scraper.Settings,
3838
cfg component.Config,
3939
) (scraper.Metrics, error) {
4040
if !supportedOS {
4141
return nil, errUnsupportedOS
4242
}
4343

44-
nfsScraper := newNfsScraper(ctx, settings, cfg.(*Config))
44+
nfsScraper := newNfsScraper(settings, cfg.(*Config))
4545

4646
return scraper.NewMetrics(
4747
nfsScraper.scrape,

‎receiver/hostmetricsreceiver/internal/scraper/nfsscraper/generated_component_test.go‎

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)