Skip to content

Commit 7642b13

Browse files
sdodsleysdodsley
authored
Merge pull request #33 from genegr/master
Proposed new minor version 1.0.4
2 parents 44ac57b + a76dca6 commit 7642b13

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

41 files changed

+1810
-132
lines changed

‎Makefile‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ GOCMD=go
33
GOTEST=$(GOCMD) test
44
GOVET=$(GOCMD) vet
55
BINARY_NAME=pure-fa-om-exporter
6-
MODULE_NAME=go mod init purestorage/fa-openmetrics-exporter
6+
MODULE_NAME=purestorage/fa-openmetrics-exporter
77
VERSION?=1.0.3
88
SERVICE_PORT?=9490
99
DOCKER_REGISTRY?= quay.io/purestorage/
@@ -21,12 +21,13 @@ all: help
2121

2222
## Build:
2323
init:
24-
$(GOCMD) init go mod init $(MODULE_NAME)
24+
$(GOCMD) mod init $(MODULE_NAME)
2525
$(GOCMD) mod tidy
2626

27-
build: ## Build your project and put the output binary in out/bin/
27+
build: ## Build project and put the output binary in out/bin/
2828
mkdir -p out/bin
29-
GO111MODULE=on $(GOCMD) build -o out/bin/$(BINARY_NAME) cmd/fa-om-exporter/main.go
29+
CGO_ENABLED=0 GO111MODULE=on $(GOCMD) build -a -tags 'netgo osusergo static_build' -o out/bin/$(BINARY_NAME) cmd/fa-om-exporter/main.go
30+
# CGO_ENABLED=1 GO111MODULE=on $(GOCMD) build -o out/bin/$(BINARY_NAME) cmd/fa-om-exporter/main.go
3031

3132
clean: ## Remove build related file
3233
rm -fr ./bin

‎README.md‎

Lines changed: 93 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -69,21 +69,59 @@ docker build -t pure-fa-ome:$VERSION .
6969

7070
**Authentication**
7171

72-
Authentication is used by the exporter as the mechanism to cross authenticate to the scraped appliance, therefore for each array it is required to provide the REST API token for an account that has a 'readonly' role. The api-token must be provided in the http request using the HTTP Authorization header of type 'Bearer'. This is achieved by specifying the api-token value as the authorization parameter of the specific job in the Prometheus configuration file.
72+
Authentication is used by the exporter as the mechanism to cross authenticate to the scraped appliance, therefore for each array it is required to provide the REST API token for an account that has a 'readonly' role. The api-token can be provided in two ways
73+
74+
- using the HTTP Authorization header of type 'Bearer', or
75+
- via a configuration map in a specific configuration file.
76+
77+
The first option requires to specify the api-token value as the authorization parameter of the specific job in the Prometheus configuration file.
78+
The second option provides the FlashArray/api-token key-pair map for a list of arrays in a simple YAML configuration file that is passed as paramether to the exporter. This makes possible to write more concise Prometheus configuration files and also to configure other scrapers that cannot use the HTTP authentication header.
79+
80+
### Usage
81+
82+
```shell
83+
84+
usage: pure-fa-om-exporter [-h|--help] [-a|--address "<value>"] [-p|--port <integer>] [-d|--debug] [-t|--tokens <file>]
85+
86+
Pure Storage FA OpenMetrics exporter
87+
88+
Arguments:
89+
90+
-h --help Print help information
91+
-a --address IP address for this exporter to bind to. Default: 0.0.0.0
92+
-p --port Port for this exporter to listen. Default: 9490
93+
-d --debug Enable debug. Default: false
94+
-t --tokens API token(s) map file
95+
```
96+
97+
The array token configuration file must have to following syntax:
98+
99+
```shell
100+
<array_id1>:
101+
address: <ip-address>|<hosname1>
102+
api_token: <api-token1>
103+
<array_id2>:
104+
address: <ip-address2>|<hostname2>
105+
api_token: <api-token2>
106+
...
107+
<array_idN>:
108+
address: <ip-addressN>|<hostnameN>
109+
api_token: <api-tokenN>
110+
```
73111

74112
### Scraping endpoints
75113

76114
The exporter uses a RESTful API schema to provide Prometheus scraping endpoints.
77115

78116

79-
URL | GET parameters | Description
80-
---|---|---
81-
http://\<exporter-host\>:\<port\>/metrics | endpoint | Full array metrics
82-
http://\<exporter-host\>:\<port\>/metrics/array | endpoint | Array only metrics
83-
http://\<exporter-host\>:\<port\>/metrics/volumes | endpoint | Volumes only metrics
84-
http://\<exporter-host\>:\<port\>/metrics/hosts | endpoint | Hosts only metrics
85-
http://\<exporter-host\>:\<port\>/metrics/pods | endpoint | Pods only metrics
86-
http://\<exporter-host\>:\<port\>/metrics/directories| endpoint | Directories only metrics
117+
|URL |GET parameters |Description |
118+
|------------------------------------------------------|---------------|-------------------------|
119+
|http://\<exporter-host\>:\<port\>/metrics |endpoint |Full array metrics |
120+
|http://\<exporter-host\>:\<port\>/metrics/array |endpoint |Array only metrics |
121+
|http://\<exporter-host\>:\<port\>/metrics/volumes |endpoint |Volumes only metrics |
122+
|http://\<exporter-host\>:\<port\>/metrics/hosts |endpoint |Hosts only metrics |
123+
|http://\<exporter-host\>:\<port\>/metrics/pods |endpoint |Pods only metrics |
124+
|http://\<exporter-host\>:\<port\>/metrics/directories |endpoint |Directories only metrics |
87125

88126
Depending on the target array, scraping for the whole set of metrics could result into timeout issues, in which case it is suggested either to increase the scraping timeout or to scrape each single endpoint instead.
89127

@@ -116,47 +154,51 @@ A complete example monitoring stack implemented in Docker Compose which can be f
116154

117155
### Metrics Collected
118156

119-
|Metric Name |Description |
120-
|--------------------------------------------|-----------------------------------------------------|
121-
|purefa_info |FlashArray system information |
122-
|purefa_alerts_open |FlashArray open alert events |
123-
|purefa_array_performance_average_bytes |FlashArray array average operations size in bytes |
124-
|purefa_array_performance_bandwidth_bytes |FlashArray array throughput in bytes per second |
125-
|purefa_array_performance_latency_usec |FlashArray array latency in microseconds |
126-
|purefa_array_performance_queue_depth_ops |FlashArray array queue depth size |
127-
|purefa_array_performance_throughput_iops |FlashArray array throughput in iops |
128-
|purefa_array_space_bytes |FlashArray array space in bytes |
129-
|purefa_array_space_data_reduction_ratio |FlashArray array space data reduction |
130-
|purefa_array_space_utilization |FlashArray array space utilization in percent |
131-
|purefa_directory_performance_average_bytes |FlashArray directory average operations size in bytes|
132-
|purefa_directory_performance_bandwidth_bytes|FlashArray directory throughput in bytes per second |
133-
|purefa_directory_performance_latency_usec |FlashArray directory latency in microseconds |
134-
|purefa_directory_performance_throughput_iops|FlashArray directory throughput in iops |
135-
|purefa_directory_space_bytes |FlashArray directory space in bytes |
136-
|purefa_directory_space_data_reduction_ratio |FlashArray directory space data reduction |
137-
|purefa_host_connections_info |FlashArray host volumes connections |
138-
|purefa_host_performance_average_bytes |FlashArray host average operations size in bytes |
139-
|purefa_host_performance_bandwidth_bytes |FlashArray host bandwidth in bytes per second |
140-
|purefa_host_performance_latency_usec |FlashArray host latency in microseconds |
141-
|purefa_host_performance_throughput_iops |FlashArray host throughput in iops |
142-
|purefa_host_space_bytes |FlashArray host space in bytes |
143-
|purefa_host_space_data_reduction_ratio |FlashArray host space data reduction |
144-
|purefa_host_space_size_bytes |FlashArray host volumes size |
145-
|purefa_hw_component_status |FlashArray hardware component status |
146-
|purefa_hw_component_temperature_celsius |FlashArray hardware component temperature in C |
147-
|purefa_hw_component_voltage_volt |FlashArray hardware component voltage |
148-
|purefa_pod_performance_average_bytes |FlashArray pod average operations size |
149-
|purefa_pod_performance_bandwidth_bytes |FlashArray pod throughput in bytes per second |
150-
|purefa_pod_performance_latency_usec |FlashArray pod latency in microseconds |
151-
|purefa_pod_performance_throughput_iops |FlashArray pod throughput in iops |
152-
|purefa_pod_space_bytes |FlashArray pod space in bytes |
153-
|purefa_pod_space_data_reduction_ratio |FlashArray pod space data reduction |
154-
|purefa_volume_performance_average_bytes |FlashArray volume average operations size in bytes |
155-
|purefa_volume_performance_bandwidth_bytes |FlashArray volume throughput in bytes per second |
156-
|purefa_volume_performance_latency_usec |FlashArray volume latency in microseconds |
157-
|purefa_volume_performance_throughput_iops |FlashArray volume throughput in iops |
158-
|purefa_volume_space_bytes |FlashArray volume space in bytes |
159-
|purefa_volume_space_data_reduction_ratio |FlashArray volume space data reduction |
157+
|Metric Name |Description |
158+
|-----------------------------------------------------|------------------------------------------------------------|
159+
|purefa_info |FlashArray system information |
160+
|purefa_alerts_open |FlashArray open alert events |
161+
|purefa_array_performance_average_bytes |FlashArray array average operations size in bytes |
162+
|purefa_array_performance_bandwidth_bytes |FlashArray array throughput in bytes per second |
163+
|purefa_array_performance_latency_usec |FlashArray array latency in microseconds |
164+
|purefa_array_performance_queue_depth_ops |FlashArray array queue depth size |
165+
|purefa_array_performance_throughput_iops |FlashArray array throughput in iops |
166+
|purefa_array_space_bytes |FlashArray array space in bytes |
167+
|purefa_array_space_data_reduction_ratio |FlashArray array space data reduction |
168+
|purefa_array_space_utilization |FlashArray array space utilization in percent |
169+
|purefa_directory_performance_average_bytes |FlashArray directory average operations size in bytes |
170+
|purefa_directory_performance_bandwidth_bytes |FlashArray directory throughput in bytes per second |
171+
|purefa_directory_performance_latency_usec |FlashArray directory latency in microseconds |
172+
|purefa_directory_performance_throughput_iops |FlashArray directory throughput in iops |
173+
|purefa_directory_space_bytes |FlashArray directory space in bytes |
174+
|purefa_directory_space_data_reduction_ratio |FlashArray directory space data reduction |
175+
|purefa_host_connections_info |FlashArray host volumes connections |
176+
|purefa_host_performance_average_bytes |FlashArray host average operations size in bytes |
177+
|purefa_host_performance_bandwidth_bytes |FlashArray host bandwidth in bytes per second |
178+
|purefa_host_performance_latency_usec |FlashArray host latency in microseconds |
179+
|purefa_host_performance_throughput_iops |FlashArray host throughput in iops |
180+
|purefa_host_space_bytes |FlashArray host space in bytes |
181+
|purefa_host_space_data_reduction_ratio |FlashArray host space data reduction |
182+
|purefa_host_space_size_bytes |FlashArray host volumes size |
183+
|purefa_hw_component_status |FlashArray hardware component status |
184+
|purefa_hw_component_temperature_celsius |FlashArray hardware component temperature in C |
185+
|purefa_hw_component_voltage_volt |FlashArray hardware component voltage |
186+
|purefa_pod_performance_average_bytes |FlashArray pod average operations size |
187+
|purefa_pod_performance_bandwidth_bytes |FlashArray pod throughput in bytes per second |
188+
|purefa_pod_performance_latency_usec |FlashArray pod latency in microseconds |
189+
|purefa_pod_performance_throughput_iops |FlashArray pod throughput in iops |
190+
|purefa_pod_space_bytes |FlashArray pod space in bytes |
191+
|purefa_pod_space_data_reduction_ratio |FlashArray pod space data reduction |
192+
|purefa_pod_performance_replication_bandwidth_bytes |FlashArray pod replication bandwidth in bytes per second |
193+
|purefa_pod_replica_links_performance_bandwidth_bytes |FlashArray pod replica links throughput in bytes per second |
194+
|purefa_pod_replica_links_lag_average_sec |FlashArray pod replica links average lag in seconds |
195+
|purefa_pod_replica_links_lag_max_sec |FlashArray pod replica links maximum lag in seconds |
196+
|purefa_volume_performance_average_bytes |FlashArray volume average operations size in bytes |
197+
|purefa_volume_performance_bandwidth_bytes |FlashArray volume throughput in bytes per second |
198+
|purefa_volume_performance_latency_usec |FlashArray volume latency in microseconds |
199+
|purefa_volume_performance_throughput_iops |FlashArray volume throughput in iops |
200+
|purefa_volume_space_bytes |FlashArray volume space in bytes |
201+
|purefa_volume_space_data_reduction_ratio |FlashArray volume space data reduction |
160202

161203
## Monitoring On-Premise with Prometheus and Grafana
162204
Take a holistic overview of your Pure Storage FlashArray estate on-premise with Prometheus and Grafana to summarize statistics such as:
@@ -174,4 +216,4 @@ For more information on dependencies, and notes to deploy -- take look at the ex
174216

175217
# License
176218

177-
This project is licensed under the Apache 2.0 License - see the [LICENSE](LICENSE) file for details
219+
This project is licensed under the Apache 2.0 License - see the [LICENSE](LICENSE) file for details.

‎build/docker/Dockerfile‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
FROM golang:alpine
1+
FROM golang:alpine as build
22

33
WORKDIR /usr/src/app
44

@@ -7,8 +7,11 @@ COPY go.mod go.sum ./
77
RUN go mod download && go mod verify
88

99
COPY . .
10-
RUN go build -v -o /usr/local/bin/pure-fa-om-exporter cmd/fa-om-exporter/main.go
10+
RUN CGO_ENABLED=1 go build -a -tags 'netgo osusergo static_build' -v -o /usr/local/bin/pure-fa-om-exporter cmd/fa-om-exporter/main.go
1111

12+
FROM scratch
13+
#FROM busybox:musl
14+
COPY --from=build /usr/local/bin/pure-fa-om-exporter /pure-fa-om-exporter
1215
EXPOSE 9490
13-
ENTRYPOINT ["/usr/local/bin/pure-fa-om-exporter"]
16+
ENTRYPOINT ["/pure-fa-om-exporter"]
1417
CMD ["--host", "0.0.0.0", "--port", "9490"]

‎cmd/fa-om-exporter/main.go‎

Lines changed: 55 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -2,29 +2,66 @@ package main
22

33
import (
44
"context"
5-
"flag"
65
"fmt"
76
"log"
87
"net/http"
8+
config "purestorage/fa-openmetrics-exporter/internal/config"
99
collectors "purestorage/fa-openmetrics-exporter/internal/openmetrics-exporter"
1010
client "purestorage/fa-openmetrics-exporter/internal/rest-client"
1111
"strings"
12+
"os"
13+
"io"
1214

15+
"gopkg.in/yaml.v3"
16+
"github.com/akamensky/argparse"
1317
"github.com/prometheus/client_golang/prometheus"
1418
"github.com/prometheus/client_golang/prometheus/promhttp"
1519
)
1620

17-
var version string = "1.0.3"
21+
var version string = "1.0.4"
1822
var debug bool = false
23+
var arraytokens config.FlashArrayList
24+
25+
func FileExists(args []string) error {
26+
_, err := os.Stat(args[0])
27+
return err
28+
}
1929

2030
func main() {
2131

22-
host := flag.String("host", "0.0.0.0", "Address of the exporter")
23-
port := flag.Int("port", 9490, "Port of the exporter")
24-
d := flag.Bool("debug", false, "Debug")
25-
flag.Parse()
26-
addr := fmt.Sprintf("%s:%d", *host, *port)
32+
parser := argparse.NewParser("pure-fa-om-exporter", "Pure Storage FA OpenMetrics exporter")
33+
host := parser.String("a", "address", &argparse.Options{Required: false, Help: "IP address for this exporter to bind to", Default: "0.0.0.0"})
34+
port := parser.Int("p", "port", &argparse.Options{Required: false, Help: "Port for this exporter to listen", Default: 9490})
35+
d := parser.Flag("d", "debug", &argparse.Options{Required: false, Help: "Enable debug", Default: false})
36+
at := parser.File("t", "tokens", os.O_RDONLY, 0600, &argparse.Options{Required: false, Validate: FileExists, Help: "API token(s) map file"})
37+
err := parser.Parse(os.Args)
38+
if err != nil {
39+
log.Fatalf("Error in token file: %v", err)
40+
}
41+
if !isNilFile(*at) {
42+
defer at.Close()
43+
buf := make([]byte, 1024)
44+
arrlist := ""
45+
for {
46+
n, err := at.Read(buf)
47+
if err == io.EOF {
48+
break
49+
}
50+
if err != nil {
51+
log.Fatalf("Reading token file: %v", err)
52+
}
53+
if n > 0 {
54+
arrlist = arrlist + string(buf[:n])
55+
}
56+
}
57+
buf = []byte(arrlist)
58+
err := yaml.Unmarshal(buf, &arraytokens)
59+
if err != nil {
60+
log.Fatalf("Unmarshalling token file: %v", err)
61+
}
62+
}
2763
debug = *d
64+
addr := fmt.Sprintf("%s:%d", *host, *port)
2865
log.Printf("Start Pure FlashArray exporter v%s on %s", version, addr)
2966

3067
http.HandleFunc("/", index)
@@ -78,11 +115,15 @@ func metricsHandler(w http.ResponseWriter, r *http.Request) {
78115
}
79116
authHeader := r.Header.Get("Authorization")
80117
authFields := strings.Fields(authHeader)
81-
if len(authFields) != 2 || strings.ToLower(authFields[0]) != "bearer" {
118+
119+
apitoken := arraytokens.GetApiToken(endpoint)
120+
if len(authFields) == 2 && strings.ToLower(authFields[0]) == "bearer" {
121+
apitoken = authFields[1]
122+
}
123+
if apitoken == "" {
82124
http.Error(w, "Target authorization token is missing", http.StatusBadRequest)
83125
return
84126
}
85-
apitoken := authFields[1]
86127

87128
registry := prometheus.NewRegistry()
88129
faclient := client.NewRestClient(endpoint, apitoken, apiver, debug)
@@ -154,3 +195,8 @@ func index(w http.ResponseWriter, r *http.Request) {
154195

155196
fmt.Fprintf(w, "%s", msg)
156197
}
198+
199+
func isNilFile(f os.File) bool {
200+
var tf os.File
201+
return f == tf
202+
}

‎go.mod‎

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,24 @@
11
module purestorage/fa-openmetrics-exporter
22

3-
go 1.19
3+
go 1.20
44

55
require (
6+
github.com/akamensky/argparse v1.4.0
67
github.com/go-resty/resty/v2 v2.7.0
78
github.com/google/go-cmp v0.5.9
89
github.com/prometheus/client_golang v1.14.0
910
github.com/prometheus/client_model v0.3.0
11+
gopkg.in/yaml.v3 v3.0.1
1012
)
1113

1214
require (
1315
github.com/beorn7/perks v1.0.1 // indirect
14-
github.com/cespare/xxhash/v2 v2.2.0 // indirect
16+
github.com/cespare/xxhash/v2 v2.1.2 // indirect
1517
github.com/golang/protobuf v1.5.2 // indirect
16-
github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
17-
github.com/prometheus/common v0.39.0 // indirect
18-
github.com/prometheus/procfs v0.9.0 // indirect
19-
golang.org/x/net v0.5.0 // indirect
20-
golang.org/x/sys v0.4.0 // indirect
18+
github.com/matttproud/golang_protobuf_extensions v1.0.1 // indirect
19+
github.com/prometheus/common v0.37.0 // indirect
20+
github.com/prometheus/procfs v0.8.0 // indirect
21+
golang.org/x/net v0.0.0-20220225172249-27dd8689420f // indirect
22+
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a // indirect
2123
google.golang.org/protobuf v1.28.1 // indirect
2224
)

0 commit comments

Comments
 (0)