Skip to content

Commit 68874e4

Browse files
HeleneHelene
committed
publish Dockerfile
-added Dockerfile which can be used to run the bridge in a docker container. -added instructions describing how to build the bridge image and run it in a container
1 parent 1cbdd4e commit 68874e4

File tree

8 files changed

+692
-6
lines changed

8 files changed

+692
-6
lines changed

‎README.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@ Grafana Bridge is a standalone Python application. It translates the IBM Spectru
66

77
## Setup
88

9+
If you want run the ***IBM Spectrum Scale bridge for Grafana*** in a docker container please follow this [link](/docs/RUNNING_AS_DOCKER_CONTAINER.md)
10+
911
### Prerequisites
1012

1113
Before installing the IBM Spectrum Scale bridge for Grafana you must install the software prerequisites. Those are:
@@ -20,7 +22,7 @@ This package could be used for:
2022
- IBM Spectrum Scale devices having mimimum release level 5.0.3 FP2 and above
2123
- Grafana 6.0.0 and above
2224

23-
To use this tool on the older IBM Spectrum Scale devices please refer to the [SUPPORT_MATRIX](SUPPORT_MATRIX.md) file.
25+
To use this tool on the older IBM Spectrum Scale devices please refer to the [SUPPORT_MATRIX](/docs/SUPPORT_MATRIX.md) file.
2426

2527

2628

‎docs/RUNNING_AS_DOCKER_CONTAINER.md

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
## Running the IBM Spectrum Scale Performance Monitoring Bridge as Docker Container
2+
3+
The IBM Spectrum Scale system must run 5.0.5 or above.
4+
5+
6+
### On the IBM Spectrum Scale cluster node running pmcollector, enable data query remote connection
7+
8+
9+
```shell
10+
# vi /opt/IBM/zimon/ZIMonSensors.cfg
11+
12+
# Interface over which incoming queries are accepted (default: 127.0.0.1 to
13+
# prevent queries from remote machines).
14+
queryinterface = "0.0.0.0"
15+
16+
# systemctl restart pmcollector
17+
18+
```
19+
20+
21+
1. On the host, where you are running the bridge, generate a private key. For example, you can use openssl command and follow the OpenSSL ‘howto’ instructions:
22+
23+
```shell
24+
# openssl genrsa -out privkey.pem 2048
25+
```
26+
27+
28+
### On the host running docker/podman perform the following steps:
29+
30+
1. Clone this repository using git in your favourite directory :
31+
32+
```shell
33+
# git clone https://github.com/IBM/ibm-spectrum-scale-bridge-for-grafana.git grafana_bridge
34+
```
35+
36+
37+
2. Copy the 'mmsdrfs' file from your IBM Spectrum Scale cluster to the 'grafana_bridge/source/gpfsConfig' directory
38+
cp grafana_bridge/source/gpfsConfig
39+
40+
```shell
41+
# scp <my_gpfs_cluster_node>:/var/mmfs/gen/mmsdrfs grafana_bridge/source/gpfsConfig
42+
43+
```
44+
45+
46+
3. Create the bridge container image
47+
48+
```shell
49+
# cd grafana_bridge/source
50+
51+
# podman build -t bridge_image:gpfs505 .
52+
```
53+
54+
55+
4. Start the bridge running in a container:
56+
57+
```shell
58+
# podman run -dt -p 4242:4242 -e "SERVER=9.XXX.XXX.XXX" --pod new:my-pod --name grafana_bridge bridge_image:gpfs505
59+
60+
# podman logs grafana_bridge
61+
62+
```
63+
64+
65+
Now you can add the host running the bridge container to the Grafana monitoring Datasource list.

‎source/Dockerfile

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
FROM registry.access.redhat.com/ubi8/ubi as pmcollector
2+
3+
4+
#
5+
# Installing tools required by pmcollector.
6+
#
7+
8+
RUN yum install -y libicu gdbm-devel boost-devel python27
9+
10+
#RUN yum install -y libicu
11+
12+
#RUN yum install -y boost-regex
13+
14+
#RUN yum list all
15+
16+
#
17+
# Installing few tools in the image that help to debug it easier.
18+
#
19+
RUN yum install -y gcc gcc-c++ make net-tools m4 libaio ksh which iproute procps-ng util-linux openssh-clients openssh-server setup elfutils-libelf-devel hostname kmod pciutils iputils bind-utils
20+
21+
#RUN yum list all
22+
23+
24+
25+
#
26+
# Installing Spectrum Scale pmcollector
27+
#
28+
ARG ARTIKEY
29+
RUN curl --fail -H "X-Jfrog-Art-Api: $ARTIKEY" -O https://na.artifactory.swg-devops.com/artifactory/res-fsaas-generic-local/ScaleRPMs/gpfs.gss.pmcollector-5.0.4-2.el8.x86_64.rpm
30+
RUN mkdir -p /opt/IBM/zimon/config
31+
RUN mkdir -p /opt/IBM/zimon/zdata
32+
RUN rpm -ivh gpfs.gss.pmcollector-5.0.4-2.el8.x86_64.rpm
33+
34+
35+
COPY ZIMonCollector.cfg /opt/IBM/zimon/config/ZIMonCollector.cfg
36+
37+
#
38+
# Create a user 'scalepm' under 'root' group
39+
#
40+
RUN groupadd -g 1099 scalepm
41+
RUN useradd -rm -d /home/1001 -s /bin/bash -g 1099 -u 1001 scalepm
42+
43+
#
44+
# Chown all the files to the zimon 'scalepm' user.
45+
#
46+
RUN chown -R 1001:1099 /opt/IBM/zimon
47+
RUN chown -R 1001:1099 /var/log/zimon
48+
RUN chown -R 1001:1099 /var/run/perfmon
49+
50+
#
51+
# Switch to user 'scalepm'
52+
#
53+
USER 1001
54+
55+
CMD ["/opt/IBM/zimon/sbin/pmcollector", "-C", "/opt/IBM/zimon/config/ZIMonCollector.cfg", "-R", "/var/run/perfmon"]
56+
57+
# helpful for debugging
58+
#CMD ["tail", "-f", "/dev/null"]
59+
60+
61+
EXPOSE 4739 9085 9084 9094

‎source/gpfsConfig/ZIMonSensors.cfg.template

Lines changed: 216 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,216 @@
1+
# ZIMonSensors configuration file
2+
#
3+
# Comments start with # and extend to the end of the line. Each
4+
# parameter definition must start on a separate line. Parameters can
5+
# be of the different types:
6+
# - flags boolean values specified as T or Y for true and F or N or
7+
# false
8+
# - strings must be quoted
9+
# - numbers must be integers
10+
# - complex parameters must be enclosed in braces { } and contain
11+
# parameter assignments.
12+
13+
# The hostname used to report this sensor's data as coming from
14+
# (default: hostname of the machine). This option should only be used
15+
# for debugging and testing, use at your own risk.
16+
#hostname = "efsNodeXYZ"
17+
18+
# Run as daemon in the background (default: N).
19+
daemonize = Y
20+
21+
# Interface over which IPFIX data is sent (default: 0.0.0.0).
22+
#ipfixinterface = "0.0.0.0"
23+
24+
# Path and filename of the logfile (default: stdout if running in
25+
# non-daemon mode, "collector.cfg" if running in daemon mode). Use
26+
# "syslog" to send logging data to the syslog daemon.
27+
logfile = "/var/log/zimon/ZIMonSensors.log"
28+
29+
# Log levels are: debug, info, warning, error (default: info).
30+
loglevel = "info"
31+
32+
# Directory that is used to store the pid file (default: /var/run).
33+
piddir = "/var/run"
34+
35+
# Specifies the path and filename of the MmpmonSockProxy.
36+
mmpmon = "/opt/IBM/zimon/MmpmonSockProxy"
37+
38+
# Specifies the path to the MM command proxy.
39+
mmcmd = "/opt/IBM/zimon/MMCmdProxy"
40+
41+
# Specifies the path to the MMDF command proxy
42+
mmdfcmd = "/opt/IBM/zimon/MMDFProxy"
43+
44+
# The list of sensors and their configuration. Sensors that are not
45+
# listed here or assigned a zero reporting period are ignored.
46+
sensors = {
47+
name = "CPU"
48+
period = 1
49+
}, {
50+
name = "Load"
51+
period = 1
52+
}, {
53+
name = "Memory"
54+
period = 1
55+
}, {
56+
name = "Network"
57+
period = 1
58+
filter = "netdev_name=veth.*"
59+
}, {
60+
name = "Netstat"
61+
period = 10
62+
}, {
63+
name = "Diskstat"
64+
period = 0
65+
}, {
66+
name = "DiskFree"
67+
period = 600
68+
filter = "mountPoint=/var/lib/docker.*"
69+
}, {
70+
name = "Infiniband"
71+
period = 0
72+
}, {
73+
name = "GPFSDisk"
74+
period = 0
75+
}, {
76+
name = "GPFSFilesystem"
77+
period = 10
78+
}, {
79+
name = "GPFSNSDDisk"
80+
period = 10
81+
# restrict is only available for GPFS-based configurations
82+
restrict = "nsdNodes"
83+
}, {
84+
name = "GPFSPoolIO"
85+
period = 0
86+
}, {
87+
name = "GPFSVFS"
88+
period = 10
89+
}, {
90+
name = "GPFSIOC"
91+
period = 0
92+
}, {
93+
name = "GPFSVIO64"
94+
# GPFSVIO64 uses 64-bit counters and replaces GPFSVIO
95+
# new metric names are gpfs_vio64_.. instead of gpfs_vio_..
96+
period = 0
97+
}, {
98+
name = "GPFSPDDisk"
99+
period = 10
100+
# restrict is only available for GPFS-based configurations
101+
restrict = "nsdNodes"
102+
}, {
103+
name = "GPFSvFLUSH"
104+
period = 0
105+
}, {
106+
name = "GPFSNode"
107+
period = 10
108+
}, {
109+
name = "GPFSNodeAPI"
110+
period = 10
111+
}, {
112+
name = "GPFSFilesystemAPI"
113+
period = 10
114+
}, {
115+
name = "GPFSLROC"
116+
period = 0
117+
}, {
118+
name = "GPFSCHMS"
119+
period = 0
120+
}, {
121+
name = "GPFSAFM"
122+
period = 0
123+
}, {
124+
name = "GPFSAFMFS"
125+
period = 0
126+
}, {
127+
name = "GPFSAFMFSET"
128+
period = 0
129+
}, {
130+
name = "GPFSRPCS"
131+
period = 10
132+
}, {
133+
name = "GPFSWaiters"
134+
period = 10
135+
}, {
136+
name = "GPFSFilesetQuota"
137+
# This sensor should be activated only on a single node in a
138+
# cluster (restrict = ...) and SHOULD not be run more often than
139+
# once per hour (3600).
140+
period = 3600
141+
restrict = "@CLUSTER_PERF_SENSOR"
142+
}, {
143+
name = "GPFSFileset"
144+
# This sensor should be activated only on a single node in a
145+
# cluster (restrict = ...) and SHOULD not be run more often
146+
# than every 300s, typically this sensor is activated by the
147+
# Spectrum Scale Health component.
148+
period = 300
149+
restrict = "@CLUSTER_PERF_SENSOR"
150+
}, {
151+
name = "GPFSPool"
152+
# This sensor should be activated only on a single node in a
153+
# cluster (restrict = ...) and SHOULD not be run more often
154+
# than every 300s, typically this sensor is activated by the
155+
# Spectrum Scale Health component.
156+
period = 300
157+
restrict = "@CLUSTER_PERF_SENSOR"
158+
}, {
159+
name = "GPFSDiskCap"
160+
# This sensor should be activated only on a single node in a
161+
# cluster (restrict = ...) and SHOULD not be run more often
162+
# than once per day (86400).
163+
period = 86400
164+
restrict = "@CLUSTER_PERF_SENSOR"
165+
}, {
166+
name = "GPFSEventProducer"
167+
# This sensor should be activated only when a policy is in place
168+
# with rules containing ... ACTION(lweSEND(...))...
169+
# Suggested frequency is every 10 seconds
170+
period = 0
171+
}, {
172+
name = "GPFSMutex"
173+
# This sensor should be activated only when determining
174+
# resource contention for GPFS mutexes
175+
# Suggested frequency is every 10 seconds
176+
period = 0
177+
}, {
178+
name = "GPFSCondvar"
179+
# This sensor should be activated only when determining
180+
# resource contention for GPFS condition variables
181+
# Suggested frequency is every 10 seconds
182+
period = 0
183+
}, {
184+
name = "TopProc"
185+
# This sensor should be activated only when determining
186+
# resource consumption of the top-k processes
187+
# Suggested frequency is every 10 seconds
188+
period = 60
189+
}
190+
191+
# This is an entry for a generic proxy such as the GaneshaProxy:
192+
#, {
193+
# NSF Ganesha statistics
194+
# name = "NFSIO"
195+
# period = 0
196+
# type = "Generic"
197+
# proxyCmd ="/opt/IBM/zimon/GaneshaProxy"
198+
# restrict is only available for GPFS-based configurations
199+
# restrict = "cesNodes"
200+
#}
201+
202+
# The number of collectors to which ZIMonSensors will connect. This
203+
# parameter is used only in conjunction with the GPFS mmperfmon tool
204+
# when creating a GPFS-managed sensor configuration.
205+
colRedundancy = 2
206+
207+
# Specifies the ZIMon backend (collectors) to which these sensors are
208+
# reporting. Each collector is specified by a hostname or IP address
209+
# and a port number. Initially, no collector is specified.
210+
# When using mmperfmon to configure ZIMon in a GPFS cluster, then the
211+
# collectors are chosen based on the colRedundancy setting and the
212+
# given set of collector nodes passed to mmperfmon.
213+
collectors = {
214+
host = ""
215+
port = "4739"
216+
}

‎source/gpfsConfig/mmsdrfs.template

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
###############################################################################
2+
#
3+
# The mmsdrfs file is kept in the sdr and contains global information
4+
# pertaining to a given GPFS cluster.
5+
#
6+
#
7+
# The following is an example of a mmsdrfs file:
8+
#
9+
# %%9999%%:00_VERSION_LINE::1:3:78::lc:c154n02.ibm.com:c154n03.ibm.com:1000:: ...
10+
# %%9999%%:03_COMMENT::1:
11+
# %%9999%%:03_COMMENT::2: This is a machine generated file. Do not edit!
12+
# %%9999%%:03_COMMENT::3:
13+
# %%home%%:10_NODESET_HDR:::3:TCP:500:1191:complete:6668::700:700:AIX:
14+
# %%home%%:20_MEMBER_NODE::1:1:c54n01:9.114.54.21:c54n01.ibm.com:manager::-1:0:
15+
# %%home%%:20_MEMBER_NODE::2:3:c54n03:9.114.54.23:c54n03.ibm.com:client::-1:2:
16+
# %%home%%:20_MEMBER_NODE::3:2:c54n02:9.114.54.22:c54n02.ibm.com:manager::-1:1:
17+
# %%home%%:30_SG_HEADR:gpfsXYZ::152::
18+
# %%home%%:40_SG_ETCFS:gpfsXYZ:1:/mmfs/gpfsXYZ:
19+
# %%home%%:40_SG_ETCFS:gpfsXYZ:2: dev = /dev/gpfsXYZ
20+
# %%home%%:40_SG_ETCFS:gpfsXYZ:3: vfs = mmfs
21+
# %%home%%:40_SG_ETCFS:gpfsXYZ:4: nodename = -
22+
# %%home%%:40_SG_ETCFS:gpfsXYZ:5: mount = false
23+
# %%home%%:40_SG_ETCFS:gpfsXYZ:6: type = mmfs
24+
# %%home%%:40_SG_ETCFS:gpfsXYZ:7: account = false
25+
# %%home%%:50_SG_MOUNT:gpfsXYZ::rw:mtime:atime:userquota;groupquota:
26+
# %%home%%:60_SG_DISKS:gpfsXYZ:1:mmfslv01:4288:4004:dataAndMetadata:
27+
# %%home%%:60_SG_DISKS:gpfsXYZ:2:mmfslv02:4288:4005:dataOnly
28+
#
29+
###############################################################################

0 commit comments

Comments
 (0)