Skip to content

Commit e11ef1d

Browse files
committed
Final (?) tweaks
1 parent 470d653 commit e11ef1d

File tree

2 files changed

+44
-28
lines changed

2 files changed

+44
-28
lines changed

samples/speedystore/README.md

Lines changed: 42 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -15,32 +15,47 @@ The diagram below, taken from the [SingleStore documentation](https://docs.singl
1515
![Overview of SingleStore Monitoring Process ](images/cluster_monitoring_hl_architecture-mfjdOm.webp)
1616

1717
### Overview of Process
18+
* Prerequisites and Set-Up
1819
* Configure SingleStore
1920
* Create the SingleStore pipeline and metrics database
2021
* Create the "S2MonitorUser" user
2122
* Configure Grafana
2223
* Create the datasource
2324
* Import the SingleStore dashboards
2425

25-
### Create the SingleStore pipeline and metrics database
26+
### Prerequisites and Set-Up
27+
#### Prerequisites
28+
* SingleStore Toolbox
2629

27-
The SingleStore Toolbox is used to deploy, administer, and manage a SingleStore cluster. The `sdb-admin start-monitoring-kube` command is used to configure and start the monitoring. It has a number of flags to control its operations. See the [SingleStore documentation](https://docs.singlestore.com/db/v8.9/reference/singlestore-tools-reference/sdb-admin-commands/start-monitoring-kube/) for more information.
30+
You will need to use the SingleStore Toolbox package to deploy, administer, and manage a SingleStore cluster. [Installation instructions for SingleStore Toolbox](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-management-with-tools/singlestore-tools-installation/singlestore-toolbox-installation/) are available on the SingleStore website.
2831

29-
>**NOTE: The SingleStore Toolbox must be installed on a host that can access the SingleStore cluster pods using the internal cluster hostnames and IP addresses. This can be a jump box (bastion) host or, if necessary, one of the Kubernetes cluster nodes. In addition to the SingleStore Toolbox, you must be able to submit `kubectl` commands from this host as well.**
32+
* SingleStore CLI or SingleStore Studio
3033

31-
To configure and start the monitoring, including the metrics database, the following command will (eventually) be submitted:
34+
You will need to use submit database commands via the SingleStore CLI to create a database user and grant them the appropriate permissions. [Installation instructions for SingleStore CLI](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-management-with-tools/singlestore-tools-installation/singlestore-client-installation/) are available on the SingleStore website.
3235

33-
`sdb-admin start-monitoring-kube --cluster-name sas-singlestore-cluster --namespace $VIYA_NS --user root --password $ROOT_PWD --exporter-host $CLUSTER_MASTER_IP`
36+
Or, alternatively, you can submit the necessary commands via the SingleStore Studio SQL Editor panel.
37+
* `kubectl`
3438

35-
But, before submitting the command, the various parameters being passed to the command and how to determine their proper values will be reviewed.
39+
You will need to use `kubectl` commands to obtain information about the your SAS Viya and SingleStore deployments.
3640

3741
#### Set the VIYA_NS environent variable
3842

3943
Since you will need to refer repeatedly to the namespace in which SAS Viya (and SingleStore) is deployed, it is helpful to define an environment variable, `VIYA_NS`, to identify the namespace and reference it in subsequent commands.
4044

4145
In the following command, the `VIYA_NS` environment variable is defined and assigned the value identifying the namespace containing our SAS Viya deployment (i.e. ***myviya*** ):
4246

43-
`export VIYA_NS=myviya`
47+
`VIYA_NS=myviya`
48+
49+
### Configure SingleStore
50+
#### Create the SingleStore pipeline and metrics database
51+
The `sdb-admin start-monitoring-kube` command is used to configure and start the monitoring. It has a number of flags to control its operations. See the [SingleStore documentation](https://docs.singlestore.com/db/v8.9/reference/singlestore-tools-reference/sdb-admin-commands/start-monitoring-kube/) for more information.
52+
53+
To configure and start the monitoring, including the metrics database, the following command will (eventually) be submitted:
54+
55+
`sdb-admin start-monitoring-kube --cluster-name sas-singlestore-cluster --namespace $VIYA_NS --user root --password $ROOT_PWD --exporter-host $CLUSTER_MASTER_IP`
56+
57+
But, before submitting the command, the various parameters being passed to the command and how to determine their proper values will be reviewed.
58+
4459

4560
#### The `cluster-name` parameter
4661
The default name for the SingleStore cluster in a SAS SpeedyStore deployment is: ***sas-singlestore-cluster***. However, since it is possible to change this name, it is important to confirm the actual cluster name before configuring the monitoring.
@@ -54,14 +69,14 @@ If the cluster name is different than the default, be sure to use the correct va
5469
#### The `user` and `password` parameters
5570
A core part of the monitoring is the exporter process which collects the metric data from the cluster. The exporter process is typically run as the SingleStore 'root' user due to the permissions required.
5671

57-
NOTE: It is possible to run the process as another user but the user must have the low level permissions needed to create and control the metrics database and pipelines. Setting up an alternate user is out-of-scope for this sample and we will use the 'root' user.
72+
NOTE: It is possible to run the process as another user but the user must have the low level permissions needed to create and control the metrics database and pipelines. Setting up an alternate user for this is out-of-scope for this sample and we will use the 'root' user.
5873

5974
You will need the password for the SingleStore 'root' user. You can use the following command to get the password for the 'root' user and store it in the `ROOT_PWD` environment variable:
6075

6176
`ROOT_PWD=$(kubectl -n ${VIYA_NS} get secret sas-singlestore-cluster -o yaml | grep "ROOT_PASSWORD"|awk '{print $2}'|base64 -d --wrap=0)`
6277

6378
#### The `exporter-host` parameter
64-
As shown in the diagram above, the export process runs on the Master Aggregator. Therefore, you need to target the SingleStore Master node; i.e. the **node-sas-singlestore-cluster-master-0** node (pod) in a SAS SpeedyStore deployment. You will need to provide the fully-qualified host name or IP address for the exporter host. And, as noted above, the IP address or host name needs to be resolvable by the host running the `sdb-admin` command. This example uses an IP address for the `exporter-host` parameter.
79+
As shown in the diagram above, the export process runs on the Master Aggregator. Therefore, you need to target the SingleStore Master node; i.e. the **node-sas-singlestore-cluster-master-0** node (pod) in a SAS SpeedyStore deployment. This example you will use that pod's IP address for the `exporter-host` parameter.
6580

6681
You can obtain the IP address for the Master node and stored it in the `CLUSTER_MASTER_IP` environment variable by submitting the following command:
6782

@@ -94,21 +109,19 @@ Automatically selected yes, non-interactive mode enabled
94109
Operation completed successfully
95110
```
96111

97-
Once completed, the exporter process, the pipeline and the metrics database have been created. You can use the SingleStore Studio, to confirm this. For example, in the screenshot below, you can see the newly created **'metrics'** database:
112+
Once completed, the exporter process, the pipeline and the '**metrics**' database have been created. You can use the SingleStore Studio, to confirm this. For example, in the screenshot below, you can see the newly created `**metrics**' database:
98113
![Screenshot showing SingleStore Studio with the 'metrics' database highlighted](images/02_MG_202508_metrics-database.png)
99114

100115
### Create the "S2MonitorUser"
101-
You need to create a specific user that Grafana can use to connect to the ***'metrics'*** database.
102-
103-
After logging into SingleStore with the admin user, you can submit `CREATE USER` and `GRANT` commands to create the user and grant the user the desired permissions.
116+
Next, you need to create a specific user that Grafana can use to connect to the '**metrics**' database. After logging into SingleStore with the admin user, you can submit the `CREATE USER` and `GRANT` commands via the SingleStore CLI (or, from the SQL Editor within SingleStore Studio) to create the user and grant the user the desired permissions.
104117

105118
For example, the following command creates a user called `S2MonitorUser` and sets its password:
106119

107120
`CREATE USER S2MonitorUser IDENTIFIED BY 'password123' REQUIRE NONE;`
108121

109122
The following command grants a minimal set of permissions that allows the newly created user to access the collected metrics and populate the Grafana dashboards:
110123

111-
`GRANT SELECT, SHOW VIEW ON metrics.* to 'S2MonitorUser'@'%';`
124+
`GRANT SELECT, SHOW VIEW, EXECUTE ON metrics.* TO 'S2MonitorUser'@'%'`
112125

113126
Alternatively, while the limited permissions above are sufficient to pull metrics from the database, it can be helpful to grant additional permissions so this same user can be used to manage the metrics database, pipelines and the exporter process.
114127

@@ -118,16 +131,16 @@ The following commands grant a broader set of permissions to allow the monitorin
118131

119132
`GRANT SELECT, CREATE, INSERT, UPDATE, DELETE, EXECUTE, INDEX, ALTER, DROP, CREATE DATABASE, LOCK TABLES, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, CREATE PIPELINE, DROP PIPELINE, ALTER PIPELINE, START PIPELINE, SHOW PIPELINE ON metrics.* to 'S2MonitorUser'@'%';`
120133

121-
You should consider your organization's specific needs before deciding whether to grant the more limited or broader set of permissions to this user.
122-
123-
### Configure the Grafana Datasource
134+
You should consider your organization's specific needs before deciding whether to grant the more limited or the broader set of permissions to this user.
135+
### Configure Grafana
136+
#### Configure the Grafana Datasource
124137
Grafana datasources provide connection information allowing Grafana to access metric information in response to user queries and to populate dashboards.
125138

126-
The file [speedystore-datasource.yaml](speedystore-datasource.yaml) in this directory defines the datasource that will allow Grafana to access the ***'metrics'*** database created above. However, before it can be used, it needs to be edited to provide the proper credentials (i.e. the ***user*** and ***password*** fields in the file). You will also need to update the ***url*** field to reflect the namespace in which SAS Viya deployment is deployed.
139+
The file [speedystore-datasource.yaml](speedystore-datasource.yaml) in this directory defines the datasource that will allow Grafana to access the '**metrics**' database created above. However, before it can be used, it needs to be edited to provide the proper credentials (i.e. the ***user*** and ***password*** fields in the file). You will also need to update the ***url*** field to reflect the namespace in which SAS Viya deployment is deployed.
127140

128141
For example, if SAS Viya is deployed into the ***myviya*** namespace, you would revise the ***url*** value from:
129142

130-
`url: svc-sas-singlestore-cluster-ddl.VIYA_NS.svc.cluster.local:3306`
143+
`svc-sas-singlestore-cluster-ddl.{**Replace with SAS VIYA Namespace**}.svc.cluster.local:3306`
131144

132145
to:
133146

@@ -139,18 +152,18 @@ Copy the file to some location, update the necessary information and save your c
139152

140153
Then submit the following command to create the datasource:
141154

142-
`kubectl -n monitoring create secret generic grafana-metrics-connection --from-file=$USER_DIR/monitoring/speedystore-datasource.yaml`
155+
`kubectl -n monitoring create secret generic grafana-speedystore-connection --from-file=$USER_DIR/monitoring/speedystore-datasource.yaml`
143156

144157
NOTE: This command assumes the metric monitoring components (including Grafana) have been deployed into the `monitoring` namespace. If they are deployed in a different namespace, update the command to reference to correct namespace.
145158

146-
After secret has been created, you need to apply a specific label to the secret to trigger the automatic provisioning (Ioading) of the datasource into Grafana.
159+
After secret has been created, you need to apply a specific label to the secret to trigger the automatic provisioning (loading) of the datasource into Grafana.
147160

148161
You can use the following command to apply the necessary label:
149162

150-
`kubectl -n monitoring label secret grafana-metrics-connection "grafana_datasource=1"`
163+
`kubectl -n monitoring label secret grafana-speedystore-connection "grafana_datasource=1"`
151164

152165

153-
### Import the SingleStore Dashboards into Grafana
166+
#### Import the SingleStore Dashboards into Grafana
154167
To import the SingleStore dashboards into Grafana, you can use the `deploy_dashboards.sh` script found in the `monitoring/bin` sub-directory of this repository.
155168

156169
You can use the following command to import all of the SingleStore dashboards:
@@ -162,9 +175,9 @@ Or, you can import specific dashboards individually using the same script. For
162175
`./monitoring/bin/deploy_dashboards.sh samples/speedystore/dashboards/clusterview.yaml`
163176

164177
### Validate
165-
Once the dashboards have been imported into Grafana, you should be ready to monitor the SingleStore instance embedded in SAS SpeedyStore.
178+
Once the dashboards have been imported into Grafana, you should be all set to monitor the SingleStore instance embedded in SAS SpeedyStore.
166179

167-
To validate the configuration, sign into Grafana and review each of the SingleStore dashboards you've imported. All of the imported dashboards have the ***"sas-speedystore"*** and ***"singlestore"*** tags. While the data shown will vary based on user activity, all of the dashboards should be available with no errors or warning icons or messages.
180+
To validate the configuration, sign into Grafana and review the SingleStore dashboards you've imported. All of the imported dashboards have the ***"sas-speedystore"*** and ***"singlestore"*** tags. While the data shown will vary based on user activity, all of the dashboards should be available with no errors or warning icons or messages.
168181

169182
## The Grafana Dashboards
170183
This is a list of the SingleStore dashboards included in this sample. Each dashboard name in the list is a link to more information about the dashboard in the SingleStore documentation including the metrics depicted and the types of questions they address.
@@ -182,8 +195,11 @@ This is a list of the SingleStore dashboards included in this sample. Each das
182195
Thank you to Michael Goddard (SAS Education) for all of his work sorting this out and allowing us to share it here.
183196

184197
## References
198+
[Blog Post: Using Grafana dashboards for monitoring SAS SpeedyStore](https://communities.sas.com/t5/SAS-Communities-Library/Using-Grafana-dashboards-for-monitoring-SAS-SpeedyStore/ta-p/973178)
199+
200+
[Workshop: SAS® SpeedyStore: Architect and Deploy the SAS® Viya® Platform with SingleStore](https://learn.sas.com/course/view.php?id=6393)
201+
185202
[SingleStore Documentation: Configure Monitoring](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/)
186203

187204
[SingleStore Documentation: Detailed Discussion of each dashboard including metrics shown and when to use them](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/view-the-dashboards/)
188205

189-

samples/speedystore/speedystore-datasource.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,10 @@ prune: true
55
datasources:
66
- name: s2monitoring
77
type: mysql
8-
url: svc-sas-singlestore-cluster-ddl.VIYA_NS.svc.cluster.local:3306
8+
url: svc-sas-singlestore-cluster-ddl.{**Replace with SAS VIYA Namespace**}.svc.cluster.local:3306
99
database: metrics
1010
user: S2MonitorUser
1111
secureJsonData:
12-
password: {Monitoring_User_Password}
12+
password: {**Replace with S2MonitorUser Password**}
1313
isDefault: false
1414
version: 1

0 commit comments

Comments
 (0)