You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: samples/speedystore/README.md
+42-26Lines changed: 42 additions & 26 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -15,32 +15,47 @@ The diagram below, taken from the [SingleStore documentation](https://docs.singl
15
15

16
16
17
17
### Overview of Process
18
+
* Prerequisites and Set-Up
18
19
* Configure SingleStore
19
20
* Create the SingleStore pipeline and metrics database
20
21
* Create the "S2MonitorUser" user
21
22
* Configure Grafana
22
23
* Create the datasource
23
24
* Import the SingleStore dashboards
24
25
25
-
### Create the SingleStore pipeline and metrics database
26
+
### Prerequisites and Set-Up
27
+
#### Prerequisites
28
+
* SingleStore Toolbox
26
29
27
-
The SingleStore Toolbox is used to deploy, administer, and manage a SingleStore cluster. The `sdb-admin start-monitoring-kube` command is used to configure and start the monitoring. It has a number of flags to control its operations. See the [SingleStore documentation](https://docs.singlestore.com/db/v8.9/reference/singlestore-tools-reference/sdb-admin-commands/start-monitoring-kube/)for more information.
30
+
You will need to use the SingleStore Toolbox package to deploy, administer, and manage a SingleStore cluster. [Installation instructions for SingleStore Toolbox](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-management-with-tools/singlestore-tools-installation/singlestore-toolbox-installation/)are available on the SingleStore website.
28
31
29
-
>**NOTE: The SingleStore Toolbox must be installed on a host that can access the SingleStore cluster pods using the internal cluster hostnames and IP addresses. This can be a jump box (bastion) host or, if necessary, one of the Kubernetes cluster nodes. In addition to the SingleStore Toolbox, you must be able to submit `kubectl` commands from this host as well.**
32
+
*SingleStore CLI or SingleStore Studio
30
33
31
-
To configure and start the monitoring, including the metrics database, the following command will (eventually) be submitted:
34
+
You will need to use submit database commands via the SingleStore CLI to create a database user and grant them the appropriate permissions. [Installation instructions for SingleStore CLI](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-management-with-tools/singlestore-tools-installation/singlestore-client-installation/) are available on the SingleStore website.
Or, alternatively, you can submit the necessary commands via the SingleStore Studio SQL Editor panel.
37
+
*`kubectl`
34
38
35
-
But, before submitting the command, the various parameters being passed to the command and how to determine their proper values will be reviewed.
39
+
You will need to use `kubectl` commands to obtain information about the your SAS Viya and SingleStore deployments.
36
40
37
41
#### Set the VIYA_NS environent variable
38
42
39
43
Since you will need to refer repeatedly to the namespace in which SAS Viya (and SingleStore) is deployed, it is helpful to define an environment variable, `VIYA_NS`, to identify the namespace and reference it in subsequent commands.
40
44
41
45
In the following command, the `VIYA_NS` environment variable is defined and assigned the value identifying the namespace containing our SAS Viya deployment (i.e. ***myviya*** ):
42
46
43
-
`export VIYA_NS=myviya`
47
+
`VIYA_NS=myviya`
48
+
49
+
### Configure SingleStore
50
+
#### Create the SingleStore pipeline and metrics database
51
+
The `sdb-admin start-monitoring-kube` command is used to configure and start the monitoring. It has a number of flags to control its operations. See the [SingleStore documentation](https://docs.singlestore.com/db/v8.9/reference/singlestore-tools-reference/sdb-admin-commands/start-monitoring-kube/) for more information.
52
+
53
+
To configure and start the monitoring, including the metrics database, the following command will (eventually) be submitted:
But, before submitting the command, the various parameters being passed to the command and how to determine their proper values will be reviewed.
58
+
44
59
45
60
#### The `cluster-name` parameter
46
61
The default name for the SingleStore cluster in a SAS SpeedyStore deployment is: ***sas-singlestore-cluster***. However, since it is possible to change this name, it is important to confirm the actual cluster name before configuring the monitoring.
@@ -54,14 +69,14 @@ If the cluster name is different than the default, be sure to use the correct va
54
69
#### The `user` and `password` parameters
55
70
A core part of the monitoring is the exporter process which collects the metric data from the cluster. The exporter process is typically run as the SingleStore 'root' user due to the permissions required.
56
71
57
-
NOTE: It is possible to run the process as another user but the user must have the low level permissions needed to create and control the metrics database and pipelines. Setting up an alternate user is out-of-scope for this sample and we will use the 'root' user.
72
+
NOTE: It is possible to run the process as another user but the user must have the low level permissions needed to create and control the metrics database and pipelines. Setting up an alternate user for this is out-of-scope for this sample and we will use the 'root' user.
58
73
59
74
You will need the password for the SingleStore 'root' user. You can use the following command to get the password for the 'root' user and store it in the `ROOT_PWD` environment variable:
As shown in the diagram above, the export process runs on the Master Aggregator. Therefore, you need to target the SingleStore Master node; i.e. the **node-sas-singlestore-cluster-master-0** node (pod) in a SAS SpeedyStore deployment. You will need to provide the fully-qualified host name or IP address for the exporter host. And, as noted above, the IP address or host name needs to be resolvable by the host running the `sdb-admin` command. This example uses an IP address for the `exporter-host` parameter.
79
+
As shown in the diagram above, the export process runs on the Master Aggregator. Therefore, you need to target the SingleStore Master node; i.e. the **node-sas-singlestore-cluster-master-0** node (pod) in a SAS SpeedyStore deployment. This example you will use that pod's IP address for the `exporter-host` parameter.
65
80
66
81
You can obtain the IP address for the Master node and stored it in the `CLUSTER_MASTER_IP` environment variable by submitting the following command:
Once completed, the exporter process, the pipeline and the metrics database have been created. You can use the SingleStore Studio, to confirm this. For example, in the screenshot below, you can see the newly created **'metrics'** database:
112
+
Once completed, the exporter process, the pipeline and the '**metrics**' database have been created. You can use the SingleStore Studio, to confirm this. For example, in the screenshot below, you can see the newly created `**metrics**' database:
98
113

99
114
100
115
### Create the "S2MonitorUser"
101
-
You need to create a specific user that Grafana can use to connect to the ***'metrics'*** database.
102
-
103
-
After logging into SingleStore with the admin user, you can submit `CREATE USER` and `GRANT` commands to create the user and grant the user the desired permissions.
116
+
Next, you need to create a specific user that Grafana can use to connect to the '**metrics**' database. After logging into SingleStore with the admin user, you can submit the `CREATE USER` and `GRANT` commands via the SingleStore CLI (or, from the SQL Editor within SingleStore Studio) to create the user and grant the user the desired permissions.
104
117
105
118
For example, the following command creates a user called `S2MonitorUser` and sets its password:
106
119
107
120
`CREATE USER S2MonitorUser IDENTIFIED BY 'password123' REQUIRE NONE;`
108
121
109
122
The following command grants a minimal set of permissions that allows the newly created user to access the collected metrics and populate the Grafana dashboards:
110
123
111
-
`GRANT SELECT, SHOW VIEWON metrics.* to 'S2MonitorUser'@'%';`
124
+
`GRANT SELECT, SHOW VIEW, EXECUTE ON metrics.* TO 'S2MonitorUser'@'%'`
112
125
113
126
Alternatively, while the limited permissions above are sufficient to pull metrics from the database, it can be helpful to grant additional permissions so this same user can be used to manage the metrics database, pipelines and the exporter process.
114
127
@@ -118,16 +131,16 @@ The following commands grant a broader set of permissions to allow the monitorin
118
131
119
132
`GRANT SELECT, CREATE, INSERT, UPDATE, DELETE, EXECUTE, INDEX, ALTER, DROP, CREATE DATABASE, LOCK TABLES, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, CREATE PIPELINE, DROP PIPELINE, ALTER PIPELINE, START PIPELINE, SHOW PIPELINE ON metrics.* to 'S2MonitorUser'@'%';`
120
133
121
-
You should consider your organization's specific needs before deciding whether to grant the more limited or broader set of permissions to this user.
122
-
123
-
### Configure the Grafana Datasource
134
+
You should consider your organization's specific needs before deciding whether to grant the more limited or the broader set of permissions to this user.
135
+
### Configure Grafana
136
+
####Configure the Grafana Datasource
124
137
Grafana datasources provide connection information allowing Grafana to access metric information in response to user queries and to populate dashboards.
125
138
126
-
The file [speedystore-datasource.yaml](speedystore-datasource.yaml) in this directory defines the datasource that will allow Grafana to access the ***'metrics'*** database created above. However, before it can be used, it needs to be edited to provide the proper credentials (i.e. the ***user*** and ***password*** fields in the file). You will also need to update the ***url*** field to reflect the namespace in which SAS Viya deployment is deployed.
139
+
The file [speedystore-datasource.yaml](speedystore-datasource.yaml) in this directory defines the datasource that will allow Grafana to access the '**metrics**' database created above. However, before it can be used, it needs to be edited to provide the proper credentials (i.e. the ***user*** and ***password*** fields in the file). You will also need to update the ***url*** field to reflect the namespace in which SAS Viya deployment is deployed.
127
140
128
141
For example, if SAS Viya is deployed into the ***myviya*** namespace, you would revise the ***url*** value from:
NOTE: This command assumes the metric monitoring components (including Grafana) have been deployed into the `monitoring` namespace. If they are deployed in a different namespace, update the command to reference to correct namespace.
145
158
146
-
After secret has been created, you need to apply a specific label to the secret to trigger the automatic provisioning (Ioading) of the datasource into Grafana.
159
+
After secret has been created, you need to apply a specific label to the secret to trigger the automatic provisioning (loading) of the datasource into Grafana.
147
160
148
161
You can use the following command to apply the necessary label:
### Import the SingleStore Dashboards into Grafana
166
+
####Import the SingleStore Dashboards into Grafana
154
167
To import the SingleStore dashboards into Grafana, you can use the `deploy_dashboards.sh` script found in the `monitoring/bin` sub-directory of this repository.
155
168
156
169
You can use the following command to import all of the SingleStore dashboards:
@@ -162,9 +175,9 @@ Or, you can import specific dashboards individually using the same script. For
Once the dashboards have been imported into Grafana, you should be ready to monitor the SingleStore instance embedded in SAS SpeedyStore.
178
+
Once the dashboards have been imported into Grafana, you should be all set to monitor the SingleStore instance embedded in SAS SpeedyStore.
166
179
167
-
To validate the configuration, sign into Grafana and review each of the SingleStore dashboards you've imported. All of the imported dashboards have the ***"sas-speedystore"*** and ***"singlestore"*** tags. While the data shown will vary based on user activity, all of the dashboards should be available with no errors or warning icons or messages.
180
+
To validate the configuration, sign into Grafana and review the SingleStore dashboards you've imported. All of the imported dashboards have the ***"sas-speedystore"*** and ***"singlestore"*** tags. While the data shown will vary based on user activity, all of the dashboards should be available with no errors or warning icons or messages.
168
181
169
182
## The Grafana Dashboards
170
183
This is a list of the SingleStore dashboards included in this sample. Each dashboard name in the list is a link to more information about the dashboard in the SingleStore documentation including the metrics depicted and the types of questions they address.
@@ -182,8 +195,11 @@ This is a list of the SingleStore dashboards included in this sample. Each das
182
195
Thank you to Michael Goddard (SAS Education) for all of his work sorting this out and allowing us to share it here.
183
196
184
197
## References
198
+
[Blog Post: Using Grafana dashboards for monitoring SAS SpeedyStore](https://communities.sas.com/t5/SAS-Communities-Library/Using-Grafana-dashboards-for-monitoring-SAS-SpeedyStore/ta-p/973178)
199
+
200
+
[Workshop: SAS® SpeedyStore: Architect and Deploy the SAS® Viya® Platform with SingleStore](https://learn.sas.com/course/view.php?id=6393)
[SingleStore Documentation: Detailed Discussion of each dashboard including metrics shown and when to use them](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/view-the-dashboards/)
0 commit comments