You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: samples/speedystore/README.md
+25-22Lines changed: 25 additions & 22 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,12 +3,12 @@
3
3
## Overview
4
4
This sample demonstrates how to extend SAS Viya Monitoring for Kubernetes to monitor the SingleStore instance embedded within SAS SpeedyStore. This allows administrators to monitor their SingleStore cluster using the same Grafana instance they use to monitor the rest of their SAS Viya deployment.
5
5
6
-
This sample is derived from the blog post [Using Grafana dashboards for monitoring SAS SpeedyStore](https://communities.sas.com/t5/SAS-Communities-Library/Using-Grafana-dashboards-for-monitoring-SAS-SpeedyStore/ta-p/973178) written by Michael Goddard from SAS Education. The blog post was, in turn, based on work Michael did in preparing to cover the topic as part of the [SAS® SpeedyStore: Architect and Deploy the SAS® Viya® Platform with SingleStore](https://learn.sas.com/course/view.php?id=6393) workshop available in [learn.sas.com](https://learn.sas.com/). This sample includes Grafana dashboards developed and made available by SingleStore.
6
+
This sample is primarily derived from the blog post [Using Grafana dashboards for monitoring SAS SpeedyStore](https://communities.sas.com/t5/SAS-Communities-Library/Using-Grafana-dashboards-for-monitoring-SAS-SpeedyStore/ta-p/973178) written by Michael Goddard from SAS Education. The blog post was, in turn, based on work Michael did in preparing to cover the topic as part of the [SAS® SpeedyStore: Architect and Deploy the SAS® Viya® Platform with SingleStore](https://learn.sas.com/course/view.php?id=6393) workshop available in [learn.sas.com](https://learn.sas.com/). This sample includes Grafana dashboards developed and made available by SingleStore.
7
7
8
8
**Note: There may be other ways to achieve these same objectives, this sample documents one possible approach.**
9
9
10
10
## Using this Sample
11
-
Enabling this monitoring will required configuring components in both SingleStore and Grafana. While we will describe how to configure the SingleStore components, you are strongly encouraged to review the official SingleStore documentation for a more comprehensive discussion of how to monitor SingleStore effectively, the options for doing so and additional implementation details. After configuring the SingleStore components, this sample covers defining the datasource within Grafana and deploying the SingleStore-specific Grafana dashboards.
11
+
Enabling this monitoring will required configuring components in both SingleStore and Grafana. While this sample describes how to configure the SingleStore components, you are strongly encouraged to review the official SingleStore documentation for a more comprehensive discussion of how to monitor SingleStore effectively, the options for doing so and additional implementation details. After configuring the SingleStore components, this sample covers defining the datasource within Grafana and deploying the SingleStore-specific Grafana dashboards.
12
12
13
13
The diagram below, taken from the [SingleStore documentation](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/), provides a high-level overview of metric collection and reporting process. As shown, the SingleStore pipeline extracts metrics from the SingleStore cluster and stores them within a metrics database. Grafana pulls metric data directly from this metrics database to populate the dashboards administrators view.
14
14
@@ -17,31 +17,33 @@ The diagram below, taken from the [SingleStore documentation](https://docs.singl
17
17
### Overview of Process
18
18
* Configure SingleStore
19
19
* Create the SingleStore pipeline and metrics database
20
-
* Create the "S2MonitorUser"
20
+
* Create the "S2MonitorUser" user
21
21
* Configure Grafana
22
-
* Create the Datasource
23
-
* Import the SingleStore Dashboards
22
+
* Create the datasource
23
+
* Import the SingleStore dashboards
24
24
25
25
### Create the SingleStore pipeline and metrics database
26
26
27
-
The SingleStore Toolbox is used to deploy, administer, and manage a SingleStore cluster. We will use the `sdb-admin start-monitoring-kube` command to configure and start the monitoring. It has a number of flags to control its operations. See the [SingleStore documentation](https://docs.singlestore.com/db/v8.9/reference/singlestore-tools-reference/sdb-admin-commands/start-monitoring-kube/) for more information.
27
+
The SingleStore Toolbox is used to deploy, administer, and manage a SingleStore cluster. The `sdb-admin start-monitoring-kube` command is used to configure and start the monitoring. It has a number of flags to control its operations. See the [SingleStore documentation](https://docs.singlestore.com/db/v8.9/reference/singlestore-tools-reference/sdb-admin-commands/start-monitoring-kube/) for more information.
28
28
29
-
To configure and start the monitoring, including the metrics database, we will (eventually) submit the following command:
29
+
To configure and start the monitoring, including the metrics database, the following command will (eventually) be submitted:
But, before submitting the command, we will review the various parameters being passed to the command and how to determine their proper values.
33
+
But, before submitting the command, the various parameters being passed to the command and how to determine their proper values will be reviewed.
34
34
35
35
#### Set the VIYA_NS environent variable
36
36
37
37
Since you will need to refer repeatedly to the namespace in which SAS Viya (and SingleStore) is deployed, it is helpful to define an environment variable, `VIYA_NS`, to identify the namespace and reference it in subsequent commands.
38
38
39
-
In the following command, we set the `VIYA_NS` environment variable to ***myviya***the namespace containing our SAS Viya deployment:
39
+
In the following command, the `VIYA_NS` environment variable is defined and assigned the value identifying the namespace containing our SAS Viya deployment (i.e. ***myviya*** ):
40
40
41
41
`export VIYA_NS=myviya`
42
42
43
43
#### The `cluster-name` parameter
44
-
The default name for the SingleStore cluster in a SAS SpeedyStore deployment is: ***sas-singlestore-cluster***. However, it is possible to change this name, so it is important to confirm the actual cluster name before configuring the monitroing. To get the cluster name, submit the following command via the SingleStore CLI:
44
+
The default name for the SingleStore cluster in a SAS SpeedyStore deployment is: ***sas-singlestore-cluster***. However, since it is possible to change this name, it is important to confirm the actual cluster name before configuring the monitoring.
45
+
46
+
To get the cluster name, submit the following command via the SingleStore CLI:
45
47
46
48
`show global variables like 'cluster_name%';`
47
49
@@ -52,19 +54,19 @@ A core part of the monitoring is the exporter process which collects the metric
52
54
53
55
NOTE: It is possible to run the process as another user but the user must have the low level permissions needed to create and control the metrics database and pipelines. Setting up an alternate user is out-of-scope for this sample and we will use the 'root' user.
54
56
55
-
In addition, we will need the password for the SingleStore 'root' user. You can use the following command to get the password for the 'root' user:
57
+
You will need the password for the SingleStore 'root' user. You can use the following command to get the password for the 'root' user and store it in the `ROOT_PWD` environment variable:
As shown in the diagram above, the export process runs on the Master Aggregator. Therefore, you need to target the SingleStore Master node; i.e. the **node-sas-singlestore-cluster-master-0** node (pod) in a SAS SpeedyStore deployment. You will need to provide the fully-qualified host name or IP address for the exporter host. And the name needs to be resolvable by the host running the `sdb-admin` command. Since the fully-qualified host name may not be resolvable, we will use the IP address for the `exporter-host` parameter instead.
61
63
62
-
You can obtain the IP address for the Master node by submitting the following command:
64
+
You can obtain the IP address for the Master node and stored it in the `CLUSTER_MASTER_IP` environment variable by submitting the following command:
By default, the `sdb-admin start-monitoring-kube` command will display some information and ask the user if they would like to continue. To skip this prompt and have the configuration continue automatically, we will incle the `--yes` parameter.
69
+
By default, the `sdb-admin start-monitoring-kube` command will display some information and ask the user if they would like to continue. To skip this prompt and have the configuration continue automatically, include the `--yes` parameter.
68
70
69
71
#### Accessing the Kubernetess Cluster
70
72
The `sb-admin` command needs to access the Kubernetes cluster on which SAS Viya and SingleStore are running. It does this through a Kubernetes configuration file. By default, the command will use the file identified in the `KUBECONFIG` environment variable or the `~/.kube/config` file are used to discover the cluster. Alternatively, the `--config-file` option can be used to specify the kube config file.
Once completed, the exporter process, the pipeline and the metrics database have been created. To confirm this, you can use the SingleStore Studio. For example, in the screenshot below, you can see the newly created **'metrics'** database:
95
+
Once completed, the exporter process, the pipeline and the metrics database have been created. You can use the SingleStore Studio, to confirm this. For example, in the screenshot below, you can see the newly created **'metrics'** database:
94
96

95
97
96
98
### Create the "S2MonitorUser"
97
-
You need to create a specific user that Grafana can use to connect to the 'metrics' database. While the permissions required by this user to pull metrics from the database are fairly limited, it can be helpful to grant additional permissions so this same user can be used to manage the metrics database, pipelines and the exporter process.
99
+
You need to create a specific user that Grafana can use to connect to the ***'metrics'*** database. While the permissions required by this user to pull metrics from the database are fairly limited, it can be helpful to grant additional permissions so this same user can be used to manage the metrics database, pipelines and the exporter process.
98
100
99
101
After logging into SingleStore with the admin user, you can submit `CREATE USER` and `GRANT` commands to create the user and grant the user the desired permissions.
100
102
101
-
For example, the following command creates a user called `S2MonitorUser` and assigns it a password:
103
+
For example, the following command creates a user called `S2MonitorUser` and sets its password:
102
104
103
105
`CREATE USER S2MonitorUser IDENTIFIED BY 'password123' REQUIRE NONE;`
104
106
105
-
Then you might grant the following permissions:
107
+
Then you can grant the desired permissions by submitting the following commands:
106
108
107
109
`GRANT CLUSTER, SHOW METADATA, SELECT, PROCESS ON *.* to 'S2MonitorUser'@'%';`
108
110
109
111
`GRANT SELECT, CREATE, INSERT, UPDATE, DELETE, EXECUTE, INDEX, ALTER, DROP, CREATE DATABASE, LOCK TABLES, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, CREATE PIPELINE, DROP PIPELINE, ALTER PIPELINE, START PIPELINE, SHOW PIPELINE ON metrics.* to 'S2MonitorUser'@'%';`
110
112
111
-
Alternatively, you might GRANT the "monitoring user" minimal permissions, and create a second user ("monitoring administrator") to manage the metrics database, pipelines and the exporter process.
113
+
Alternatively, you could GRANT the "monitoring user" minimal permissions, and create a second user ("monitoring administrator") to manage the metrics database, pipelines and the exporter process.
112
114
113
115
|**TO DO: Does the first GRANT statement only grant the _minimal_ permissions needed to pull metirics? And the 2nd grants the extended permissions needed to be an administrator?**
114
116
@@ -117,7 +119,7 @@ Alternatively, you might GRANT the "monitoring user" minimal permissions, and cr
117
119
### Configure the Grafana Datasource
118
120
Grafana datasources provide connection information allowing Grafana to access metric information in response to user queries and to populate dashboards.
119
121
120
-
The file [speedystore-datasource.yaml](speedystore-datasource.yaml) in this directory defines the datasource that will allow Grafana to access the 'metrics' database created above. However, before it can be used, it needs to be edited to provide the proper credentials (i.e. the ***user*** and ***password*** fields in the file). You will also need to update the ***url*** field to reflect the namespace in which SAS Viya deployment is deployed.
122
+
The file [speedystore-datasource.yaml](speedystore-datasource.yaml) in this directory defines the datasource that will allow Grafana to access the ***'metrics'*** database created above. However, before it can be used, it needs to be edited to provide the proper credentials (i.e. the ***user*** and ***password*** fields in the file). You will also need to update the ***url*** field to reflect the namespace in which SAS Viya deployment is deployed.
121
123
122
124
For example, if SAS Viya is deployed into the ***myviya*** namespace, you would revise the ***url*** value from:
123
125
@@ -131,7 +133,7 @@ to:
131
133
132
134
If the name of the SingleStore cluster is not ***sas-singlestore-cluster***, you will need to update that portion of the ***url*** field in the file as well.
133
135
134
-
Copy the file to some location, update the necessary information and save your changes. We suggest copying the file into your `$USER_DIR/monitoring` sub-directory, i.e. the same directory used for any other customizations related to the metric monitoring components you have made to your deployment of SAS Viya Monitoring.
136
+
Copy the file to some location, update the necessary information and save your changes. We suggest copying the file into your `$USER_DIR/monitoring` sub-directory, i.e. the same directory used for any other customizations related to the metric monitoring components you have made to your deployment of SAS Viya Monitoring. This will ensure all of the files related to this deployment of SAS Viya Monitoring are in one place.
135
137
136
138
Then submit the following command to create the datasource:
137
139
@@ -151,16 +153,17 @@ You can use the following command to import all of the SingleStore dashboards:
Or, if you can import specific dashboards individually using the same script. For example, following command imports the ***Cluster View*** dashboard into Grafana:
156
+
Or, you can import specific dashboards individually using the same script. For example, following command imports the ***Cluster View*** dashboard into Grafana:
Once the dashboards have been imported into Grafana, you should be all set to monitor the SingleStore instance embedded in SAS SpeedyStore.
161
+
Once the dashboards have been imported into Grafana, you should be ready to monitor the SingleStore instance embedded in SAS SpeedyStore.
160
162
161
163
To validate the configuration, sign into Grafana and review each of the SingleStore dashboards you've imported. All of the imported dashboards have the ***"sas-speedystore"*** and ***"singlestore"*** tags. While the data shown will vary based on user activity, all of the dashboards should be available with no errors or warning icons or messages.
162
164
163
165
## The Grafana Dashboards
166
+
This is a list of the SingleStore dashboards included in this sample. Each dashboard name in the list is a link to more information about the dashboard in the SingleStore documentation including the metrics depicted and the types of questions they address.
*[Detailed Cluster View by Node](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/view-the-dashboards/#detailed-cluster-view-by-node)
0 commit comments