Skip to content

Commit cdbf84d

Browse files
committed
Tweaks
1 parent db2e1b7 commit cdbf84d

File tree

2 files changed

+36
-22
lines changed

2 files changed

+36
-22
lines changed

samples/speedystore/README.md

Lines changed: 35 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -3,44 +3,41 @@
33
## Overview
44
This sample demonstrates how to extend SAS Viya Monitoring for Kubernetes to monitor the SingleStore instance embedded within SAS SpeedyStore. This allows administrators to monitor their SingleStore cluster using the same Grafana instance they use to monitor the rest of their SAS Viya deployment.
55

6-
This sample is derived from the blog post Using [Grafana dashboards for monitoring SAS SpeedyStore](https://communities.sas.com/t5/SAS-Communities-Library/Using-Grafana-dashboards-for-monitoring-SAS-SpeedyStore/ta-p/973178) written by Michael Goddard from SAS Education. The blog post was, in turn, based on work Michael did in preparing to cover the topic as part of the [SAS® SpeedyStore: Architect and Deploy the SAS® Viya® Platform with SingleStore](https://learn.sas.com/course/view.php?id=6393) workshop available in [learn.sas.com](https://learn.sas.com/). This sample includes Grafana dashboards developed and made available by SingleStore.
6+
This sample is derived from the blog post [Using Grafana dashboards for monitoring SAS SpeedyStore](https://communities.sas.com/t5/SAS-Communities-Library/Using-Grafana-dashboards-for-monitoring-SAS-SpeedyStore/ta-p/973178) written by Michael Goddard from SAS Education. The blog post was, in turn, based on work Michael did in preparing to cover the topic as part of the [SAS® SpeedyStore: Architect and Deploy the SAS® Viya® Platform with SingleStore](https://learn.sas.com/course/view.php?id=6393) workshop available in [learn.sas.com](https://learn.sas.com/). This sample includes Grafana dashboards developed and made available by SingleStore.
77

88
**Note: There may be other ways to achieve these same objectives, this sample documents one possible approach.**
99

1010
## Using this Sample
1111
Enabling this monitoring will required configuring components in both SingleStore and Grafana. While we will describe how to configure the SingleStore components, you are strongly encouraged to review the official SingleStore documentation for a more comprehensive discussion of how to monitor SingleStore effectively, the options for doing so and additional implementation details. After configuring the SingleStore components, this sample covers defining the datasource within Grafana and deploying the SingleStore-specific Grafana dashboards.
1212

13-
The diagram below. taken from the [SingleStore documentation](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/), provides a high-level overview of metric collection and reporting process. As shown, the SingleStore pipeline extracts metrics from the SingleStore cluster and stores them within a metrics database. Grafana pulls metric data directly from this metrics database to populate the dashboards administrators view.
13+
The diagram below, taken from the [SingleStore documentation](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/), provides a high-level overview of metric collection and reporting process. As shown, the SingleStore pipeline extracts metrics from the SingleStore cluster and stores them within a metrics database. Grafana pulls metric data directly from this metrics database to populate the dashboards administrators view.
1414

1515
![Overview of SingleStore Monitoring Process ](images/cluster_monitoring_hl_architecture-mfjdOm.webp)
1616

1717
### Overview of Process
18-
* Set-up
19-
* Set the VIYA_NS environent variable
2018
* SingleStore
21-
* Configure the SingleStore pipeline
22-
* Configure metrics database
23-
* Configure "monitoring user"
19+
* Configure the SingleStore pipeline and metrics database
20+
* Configure the "monitoring user"
2421
* Grafana
25-
* Configure Grafana
26-
* Load SingleStoer Grafana Dashboards
22+
* Configure the Grafana Datasource
23+
* Import the SingleStore Dashboards into Grafana
2724

2825
### Configure the SingleStore pipeline and metrics database
2926

30-
The SingleStore Toolbox is used to deploy, administer, and manage a SingleStore cluster. We will use the `sdb-admin start-monitoring-kube` command to configure and start the monitoring. It has a number of flags to control its operations. See the SingleStore documentation for more information: [start-monitoring-kube](https://docs.singlestore.com/db/v8.9/reference/singlestore-tools-reference/sdb-admin-commands/start-monitoring-kube/).
27+
The SingleStore Toolbox is used to deploy, administer, and manage a SingleStore cluster. We will use the `sdb-admin start-monitoring-kube` command to configure and start the monitoring. It has a number of flags to control its operations. See the [SingleStore documentation](https://docs.singlestore.com/db/v8.9/reference/singlestore-tools-reference/sdb-admin-commands/start-monitoring-kube/) for more information.
3128

3229

33-
To configure and start the monitoring, including the metrics database, we will submit the following command:
30+
To configure and start the monitoring, including the metrics database, we will (eventually) submit the following command:
3431

3532
`sdb-admin start-monitoring-kube --cluster-name sas-singlestore-cluster --namespace {VIYA_NS} --user root --password {ROOT_PWD} --exporter-host {CLUSTER_MASTER_IP}`
3633

37-
But before submitting the command, we will review the parameters being passed to the command.
34+
But, before submitting the command, we will review the various parameters being passed to the command and how to determine their proper values.
3835

3936
#### Set the VIYA_NS environent variable
4037

41-
Since you will need to refer repeatedly to the namespace in which SAS Viya (and SingleStore) is deployed, it is helpful to define an environment variable, `VIYA_NS`,to identify the namespace and reference it in subsequent commands.
38+
Since you will need to refer repeatedly to the namespace in which SAS Viya (and SingleStore) is deployed, it is helpful to define an environment variable, `VIYA_NS`, to identify the namespace and reference it in subsequent commands.
4239

43-
You can use the following command to do this:
40+
In the following command, we set the `VIYA_NS` environment variable to ***myviya*** the namespace containing our SAS Viya deployment:
4441

4542
`export VIYA_NS=myviya`
4643

@@ -49,22 +46,26 @@ The default name for the SingleStore cluster in a SAS SpeedyStore deployment is:
4946

5047
`show global variables like 'cluster_name%';`
5148

49+
If the cluster name is different than the default, be sure to use the correct value in subsequent commands.
50+
5251
#### The `user` and `password` parameters
53-
A core part of the monitoring is the exporter process which collects the metric data from the cluster. The exporter process is typically run as the SingleStore 'root' user due to the permissions required. In addition, we will need the password for the SingleStore 'root' user. Note: It is possible to run the process as another user but the user must have the low level permissions needed to create and control the metrics database and pipelines. Setting up an alternate user is out-of-scope for this sample and we will use the 'root' user.
52+
A core part of the monitoring is the exporter process which collects the metric data from the cluster. The exporter process is typically run as the SingleStore 'root' user due to the permissions required. In addition, we will need the password for the SingleStore 'root' user.
53+
54+
NOTE: It is possible to run the process as another user but the user must have the low level permissions needed to create and control the metrics database and pipelines. Setting up an alternate user is out-of-scope for this sample and we will use the 'root' user.
5455

5556
You can use the following command to get the password for the 'root' user:
5657

5758
`ROOT_PWD=$(kubectl -n ${VIYA_NS} get secret sas-singlestore-cluster -o yaml | grep "ROOT_PASSWORD"|awk '{print $2}'|base64 -d --wrap=0)`
5859

5960
#### The `exporter-host` parameter
60-
As shown in the diagram above, the export process runs on the Master Aggregator. Therefore, you need to target the SingleStore Master node; i.e. the **node-sas-singlestore-cluster-master-0** node (pod) in a SAS SpeedyStore deployment. You need to provide the fully-qualified host name or IP address for the exporter host. The name needs to be resolvable by the host running the `sdb-admin` command. Since the fully-qualified host name may not be resolvable, we will use the IP address for the `exporter-host` parameter instead.
61+
As shown in the diagram above, the export process runs on the Master Aggregator. Therefore, you need to target the SingleStore Master node; i.e. the **node-sas-singlestore-cluster-master-0** node (pod) in a SAS SpeedyStore deployment. You will need to provide the fully-qualified host name or IP address for the exporter host. And the name needs to be resolvable by the host running the `sdb-admin` command. Since the fully-qualified host name may not be resolvable, we will use the IP address for the `exporter-host` parameter instead.
6162

6263
You can obtain the IP address for the Master node by submitting the following command:
6364

6465
`CLUSTER_MASTER_IP=$(kubectl -n ${NS} get pods -o wide | grep 'node-sas-singlestore-cluster-master-0' | awk '{print $6}')`
6566

6667
#### Accessing the Kubernetess Cluster
67-
The `sb-admin` command needs to access the Kubernetes cluster on which SAS Viya and SingleStore are running. It does this through a Kubernetes configuration file. By default, the command will use the file defined in the KUBECONFIG environment variable or the ~/.kube/config file are used to discover the cluster. Alternatively, the `--config-file` option can be used to specify the kube config.
68+
The `sb-admin` command needs to access the Kubernetes cluster on which SAS Viya and SingleStore are running. It does this through a Kubernetes configuration file. By default, the command will use the file identified in the `KUBECONFIG` environment variable or the `~/.kube/config` file are used to discover the cluster. Alternatively, the `--config-file` option can be used to specify the kube config.
6869

6970
#### Run the `sb-admin start-monitoring-kube` command
7071
After setting all of the required parameters, submit the following command to configure and start the monitoring, including the metrics database:
@@ -74,7 +75,7 @@ After setting all of the required parameters, submit the following command to co
7475
After running the command, the exporter process, the pipeline and the metrics database are created. To confirm this, you can use the SingleStore Studio. For example, in the screenshot below, you can see the newly created **'metrics'** database:
7576
![Screenshot showing SingleStore Studio with the 'metrics' database highlighted](images/02_MG_202508_metrics-database.png)
7677

77-
### Configure "monitoring user"
78+
### Configure the "monitoring user"
7879
Grafana will need to connect to the 'metrics' database and you should create a specific user to be used for that purpose. While the permissions required by this user to pull metrics from the database are fairly limited, it can be helpful to grant additional permissions so the user can be used to manage the metrics database, pipelines and the exporter process.
7980

8081
For example, you might grant the following permissions:
@@ -91,10 +92,20 @@ Alternatively, you might GRANT the "monitoring user" minimal permissions, and cr
9192

9293
|**TO DO: Is the user name *S2MonitorUser*?**
9394

94-
### Configure Grafana Datasource
95+
### Configure the Grafana Datasource
9596
Grafana datasources provide connection information allowing Grafana to access metric information in response to user queries and to populate dashboards.
9697

97-
The file [speedystore-datasource.yaml](speedystore-datasource.yaml) in this directory defines the datasource that will allow Grafana to access the 'metrics' database created above. However, before it can be used, it needs to be edited to provide the proper credentials (i.e. the ***user*** and ***password*** fields in the file). If the name of the SingleStore cluster is not ***sas-singlestore-cluster***, you will need to update the ***url*** field in the file as well.
98+
The file [speedystore-datasource.yaml](speedystore-datasource.yaml) in this directory defines the datasource that will allow Grafana to access the 'metrics' database created above. However, before it can be used, it needs to be edited to provide the proper credentials (i.e. the ***user*** and ***password*** fields in the file). You will also need to update the ***url*** field to reflect the namespace in which SAS Viya deployment is deployed.
99+
100+
For example, if SAS Viya is deployed into the ***myviya*** namespace, you would revise the ***url*** value from:
101+
102+
`url: svc-sas-singlestore-cluster-ddl.VIYA_NS.svc.cluster.local:3306`
103+
104+
to:
105+
106+
`url: svc-sas-singlestore-cluster-ddl.myviya.svc.cluster.local:3306`
107+
108+
If the name of the SingleStore cluster is not ***sas-singlestore-cluster***, you will need to update that portion of the ***url*** field in the file as well.
98109

99110
Copy the file to some location, update the necessary information and save your changes. We suggest copying the file into your `$USER_DIR/monitoring` sub-directory, i.e. the same directory used for any other customizations related to the metric monitoring components you have made to your deployment of SAS Viya Monitoring.
100111

@@ -108,7 +119,9 @@ You can use the following command to apply the necessary label:
108119

109120
`kubectl -n $VIYA_NS label secret grafana-metrics-connection "grafana_datasource=1"`
110121

111-
### Import SingleStore Dashboards
122+
|**TO DO**: Is the service-name always _name_of_singlestore_cluster_ + "-dll"
123+
124+
### Import the SingleStore Dashboards into Grafana
112125
To import the SingleStore dashboards into Grafana, you can use the `deploy_dashboards.sh` script found in the `monitoring/bin` sub-directory of this repository.
113126

114127
You can use the following command to import all of the SingleStore dashboards:
@@ -136,6 +149,7 @@ To validate the configuration, sign into Grafana and review each of the SingleSt
136149
* [Resource Pool Monitoring](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/view-the-dashboards/#resource-pool-monitoring)
137150

138151
## Acknowledgements
152+
Thank you to Michael Goddard (SAS Education) for all of his work sorting this out and allowing us to share it here.
139153

140154
## References
141155
[SingleStore Documentation: Configure Monitoring](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/)

samples/speedystore/speedystore-datasource.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ apiVersion: 1
33
datasources:
44
- name: monitoring
55
type: mysql
6-
url: svc-sas-singlestore-cluster-ddl.viya.svc.cluster.local:3306
6+
url: svc-sas-singlestore-cluster-ddl.VIYA_NS.svc.cluster.local:3306
77
database: metrics
88
user: S2MonitorUser
99
secureJsonData:

0 commit comments

Comments
 (0)