diff --git a/samples/speedystore/README.md b/samples/speedystore/README.md new file mode 100644 index 00000000..6a214c78 --- /dev/null +++ b/samples/speedystore/README.md @@ -0,0 +1,205 @@ + +# Monitoring SAS SpeedyStore (SingleStore) +## Overview +This sample demonstrates how to extend SAS Viya Monitoring for Kubernetes to monitor the SingleStore instance embedded within SAS SpeedyStore. This allows administrators to monitor their SingleStore cluster using the same Grafana instance they use to monitor the rest of their SAS Viya deployment. + +This sample is primarily derived from the blog post [Using Grafana dashboards for monitoring SAS SpeedyStore](https://communities.sas.com/t5/SAS-Communities-Library/Using-Grafana-dashboards-for-monitoring-SAS-SpeedyStore/ta-p/973178) written by Michael Goddard from SAS Education. The blog post was, in turn, based on work Michael did in preparing to cover the topic as part of the [SAS® SpeedyStore: Architect and Deploy the SAS® Viya® Platform with SingleStore](https://learn.sas.com/course/view.php?id=6393) workshop available in [learn.sas.com](https://learn.sas.com/). This sample includes Grafana dashboards developed and made available by SingleStore. + +**Note: This sample documents one possible approach, there might be other ways to achieve the same objectives.** + +## Using this Sample +Enabling this monitoring will required configuring components in both SingleStore and Grafana. While this sample describes how to configure the SingleStore components, you are strongly encouraged to review the official SingleStore documentation for a more comprehensive discussion of how to monitor SingleStore effectively, the options for doing so, and additional implementation details. After configuring the SingleStore components, this sample covers defining the datasource within Grafana and deploying the SingleStore-specific Grafana dashboards. + +The diagram below, taken from the [SingleStore documentation](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/), provides a high-level overview of metric collection and reporting process. As shown, the SingleStore pipeline extracts metrics from the SingleStore cluster and stores them within a metrics database. Grafana pulls metric data directly from this metrics database to populate the dashboards administrators view. + +![Overview of SingleStore Monitoring Process ](images/cluster_monitoring_hl_architecture-mfjdOm.webp) + +### Overview of Process +* Prerequisites and Set-Up +* Configure SingleStore + * Create the SingleStore pipeline and metrics database + * Create the "S2MonitorUser" user +* Configure Grafana + * Create the datasource + * Import the SingleStore dashboards + +### Prerequisites and Set-Up +#### Prerequisites +* SingleStore Toolbox + +You will need to use the SingleStore Toolbox package to deploy, administer, and manage a SingleStore cluster. [Installation instructions for SingleStore Toolbox](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-management-with-tools/singlestore-tools-installation/singlestore-toolbox-installation/) are available on the SingleStore website. + +* SingleStore CLI or SingleStore Studio + +You will need to use submit database commands via the SingleStore CLI to create a database user and grant them the appropriate permissions. [Installation instructions for SingleStore CLI](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-management-with-tools/singlestore-tools-installation/singlestore-client-installation/) are available on the SingleStore website. + +Or, alternatively, you can submit the necessary commands via the SingleStore Studio SQL Editor panel. +* `kubectl` + +You will need to use `kubectl` commands to obtain information about your SAS Viya and SingleStore deployments. + +#### Set the VIYA_NS environent variable + +Since you will need to refer repeatedly to the namespace in which SAS Viya (and SingleStore) is deployed, it is helpful to define an environment variable, `VIYA_NS`, to identify the namespace and reference it in subsequent commands. + +In the following command, the `VIYA_NS` environment variable is defined and assigned the value identifying the namespace containing our SAS Viya deployment (i.e. ***myviya*** ): + + `VIYA_NS=myviya` + +### Configure SingleStore +#### Create the SingleStore pipeline and metrics database +The `sdb-admin start-monitoring-kube` command is used to configure and start the monitoring. It has a number of flags to control its operations. See the [SingleStore documentation](https://docs.singlestore.com/db/v8.9/reference/singlestore-tools-reference/sdb-admin-commands/start-monitoring-kube/) for more information. + +Before submitting the `sdb-admin start-monitoring-kube` command, the various parameters being passed to the command and how to determine their proper values will be reviewed. + +#### The `cluster-name` parameter +The default name for the SingleStore cluster in a SAS SpeedyStore deployment is: ***sas-singlestore-cluster***. However, since it is possible to change this name, it is important to confirm the actual cluster name before configuring the monitoring. + +To get the cluster name, submit the following command via the SingleStore CLI: + + `show global variables like 'cluster_name%';` + +If the cluster name is different than the default, be sure to use the correct value in subsequent commands. + +#### The `user` and `password` parameters +A core part of the monitoring is the exporter process which collects the metric data from the cluster. The exporter process is typically run as the SingleStore 'root' user due to the permissions required. + +NOTE: It is possible to run the process as another user but the user must have the low level permissions needed to create and control the metrics database and pipelines. Setting up an alternate user for this is out-of-scope for this sample and we will use the 'root' user. + +You need the password for the SingleStore 'root' user. You can use the following command to get the password for the 'root' user and store it in the `ROOT_PWD` environment variable: + +`ROOT_PWD=$(kubectl -n ${VIYA_NS} get secret sas-singlestore-cluster -o yaml | grep "ROOT_PASSWORD"|awk '{print $2}'|base64 -d --wrap=0)` + +#### The `exporter-host` parameter +As shown in the diagram above, the export process runs on the Master Aggregator. Therefore, you need to target the SingleStore Master node; i.e. the **node-sas-singlestore-cluster-master-0** node (pod) in a SAS SpeedyStore deployment. In this example, that pod's IP address will be used for the `exporter-host` parameter. + +You can obtain the IP address for the Master node and store it in the `CLUSTER_MASTER_IP` environment variable by submitting the following command: + +`CLUSTER_MASTER_IP=$(kubectl -n ${VIYA_NS} get pods -o wide | grep 'node-sas-singlestore-cluster-master-0' | awk '{print $6}')` + +#### Disabling interactive mode +By default, the `sdb-admin start-monitoring-kube` command displays some information and asks the user if they would like to continue. To skip this prompt and have the configuration continue automatically, this example includes the `--yes` parameter. + +#### Accessing the Kubernetess Cluster +The `sb-admin` command needs to access the Kubernetes cluster on which SAS Viya and SingleStore are running. It does this through a Kubernetes configuration file. By default, the command uses the file identified in the `KUBECONFIG` environment variable (if defined), or the `~/.kube/config` file (if it exists) to discover the cluster. Alternatively, the `--config-file` option can be used to specify the kube config file to use. + +#### Run the `sb-admin start-monitoring-kube` command +After setting all of the required parameters, submit the following command to configure and start the monitoring, including the metrics database: + +`sdb-admin start-monitoring-kube --cluster-name sas-singlestore-cluster --namespace $VIYA_NS --user root --password $ROOT_PWD --exporter-host $CLUSTER_MASTER_IP --yes` + +After running the command, you should see output similar to the following: + +``` +✓ Successfully discovered resources +Toolbox will perform the following actions: + · create a database metrics with tables and procedures for monitoring + · set retention period to 7 days for 192.168.0.164:9104 + · set purge frequency to 60 minutes for 192.168.0.164:9104 + · set purge log retention period to 365 days + +Would you like to continue? [Y/n]: +Automatically selected yes, non-interactive mode enabled + +Operation completed successfully +``` + +Once completed, the exporter process, the pipeline, and the '**metrics**' database have been created. You can use the SingleStore Studio, to confirm this. For example, in the screenshot below, you can see the newly created `**metrics**' database: +![Screenshot showing SingleStore Studio with the 'metrics' database highlighted](images/02_MG_202508_metrics-database.png) + +### Create the "S2MonitorUser" user +Next, you need to create a specific user that Grafana can use to connect to the '**metrics**' database. After logging into SingleStore with the admin user, you can submit the `CREATE USER` and `GRANT` commands via the SingleStore CLI (or, from the SQL Editor within SingleStore Studio) to create the user and grant the user the desired permissions. + +For example, the following command creates a user called `S2MonitorUser` and sets its password: + +`CREATE USER S2MonitorUser IDENTIFIED BY 'password123' REQUIRE NONE;` + +The following command grants a minimal set of permissions that allows the newly created user to access the collected metrics and populate the Grafana dashboards: + +`GRANT SELECT, SHOW VIEW, EXECUTE ON metrics.* TO 'S2MonitorUser'@'%'` + +Alternatively, while the limited permissions above are sufficient to pull metrics from the database, it can be helpful to grant additional permissions so this same user can be used to manage the metrics database, pipelines, and the exporter process. + +The following commands grant a broader set of permissions to allow the monitoring user to perform these administrator duties: + +`GRANT CLUSTER, SHOW METADATA, SELECT, PROCESS ON *.* to 'S2MonitorUser'@'%';` + +`GRANT SELECT, CREATE, INSERT, UPDATE, DELETE, EXECUTE, INDEX, ALTER, DROP, CREATE DATABASE, LOCK TABLES, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, CREATE PIPELINE, DROP PIPELINE, ALTER PIPELINE, START PIPELINE, SHOW PIPELINE ON metrics.* to 'S2MonitorUser'@'%';` + +You should consider your organization's specific needs before deciding whether to grant the more limited or the broader set of permissions to this user. +### Configure Grafana +#### Configure the Grafana Datasource +Grafana datasources provide connection information allowing Grafana to access metric information in response to user queries and to populate dashboards. + +The file [speedystore-datasource.yaml](speedystore-datasource.yaml) in this directory defines the datasource that will allow Grafana to access the '**metrics**' database created above. However, before it can be used, it needs to be edited to provide the proper credentials (i.e. the ***user*** and ***password*** fields in the file). You also need to update the ***url*** field to reflect the namespace in which SAS Viya deployment is deployed. + +For example, if SAS Viya is deployed into the ***myviya*** namespace, you would revise the ***url*** value from: + +`svc-sas-singlestore-cluster-ddl.{**Replace with SAS VIYA Namespace**}.svc.cluster.local:3306` + +to: + +`url: svc-sas-singlestore-cluster-ddl.myviya.svc.cluster.local:3306` + + If the name of the SingleStore cluster is not ***sas-singlestore-cluster***, you need to update that portion of the ***url*** field in the file as well. + +Copy the file to some location, update the necessary information, and save your changes. We suggest copying the file into your `$USER_DIR/monitoring` sub-directory, i.e. the same directory used for any other customizations related to the metric monitoring components you have made to your deployment of SAS Viya Monitoring. This will ensure all of the files related to this deployment of SAS Viya Monitoring are in one place. + +Then submit the following command to create the datasource: + +`kubectl -n monitoring create secret generic grafana-speedystore-connection --from-file=$USER_DIR/monitoring/speedystore-datasource.yaml` + +NOTE: This command assumes the metric monitoring components (including Grafana) have been deployed into the `monitoring` namespace. If they are deployed in a different namespace, update the command to reference the correct namespace. + +After the secret has been created, you need to apply a specific label to the secret to trigger the automatic provisioning (loading) of the datasource into Grafana. + +You can use the following command to apply the necessary label: + +`kubectl -n monitoring label secret grafana-speedystore-connection "grafana_datasource=1"` + + +#### Import the SingleStore Dashboards into Grafana +To import the SingleStore dashboards into Grafana, you can use the `deploy_dashboards.sh` script found in the `monitoring/bin` sub-directory of this repository. + +You can use the following command to import all of the SingleStore dashboards: + +`./monitoring/bin/deploy_dashboards.sh samples/speedystore/dashboards` + +Or, you can import specific dashboards individually using the same script. For example, following command imports the ***Cluster View*** dashboard into Grafana: + +`./monitoring/bin/deploy_dashboards.sh samples/speedystore/dashboards/clusterview.yaml` + +### Validate everything is working +Once the dashboards have been imported into Grafana, you should be all set to monitor the SingleStore instance embedded in SAS SpeedyStore. + +To validate the configuration, sign into Grafana and review the SingleStore dashboards you've imported. All of the imported dashboards have the ***"sas-speedystore"*** and ***"singlestore"*** tags. While the data shown will vary based on user activity, all of the dashboards should be available with no errors or warning icons or messages. + +## The Grafana Dashboards +This is a list of the SingleStore dashboards included in this sample. Each dashboard name in the list is a link to more information about the dashboard in the SingleStore documentation including the metrics depicted and the types of questions they address. + +* [Cluster View](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/view-the-dashboards/#cluster-view) +* [Detailed Cluster View by Node](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/view-the-dashboards/#detailed-cluster-view-by-node) +* [Disk Usage](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/view-the-dashboards/#disk-usage) +* [Historical Workload Monitoring](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/view-the-dashboards/#historical-workload-monitoring) +* [Memory Usage](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/view-the-dashboards/#memory-usage) +* [Pipeline Summary](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/view-the-dashboards/#pipeline-summary) +* [Pipeline Performance](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/view-the-dashboards/#pipeline-performance) +* [Query History](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/view-the-dashboards/#query-history) +* [Resource Pool Monitoring](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/view-the-dashboards/#resource-pool-monitoring) + +NOTE: The following modifications have been made to the original SingleStore dashboards definitions: +* the json files were renamed to use all lowercase and remove spaces; +* the name of the datasource was changed from `monitoring` to `s2monitoring`; and, +* the tags `sas-speedystore` and `singlestore` were added to the dashboards. +## Acknowledgements +Thank you to Michael Goddard (SAS Education) for all of his work sorting this out and allowing us to share it here. + +## References +[Blog Post: Using Grafana dashboards for monitoring SAS SpeedyStore](https://communities.sas.com/t5/SAS-Communities-Library/Using-Grafana-dashboards-for-monitoring-SAS-SpeedyStore/ta-p/973178) + +[Workshop: SAS® SpeedyStore: Architect and Deploy the SAS® Viya® Platform with SingleStore](https://learn.sas.com/course/view.php?id=6393) + +[SingleStore Documentation: Configure Monitoring](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/) + +[SingleStore Documentation: Detailed Discussion of each dashboard including metrics shown and when to use them](https://docs.singlestore.com/db/v8.9/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/view-the-dashboards/) + diff --git a/samples/speedystore/dashboards/clusterview.json b/samples/speedystore/dashboards/clusterview.json new file mode 100644 index 00000000..2d681deb --- /dev/null +++ b/samples/speedystore/dashboards/clusterview.json @@ -0,0 +1,2449 @@ +{ + "annotations": { + "list": [ + { + "builtIn": 1, + "datasource": { + "type": "datasource", + "uid": "grafana" + }, + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "limit": 100, + "name": "Annotations & Alerts", + "showIn": 0, + "target": { + "limit": 100, + "matchAny": false, + "tags": [], + "type": "dashboard" + }, + "type": "dashboard" + } + ] + }, + "editable": true, + "fiscalYearStartMonth": 0, + "graphTooltip": 1, + "id": 261, + "links": [ + { + "asDropdown": false, + "icon": "external link", + "includeVars": false, + "keepTime": false, + "tags": [], + "targetBlank": true, + "title": "Documentation", + "tooltip": "", + "type": "link", + "url": "https://docs.singlestore.com/db/v8.1/en/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/view-the-dashboards.html#cluster-view-793226" + }, + { + "asDropdown": true, + "icon": "external link", + "includeVars": true, + "keepTime": true, + "tags": [], + "targetBlank": true, + "title": "Dashboards", + "type": "dashboards" + } + ], + "panels": [ + { + "collapsed": false, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 0 + }, + "id": 45, + "panels": [], + "title": "Resource Utilization", + "type": "row" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "CPU", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "decimals": 1, + "mappings": [], + "max": 100, + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "percent" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "max single-core load" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "green", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "min single-core load" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "blue", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 0, + "y": 1 + }, + "id": 20, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true, + "sortBy": "Last *", + "sortDesc": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "hide": false, + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t1 AS (WITH t AS (\n SELECT *, lag(value) OVER w value_l, lag(time_sec) OVER w time_sec_l\n FROM\n (SELECT\n-- take the last datapoint in the interval\ntime_sec as time_sec,\nmemsql_sysinfo_cpu_used_ms/memsql_sysinfo_cpu_num_cpus value,\nhost\n FROM\n (SELECT\n max(time_sec) time_sec,\n -- this is the last counter in the time group\n max(intval) as value,\n name as metric,\n trim_host(host) host,\n $__unixEpochGroup(time_sec, $interval) as tg\n FROM metrics\n WHERE\n (cluster = '$cluster')\n and (host in ($host) or $host = '%')\n and (extractor = 'memsql')\n and (subsystem = 'sysinfo')\n and (name = 'memsql_sysinfo_cpu_used_ms' or name = 'memsql_sysinfo_cpu_num_cpus')\n and $__unixEpochFilter(time_sec)\n group by 3, 4, 5\n ) as pvt_data\n PIVOT\n (\n AVG(value)\n FOR metric\n IN ('memsql_sysinfo_cpu_used_ms', 'memsql_sysinfo_cpu_num_cpus')\n ) as pvt_table\n ) X\n WINDOW w AS (partition by host order by time_sec desc)\n)\nselect a.time_sec, (value - value_l)/(1000*(time_sec - time_sec_l)) * 100 value, a.host from t a\nWHERE time_sec_l is not null and value_l is not null\norder by 1, 2 desc, 3)\nSELECT time_sec, min(value) as `min single-core load`, avg(value) 'avg core load', max(value) as `max single-core load` from t1\ngroup by time_sec\norder by time_sec\n", + "refId": "B", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "CPU Utilization", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "Memory", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "max": 1, + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "percentunit" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "Memory Usage" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "green", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 8, + "y": 1 + }, + "id": 22, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "select time_sec, 'Memory Usage' as metric, max(used) from (\nselect time_sec,\nhost,\nmemsql_sysinfo_mem_used_b / memsql_sysinfo_mem_total_b as used\nFROM\n(SELECT\nmax(time_sec) time_sec,\navg(value) value,\nmetric,\nhost\nFROM\n(SELECT\n time_sec,\n $__unixEpochGroup(time_sec,$interval) tg,\n intval as value,\n name as metric,\n CONCAT('{', trim_host(host), '}') as host\nFROM metrics\nWHERE cluster='$cluster'\nand (host in ($host) or $host = '%')\nand extractor = 'memsql'\nand subsystem = 'sysinfo'\nand name in ('memsql_sysinfo_mem_used_b','memsql_sysinfo_mem_total_b')\nand $__unixEpochFilter(time_sec)\n) X\nGROUP BY tg, metric, host\n) as pvt_data\nPIVOT (\n AVG(value) \n FOR metric \n IN ('memsql_sysinfo_mem_used_b','memsql_sysinfo_mem_total_b')\n) as pvt_table) a\ngroup by time_sec\norder by time_sec", + "refId": "B", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Memory Utilization", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "Disk Utilization", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "max": 1, + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "percentunit" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 16, + "y": 1 + }, + "id": 51, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "select time_sec, 'Disk Usage' as metric, max(used) from (\nselect time_sec,\nhost,\nmemsql_sysinfo_disk_used_b / memsql_sysinfo_disk_total_b as used\nFROM\n(SELECT\nmax(time_sec) time_sec,\navg(value) value,\nmetric,\nhost\nFROM\n(SELECT\n time_sec,\n $__unixEpochGroup(time_sec,$interval) tg,\n intval as value,\n name as metric,\n CONCAT('{', trim_host(host), '}') as host\nFROM metrics\nWHERE cluster='$cluster'\nand (host in ($host) or $host = '%')\nand extractor = 'memsql'\nand subsystem = 'sysinfo'\nand name in ('memsql_sysinfo_disk_used_b','memsql_sysinfo_disk_total_b')\nand $__unixEpochFilter(time_sec)\n) X\nGROUP BY tg, metric, host\n) as pvt_data\nPIVOT (\n AVG(value) \n FOR metric \n IN ('memsql_sysinfo_disk_used_b','memsql_sysinfo_disk_total_b')\n) as pvt_table) a\ngroup by time_sec\norder by time_sec", + "refId": "B", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Disk Utilization", + "type": "timeseries" + }, + { + "collapsed": false, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 9 + }, + "id": 44, + "panels": [], + "title": "Connections & Threads", + "type": "row" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "Number of open connections (threads_connected) to the database relative to the maximum limit (max_connections). If the utilization is near 100%, it means DB is approaching the maximum allowed connections; this can potentially lead to performance issues, as queries may need to wait in a queue until a thread becomes available to process them.", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "Threads - Connected", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "decimals": 1, + "mappings": [], + "max": 100, + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "percent" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 10 + }, + "id": 46, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "table", + "hide": false, + "rawQuery": true, + "rawSql": "select max(used_c / total_C) * 100 as `Threads - Connected`, \n tg as time \nfrom (SELECT time_sec as time, \n $__unixEpochGroup(time_sec, $interval) as tg, \n sum(case when name = 'memsql_variables_max_connections' then value else 0 end) as total_c,\n sum(case when name = 'memsql_status_threads_connected' then value else 0 end) as used_c,\n host, port\n FROM metrics\n WHERE $__unixEpochFilter(time_sec) AND\n (cluster = '$cluster') AND\n NAME in ('memsql_variables_max_connections','memsql_status_threads_connected') \n group by time, tg, host, port)\ngroup by tg\norder by tg;", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Threads - Connected", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "Number of threads actively running queries (threads_running) relative to the maximum limit (max_connection_threads). If the utilization is near 100%, it suggests that the system might be reaching its capacity in terms of how many queries can be executed in parallel. This situation can lead to resource pressure, unresponsiveness, latency spikes, and even failures", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "Threads - Running", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "decimals": 1, + "mappings": [], + "max": 100, + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "percent" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 10 + }, + "id": 47, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "table", + "hide": false, + "rawQuery": true, + "rawSql": "select tg as time,\n max(used_c / total_C) * 100 as 'Threads - Running:' \nfrom (SELECT time_sec as time, \n $__unixEpochGroup(time_sec, $interval) as tg, \n sum(case when name = 'memsql_variables_max_connection_threads' then value else 0 end) as total_c,\n sum(case when name = 'memsql_status_threads_running' then value else 0 end) as used_c,\n host, port\n FROM metrics\n WHERE $__unixEpochFilter(time_sec) AND\n (cluster = '$cluster') AND\n NAME in ('memsql_variables_max_connection_threads','memsql_status_threads_running') \n group by time, tg, host, port)\ngroup by tg\norder by tg;", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Threads - Running", + "type": "timeseries" + }, + { + "collapsed": false, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 18 + }, + "id": 43, + "panels": [], + "title": "Throughput", + "type": "row" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "wps" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "Write queries per second" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "green", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 7, + "w": 12, + "x": 0, + "y": 19 + }, + "id": 36, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "hide": false, + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\n SELECT \n *, \n lag(value) OVER w value_l, \n lag(time_sec) OVER w time_sec_l \n FROM \n (\n SELECT \n max(time_sec) as time_sec, \n avg(value) as value, \n metric \n FROM \n (\n SELECT \n time_sec, \n $__unixEpochGroup(time_sec, $interval) as tg, \n intval as value, \n concat(name, host) as metric \n FROM \n `metrics` \n WHERE \n $__unixEpochFilter(time_sec) \n and (cluster = '$cluster') \n and (host in ($host) or $host = '%')\n and extractor = 'memsql' \n and subsystem = 'status' \n and name like 'memsql_status_%_write_queries'\n and role <> 'LEAF') subq \n GROUP BY \n metric, \n tg\n ) X WINDOW w AS (\n partition by metric \n order by \n time_sec\n )\n) \nSELECT \n time_sec, \n sum((value - value_l)/(time_sec - time_sec_l)) as 'Write queries per second'\nFROM \n t a \nwhere \n time_sec_l is not null \n and value_l is not null \n and value >= value_l \ngroup by\n time_sec\norder by \n time_sec", + "refId": "A", + "select": [ + [ + { + "params": [ + "CPU_TIME_MS" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "act_samples", + "timeColumn": "ts", + "timeColumnType": "timestamp", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Write Queries per Second", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "normal" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "wps" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "Rows written per second" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "green", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 7, + "w": 12, + "x": 12, + "y": 19 + }, + "id": 34, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "hide": false, + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\n SELECT \n *, \n lag(value) OVER w value_l, \n lag(time_sec) OVER w time_sec_l \n FROM \n (\n SELECT \n max(time_sec) as time_sec, \n max(value) as value, \n metric \n FROM \n (\n SELECT \n time_sec, \n $__unixEpochGroup(time_sec, $interval) as tg, \n intval as value, \n concat(name, host) as metric \n FROM \n `metrics` \n WHERE \n $__unixEpochFilter(time_sec) \n and (cluster = '$cluster') \n and (host in ($host) or $host = '%')\n and extractor = 'memsql' \n and subsystem = 'status' \n and name = 'memsql_status_rows_affected_by_writes'\n and role <> 'LEAF') subq \n GROUP BY \n metric, \n tg\n ) X WINDOW w AS (\n partition by metric \n order by \n time_sec\n )\n) \nSELECT \n time_sec, \n sum((value - value_l)/(time_sec - time_sec_l)) as 'Rows written per second'\nFROM \n t a \nwhere \n time_sec_l is not null \n and value_l is not null \n and value >= value_l \ngroup by time_sec\norder by \n 1", + "refId": "A", + "select": [ + [ + { + "params": [ + "CPU_TIME_MS" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "act_samples", + "timeColumn": "ts", + "timeColumnType": "timestamp", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Rows Written per Second", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "wps" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "Failed write queries per second" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "yellow", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 7, + "w": 12, + "x": 0, + "y": 26 + }, + "id": 41, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "table", + "hide": false, + "rawQuery": true, + "rawSql": "WITH t AS (\n SELECT \n *, \n lag(value) OVER w value_l, \n lag(time_sec) OVER w time_sec_l \n FROM \n (\n SELECT \n max(time_sec) as time_sec, \n avg(value) as value, \n metric \n FROM \n (\n SELECT \n time_sec, \n $__unixEpochGroup(time_sec, $interval) as tg, \n intval as value, \n concat(name, host) as metric \n FROM \n `metrics` \n WHERE \n $__unixEpochFilter(time_sec) \n and (cluster = '$cluster') \n and (host in ($host) or $host = '%')\n and extractor = 'memsql' \n and subsystem = 'status' \n and name like 'memsql_status_failed_write_queries'\n and role <> 'LEAF') subq \n GROUP BY \n metric, \n tg\n ) X WINDOW w AS (\n partition by metric \n order by \n time_sec\n )\n) \nSELECT \n time_sec, \n sum((value - value_l)/(time_sec - time_sec_l)) as 'Failed write queries per second'\nFROM \n t a \nwhere \n time_sec_l is not null \n and value_l is not null \n and value >= value_l \ngroup by\n time_sec\norder by \n time_sec\n", + "refId": "B", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Failed Write Queries per Second", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "Changes in the pattern of execution time per write query from the historical norm may indicate an issue with your application or changes in your workload.", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "decimals": 1, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "ms" + }, + "overrides": [] + }, + "gridPos": { + "h": 7, + "w": 12, + "x": 12, + "y": 26 + }, + "id": 49, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "table", + "rawQuery": true, + "rawSql": "WITH t AS (\n SELECT \n *, \n lag(value) OVER w value_l, \n lag(time) OVER w time_l\n from (\n select max(tg) as time,\n max(value) as value,\n metric, name\n from (\n SELECT time_sec,\n $__unixEpochGroup(time_sec, $interval) as tg, \n concat(name, host) as metric,\n name,\n value\n FROM metrics\n WHERE $__unixEpochFilter(time_sec) AND\n cluster = '$cluster' AND\n (host in ($host) or $host = '%') AND\n role <> 'LEAF' AND\n name in ('memsql_status_failed_write_queries','memsql_status_successful_write_queries','memsql_status_execution_time_of_write')\n )\n group by metric, tg) X \n WINDOW w AS (partition by metric order by time))\nselect time, \n sum(case when name = 'memsql_status_execution_time_of_write' then (value - value_l) end) /\n sum(case when name in ('memsql_status_failed_write_queries','memsql_status_successful_write_queries') then (value - value_l) end) as 'Execution Time per Write Query'\nfrom t\nwhere time_l is not null \n and value_l is not null \n and value >= value_l\ngroup by time\norder by time", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Execution Time per Write Query", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "rps" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "Read queries per second" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "green", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 7, + "w": 12, + "x": 0, + "y": 33 + }, + "id": 33, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "hide": false, + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\n SELECT \n *, \n lag(value) OVER w value_l, \n lag(time_sec) OVER w time_sec_l \n FROM \n (\n SELECT \n max(time_sec) as time_sec, \n avg(value) as value, \n metric \n FROM \n (\n SELECT \n time_sec, \n $__unixEpochGroup(time_sec, $interval) as tg, \n intval as value, \n concat(name, host) as metric \n FROM \n `metrics` \n WHERE \n $__unixEpochFilter(time_sec) \n and (cluster = '$cluster') \n and (host in ($host) or $host = '%')\n and extractor = 'memsql' \n and subsystem = 'status' \n and name like 'memsql_status_%_read_queries'\n and role <> 'LEAF') subq \n GROUP BY \n metric, \n tg\n ) X WINDOW w AS (\n partition by metric \n order by \n time_sec\n )\n) \nSELECT \n time_sec, \n sum((value - value_l)/(time_sec - time_sec_l)) as 'Read queries per second'\nFROM \n t a \nwhere \n time_sec_l is not null \n and value_l is not null \n and value >= value_l \ngroup by\n time_sec\norder by \n time_sec", + "refId": "A", + "select": [ + [ + { + "params": [ + "CPU_TIME_MS" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "act_samples", + "timeColumn": "ts", + "timeColumnType": "timestamp", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Read Queries per Second", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "normal" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "rps" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "Rows Read per second" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "green", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 7, + "w": 12, + "x": 12, + "y": 33 + }, + "id": 35, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "hide": false, + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\n SELECT \n *, \n lag(value) OVER w value_l, \n lag(time_sec) OVER w time_sec_l \n FROM \n (\n SELECT \n max(time_sec) as time_sec, \n max(value) as value, \n metric \n FROM \n (\n SELECT \n time_sec, \n $__unixEpochGroup(time_sec, $interval) as tg, \n intval as value, \n concat(name, host) as metric \n FROM \n `metrics` \n WHERE \n $__unixEpochFilter(time_sec) \n and (cluster = '$cluster') \n and (host in ($host) or $host = '%')\n and extractor = 'memsql' \n and subsystem = 'status' \n and name = 'memsql_status_rows_returned_by_reads'\n and role <> 'LEAF') subq \n GROUP BY \n metric, \n tg\n ) X WINDOW w AS (\n partition by metric \n order by \n time_sec\n )\n) \nSELECT \n time_sec, \n sum((value - value_l)/(time_sec - time_sec_l)) as 'Rows Read per second'\nFROM \n t a \nwhere \n time_sec_l is not null \n and value_l is not null \n and value >= value_l \ngroup by time_sec\norder by \n 1", + "refId": "A", + "select": [ + [ + { + "params": [ + "CPU_TIME_MS" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "act_samples", + "timeColumn": "ts", + "timeColumnType": "timestamp", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Rows Read per Second", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "rps" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "Failed read queries per second" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "yellow", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 7, + "w": 12, + "x": 0, + "y": 40 + }, + "id": 42, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "table", + "hide": false, + "rawQuery": true, + "rawSql": "WITH t AS (\n SELECT \n *, \n lag(value) OVER w value_l, \n lag(time_sec) OVER w time_sec_l \n FROM \n (\n SELECT \n max(time_sec) as time_sec, \n avg(value) as value, \n metric \n FROM \n (\n SELECT \n time_sec, \n $__unixEpochGroup(time_sec, $interval) as tg, \n intval as value, \n concat(name, host) as metric \n FROM \n `metrics` \n WHERE \n $__unixEpochFilter(time_sec) \n and (cluster = '$cluster') \n and (host in ($host) or $host = '%')\n and extractor = 'memsql' \n and subsystem = 'status' \n and name like 'memsql_status_failed_read_queries'\n and role <> 'LEAF') subq \n GROUP BY \n metric, \n tg\n ) X WINDOW w AS (\n partition by metric \n order by \n time_sec\n )\n) \nSELECT \n time_sec, \n sum((value - value_l)/(time_sec - time_sec_l)) as 'Failed read queries per second'\nFROM \n t a \nwhere \n time_sec_l is not null \n and value_l is not null \n and value >= value_l \ngroup by\n time_sec\norder by \n time_sec\n", + "refId": "B", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Failed Read Queries per Second", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "Changes in the pattern of execution time per read query from the historical norm may indicate an issue with your application or changes in your workload.", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "decimals": 1, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "ms" + }, + "overrides": [] + }, + "gridPos": { + "h": 7, + "w": 12, + "x": 12, + "y": 40 + }, + "id": 50, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "table", + "rawQuery": true, + "rawSql": "WITH t AS (\n SELECT \n *, \n lag(value) OVER w value_l, \n lag(time) OVER w time_l\n from (\n select max(tg) as time,\n max(value) as value,\n metric, name\n from (\n SELECT time_sec,\n $__unixEpochGroup(time_sec, $interval) as tg, \n concat(name, host) as metric,\n name,\n value\n FROM metrics\n WHERE $__unixEpochFilter(time_sec) AND\n cluster = '$cluster' AND\n (host in ($host) or $host = '%') AND\n role <> 'LEAF' AND\n name in ('memsql_status_failed_read_queries','memsql_status_successful_read_queries','memsql_status_execution_time_of_reads')\n )\n group by metric, tg) X \n WINDOW w AS (partition by metric order by time))\nselect time, \n sum(case when name = 'memsql_status_execution_time_of_reads' then (value - value_l) end) /\n sum(case when name in ('memsql_status_failed_read_queries','memsql_status_successful_read_queries') then (value - value_l) end) as 'Execution Time per Read Query'\nfrom t\nwhere time_l is not null \n and value_l is not null \n and value >= value_l\ngroup by time\norder by time", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Execution Time per Read Query", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "Bps" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "Network Sent Bytes" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "green", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 7, + "w": 12, + "x": 0, + "y": 47 + }, + "id": 40, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "hide": false, + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\nSELECT *, lag(value) OVER w value_l, lag(time_sec) OVER w time_sec_l\nFROM \n(SELECT\n max(time_sec) time_sec,\n $__unixEpochGroup(time_sec,$interval) tg,\n avg(intval) as value,\n trim_metric(name, 'memsql_status_bytes_') as metric,\n trim_host(host) host,\n port\nFROM metrics\nWHERE (cluster = '$cluster')\nand (extractor = 'memsql')\nand (subsystem = 'status')\nand name in ('memsql_status_bytes_sent')\nand $__unixEpochFilter(time_sec)\nGROUP BY metric, host, port, tg\n) X\nWINDOW w AS (partition by metric, host, port order by time_sec desc)\n)\nselect time_sec, sum((value - value_l)/(time_sec - time_sec_l)) as 'Network Sent Bytes' \nfrom t a\nwhere time_sec_l is not null and value_l is not null\ngroup by time_sec\norder by time_sec", + "refId": "B", + "select": [ + [ + { + "params": [ + "CPU_TIME_MS" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "act_samples", + "timeColumn": "ts", + "timeColumnType": "timestamp", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Network Sent Bytes", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "Bps" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "Network Received Bytes" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "green", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 7, + "w": 12, + "x": 12, + "y": 47 + }, + "id": 39, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "hide": false, + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\nSELECT *, lag(value) OVER w value_l, lag(time_sec) OVER w time_sec_l\nFROM \n(SELECT\n max(time_sec) time_sec,\n $__unixEpochGroup(time_sec,$interval) tg,\n avg(intval) as value,\n trim_metric(name, 'memsql_status_bytes_') as metric,\n trim_host(host) host,\n port\nFROM metrics\nWHERE (cluster = '$cluster')\nand (extractor = 'memsql')\nand (subsystem = 'status')\nand name in ('memsql_status_bytes_received')\nand $__unixEpochFilter(time_sec)\nGROUP BY metric, host, port, tg\n) X\nWINDOW w AS (partition by metric, host, port order by time_sec desc)\n)\nselect time_sec, sum((value - value_l)/(time_sec - time_sec_l)) as 'Network Received Bytes' \nfrom t a\nwhere time_sec_l is not null and value_l is not null\ngroup by time_sec\norder by time_sec", + "refId": "B", + "select": [ + [ + { + "params": [ + "CPU_TIME_MS" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "act_samples", + "timeColumn": "ts", + "timeColumnType": "timestamp", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Network Received Bytes", + "type": "timeseries" + } + ], + "preload": false, + "refresh": "", + "schemaVersion": 41, + "tags": ["singlestore","sas-speedystore"], + "templating": { + "list": [ + { + "allValue": "'%'", + "current": { + "text": "memsql_cluster", + "value": "memsql_cluster" + }, + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "definition": "", + "includeAll": false, + "name": "cluster", + "options": [], + "query": "select distinct(cluster) from metrics where $__unixEpochFilter(time_sec)", + "refresh": 2, + "regex": "", + "type": "query" + }, + { + "allValue": "'%'", + "current": { + "text": "All", + "value": "$__all" + }, + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "definition": "select distinct(database_name) from act_samples where $__timeFilter(ts) and (cluster = '$cluster')", + "hide": 2, + "includeAll": true, + "multi": true, + "name": "database_name", + "options": [], + "query": "select distinct(database_name) from act_samples where $__timeFilter(ts) and (cluster = '$cluster')", + "refresh": 2, + "regex": "", + "sort": 5, + "type": "query" + }, + { + "allValue": "'%'", + "current": { + "text": "All", + "value": "$__all" + }, + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "definition": "", + "hide": 2, + "includeAll": true, + "multi": true, + "name": "host", + "options": [], + "query": "select distinct(host) from metrics where host is not NULL and $__unixEpochFilter(time_sec) and (cluster='$cluster')", + "refresh": 2, + "regex": "", + "type": "query" + }, + { + "auto": true, + "auto_count": 50, + "auto_min": "1m", + "current": { + "text": "$__auto", + "value": "$__auto" + }, + "name": "interval", + "options": [ + { + "selected": false, + "text": "1m", + "value": "1m" + }, + { + "selected": false, + "text": "10m", + "value": "10m" + }, + { + "selected": false, + "text": "30m", + "value": "30m" + }, + { + "selected": false, + "text": "1h", + "value": "1h" + }, + { + "selected": false, + "text": "6h", + "value": "6h" + }, + { + "selected": false, + "text": "12h", + "value": "12h" + }, + { + "selected": false, + "text": "1d", + "value": "1d" + }, + { + "selected": false, + "text": "7d", + "value": "7d" + }, + { + "selected": false, + "text": "14d", + "value": "14d" + }, + { + "selected": false, + "text": "30d", + "value": "30d" + } + ], + "query": "1m,10m,30m,1h,6h,12h,1d,7d,14d,30d", + "refresh": 2, + "type": "interval" + } + ] + }, + "time": { + "from": "now-1h", + "to": "now" + }, + "timepicker": { + "refresh_intervals": [ + "5m", + "15m", + "30m", + "1h", + "2h", + "1d" + ] + }, + "timezone": "", + "title": "SingleStore: Cluster View", + "uid": "detailed-cluster-view2", + "version": 1 +} diff --git a/samples/speedystore/dashboards/detailedclusterviewbynode.json b/samples/speedystore/dashboards/detailedclusterviewbynode.json new file mode 100644 index 00000000..b1cae738 --- /dev/null +++ b/samples/speedystore/dashboards/detailedclusterviewbynode.json @@ -0,0 +1,2483 @@ +{ + "annotations": { + "list": [ + { + "builtIn": 1, + "datasource": { + "type": "datasource", + "uid": "grafana" + }, + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "limit": 100, + "name": "Annotations & Alerts", + "showIn": 0, + "target": { + "limit": 100, + "matchAny": false, + "tags": [], + "type": "dashboard" + }, + "type": "dashboard" + } + ] + }, + "editable": true, + "fiscalYearStartMonth": 0, + "graphTooltip": 1, + "id": 262, + "links": [ + { + "asDropdown": false, + "icon": "external link", + "includeVars": false, + "keepTime": false, + "tags": [], + "targetBlank": true, + "title": "Documentation", + "tooltip": "", + "type": "link", + "url": "https://docs.singlestore.com/db/v8.1/en/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/view-the-dashboards.html#cluster-view-793226" + }, + { + "asDropdown": true, + "icon": "external link", + "includeVars": true, + "keepTime": true, + "tags": [], + "targetBlank": true, + "title": "Dashboards", + "type": "dashboards" + } + ], + "panels": [ + { + "collapsed": false, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 0 + }, + "id": 47, + "panels": [], + "title": "Resource Utilization", + "type": "row" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "CPU", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "decimals": 1, + "mappings": [], + "max": 100, + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "percent" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "max single-core load" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "dark-green", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "min single-core load" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "blue", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 0, + "y": 1 + }, + "id": 20, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "hide": false, + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t1 AS (WITH t AS (\n SELECT *, lag(value) OVER w value_l, lag(time_sec) OVER w time_sec_l\n FROM\n (SELECT\n-- take the last datapoint in the interval\ntime_sec as time_sec,\nmemsql_sysinfo_cpu_used_ms/memsql_sysinfo_cpu_num_cpus value,\nhost\n FROM\n (SELECT\n max(time_sec) time_sec,\n -- this is the last counter in the time group\n max(intval) as value,\n name as metric,\n trim_host(host) host,\n $__unixEpochGroup(time_sec, $interval) as tg\n FROM metrics\n WHERE\n (cluster = '$cluster')\n and (host in ($host) or concat($host) = '%')\n and (extractor = 'memsql')\n and (subsystem = 'sysinfo')\n and (name = 'memsql_sysinfo_cpu_used_ms' or name = 'memsql_sysinfo_cpu_num_cpus')\n and $__unixEpochFilter(time_sec)\n group by metric, host, tg\n ) as pvt_data\n PIVOT\n (\n AVG(value)\n FOR metric\n IN ('memsql_sysinfo_cpu_used_ms', 'memsql_sysinfo_cpu_num_cpus')\n ) as pvt_table\n ) X\n WINDOW w AS (partition by host order by time_sec desc)\n)\nselect a.time_sec, (value - value_l)/(1000*(time_sec - time_sec_l)) * 100 value, a.host from t a\nWHERE time_sec_l is not null and value_l is not null\norder by 1, 2 desc, 3)\nSELECT time_sec, host, max(value) 'CPU:' from t1\ngroup by time_sec, host\norder by time_sec, host\n", + "refId": "B", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "CPU Utilization per Host", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "Memory", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "max": 1, + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "percentunit" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 8, + "y": 1 + }, + "id": 22, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "select * from (\nselect time_sec,\nhost,\nmemsql_sysinfo_mem_used_b / memsql_sysinfo_mem_total_b as 'Memory:'\nFROM\n(SELECT\n max(time_sec) time_sec,\n avg(value) value,\n metric,\n host\n FROM\n (SELECT\n time_sec,\n $__unixEpochGroup(time_sec,$interval) tg,\n intval as value,\n name as metric,\n trim_host(host) as host\n FROM metrics\n WHERE cluster='$cluster'\n and (host in ($host) or concat($host) = '%')\n and extractor = 'memsql'\n and subsystem = 'sysinfo'\n and name in ('memsql_sysinfo_mem_used_b','memsql_sysinfo_mem_total_b')\n and $__unixEpochFilter(time_sec)\n ) X\n GROUP BY tg, metric, host\n) as pvt_data\nPIVOT (\n AVG(value) \n FOR metric \n IN ('memsql_sysinfo_mem_used_b','memsql_sysinfo_mem_total_b')\n) as pvt_table) a\norder by time_sec", + "refId": "B", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Memory Utilization per Host", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "Disk Utilization", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "max": 1, + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "percentunit" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 16, + "y": 1 + }, + "id": 58, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "select * from (\nselect time_sec,\nhost,\nmemsql_sysinfo_disk_used_b / memsql_sysinfo_disk_total_b as 'Disk:'\nFROM\n(SELECT\n max(time_sec) time_sec,\n avg(value) value,\n metric,\n host\n FROM\n (SELECT\n time_sec,\n $__unixEpochGroup(time_sec,$interval) tg,\n intval as value,\n name as metric,\n trim_host(host) as host\n FROM metrics\n WHERE cluster='$cluster'\n and (host in ($host) or concat($host) = '%')\n and extractor = 'memsql'\n and subsystem = 'sysinfo'\n and name in ('memsql_sysinfo_disk_used_b','memsql_sysinfo_disk_total_b')\n and $__unixEpochFilter(time_sec)\n ) X\n GROUP BY tg, metric, host\n) as pvt_data\nPIVOT (\n AVG(value) \n FOR metric \n IN ('memsql_sysinfo_disk_used_b','memsql_sysinfo_disk_total_b')\n) as pvt_table) a\norder by time_sec", + "refId": "B", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Disk Utilization per Host", + "type": "timeseries" + }, + { + "collapsed": false, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 9 + }, + "id": 46, + "panels": [], + "title": "Connections & Threads", + "type": "row" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "Number of open connections (threads_connected) to the database relative to the maximum limit (max_connections). If the utilization is near 100%, it means DB is approaching the maximum allowed connections; this can potentially lead to performance issues, as queries may need to wait in a queue until a thread becomes available to process them.", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "Threads - Connected", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "decimals": 1, + "mappings": [], + "max": 100, + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "percent" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "max single-core load" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "dark-green", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "min single-core load" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "blue", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 10 + }, + "id": 55, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "hide": false, + "metricColumn": "none", + "rawQuery": true, + "rawSql": "select tg as time,\n concat(host,':',port,' {',role,'}') as node,\n max(used_c / total_C) * 100 as 'Threads - Connected:'\nfrom (SELECT time_sec as time, \n $__unixEpochGroup(time_sec, $interval) as tg, \n sum(case when name = 'memsql_variables_max_connections' then value else 0 end) as total_c,\n sum(case when name = 'memsql_status_threads_connected' then value else 0 end) as used_c,\n host, port, role\n FROM metrics\n WHERE $__unixEpochFilter(time_sec) AND\n (cluster = '$cluster') AND\n (host in ($host) or concat($host) = '%') AND\n (port in ($port) or concat($port) = '%') AND\n NAME in ('memsql_variables_max_connections','memsql_status_threads_connected') \n group by time, tg, host, port, role)\ngroup by tg, node\norder by tg, node;", + "refId": "B", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Threads - Connected", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "Number of threads actively running queries (threads_running) relative to the maximum limit (max_connection_threads). If the utilization is near 100%, it suggests that the system might be reaching its capacity in terms of how many queries can be executed in parallel. This situation can lead to resource pressure, unresponsiveness, latency spikes, and even failures", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "Threads - Running", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "decimals": 1, + "mappings": [], + "max": 100, + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "percent" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "max single-core load" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "dark-green", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "min single-core load" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "blue", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 10 + }, + "id": 56, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "hide": false, + "metricColumn": "none", + "rawQuery": true, + "rawSql": "select tg as time,\n concat(host,':',port,' {',role,'}') as node,\n max(used_c / total_C) * 100 as 'Threads - Running:' \nfrom (SELECT time_sec as time, \n $__unixEpochGroup(time_sec, $interval) as tg, \n sum(case when name = 'memsql_variables_max_connection_threads' then value else 0 end) as total_c,\n sum(case when name = 'memsql_status_threads_running' then value else 0 end) as used_c,\n host, port, role\n FROM metrics\n WHERE $__unixEpochFilter(time_sec) AND\n (cluster = '$cluster') AND\n (host in ($host) or concat($host) = '%') AND\n (port in ($port) or concat($port) = '%') AND\n NAME in ('memsql_variables_max_connection_threads','memsql_status_threads_running') \n group by time, tg, host, port, role)\ngroup by tg, node\norder by tg, node;", + "refId": "B", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Threads - Running", + "type": "timeseries" + }, + { + "collapsed": false, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 18 + }, + "id": 45, + "panels": [], + "title": "Throughput", + "type": "row" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "normal" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "wps" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 19 + }, + "id": 36, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "hide": false, + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\n SELECT \n *, \n lag(value) OVER w value_l, \n lag(time_sec) OVER w time_sec_l \n FROM \n (\n SELECT \n max(time_sec) as time_sec, \n avg(value) as value, \n host , port, name, role\n FROM \n (\n SELECT \n time_sec, \n $__unixEpochGroup(time_sec, $interval) as tg, \n intval as value, \n host , port, name, role\n FROM \n `metrics` \n WHERE \n $__unixEpochFilter(time_sec) \n and (cluster = '$cluster') \n and (host in ($host) or concat($host) = '%')\n and (port in ($port) or concat($port) = '%')\n and extractor = 'memsql' \n and subsystem = 'status' \n and name like 'memsql_status_%_write_queries'\n and role <> 'LEAF') subq \n GROUP BY \n host, port, name, role,\n tg\n ) X WINDOW w AS (\n partition by host, port , name\n order by \n time_sec\n )\n) \nSELECT \n time_sec, \n concat(host,':',port,' {',role,'}') as node,\n sum((value - value_l)/(time_sec - time_sec_l)) as 'Write queries per second:'\nFROM \n t a \nwhere \n time_sec_l is not null \n and value_l is not null \n and value >= value_l \ngroup by\n 1,2\norder by \n 1,2", + "refId": "A", + "select": [ + [ + { + "params": [ + "CPU_TIME_MS" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "act_samples", + "timeColumn": "ts", + "timeColumnType": "timestamp", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Write Queries per Second", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "normal" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "wps" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 19 + }, + "id": 34, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "hide": false, + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\n SELECT \n *, \n lag(value) OVER w value_l, \n lag(time_sec) OVER w time_sec_l \n FROM \n (\n SELECT \n max(time_sec) as time_sec, \n max(value) as value, \n host, port, role\n FROM \n (\n SELECT \n time_sec, \n $__unixEpochGroup(time_sec, $interval) as tg, \n intval as value, \n host, port , role\n FROM \n `metrics` \n WHERE \n $__unixEpochFilter(time_sec) \n and (cluster = '$cluster') \n and (host in ($host) or concat($host) = '%')\n and (port in ($port) or concat($port) = '%')\n and extractor = 'memsql' \n and subsystem = 'status' \n and name = 'memsql_status_rows_affected_by_writes'\n and role <> 'LEAF') subq \n GROUP BY \n host, port, role,\n tg\n ) X WINDOW w AS (\n partition by host, port \n order by \n time_sec\n )\n) \nSELECT \n time_sec, \n concat(host,':',port,' {',role,'}') as node,\n sum((value - value_l)/(time_sec - time_sec_l)) as 'Rows written per second:'\nFROM \n t a \nwhere \n time_sec_l is not null \n and value_l is not null \n and value >= value_l \ngroup by 1,2\norder by \n 1,2", + "refId": "A", + "select": [ + [ + { + "params": [ + "CPU_TIME_MS" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "act_samples", + "timeColumn": "ts", + "timeColumnType": "timestamp", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Rows Written per Second", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "normal" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "wps" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 27 + }, + "id": 43, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "9.3.6", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "hide": false, + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\n SELECT \n *, \n lag(value) OVER w value_l, \n lag(time_sec) OVER w time_sec_l \n FROM \n (\n SELECT \n max(time_sec) as time_sec, \n avg(value) as value, \n host , port, role\n FROM \n (\n SELECT \n time_sec, \n $__unixEpochGroup(time_sec, $interval) as tg, \n intval as value, \n host , port, role\n FROM \n `metrics` \n WHERE \n $__unixEpochFilter(time_sec) \n and (cluster = '$cluster') \n and (host in ($host) or concat($host) = '%')\n and (port in ($port) or concat($port) = '%')\n and extractor = 'memsql' \n and subsystem = 'status' \n and name like 'memsql_status_failed_write_queries'\n and role <> 'LEAF') subq \n GROUP BY \n host, port, role,\n tg\n ) X WINDOW w AS (\n partition by host, port \n order by \n time_sec\n )\n) \nSELECT \n time_sec, \n concat(host,':',port,' {',role,'}') as node,\n sum((value - value_l)/(time_sec - time_sec_l)) as 'Failed Write queries per second:'\nFROM \n t a \nwhere \n time_sec_l is not null \n and value_l is not null \n and value >= value_l \ngroup by\n 1,2\norder by \n 1,2", + "refId": "A", + "select": [ + [ + { + "params": [ + "CPU_TIME_MS" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "act_samples", + "timeColumn": "ts", + "timeColumnType": "timestamp", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Failed Write Queries per Second", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "Changes in the pattern of execution time per write query from the historical norm may indicate an issue with your application or changes in your workload.", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "decimals": 1, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "ms" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 27 + }, + "id": 53, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "9.3.6", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "hide": false, + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\n SELECT \n *, \n lag(value) OVER w value_l, \n lag(time) OVER w time_l\n from (\n select max(tg) as time,\n max(value) as value,\n host,port, name, role\n from (\n SELECT time_sec,\n $__unixEpochGroup(time_sec, $interval) as tg, \n host,\n port,role,\n name,\n value\n FROM metrics\n WHERE $__unixEpochFilter(time_sec) AND\n cluster = '$cluster' AND\n (host in ($host) or concat($host) = '%') AND\n (port in ($port) or concat($port) = '%') AND\n role <> 'LEAF' AND\n name in ('memsql_status_failed_write_queries','memsql_status_successful_write_queries','memsql_status_execution_time_of_write')\n )\n group by name, host, port, role, tg) X \n WINDOW w AS (partition by host,port,name order by time))\nselect time, \n concat(host,':',port,' {',role,'}') as node,\n sum(case when name = 'memsql_status_execution_time_of_write' then (value - value_l) end) /\n sum(case when name in ('memsql_status_failed_write_queries','memsql_status_successful_write_queries') then (value - value_l) end) as 'Execution Time per Write Query:'\nfrom t\nwhere time_l is not null \n and value_l is not null \n and value >= value_l\ngroup by time, node\norder by time, node", + "refId": "A", + "select": [ + [ + { + "params": [ + "CPU_TIME_MS" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "act_samples", + "timeColumn": "ts", + "timeColumnType": "timestamp", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Execution Time per Write Query", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "normal" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "rps" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 35 + }, + "id": 33, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "9.3.6", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "hide": false, + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\n SELECT \n *, \n lag(value) OVER w value_l, \n lag(time_sec) OVER w time_sec_l \n FROM \n (\n SELECT \n max(time_sec) as time_sec, \n avg(value) as value, \n host , port, name, role\n FROM \n (\n SELECT \n time_sec, \n $__unixEpochGroup(time_sec, $interval) as tg, \n intval as value, \n host , port, name, role\n FROM \n `metrics` \n WHERE \n $__unixEpochFilter(time_sec) \n and (cluster = '$cluster') \n and (host in ($host) or concat($host) = '%')\n and (port in ($port) or concat($port) = '%')\n and extractor = 'memsql' \n and subsystem = 'status' \n and name like 'memsql_status_%_read_queries'\n and role <> 'LEAF') subq \n GROUP BY \n host, port, name, role,\n tg\n ) X WINDOW w AS (\n partition by host, port , name\n order by \n time_sec\n )\n) \nSELECT \n time_sec, concat(host,':',port,' {',role,'}') as node,\n sum((value - value_l)/(time_sec - time_sec_l)) as 'Read queries per second:'\nFROM \n t a \nwhere \n time_sec_l is not null \n and value_l is not null \n and value >= value_l \ngroup by\n 1,2\norder by \n 1,2", + "refId": "A", + "select": [ + [ + { + "params": [ + "CPU_TIME_MS" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "act_samples", + "timeColumn": "ts", + "timeColumnType": "timestamp", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Read Queries per Second", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "normal" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "rps" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 35 + }, + "id": 35, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "9.3.6", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "hide": false, + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\n SELECT\n *,\n lag(value) OVER w value_l,\n lag(time_sec) OVER w time_sec_l\n FROM\n (\n SELECT\n max(time_sec) as time_sec,\n max(value) as value, \n host, port, role\n FROM\n (\n SELECT\n time_sec,\n $__unixEpochGroup(time_sec, $interval) as tg,\n intval as value,\n host, port, role\n FROM\n `metrics`\n WHERE\n $__unixEpochFilter(time_sec)\n and (cluster = '$cluster')\n and (host in ($host) or concat($host) = '%')\n and (port in ($port) or concat($port) = '%')\n and extractor = 'memsql'\n and subsystem = 'status'\n and name = 'memsql_status_rows_returned_by_reads'\n and role <> 'LEAF') subq\n GROUP BY\n host, port, role,\n tg\n ) X WINDOW w AS (\n partition by host, port\n order by\n time_sec\n )\n)\nSELECT\n time_sec,\n concat(host,':',port,' {',role,'}') as node,\n sum((value - value_l)/(time_sec - time_sec_l)) as 'Rows Read per second:'\nFROM\n t a\nwhere\n time_sec_l is not null\n and value_l is not null\n and value >= value_l\ngroup by 1,2\norder by\n 1,2", + "refId": "A", + "select": [ + [ + { + "params": [ + "CPU_TIME_MS" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "act_samples", + "timeColumn": "ts", + "timeColumnType": "timestamp", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Rows Read per Second", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "normal" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "rps" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 43 + }, + "id": 44, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "9.3.6", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "hide": false, + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\n SELECT \n *, \n lag(value) OVER w value_l, \n lag(time_sec) OVER w time_sec_l \n FROM \n (\n SELECT \n max(time_sec) as time_sec, \n avg(value) as value, \n host , port, role\n FROM \n (\n SELECT \n time_sec, \n $__unixEpochGroup(time_sec, $interval) as tg, \n intval as value, \n host , port, role\n FROM \n `metrics` \n WHERE \n $__unixEpochFilter(time_sec) \n and (cluster = '$cluster') \n and (host in ($host) or concat($host) = '%')\n and (port in ($port) or concat($port) = '%')\n and extractor = 'memsql' \n and subsystem = 'status' \n and name like 'memsql_status_failed_read_queries'\n and role <> 'LEAF') subq \n GROUP BY \n host, port, role,\n tg\n ) X WINDOW w AS (\n partition by host, port \n order by \n time_sec\n )\n) \nSELECT \n time_sec, concat(host,':',port,' {',role,'}') as node,\n sum((value - value_l)/(time_sec - time_sec_l)) as 'Failed Read queries per second:'\nFROM \n t a \nwhere \n time_sec_l is not null \n and value_l is not null \n and value >= value_l \ngroup by\n 1,2\norder by \n 1,2", + "refId": "A", + "select": [ + [ + { + "params": [ + "CPU_TIME_MS" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "act_samples", + "timeColumn": "ts", + "timeColumnType": "timestamp", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Failed Read Queries per Second", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "Changes in the pattern of execution time per read query from the historical norm may indicate an issue with your application or changes in your workload.", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "decimals": 1, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "ms" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 43 + }, + "id": 54, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "9.3.6", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "hide": false, + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\n SELECT \n *, \n lag(value) OVER w value_l, \n lag(time) OVER w time_l\n from (\n select max(tg) as time,\n max(value) as value,\n host,port, name, role\n from (\n SELECT time_sec,\n $__unixEpochGroup(time_sec, $interval) as tg, \n host,\n port, role,\n name,\n value\n FROM metrics\n WHERE $__unixEpochFilter(time_sec) AND\n cluster = '$cluster' AND\n (host in ($host) or concat($host) = '%') AND\n (port in ($port) or concat($port) = '%') AND\n role <> 'LEAF' AND\n name in ('memsql_status_failed_read_queries','memsql_status_successful_read_queries','memsql_status_execution_time_of_reads')\n )\n group by name, host, port,role, tg) X \n WINDOW w AS (partition by host,port,name order by time))\nselect time, \n concat(host,':',port,' {',role,'}') as node,\n sum(case when name = 'memsql_status_execution_time_of_reads' then (value - value_l) end) /\n sum(case when name in ('memsql_status_failed_read_queries','memsql_status_successful_read_queries') then (value - value_l) end) as 'Execution Time per read Query:'\nfrom t\nwhere time_l is not null \n and value_l is not null \n and value >= value_l\ngroup by time, node\norder by time, node", + "refId": "A", + "select": [ + [ + { + "params": [ + "CPU_TIME_MS" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "act_samples", + "timeColumn": "ts", + "timeColumnType": "timestamp", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Execution Time per Read Query", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "normal" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "Bps" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 51 + }, + "id": 39, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "9.3.6", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "hide": false, + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\nSELECT *, lag(value) OVER w value_l, lag(time_sec) OVER w time_sec_l\nFROM \n(SELECT\n max(time_sec) time_sec,\n $__unixEpochGroup(time_sec,$interval) tg,\n avg(intval) as value,\n trim_metric(name, 'memsql_status_bytes_') as metric,\n trim_host(host) host,\n port, role\nFROM metrics\nWHERE (cluster = '$cluster')\nand (extractor = 'memsql')\nand (subsystem = 'status')\nand name in ('memsql_status_bytes_received')\nand (host in ($host) or concat($host) = '%')\nand (port in ($port) or concat($port) = '%')\nand $__unixEpochFilter(time_sec)\nGROUP BY metric, host, port, role, tg\n) X\nWINDOW w AS (partition by metric, host, port order by time_sec desc)\n)\nselect time_sec, concat(host,':',port,' {',role,'}') as node, sum((value - value_l)/(time_sec - time_sec_l)) 'Network Received bytes:'\nfrom t a\nwhere time_sec_l is not null and value_l is not null\ngroup by 1,2\norder by 1,2", + "refId": "B", + "select": [ + [ + { + "params": [ + "CPU_TIME_MS" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "act_samples", + "timeColumn": "ts", + "timeColumnType": "timestamp", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Network Received Bytes", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "normal" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "Bps" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 51 + }, + "id": 40, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "9.3.6", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "hide": false, + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\nSELECT *, lag(value) OVER w value_l, lag(time_sec) OVER w time_sec_l\nFROM \n(SELECT\n max(time_sec) time_sec,\n $__unixEpochGroup(time_sec,$interval) tg,\n avg(intval) as value,\n trim_metric(name, 'memsql_status_bytes_') as metric,\n trim_host(host) host,\n port, role\nFROM metrics\nWHERE (cluster = '$cluster')\nand (extractor = 'memsql')\nand (subsystem = 'status')\nand name in ('memsql_status_bytes_sent')\nand (host in ($host) or concat($host) = '%')\nand (port in ($port) or concat($port) = '%')\nand $__unixEpochFilter(time_sec)\nGROUP BY metric, host, port, role, tg\n) X\nWINDOW w AS (partition by metric, host, port order by time_sec desc)\n)\nselect time_sec, concat(host,':',port,' {',role,'}') as node, sum((value - value_l)/(time_sec - time_sec_l)) 'Network Sent bytes:' \nfrom t a\nwhere time_sec_l is not null and value_l is not null\ngroup by 1,2\norder by 1,2", + "refId": "B", + "select": [ + [ + { + "params": [ + "CPU_TIME_MS" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "act_samples", + "timeColumn": "ts", + "timeColumnType": "timestamp", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Network Sent Bytes", + "type": "timeseries" + } + ], + "preload": false, + "refresh": "", + "schemaVersion": 41, + "tags": ["singlestore","sas-speedystore"], + "templating": { + "list": [ + { + "allValue": "'%'", + "current": { + "text": "memsql_cluster", + "value": "memsql_cluster" + }, + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "definition": "", + "includeAll": false, + "name": "cluster", + "options": [], + "query": "select distinct(cluster) from metrics where $__unixEpochFilter(time_sec)", + "refresh": 2, + "regex": "", + "type": "query" + }, + { + "allValue": "'%'", + "current": { + "text": "All", + "value": "$__all" + }, + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "definition": "select distinct(database_name) from act_samples where $__timeFilter(ts) and (cluster = '$cluster')", + "hide": 2, + "includeAll": true, + "multi": true, + "name": "database_name", + "options": [], + "query": "select distinct(database_name) from act_samples where $__timeFilter(ts) and (cluster = '$cluster')", + "refresh": 2, + "regex": "", + "sort": 5, + "type": "query" + }, + { + "allValue": "'%'", + "current": { + "text": "All", + "value": "$__all" + }, + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "definition": "", + "includeAll": true, + "multi": true, + "name": "host", + "options": [], + "query": "select distinct(host) from metrics where host is not NULL and $__unixEpochFilter(time_sec) and (cluster='$cluster')", + "refresh": 2, + "regex": "", + "type": "query" + }, + { + "allValue": "'%'", + "current": { + "text": "All", + "value": "$__all" + }, + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "definition": "select distinct(port) from metrics where host is not NULL and $__unixEpochFilter(time_sec) and (cluster='$cluster') and (host in ($host) or concat($host) = '%') and port is not null", + "includeAll": true, + "multi": true, + "name": "port", + "options": [], + "query": "select distinct(port) from metrics where host is not NULL and $__unixEpochFilter(time_sec) and (cluster='$cluster') and (host in ($host) or concat($host) = '%') and port is not null", + "refresh": 1, + "regex": "", + "sort": 3, + "type": "query" + }, + { + "auto": true, + "auto_count": 50, + "auto_min": "1m", + "current": { + "text": "$__auto", + "value": "$__auto" + }, + "name": "interval", + "options": [ + { + "selected": false, + "text": "1m", + "value": "1m" + }, + { + "selected": false, + "text": "10m", + "value": "10m" + }, + { + "selected": false, + "text": "30m", + "value": "30m" + }, + { + "selected": false, + "text": "1h", + "value": "1h" + }, + { + "selected": false, + "text": "6h", + "value": "6h" + }, + { + "selected": false, + "text": "12h", + "value": "12h" + }, + { + "selected": false, + "text": "1d", + "value": "1d" + }, + { + "selected": false, + "text": "7d", + "value": "7d" + }, + { + "selected": false, + "text": "14d", + "value": "14d" + }, + { + "selected": false, + "text": "30d", + "value": "30d" + } + ], + "query": "1m,10m,30m,1h,6h,12h,1d,7d,14d,30d", + "refresh": 2, + "type": "interval" + } + ] + }, + "time": { + "from": "now-1h", + "to": "now" + }, + "timepicker": { + "refresh_intervals": [ + "5m", + "15m", + "30m", + "1h", + "2h", + "1d" + ] + }, + "timezone": "", + "title": "SingleStore: Detailed Cluster View By Node", + "uid": "TVggJLbVk", + "version": 1 +} diff --git a/samples/speedystore/dashboards/diskusage.json b/samples/speedystore/dashboards/diskusage.json new file mode 100644 index 00000000..305d3b5f --- /dev/null +++ b/samples/speedystore/dashboards/diskusage.json @@ -0,0 +1,1254 @@ +{ + "annotations": { + "list": [ + { + "builtIn": 1, + "datasource": { + "type": "datasource", + "uid": "grafana" + }, + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "name": "Annotations & Alerts", + "target": { + "limit": 100, + "matchAny": false, + "tags": [], + "type": "dashboard" + }, + "type": "dashboard" + } + ] + }, + "editable": true, + "fiscalYearStartMonth": 0, + "graphTooltip": 0, + "id": 263, + "links": [], + "panels": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "Disk Utilization", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "max": 1, + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "percentunit" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 0, + "y": 0 + }, + "id": 13, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "select * from (\nselect time_sec,\nhost,\nmemsql_sysinfo_disk_used_b / memsql_sysinfo_disk_total_b as 'Disk:'\nFROM\n(SELECT\n max(time_sec) time_sec,\n avg(value) value,\n metric,\n host\n FROM\n (SELECT\n time_sec,\n $__unixEpochGroup(time_sec,$interval) tg,\n intval as value,\n name as metric,\n trim_host(host) as host\n FROM metrics\n WHERE cluster='$cluster'\n and (host in ('$host') or concat('$host') = '%')\n and extractor = 'memsql'\n and subsystem = 'sysinfo'\n and name in ('memsql_sysinfo_disk_used_b','memsql_sysinfo_disk_total_b')\n and $__unixEpochFilter(time_sec)\n ) X\n GROUP BY tg, metric, host\n) as pvt_data\nPIVOT (\n AVG(value) \n FOR metric \n IN ('memsql_sysinfo_disk_used_b','memsql_sysinfo_disk_total_b')\n) as pvt_table) a\norder by time_sec", + "refId": "B", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Disk Utilization per Host", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "bytes" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "Total Disk Size" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "dark-red", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Total Used" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "dark-green", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byFrameRefID", + "options": "A" + }, + "properties": [ + { + "id": "custom.stacking", + "value": { + "group": "A", + "mode": "normal" + } + }, + { + "id": "custom.fillOpacity", + "value": 30 + } + ] + } + ] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 8, + "y": 0 + }, + "id": 14, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true, + "sortBy": "Last *", + "sortDesc": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "select sum(used_c) as 'Total Used',\n sum(total_c) as 'Total Disk Size', \n tg as time\nfrom (SELECT $__unixEpochGroup(time_sec, $interval) as tg,\n max(case when name = 'memsql_sysinfo_disk_total_b' then value else 0 end) as total_c,\n max(case when name = 'memsql_sysinfo_disk_used_b' then value else 0 end) as used_c,\n host\n FROM metrics\n WHERE cluster='$cluster'\n and (host in ('$host') or concat('$host') = '%')\n and extractor = 'memsql'\n and subsystem = 'sysinfo'\n and name in ('memsql_sysinfo_disk_used_b','memsql_sysinfo_disk_total_b')\n and $__unixEpochFilter(time_sec)\n group by tg, host)\nwhere used_c > 0 & total_c > 0\ngroup by tg\norder by tg;", + "refId": "B", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "table", + "hide": false, + "rawQuery": true, + "rawSql": "select tg as time,\n sum(memsql_disk_usage_tracelogs_b) tracelogs,\n sum(memsql_disk_usage_plancache_b) plancache,\n sum(memsql_disk_usage_auditlogs_b) auditlogs,\n sum(memsql_disk_usage_user_data_b + memsql_disk_usage_other_data_b) data\nfrom \n (\n SELECT $__unixEpochGroup(time_sec, $interval) AS tg, \n max(case when name = 'memsql_disk_usage_tracelogs_b' then value else 0 end) as memsql_disk_usage_tracelogs_b,\n max(case when name = 'memsql_disk_usage_plancache_b' then value else 0 end) as memsql_disk_usage_plancache_b,\n max(case when name = 'memsql_disk_usage_auditlogs_b' then value else 0 end) as memsql_disk_usage_auditlogs_b,\n max(case when name = 'memsql_disk_usage_other_data_b' then value else 0 end) as memsql_disk_usage_other_data_b,\n max(case when name = 'memsql_disk_usage_user_data_b' then value else 0 end) as memsql_disk_usage_user_data_b,\n host AS host\n FROM metrics\n WHERE cluster='$cluster'\n and (host in ('$host') or concat('$host') = '%')\n and $__unixEpochFilter(time_sec)\n and NAME in ('memsql_disk_usage_tracelogs_b','memsql_disk_usage_plancache_b', 'memsql_disk_usage_auditlogs_b', 'memsql_disk_usage_other_data_b', 'memsql_disk_usage_user_data_b')\n group by tg, host\n ) as used\ngroup by \n time \norder by \n time", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Distribution of Components using Disk", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "Breaksdown consumption of disk by \"Data\" category (in the left graph), adding utilization across all components will be equal to disk utilized by \"Data\"", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "bytes" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "Total Available Cache" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "dark-red", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Total Cache Used" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "dark-green", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byFrameRefID", + "options": "A" + }, + "properties": [ + { + "id": "custom.stacking", + "value": { + "group": "A", + "mode": "normal" + } + }, + { + "id": "custom.fillOpacity", + "value": 30 + } + ] + } + ] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 16, + "y": 0 + }, + "id": 15, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true, + "sortBy": "Last *", + "sortDesc": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "table", + "hide": false, + "rawQuery": true, + "rawSql": "select tg as time,\n sum(memsql_data_disk_usage_tempblobs_b) 'data/tempblobs',\n sum(memsql_data_disk_usage_snapshots_b) 'data/snapshots',\n sum(memsql_data_disk_usage_blobs_b) 'data/blobs',\n sum(memsql_data_disk_usage_logs_b) 'data/transaction logs',\n sum(memsql_data_disk_usage_other_b + memsql_disk_usage_other_data_b) 'data/other'\nfrom \n (\n SELECT $__unixEpochGroup(time_sec, $interval) AS tg, \n labels::$database_name as db,\n host AS host, \n max(case when name = 'memsql_data_disk_usage_tempblobs_b' then value else 0 end) as memsql_data_disk_usage_tempblobs_b,\n max(case when name = 'memsql_data_disk_usage_snapshots_b' then value else 0 end) as memsql_data_disk_usage_snapshots_b,\n max(case when name = 'memsql_data_disk_usage_blobs_b' then value else 0 end) as memsql_data_disk_usage_blobs_b,\n max(case when name = 'memsql_data_disk_usage_logs_b' then value else 0 end) as memsql_data_disk_usage_logs_b,\n max(case when name = 'memsql_data_disk_usage_other_b' then value else 0 end) as memsql_data_disk_usage_other_b,\n max(case when name = 'memsql_disk_usage_other_data_b' then value else 0 end) as memsql_disk_usage_other_data_b\n FROM metrics\n WHERE $__unixEpochFilter(time_sec) AND\n cluster='$cluster' and \n (host in ('$host') or concat('$host') = '%') and\n NAME in ('memsql_disk_usage_other_data_b','memsql_data_disk_usage_tempblobs_b','memsql_data_disk_usage_snapshots_b', 'memsql_data_disk_usage_blobs_b', 'memsql_data_disk_usage_logs_b', 'memsql_data_disk_usage_other_b')\n group by 1,2,3\n ) as used\ngroup by \n time \norder by \n time", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Breakdown of Disk Utilization by Data", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "Breaksdown consumption of disk by databases, adding utilization across all databases will be equal to disk utilized by \"Data\"", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "bytes" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "Total Available Cache" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "dark-red", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Total Cache Used" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "dark-green", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byFrameRefID", + "options": "A" + }, + "properties": [ + { + "id": "custom.stacking", + "value": { + "group": "A", + "mode": "normal" + } + }, + { + "id": "custom.fillOpacity", + "value": 30 + } + ] + } + ] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 0, + "y": 8 + }, + "id": 16, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true, + "sortBy": "Last *", + "sortDesc": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "table", + "hide": false, + "rawQuery": true, + "rawSql": "select tg as time,\n db,\n sum(value) as ' '\nfrom (\n SELECT $__unixEpochGroup(time_sec, $interval) AS tg,\n name,\n labels::$database_name as db,\n host AS host, \n max(value) as value\n FROM metrics\n WHERE $__unixEpochFilter(time_sec) and\n cluster='$cluster' and \n (host in ('$host') or concat('$host') = '%') and\n value > 0 and\n name in ('memsql_data_disk_usage_tempblobs_b','memsql_data_disk_usage_snapshots_b', 'memsql_data_disk_usage_blobs_b', 'memsql_data_disk_usage_logs_b', 'memsql_data_disk_usage_other_b')\n GROUP by 1,2,3,4\n ) as pvt_data\ngroup by 1,2\norder by 1,2", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Distribution of Databases using Disk", + "transformations": [ + { + "id": "prepareTimeSeries", + "options": { + "format": "multi" + } + } + ], + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "It provides insights into the performance of your SingleStoreDB's blob cache activity. By understanding and monitoring the rate at which the blob cache is downloading files from remote storage, you can identify potential performance bottlenecks or issues related to blob cache activity. \nFor instance: if you observe high download rates given the size of your database and scale of your hardware, you might consider increasing the local cache size.\nBy keeping an eye on this metric, you can make well-informed decisions regarding your cluster's performance and resource allocation to ensure optimal system operation.", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "MiBs" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "Total Available Cache" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "dark-red", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Total Cache Used" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "dark-green", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byFrameRefID", + "options": "A" + }, + "properties": [ + { + "id": "custom.stacking", + "value": { + "group": "A", + "mode": "normal" + } + }, + { + "id": "custom.fillOpacity", + "value": 30 + } + ] + } + ] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 8, + "y": 8 + }, + "id": 17, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true, + "sortBy": "Last *", + "sortDesc": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "table", + "hide": false, + "rawQuery": true, + "rawSql": "select tg as time,\n db,\n sum(value) as ' '\n from (\n SELECT $__unixEpochGroup(time_sec, $interval) AS tg,\n name,\n labels::$database_name as db,\n host AS host, \n max(value) as value\n FROM metrics\n WHERE $__unixEpochFilter(time_sec) and\n cluster='$cluster' and\n (host in ('$host') or concat('$host') = '%') and\n name in ('memsql_bottomless_status_blob_cache_download_mb_per_sec')\n GROUP by 1,2,3,4\n ) as pvt_data\ngroup by 1,2\norder by time", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Blob Cache Downloaded per Second (by database)", + "transformations": [ + { + "id": "prepareTimeSeries", + "options": { + "format": "multi" + } + } + ], + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "It provides insights into the performance of your SingleStoreDB's blob cache activity. By tracking this metric, you can optimize system resource utilization based on your data management needs. If evictions occur at a high rate, it may indicate that the cache size is too small or that the workload is causing a lot of cache turnover. In such cases, you can take appropriate measures, such as adjusting cache size or reviewing data access patterns, to improve overall cluster performance.\nMonitoring this metric, helps you identify potential performance bottlenecks related to the blob cache and make well-informed decisions for optimizing your SingleStore cluster's performance.", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "MiBs" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "Total Available Cache" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "dark-red", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Total Cache Used" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "dark-green", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byFrameRefID", + "options": "A" + }, + "properties": [ + { + "id": "custom.stacking", + "value": { + "group": "A", + "mode": "normal" + } + }, + { + "id": "custom.fillOpacity", + "value": 30 + } + ] + } + ] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 16, + "y": 8 + }, + "id": 18, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true, + "sortBy": "Last *", + "sortDesc": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "table", + "hide": false, + "rawQuery": true, + "rawSql": "select tg as time,\n db,\n sum(value) as ' '\n from (\n SELECT $__unixEpochGroup(time_sec, $interval) AS tg,\n name,\n labels::$database_name as db,\n host AS host, \n max(value) as value\n FROM metrics\n WHERE $__unixEpochFilter(time_sec) and\n cluster='$cluster' and\n (host in ('$host') or concat('$host') = '%') and\n name in ('memsql_bottomless_status_blob_cache_eviction_mb_per_sec')\n GROUP by 1,2,3,4\n ) as pvt_data\ngroup by 1,2\norder by time", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Blob Cache Evicted per Second (by database)", + "transformations": [ + { + "id": "prepareTimeSeries", + "options": { + "format": "multi" + } + } + ], + "type": "timeseries" + } + ], + "preload": false, + "refresh": "", + "schemaVersion": 41, + "tags": ["singlestore","sas-speedystore"], + "templating": { + "list": [ + { + "current": { + "text": "memsql_cluster", + "value": "memsql_cluster" + }, + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "definition": "", + "includeAll": false, + "name": "cluster", + "options": [], + "query": "select distinct(cluster) from metrics where $__unixEpochFilter(time_sec)", + "refresh": 2, + "regex": "", + "type": "query" + }, + { + "current": { + "text": "127.0.0.1", + "value": "127.0.0.1" + }, + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "definition": "select distinct(host) from metrics where cluster = '$cluster' and $__unixEpochFilter(time_sec) and host is not NULL and host != \"\"", + "includeAll": false, + "name": "host", + "options": [], + "query": "select distinct(host) from metrics where cluster = '$cluster' and $__unixEpochFilter(time_sec) and host is not NULL and host != \"\"", + "refresh": 2, + "regex": "", + "type": "query" + }, + { + "current": { + "text": "3307", + "value": "3307" + }, + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "definition": "", + "hide": 2, + "includeAll": false, + "name": "port", + "options": [], + "query": "select distinct(port) from metrics where cluster = '$cluster' and $__unixEpochFilter(time_sec) and port is not NULL", + "refresh": 2, + "regex": "", + "type": "query" + }, + { + "auto": true, + "auto_count": 50, + "auto_min": "1m", + "current": { + "text": "1m", + "value": "1m" + }, + "name": "interval", + "options": [ + { + "selected": true, + "text": "1m", + "value": "1m" + }, + { + "selected": false, + "text": "10m", + "value": "10m" + }, + { + "selected": false, + "text": "30m", + "value": "30m" + }, + { + "selected": false, + "text": "1h", + "value": "1h" + }, + { + "selected": false, + "text": "6h", + "value": "6h" + }, + { + "selected": false, + "text": "12h", + "value": "12h" + }, + { + "selected": false, + "text": "1d", + "value": "1d" + }, + { + "selected": false, + "text": "7d", + "value": "7d" + }, + { + "selected": false, + "text": "14d", + "value": "14d" + }, + { + "selected": false, + "text": "30d", + "value": "30d" + } + ], + "query": "1m,10m,30m,1h,6h,12h,1d,7d,14d,30d", + "refresh": 2, + "type": "interval" + } + ] + }, + "time": { + "from": "now-1h", + "to": "now" + }, + "timepicker": { + "refresh_intervals": [] + }, + "timezone": "", + "title": "SingleStore: Disk Usage", + "uid": "ccd894b4-7e3b-43e0-966b-579ac7eb3714", + "version": 1 +} diff --git a/samples/speedystore/dashboards/historicalworkloadmonitoring.json b/samples/speedystore/dashboards/historicalworkloadmonitoring.json new file mode 100644 index 00000000..e810585d --- /dev/null +++ b/samples/speedystore/dashboards/historicalworkloadmonitoring.json @@ -0,0 +1,1715 @@ +{ + "annotations": { + "list": [ + { + "builtIn": 1, + "datasource": { + "type": "datasource", + "uid": "grafana" + }, + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "name": "Annotations & Alerts", + "target": { + "limit": 100, + "matchAny": false, + "tags": [], + "type": "dashboard" + }, + "type": "dashboard" + } + ] + }, + "editable": true, + "fiscalYearStartMonth": 0, + "graphTooltip": 1, + "id": 264, + "links": [ + { + "asDropdown": true, + "icon": "external link", + "includeVars": true, + "keepTime": true, + "tags": [], + "targetBlank": true, + "title": "Dashboards", + "tooltip": "", + "type": "dashboards", + "url": "" + } + ], + "panels": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": true, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "ms" + }, + "overrides": [] + }, + "gridPos": { + "h": 9, + "w": 12, + "x": 0, + "y": 0 + }, + "id": 22, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true, + "sortBy": "Last *", + "sortDesc": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\nSELECT *,\n lag(ts) OVER w ts_l,\n lag(total) OVER w total_l,\n lag(elp) OVER w elp_l\nFROM (\n select ts,\n cluster,\n activity_name name,\n ifnull(aggregator_activity_name, activity_name) agg_name,\n database_name db,\n sum(elapsed_time_ms) as elp,\n sum(IFNULL(memory_major_faults,0)+IFNULL(success_count,0)+IFNULL(failure_count,0)) total\n from `act_samples` act\n where $__timeFilter(ts)\n and (cluster = '$cluster')\n and (database_name in ($database_name) or '%' in ($database_name) or ('-' in ($database_name) and database_name is null))\n and (activity_type = 'Query')\n and ('All' in ('$hidden_activity_name') OR 'Select a query in the table below' in ('$hidden_activity_name') OR activity_name = '$hidden_activity_name')\n and activity_name = agg_name\n group by 1,2,3,4,5\n ) q\nWINDOW w AS(partition by cluster, db, name order by ts) )\nselect $__timeGroup(ts,$interval) time_sec,\n a.db,\n sum(a.elp - a.elp_l)/sum(a.total- a.total_l) ' '\nfrom t a\nwhere a.total >a.total_l and\n a.elp >a.elp_l\ngroup by 1,2\norder by 1", + "refId": "A", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Elapsed Time per Execution by Database", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "Number of query executions", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "Query Execution Count", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": true, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 9, + "w": 12, + "x": 12, + "y": 0 + }, + "id": 21, + "options": { + "legend": { + "calcs": [ + "mean", + "max", + "min", + "sum" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\nSELECT *,\n lag(ts) OVER w ts_l,\n lag(total) OVER w total_l\nFROM (\n select ts,\n cluster,\n activity_name name,\n ifnull(aggregator_activity_name, activity_name) agg_name,\n database_name db,\n sum(IFNULL(memory_major_faults,0)+IFNULL(success_count,0)+IFNULL(failure_count,0)) total\n from `act_samples` act\n where $__timeFilter(ts)\n and (cluster = '$cluster')\n and (database_name in ($database_name) or '%' in ($database_name) or ('-' in ($database_name) and database_name is null))\n and (activity_type = 'Query')\n and ('All' in ('$hidden_activity_name') OR 'Select a query in the table below' in ('$hidden_activity_name') OR activity_name = '$hidden_activity_name')\n and activity_name = agg_name\n group by 1,2,3,4,5\n ) q\nWINDOW w AS(partition by cluster, db, name order by ts))\nselect $__timeGroup(ts,$interval) time_sec,\n sum(a.total- a.total_l) execution_count\nfrom t a\nwhere a.total>a.total_l\ngroup by 1\norder by 1", + "refId": "A", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Execution Count", + "type": "timeseries" + }, + { + "collapsed": true, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 9 + }, + "id": 19, + "panels": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "Time spent running on all the CPUs across the workspace reported by database", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "CPU time (core ms)", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": true, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "ms" + }, + "overrides": [] + }, + "gridPos": { + "h": 10, + "w": 6, + "x": 0, + "y": 10 + }, + "id": 5, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\nSELECT *,\n lag(ts) OVER w ts_l,\n lag(total) OVER w total_l,\n lag(cpu) OVER w cpu_l\nFROM (\n select ts,\n cluster,\n ifnull(act.aggregator_activity_name, act.activity_name) name,\n database_name db,\n sum(cpu_time_ms) as cpu,\n sum(if(activity_name = name, IFNULL(memory_major_faults,0), 0) +\n if(activity_name = name, IFNULL(success_count,0), 0) +\n if(activity_name = name, IFNULL(failure_count,0), 0)) total\n from `act_samples` act\n where $__timeFilter(ts)\n and (cluster = '$cluster')\n and (database_name in ($database_name) or '%' in ($database_name) or ('-' in ($database_name) and database_name is null))\n and (activity_type = 'Query')\n and ('All' in ('$hidden_activity_name') OR 'Select a query in the table below' in ('$hidden_activity_name') OR name = '$hidden_activity_name')\n group by 1,2,3,4\n ) q\nWINDOW w AS(partition by cluster, db, name order by ts) )\nselect $__timeGroup(ts,$interval) time_sec,\n a.db,\n sum(a.cpu - a.cpu_l)/sum(a.total- a.total_l) ' '\nfrom t a\nwhere a.total > 0 and\n a.total >a.total_l and\n a.cpu >a.cpu_l\ngroup by 1,2\norder by 1", + "refId": "A", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "CPU Time per Execution by Database", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": true, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "bytes" + }, + "overrides": [] + }, + "gridPos": { + "h": 10, + "w": 6, + "x": 6, + "y": 10 + }, + "id": 14, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\nSELECT *,\n lag(ts) OVER w ts_l,\n lag(elp) OVER w elp_l,\n lag(memory_bs) OVER w memory_bs_l\nFROM (\n select ts,\n cluster,\n ifnull(act.aggregator_activity_name, act.activity_name) name,\n database_name db,\n sum(if(activity_name = name, elapsed_time_ms,0)) as elp,\n sum(memory_bs) as memory_bs\n from `act_samples` act\n where $__timeFilter(ts)\n and (cluster = '$cluster')\n and (database_name in ($database_name) or '%' in ($database_name) or ('-' in ($database_name) and database_name is null))\n and (activity_type = 'Query')\n and ('All' in ('$hidden_activity_name') OR 'Select a query in the table below' in ('$hidden_activity_name') OR name = '$hidden_activity_name')\n group by 1,2,3,4\n ) q\nWINDOW w AS(partition by cluster, db, name order by ts) )\nselect $__timeGroup(ts,$interval) time_sec,\n a.db,\n sum(1000*(a.memory_bs - a.memory_bs_l))/sum(a.elp- a.elp_l) ' '\nfrom t a\nwhere a.elp > 0 and\n a.memory_bs >a.memory_bs_l and\n a.elp >a.elp_l\ngroup by 1,2\norder by 1", + "refId": "A", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Memory Bytes per Execution by Database", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": true, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "bytes" + }, + "overrides": [] + }, + "gridPos": { + "h": 10, + "w": 6, + "x": 12, + "y": 10 + }, + "id": 23, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\nSELECT *,\n lag(ts) OVER w ts_l,\n lag(total) OVER w total_l,\n lag(disk_b) OVER w disk_b_l\nFROM (\n select ts,\n cluster,\n ifnull(act.aggregator_activity_name, act.activity_name) name,\n database_name db,\n sum(disk_b) as disk_b,\n sum(if(activity_name = name, IFNULL(memory_major_faults,0), 0) +\n if(activity_name = name, IFNULL(success_count,0), 0) +\n if(activity_name = name, IFNULL(failure_count,0), 0)) total\n from `act_samples` act\n where $__timeFilter(ts)\n and (cluster = '$cluster')\n and (database_name in ($database_name) or '%' in ($database_name) or ('-' in ($database_name) and database_name is null))\n and (activity_type = 'Query')\n and ('All' in ('$hidden_activity_name') OR 'Select a query in the table below' in ('$hidden_activity_name') OR name = '$hidden_activity_name')\n group by 1,2,3,4\n ) q\nWINDOW w AS(partition by cluster, db, name order by ts) )\nselect $__timeGroup(ts,$interval) time_sec,\n a.db,\n sum(a.disk_b - a.disk_b_l)/sum(a.total- a.total_l) ' '\nfrom t a\nwhere a.total > 0 and\n a.total >a.total_l and\n a.disk_b >a.disk_b_l\ngroup by 1,2\norder by 1", + "refId": "A", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Disk Bytes per Execution by Database", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": true, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "bytes" + }, + "overrides": [] + }, + "gridPos": { + "h": 10, + "w": 6, + "x": 18, + "y": 10 + }, + "id": 17, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\nSELECT *,\n lag(ts) OVER w ts_l,\n lag(total) OVER w total_l,\n lag(network_b) OVER w network_b_l\nFROM (\n select ts,\n cluster,\n ifnull(act.aggregator_activity_name, act.activity_name) name,\n database_name db,\n sum(network_b) as network_b,\n sum(if(activity_name = name, IFNULL(memory_major_faults,0), 0) +\n if(activity_name = name, IFNULL(success_count,0), 0) +\n if(activity_name = name, IFNULL(failure_count,0), 0)) total\n from `act_samples` act\n where $__timeFilter(ts)\n and (cluster = '$cluster')\n and (database_name in ($database_name) or '%' in ($database_name) or ('-' in ($database_name) and database_name is null))\n and (activity_type = 'Query')\n and ('All' in ('$hidden_activity_name') OR 'Select a query in the table below' in ('$hidden_activity_name') OR name = '$hidden_activity_name')\n group by 1,2,3,4\n ) q\nWINDOW w AS(partition by cluster, db, name order by ts) )\nselect $__timeGroup(ts,$interval) time_sec,\n a.db,\n sum(a.network_b - a.network_b_l)/sum(a.total- a.total_l) ' '\nfrom t a\nwhere a.total > 0 and\n a.total >a.total_l and\n a.network_b >a.network_b_l\ngroup by 1,2\norder by 1", + "refId": "A", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Network Bytes per Execution by Database", + "type": "timeseries" + } + ], + "title": "Resource Usage", + "type": "row" + }, + { + "collapsed": false, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 10 + }, + "id": 20, + "panels": [], + "title": "Metrics by Query Plan", + "type": "row" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "Elapsed Time - Wall clock milliseconds elapsed during execution; CPU Time - Milliseconds spent running on all the CPUs across the workspace. This is likely to exceed the observed elapsed time because the tasks are likely to have executed concurrently", + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "custom": { + "align": "auto", + "cellOptions": { + "type": "auto" + }, + "filterable": false, + "inspect": false + }, + "mappings": [ + { + "options": { + "total_time": { + "index": 0, + "text": "Total Elapsed Time" + } + }, + "type": "value" + } + ], + "thresholds": { + "mode": "percentage", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 50 + } + ] + } + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "uniq_act" + }, + "properties": [ + { + "id": "custom.hidden", + "value": false + }, + { + "id": "custom.inspect", + "value": true + }, + { + "id": "custom.filterable", + "value": true + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "type" + }, + "properties": [ + { + "id": "custom.hidden", + "value": true + } + ] + }, + { + "matcher": { + "id": "byRegexp", + "options": "/Time/" + }, + "properties": [ + { + "id": "unit", + "value": "ms" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "query_plan" + }, + "properties": [ + { + "id": "custom.width", + "value": 545 + }, + { + "id": "links", + "value": [ + { + "targetBlank": true, + "title": "", + "url": "/d/1xNFf_iSk/historical-workload-monitoring?var-cluster=[[cluster]]&var-hidden_activity_name=${__data.fields.uniq_act}&${__url_time_range}" + } + ] + }, + { + "id": "custom.inspect", + "value": true + }, + { + "id": "custom.filterable", + "value": true + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Failure Rate" + }, + "properties": [ + { + "id": "unit", + "value": "percentunit" + }, + { + "id": "custom.width", + "value": 98 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Total Elapsed Time" + }, + "properties": [ + { + "id": "custom.width", + "value": 149 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Avg. Elapsed Time" + }, + "properties": [ + { + "id": "custom.width", + "value": 145 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Execution Count" + }, + "properties": [ + { + "id": "custom.width", + "value": 130 + }, + { + "id": "unit", + "value": "short" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Failure Count" + }, + "properties": [ + { + "id": "custom.width", + "value": 106 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "db" + }, + "properties": [ + { + "id": "custom.width", + "value": 151 + }, + { + "id": "custom.filterable", + "value": true + }, + { + "id": "custom.inspect", + "value": true + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Query_plan_id" + }, + "properties": [ + { + "id": "custom.width", + "value": 158 + } + ] + }, + { + "matcher": { + "id": "byRegexp", + "options": "/Bytes/" + }, + "properties": [ + { + "id": "unit", + "value": "bytes" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Avg. Disk Bytes" + }, + "properties": [ + { + "id": "custom.width", + "value": 127 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Total CPU Time" + }, + "properties": [ + { + "id": "custom.width", + "value": 124 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Avg. CPU Time" + }, + "properties": [ + { + "id": "custom.width", + "value": 119 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Avg. Disk Read Bytes (logical)" + }, + "properties": [ + { + "id": "custom.width", + "value": 224 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Avg. Disk Write Bytes (logical)" + }, + "properties": [ + { + "id": "custom.width", + "value": 225 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Avg. Network Recieved Bytes (logical)" + }, + "properties": [ + { + "id": "custom.width", + "value": 281 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Avg. Network Send Bytes (logical)" + }, + "properties": [ + { + "id": "custom.width", + "value": 252 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Avg. Network Received Bytes (logical)" + }, + "properties": [ + { + "id": "custom.width", + "value": 275 + } + ] + } + ] + }, + "gridPos": { + "h": 14, + "w": 24, + "x": 0, + "y": 11 + }, + "id": 4, + "options": { + "cellHeight": "sm", + "footer": { + "countRows": false, + "enablePagination": true, + "fields": "", + "reducer": [ + "sum" + ], + "show": false + }, + "showHeader": true, + "sortBy": [] + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "table", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "select res.*, q.query_text as query_plan from ( \nWITH t AS (\n\nSELECT *,\nlag(cpu) OVER w cpu_l,\nlag(disk_b) OVER w disk_b_l,\nlag(disk_logical_read_b) OVER w disk_logical_read_b_l,\nlag(disk_logical_write_b) OVER w disk_logical_write_b_l,\nlag(network_b) OVER w network_b_l,\nlag(network_logical_recv_b) OVER w network_logical_recv_b_l,\nlag(network_logical_send_b) OVER w network_logical_send_b_l,\nlag(memory_bs) OVER w memory_bs_l,\nlag(elp) OVER w elp_l,\nlag(fail) OVER w fail_l,\nlag(total) OVER w total_l\nFROM\n(\nselect\nact.cluster,\nact.ts,\nifnull(database_name,'-') db,\nifnull(act.aggregator_activity_name, act.activity_name) uniq_act,\nsum(cpu_time_ms) cpu,\nsum(disk_b) disk_b,\nsum(disk_logical_read_b) disk_logical_read_b,\nsum(disk_logical_write_b) disk_logical_write_b,\nsum(network_b) network_b,\nsum(network_logical_recv_b) network_logical_recv_b,\nsum(network_logical_send_b) network_logical_send_b,\nsum(memory_bs) memory_bs,\nsum(if (activity_name = uniq_act, elapsed_time_ms, 0)) elp,\nsum(if(activity_name = uniq_act, IFNULL(failure_count,0), 0)) fail,\nsum(if(activity_name = uniq_act, IFNULL(memory_major_faults,0), 0) +\n if(activity_name = uniq_act, IFNULL(success_count,0), 0) +\n if(activity_name = uniq_act, IFNULL(failure_count,0), 0)) total\nfrom `act_samples` act\nwhere $__timeFilter(act.ts)\nand (cluster = '$cluster')\nand (database_name in ($database_name) or '%' in ($database_name) or ('-' in ($database_name) and database_name is null))\nand ('All' in ('$hidden_activity_name') OR 'Select a query in the table below' in ('$hidden_activity_name') \n OR (uniq_act in ('$hidden_activity_name'))\n )\nand (act.activity_type = 'Query')\ngroup by 1,2,3,4\n) q\nWINDOW w AS(partition by cluster, db, uniq_act order by ts) \n)\nselect \ndb, \nelp as 'Total Elapsed Time', \nelp/total 'Avg. Elapsed Time', \ncpu as 'Total CPU Time', \ncpu/total as 'Avg. CPU Time',\ndisk_b/total as 'Avg. Disk Bytes',\ndisk_logical_read_b/total as 'Avg. Disk Read Bytes (logical)',\ndisk_logical_write_b/total as 'Avg. Disk Write Bytes (logical)',\nnetwork_logical_recv_b/total as 'Avg. Network Received Bytes (logical)',\nnetwork_logical_send_b/total as 'Avg. Network Send Bytes (logical)',\nnetwork_b/total as 'Avg. Network Bytes',\n(1000*memory_bs)/elp as 'Avg. Memory Bytes',\ntotal as 'Execution Count', \nfail as 'Failure Count',\nfail/total as 'Failure Rate', \nuniq_act\nFROM (select \na.cluster, a.db,\nsum(if(a.cpu>a.cpu_l, a.cpu- a.cpu_l,0)) cpu, \nsum(if(a.disk_b>a.disk_b_l, a.disk_b- a.disk_b_l,0)) disk_b, \nsum(if(a.disk_logical_read_b>a.disk_logical_read_b_l, a.disk_logical_read_b- a.disk_logical_read_b_l,0)) disk_logical_read_b, \nsum(if(a.disk_logical_write_b>a.disk_logical_write_b_l, a.disk_logical_write_b- a.disk_logical_write_b_l,0)) disk_logical_write_b, \nsum(if(a.network_b>a.network_b_l, a.network_b- a.network_b_l,0)) network_b, \nsum(if(a.network_logical_recv_b>a.network_logical_recv_b_l, a.network_logical_recv_b- a.network_logical_recv_b_l,0)) network_logical_recv_b, \nsum(if(a.network_logical_send_b>a.network_logical_send_b_l, a.network_logical_send_b- a.network_logical_send_b_l,0)) network_logical_send_b,\nsum(if(a.memory_bs>a.memory_bs_l, a.memory_bs- a.memory_bs_l,0)) memory_bs,\nsum(if(a.elp>a.elp_l, a.elp- a.elp_l,0)) elp, \nsum(if(a.fail>a.fail_l, a.fail- a.fail_l,0)) fail, \nsum(if(a.total>a.total_l, a.total- a.total_l,0)) total,\na.uniq_act\nfrom t a\nwhere total > 0 and elp > 0\ngroup by uniq_act) subq\nWHERE (subq.total > 0)\n) res, `mv_queries` q\nwhere res.uniq_act = q.activity_name\norder by `$sortby` desc\nlimit $topn", + "refId": "A", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Metrics by Query Plan", + "transformations": [ + { + "id": "organize", + "options": { + "excludeByName": {}, + "indexByName": { + "Avg. CPU Time": 7, + "Avg. Disk Bytes": 9, + "Avg. Disk Read Bytes (logical)": 10, + "Avg. Disk Write Bytes (logical)": 11, + "Avg. Elapsed Time": 5, + "Avg. Memory Bytes": 8, + "Avg. Network Bytes": 12, + "Avg. Network Received Bytes (logical)": 13, + "Avg. Network Send Bytes (logical)": 14, + "Execution Count": 3, + "Failure Count": 15, + "Failure Rate": 16, + "Total CPU Time": 6, + "Total Elapsed Time": 4, + "db": 0, + "query_plan": 2, + "uniq_act": 1 + }, + "renameByName": { + "uniq_act": "Query_plan_id" + } + } + } + ], + "type": "table" + }, + { + "fieldConfig": { + "defaults": {}, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 25 + }, + "id": 7, + "options": { + "code": { + "language": "sql", + "showLineNumbers": true, + "showMiniMap": true + }, + "content": "${activity_name}", + "mode": "code" + }, + "pluginVersion": "12.1.0", + "title": "select a query in the table above to view its text", + "type": "text" + }, + { + "fieldConfig": { + "defaults": {}, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 25 + }, + "id": 9, + "options": { + "code": { + "language": "plaintext", + "showLineNumbers": false, + "showMiniMap": false + }, + "content": "**Elapsed Time** - Wall clock milliseconds elapsed during execution \n\n**CPU Time** - The units of CPU Time are 'core ms' because it's the time spent running on each CPU core across the workspace. This is likely to exceed the observed elapsed time because the tasks are likely to have executed concurrently\n\n**query_plan** - Query plans are compiled to machine code from the query by parameterizing the values and cached to expedite subsequent executions.
\n      For Instance: Let's say you execute the below 2 queries,
\n        Select * from table_a where colum_a = \"Value_A\"
\n        Select * from table_a where column_a = \"Value_B\"
\n      Query Plan for the above two queries would look something like the below,
\n        Select * from table_a where column_a = ^
", + "mode": "markdown" + }, + "pluginVersion": "12.1.0", + "title": "Notes", + "type": "text" + } + ], + "preload": false, + "refresh": "", + "schemaVersion": 41, + "tags": ["singlestore","sas-speedystore"], + "templating": { + "list": [ + { + "current": { + "text": "0cda4387b1a7", + "value": "0cda4387b1a7" + }, + "hide": 2, + "name": "viewName", + "options": [ + { + "selected": true, + "text": "0cda4387b1a7", + "value": "0cda4387b1a7" + } + ], + "query": "0cda4387b1a7", + "type": "textbox" + }, + { + "current": { + "text": "memsql_cluster", + "value": "memsql_cluster" + }, + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "definition": "select distinct(cluster) from `act_samples` where $__timeFilter(ts)", + "includeAll": false, + "name": "cluster", + "options": [], + "query": "select distinct(cluster) from `act_samples` where $__timeFilter(ts)", + "refresh": 1, + "regex": "", + "type": "query" + }, + { + "allValue": "'%'", + "current": { + "text": [ + "All" + ], + "value": [ + "$__all" + ] + }, + "datasource": { + "uid": "s2monitoring" + }, + "definition": "select distinct(ifnull(database_name,'-')) from `act_samples` where (cluster = '$cluster') and $__timeFilter(ts)", + "includeAll": true, + "multi": true, + "name": "database_name", + "options": [], + "query": "select distinct(ifnull(database_name,'-')) from `act_samples` where (cluster = '$cluster') and $__timeFilter(ts)", + "refresh": 2, + "regex": "", + "sort": 1, + "type": "query" + }, + { + "auto": true, + "auto_count": 100, + "auto_min": "30s", + "current": { + "text": "1m", + "value": "1m" + }, + "name": "interval", + "options": [ + { + "selected": true, + "text": "1m", + "value": "1m" + }, + { + "selected": false, + "text": "5m", + "value": "5m" + }, + { + "selected": false, + "text": "10m", + "value": "10m" + }, + { + "selected": false, + "text": "30m", + "value": "30m" + }, + { + "selected": false, + "text": "1h", + "value": "1h" + }, + { + "selected": false, + "text": "3h", + "value": "3h" + }, + { + "selected": false, + "text": "6h", + "value": "6h" + }, + { + "selected": false, + "text": "12h", + "value": "12h" + }, + { + "selected": false, + "text": "1d", + "value": "1d" + } + ], + "query": "1m,5m,10m,30m,1h,3h,6h,12h,1d", + "refresh": 2, + "type": "interval" + }, + { + "current": { + "text": "All", + "value": "All" + }, + "description": "Hidden activity name to get query", + "includeAll": false, + "label": "Query Plan ID", + "name": "hidden_activity_name", + "options": [ + { + "selected": true, + "text": "All", + "value": "All" + }, + { + "selected": false, + "text": "Select a query in the table below", + "value": "Select a query in the table below" + } + ], + "query": "All, Select a query in the table below", + "type": "custom" + }, + { + "current": { + "text": "All", + "value": "All" + }, + "hide": 2, + "includeAll": false, + "label": "Query Plan", + "name": "query_plan_hidden", + "options": [ + { + "selected": true, + "text": "All", + "value": "All" + }, + { + "selected": false, + "text": "Select a query in the table below", + "value": "Select a query in the table below" + } + ], + "query": "All, Select a query in the table below", + "type": "custom" + }, + { + "current": { + "text": "None", + "value": "" + }, + "datasource": { + "uid": "s2monitoring" + }, + "definition": "select case when '$hidden_activity_name' = 'All' then 'All' else query_text end \nfrom `mv_queries` \nwhere (cluster = '$cluster') and \nactivity_name in ('$hidden_activity_name')", + "hide": 2, + "includeAll": false, + "name": "activity_name", + "options": [], + "query": "select case when '$hidden_activity_name' = 'All' then 'All' else query_text end \nfrom `mv_queries` \nwhere (cluster = '$cluster') and \nactivity_name in ('$hidden_activity_name')", + "refresh": 2, + "regex": "", + "type": "query" + }, + { + "current": { + "text": "100", + "value": "100" + }, + "includeAll": false, + "label": "Top N (Query Plan)", + "name": "topn", + "options": [ + { + "selected": true, + "text": "100", + "value": "100" + }, + { + "selected": false, + "text": "250", + "value": "250" + }, + { + "selected": false, + "text": "500", + "value": "500" + }, + { + "selected": false, + "text": "1000", + "value": "1000" + } + ], + "query": "100,250,500,1000", + "type": "custom" + }, + { + "current": { + "text": "Execution Count", + "value": "Execution Count" + }, + "includeAll": false, + "label": "Sort By", + "name": "sortby", + "options": [ + { + "selected": false, + "text": "Avg. Elapsed Time", + "value": "Avg. Elapsed Time" + }, + { + "selected": false, + "text": "Total Elapsed Time", + "value": "Total Elapsed Time" + }, + { + "selected": false, + "text": "Avg. CPU Time", + "value": "Avg. CPU Time" + }, + { + "selected": false, + "text": "Total CPU Time", + "value": "Total CPU Time" + }, + { + "selected": false, + "text": "Avg. Memory Bytes", + "value": "Avg. Memory Bytes" + }, + { + "selected": false, + "text": "Avg. Disk Bytes", + "value": "Avg. Disk Bytes" + }, + { + "selected": false, + "text": "Avg. Disk Read Bytes (logical)", + "value": "Avg. Disk Read Bytes (logical)" + }, + { + "selected": false, + "text": "Avg. Disk Write Bytes (logical)", + "value": "Avg. Disk Write Bytes (logical)" + }, + { + "selected": false, + "text": "Avg. Network Bytes", + "value": "Avg. Network Bytes" + }, + { + "selected": false, + "text": "Avg. Network Received Bytes (logical)", + "value": "Avg. Network Received Bytes (logical)" + }, + { + "selected": false, + "text": "Avg. Network Send Bytes (logical)", + "value": "Avg. Network Send Bytes (logical)" + }, + { + "selected": true, + "text": "Execution Count", + "value": "Execution Count" + }, + { + "selected": false, + "text": "Failure Count", + "value": "Failure Count" + }, + { + "selected": false, + "text": "Failure Rate", + "value": "Failure Rate" + } + ], + "query": "Avg. Elapsed Time,Total Elapsed Time,Avg. CPU Time,Total CPU Time,Avg. Memory Bytes, Avg. Disk Bytes, Avg. Disk Read Bytes (logical), Avg. Disk Write Bytes (logical), Avg. Network Bytes,Avg. Network Received Bytes (logical), Avg. Network Send Bytes (logical), Execution Count,Failure Count,Failure Rate", + "type": "custom" + } + ] + }, + "time": { + "from": "now-30m", + "to": "now" + }, + "timepicker": {}, + "timezone": "", + "title": "SingleStore: Historical Workload Monitoring", + "uid": "1xNFf_iSk", + "version": 1 +} diff --git a/samples/speedystore/dashboards/memoryusage.json b/samples/speedystore/dashboards/memoryusage.json new file mode 100644 index 00000000..09388de1 --- /dev/null +++ b/samples/speedystore/dashboards/memoryusage.json @@ -0,0 +1,810 @@ +{ + "annotations": { + "list": [ + { + "builtIn": 1, + "datasource": { + "type": "datasource", + "uid": "grafana" + }, + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "name": "Annotations & Alerts", + "target": { + "limit": 100, + "matchAny": false, + "tags": [], + "type": "dashboard" + }, + "type": "dashboard" + } + ] + }, + "editable": true, + "fiscalYearStartMonth": 0, + "graphTooltip": 0, + "id": 265, + "links": [], + "panels": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "Total_Server_Memory used, out of Maximum_Memory limit.\nUsed memory is broken into data, queries, or internal.", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 35, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "normal" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "mbytes" + }, + "overrides": [ + { + "matcher": { + "id": "byRegexp", + "options": "/MAXIMUM_MEMORY/" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "rgb(202, 199, 199)", + "mode": "fixed" + } + }, + { + "id": "custom.fillOpacity", + "value": 0 + }, + { + "id": "custom.lineWidth", + "value": 5 + }, + { + "id": "custom.stacking", + "value": { + "group": "A", + "mode": "none" + } + }, + { + "id": "custom.lineStyle", + "value": { + "dash": [ + 10, + 10 + ], + "fill": "dash" + } + } + ] + }, + { + "matcher": { + "id": "byRegexp", + "options": "/Total Memory Used/" + }, + "properties": [ + { + "id": "custom.fillOpacity", + "value": 0 + }, + { + "id": "custom.lineWidth", + "value": 2 + }, + { + "id": "custom.stacking", + "value": { + "group": "A", + "mode": "none" + } + } + ] + } + ] + }, + "gridPos": { + "h": 9, + "w": 12, + "x": 0, + "y": 0 + }, + "id": 9, + "options": { + "legend": { + "calcs": [ + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true, + "sortBy": "Last *", + "sortDesc": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "hide": false, + "rawQuery": true, + "rawSql": "SELECT\n time_sec,\n concat(\n case when name = 'memsql_variables_maximum_memory' then 'MAXIMUM_MEMORY'\n when name = 'memsql_status_total_server_memory' then 'Total Memory Used'\n when name = 'memsql_status_buffer_manager_cached_memory' then 'Cached Memory'\n when name = 'memsql_status_alloc_table_memory' then 'Data'\n when name in ('memsql_status_alloc_query_execution',\n 'memsql_status_alloc_aggregator_query_execution',\n 'memsql_status_alloc_columnar_query_execution',\n 'memsql_status_alloc_query_background_tasks',\n 'memsql_status_alloc_query_execution_temp_table',\n 'memsql_status_alloc_query_ingest',\n 'memsql_status_alloc_plan_cache',\n 'memsql_status_alloc_unit_images',\n 'memsql_status_alloc_object_code_images',\n 'memsql_status_alloc_compiled_unit_sections',\n 'memsql_status_alloc_analyze',\n 'memsql_status_alloc_arena',\n 'memsql_status_alloc_autostats',\n 'memsql_status_alloc_backup_restore',\n 'memsql_status_alloc_column_store',\n 'memsql_status_alloc_columnstore_blob_cache',\n 'memsql_status_alloc_distributed_transaction',\n 'memsql_status_alloc_dqo_cache',\n 'memsql_status_alloc_external_formatter',\n 'memsql_status_alloc_external_functions',\n 'memsql_status_alloc_fulltext_search',\n 'memsql_status_alloc_json',\n 'memsql_status_alloc_mmap_memory',\n 'memsql_status_alloc_mmap_wasm',\n 'memsql_status_alloc_parametrizer',\n 'memsql_status_alloc_pipelines',\n 'memsql_status_alloc_profile_stats',\n 'memsql_status_alloc_shared_memory',\n 'memsql_status_alloc_spatial',\n 'memsql_status_alloc_spill',\n 'memsql_status_alloc_stack_first_segmented_array',\n 'memsql_status_alloc_warnings') then 'Query' else 'Other' end\n , ' ', trim_host(host), ':', port, ' (', role, ')') as metric,\n sum(intval) as value\nFROM metrics\nWHERE cluster = '$cluster'\nand extractor = 'memsql'\nand host = '$host'\nand port = '$port'\nand name in ('memsql_variables_maximum_memory',\n 'memsql_status_total_server_memory',\n 'memsql_status_buffer_manager_cached_memory',\n 'memsql_status_alloc_aggregator_query_execution',\n 'memsql_status_alloc_columnar_query_execution',\n 'memsql_status_alloc_query_background_tasks',\n 'memsql_status_alloc_query_execution_temp_table',\n 'memsql_status_alloc_query_ingest',\n 'memsql_status_alloc_column_store',\n 'memsql_status_alloc_analyze',\n 'memsql_status_alloc_arena',\n 'memsql_status_alloc_autostats',\n 'memsql_status_alloc_background_tasks',\n 'memsql_status_alloc_backup_restore',\n 'memsql_status_alloc_client_connection',\n 'memsql_status_alloc_code_generator',\n 'memsql_status_alloc_columnstore_blob_cache',\n 'memsql_status_alloc_databases_list_entry',\n 'memsql_status_alloc_db_tasks',\n 'memsql_status_alloc_distributed_transaction',\n 'memsql_status_alloc_dqo_cache',\n 'memsql_status_alloc_external_formatter',\n 'memsql_status_alloc_fulltext_search',\n 'memsql_status_alloc_global_func',\n 'memsql_status_alloc_json',\n 'memsql_status_alloc_none',\n 'memsql_status_alloc_parametrizer',\n 'memsql_status_alloc_pipelines',\n 'memsql_status_alloc_profile_stats',\n 'memsql_status_alloc_protocol_packet',\n 'memsql_status_alloc_replication',\n 'memsql_status_alloc_security',\n 'memsql_status_alloc_segmented_list',\n 'memsql_status_alloc_sharding_partitions',\n 'memsql_status_alloc_sharding_tables',\n 'memsql_status_alloc_show_statement',\n 'memsql_status_alloc_skynet_consensus',\n 'memsql_status_alloc_snapshot',\n 'memsql_status_alloc_spatial',\n 'memsql_status_alloc_stack_first_segmented_array',\n 'memsql_status_alloc_system_tasks',\n 'memsql_status_alloc_table_metadata_cache',\n 'memsql_status_alloc_transaction',\n 'memsql_status_alloc_warnings',\n 'memsql_status_buffer_manager_unrecycled_memory',\n 'memsql_status_alloc_table_memory',\n 'memsql_status_malloc_active_memory',\n 'memsql_status_alloc_replication_large',\n 'memsql_status_alloc_thread_stacks',\n 'memsql_status_alloc_durability_large',\n 'memsql_status_alloc_external_functions',\n 'memsql_status_alloc_mmap_memory',\n 'memsql_status_alloc_mmap_wasm',\n 'memsql_status_alloc_shared_memory',\n 'memsql_status_alloc_spill',\n 'memsql_status_alloc_thread_signal_handler_altstack',\n 'memsql_status_alloc_unit_ifn_thunks',\n 'memsql_status_total_io_pool_memory',\n 'memsql_status_alloc_compiled_unit_sections',\n 'memsql_status_alloc_object_code_images',\n 'memsql_status_alloc_plan_cache',\n 'memsql_status_alloc_query_execution',\n 'memsql_status_alloc_unit_images')\nand $__unixEpochFilter(time_sec)\ngroup by 1,2\nORDER BY 1,2\n", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Total Memory Used vs Total Limit", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "Memory used for query execution, code gen, and plancaching, out of Maximum_Memory limit. Query execution can consume the cache, as well as any available memory less than the total limit.", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 20, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "normal" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "mbytes" + }, + "overrides": [ + { + "matcher": { + "id": "byRegexp", + "options": "/MAXIMUM_MEMORY/" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "rgb(202, 199, 199)", + "mode": "fixed" + } + }, + { + "id": "custom.fillOpacity", + "value": 0 + }, + { + "id": "custom.lineWidth", + "value": 5 + }, + { + "id": "custom.stacking", + "value": { + "group": "A", + "mode": "none" + } + }, + { + "id": "custom.lineStyle", + "value": { + "dash": [ + 10, + 10 + ], + "fill": "dash" + } + } + ] + } + ] + }, + "gridPos": { + "h": 9, + "w": 12, + "x": 12, + "y": 0 + }, + "id": 10, + "options": { + "legend": { + "calcs": [ + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true, + "sortBy": "Last *", + "sortDesc": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "hide": false, + "rawQuery": true, + "rawSql": "SELECT\n time_sec,\n concat(\n case \n when name like 'memsql_status_alloc%' then trim_metric(name, 'memsql_status_alloc_')\n when name like 'memsql_variables_%' then 'MAXIMUM_MEMORY'\n end\n , ' ', trim_host(host), ':', port, ' (', role, ')') as metric,\n sum(intval) as value\nFROM metrics\nWHERE cluster = '$cluster'\nand extractor = 'memsql'\nand host = '$host'\nand port = '$port'\nand name in ('memsql_variables_maximum_memory',\n 'memsql_status_alloc_query_execution',\n 'memsql_status_alloc_aggregator_query_execution',\n 'memsql_status_alloc_columnar_query_execution',\n 'memsql_status_alloc_query_background_tasks',\n 'memsql_status_alloc_query_execution_temp_table',\n 'memsql_status_alloc_query_ingest',\n 'memsql_status_alloc_plan_cache',\n 'memsql_status_alloc_unit_images',\n 'memsql_status_alloc_object_code_images',\n 'memsql_status_alloc_compiled_unit_sections',\n 'memsql_status_alloc_analyze',\n 'memsql_status_alloc_arena',\n 'memsql_status_alloc_autostats',\n 'memsql_status_alloc_backup_restore',\n 'memsql_status_alloc_column_store',\n 'memsql_status_alloc_columnstore_blob_cache',\n 'memsql_status_alloc_distributed_transaction',\n 'memsql_status_alloc_dqo_cache',\n 'memsql_status_alloc_external_formatter',\n 'memsql_status_alloc_external_functions',\n 'memsql_status_alloc_fulltext_search',\n 'memsql_status_alloc_json',\n 'memsql_status_alloc_mmap_memory',\n 'memsql_status_alloc_mmap_wasm',\n 'memsql_status_alloc_parametrizer',\n 'memsql_status_alloc_pipelines',\n 'memsql_status_alloc_profile_stats',\n 'memsql_status_alloc_shared_memory',\n 'memsql_status_alloc_spatial',\n 'memsql_status_alloc_spill',\n 'memsql_status_alloc_stack_first_segmented_array',\n 'memsql_status_alloc_warnings')\nand $__unixEpochFilter(time_sec)\ngroup by 1,2\nORDER BY 1,2\n", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Query Memory Used vs Total Limit", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "Table_Memory used, out of Maximum_Table_Memory limit.\nTable_primary and Variable should be the greatest allocators. The others should be small.", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 20, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "normal" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "mbytes" + }, + "overrides": [ + { + "matcher": { + "id": "byRegexp", + "options": "/TABLE_MEMORY/" + }, + "properties": [ + { + "id": "custom.fillOpacity", + "value": 0 + }, + { + "id": "custom.lineWidth", + "value": 5 + }, + { + "id": "custom.stacking", + "value": { + "group": "A", + "mode": "normal" + } + } + ] + }, + { + "matcher": { + "id": "byRegexp", + "options": "/MAXIMUM_TABLE_MEMORY/" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "rgb(202, 199, 199)", + "mode": "fixed" + } + }, + { + "id": "custom.fillOpacity", + "value": 0 + }, + { + "id": "custom.lineWidth", + "value": 5 + }, + { + "id": "custom.stacking", + "value": { + "group": "B", + "mode": "normal" + } + }, + { + "id": "custom.lineStyle", + "value": { + "dash": [ + 10, + 10 + ], + "fill": "dash" + } + } + ] + } + ] + }, + "gridPos": { + "h": 9, + "w": 12, + "x": 0, + "y": 9 + }, + "id": 8, + "options": { + "legend": { + "calcs": [ + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true, + "sortBy": "Last *", + "sortDesc": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "hide": false, + "rawQuery": true, + "rawSql": "SELECT\n time_sec,\n intval as value,\n concat(\n case \n when name like 'memsql_status_alloc_%' then trim_metric(name, 'memsql_status_alloc_')\n when name like 'memsql_variables_%' then trim_metric(name, 'memsql_variables_')\n end\n , ' ', trim_host(host), ':', port, ' (', role, ')') as metric\nFROM metrics\nWHERE cluster = '$cluster'\nand extractor = 'memsql'\nand host = '$host'\nand port = '$port'\nand name in ( 'memsql_variables_maximum_table_memory',\n 'memsql_status_alloc_variable',\n 'memsql_status_alloc_table_primary',\n 'memsql_status_alloc_deleted_version',\n 'memsql_status_alloc_skiplist_tower',\n 'memsql_status_alloc_large_variable',\n 'memsql_status_alloc_hash_buckets',\n 'memsql_status_alloc_internal_key_node',\n 'memsql_status_alloc_table_autostats',\n 'memsql_status_alloc_varbuffer_cache')\nand $__unixEpochFilter(time_sec)\nORDER BY time_sec ASC, value ASC\n", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Data Memory Used vs Data Limit", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "Memory used for internal MemSQL allocations, out of Maximum_Memory limit.\nThe cache holds memory in MemSQL for faster internal allocation (eg for executing queries quickly).", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 20, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "normal" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "mbytes" + }, + "overrides": [ + { + "matcher": { + "id": "byRegexp", + "options": "/MAXIMUM_MEMORY/" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "rgb(202, 199, 199)", + "mode": "fixed" + } + }, + { + "id": "custom.fillOpacity", + "value": 0 + }, + { + "id": "custom.lineWidth", + "value": 5 + }, + { + "id": "custom.stacking", + "value": { + "group": "A", + "mode": "none" + } + }, + { + "id": "custom.lineStyle", + "value": { + "dash": [ + 10, + 10 + ], + "fill": "dash" + } + } + ] + } + ] + }, + "gridPos": { + "h": 9, + "w": 12, + "x": 12, + "y": 9 + }, + "id": 12, + "options": { + "legend": { + "calcs": [ + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true, + "sortBy": "Last *", + "sortDesc": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "alias": "", + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "hide": false, + "rawQuery": true, + "rawSql": "SELECT\n time_sec,\n intval as value,\n concat(\n case \n when name like 'memsql_status_alloc%' then trim_metric(name, 'memsql_status_alloc_')\n when name like 'memsql_status_buffer_manager_%' then trim_metric(name, 'memsql_status_buffer_manager_')\n when name like 'memsql_status_%' then trim_metric(name, 'memsql_status_')\n when name like 'memsql_variables_%' then trim_metric(name, 'memsql_variables_')\n end\n , ' ', trim_host(host), ':', port, ' (', role, ')') as metric\nFROM metrics\nWHERE cluster = '$cluster'\nand extractor = 'memsql'\nand host = '$host'\nand port = '$port'\nand name in ('memsql_variables_maximum_memory',\n 'memsql_status_alloc_background_tasks',\n 'memsql_status_alloc_client_connection',\n 'memsql_status_alloc_code_generator',\n 'memsql_status_alloc_databases_list_entry',\n 'memsql_status_alloc_db_tasks',\n 'memsql_status_alloc_global_func',\n 'memsql_status_alloc_none',\n 'memsql_status_alloc_protocol_packet',\n 'memsql_status_alloc_replication',\n 'memsql_status_alloc_security',\n 'memsql_status_alloc_segmented_list',\n 'memsql_status_alloc_sharding_partitions',\n 'memsql_status_alloc_sharding_tables',\n 'memsql_status_alloc_show_statement',\n 'memsql_status_alloc_skynet_consensus',\n 'memsql_status_alloc_snapshot',\n 'memsql_status_alloc_system_tasks',\n 'memsql_status_alloc_table_metadata_cache',\n 'memsql_status_alloc_transaction',\n 'memsql_status_buffer_manager_unrecycled_memory',\n 'memsql_status_malloc_active_memory',\n 'memsql_status_alloc_replication_large',\n 'memsql_status_alloc_thread_stacks',\n 'memsql_status_alloc_durability_large',\n 'memsql_status_alloc_thread_signal_handler_altstack',\n 'memsql_status_alloc_unit_ifn_thunks',\n 'memsql_status_total_io_pool_memory')\nand $__unixEpochFilter(time_sec)\nORDER BY time_sec ASC, value ASC\n", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Internal Memory Allocators vs Total Limit", + "type": "timeseries" + } + ], + "preload": false, + "refresh": "", + "schemaVersion": 41, + "tags": ["singlestore","sas-speedystore"], + "templating": { + "list": [ + { + "current": { + "text": "memsql_cluster", + "value": "memsql_cluster" + }, + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "definition": "", + "includeAll": false, + "name": "cluster", + "options": [], + "query": "select distinct(cluster) from metrics where $__unixEpochFilter(time_sec)", + "refresh": 2, + "regex": "", + "type": "query" + }, + { + "current": { + "text": "127.0.0.1", + "value": "127.0.0.1" + }, + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "definition": "select distinct(host) from metrics where cluster = '$cluster' and $__unixEpochFilter(time_sec) and host is not NULL and host != \"\"", + "includeAll": false, + "name": "host", + "options": [], + "query": "select distinct(host) from metrics where cluster = '$cluster' and $__unixEpochFilter(time_sec) and host is not NULL and host != \"\"", + "refresh": 2, + "regex": "", + "type": "query" + }, + { + "current": { + "text": "3307", + "value": "3307" + }, + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "definition": "", + "includeAll": false, + "name": "port", + "options": [], + "query": "select distinct(port) from metrics where cluster = '$cluster' and $__unixEpochFilter(time_sec) and port is not NULL", + "refresh": 2, + "regex": "", + "type": "query" + } + ] + }, + "time": { + "from": "now-1h", + "to": "now" + }, + "timepicker": { + "refresh_intervals": [] + }, + "timezone": "", + "title": "SingleStore: Memory Usage", + "uid": "memory-usage", + "version": 1 +} diff --git a/samples/speedystore/dashboards/pipelineperformance.json b/samples/speedystore/dashboards/pipelineperformance.json new file mode 100644 index 00000000..b06ab853 --- /dev/null +++ b/samples/speedystore/dashboards/pipelineperformance.json @@ -0,0 +1,1508 @@ +{ + "annotations": { + "list": [ + { + "builtIn": 1, + "datasource": { + "type": "grafana", + "uid": "-- Grafana --" + }, + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "name": "Annotations & Alerts", + "target": { + "limit": 100, + "matchAny": false, + "tags": [], + "type": "dashboard" + }, + "type": "dashboard" + } + ] + }, + "editable": true, + "fiscalYearStartMonth": 0, + "graphTooltip": 0, + "id": 266, + "links": [ + { + "asDropdown": false, + "icon": "external link", + "includeVars": false, + "keepTime": false, + "tags": [], + "targetBlank": true, + "title": "Documentation", + "tooltip": "", + "type": "link", + "url": "https://docs.singlestore.com/managed-service/en/reference/troubleshooting-reference/pipeline-errors/detect-and-address-slow-performance-and-high-memory-usage-of-pipelines.html" + }, + { + "asDropdown": true, + "icon": "external link", + "includeVars": true, + "keepTime": true, + "tags": [], + "targetBlank": true, + "title": "Dashboards", + "tooltip": "", + "type": "dashboards", + "url": "" + } + ], + "panels": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 0, + "y": 0 + }, + "id": 7, + "options": { + "legend": { + "calcs": [ + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "dataset": "metrics", + "datasource": { + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "table", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\nSELECT *,\n lag(ts) OVER w ts_l,\n lag(total) OVER w total_l\nFROM (\n select ts,\n cluster,\n activity_name name,\n database_name db,\n max(IFNULL(run_count,0)+IFNULL(success_count,0)+IFNULL(failure_count,0)) total\n from `act_samples` act\n where $__timeFilter(ts)\n and (cluster = '$cluster')\n and activity_name like '%RunPipeline%' \n and (activity_name like '%${pipeline_name:raw}%' or '%' in ($pipeline_name))\n and (database_name in ($database_name) or '%' in ($database_name) or ('-' in ($database_name) and database_name is null))\n and (activity_name = aggregator_activity_name or aggregator_activity_name is NULL)\n group by cluster, database_name, activity_type, activity_name, ts) q\nWINDOW w AS(partition by cluster, db, name order by ts) )\nselect $__timeGroup(time_sec,$interval) time_sec,\n sum(exec_cnt) execution_count\nFROM (select a.cluster,\n a.db,\n a.name,\n a.ts time_sec,\n sum(if(a.total>a.total_l, a.total- a.total_l, 0)) exec_cnt\n from t a\n group by 1, 2, 3, 4) subq\nwhere exec_cnt > 0\ngroup by 1\norder by 1", + "refId": "A", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [ + { + "name": "avg_cpu_time_ms", + "type": "functionParameter" + } + ], + "type": "function" + }, + { + "parameters": [ + { + "name": "avg_memory_mb", + "type": "functionParameter" + } + ], + "type": "function" + }, + { + "parameters": [ + { + "name": "ts", + "type": "functionParameter" + } + ], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "pipeline_perfmetrics", + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "hide": false, + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\nSELECT *,\n lag(ts) OVER w ts_l,\n lag(total) OVER w total_l\nFROM (\n select ts,\n cluster,\n activity_name name,\n database_name db,\n max(IFNULL(failure_count,0)) total\n from `act_samples` act\n where $__timeFilter(ts)\n and (cluster = '$cluster')\n and activity_name like '%RunPipeline%' \n and (activity_name like '%${pipeline_name:raw}%' or '%' in ($pipeline_name))\n and (database_name in ($database_name) or '%' in ($database_name) or ('-' in ($database_name) and database_name is null))\n and (activity_name = aggregator_activity_name or aggregator_activity_name is NULL)\n group by cluster, database_name, activity_type, activity_name, ts) q\nWINDOW w AS(partition by cluster, db, name order by ts) )\nselect $__timeGroup(time_sec,$interval) time_sec,\n sum(exec_cnt) as failure_count\nFROM (select a.cluster,\n a.db,\n a.name,\n a.ts time_sec,\n sum(if(a.total>a.total_l, a.total- a.total_l, 0)) exec_cnt\n from t a\n group by 1, 2, 3, 4) subq\ngroup by 1\norder by 1", + "refId": "B", + "select": [ + [ + { + "params": [ + "cpu_time_ms" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "08c783c1c927b993824179b6d7a31908_act_samples", + "timeColumn": "ts", + "timeColumnType": "timestamp", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Execution Count", + "transformations": [ + { + "id": "convertFieldType", + "options": { + "conversions": [ + { + "destinationType": "time", + "targetField": "time_sec" + } + ], + "fields": {} + } + } + ], + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "axisSoftMin": 0, + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "ms" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 8, + "y": 0 + }, + "id": 2, + "options": { + "legend": { + "calcs": [ + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "dataset": "metrics", + "datasource": { + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\nSELECT *,\n lag(ts) OVER w ts_l,\n lag(cpu) OVER w cpu_l,\n lag(total) OVER w total_l\nFROM (\n select ts,\n cluster,\n activity_name name,\n database_name db,\n max(cpu_time_ms) cpu,\n max(IFNULL(run_count,0)+IFNULL(success_count,0)+IFNULL(failure_count,0)) total\n from `act_samples` act\n where $__timeFilter(ts) and \n (cluster = '$cluster') and \n activity_name like '%RunPipeline%' and \n (activity_name like '%${pipeline_name:raw}%' or '%' in ($pipeline_name)) and\n (database_name in ($database_name) or '%' in ($database_name) or ('-' in ($database_name) and database_name is null)) and\n (activity_name = aggregator_activity_name or aggregator_activity_name is NULL)\n group by cluster, database_name, activity_type, activity_name, ts) q\nWINDOW w AS(partition by cluster, db, name order by ts) )\nselect $__timeGroup(time_sec,$interval) time_sec,\n sum(cpu)/sum(total) as 'Avg CPU Time per Execution'\nFROM (select a.cluster,\n a.db,\n a.name,\n a.ts time_sec,\n sum(if(a.cpu>a.cpu_l, a.cpu- a.cpu_l, 0)) cpu,\n sum(if(a.total>a.total_l, a.total- a.total_l, 0)) total\n from t a\n group by 1, 2, 3, 4) subq\nwhere cpu > 0\ngroup by 1\norder by 1", + "refId": "A", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [ + { + "name": "avg_cpu_time_ms", + "type": "functionParameter" + } + ], + "type": "function" + }, + { + "parameters": [ + { + "name": "avg_memory_mb", + "type": "functionParameter" + } + ], + "type": "function" + }, + { + "parameters": [ + { + "name": "ts", + "type": "functionParameter" + } + ], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "pipeline_perfmetrics", + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Avg CPU Time per Execution", + "transformations": [ + { + "id": "convertFieldType", + "options": { + "conversions": [ + { + "destinationType": "time", + "targetField": "time_sec" + } + ], + "fields": {} + } + } + ], + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "axisSoftMin": 0, + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "ms" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 16, + "y": 0 + }, + "id": 9, + "options": { + "legend": { + "calcs": [ + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "dataset": "metrics", + "datasource": { + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\nSELECT *,\n lag(ts) OVER w ts_l,\n lag(elp) OVER w elp_l,\n lag(total) OVER w total_l\nFROM (\n select ts,\n cluster,\n activity_name name,\n database_name db,\n max(elapsed_time_ms) elp,\n max(IFNULL(run_count,0)+IFNULL(success_count,0)+IFNULL(failure_count,0)) total\n from `act_samples` act\n where $__timeFilter(ts) and \n (cluster = '$cluster') and \n activity_name like '%RunPipeline%' and \n (activity_name like '%${pipeline_name:raw}%' or '%' in ($pipeline_name)) and\n (database_name in ($database_name) or '%' in ($database_name) or ('-' in ($database_name) and database_name is null)) and\n (activity_name = aggregator_activity_name or aggregator_activity_name is NULL)\n group by cluster, database_name, activity_type, activity_name, ts) q\nWINDOW w AS(partition by cluster, db, name order by ts) )\nselect $__timeGroup(time_sec,$interval) time_sec,\n sum(elp)/sum(total) as 'Avg Elapsed Time per Execution'\nFROM (select a.cluster,\n a.db,\n a.name,\n a.ts time_sec,\n sum(if(a.elp>a.elp_l, a.elp- a.elp_l, 0)) elp,\n sum(if(a.total>a.total_l, a.total- a.total_l, 0)) total\n from t a\n group by 1, 2, 3, 4) subq\nwhere elp > 0\ngroup by 1\norder by 1", + "refId": "A", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [ + { + "name": "avg_cpu_time_ms", + "type": "functionParameter" + } + ], + "type": "function" + }, + { + "parameters": [ + { + "name": "avg_memory_mb", + "type": "functionParameter" + } + ], + "type": "function" + }, + { + "parameters": [ + { + "name": "ts", + "type": "functionParameter" + } + ], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "pipeline_perfmetrics", + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Avg Elapsed Time per Execution", + "transformations": [ + { + "id": "convertFieldType", + "options": { + "conversions": [ + { + "destinationType": "time", + "targetField": "time_sec" + } + ], + "fields": {} + } + } + ], + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "axisSoftMin": 0, + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "bytes" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 0, + "y": 8 + }, + "id": 13, + "options": { + "legend": { + "calcs": [ + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "dataset": "metrics", + "datasource": { + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\nSELECT *,\n lag(ts) OVER w ts_l,\n lag(disk_b) OVER w disk_b_l,\n lag(total) OVER w total_l\nFROM (\n select ts,\n cluster,\n activity_name name,\n database_name db,\n max(disk_b) disk_b,\n max(IFNULL(run_count,0)+IFNULL(success_count,0)+IFNULL(failure_count,0)) total\n from `act_samples` act\n where $__timeFilter(ts) and \n (cluster = '$cluster') and \n activity_name like '%RunPipeline%' and \n (activity_name like '%${pipeline_name:raw}%' or '%' in ($pipeline_name)) and\n (database_name in ($database_name) or '%' in ($database_name) or ('-' in ($database_name) and database_name is null)) and\n (activity_name = aggregator_activity_name or aggregator_activity_name is NULL)\n group by cluster, database_name, activity_type, activity_name, ts) q\nWINDOW w AS(partition by cluster, db, name order by ts) )\nselect $__timeGroup(time_sec,$interval) time_sec,\n sum(disk_b)/sum(total) as 'Avg IO per Execution'\nFROM (select a.cluster,\n a.db,\n a.name,\n a.ts time_sec,\n sum(if(a.disk_b>a.disk_b_l, a.disk_b- a.disk_b_l, 0)) disk_b,\n sum(if(a.total>a.total_l, a.total- a.total_l, 0)) total\n from t a\n group by 1, 2, 3, 4) subq\ngroup by 1\norder by 1", + "refId": "A", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [ + { + "name": "avg_cpu_time_ms", + "type": "functionParameter" + } + ], + "type": "function" + }, + { + "parameters": [ + { + "name": "avg_memory_mb", + "type": "functionParameter" + } + ], + "type": "function" + }, + { + "parameters": [ + { + "name": "ts", + "type": "functionParameter" + } + ], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "pipeline_perfmetrics", + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Avg IO per Execution", + "transformations": [ + { + "id": "convertFieldType", + "options": { + "conversions": [ + { + "destinationType": "time", + "targetField": "time_sec" + } + ], + "fields": {} + } + } + ], + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "axisSoftMin": 0, + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "bytes" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 8, + "y": 8 + }, + "id": 11, + "options": { + "legend": { + "calcs": [ + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "dataset": "metrics", + "datasource": { + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\nSELECT *,\n lag(ts) OVER w ts_l,\n lag(elp) OVER w elp_l,\n lag(memory_b) OVER w memory_b_l\nFROM (\n select ts,\n cluster,\n activity_name name,\n database_name db,\n max(elapsed_time_ms) elp,\n max(1000*memory_bs) memory_b\n from `act_samples` act\n where $__timeFilter(ts) and \n (cluster = '$cluster') and \n activity_name like '%RunPipeline%' and \n (activity_name like '%${pipeline_name:raw}%' or '%' in ($pipeline_name)) and\n (database_name in ($database_name) or '%' in ($database_name) or ('-' in ($database_name) and database_name is null)) and\n (activity_name = aggregator_activity_name or aggregator_activity_name is NULL)\n group by cluster, database_name, activity_type, activity_name, ts) q\nWINDOW w AS(partition by cluster, db, name order by ts) )\nselect $__timeGroup(time_sec,$interval) time_sec,\n sum(memory_b)/sum(elp) as 'Avg Memory Used per Execution'\nFROM (select a.cluster,\n a.db,\n a.name,\n a.ts time_sec,\n sum(if(a.elp>a.elp_l, a.elp- a.elp_l, 0)) elp,\n sum(if(a.memory_b>a.memory_b_l, a.memory_b- a.memory_b_l, 0)) memory_b\n from t a\n group by 1, 2, 3, 4) subq\nwhere elp > 0\ngroup by 1\norder by 1", + "refId": "A", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [ + { + "name": "avg_cpu_time_ms", + "type": "functionParameter" + } + ], + "type": "function" + }, + { + "parameters": [ + { + "name": "avg_memory_mb", + "type": "functionParameter" + } + ], + "type": "function" + }, + { + "parameters": [ + { + "name": "ts", + "type": "functionParameter" + } + ], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "pipeline_perfmetrics", + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Avg Memory Used per Execution", + "transformations": [ + { + "id": "convertFieldType", + "options": { + "conversions": [ + { + "destinationType": "time", + "targetField": "time_sec" + } + ], + "fields": {} + } + } + ], + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "axisSoftMin": 0, + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "bytes" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 16, + "y": 8 + }, + "id": 12, + "options": { + "legend": { + "calcs": [ + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "dataset": "metrics", + "datasource": { + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "WITH t AS (\nSELECT *,\n lag(ts) OVER w ts_l,\n lag(network_b) OVER w network_b_l,\n lag(total) OVER w total_l\nFROM (\n select ts,\n cluster,\n activity_name name,\n database_name db,\n max(network_b) network_b,\n max(IFNULL(run_count,0)+IFNULL(success_count,0)+IFNULL(failure_count,0)) total\n from `act_samples` act\n where $__timeFilter(ts) and \n (cluster = '$cluster') and \n activity_name like '%RunPipeline%' and \n (activity_name like '%${pipeline_name:raw}%' or '%' in ($pipeline_name)) and\n (database_name in ($database_name) or '%' in ($database_name) or ('-' in ($database_name) and database_name is null)) and\n (activity_name = aggregator_activity_name or aggregator_activity_name is NULL)\n group by cluster, database_name, activity_type, activity_name, ts) q\nWINDOW w AS(partition by cluster, db, name order by ts) )\nselect $__timeGroup(time_sec,$interval) time_sec,\n sum(network_b)/sum(total) as 'Avg Network Bytes per Execution'\nFROM (select a.cluster,\n a.db,\n a.name,\n a.ts time_sec,\n sum(if(a.network_b>a.network_b_l, a.network_b- a.network_b_l, 0)) network_b,\n sum(if(a.total>a.total_l, a.total- a.total_l, 0)) total\n from t a\n group by 1, 2, 3, 4) subq\nwhere network_b > 0\ngroup by 1\norder by 1", + "refId": "A", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [ + { + "name": "avg_cpu_time_ms", + "type": "functionParameter" + } + ], + "type": "function" + }, + { + "parameters": [ + { + "name": "avg_memory_mb", + "type": "functionParameter" + } + ], + "type": "function" + }, + { + "parameters": [ + { + "name": "ts", + "type": "functionParameter" + } + ], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "pipeline_perfmetrics", + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Avg Network Bytes per Execution", + "transformations": [ + { + "id": "convertFieldType", + "options": { + "conversions": [ + { + "destinationType": "time", + "targetField": "time_sec" + } + ], + "fields": {} + } + } + ], + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "custom": { + "align": "auto", + "cellOptions": { + "type": "auto" + }, + "inspect": false + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "error_id" + }, + "properties": [ + { + "id": "custom.width", + "value": 113 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "error_code" + }, + "properties": [ + { + "id": "custom.width", + "value": 102 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "error_message" + }, + "properties": [ + { + "id": "custom.width", + "value": 496 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "database_name" + }, + "properties": [ + { + "id": "custom.width", + "value": 194 + } + ] + } + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 16 + }, + "id": 6, + "options": { + "cellHeight": "sm", + "footer": { + "countRows": false, + "fields": "", + "reducer": [ + "sum" + ], + "show": false + }, + "showHeader": true, + "sortBy": [ + { + "desc": false, + "displayName": "PipelineName" + } + ] + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "dataset": "metrics", + "datasource": { + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "table", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "SELECT pipeline_name as PipelineName, \n database_name, \n error_id, \n error_code,\n error_message,\n from_unixtime(error_unix_time) as error_time\nFROM `pipeline_errors`\nwhere (pipeline_name in ($pipeline_name) or '%' in ($pipeline_name) or ('-' in ($pipeline_name) and pipeline_name is null)) and\n cluster = '$cluster' and\n $__unixEpochFilter(error_unix_time) and\n (database_name in ($database_name) or '%' in ($database_name) or ('-' in ($database_name) and database_name is null))", + "refId": "A", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [ + { + "name": "pipeline_name", + "type": "functionParameter" + } + ], + "type": "function" + }, + { + "parameters": [ + { + "name": "database_name", + "type": "functionParameter" + } + ], + "type": "function" + }, + { + "parameters": [ + { + "name": "error_id", + "type": "functionParameter" + } + ], + "type": "function" + }, + { + "parameters": [ + { + "name": "error_unix_time", + "type": "functionParameter" + } + ], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "pipeline_errors", + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Pipeline Errors", + "type": "table" + } + ], + "preload": false, + "refresh": "", + "schemaVersion": 41, + "tags": ["singlestore","sas-speedystore"], + "templating": { + "list": [ + { + "current": { + "text": "memsql_cluster", + "value": "memsql_cluster" + }, + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "definition": "select distinct(cluster) from pipeline_metrics where $__timeFilter(ts)", + "includeAll": false, + "name": "cluster", + "options": [], + "query": "select distinct(cluster) from pipeline_metrics where $__timeFilter(ts)", + "refresh": 2, + "regex": "", + "type": "query" + }, + { + "allValue": "'%'", + "current": { + "text": [ + "All" + ], + "value": [ + "$__all" + ] + }, + "datasource": { + "uid": "s2monitoring" + }, + "definition": "SELECT distinct pipeline_name FROM `pipeline_metrics` where $__timeFilter(ts) and cluster = '$cluster'", + "includeAll": true, + "label": "Pipeline", + "multi": true, + "name": "pipeline_name", + "options": [], + "query": "SELECT distinct pipeline_name FROM `pipeline_metrics` where $__timeFilter(ts) and cluster = '$cluster'", + "refresh": 2, + "regex": "", + "type": "query" + }, + { + "allValue": "'%'", + "current": { + "text": "All", + "value": "$__all" + }, + "datasource": { + "uid": "s2monitoring" + }, + "definition": "SELECT distinct database_name FROM `pipeline_metrics` where $__timeFilter(ts) and cluster = '$cluster'", + "includeAll": true, + "label": "Database", + "multi": true, + "name": "database_name", + "options": [], + "query": "SELECT distinct database_name FROM `pipeline_metrics` where $__timeFilter(ts) and cluster = '$cluster'", + "refresh": 2, + "regex": "", + "sort": 1, + "type": "query" + }, + { + "auto": true, + "auto_count": 30, + "auto_min": "10s", + "current": { + "text": "1m", + "value": "1m" + }, + "hide": 2, + "name": "interval", + "options": [ + { + "selected": true, + "text": "1m", + "value": "1m" + }, + { + "selected": false, + "text": "5m", + "value": "5m" + }, + { + "selected": false, + "text": "10m", + "value": "10m" + }, + { + "selected": false, + "text": "15m", + "value": "15m" + }, + { + "selected": false, + "text": "30m", + "value": "30m" + }, + { + "selected": false, + "text": "1h", + "value": "1h" + }, + { + "selected": false, + "text": "6h", + "value": "6h" + }, + { + "selected": false, + "text": "12h", + "value": "12h" + }, + { + "selected": false, + "text": "1d", + "value": "1d" + }, + { + "selected": false, + "text": "7d", + "value": "7d" + } + ], + "query": "1m,5m,10m,15m,30m,1h,6h,12h,1d,7d", + "refresh": 2, + "type": "interval" + } + ] + }, + "time": { + "from": "now-30m", + "to": "now" + }, + "timepicker": {}, + "timezone": "", + "title": "SingleStore: Pipeline Performance", + "uid": "pipeline-performance", + "version": 1 +} diff --git a/samples/speedystore/dashboards/pipelinesummary.json b/samples/speedystore/dashboards/pipelinesummary.json new file mode 100644 index 00000000..beadd3cf --- /dev/null +++ b/samples/speedystore/dashboards/pipelinesummary.json @@ -0,0 +1,619 @@ +{ + "annotations": { + "list": [ + { + "builtIn": 1, + "datasource": { + "type": "grafana", + "uid": "-- Grafana --" + }, + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "name": "Annotations & Alerts", + "target": { + "limit": 100, + "matchAny": false, + "tags": [], + "type": "dashboard" + }, + "type": "dashboard" + } + ] + }, + "editable": true, + "fiscalYearStartMonth": 0, + "graphTooltip": 0, + "id": 267, + "links": [ + { + "asDropdown": true, + "icon": "external link", + "includeVars": true, + "keepTime": true, + "tags": [], + "targetBlank": true, + "title": "Dashboards", + "tooltip": "", + "type": "dashboards", + "url": "" + } + ], + "panels": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + } + }, + "mappings": [] + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 0 + }, + "id": 2, + "options": { + "displayLabels": [ + "percent" + ], + "legend": { + "displayMode": "table", + "placement": "right", + "showLegend": true, + "values": [ + "value" + ] + }, + "pieType": "pie", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": true + }, + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "dataset": "metrics", + "datasource": { + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "table", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "SELECT state,\n COUNT(distinct pipeline_name) as '' \nFROM pipeline_metrics\nwhere UNIX_TIMESTAMP(ts) = $max_ts and\n cluster = '$cluster'\ngroup by state", + "refId": "A", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "name": "COUNT", + "parameters": [ + { + "name": "state", + "type": "functionParameter" + } + ], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "pipeline_metrics", + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Pipeline State Distribution", + "type": "piechart" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "# of Pipelines", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": true, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "decimals": 0, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "none" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "Stopped" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "yellow", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Running Fo" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "blue", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 0 + }, + "id": 9, + "options": { + "legend": { + "calcs": [ + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true, + "sortBy": "Last", + "sortDesc": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "time_series", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "select $__timeGroup(ts, $interval) as time,\n state,\n count(distinct pipeline_name) as ''\nfrom pipeline_metrics\nwhere cluster = '$cluster' and \n $__timeFilter(ts)\ngroup by 1,2\norder by 1", + "refId": "A", + "select": [ + [ + { + "params": [ + "cpu_time_ms" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "08c783c1c927b993824179b6d7a31908_act_samples", + "timeColumn": "ts", + "timeColumnType": "timestamp", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Historical Pipeline State", + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "custom": { + "align": "auto", + "cellOptions": { + "type": "auto" + }, + "inspect": false + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "PipelineName" + }, + "properties": [ + { + "id": "links", + "value": [ + { + "targetBlank": true, + "title": "", + "url": "d/pipeline-performance/pipeline-performance?var-cluster=[[cluster]]&var-pipeline_name=${__data.fields.PipelineName}&var-database_name=${__data.fields.DatabaseName}&${__url_time_range}" + } + ] + } + ] + } + ] + }, + "gridPos": { + "h": 12, + "w": 24, + "x": 0, + "y": 8 + }, + "id": 4, + "options": { + "cellHeight": "sm", + "footer": { + "countRows": false, + "fields": "", + "reducer": [ + "sum" + ], + "show": false + }, + "showHeader": true, + "sortBy": [ + { + "desc": false, + "displayName": "PipelineName" + } + ] + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "dataset": "metrics", + "datasource": { + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "table", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "SELECT pipeline_name as PipelineName, \n database_name as DatabaseName, \n state as State, \n create_time as 'Created on', \n ts as 'Last State Check'\nFROM pipeline_metrics\nwhere UNIX_TIMESTAMP(ts) = '$max_ts' and\n cluster = '$cluster'\ngroup by pipeline_name, database_name ", + "refId": "A", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [ + { + "name": "pipeline_name", + "type": "functionParameter" + } + ], + "type": "function" + }, + { + "parameters": [ + { + "name": "database_name", + "type": "functionParameter" + } + ], + "type": "function" + }, + { + "parameters": [ + { + "name": "state", + "type": "functionParameter" + } + ], + "type": "function" + }, + { + "parameters": [ + { + "name": "create_time", + "type": "functionParameter" + } + ], + "type": "function" + }, + { + "parameters": [ + { + "name": "alter_time", + "type": "functionParameter" + } + ], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "table": "pipeline_metrics", + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Summary", + "type": "table" + } + ], + "preload": false, + "refresh": "15m", + "schemaVersion": 41, + "tags": ["singlestore","sas-speedystore"], + "templating": { + "list": [ + { + "current": { + "text": "memsql_cluster", + "value": "memsql_cluster" + }, + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "definition": "select distinct(cluster) from pipeline_metrics where $__timeFilter(ts)", + "includeAll": false, + "name": "cluster", + "options": [], + "query": "select distinct(cluster) from pipeline_metrics where $__timeFilter(ts)", + "refresh": 2, + "regex": "", + "type": "query" + }, + { + "current": { + "text": "1693284241", + "value": "1693284241" + }, + "datasource": { + "uid": "s2monitoring" + }, + "definition": "select UNIX_TIMESTAMP(max(ts)) from pipeline_metrics where $__timeFilter(ts)", + "hide": 2, + "includeAll": false, + "name": "max_ts", + "options": [], + "query": "select UNIX_TIMESTAMP(max(ts)) from pipeline_metrics where $__timeFilter(ts)", + "refresh": 2, + "regex": "", + "type": "query" + }, + { + "auto": true, + "auto_count": 100, + "auto_min": "10s", + "current": { + "text": "1m", + "value": "1m" + }, + "hide": 2, + "name": "interval", + "options": [ + { + "selected": true, + "text": "1m", + "value": "1m" + }, + { + "selected": false, + "text": "5m", + "value": "5m" + }, + { + "selected": false, + "text": "10m", + "value": "10m" + }, + { + "selected": false, + "text": "30m", + "value": "30m" + }, + { + "selected": false, + "text": "1h", + "value": "1h" + }, + { + "selected": false, + "text": "3h", + "value": "3h" + }, + { + "selected": false, + "text": "6h", + "value": "6h" + }, + { + "selected": false, + "text": "12h", + "value": "12h" + }, + { + "selected": false, + "text": "1d", + "value": "1d" + } + ], + "query": "1m,5m,10m,30m,1h,3h,6h,12h,1d", + "refresh": 2, + "type": "interval" + } + ] + }, + "time": { + "from": "now-30m", + "to": "now" + }, + "timepicker": {}, + "timezone": "", + "title": "SingleStore: Pipeline Summary", + "uid": "pipeline-summary", + "version": 1 +} diff --git a/samples/speedystore/dashboards/queryhistory.json b/samples/speedystore/dashboards/queryhistory.json new file mode 100644 index 00000000..fe535e67 --- /dev/null +++ b/samples/speedystore/dashboards/queryhistory.json @@ -0,0 +1,394 @@ +{ + "annotations": { + "list": [ + { + "builtIn": 1, + "datasource": { + "type": "datasource", + "uid": "grafana" + }, + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "name": "Annotations & Alerts", + "target": { + "limit": 100, + "matchAny": false, + "tags": [], + "type": "dashboard" + }, + "type": "dashboard" + } + ] + }, + "editable": true, + "fiscalYearStartMonth": 0, + "graphTooltip": 1, + "id": 268, + "links": [ + { + "asDropdown": true, + "icon": "external link", + "includeVars": true, + "keepTime": true, + "tags": [], + "targetBlank": true, + "title": "Dashboards", + "tooltip": "", + "type": "dashboards", + "url": "" + } + ], + "panels": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "custom": { + "align": "auto", + "cellOptions": { + "type": "auto" + }, + "filterable": false, + "inspect": false + }, + "mappings": [ + { + "options": { + "total_time": { + "index": 0, + "text": "Total Elapsed Time" + } + }, + "type": "value" + } + ], + "thresholds": { + "mode": "percentage", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 50 + } + ] + } + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "activity_name" + }, + "properties": [ + { + "id": "custom.inspect", + "value": true + }, + { + "id": "custom.filterable", + "value": true + } + ] + }, + { + "matcher": { + "id": "byRegexp", + "options": "/duration/" + }, + "properties": [ + { + "id": "unit", + "value": "ms" + }, + { + "id": "custom.cellOptions", + "value": { + "type": "gauge" + } + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "query_text" + }, + "properties": [ + { + "id": "custom.width", + "value": 538 + }, + { + "id": "custom.inspect", + "value": true + }, + { + "id": "custom.filterable", + "value": true + }, + { + "id": "custom.cellOptions", + "value": { + "type": "color-text" + } + }, + { + "id": "color", + "value": { + "fixedColor": "blue", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Query_plan_id" + }, + "properties": [ + { + "id": "custom.width", + "value": 158 + } + ] + }, + { + "matcher": { + "id": "byRegexp", + "options": "/error/" + }, + "properties": [ + { + "id": "custom.filterable", + "value": true + }, + { + "id": "custom.inspect", + "value": true + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "duration_ms" + }, + "properties": [ + { + "id": "custom.width", + "value": 172 + } + ] + }, + { + "matcher": { + "id": "byRegexp", + "options": "/name/" + }, + "properties": [ + { + "id": "custom.inspect", + "value": true + }, + { + "id": "custom.filterable", + "value": true + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "resource_pool_name" + }, + "properties": [ + { + "id": "custom.width", + "value": 176 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "success" + }, + "properties": [ + { + "id": "custom.width", + "value": 78 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "start_time" + }, + "properties": [ + { + "id": "custom.width", + "value": 158 + } + ] + } + ] + }, + "gridPos": { + "h": 21, + "w": 24, + "x": 0, + "y": 0 + }, + "id": 4, + "options": { + "cellHeight": "sm", + "footer": { + "countRows": false, + "enablePagination": true, + "fields": "", + "reducer": [ + "sum" + ], + "show": false + }, + "showHeader": true, + "sortBy": [ + { + "desc": true, + "displayName": "start_time" + } + ] + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "table", + "group": [], + "metricColumn": "none", + "rawQuery": true, + "rawSql": "select start_time,\n activity_name, \n query_text,\n duration_ms,\n database_name,\n user_name,\n resource_pool_name,\n success,\n error_code,\n error_message\nfrom query_event_history\nwhere $__timeFilter(start_time) AND \n cluster = '$cluster'", + "refId": "A", + "select": [ + [ + { + "params": [ + "value" + ], + "type": "column" + } + ] + ], + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + }, + "timeColumn": "time", + "where": [ + { + "name": "$__timeFilter", + "params": [], + "type": "macro" + } + ] + } + ], + "title": "Query History", + "transformations": [ + { + "id": "organize", + "options": { + "excludeByName": {}, + "indexByName": { + "Avg. CPU Time": 7, + "Avg. Disk Bytes": 9, + "Avg. Disk Read Bytes (logical)": 10, + "Avg. Disk Write Bytes (logical)": 11, + "Avg. Elapsed Time": 5, + "Avg. Memory Bytes": 8, + "Avg. Network Bytes": 12, + "Avg. Network Received Bytes (logical)": 13, + "Avg. Network Send Bytes (logical)": 14, + "Execution Count": 3, + "Failure Count": 15, + "Failure Rate": 16, + "Total CPU Time": 6, + "Total Elapsed Time": 4, + "db": 0, + "query_plan": 2, + "uniq_act": 1 + }, + "renameByName": { + "uniq_act": "Query_plan_id" + } + } + } + ], + "type": "table" + } + ], + "preload": false, + "refresh": "", + "schemaVersion": 41, + "tags": ["singlestore","sas-speedystore"], + "templating": { + "list": [ + { + "current": { + "text": "memsql_cluster", + "value": "memsql_cluster" + }, + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "definition": "select distinct(cluster) from `query_event_history` where $__timeFilter(ts)", + "includeAll": false, + "name": "cluster", + "options": [], + "query": "select distinct(cluster) from `query_event_history` where $__timeFilter(ts)", + "refresh": 2, + "regex": "", + "type": "query" + } + ] + }, + "time": { + "from": "now-30m", + "to": "now" + }, + "timepicker": {}, + "timezone": "", + "title": "SingleStore: Query History", + "uid": "efacc80e-2e33-4bfb-99fe-223b45e4197a", + "version": 1 +} diff --git a/samples/speedystore/dashboards/resourcepoolmonitoring.json b/samples/speedystore/dashboards/resourcepoolmonitoring.json new file mode 100644 index 00000000..94bb9f7a --- /dev/null +++ b/samples/speedystore/dashboards/resourcepoolmonitoring.json @@ -0,0 +1,717 @@ +{ + "annotations": { + "list": [ + { + "builtIn": 1, + "datasource": { + "type": "datasource", + "uid": "grafana" + }, + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "limit": 100, + "name": "Annotations & Alerts", + "showIn": 0, + "target": { + "limit": 100, + "matchAny": false, + "tags": [], + "type": "dashboard" + }, + "type": "dashboard" + } + ] + }, + "editable": true, + "fiscalYearStartMonth": 0, + "graphTooltip": 1, + "id": 269, + "links": [ + { + "asDropdown": false, + "icon": "external link", + "includeVars": false, + "keepTime": false, + "tags": [], + "targetBlank": true, + "title": "Documentation", + "tooltip": "", + "type": "link", + "url": "https://docs.singlestore.com/db/v8.1/en/user-and-cluster-administration/cluster-health-and-performance/configure-monitoring/view-the-dashboards.html#cluster-view-793226" + }, + { + "asDropdown": true, + "icon": "external link", + "includeVars": true, + "keepTime": true, + "tags": [], + "targetBlank": true, + "title": "Dashboards", + "type": "dashboards" + } + ], + "panels": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 0 + }, + "id": 1, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "table", + "rawQuery": true, + "rawSql": "WITH t AS (\n SELECT \n *, \n lag(value) OVER w value_l\n from (\n select max(tg) as time,\n max(value) as value,\n metric,\n pool_name\n from (\n SELECT time_sec as time,\n $__unixEpochGroup(time_sec, $interval) as tg, \n host as metric,\n labels::$pool_name as pool_name,\n value\n FROM metrics\n WHERE $__unixEpochFilter(time_sec) AND\n (cluster = '$cluster') AND\n name = 'memsql_resource_pool_status_finished_queries' AND\n ('%' in ($pool) or labels::$pool_name in ($pool))\n )\n group by metric, pool_name, tg) X \n WINDOW w AS (partition by metric, pool_name order by time))\nselect time, \n pool_name,\n sum((value - value_l)) as 'Finished Queries:'\nfrom t\nwhere value_l is not null \n and value >= value_l\ngroup by time, pool_name\norder by time", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Finished Queries", + "transformations": [ + { + "id": "prepareTimeSeries", + "options": { + "format": "multi" + } + } + ], + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 0 + }, + "id": 2, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "table", + "rawQuery": true, + "rawSql": "WITH t AS (\n SELECT \n *, \n lag(value) OVER w value_l\n from (\n select max(tg) as time,\n max(value) as value,\n metric,\n pool_name\n from (\n SELECT time_sec as time,\n $__unixEpochGroup(time_sec, $interval) as tg, \n host as metric,\n labels::$pool_name as pool_name,\n value\n FROM metrics\n WHERE $__unixEpochFilter(time_sec) AND\n (cluster = '$cluster') AND\n name = 'memsql_resource_pool_status_killed_queries' AND\n ('%' in ($pool) or labels::$pool_name in ($pool))\n )\n group by metric, pool_name, tg) X \n WINDOW w AS (partition by metric, pool_name order by time))\nselect time, \n pool_name,\n sum((value - value_l)) as 'Killed Queries:'\nfrom t\nwhere value_l is not null \n and value >= value_l\ngroup by time, pool_name\norder by time", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Killed Queries", + "transformations": [ + { + "id": "prepareTimeSeries", + "options": { + "format": "multi" + } + } + ], + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 8 + }, + "id": 3, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "table", + "rawQuery": true, + "rawSql": "SELECT $__unixEpochGroup(time_sec, $interval) as time, \n labels::$pool_name as pool_name,\n sum(value) as 'Queueing Queries:'\nFROM metrics\nWHERE $__unixEpochFilter(time_sec) AND\n (cluster = '$cluster') AND\n host not like '%leaf%' and\n NAME in ('memsql_resource_pool_status_queueing_queries') AND\n ('%' in ($pool) or labels::$pool_name in ($pool))\ngroup by 1, 2\norder by 1;", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Queueing Queries", + "transformations": [ + { + "id": "prepareTimeSeries", + "options": { + "format": "multi" + } + } + ], + "type": "timeseries" + }, + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "ms" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 8 + }, + "id": 4, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.1.0", + "targets": [ + { + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "editorMode": "code", + "format": "table", + "rawQuery": true, + "rawSql": "WITH t AS (\n SELECT \n *, \n lag(total_queue_time_ms) OVER w total_queue_time_ms_l\n from (\n select max(tg) as time,\n max(case when name = 'memsql_resource_pool_status_total_queue_time_ms' then value end) as total_queue_time_ms,\n max(case when name = 'memsql_resource_pool_status_queueing_queries' then value end) as queueing_queries,\n metric,\n pool_name\n from (\n SELECT time_sec as time,\n $__unixEpochGroup(time_sec, $interval) as tg, \n name,\n host as metric,\n labels::$pool_name as pool_name,\n value\n FROM metrics\n WHERE $__unixEpochFilter(time_sec) AND\n (cluster = '$cluster') AND\n name in ('memsql_resource_pool_status_total_queue_time_ms','memsql_resource_pool_status_queueing_queries') AND\n ('%' in ($pool) or labels::$pool_name in ($pool))\n )\n group by metric, pool_name, tg) X \n WINDOW w AS (partition by metric, pool_name order by time))\nselect time, \n pool_name,\n sum((total_queue_time_ms - total_queue_time_ms_l))/sum(queueing_queries) as 'Avg. Queue Time:'\nfrom t\nwhere total_queue_time_ms_l is not null \n and total_queue_time_ms >= total_queue_time_ms_l\ngroup by time, pool_name\norder by time", + "refId": "A", + "sql": { + "columns": [ + { + "parameters": [], + "type": "function" + } + ], + "groupBy": [ + { + "property": { + "type": "string" + }, + "type": "groupBy" + } + ], + "limit": 50 + } + } + ], + "title": "Queue Time per Queued Query", + "transformations": [ + { + "id": "prepareTimeSeries", + "options": { + "format": "multi" + } + } + ], + "type": "timeseries" + } + ], + "preload": false, + "refresh": "", + "schemaVersion": 41, + "tags": ["singlestore","sas-speedystore"], + "templating": { + "list": [ + { + "allValue": "'%'", + "current": { + "text": "memsql_cluster", + "value": "memsql_cluster" + }, + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "definition": "", + "includeAll": false, + "name": "cluster", + "options": [], + "query": "select distinct(cluster) from metrics where $__unixEpochFilter(time_sec)", + "refresh": 2, + "regex": "", + "type": "query" + }, + { + "allValue": "'%'", + "current": { + "text": "All", + "value": "$__all" + }, + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "definition": "select distinct labels::$pool_name\nfrom metrics\nwhere name = 'memsql_resource_pool_status_finished_queries' and\n $__unixEpochFilter(time_sec) and (cluster='$cluster')", + "includeAll": true, + "label": "Resource Pool", + "multi": true, + "name": "pool", + "options": [], + "query": "select distinct labels::$pool_name\nfrom metrics\nwhere name = 'memsql_resource_pool_status_finished_queries' and\n $__unixEpochFilter(time_sec) and (cluster='$cluster')", + "refresh": 2, + "regex": "", + "sort": 5, + "type": "query" + }, + { + "allValue": "'%'", + "current": { + "text": "All", + "value": "$__all" + }, + "datasource": { + "type": "mysql", + "uid": "s2monitoring" + }, + "definition": "", + "hide": 2, + "includeAll": true, + "multi": true, + "name": "host", + "options": [], + "query": "select distinct(host) from metrics where host is not NULL and $__unixEpochFilter(time_sec) and (cluster='$cluster')", + "refresh": 2, + "regex": "", + "type": "query" + }, + { + "auto": true, + "auto_count": 50, + "auto_min": "1m", + "current": { + "text": "$__auto", + "value": "$__auto" + }, + "name": "interval", + "options": [ + { + "selected": false, + "text": "1m", + "value": "1m" + }, + { + "selected": false, + "text": "10m", + "value": "10m" + }, + { + "selected": false, + "text": "30m", + "value": "30m" + }, + { + "selected": false, + "text": "1h", + "value": "1h" + }, + { + "selected": false, + "text": "6h", + "value": "6h" + }, + { + "selected": false, + "text": "12h", + "value": "12h" + }, + { + "selected": false, + "text": "1d", + "value": "1d" + }, + { + "selected": false, + "text": "7d", + "value": "7d" + }, + { + "selected": false, + "text": "14d", + "value": "14d" + }, + { + "selected": false, + "text": "30d", + "value": "30d" + } + ], + "query": "1m,10m,30m,1h,6h,12h,1d,7d,14d,30d", + "refresh": 2, + "type": "interval" + } + ] + }, + "time": { + "from": "now-1h", + "to": "now" + }, + "timepicker": { + "refresh_intervals": [ + "5m", + "15m", + "30m", + "1h", + "2h", + "1d" + ] + }, + "timezone": "", + "title": "SingleStore: Resource Pool Monitoring", + "uid": "b706aab4-2dc9-41f0-89c1-2418f8da5005", + "version": 1 +} diff --git a/samples/speedystore/images/02_MG_202508_metrics-database.png b/samples/speedystore/images/02_MG_202508_metrics-database.png new file mode 100644 index 00000000..06bf140b Binary files /dev/null and b/samples/speedystore/images/02_MG_202508_metrics-database.png differ diff --git a/samples/speedystore/images/cluster_monitoring_hl_architecture-mfjdOm.webp b/samples/speedystore/images/cluster_monitoring_hl_architecture-mfjdOm.webp new file mode 100644 index 00000000..1c600444 Binary files /dev/null and b/samples/speedystore/images/cluster_monitoring_hl_architecture-mfjdOm.webp differ diff --git a/samples/speedystore/speedystore-datasource.yaml b/samples/speedystore/speedystore-datasource.yaml new file mode 100644 index 00000000..e89b04af --- /dev/null +++ b/samples/speedystore/speedystore-datasource.yaml @@ -0,0 +1,14 @@ +apiVersion: 1 +deleteDatasources: +- name: s2monitoring +prune: true +datasources: + - name: s2monitoring + type: mysql + url: svc-sas-singlestore-cluster-ddl.{**Replace with SAS VIYA Namespace**}.svc.cluster.local:3306 + database: metrics + user: S2MonitorUser + secureJsonData: + password: {**Replace with S2MonitorUser Password**} + isDefault: false + version: 1 \ No newline at end of file