Make creation of Grafana Data Source and Folder configurable (#145)

* Made the cluster variable visible in dashboards

* Made the repo reusable with multiple EKS clusters

* Relabled the input variable from create_grafana_data_source to create_prometheus_data_source

Author: vchintal

README.md

@@ -188,6 +188,8 @@ No modules.
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
 | <a name="input_aws_region"></a> [aws\_region](#input\_aws\_region) | AWS Region | `string` | n/a | yes |
+| <a name="input_create_dashboard_folder"></a> [create\_dashboard\_folder](#input\_create\_dashboard\_folder) | Boolean flag to enable Amazon Managed Grafana folder and dashboards | `bool` | `false` | no |
+| <a name="input_create_prometheus_data_source"></a> [create\_prometheus\_data\_source](#input\_create\_prometheus\_data\_source) | Boolean flag to enable Amazon Managed Grafana datasource | `bool` | `false` | no |
 | <a name="input_enable_alertmanager"></a> [enable\_alertmanager](#input\_enable\_alertmanager) | Creates Amazon Managed Service for Prometheus AlertManager for all workloads | `bool` | `false` | no |
 | <a name="input_enable_managed_prometheus"></a> [enable\_managed\_prometheus](#input\_enable\_managed\_prometheus) | Creates a new Amazon Managed Service for Prometheus Workspace | `bool` | `true` | no |
 | <a name="input_grafana_api_key"></a> [grafana\_api\_key](#input\_grafana\_api\_key) | Grafana API key for the Amazon Managed Grafana workspace | `string` | n/a | yes |

main.tf

@@ -25,6 +25,7 @@ provider "grafana" {
 }
 
 resource "grafana_data_source" "amp" {
+  count      = var.create_prometheus_data_source ? 1 : 0
   type       = "prometheus"
   name       = local.name
   is_default = true
@@ -39,5 +40,6 @@ resource "grafana_data_source" "amp" {
 
 # dashboards
 resource "grafana_folder" "this" {
+  count = var.create_dashboard_folder ? 1 : 0
   title = "Observability Accelerator Dashboards"
 }

modules/eks-monitoring/README.md

@@ -75,11 +75,12 @@ This module makes use of the open source [kube-prometheus-stack](https://github.
 | <a name="input_enable_logs"></a> [enable\_logs](#input\_enable\_logs) | Using AWS For FluentBit to collect cluster and application logs to Amazon CloudWatch | `bool` | `true` | no |
 | <a name="input_enable_nginx"></a> [enable\_nginx](#input\_enable\_nginx) | Enable NGINX workloads monitoring, alerting and default dashboards | `bool` | `false` | no |
 | <a name="input_enable_node_exporter"></a> [enable\_node\_exporter](#input\_enable\_node\_exporter) | Enables or disables Node exporter. Disabling this might affect some data in the dashboards | `bool` | `true` | no |
+| <a name="input_enable_recording_rules"></a> [enable\_recording\_rules](#input\_enable\_recording\_rules) | Enables or disables Managed Prometheus recording rules | `bool` | `true` | no |
 | <a name="input_enable_tracing"></a> [enable\_tracing](#input\_enable\_tracing) | (Experimental) Enables tracing with AWS X-Ray. This changes the deploy mode of the collector to daemon set. Requirement: adot add-on <= 0.58-build.0 | `bool` | `false` | no |
 | <a name="input_helm_config"></a> [helm\_config](#input\_helm\_config) | Helm Config for Prometheus | `any` | `{}` | no |
 | <a name="input_irsa_iam_permissions_boundary"></a> [irsa\_iam\_permissions\_boundary](#input\_irsa\_iam\_permissions\_boundary) | IAM permissions boundary for IRSA roles | `string` | `null` | no |
 | <a name="input_irsa_iam_role_path"></a> [irsa\_iam\_role\_path](#input\_irsa\_iam\_role\_path) | IAM role path for IRSA roles | `string` | `"/"` | no |
-| <a name="input_java_config"></a> [java\_config](#input\_java\_config) | Configuration object for Java/JMX monitoring | <pre>object({<br>    enable_alerting_rules = bool<br>    scrape_sample_limit   = number<br>  })</pre> | <pre>{<br>  "enable_alerting_rules": true,<br>  "scrape_sample_limit": 1000<br>}</pre> | no |
+| <a name="input_java_config"></a> [java\_config](#input\_java\_config) | Configuration object for Java/JMX monitoring | <pre>object({<br>    enable_alerting_rules  = bool<br>    enable_recording_rules = bool<br>    scrape_sample_limit    = number<br>  })</pre> | <pre>{<br>  "enable_alerting_rules": true,<br>  "enable_recording_rules": true,<br>  "scrape_sample_limit": 1000<br>}</pre> | no |
 | <a name="input_ksm_config"></a> [ksm\_config](#input\_ksm\_config) | Kube State metrics configuration | <pre>object({<br>    create_namespace   = bool<br>    k8s_namespace      = string<br>    helm_chart_name    = string<br>    helm_chart_version = string<br>    helm_release_name  = string<br>    helm_repo_url      = string<br>    helm_settings      = map(string)<br>    helm_values        = map(any)<br><br>    scrape_interval = string<br>    scrape_timeout  = string<br>  })</pre> | <pre>{<br>  "create_namespace": true,<br>  "helm_chart_name": "kube-state-metrics",<br>  "helm_chart_version": "4.24.0",<br>  "helm_release_name": "kube-state-metrics",<br>  "helm_repo_url": "https://prometheus-community.github.io/helm-charts",<br>  "helm_settings": {},<br>  "helm_values": {},<br>  "k8s_namespace": "kube-system",<br>  "scrape_interval": "60s",<br>  "scrape_timeout": "15s"<br>}</pre> | no |
 | <a name="input_logs_config"></a> [logs\_config](#input\_logs\_config) | Configuration object for logs collection | <pre>object({<br>    cw_log_retention_days = number<br>  })</pre> | <pre>{<br>  "cw_log_retention_days": 90<br>}</pre> | no |
 | <a name="input_managed_prometheus_workspace_endpoint"></a> [managed\_prometheus\_workspace\_endpoint](#input\_managed\_prometheus\_workspace\_endpoint) | Amazon Managed Prometheus Workspace Endpoint | `string` | `""` | no |

modules/eks-monitoring/dashboards/cluster.json

@@ -2945,7 +2945,7 @@
           "uid": "$datasource"
         },
         "definition": "",
-        "hide": 2,
+        "hide": 0,
         "includeAll": false,
         "multi": false,
         "name": "cluster",

modules/eks-monitoring/dashboards/kubelet.json

@@ -2148,7 +2148,7 @@
           "uid": "$datasource"
         },
         "definition": "",
-        "hide": 2,
+        "hide": 0,
         "includeAll": false,
         "label": "cluster",
         "multi": false,

modules/eks-monitoring/dashboards/namespace-workloads.json

@@ -2542,7 +2542,7 @@
           "uid": "$datasource"
         },
         "definition": "",
-        "hide": 2,
+        "hide": 0,
         "includeAll": false,
         "multi": false,
         "name": "cluster",

modules/eks-monitoring/dashboards/nodes.json

@@ -1402,7 +1402,7 @@
           "uid": "$datasource"
         },
         "definition": "",
-        "hide": 2,
+        "hide": 0,
         "includeAll": false,
         "multi": false,
         "name": "cluster",

modules/eks-monitoring/dashboards/workloads.json

@@ -2139,7 +2139,7 @@
           "uid": "$datasource"
         },
         "definition": "",
-        "hide": 2,
+        "hide": 0,
         "includeAll": false,
         "multi": false,
         "name": "cluster",

modules/eks-monitoring/main.tf

@@ -153,11 +153,13 @@ module "helm_addon" {
 }
 
 module "java_monitoring" {
-  source = "./patterns/java"
-  count  = var.enable_java ? 1 : 0
+  source            = "./patterns/java"
+  count             = var.enable_java ? 1 : 0
+  enable_dashboards = var.enable_dashboards
 
   managed_prometheus_workspace_id = var.managed_prometheus_workspace_id
   enable_alerting_rules           = var.java_config.enable_alerting_rules
+  enable_recording_rules          = var.java_config.enable_recording_rules
   dashboards_folder_id            = var.dashboards_folder_id
 }
 

modules/eks-monitoring/patterns/java/README.md

@@ -42,6 +42,8 @@ No modules.
 |------|-------------|------|---------|:--------:|
 | <a name="input_dashboards_folder_id"></a> [dashboards\_folder\_id](#input\_dashboards\_folder\_id) | Grafana folder ID for automatic dashboards | `string` | n/a | yes |
 | <a name="input_enable_alerting_rules"></a> [enable\_alerting\_rules](#input\_enable\_alerting\_rules) | Enables or disables Managed Prometheus alerting rules | `bool` | `true` | no |
+| <a name="input_enable_dashboards"></a> [enable\_dashboards](#input\_enable\_dashboards) | Enables or disables curated dashboards | `bool` | `true` | no |
+| <a name="input_enable_recording_rules"></a> [enable\_recording\_rules](#input\_enable\_recording\_rules) | Enables or disables Managed Prometheus recording rules | `bool` | `true` | no |
 | <a name="input_managed_prometheus_workspace_id"></a> [managed\_prometheus\_workspace\_id](#input\_managed\_prometheus\_workspace\_id) | Amazon Managed Prometheus Workspace ID | `string` | `null` | no |
 
 ## Outputs