Skip to content

Commit 9a761d9

Browse files
freschrifreschri
authored
5 pattern for java workload monitoring (#76)
* added rules configurator and rule files * removed ACK, using CfnRuleGroupsNamespace * first commit * adding dashboard images to doc * moved code from index file to doc, to avoid interference with single-new-eks-opensource-observability-construct pattern * removing property from AmpRulesConfiguratorAddOn * applied update from branch: 64-single-new-eks-cluster-open-source-observability-accelerator-pattern-does-not-show-all-metrics-in-grafana * added java flag, amended doc * adapted to flux interface changes; cdk version updated * updated all new pattern to use .version("auto") * missed in previous commit * added java rules, corrected doc * adding java to graviton * added java to existing cluster oss pattern
1 parent a608876 commit 9a761d9

File tree

15 files changed

+2977
-151
lines changed

15 files changed

+2977
-151
lines changed

‎cdk.json‎

Lines changed: 23 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -17,12 +17,29 @@
1717
]
1818
},
1919
"context": {
20-
"cluster.dashboard.url": "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/cluster.json",
21-
"kubelet.dashboard.url": "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/kubelet.json",
22-
"namespaceworkloads.dashboard.url": "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/namespace-workloads.json",
23-
"nodeexporter.dashboard.url": "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodeexporter-nodes.json",
24-
"nodes.dashboard.url": "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodes.json",
25-
"workloads.dashboard.url": "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/workloads.json",
20+
"fluxRepository": {
21+
"name": "grafana-dashboards",
22+
"namespace": "grafana-operator",
23+
"repository": {
24+
"repoUrl": "https://github.com/aws-observability/aws-observability-accelerator",
25+
"name": "grafana-dashboards",
26+
"targetRevision": "main",
27+
"path": "./artifacts/grafana-operator-manifests/eks/infrastructure"
28+
},
29+
"values": {
30+
"GRAFANA_CLUSTER_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/cluster.json",
31+
"GRAFANA_KUBELET_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/kubelet.json",
32+
"GRAFANA_NSWRKLDS_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/namespace-workloads.json",
33+
"GRAFANA_NODEEXP_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodeexporter-nodes.json",
34+
"GRAFANA_NODES_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodes.json",
35+
"GRAFANA_WORKLOADS_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/workloads.json"
36+
},
37+
"kustomizations": [
38+
{
39+
"kustomizationPath": "./artifacts/grafana-operator-manifests/eks/infrastructure"
40+
}
41+
]
42+
},
2643
"existing.cluster.name": "single-new-eks-observability-accelerator",
2744
"existing.kubectl.rolename": "YOUR_KUBECTL_ROLE"
2845
}

‎docs/patterns/existing-eks-observability-accelerators/existing-eks-opensource-observability.md‎

Lines changed: 60 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -97,15 +97,67 @@ aws ssm put-parameter --name "/cdk-accelerator/grafana-api-key" \
9797

9898
Example settings: Update the context in `cdk.json` file located in `cdk-eks-blueprints-patterns` directory
9999

100+
```typescript
101+
"context": {
102+
"fluxRepository": {
103+
"name": "grafana-dashboards",
104+
"namespace": "grafana-operator",
105+
"repository": {
106+
"repoUrl": "https://github.com/aws-observability/aws-observability-accelerator",
107+
"name": "grafana-dashboards",
108+
"targetRevision": "main",
109+
"path": "./artifacts/grafana-operator-manifests/eks/infrastructure"
110+
},
111+
"values": {
112+
"GRAFANA_CLUSTER_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/cluster.json",
113+
"GRAFANA_KUBELET_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/kubelet.json",
114+
"GRAFANA_NSWRKLDS_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/namespace-workloads.json",
115+
"GRAFANA_NODEEXP_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodeexporter-nodes.json",
116+
"GRAFANA_NODES_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodes.json",
117+
"GRAFANA_WORKLOADS_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/workloads.json"
118+
},
119+
"kustomizations": [
120+
{
121+
"kustomizationPath": "./artifacts/grafana-operator-manifests/eks/infrastructure"
122+
}
123+
]
124+
},
125+
}
100126
```
101-
"context": {
102-
"cluster.dashboard.url": "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/cluster.json",
103-
"kubelet.dashboard.url": "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/kubelet.json",
104-
"namespaceworkloads.dashboard.url": "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/namespace-workloads.json",
105-
"nodeexporter.dashboard.url": "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodeexporter-nodes.json",
106-
"nodes.dashboard.url": "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodes.json",
107-
"workloads.dashboard.url": "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/workloads.json"
108-
}
127+
128+
If you need Java observability you can instead use:
129+
130+
```typescript
131+
"context": {
132+
"fluxRepository": {
133+
"name": "grafana-dashboards",
134+
"namespace": "grafana-operator",
135+
"repository": {
136+
"repoUrl": "https://github.com/aws-observability/aws-observability-accelerator",
137+
"name": "grafana-dashboards",
138+
"targetRevision": "main",
139+
"path": "./artifacts/grafana-operator-manifests/eks/infrastructure"
140+
},
141+
"values": {
142+
"GRAFANA_CLUSTER_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/cluster.json",
143+
"GRAFANA_KUBELET_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/kubelet.json",
144+
"GRAFANA_NSWRKLDS_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/namespace-workloads.json",
145+
"GRAFANA_NODEEXP_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodeexporter-nodes.json",
146+
"GRAFANA_NODES_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodes.json",
147+
"GRAFANA_WORKLOADS_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/workloads.json",
148+
"GRAFANA_JAVA_JMX_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/java/default.json"
149+
},
150+
"kustomizations": [
151+
{
152+
"kustomizationPath": "./artifacts/grafana-operator-manifests/eks/infrastructure"
153+
},
154+
{
155+
"kustomizationPath": "./artifacts/grafana-operator-manifests/eks/java"
156+
}
157+
]
158+
},
159+
"java.pattern.enabled": true
160+
}
109161
```
110162

111163
8. Once all pre-requisites are set you are ready to deploy the pipeline. Run the following command from the root of this repository to deploy the pipeline stack:

‎docs/patterns/images/all-dashboards-java.png‎

407 KB
Loading

‎docs/patterns/images/java-dashboard.png‎

500 KB
Loading

‎docs/patterns/single-new-eks-observability-accelerators/single-new-eks-graviton-opensource-observability.md‎

Lines changed: 60 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -96,15 +96,67 @@ aws secretsmanager create-secret \
9696

9797
Example settings: Update the context in `cdk.json` file located in `cdk-eks-blueprints-patterns` directory
9898

99+
```typescript
100+
"context": {
101+
"fluxRepository": {
102+
"name": "grafana-dashboards",
103+
"namespace": "grafana-operator",
104+
"repository": {
105+
"repoUrl": "https://github.com/aws-observability/aws-observability-accelerator",
106+
"name": "grafana-dashboards",
107+
"targetRevision": "main",
108+
"path": "./artifacts/grafana-operator-manifests/eks/infrastructure"
109+
},
110+
"values": {
111+
"GRAFANA_CLUSTER_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/cluster.json",
112+
"GRAFANA_KUBELET_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/kubelet.json",
113+
"GRAFANA_NSWRKLDS_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/namespace-workloads.json",
114+
"GRAFANA_NODEEXP_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodeexporter-nodes.json",
115+
"GRAFANA_NODES_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodes.json",
116+
"GRAFANA_WORKLOADS_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/workloads.json"
117+
},
118+
"kustomizations": [
119+
{
120+
"kustomizationPath": "./artifacts/grafana-operator-manifests/eks/infrastructure"
121+
}
122+
]
123+
},
124+
}
99125
```
100-
"context": {
101-
"cluster.dashboard.url": "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/cluster.json",
102-
"kubelet.dashboard.url": "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/kubelet.json",
103-
"namespaceworkloads.dashboard.url": "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/namespace-workloads.json",
104-
"nodeexporter.dashboard.url": "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodeexporter-nodes.json",
105-
"nodes.dashboard.url": "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodes.json",
106-
"workloads.dashboard.url": "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/workloads.json"
107-
}
126+
127+
If you need Java observability you can instead use:
128+
129+
```typescript
130+
"context": {
131+
"fluxRepository": {
132+
"name": "grafana-dashboards",
133+
"namespace": "grafana-operator",
134+
"repository": {
135+
"repoUrl": "https://github.com/aws-observability/aws-observability-accelerator",
136+
"name": "grafana-dashboards",
137+
"targetRevision": "main",
138+
"path": "./artifacts/grafana-operator-manifests/eks/infrastructure"
139+
},
140+
"values": {
141+
"GRAFANA_CLUSTER_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/cluster.json",
142+
"GRAFANA_KUBELET_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/kubelet.json",
143+
"GRAFANA_NSWRKLDS_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/namespace-workloads.json",
144+
"GRAFANA_NODEEXP_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodeexporter-nodes.json",
145+
"GRAFANA_NODES_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodes.json",
146+
"GRAFANA_WORKLOADS_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/workloads.json",
147+
"GRAFANA_JAVA_JMX_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/java/default.json"
148+
},
149+
"kustomizations": [
150+
{
151+
"kustomizationPath": "./artifacts/grafana-operator-manifests/eks/infrastructure"
152+
},
153+
{
154+
"kustomizationPath": "./artifacts/grafana-operator-manifests/eks/java"
155+
}
156+
]
157+
},
158+
"java.pattern.enabled": true
159+
}
108160
```
109161

110162
8. Once all pre-requisites are set you are ready to deploy the pipeline. Run the following command from the root of this repository to deploy the pipeline stack:

‎docs/patterns/single-new-eks-observability-accelerators/single-new-eks-java-opensource-observability.md‎

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
# Single New EKS Cluster Open Source Observability Accelerator - monitoring Java applications
2+
3+
## Objective
4+
5+
This pattern demonstrates how to use the _New EKS Cluster Open Source Observability Accelerator_ with Java based workloads.
6+
7+
## Prerequisites
8+
9+
Ensure that you have installed the following tools on your machine.
10+
11+
1. [aws cli](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html)
12+
2. [kubectl](https://Kubernetes.io/docs/tasks/tools/)
13+
3. [cdk](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html#getting_started_install)
14+
4. [npm](https://docs.npmjs.com/cli/v8/commands/npm-install)
15+
16+
## Deploying
17+
18+
Please follow the _Deploying_ instructions of the [New EKS Cluster Open Source Observability Accelerator](./single-new-eks-opensource-observability.md) pattern, except for step 7, where you need to replace "context" in `~/.cdk.json` with the following:
19+
20+
```typescript
21+
"context": {
22+
"fluxRepository": {
23+
"name": "grafana-dashboards",
24+
"namespace": "grafana-operator",
25+
"repository": {
26+
"repoUrl": "https://github.com/aws-observability/aws-observability-accelerator",
27+
"name": "grafana-dashboards",
28+
"targetRevision": "main",
29+
"path": "./artifacts/grafana-operator-manifests/eks/infrastructure"
30+
},
31+
"values": {
32+
"GRAFANA_CLUSTER_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/cluster.json",
33+
"GRAFANA_KUBELET_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/kubelet.json",
34+
"GRAFANA_NSWRKLDS_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/namespace-workloads.json",
35+
"GRAFANA_NODEEXP_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodeexporter-nodes.json",
36+
"GRAFANA_NODES_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/nodes.json",
37+
"GRAFANA_WORKLOADS_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/infrastructure/workloads.json",
38+
"GRAFANA_JAVA_JMX_DASH_URL" : "https://raw.githubusercontent.com/aws-observability/aws-observability-accelerator/main/artifacts/grafana-dashboards/eks/java/default.json"
39+
},
40+
"kustomizations": [
41+
{
42+
"kustomizationPath": "./artifacts/grafana-operator-manifests/eks/infrastructure"
43+
},
44+
{
45+
"kustomizationPath": "./artifacts/grafana-operator-manifests/eks/java"
46+
}
47+
]
48+
},
49+
"java.pattern.enabled": true
50+
}
51+
```
52+
53+
Once completed the rest of the _Deploying_ steps, you can move on with the deployment of the Java workload.
54+
55+
## Deploy an example Java application
56+
57+
In this section we will reuse an example from the AWS OpenTelemetry collector [repository](https://github.com/aws-observability/aws-otel-collector/blob/main/docs/developers/container-insights-eks-jmx.md). For convenience, the steps can be found below.
58+
59+
1. Clone [this repository](https://github.com/aws-observability/aws-otel-test-framework) and navigate to the `sample-apps/jmx/` directory.
60+
61+
2. Authenticate to Amazon ECR (_AWS_REGION_ was set during the deployment stage)
62+
63+
```bash
64+
export AWS_ACCOUNT_ID=`aws sts get-caller-identity --query Account --output text`
65+
aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com
66+
```
67+
68+
3. Create an Amazon ECR repository
69+
70+
```bash
71+
aws ecr create-repository --repository-name prometheus-sample-tomcat-jmx \
72+
--image-scanning-configuration scanOnPush=true \
73+
--region $AWS_REGION
74+
```
75+
76+
4. Build Docker image and push to ECR.
77+
78+
```bash
79+
docker build -t $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/prometheus-sample-tomcat-jmx:latest .
80+
docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/prometheus-sample-tomcat-jmx:latest
81+
```
82+
83+
5. Install the sample application in the cluster
84+
85+
```bash
86+
SAMPLE_TRAFFIC_NAMESPACE=javajmx-sample
87+
curl https://raw.githubusercontent.com/aws-observability/aws-otel-test-framework/terraform/sample-apps/jmx/examples/prometheus-metrics-sample.yaml |
88+
sed "s/{{aws_account_id}}/$AWS_ACCOUNT_ID/g" |
89+
sed "s/{{region}}/$AWS_REGION/g" |
90+
sed "s/{{namespace}}/$SAMPLE_TRAFFIC_NAMESPACE/g" |
91+
kubectl apply -f -
92+
```
93+
94+
## Verify the resources
95+
96+
```bash
97+
kubectl get pods -n $SAMPLE_TRAFFIC_NAMESPACE
98+
99+
NAME READY STATUS RESTARTS AGE
100+
tomcat-bad-traffic-generator 1/1 Running 0 90m
101+
tomcat-example-77b46cc546-z22jf 1/1 Running 0 25m
102+
tomcat-traffic-generator 1/1 Running 0 90m
103+
```
104+
105+
## Visualization
106+
107+
Login to your Grafana workspace and navigate to the Dashboards panel. You should see a new dashboard named `Java/JMX`, under `Observability Accelerator Dashboards`:
108+
109+
![Dashboard](../images/all-dashboards-java.png)
110+
111+
Open the `Java/JMX` dashboard and you should be able to view its visualization as shown below:
112+
113+
![NodeExporter_Dashboard](../images/java-dashboard.png)
114+
115+
## Teardown
116+
117+
You can teardown the whole CDK stack with the following command:
118+
119+
```bash
120+
make pattern single-new-eks-opensource-observability destroy
121+
```

0 commit comments

Comments
 (0)