Skip to content

Commit b9bc063

Browse files
elamaran11elamaran11
authored
Merge pull request #154 from Howlla/cost_mon
Cost monitoring with kubecost and AMP
2 parents 0e40005 + 3537896 commit b9bc063

File tree

15 files changed

+680
-6
lines changed

15 files changed

+680
-6
lines changed

‎bin/single-new-eks-cost-monitoring.ts‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
import { configureApp, errorHandler } from '../lib/common/construct-utils';
2+
import SingleNewEksCostMonitoringPattern from '../lib/single-new-eks-cost-monitoring-pattern';
3+
4+
const app = configureApp();
5+
6+
new SingleNewEksCostMonitoringPattern()
7+
.buildAsync(app, 'single-new-eks-cost-monitoring')
8+
.catch((e) => {
9+
errorHandler(app, "Secure Ingress Cost monitoring Pattern could not be deployed. \
10+
See Secure Ingress Cost monitoring pattern in the readme for instructions", e);
11+
});

‎cdk.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,4 +58,4 @@
5858
"existing.cluster.name": "single-new-eks-observability-accelerator",
5959
"existing.kubectl.rolename": "YOUR_KUBECTL_ROLE"
6060
}
61-
}
61+
}

‎docs/patterns/images/costmonitoring-ingress.png‎

94.4 KB
Loading

‎docs/patterns/images/kubecost-dashboard.png‎

243 KB
Loading

‎docs/patterns/images/kubecost-namespace-dashboard.png‎

289 KB
Loading

‎docs/patterns/single-new-eks-observability-accelerators/single-new-eks-cost-monitoring-ingress-observability.md‎

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
# Single Cluster Observability - Kubecost Cost Monitoring with Secure Ingress using Cognito
2+
3+
Implementing Kubecost for monitoring EKS clusters provides invaluable insights into resource utilization and cost management. Kubecost offers granular visibility into the cost breakdown of Kubernetes workloads, enabling efficient allocation of resources and optimization of infrastructure spending. By integrating with Amazon Managed Prometheus (AMP) and AWS services such as Application Load Balancer, Amazon Cognito, and Amazon Route 53, Kubecost ensures a comprehensive monitoring solution with secure access control mechanisms. With alerts and recording rules provided by Amazon Managed Service for Prometheus, teams can proactively identify and address potential issues, ensuring optimal performance and cost-effectiveness of EKS deployments. Kubecost's user-friendly dashboard and reporting capabilities empower organizations to make informed decisions, maximize resource efficiency, and maintain cost predictability in their EKS environments, ultimately enhancing operational excellence and driving business growth.
4+
5+
## Architecture
6+
7+
The following figure illustrates the architecture of the pattern we will be deploying for Single EKS cost monitoring (Kubecost) pattern with Application Load Balancer, Amazon Cognito, and a Transport Layer Security (TLS) Certificate on AWS Certificate Manager (ACM) with Amazon Route 53 hosted zone to authenticate users to Kubecost
8+
9+
![Architecture](../images/costmonitoring-ingress.png)
10+
11+
## Objective
12+
13+
- Deploys one production grade Amazon EKS cluster.
14+
- AWS Kubecost with Amazon Managed Prometheus (AMP) integration
15+
- [Secure Ingress with AWS Cognito](https://aws.amazon.com/blogs/containers/securing-kubecost-access-with-amazon-cognito/)
16+
- AWS Certificate Manager with Amazon Route 53 hosted zone
17+
- Alerts and recording rules with Amazon Managed Service for Prometheus
18+
19+
## Prerequisites:
20+
21+
An existing hosted zone in Route53 with the ability to add records.
22+
23+
Ensure that you have installed the following tools on your machine.
24+
25+
1. [aws cli](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html)
26+
2. [kubectl](https://Kubernetes.io/docs/tasks/tools/)
27+
3. [cdk](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html#getting_started_install)
28+
4. [npm](https://docs.npmjs.com/cli/v8/commands/npm-install)
29+
30+
## Configuring domain
31+
32+
The CDK code expects the allowed domain and subdomain names in the CDK context file (cdk.json).
33+
34+
Create two environment variables. The PARENT_HOSTED_ZONE variable contains the name of your Route 53 public hosted zone. The DEV_SUBZONE_NAME will be the address for your Kubecost dashboard.
35+
36+
When users register to cognito they will have to provide an email address, using the `allowed.domains.list` you can specify you enterprise's email domain to only allow your employees to sign up for the service
37+
38+
Generate the cdk.json file:
39+
40+
```bash
41+
PARENT_HOSTED_ZONE=mycompany.a2z.com
42+
DEV_SUBZONE_NAME=kubecost.mycompany.a2z.com
43+
ALLOWED_DOMAIN_LIST=amazon.com
44+
cat << EOF > cdk.json
45+
{
46+
"app": "npx ts-node dist/lib/common/default-main.js",
47+
"context": {
48+
"parent.hostedzone.name": "${PARENT_HOSTED_ZONE}",
49+
"dev.subzone.name": "${DEV_SUBZONE_NAME}",
50+
"allowed.domains.list": "${ALLOWED_DOMAIN_LIST}"
51+
}
52+
}
53+
EOF
54+
```
55+
56+
57+
## Deploying
58+
59+
Please follow the _Deploying_ instructions of the [New EKS Cluster Open Source Observability Accelerator](./single-new-eks-opensource-observability.md) pattern till step 7.
60+
At step 8, execute the following
61+
62+
```bash
63+
make build
64+
make pattern single-new-eks-cost-monitoring deploy
65+
```
66+
## Verifying A record for Route53
67+
68+
Open the AWS console once the deployment is complete.
69+
Navigate to Route53 in AWS console and select the hosted zone you used for the deployment.
70+
Verify the entry of a record matching the DEV_SUBZONE_NAME we used.
71+
72+
## Verify the resources
73+
74+
Run update-kubeconfig command. You should be able to get the command from CDK output message.
75+
76+
```bash
77+
aws eks update-kubeconfig --name single-new-eks-fargate-opensource-observability-accelerator --region <your region> --role-arn arn:aws:iam::xxxxxxxxx:role/single-new-eks-fargate-op-singleneweksfargateopens-xxxxxxxx
78+
```
79+
80+
81+
Let’s verify the resources created by steps above.
82+
83+
```bash
84+
kubectl get pods -o wide -A
85+
```
86+
```
87+
Output:
88+
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
89+
amazon-guardduty aws-guardduty-agent-5lblf 1/1 Running 0 3h43m 10.0.184.135 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
90+
amazon-guardduty aws-guardduty-agent-qzm4j 1/1 Running 0 3h43m 10.0.153.58 ip-10-0-153-58.us-west-2.compute.internal <none> <none>
91+
argocd blueprints-addon-argocd-application-controller-0 1/1 Running 0 3h40m 10.0.128.187 ip-10-0-153-58.us-west-2.compute.internal <none> <none>
92+
argocd blueprints-addon-argocd-applicationset-controller-7d77d5cdjjhm8 1/1 Running 0 3h40m 10.0.148.136 ip-10-0-153-58.us-west-2.compute.internal <none> <none>
93+
argocd blueprints-addon-argocd-dex-server-84dc54844f-lwgss 1/1 Running 0 3h40m 10.0.178.159 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
94+
argocd blueprints-addon-argocd-notifications-controller-597477df8q4btr 1/1 Running 0 3h40m 10.0.166.196 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
95+
argocd blueprints-addon-argocd-redis-79cb6b87dc-tddlm 1/1 Running 0 3h40m 10.0.160.149 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
96+
argocd blueprints-addon-argocd-repo-server-584549c456-5gfs8 1/1 Running 0 3h40m 10.0.146.88 ip-10-0-153-58.us-west-2.compute.internal <none> <none>
97+
argocd blueprints-addon-argocd-server-7b7b488dd4-686tx 1/1 Running 0 3h40m 10.0.175.70 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
98+
aws-for-fluent-bit blueprints-addon-aws-fluent-bit-for-cw-aws-for-fluent-bit-lr99l 1/1 Running 0 3h40m 10.0.160.194 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
99+
aws-for-fluent-bit blueprints-addon-aws-fluent-bit-for-cw-aws-for-fluent-bit-z2pm7 1/1 Running 0 3h40m 10.0.146.233 ip-10-0-153-58.us-west-2.compute.internal <none> <none>
100+
cert-manager cert-manager-6d988558d6-wm746 1/1 Running 0 3h40m 10.0.188.100 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
101+
cert-manager cert-manager-cainjector-6976895488-mk9sw 1/1 Running 0 3h40m 10.0.173.79 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
102+
cert-manager cert-manager-webhook-fcf48cc54-92wqm 1/1 Running 0 3h40m 10.0.133.37 ip-10-0-153-58.us-west-2.compute.internal <none> <none>
103+
default otel-collector-amp-collector-6d768bcbf5-vbmqr 1/1 Running 0 3h39m 10.0.171.253 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
104+
external-dns blueprints-addon-external-dns-78bcd6c7c5-df74q 1/1 Running 0 3h40m 10.0.180.87 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
105+
external-secrets blueprints-addon-external-secrets-675f847b97-kbn98 1/1 Running 0 3h40m 10.0.178.180 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
106+
external-secrets blueprints-addon-external-secrets-cert-controller-68cbb65dspf8c 1/1 Running 0 3h40m 10.0.154.4 ip-10-0-153-58.us-west-2.compute.internal <none> <none>
107+
external-secrets blueprints-addon-external-secrets-webhook-6cfdbdf896-j9ng7 1/1 Running 0 3h40m 10.0.142.78 ip-10-0-153-58.us-west-2.compute.internal <none> <none>
108+
kube-system aws-load-balancer-controller-7cd4b895d4-gvxtv 1/1 Running 0 3h40m 10.0.131.188 ip-10-0-153-58.us-west-2.compute.internal <none> <none>
109+
kube-system aws-load-balancer-controller-7cd4b895d4-m2dh5 1/1 Running 0 3h40m 10.0.173.13 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
110+
kube-system aws-node-7l22p 2/2 Running 0 3h43m 10.0.184.135 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
111+
kube-system aws-node-rfc76 2/2 Running 0 3h43m 10.0.153.58 ip-10-0-153-58.us-west-2.compute.internal <none> <none>
112+
kube-system blueprints-addon-metrics-server-7cb6564d98-jhwmj 1/1 Running 0 3h40m 10.0.182.218 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
113+
kube-system blueprints-addon-secret-store-csi-driver-secrets-store-csi5fbjj 3/3 Running 0 3h40m 10.0.190.108 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
114+
kube-system blueprints-addon-secret-store-csi-driver-secrets-store-csigdgfd 3/3 Running 0 3h40m 10.0.148.234 ip-10-0-153-58.us-west-2.compute.internal <none> <none>
115+
kube-system coredns-5b8cc885bc-t9dpp 1/1 Running 0 3h47m 10.0.132.167 ip-10-0-153-58.us-west-2.compute.internal <none> <none>
116+
kube-system coredns-5b8cc885bc-tkq6g 1/1 Running 0 3h47m 10.0.152.126 ip-10-0-153-58.us-west-2.compute.internal <none> <none>
117+
kube-system csi-secrets-store-provider-aws-ktklg 1/1 Running 0 3h40m 10.0.190.207 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
118+
kube-system csi-secrets-store-provider-aws-qmg44 1/1 Running 0 3h40m 10.0.142.192 ip-10-0-153-58.us-west-2.compute.internal <none> <none>
119+
kube-system ebs-csi-controller-5c4b7b9549-cvv8b 6/6 Running 0 3h40m 10.0.163.2 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
120+
kube-system ebs-csi-controller-5c4b7b9549-d9wfc 6/6 Running 0 3h40m 10.0.146.91 ip-10-0-153-58.us-west-2.compute.internal <none> <none>
121+
kube-system ebs-csi-node-9sxtr 3/3 Running 0 3h40m 10.0.155.48 ip-10-0-153-58.us-west-2.compute.internal <none> <none>
122+
kube-system ebs-csi-node-bcsjk 3/3 Running 0 3h40m 10.0.187.96 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
123+
kube-system kube-proxy-djbgh 1/1 Running 0 3h43m 10.0.153.58 ip-10-0-153-58.us-west-2.compute.internal <none> <none>
124+
kube-system kube-proxy-mck62 1/1 Running 0 3h43m 10.0.184.135 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
125+
kube-system kube-state-metrics-6cf6f65cf7-nzqkb 1/1 Running 0 3h40m 10.0.186.50 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
126+
kube-system ssm-installer-fsfjn 1/1 Running 0 3h41m 10.0.189.79 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
127+
kube-system ssm-installer-vbqqm 1/1 Running 0 3h41m 10.0.154.228 ip-10-0-153-58.us-west-2.compute.internal <none> <none>
128+
kubecost kubecost-cost-analyzer-5769d5f47f-fjwkz 3/3 Running 0 3h40m 10.0.137.248 ip-10-0-153-58.us-west-2.compute.internal <none> <none>
129+
kubecost kubecost-cost-analyzer-prometheus-server-6f48bdc56c-d6789 2/2 Running 0 3h40m 10.0.187.76 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
130+
opentelemetry-operator-system opentelemetry-operator-98f5b9c89-7kp6x 2/2 Running 0 3h39m 10.0.148.126 ip-10-0-153-58.us-west-2.compute.internal <none> <none>
131+
prometheus-node-exporter prometheus-node-exporter-czsx8 1/1 Running 0 3h40m 10.0.184.135 ip-10-0-184-135.us-west-2.compute.internal <none> <none>
132+
prometheus-node-exporter prometheus-node-exporter-jg9tw 1/1 Running 0 3h40m 10.0.153.58 ip-10-0-153-58.us-west-2.compute.internal <none> <none>
133+
```
134+
135+
Now, lets navigate to the URL described as our dev.subzone.name in the cdk.json file and signup with a new cognito user profile.
136+
137+
- **Kubecost Dashboards**
138+
139+
![kubecost-dashboard](../images/kubecost-dashboard.png)
140+
141+
- **Kubecost Namespace Dashboards**
142+
143+
![kubecost-dashboard](../images/kubecost-namespace-dashboard.png)
144+
145+
## Teardown
146+
147+
You can teardown the whole CDK stack with the following command:
148+
149+
```bash
150+
make pattern single-new-eks-cost-monitoring destroy
151+
```

‎lib/common/cognito/cognito-idp-stack.ts‎

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
import * as cdk from 'aws-cdk-lib';
2+
import * as blueprints from '@aws-quickstart/eks-blueprints';
3+
import { Construct } from 'constructs';
4+
import * as cognito from 'aws-cdk-lib/aws-cognito';
5+
import * as lambda from 'aws-cdk-lib/aws-lambda';
6+
import * as iam from 'aws-cdk-lib/aws-iam';
7+
8+
export default class CognitoIdpStack extends cdk.Stack {
9+
10+
public readonly userPoolOut: cognito.UserPool;
11+
public readonly userPoolClientOut: cognito.UserPoolClient;
12+
public readonly userPoolDomainOut: cognito.UserPoolDomain;
13+
14+
constructor(scope: Construct, id: string, subDomain: string, props?: cdk.StackProps) {
15+
super(scope, id, props);
16+
17+
const lambdaExecutionRole = new iam.Role(this, 'Lambda Execution Role', {
18+
assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'),
19+
});
20+
21+
lambdaExecutionRole.addManagedPolicy(iam.ManagedPolicy.fromAwsManagedPolicyName("service-role/AWSLambdaBasicExecutionRole"));
22+
lambdaExecutionRole.addManagedPolicy(iam.ManagedPolicy.fromAwsManagedPolicyName("AmazonSSMReadOnlyAccess"));
23+
24+
const authChallengeFn = new lambda.Function(this, 'authChallengeFn', {
25+
runtime: lambda.Runtime.PYTHON_3_12,
26+
code: lambda.Code.fromAsset('./lib/common/cognito/lambda'),
27+
handler: 'lambda_function.lambda_handler',
28+
role: lambdaExecutionRole,
29+
environment: {
30+
"ALLOWED_DOMAINS_LIST": blueprints.utils.valueFromContext(scope, "allowed.domains.list", "amazon.com")
31+
}
32+
});
33+
34+
35+
// Cognito User Pool
36+
const userPool = new cognito.UserPool(this, 'CognitoIDPUserPool', {
37+
userPoolName: 'CognitoIDPUserPool',
38+
selfSignUpEnabled: true,
39+
signInAliases: {
40+
email: true,
41+
username: true
42+
},
43+
standardAttributes: {
44+
email: {
45+
mutable: true,
46+
required: true
47+
},
48+
givenName: {
49+
mutable: true,
50+
required: true
51+
},
52+
familyName: {
53+
mutable: true,
54+
required: true
55+
}
56+
},
57+
lambdaTriggers: {
58+
preSignUp: authChallengeFn,
59+
preAuthentication: authChallengeFn,
60+
},
61+
});
62+
63+
64+
// Output the User Pool ID
65+
66+
this.userPoolOut = userPool;
67+
68+
new cdk.CfnOutput(this, 'CognitoIDPUserPoolOut', {
69+
value: userPool.userPoolId,
70+
exportName: 'CognitoIDPUserPoolId'
71+
});
72+
73+
new cdk.CfnOutput(this, 'CognitoIDPUserPoolArnOut', {
74+
value: userPool.userPoolArn,
75+
exportName: 'CognitoIDPUserPoolArn'
76+
});
77+
78+
79+
// We will ask the IDP to redirect back to our domain's index page
80+
const redirectUri = `https://${subDomain}/oauth2/idpresponse`;
81+
82+
// Configure the user pool client application
83+
const userPoolClient = new cognito.UserPoolClient(this, 'CognitoAppClient', {
84+
userPool,
85+
authFlows: {
86+
userPassword: true
87+
},
88+
oAuth: {
89+
flows: {
90+
authorizationCodeGrant: true
91+
},
92+
scopes: [
93+
cognito.OAuthScope.OPENID
94+
],
95+
callbackUrls: [redirectUri]
96+
// TODO - What about logoutUrls?
97+
},
98+
generateSecret: true,
99+
userPoolClientName: 'Web',
100+
supportedIdentityProviders: [cognito.UserPoolClientIdentityProvider.COGNITO]
101+
});
102+
103+
// Output the User Pool App Client ID
104+
this.userPoolClientOut = userPoolClient;
105+
106+
new cdk.CfnOutput(this, 'CognitoIDPUserPoolClientOut', {
107+
value: userPoolClient.userPoolClientId,
108+
exportName: 'CognitoIDPUserPoolClientId'
109+
});
110+
111+
// Add the domain to the user pool
112+
const randomText = (Math.random() + 1).toString(36).substring(7);
113+
const userPoolDomain = userPool.addDomain('CognitoDomain', {
114+
cognitoDomain: {
115+
domainPrefix: `my-cdk-blueprint-${randomText}`,
116+
},
117+
});
118+
119+
// Output the User Pool App Client ID
120+
121+
this.userPoolDomainOut = userPoolDomain;
122+
123+
new cdk.CfnOutput(this, 'CognitoIDPUserPoolDomainOut', {
124+
value: userPoolDomain.domainName,
125+
exportName: 'CognitoIDPUserPoolDomain'
126+
});
127+
128+
}
129+
}

0 commit comments

Comments
 (0)