Skip to content

Commit 008d44b

Browse files
Friz-zyFriz-zy
committed
up readme and example
1 parent adf43b7 commit 008d44b

File tree

5 files changed

+127
-25
lines changed

5 files changed

+127
-25
lines changed

‎README.md‎

Lines changed: 48 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,52 @@
33
[![#StandWithBelarus](https://img.shields.io/badge/Belarus-red?label=%23%20Stand%20With&labelColor=white&color=red)
44
<img src="https://upload.wikimedia.org/wikipedia/commons/thumb/e/ea/Presidential_Standard_of_Belarus_%28fictional%29.svg/240px-Presidential_Standard_of_Belarus_%28fictional%29.svg.png" width="20" height="20" alt="Voices From Belarus" />](https://bysol.org/en/) [![Stand With Ukraine](https://raw.githubusercontent.com/vshymanskyy/StandWithUkraine/main/badges/StandWithUkraine.svg)](https://vshymanskyy.github.io/StandWithUkraine)
55

6-
Setup basic EKS cluster with necessary controllers. Examples for further configuring the EKS cluster can be found in [eks blueprints](https://github.com/aws-ia/terraform-aws-eks-blueprints/tree/main), [tEKS](https://github.com/particuleio/teks) and [eks demo](https://github.com/awslabs/eksdemo) repos.
6+
Setup EKS cluster with necessary controllers, operators and monitoring stack. Similar projects:
7+
- [eks blueprints](https://github.com/aws-ia/terraform-aws-eks-blueprints/tree/main)
8+
- [tEKS](https://github.com/particuleio/teks)
9+
- [eks demo](https://github.com/awslabs/eksdemo)
10+
11+
## What is included
12+
13+
|| Description | Purpose | Enabled | DNS ||
14+
|EKS cluster module based on [terraform-aws-modules/eks/aws](https://github.com/terraform-aws-modules/terraform-aws-eks) v19|Base|True||
15+
|Templates for Managed Node Groups and Fargate Profile to link them to each availability zone instead of all zones at once|Base|True||
16+
|Integration of modules with each other and reasonable default values|Base|True||
17+
|[CoreDNS EKS addon](https://docs.aws.amazon.com/eks/latest/userguide/managing-coredns.html)|Core|True||
18+
|[Kube-Proxy EKS addon](https://docs.aws.amazon.com/eks/latest/userguide/managing-kube-proxy.html)|Core|True||
19+
|[VPC CNI EKS addon](https://docs.aws.amazon.com/eks/latest/userguide/managing-vpc-cni.html)|Core|True||
20+
|[AWS EBS CSI driver EKS addon](https://docs.aws.amazon.com/eks/latest/userguide/managing-ebs-csi.html)|Core|True||
21+
|[Snapshot Controller EKS addon](https://docs.aws.amazon.com/eks/latest/userguide/csi-snapshot-controller.html)|Core|True||
22+
|[AWS EFS CSI driver](https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/blob/main/docs/addons/aws-efs-csi-driver.md)|Core|True||
23+
|[AWS Node Termination Handler](https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/blob/main/docs/addons/aws-node-termination-handler.md)|Core|True||
24+
|[Cert Manager](https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/blob/main/docs/addons/cert-manager.md)|Core|True||
25+
|[Cluster Autoscaler](https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/blob/main/docs/addons/cluster-autoscaler.md)|Core|True||
26+
|[Metrics Server](https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/blob/main/docs/addons/metrics-server.md)|Core|True||
27+
|[Vertical Pod Autoscaler](https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/blob/main/docs/addons/vertical-pod-autoscaler.md)|Core|True||
28+
|[Ingress Apisix](https://github.com/apache/apisix-ingress-controller)|Ingress|True||
29+
|[Ingress Nginx](https://github.com/kubernetes/ingress-nginx)|Ingress|False||
30+
|[Victoriametrics Operator](https://github.com/VictoriaMetrics/operator)|Operator|True||
31+
|[Opentelemetry Operator](https://github.com/open-telemetry/opentelemetry-operator)|Operator|False||
32+
|[Clickhouse Operator](https://github.com/Altinity/clickhouse-operator)|Operator|False||
33+
|[Grafana Operator](https://artifacthub.io/packages/helm/bitnami/grafana-operator)|Operator|True||
34+
|[Victoriametrics](https://github.com/VictoriaMetrics/helm-charts/blob/master/charts/victoria-metrics-k8s-stack/README.md)|Monitoring|True|vmauth.${var.ingress_domain}<br>victoriametrics.${var.ingress_domain}<br>vmalertmanager.${var.ingress_domain}<br>vmagent.${var.ingress_domain}<br>vmalert.${var.ingress_domain}|
35+
|[Grafana](https://grafana.com/oss/grafana/)|Monitoring|True|grafana.${var.ingress_domain}|
36+
|[Uptrace](https://uptrace.dev/)|Monitoring|True|uptrace.${var.ingress_domain}|
37+
|[Vector](https://vector.dev/)|Monitoring|True||
38+
|[Qryn](https://qryn.metrico.in)|Monitoring|False|qryn.${var.ingress_domain}|
39+
|[Openobserve](https://openobserve.ai/)|Monitoring|False|openobserve.${var.ingress_domain}|
40+
|[Kubernetes Dashboard](https://github.com/kubernetes/dashboard)|Control|False|k8s-dashboard.${var.ingress_domain}|
41+
42+
## What is not included right now
43+
44+
- email integration
45+
- dns integration
46+
- alert rules
47+
- resources limits
48+
- ci & cd integration
49+
- network policies
50+
- host-based pod segregation
51+
- ...
752

853
## Depend on
954
- terraform
@@ -45,21 +90,11 @@ After `terraform destroy` check ec2 volumes for unused disks as aws-ebs-csi-driv
4590

4691
Helm upgrade `reset_values` flag set to `true` for everything except databases like postgresql and clickhouse, see this [explain](https://shipmight.com/blog/understanding-helm-upgrade-reset-reuse-values)
4792

48-
## Variables
49-
5093
## Outputs
5194

52-
| Name | Description |
53-
|------|-------------|
54-
|region|The AWS region|
55-
|vpc_id|The ID of the target VPC|
56-
|cluster_name|The name of the EKS|
57-
|cluster_endpoint|Endpoint for your Kubernetes API server|
58-
|cluster_certificate_authority_data|Base64 encoded certificate data required to communicate with the cluster|
59-
60-
Also `~/.kube/eks-${account_id}-${region}-${cluster_name}` will be created by `aws eks` utility.
95+
Check the [./example/outputs.example](./example/outputs.example) file to get an example of the output. For setting DNS you can describe ingress external address with kubectl: `kubectl get service/apisix-ingress-controller-apisix-gateway -n ingress-apisix`.
6196

62-
You can describe apisix ingress external address with kubectl: `kubectl get service/apisix-ingress-controller-apisix-gateway -n ingress-apisix`.
97+
Also `~/.kube/eks-${account_id}-${region}-${cluster_name}` kubeconfig will be created by `aws eks` utility.
6398

6499
## About the Author
65100

‎example/main.tf‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,8 @@ locals {
2222

2323
# Limit number of availability zones in generating from templates.
2424
# Could be from 0 and up to ${number_of_multi_az}
25+
# In given example 2 AZ would be used for each node group for fault tolerance.
26+
# The price for that would be to pay for multi AZ traffic
2527
self_managed_node_group_number_of_multi_az = 2
2628
eks_managed_node_group_number_of_multi_az = 2
2729
fargate_profile_number_of_multi_az = 2
@@ -36,7 +38,7 @@ locals {
3638
desired_size = 1
3739
max_size = 3
3840

39-
instance_types = ["m6a.large"] # 2cpu 8gb ram 62$\mo https://instances.vantage.sh/aws/ec2/m6a.large
41+
instance_types = ["m6a.large"] # 2cpu 8gb ram 63$\mo https://instances.vantage.sh/aws/ec2/m6a.large
4042

4143
labels = {
4244
"node.kubernetes.io/purpose" = "management"
@@ -150,8 +152,8 @@ module "eks" {
150152
enable_ingress_apisix = true
151153
enable_ingress_nginx = false
152154
enable_victoriametrics_operator = true
153-
enable_opentelemetry_operator = true
154-
enable_clickhouse_operator = true
155+
enable_opentelemetry_operator = false
156+
enable_clickhouse_operator = false
155157
enable_grafana_operator = true
156158
enable_victoriametrics = true
157159
enable_grafana = true

‎example/top.example‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
$ kubectl top node
2+
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
3+
ip-10-0-1-18.us-east-2.compute.internal 1420m 73% 2866Mi 40%
4+
ip-10-0-3-81.us-east-2.compute.internal 421m 21% 2070Mi 29%
5+
6+
$ kubectl top pod --all-namespaces
7+
NAMESPACE NAME CPU(cores) MEMORY(bytes)
8+
cert-manager cert-manager-796cbd6574-2t4ck 1m 21Mi
9+
cert-manager cert-manager-cainjector-6b6c9ff467-vfxzm 2m 40Mi
10+
cert-manager cert-manager-webhook-d969fd76f-vxwzz 1m 8Mi
11+
clickhouse altinity-clickhouse-operator-6c64dbbd87-lfl82 1m 17Mi
12+
ingress-apisix apisix-ingress-controller-75579d89db-rlzjq 4m 89Mi
13+
kube-system aws-node-hrd4s 4m 60Mi
14+
kube-system aws-node-msngk 4m 60Mi
15+
kube-system cluster-autoscaler-aws-cluster-autoscaler-69b4dcd58f-q5drj 2m 34Mi
16+
kube-system coredns-656f8bb969-gl8j2 2m 16Mi
17+
kube-system coredns-656f8bb969-lm69l 2m 15Mi
18+
kube-system ebs-csi-controller-7f8c48d5b9-6gttw 2m 47Mi
19+
kube-system ebs-csi-controller-7f8c48d5b9-rv7zk 2m 53Mi
20+
kube-system ebs-csi-node-6s7b6 1m 19Mi
21+
kube-system ebs-csi-node-czdzb 1m 19Mi
22+
kube-system efs-csi-controller-7c77bfcd6b-bkwx9 2m 42Mi
23+
kube-system efs-csi-controller-7c77bfcd6b-fp5tz 2m 40Mi
24+
kube-system efs-csi-node-hzdbf 3m 41Mi
25+
kube-system efs-csi-node-w64r7 3m 39Mi
26+
kube-system kube-proxy-j4z7p 1m 14Mi
27+
kube-system kube-proxy-vt95s 1m 15Mi
28+
kube-system metrics-server-b7f8d757d-6tjgm 4m 19Mi
29+
kube-system snapshot-controller-85bbdcdf64-4jnct 1m 6Mi
30+
kube-system snapshot-controller-85bbdcdf64-6rz7p 1m 7Mi
31+
kubernetes-dashboard kubernetes-dashboard-api-7ff949576d-4knct 1m 8Mi
32+
kubernetes-dashboard kubernetes-dashboard-auth-7b6cfb784d-vdcsk 1m 7Mi
33+
kubernetes-dashboard kubernetes-dashboard-kong-75bb76dd5f-tvgm9 2m 74Mi
34+
kubernetes-dashboard kubernetes-dashboard-metrics-scraper-7f6f977dc9-pp8sl 1m 7Mi
35+
kubernetes-dashboard kubernetes-dashboard-web-7cccf8c454-x75gm 0m 7Mi
36+
monitoring grafana-6fbfd67f58-whkmm 3m 47Mi
37+
monitoring grafana-operator-67db8569bd-pg6sb 2m 28Mi
38+
monitoring opentelemetry-operator-769898647b-2nqhr 1m 31Mi
39+
monitoring qryn-clickhouse-shard0-0 422m 459Mi
40+
monitoring qryn-helm-877cc59c-cds4d 2m 113Mi
41+
monitoring qryn-helm-877cc59c-dclxn 2m 104Mi
42+
monitoring qryn-helm-877cc59c-lv4zk 952m 449Mi
43+
monitoring qryn-helm-877cc59c-tnsbj 2m 114Mi
44+
monitoring qryn-helm-877cc59c-trh4b 2m 115Mi
45+
monitoring uptrace-0 132m 125Mi
46+
monitoring uptrace-clickhouse-shard0-0 640m 609Mi
47+
monitoring uptrace-postgresql-0 6m 41Mi
48+
monitoring vector-77h76 6m 26Mi
49+
monitoring vector-vznj2 2m 29Mi
50+
monitoring victoria-metrics-auth-574d6768dc-zhbgv 1m 4Mi
51+
monitoring victoria-metrics-k8s-stack-kube-state-metrics-889cfcf97-9rfrk 3m 14Mi
52+
monitoring victoria-metrics-k8s-stack-prometheus-node-exporter-c85sm 1m 7Mi
53+
monitoring victoria-metrics-k8s-stack-prometheus-node-exporter-cd9vk 1m 7Mi
54+
monitoring victoria-metrics-operator-677fdb8948-bv9kx 2m 21Mi
55+
monitoring vmagent-victoria-metrics-k8s-stack-6d4655df46-67pv6 50m 125Mi
56+
monitoring vmalert-victoria-metrics-k8s-stack-6b9496f4f-6dxd5 17m 21Mi
57+
monitoring vmalertmanager-victoria-metrics-k8s-stack-0 1m 23Mi
58+
monitoring vmsingle-victoria-metrics-k8s-stack-59d857bcd4-4z8gl 54m 249Mi
59+
vpa vpa-admission-controller-7ffd6d94f9-l6l7x 1m 11Mi
60+
vpa vpa-recommender-75bf5fbc55-g9fmw 1m 14Mi
61+
vpa vpa-updater-64c774dbb6-mjk28 1m 16Mi

‎main.tf‎

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -426,35 +426,35 @@ module "addons" {
426426
eks_addons = local.eks_addons
427427
eks_addons_timeouts = var.eks_addons_timeouts
428428

429-
# https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/blob/0e9d6c9b7115ecf0404c377c9c2529bffa56d10d/docs/addons/aws-efs-csi-driver.md
429+
# https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/blob/main/docs/addons/aws-efs-csi-driver.md
430430
# https://github.com/kubernetes-sigs/aws-efs-csi-driver/blob/master/charts/aws-efs-csi-driver/values.yaml
431431
enable_aws_efs_csi_driver = var.enable_aws_efs_csi_driver
432432
aws_efs_csi_driver = local.aws_efs_csi_driver_config
433433

434-
# https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/blob/0e9d6c9b7115ecf0404c377c9c2529bffa56d10d/docs/addons/aws-node-termination-handler.md
434+
# https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/blob/main/docs/addons/aws-node-termination-handler.md
435435
# https://github.com/aws/aws-node-termination-handler/blob/main/config/helm/aws-node-termination-handler/values.yaml
436436
enable_aws_node_termination_handler = var.enable_aws_node_termination_handler && length(local.aws_node_termination_handler_asg_arns) > 0
437437
aws_node_termination_handler = local.aws_node_termination_handler_config
438438
aws_node_termination_handler_sqs = var.aws_node_termination_handler_sqs
439439
aws_node_termination_handler_asg_arns = local.aws_node_termination_handler_asg_arns
440440

441-
# https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/blob/0e9d6c9b7115ecf0404c377c9c2529bffa56d10d/docs/addons/cert-manager.md
441+
# https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/blob/main/docs/addons/cert-manager.md
442442
# https://github.com/cert-manager/cert-manager/blob/master/deploy/charts/cert-manager/values.yaml
443443
enable_cert_manager = var.enable_cert_manager
444444
cert_manager = local.cert_manager_config
445445
cert_manager_route53_hosted_zone_arns = var.cert_manager_route53_hosted_zone_arns
446446

447-
# https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/blob/0e9d6c9b7115ecf0404c377c9c2529bffa56d10d/docs/addons/cluster-autoscaler.md
447+
# https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/blob/main/docs/addons/cluster-autoscaler.md
448448
# https://github.com/kubernetes/autoscaler/blob/master/charts/cluster-autoscaler/values.yaml
449449
enable_cluster_autoscaler = var.enable_cluster_autoscaler
450450
cluster_autoscaler = local.cluster_autoscaler_config
451451

452-
# https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/blob/0e9d6c9b7115ecf0404c377c9c2529bffa56d10d/docs/addons/metrics-server.md
452+
# https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/blob/main/docs/addons/metrics-server.md
453453
# https://github.com/kubernetes-sigs/metrics-server/blob/master/charts/metrics-server/values.yaml
454454
enable_metrics_server = var.enable_metrics_server
455455
metrics_server = local.metrics_server_config
456456

457-
# https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/blob/0e9d6c9b7115ecf0404c377c9c2529bffa56d10d/docs/addons/vertical-pod-autoscaler.md
457+
# https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/blob/main/docs/addons/vertical-pod-autoscaler.md
458458
# https://github.com/FairwindsOps/charts/blob/master/stable/vpa/values.yaml
459459
enable_vpa = var.enable_vpa
460460
vpa = local.vpa_config
@@ -942,7 +942,11 @@ module "uptrace" {
942942
uptrace:
943943
config:
944944
site:
945+
%{~ if coalesce(var.uptrace_cert_manager_issuer, var.cert_manager_issuer, false) ~}
945946
addr: 'https://uptrace.${var.ingress_domain}/'
947+
%{ else }
948+
addr: 'http://uptrace.${var.ingress_domain}/'
949+
%{ endif }
946950
ingress:
947951
enabled: true
948952
className: ${var.ingress_class_name}

‎variables.tf‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -431,7 +431,7 @@ variable "victoriametrics_operator_values" {
431431
variable "enable_opentelemetry_operator" {
432432
description = "Install Opentelemetry Operator"
433433
type = bool
434-
default = true
434+
default = false
435435
}
436436

437437
variable "opentelemetry_operator_chart_version" {
@@ -463,7 +463,7 @@ variable "opentelemetry_operator_values" {
463463
variable "enable_clickhouse_operator" {
464464
description = "Install Clickhouse Operator"
465465
type = bool
466-
default = true
466+
default = false
467467
}
468468

469469
variable "clickhouse_operator_chart_version" {
@@ -893,7 +893,7 @@ variable "vector_agent_values" {
893893
variable "enable_kubernetes_dashboard" {
894894
description = "Install Kubernetes Dashboard"
895895
type = bool
896-
default = true
896+
default = false
897897
}
898898

899899
variable "kubernetes_dashboard_chart_version" {

0 commit comments

Comments
 (0)