Skip to content

Commit 7a6901d

Browse files
gambol99gambol99
committed
feat: added the pod identity role, permissions and configuration for kubecosts
1 parent 24b4d07 commit 7a6901d

File tree

7 files changed

+518
-149
lines changed

7 files changed

+518
-149
lines changed

‎README.md‎

Lines changed: 182 additions & 108 deletions
Original file line numberDiff line numberDiff line change
@@ -41,13 +41,7 @@ This module provides a production-ready EKS cluster with integrated platform ser
4141
- **Terranetes**: Terraform-as-a-Service platform
4242
- **AWS ACK IAM**: AWS Controllers for Kubernetes
4343
- **CloudWatch Observability**: Monitoring and logging
44-
45-
### 🏗️ **Networking**
46-
47-
- Optional VPC creation with configurable CIDR blocks
48-
- Transit Gateway integration for multi-VPC connectivity
49-
- NAT Gateway support (single AZ or all AZs)
50-
- Subnet tagging for Kubernetes service discovery
44+
- **Kubecost**: Cost monitoring and optimization with AWS CUR integration
5145

5246
## Usage
5347

@@ -112,36 +106,6 @@ module "eks" {
112106
}
113107
```
114108

115-
### EKS Cluster with Custom Networking
116-
117-
```hcl
118-
module "eks" {
119-
source = "appvia/eks/aws"
120-
version = "1.0.0"
121-
122-
cluster_name = "custom-network-eks"
123-
tags = {
124-
Environment = "Production"
125-
Product = "EKS"
126-
Owner = "Engineering"
127-
}
128-
129-
# Custom VPC configuration
130-
vpc_cidr = "10.100.0.0/21"
131-
availability_zones = 3
132-
nat_gateway_mode = "all_azs"
133-
private_subnet_netmask = 24
134-
public_subnet_netmask = 24
135-
136-
# Transit Gateway integration
137-
transit_gateway_id = "tgw-1234567890"
138-
transit_gateway_routes = {
139-
private = "10.0.0.0/8"
140-
database = "10.1.0.0/16"
141-
}
142-
}
143-
```
144-
145109
### EKS Cluster with Pod Identity
146110

147111
```hcl
@@ -208,6 +172,86 @@ module "eks" {
208172
}
209173
```
210174

175+
## Networking Options
176+
177+
### VPC & Networking
178+
179+
The module assumes the account alread has an existing VPC to provision the cluster within. We need the VPC ID and the subnet IDs for the private subnets where the cluster should be located.
180+
181+
```hcl
182+
# Use existing VPC
183+
vpc_id = "vpc-1234567890"
184+
private_subnet_ids = ["subnet-1234567890", "subnet-0987654321"]
185+
```
186+
187+
## Security Features
188+
189+
### Access Control
190+
191+
Configure cluster access using access entries.
192+
193+
```hcl
194+
access_entries = {
195+
admin = {
196+
principal_arn = "arn:aws:iam::123456789012:role/AdminRole"
197+
policy_associations = {
198+
cluster_admin = {
199+
policy_arn = "arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy"
200+
access_scope = {
201+
type = "cluster"
202+
}
203+
}
204+
}
205+
}
206+
}
207+
```
208+
209+
### Pod Identity
210+
211+
Secure workload-to-AWS service authentication.
212+
213+
```hcl
214+
pod_identity = {
215+
my-workload = {
216+
enabled = true
217+
name = "my-workload-identity"
218+
namespace = "my-namespace"
219+
service_account = "my-service-account"
220+
managed_policy_arns = {
221+
"S3Access" = "arn:aws:iam::aws:policy/AmazonS3FullAccess"
222+
}
223+
}
224+
}
225+
```
226+
227+
### Security Groups
228+
229+
Customize security group rules for cluster and nodes.
230+
231+
```hcl
232+
cluster_security_group_additional_rules = {
233+
custom_ingress = {
234+
description = "Custom ingress rule"
235+
protocol = "tcp"
236+
from_port = 8080
237+
to_port = 8080
238+
type = "ingress"
239+
cidr_blocks = ["10.0.0.0/8"]
240+
}
241+
}
242+
243+
node_security_group_additional_rules = {
244+
custom_egress = {
245+
description = "Custom egress rule"
246+
protocol = "tcp"
247+
from_port = 443
248+
to_port = 443
249+
type = "egress"
250+
cidr_blocks = ["0.0.0.0/0"]
251+
}
252+
}
253+
```
254+
211255
## Platform Services
212256

213257
### ArgoCD
@@ -302,110 +346,137 @@ cloudwatch_observability = {
302346
}
303347
```
304348

305-
## Networking Options
349+
## Kubecost Cost Monitoring
306350

307-
### VPC Creation
351+
Kubecost provides comprehensive cost monitoring and optimization for Kubernetes clusters with advanced AWS integration capabilities.
308352

309-
The module can create a new VPC or use an existing one.
353+
### Overview
310354

311-
```hcl
312-
# Create new VPC
313-
vpc_cidr = "10.0.0.0/21"
314-
availability_zones = 3
355+
Kubecost offers three main deployment modes:
315356

316-
# Use existing VPC
317-
vpc_id = "vpc-1234567890"
318-
private_subnet_ids = ["subnet-1234567890", "subnet-0987654321"]
319-
```
357+
1. **Standalone**: Single cluster cost monitoring
358+
2. **Federated Storage**: Multi-cluster aggregation for centralized monitoring
359+
3. **Cloud Costs**: Integration with AWS Cost and Usage Reports (CUR) via Athena
320360

321-
### Transit Gateway Integration
361+
### Prerequisites
322362

323-
Connect to existing transit gateways for multi-VPC connectivity.
363+
Before setting up Kubecost, ensure you have:
324364

325-
```hcl
326-
transit_gateway_id = "tgw-1234567890"
327-
transit_gateway_routes = {
328-
private = "10.0.0.0/8"
329-
database = "10.1.0.0/16"
330-
}
331-
```
365+
- An active AWS account with appropriate permissions
366+
- S3 buckets for data storage and Athena query results
367+
- AWS Cost and Usage Report (CUR) configured (for cloud costs feature)
368+
- Amazon Athena setup with Glue database and table (for cloud costs feature)
332369

333-
### NAT Gateway Configuration
370+
### AWS CUR Setup
334371

335-
Configure NAT gateways for outbound internet access.
372+
To enable cloud costs analysis, you need to set up AWS Cost and Usage Reports:
336373

337-
```hcl
338-
nat_gateway_mode = "single_az" # or "all_azs"
339-
```
374+
1. **Create CUR in AWS Billing Console**:
375+
- Navigate to AWS Billing Dashboard
376+
- Create a new Cost and Usage Report with daily granularity
377+
- Enable Resource IDs and Athena integration
378+
- Specify an S3 bucket for CUR data storage
340379

341-
## Security Features
380+
2. **Set up Athena Integration**:
381+
- Use the AWS-provided CloudFormation template to create Athena resources
382+
- Create an S3 bucket for Athena query results
383+
- Configure Athena workgroup and database
342384

343-
### Access Control
385+
### IAM Permissions
344386

345-
Configure cluster access using access entries.
387+
The module automatically provisions the necessary IAM roles and policies:
388+
389+
- **S3 Access**: Read/write access to federated and CUR buckets
390+
- **Athena Operations**: Query execution, monitoring, and result retrieval
391+
- **Glue Metadata**: Database and table schema access for CUR data
392+
393+
Cost monitoring and optimization for Kubernetes clusters with AWS integration.
394+
395+
#### Standalone Kubecost
396+
397+
Basic cost monitoring for a single cluster.
346398

347399
```hcl
348-
access_entries = {
349-
admin = {
350-
principal_arn = "arn:aws:iam::123456789012:role/AdminRole"
351-
policy_associations = {
352-
cluster_admin = {
353-
policy_arn = "arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy"
354-
access_scope = {
355-
type = "cluster"
356-
}
357-
}
358-
}
359-
}
400+
kubecosts = {
401+
enabled = true
360402
}
361403
```
362404

363-
### Pod Identity
405+
#### Federated Storage (Multi-Cluster Aggregation)
364406

365-
Secure workload-to-AWS service authentication.
407+
Aggregate cost data from multiple clusters into a primary cluster for centralized monitoring.
366408

367409
```hcl
368-
pod_identity = {
369-
my-workload = {
370-
enabled = true
371-
name = "my-workload-identity"
372-
namespace = "my-namespace"
373-
service_account = "my-service-account"
374-
managed_policy_arns = {
375-
"S3Access" = "arn:aws:iam::aws:policy/AmazonS3FullAccess"
376-
}
410+
# Primary cluster (aggregates data from all clusters)
411+
kubecosts = {
412+
enabled = true
413+
federated_storage = {
414+
federated_bucket_arn = "arn:aws:s3:::kubecost-federated-bucket"
415+
create_bucket = true
416+
allowed_principals = [
417+
"ACCOUNT_ID"
418+
]
377419
}
378420
}
421+
422+
# Secondary clusters (send data to primary)
423+
kubecosts_agent = {
424+
enabled = true
425+
federated_bucket_name = "arn:aws:s3:::kubecost-federated-bucket"
426+
}
379427
```
380428

381-
### Security Groups
429+
#### Cloud Costs via AWS CUR
382430

383-
Customize security group rules for cluster and nodes.
431+
Integrate with AWS Cost and Usage Reports (CUR) via Amazon Athena for comprehensive cloud cost analysis.
384432

385433
```hcl
386-
cluster_security_group_additional_rules = {
387-
custom_ingress = {
388-
description = "Custom ingress rule"
389-
protocol = "tcp"
390-
from_port = 8080
391-
to_port = 8080
392-
type = "ingress"
393-
cidr_blocks = ["10.0.0.0/8"]
434+
kubecosts = {
435+
enabled = true
436+
fedarated_storage = {
437+
federated_bucket_arn = "my-kubecost-bucket"
394438
}
395-
}
396-
397-
node_security_group_additional_rules = {
398-
custom_egress = {
399-
description = "Custom egress rule"
400-
protocol = "tcp"
401-
from_port = 443
402-
to_port = 443
403-
type = "egress"
404-
cidr_blocks = ["0.0.0.0/0"]
439+
440+
# Cloud costs integration with AWS CUR via Athena
441+
cloud_costs = {
442+
enable = true
443+
cur_bucket_name = "my-cur-bucket"
444+
athena_bucket_arn = "arn:s3:aws:::my-athena-results-bucket"
445+
athena_database_name = "cost_and_usage_data"
446+
athena_table_name = "cur_table"
405447
}
406448
}
407449
```
408450

451+
### Verification
452+
453+
After deployment, verify Kubecost is working correctly:
454+
455+
1. **Access the Dashboard**:
456+
457+
```bash
458+
kubectl port-forward -n kubecost svc/kubecost-cost-analyzer 9090:9090
459+
```
460+
461+
2. **Check Cloud Integration**:
462+
- Navigate to Settings → Cloud Cost Settings
463+
- Verify AWS integration is active
464+
- Check for any error messages
465+
466+
3. **Monitor Logs**:
467+
468+
```bash
469+
kubectl logs -n kubecost deployment/kubecost-cost-analyzer
470+
```
471+
472+
### Additional Resources
473+
474+
- [Kubecost Documentation](https://docs.kubecost.com/)
475+
- [AWS Cloud Billing Integration](https://docs.kubecost.com/cloud-integration/aws-cloud-integration)
476+
- [Multi-Cluster Setup](https://docs.kubecost.com/install-and-configure/install/multi-cluster)
477+
- [AWS CUR Setup Guide](https://docs.aws.amazon.com/cur/latest/userguide/)
478+
- [Athena Integration](https://docs.aws.amazon.com/athena/latest/ug/)
479+
409480
## Examples
410481

411482
See the [examples](./examples/) directory for complete usage examples:
@@ -414,6 +485,7 @@ See the [examples](./examples/) directory for complete usage examples:
414485
- [Platform Services](./examples/platform/) - EKS with integrated platform services
415486
- [Custom Networking](./examples/networking/) - EKS with custom VPC and transit gateway
416487
- [Pod Identity](./examples/pod-identity/) - EKS with custom pod identities
488+
- [Kubecost Cost Monitoring](./examples/kubecost/) - EKS with Kubecost cost monitoring and AWS integration
417489

418490
## Requirements
419491

@@ -474,6 +546,8 @@ The `terraform-docs` utility is used to generate this README. Follow the below s
474546
| <a name="input_kms_key_administrators"></a> [kms\_key\_administrators](#input\_kms\_key\_administrators) | A list of IAM ARNs for EKS key administrators. If no value is provided, the current caller identity is used to ensure at least one key admin is available. | `list(string)` | `[]` | no |
475547
| <a name="input_kms_key_service_users"></a> [kms\_key\_service\_users](#input\_kms\_key\_service\_users) | A list of IAM ARNs for EKS key service users. | `list(string)` | `[]` | no |
476548
| <a name="input_kms_key_users"></a> [kms\_key\_users](#input\_kms\_key\_users) | A list of IAM ARNs for EKS key users. | `list(string)` | `[]` | no |
549+
| <a name="input_kubecosts"></a> [kubecosts](#input\_kubecosts) | The Kubecost configuration | <pre>object({<br/> ## Indicates if we should enable the Kubecost platform<br/> enable = optional(bool, false)<br/> ## The namespace to deploy the Kubecost platform to<br/> namespace = optional(string, "kubecost")<br/> ## The service account to deploy the Kubecost platform to<br/> service_account = optional(string, "kubecost")<br/> ## Fedarate storage configuration<br/> federated_storage = optional(object({<br/> ## Indicates if we should create the federated bucket<br/> create_bucket = optional(bool, false)<br/> ## KMS key ARN to use for the federated bucket<br/> kms_key_arn = optional(string, null)<br/> ## The ARN of the federated bucket to use for the Kubecost platform<br/> federated_bucket_arn = optional(string, null)<br/> ## List of principals to allowed to write to the federated bucket<br/> allowed_principals = optional(list(string), [])<br/> }), {})<br/> ## Cloud Costs feature <br/> cloud_costs = optional(object({<br/> ## Indicates if we should enable cloud costs via Athena<br/> enable = optional(bool, false)<br/> ## The ARN of the S3 bucket for Cost and Usage Report (CUR) data<br/> cur_bucket_arn = optional(string, null)<br/> ## The ARN of the S3 bucket for Athena query results<br/> athena_bucket_arn = optional(string, null)<br/> ## The name of the Athena database for CUR data<br/> athena_database_name = optional(string, null)<br/> ## The ARN of the Athena table for CUR data<br/> athena_table_name = optional(string, null)<br/> }), {})<br/> })</pre> | `null` | no |
550+
| <a name="input_kubecosts_agent"></a> [kubecosts\_agent](#input\_kubecosts\_agent) | The Kubecost Agent configuration | <pre>object({<br/> ## Indicates if we should enable the Kubecost Agent platform<br/> enable = optional(bool, false)<br/> ## The namespace to deploy the Kubecost Agent platform to<br/> namespace = optional(string, "kubecost")<br/> ## The service account to deploy the Kubecost Agent platform to<br/> service_account = optional(string, "kubecost-agent")<br/> ## The ARN of the federated bucket to use for the Kubecost Agent platform<br/> federated_bucket_arn = string<br/> })</pre> | `null` | no |
477551
| <a name="input_kubernetes_version"></a> [kubernetes\_version](#input\_kubernetes\_version) | Kubernetes version for the EKS cluster | `string` | `"1.34"` | no |
478552
| <a name="input_node_pools"></a> [node\_pools](#input\_node\_pools) | Collection of nodepools to create via auto-mote karpenter | `list(string)` | <pre>[<br/> "system"<br/>]</pre> | no |
479553
| <a name="input_node_security_group_additional_rules"></a> [node\_security\_group\_additional\_rules](#input\_node\_security\_group\_additional\_rules) | List of additional security group rules to add to the node security group created. Set `source_cluster_security_group = true` inside rules to set the `cluster_security_group` as source. | `any` | `{}` | no |

0 commit comments

Comments
 (0)