tf-aws-module_primitive-efs_file_system

Overview

A Terraform primitive module for creating and managing AWS Elastic File System (EFS) file systems. This module provides a standardized way to create EFS file systems with configurable encryption, performance modes, throughput settings, and lifecycle policies for cost optimization.

Features

• Encryption at Rest: Supports both AWS managed and customer managed KMS keys
• Flexible Storage Options: Multi-AZ (high availability) or One Zone (cost optimized) storage classes
• Performance Modes: Choose between General Purpose (lower latency) or Max I/O (higher throughput)
• Throughput Modes: Bursting, Elastic (auto-scaling), or Provisioned throughput
• Lifecycle Management: Automated transitions between storage classes (Standard, IA, Archive) for cost optimization
• Protection Controls: Configurable replication overwrite protection
• Comprehensive Tagging: Support for custom tags with automatic ManagedBy tag and optional Name tag
• Input Validation: Built-in validation for performance mode and throughput mode values

Usage

Basic Example

module "efs" {
  source = "github.com/launchbynttdata/tf-aws-module_primitive-efs_file_system?ref=1.0.0"

  creation_token = "my-app-efs"
  name           = "My Application EFS"
  encrypted      = true

  tags = {
    Environment = "production"
    Application = "my-app"
  }
}

Complete Example with All Features

module "efs" {
  source = "github.com/launchbynttdata/tf-aws-module_primitive-efs_file_system?ref=1.0.0"

  creation_token   = "my-app-efs"
  name             = "Production EFS"
  encrypted        = true
  kms_key_id       = "arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012"
  performance_mode = "generalPurpose"
  throughput_mode  = "elastic"

  # Cost optimization with lifecycle policies
  lifecycle_policy = {
    transition_to_ia                    = "AFTER_30_DAYS"
    transition_to_primary_storage_class = "AFTER_1_ACCESS"
    transition_to_archive               = "AFTER_90_DAYS"
  }

  protection = {
    replication_overwrite = "DISABLED"
  }

  tags = {
    Environment = "production"
    Application = "my-app"
    CostCenter  = "engineering"
  }
}

One Zone Storage Example

module "efs_one_zone" {
  source = "github.com/launchbynttdata/tf-aws-module_primitive-efs_file_system?ref=1.0.0"

  creation_token         = "my-dev-efs"
  name                   = "Development EFS"
  availability_zone_name = "us-east-1a"  # Creates One Zone storage
  encrypted              = true
  throughput_mode        = "bursting"

  tags = {
    Environment = "development"
  }
}

Important Notes

Lifecycle Policy Implementation

This module uses separate lifecycle_policy blocks for each transition type as required by AWS Provider version 5.100.0. Each transition rule (to IA, to Archive, to Primary Storage) must be defined in its own block. See the main.tf file for detailed implementation comments.

Mount Targets

This module creates only the EFS file system resource itself. To mount the file system to EC2 instances or other compute resources, you'll need to create EFS mount targets separately using a mount target module or resource. Mount targets require:

• VPC subnet IDs
• Security groups for access control
• Network connectivity to your compute resources

Storage Classes and Cost Optimization

• Standard: Default storage class for frequently accessed files
• Infrequent Access (IA): Lower cost for files accessed less frequently
• Archive: Lowest cost for long-term storage, rarely accessed files
• Use lifecycle policies to automatically transition files between storage classes based on access patterns

Throughput Modes

• Bursting: Throughput scales with file system size (good for baseline workloads)
• Elastic: Automatically scales throughput up/down based on workload (recommended for variable workloads)
• Provisioned: Fixed throughput independent of storage size (use when you need guaranteed throughput)

Install required development dependencies:

make configure-dependencies
make configure-git-hooks

This installs:

• Terraform
• Go
• Pre-commit hooks
• Other development tools specified in .tool-versions

---

HOWTO: Developing a Primitive Module

Step 1: Define Your Resource

1. Identify the AWS resource you're wrapping (e.g., aws_eks_cluster)
2. Review AWS documentation for the resource to understand all available parameters
3. Study similar primitive modules for patterns and best practices

Step 2: Create the Module Structure

Your primitive module should include these core files:

main.tf

• Contains the primary resource declaration
• Should be clean and focused on the single resource
• Example:

resource "aws_eks_cluster" "this" {
  name     = var.name
  role_arn = var.role_arn
  version  = var.kubernetes_version

  vpc_config {
    subnet_ids              = var.vpc_config.subnet_ids
    security_group_ids      = var.vpc_config.security_group_ids
    endpoint_private_access = var.vpc_config.endpoint_private_access
    endpoint_public_access  = var.vpc_config.endpoint_public_access
    public_access_cidrs     = var.vpc_config.public_access_cidrs
  }

  tags = merge(
    var.tags,
    local.default_tags
  )
}

variables.tf

• Define all configurable parameters
• Include clear descriptions for each variable
• Set sensible defaults where appropriate
• Use validation rules to enforce constraints, but only when the validations can be made precise.
• Alternatively, use check blocks to create more complicated validations. (Requires terraform ~> 1.12)
• Example:

variable "name" {
  description = "Name of the EKS cluster"
  type        = string

  validation {
    condition     = length(var.name) <= 100
    error_message = "Cluster name must be 100 characters or less"
  }
}

variable "kubernetes_version" {
  description = "Kubernetes version to use for the EKS cluster"
  type        = string
  default     = null

  validation {
    condition     = var.kubernetes_version == null || can(regex("^1\\.(2[89]|[3-9][0-9])$", var.kubernetes_version))
    error_message = "Kubernetes version must be 1.28 or higher"
  }
}

outputs.tf

• Export all useful attributes of the resource
• Include comprehensive outputs for downstream consumption
• Document what each output provides
• Example:

output "id" {
  description = "The ID of the EKS cluster"
  value       = aws_eks_cluster.this.id
}

output "arn" {
  description = "The ARN of the EKS cluster"
  value       = aws_eks_cluster.this.arn
}

output "endpoint" {
  description = "The endpoint for the EKS cluster API server"
  value       = aws_eks_cluster.this.endpoint
}

locals.tf

• Define local values and transformations
• Include standard tags (e.g., provisioner = "Terraform")
• Example:

locals {
  default_tags = {
    provisioner = "Terraform"
  }
}

versions.tf

• Specify required Terraform and provider versions
• Example:

terraform {
  required_version = "~> 1.5"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.100"
    }
  }
}

Step 3: Create Examples

Create example configurations in the examples/ directory:

examples/simple/

• Minimal, working configuration
• Uses only required variables
• Good for quick starts and basic testing

examples/complete/

• Comprehensive configuration showing all features
• Demonstrates advanced options
• Includes comments explaining choices

Each example should include:

• main.tf - The module invocation
• variables.tf - Example variables
• outputs.tf - Pass-through outputs
• test.tfvars - Test values for automated testing
• README.md - Documentation for the example

Step 4: Write Tests

Update the test files in tests/:

tests/testimpl/test_impl.go

Write functional tests that verify:

• The resource is created successfully
• Resource properties match expectations
• Outputs are correct
• Integration with AWS SDK to verify actual state

tests/testimpl/types.go

Define the configuration structure for your tests:

type ThisTFModuleConfig struct {
    Name              string `json:"name"`
    KubernetesVersion string `json:"kubernetes_version"`
    // ... other fields
}

tests/post_deploy_functional/main_test.go

• Update test names to match your module
• Configure test flags (e.g., idempotency settings)
• Adjust test context as needed

Step 5: Update Documentation

1. Update README.md with:

   • Overview of the module
   • Feature list
   • Usage examples
   • Input/output documentation
   • Validation rules

2. Document validation rules clearly so users understand constraints.

Step 6: Test Your Module

1. Run local validation:

make check

This runs:

• Terraform fmt, validate, and plan
• Go tests with Terratest
• Pre-commit hooks
• Security scans

1. Test with real infrastructure:

cd examples/simple
terraform init
terraform plan -var-file=test.tfvars -out=the.tfplan
terraform apply the.tfplan

1. Verify outputs:

terraform output

1. Clean up:

terraform destroy -var-file=test.tfvars

Step 7: Document and Release

1. Write a comprehensive README following the pattern in the example modules
2. Add files to commit git add .
3. Run pre-commit hooks manually pre-commit run
4. Resolve any pre-commit issues
5. Push branch to github

---

Module Best Practices

Naming Conventions

• Repository: tf-aws-module_primitive-<resource_name>
• Resource identifier: Use this for the primary resource.
• Variables: Use snake_case.
• Match AWS resource parameter names where possible.

Input Variables

• Provide sensible defaults when safe to do so.
• Use null as default for optional complex objects.
• Include validation rules with clear error messages.
• Group related parameters using object types.
• Document expected formats and constraints.

Outputs

• Export all significant resource attributes.
• Use clear, descriptive output names.
• Include descriptions for all outputs.
• Consider downstream module needs.

Tags

• Always include a tags variable, unless the resource does not support tags.
• Merge with local.default_tags including provisioner = "Terraform".
• Use provider default tags when appropriate.

Validation

• Validate input constraints at the variable level.
• Provide helpful error messages.
• Check for common misconfigurations.
• Validate relationships between variables.

Testing

• Test the minimal example (required parameters only).
• Test the complete example (all features).
• Verify resource creation and properties.
• Test idempotency where applicable.
• Test validation rules by expecting failures.

Documentation

• Clear overview of the module's purpose.
• Feature list highlighting key capabilities.
• Multiple usage examples (minimal and complete).
• Comprehensive input/output tables.
• Document validation rules and constraints.
• Include links to relevant AWS documentation.

---

File Structure

After initialization, your module should have this structure:

tf-aws-module_primitive-<resource_name>/
├── .github/
│   └── workflows/          # CI/CD workflows
├── examples/
│   ├── simple/            # Minimal example
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   ├── outputs.tf
│   │   ├── test.tfvars
│   │   └── README.md
│   └── complete/          # Comprehensive example
│       ├── main.tf
│       ├── variables.tf
│       ├── outputs.tf
│       ├── test.tfvars
│       └── README.md
├── tests/
│   ├── post_deploy_functional/
│   │   └── main_test.go
│   ├── testimpl/
│   │   ├── test_impl.go
│   │   └── types.go
├── .gitignore
├── .pre-commit-config.yaml
├── .tool-versions
├── go.mod
├── go.sum
├── LICENSE
├── locals.tf
├── main.tf
├── Makefile
├── outputs.tf
├── README.md
├── variables.tf
└── versions.tf

---

Common Makefile Targets

Target	Description
make init-module	Initialize new module from template (run once after creating from template)
make configure-dependencies	Install required development tools
make configure-git-hooks	Set up pre-commit hooks
make check	Run all validation and tests
make configure	Full setup (dependencies + hooks + repo sync)
make clean	Remove downloaded components

---

Getting Help

• Review example modules: EKS Cluster, KMS Key
• Check the Launch Common Automation Framework documentation.
• Reach out to the platform team for guidance.

---

Contributing

Follow the established patterns in existing primitive modules. All modules should:

• Pass make check validation.
• Include comprehensive tests.
• Follow naming conventions.
• Include clear documentation.
• Use semantic versioning.

Requirements

Name	Version
terraform	~> 1.5
aws	~> 5.100

Modules

No modules.

Resources

Name	Type
aws_efs_file_system.this	resource

Inputs

Name	Description	Type	Default	Required
creation_token	Required unique identifier for the EFS file system. Must be unique within your AWS account and region and cannot be empty	string	n/a	yes
name	Required friendly name for the EFS file system. Will be added as a 'Name' tag and cannot be empty	string	n/a	yes
availability_zone_name	The AWS Availability Zone in which to create the file system. Used to create a file system that uses One Zone storage classes. If omitted, Multi-AZ storage will be used	string	null	no
encrypted	If true, the disk will be encrypted	bool	true	no
kms_key_id	ARN for the KMS encryption key. If set, the EFS file system will be encrypted at rest using this key	string	null	no
performance_mode	The file system performance mode. Valid values: 'generalPurpose' (default, lower latency for most workloads) or 'maxIO' (higher aggregate throughput for highly parallelized workloads)	string	"generalPurpose"	no
throughput_mode	Throughput mode for the file system. Valid values: 'bursting' (scales with file system size), 'elastic' (automatically scales based on workload), or 'provisioned' (fixed throughput - requires provisioned_throughput_in_mibps to be set)	string	"bursting"	no
provisioned_throughput_in_mibps	The throughput, measured in MiB/s, that you want to provision for the file system. Only applicable when throughput_mode is set to 'provisioned'	number	null	no
lifecycle_policy	Lifecycle policy for the file system. Supports transition_to_ia (AFTER_7_DAYS, AFTER_14_DAYS, AFTER_30_DAYS, AFTER_60_DAYS, AFTER_90_DAYS, AFTER_1_DAY, AFTER_180_DAYS, AFTER_270_DAYS, AFTER_365_DAYS), transition_to_primary_storage_class (AFTER_1_ACCESS), and transition_to_archive (AFTER_1_DAY, AFTER_7_DAYS, AFTER_14_DAYS, AFTER_30_DAYS, AFTER_60_DAYS, AFTER_90_DAYS, AFTER_180_DAYS, AFTER_270_DAYS, AFTER_365_DAYS)	object({ transition_to_ia = optional(string) transition_to_primary_storage_class = optional(string) transition_to_archive = optional(string) })	null	no
protection	Protection configuration for the file system. Supports replication_overwrite (ENABLED, DISABLED, REPLICATING)	object({ replication_overwrite = optional(string) })	null	no
tags	A map of tags to assign to the EFS file system	map(string)	{}	no

Outputs

Name	Description
file_system_id	The ID of the EFS file system
file_system_arn	Amazon Resource Name of the file system
file_system_dns_name	The DNS name for the filesystem
file_system_creation_token	The creation token of the EFS file system
file_system_availability_zone_id	The identifier of the Availability Zone in which the file system's One Zone storage classes exist
file_system_availability_zone_name	The Availability Zone name in which the file system's One Zone storage classes exist
file_system_number_of_mount_targets	The current number of mount targets that the file system has
file_system_owner_id	The AWS account that created the file system
file_system_size_in_bytes	The latest known metered size (in bytes) of data stored in the file system
file_system_name	The value of the file system's Name tag