Skip to content

Commit 2ceb32f

Browse files
luigidifraiaworkantonbabenko
luigidifraiawork
and
antonbabenko
authored
feat: Add example for S3 bucket access through VPC Endpoint (#349)
Co-authored-by: Anton Babenko <[email protected]>
1 parent 8f99aaa commit 2ceb32f

File tree

7 files changed

+418
-0
lines changed

7 files changed

+418
-0
lines changed

‎README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -586,6 +586,7 @@ Q4: What does this error mean - `"We currently do not support adding policies fo
586586
- [Deploy](https://github.com/terraform-aws-modules/terraform-aws-lambda/tree/master/examples/deploy) - Complete end-to-end build/update/deploy process using AWS CodeDeploy.
587587
- [Async Invocations](https://github.com/terraform-aws-modules/terraform-aws-lambda/tree/master/examples/async) - Create Lambda Function with async event configuration (with SQS, SNS, and EventBridge integration).
588588
- [With VPC](https://github.com/terraform-aws-modules/terraform-aws-lambda/tree/master/examples/with-vpc) - Create Lambda Function with VPC.
589+
- [With VPC and VPC Endpoint for S3](https://github.com/terraform-aws-modules/terraform-aws-lambda/tree/master/examples/with-vpc-s3-endpoint) - Create Lambda Function with VPC and VPC Endpoint for S3.
589590
- [With EFS](https://github.com/terraform-aws-modules/terraform-aws-lambda/tree/master/examples/with-efs) - Create Lambda Function with Elastic File System attached (Terraform 0.13+ is recommended).
590591
- [Multiple regions](https://github.com/terraform-aws-modules/terraform-aws-lambda/tree/master/examples/multiple-regions) - Create the same Lambda Function in multiple regions with non-conflicting IAM roles and policies.
591592
- [Event Source Mapping](https://github.com/terraform-aws-modules/terraform-aws-lambda/tree/master/examples/event-source-mapping) - Create Lambda Function with event source mapping configuration (SQS, DynamoDB, Amazon MQ, and Kinesis).

‎examples/fixtures/python3.8-app2/index.py

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
import logging
2+
import boto3
3+
import os
4+
from uuid import uuid4
5+
6+
# See https://docs.aws.amazon.com/lambda/latest/dg/python-logging.html
7+
logger = logging.getLogger()
8+
logger.setLevel(logging.INFO)
9+
10+
logging.getLogger('boto3').setLevel(logging.DEBUG)
11+
logging.getLogger('botocore').setLevel(logging.DEBUG)
12+
13+
bucketName = os.environ['BUCKET_NAME']
14+
regionName = os.environ['REGION_NAME']
15+
16+
def lambda_handler(event, context):
17+
client = boto3.client('s3', regionName)
18+
response = client.put_object(
19+
Bucket=bucketName,
20+
Key=str(uuid4()),
21+
Body=bytearray("Hello, World!", 'utf-8')
22+
)
23+
24+
logger.info(response)
25+
26+
return response

‎examples/with-vpc-s3-endpoint/README.md

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
# AWS Lambda with VPC and VPC Endpoint for S3 example
2+
3+
The configuration in this directory creates an AWS Lambda Function deployed within a VPC with a VPC Endpoint for S3 and no Internet access. The Function writes a single object to an S3 bucket that is created as part of the supporting resources.
4+
5+
Be aware, that deletion of AWS Lambda with VPC can take a long time (e.g., 10 minutes).
6+
7+
## Usage
8+
9+
To run this example you need to execute:
10+
11+
```bash
12+
$ terraform init
13+
$ terraform plan
14+
$ terraform apply
15+
```
16+
17+
Note that this example may create resources which cost money. Run `terraform destroy` when you don't need these resources.
18+
19+
<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
20+
## Requirements
21+
22+
| Name | Version |
23+
|------|---------|
24+
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.14 |
25+
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.33 |
26+
| <a name="requirement_random"></a> [random](#requirement\_random) | >= 3.4 |
27+
28+
## Providers
29+
30+
| Name | Version |
31+
|------|---------|
32+
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 4.33 |
33+
| <a name="provider_random"></a> [random](#provider\_random) | >= 3.4 |
34+
35+
## Modules
36+
37+
| Name | Source | Version |
38+
|------|--------|---------|
39+
| <a name="module_kms"></a> [kms](#module\_kms) | terraform-aws-modules/kms/aws | ~> 1.0 |
40+
| <a name="module_lambda_s3_write"></a> [lambda\_s3\_write](#module\_lambda\_s3\_write) | ../../ | n/a |
41+
| <a name="module_s3_bucket"></a> [s3\_bucket](#module\_s3\_bucket) | terraform-aws-modules/s3-bucket/aws | ~> 3.0 |
42+
| <a name="module_security_group_lambda"></a> [security\_group\_lambda](#module\_security\_group\_lambda) | terraform-aws-modules/security-group/aws | ~> 4.0 |
43+
| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 3.0 |
44+
| <a name="module_vpc_endpoints"></a> [vpc\_endpoints](#module\_vpc\_endpoints) | terraform-aws-modules/vpc/aws//modules/vpc-endpoints | ~> 3.0 |
45+
46+
## Resources
47+
48+
| Name | Type |
49+
|------|------|
50+
| [random_pet.this](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) | resource |
51+
| [aws_iam_policy_document.bucket](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
52+
| [aws_iam_policy_document.endpoint](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
53+
| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |
54+
55+
## Inputs
56+
57+
No inputs.
58+
59+
## Outputs
60+
61+
| Name | Description |
62+
|------|-------------|
63+
| <a name="output_lambda_cloudwatch_log_group_arn"></a> [lambda\_cloudwatch\_log\_group\_arn](#output\_lambda\_cloudwatch\_log\_group\_arn) | The ARN of the Cloudwatch Log Group |
64+
| <a name="output_lambda_function_arn"></a> [lambda\_function\_arn](#output\_lambda\_function\_arn) | The ARN of the Lambda Function |
65+
| <a name="output_lambda_function_invoke_arn"></a> [lambda\_function\_invoke\_arn](#output\_lambda\_function\_invoke\_arn) | The Invoke ARN of the Lambda Function |
66+
| <a name="output_lambda_function_kms_key_arn"></a> [lambda\_function\_kms\_key\_arn](#output\_lambda\_function\_kms\_key\_arn) | The ARN for the KMS encryption key of Lambda Function |
67+
| <a name="output_lambda_function_last_modified"></a> [lambda\_function\_last\_modified](#output\_lambda\_function\_last\_modified) | The date Lambda Function resource was last modified |
68+
| <a name="output_lambda_function_name"></a> [lambda\_function\_name](#output\_lambda\_function\_name) | The name of the Lambda Function |
69+
| <a name="output_lambda_function_qualified_arn"></a> [lambda\_function\_qualified\_arn](#output\_lambda\_function\_qualified\_arn) | The ARN identifying your Lambda Function Version |
70+
| <a name="output_lambda_function_source_code_hash"></a> [lambda\_function\_source\_code\_hash](#output\_lambda\_function\_source\_code\_hash) | Base64-encoded representation of raw SHA-256 sum of the zip file |
71+
| <a name="output_lambda_function_source_code_size"></a> [lambda\_function\_source\_code\_size](#output\_lambda\_function\_source\_code\_size) | The size in bytes of the function .zip file |
72+
| <a name="output_lambda_function_version"></a> [lambda\_function\_version](#output\_lambda\_function\_version) | Latest published version of Lambda Function |
73+
| <a name="output_lambda_layer_arn"></a> [lambda\_layer\_arn](#output\_lambda\_layer\_arn) | The ARN of the Lambda Layer with version |
74+
| <a name="output_lambda_layer_created_date"></a> [lambda\_layer\_created\_date](#output\_lambda\_layer\_created\_date) | The date Lambda Layer resource was created |
75+
| <a name="output_lambda_layer_layer_arn"></a> [lambda\_layer\_layer\_arn](#output\_lambda\_layer\_layer\_arn) | The ARN of the Lambda Layer without version |
76+
| <a name="output_lambda_layer_source_code_size"></a> [lambda\_layer\_source\_code\_size](#output\_lambda\_layer\_source\_code\_size) | The size in bytes of the Lambda Layer .zip file |
77+
| <a name="output_lambda_layer_version"></a> [lambda\_layer\_version](#output\_lambda\_layer\_version) | The Lambda Layer version |
78+
| <a name="output_lambda_role_arn"></a> [lambda\_role\_arn](#output\_lambda\_role\_arn) | The ARN of the IAM role created for the Lambda Function |
79+
| <a name="output_lambda_role_name"></a> [lambda\_role\_name](#output\_lambda\_role\_name) | The name of the IAM role created for the Lambda Function |
80+
| <a name="output_local_filename"></a> [local\_filename](#output\_local\_filename) | The filename of zip archive deployed (if deployment was from local) |
81+
| <a name="output_s3_object"></a> [s3\_object](#output\_s3\_object) | The map with S3 object data of zip archive deployed (if deployment was from S3) |
82+
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

‎examples/with-vpc-s3-endpoint/main.tf

Lines changed: 196 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,196 @@
1+
provider "aws" {
2+
region = "eu-west-1"
3+
4+
# Make it faster by skipping something
5+
skip_get_ec2_platforms = true
6+
skip_metadata_api_check = true
7+
skip_region_validation = true
8+
skip_credentials_validation = true
9+
skip_requesting_account_id = true
10+
}
11+
12+
data "aws_region" "current" {}
13+
14+
################################################################################
15+
# Lambda Module
16+
################################################################################
17+
18+
module "lambda_s3_write" {
19+
source = "../../"
20+
21+
description = "Lambda demonstrating writes to an S3 bucket from within a VPC without Internet access"
22+
23+
function_name = random_pet.this.id
24+
handler = "index.lambda_handler"
25+
runtime = "python3.8"
26+
27+
source_path = "${path.module}/../fixtures/python3.8-app2"
28+
29+
environment_variables = {
30+
BUCKET_NAME = module.s3_bucket.s3_bucket_id
31+
REGION_NAME = data.aws_region.current.name
32+
}
33+
34+
# Let the module create a role for us
35+
create_role = true
36+
attach_cloudwatch_logs_policy = true
37+
attach_network_policy = true
38+
39+
# There's no need to attach any extra permission for S3 writes as that's added by the bucket policy when a session is created
40+
# See https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html
41+
42+
vpc_security_group_ids = [module.security_group_lambda.security_group_id]
43+
vpc_subnet_ids = module.vpc.intra_subnets
44+
45+
tags = {
46+
Module = "lambda_s3_write"
47+
}
48+
}
49+
50+
################################################################################
51+
# Extra Resources
52+
################################################################################
53+
54+
resource "random_pet" "this" {
55+
length = 2
56+
}
57+
58+
module "vpc" {
59+
source = "terraform-aws-modules/vpc/aws"
60+
version = "~> 3.0"
61+
62+
name = random_pet.this.id
63+
cidr = "10.0.0.0/16"
64+
65+
azs = ["${data.aws_region.current.name}a", "${data.aws_region.current.name}b", "${data.aws_region.current.name}c"]
66+
67+
# Intra subnets are designed to have no Internet access via NAT Gateway.
68+
intra_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
69+
}
70+
71+
module "vpc_endpoints" {
72+
source = "terraform-aws-modules/vpc/aws//modules/vpc-endpoints"
73+
version = "~> 3.0"
74+
75+
vpc_id = module.vpc.vpc_id
76+
77+
endpoints = {
78+
s3 = {
79+
service = "s3"
80+
service_type = "Gateway"
81+
route_table_ids = module.vpc.intra_route_table_ids
82+
policy = data.aws_iam_policy_document.endpoint.json
83+
}
84+
}
85+
}
86+
87+
data "aws_iam_policy_document" "endpoint" {
88+
statement {
89+
sid = "RestrictBucketAccessToIAMRole"
90+
91+
principals {
92+
type = "AWS"
93+
identifiers = ["*"]
94+
}
95+
96+
actions = [
97+
"s3:PutObject",
98+
]
99+
100+
resources = [
101+
"${module.s3_bucket.s3_bucket_arn}/*",
102+
]
103+
104+
# See https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-s3.html#edit-vpc-endpoint-policy-s3
105+
condition {
106+
test = "ArnEquals"
107+
variable = "aws:PrincipalArn"
108+
values = [module.lambda_s3_write.lambda_role_arn]
109+
}
110+
}
111+
}
112+
113+
module "kms" {
114+
source = "terraform-aws-modules/kms/aws"
115+
version = "~> 1.0"
116+
117+
description = "S3 encryption key"
118+
119+
# Grants
120+
grants = {
121+
lambda = {
122+
grantee_principal = module.lambda_s3_write.lambda_role_arn
123+
operations = [
124+
"GenerateDataKey",
125+
]
126+
}
127+
}
128+
}
129+
130+
module "s3_bucket" {
131+
source = "terraform-aws-modules/s3-bucket/aws"
132+
version = "~> 3.0"
133+
134+
bucket_prefix = "${random_pet.this.id}-"
135+
force_destroy = true
136+
137+
# S3 bucket-level Public Access Block configuration
138+
block_public_acls = true
139+
block_public_policy = true
140+
ignore_public_acls = true
141+
restrict_public_buckets = true
142+
143+
versioning = {
144+
enabled = true
145+
}
146+
147+
# Bucket policy
148+
attach_policy = true
149+
policy = data.aws_iam_policy_document.bucket.json
150+
151+
server_side_encryption_configuration = {
152+
rule = {
153+
apply_server_side_encryption_by_default = {
154+
kms_master_key_id = module.kms.key_id
155+
sse_algorithm = "aws:kms"
156+
}
157+
}
158+
}
159+
}
160+
161+
data "aws_iam_policy_document" "bucket" {
162+
statement {
163+
sid = "RestrictBucketAccessToIAMRole"
164+
165+
principals {
166+
type = "AWS"
167+
identifiers = [module.lambda_s3_write.lambda_role_arn]
168+
}
169+
170+
actions = [
171+
"s3:PutObject",
172+
]
173+
174+
resources = [
175+
"${module.s3_bucket.s3_bucket_arn}/*",
176+
]
177+
}
178+
}
179+
180+
module "security_group_lambda" {
181+
source = "terraform-aws-modules/security-group/aws"
182+
version = "~> 4.0"
183+
184+
name = random_pet.this.id
185+
description = "Security Group for Lambda Egress"
186+
187+
vpc_id = module.vpc.vpc_id
188+
189+
egress_cidr_blocks = []
190+
egress_ipv6_cidr_blocks = []
191+
192+
# Prefix list ids to use in all egress rules in this module
193+
egress_prefix_list_ids = [module.vpc_endpoints.endpoints["s3"]["prefix_list_id"]]
194+
195+
egress_rules = ["https-443-tcp"]
196+
}

0 commit comments

Comments
 (0)