stack-lemp (Linux, NGINX, MySQL, PHP)

This stack will create a full LEMP infrastructure based on Auto Scaling Groups (ASG) and an RDS database on AWS

• Linux operating system
• NGINX (Pronounced as Engine-X) web server
• MySQL (RDS) database server
• PHP-FPM for dynamic data processing

Architecture

• ALB: Amazon Application Load Balancer
• ASG: Amazon Auto Scaling group for fronts
• front: EC2 instances from builded AMI
• RDS (optional): Amazon RDS database (mysql)
• ElastiCache (optional): Amazon ElastiCache (Redis)
• S3 bucket (optional): public medias bucket

Requirements

In order to run this task, couple elements are required within the infrastructure:

• Having a VPC with private & public subnets Here
• Having a bastion server to run Ansible like described Here
• Having an S3 bucket to store Terraform remote states Here
• Having an S3 bucket for website code WITH versioning enable Here

Details

Pipeline

Note The pipeline contains a manual approval between terraform plan and terraform apply. That means if you trigger a terraform plan, to apply it, you have to go on terraform apply job and click on the + button to trigger it.

Jobs description

• build-ami-front : Build Front Amazon image AMI using Packer and Ansible.
• build-application: Runs the appropriate php/composer commands and build a release file of the code to put it on S3.
• terraform-plan: Terraform job that will simply make a plan of the stack.
• terraform-apply: Terraform job similar to the plan one, but will actually create/update everything that needs to. Please see the plan diff for a better understanding.
• unittests: Dummy job meant to eventually be replaced by proper tests or removed.
• application-deployment: Simply trigger a deployment using Ansible of the last version of the code on existing instances.
• functional-tests: Dummy job meant to eventually be replaced by proper functional or removed.
• terraform-destroy: ⚠️ Terraform job meant to destroy the whole stack - NO CONFIRMATION ASKED. If triggered, the full project WILL be destroyed. Use with caution.

Params

Name	Description	Type	Default	Required
ansible_vault_password	Password used by ansible vault to decrypt your vaulted files.	-	((custom_ansible_vault_password))	False
ansible_version	Ansible version used in packer and cycloid-toolkit ansible runner	-	"2.9"	True
aws_access_key	Amazon AWS access key for Terraform. See value format here	-	((aws.access_key))	True
aws_default_region	Amazon AWS region to use for Terraform.	-	eu-west-1	True
aws_secret_key	Amazon AWS secret key for Terraform. See value format here	-	((aws.secret_key))	True
bastion_private_key_pair	bastion SSH private key used by ansible to connect on AWS EC2 instances and the bastion itself.	-	((ssh_bastion.ssh_key))	True
bastion_url	bastion URL used by ansible to connect on AWS EC2 instances.	-	user@bastion.server.com	True
config_ansible_path	Path of Ansible files in the config Git repository	-	($ .project $)/ansible	True
config_git_branch	Branch of the config Git repository.	-	master	True
config_git_private_key	SSH key pair to fetch the config Git repository.	-	((ssh_config.ssh_key))	True
config_git_repository	Git repository URL containing the config of the stack.	-	git@github.com:MyUser/config-lemp-app.git	True
config_pipeline_path	Path of pipeline task yml files in the config Git repository. Used to override pipeline yask like build-release.yml	-	($ .project $)/pipeline	True
config_terraform_path	Path of Terraform files in the config git repository	-	($ .project $)/terraform/($ .environment $)	True
customer	Name of the Cycloid Organization, used as customer variable name.	-	($ .organization_canonical $)	True
cycloid_toolkit_tag_prefix	Prefix used with ansible_version to match cycloid-toolkit docker image tag. (example with "a": cycloid/cycloid-toolkit:a2.9).	-	"a"	True
debug_public_key	SSH pubkey injected by packer during the ec2 ami build. Used only to debug failure.	-	""	False
deploy_bucket_name	AWS S3 bucket name in which store the builded code of the website.	-	($ .project $)-deploy	True
deploy_bucket_object_path	AWS S3 bucket path in which store the builded code of the website.	-	/catalog-lemp-app/($ .environment $)/lemp-app.tar.gz	True
env	Name of the project's environment.	-	($ .environment $)	True
lemp_git_branch	Branch of the LEMP source code Git repository.	-	master	True
lemp_git_private_key	SSH key pair to fetch LEMP source code Git repository.	-	((ssh_lemp_app.ssh_key))	True
lemp_git_repository	URL to the Git repository containing LEMP website source code.	-	git@github.com:MyUser/code-lemp.git	True
project	Name of the project.	-	($ .project $)	True
rds_password	Password used for your rds. Set "empty" if you dont use databases	-	((custom_rds_password))	True
stack_git_branch	Branch to use on the public stack Git repository	-	master	True
terraform_storage_bucket_name	AWS S3 bucket name to store terraform remote state file.	-	($ .organization_canonical $)-terraform-remote-state	True
terraform_version	terraform version used to execute your code.	-	'1.0.1'	True

Terraform

Inputs

Name	Description	Type	Default	Required
application_health_check_matcher	n/a	number	200	no
application_health_check_path	n/a	string	"/health-check"	no
application_path_health_interval	n/a	number	45	no
application_path_health_timeout	n/a	number	15	no
application_ssl_cert	n/a	string	""	no
application_ssl_policy	n/a	string	"ELBSecurityPolicy-TLS-1-2-2017-01"	no
cache_subnet_group	n/a	string	""	no
cloudfront_aliases	n/a	list(string)	[]	no
cloudfront_cached_methods	n/a	list	[ "GET", "HEAD" ]	no
cloudfront_compress	n/a	bool	true	no
cloudfront_default_ttl	n/a	number	300	no
cloudfront_max_ttl	n/a	number	1200	no
cloudfront_min_ttl	n/a	number	0	no
cloudfront_minimum_protocol_version	n/a	string	"TLSv1"	no
cloudfront_price_class	n/a	string	"PriceClass_200"	no
cloudfront_ssl_certificate	n/a	string	"arn:aws:acm:us-east-1:xxxxxxxx:certificate/xxxxxxx"	no
component	n/a	any	n/a	yes
create_cloudfront_medias	n/a	bool	false	no
create_elasticache	n/a	bool	false	no
create_rds	n/a	bool	false	no
create_s3_medias	n/a	bool	false	no
create_ses_access	n/a	bool	false	no
debian_ami_name	n/a	string	"debian-11-amd64-*"	no
default_short_name	n/a	string	""	no
deploy_bucket_name	n/a	string	"application-deployment"	no
elasticache_cluster_id	n/a	string	""	no
elasticache_engine	n/a	string	"redis"	no
elasticache_engine_version	n/a	string	"8.0"	no
elasticache_nodes	n/a	number	1	no
elasticache_parameter_group_name	n/a	string	"default.redis8.0"	no
elasticache_port	n/a	string	"6379"	no
elasticache_type	n/a	string	"cache.t2.micro"	no
env	n/a	any	n/a	yes
extra_tags	n/a	map	{}	no
front_ami_id	n/a	string	""	no
front_asg_max_size	n/a	number	5	no
front_asg_min_size	n/a	number	1	no
front_asg_scale_down_cooldown	n/a	number	500	no
front_asg_scale_down_scaling_adjustment	n/a	number	-1	no
front_asg_scale_down_threshold	n/a	number	30	no
front_asg_scale_up_cooldown	n/a	number	300	no
front_asg_scale_up_scaling_adjustment	n/a	number	2	no
front_asg_scale_up_threshold	n/a	number	85	no
front_associate_public_ip_address	n/a	bool	false	no
front_count	n/a	number	1	no
front_disk_size	n/a	number	30	no
front_disk_type	n/a	string	"gp2"	no
front_ebs_optimized	n/a	bool	false	no
front_type	n/a	string	"t3.small"	no
front_update_min_in_service	n/a	number	1	no
keypair_name	n/a	string	"cycloid"	no
metrics_sg_allow	n/a	string	""	no
nameregex	Used to only keep few char for component like ALB name	string	"/[^0-9A-Za-z-]/"	no
organization	n/a	any	n/a	yes
private_subnets_ids	n/a	list(string)	[]	no
project	n/a	any	n/a	yes
public_subnets_ids	n/a	list(string)	[]	no
rds_backup_retention	n/a	number	7	no
rds_database	n/a	string	"application"	no
rds_disk_size	n/a	number	10	no
rds_engine	n/a	string	"mysql"	no
rds_engine_version	n/a	string	"8.0"	no
rds_extra_sg_allow	n/a	string	""	no
rds_multiaz	n/a	bool	false	no
rds_parameters	n/a	string	"default.mysql8.0"	no
rds_password	n/a	string	"ChangeMePls"	no
rds_skip_final_snapshot	n/a	bool	true	no
rds_storage_type	n/a	string	"gp2"	no
rds_subnet_group	n/a	string	""	no
rds_type	n/a	string	"db.t3.small"	no
rds_username	n/a	string	"application"	no
s3_medias_acl	n/a	string	"private"	no
s3_medias_policy_json	n/a	string	""	no
ses_resource_arn	n/a	string	"*"	no
vpc_id	n/a	string	""	no
zones	To use specific AWS Availability Zones.	list	[]	no

Outputs

Name	Description
alb_front_dns_name	n/a
alb_front_zone_id	n/a
cloudfront_medias_domain_name	n/a
elasticache_address	n/a
elasticache_cluster_id	n/a
iam_ses_smtp_user_key	n/a
iam_ses_smtp_user_secret	n/a
iam_ses_user_key	n/a
iam_ses_user_secret	n/a
rds_address	n/a
rds_database	n/a
rds_port	n/a
rds_username	n/a
s3_medias	n/a

Ansible

• Playbook and packer config to build a debian image with telegraf, fluentd, nginx and php-fpm installed

Name	Description	Type	Default	Required
cycloid_files_watched	Provide log files you want to export to Cycloid logs.	-	<Default log files watched>	False
nginx_sites	Contain Nginx vhosts to create on front servers. A default application and metrics vhosts are already provided.	dict	<metric and application vhost>	False
nginx_vhost_extra_directive	If you need extra directive to add in the default application vhost. Example basic auth, https redirect ...	dict	``	False
php_version_to_install	PHP fpm version to install.	-	7.2	False
telegraf_install	Install telegraf	bool	true	False

Molecule tests

Requires a bucket which contains a build of magento sources and AWS access key

virtualenv if needed

virtualenv    .env  --clear
source .env/bin/activate

pip install ansible==2.7 molecule==3.0a4 docker-py

Run the test

cd ansible

export AWS_ACCESS_KEY_ID=AKI...
export AWS_SECRET_ACCESS_KEY=....

export DEPLOY_BUCKET_NAME=cycloid-deploy
export DEPLOY_BUCKET_OBJECT_PATH=catalog-lemp-app/ci/lemp-app.tar.gz
export DEPLOY_BUCKET_REGION=eu-west-1

# Run molecule
molecule destroy
molecule converge
molecule verify