added env variable docs, added MEM_LIMIT variable, support creation of 2 submits in aws-setup.sh

Author: dsperling

README.md

@@ -12,7 +12,7 @@ The `smithmicro/lucy` Docker image can be run as-is with a number of required en
 
 Prerequisites to use this image:
 * Create a VPC with at least one subnet as ECS requires the use of VPC **
-* Create a VPC security group that allows ports 22, 1099, 50000 and 51000 (tcp) to the VPC **
+* Create a VPC security group that allows ports 22, 1099, 50000 and 51000 (tcp) and 4445 (udp) to the VPC **
 * Create a security key pair and place in the `keys` subdirectory
 * Have your AWS CLI Access Key ID/Secret Access Key handy
 * Replace or edit the included `plans/demo.jmx` to run your specific tests
@@ -31,7 +31,6 @@ docker run -v <oath to jmx>:/plans -v <path to pem>:/keys -v <path to logs>:/log
     --env SUBNET_ID=<subnet ID within your VPC> \
     --env KEY_NAME=<key pair name without extension> \
     --env MINION_COUNT=<number of minions> \
-    --env INSTANCE_TYPE=<valid ECS instance type> \
     smithmicro/lucy /plans/demo.jmx
 ```
 For 5 test instances in N. Virginia, `docker run` would look like this, assuming your `jmeter-key.pem` file is located in the `keys` subdirectory:
@@ -44,7 +43,6 @@ docker run -v $PWD/plans:/plans -v $PWD/keys:/keys -v $PWD/logs:/logs \
     --env SUBNET_ID=subnet-12345678 \
     --env KEY_NAME=jmeter-key \
     --env MINION_COUNT=5 \
-    --env INSTANCE_TYPE=t2.small \
     smithmicro/lucy /plans/demo.jmx
 ```
 
@@ -123,6 +121,28 @@ docker-compose up
 ```
 Using the `docker-compose scale` command does not work as it creates hostnames like `minion_1`.  This causes an error in JMeter as it uses the hostname in URL form and sees the underscore as an illegal URL character.
 
+## Notes
+The following required and optional environment variables are supported:
+
+| Variable | Required | Default | Notes |
+|---|---|---|---|
+|AWS_DEFAULT_REGION|Yes|None|AWS Region (e.g. us-east-1)|
+|AWS_ACCESS_KEY_ID|Yes|None|AWS Access Key|
+|AWS_SECRET_ACCESS_KEY|Yes|None|AWS Secret Key|
+|INPUT_JMX|Yes|None|File path of JMeter Test file to run (.jmx).  You can optionally specify this as the first command line option of `docker run`|
+|KEY_NAME|Yes|None|AWS Security Key Pair .pem file (do not specify the .pem extension)|
+|SECURITY_GROUP|Yes|None|AWS Secuirty group that allows ports 22,1099,50000,51000/tcp and 4445/udp from all ports (e.g. sg-12345678)|
+|SUBNET_ID|Yes|None|One or more Subnets that are assigned to your VPC|
+|VPC_ID||VPC assigned to SUBNET_ID|We dautomatically erive this from your SUBNET_ID|
+|JMETER_VERSION||latest|smithmicro/lucy Image tag.  See Docker Hub for [available versions](https://hub.docker.com/r/smithmicro/jmeter/tags/).|
+|INSTANCE_TYPE||t2.micro|To double your memory, pass t2.small|
+|MEM_LIMIT||950m|If you are using t2.small, set MEM_LIMIT to 1995m|
+|MINION_COUNT||2||
+|PEM_PATH||/keys|This must match your Volume map.  See Volume section above.|
+|MINION_CLUSTER_NAME||JMeterMinion|Name that appears in your AWS Cluster UI|
+|GRU_CLUSTER_NAME||JMeterGru|Name that appears in your AWS Cluster UI|
+|GRU_PRIVATE_IP||(blank)|Set to true if you would like to run Lucy within AWS.  See GitHub [Issue 8](https://github.com/smithmicro/jmeter-ecs/issues/8) for details.|
+
 ## Notes
 This Docker image uses the Instance Metadata API documented here:
 * http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html
@@ -133,11 +153,6 @@ To get the instance public hostname within the `entrypoint.sh` script, we call:
 For more information on JMeter Distributed Testing, see:
 * http://jmeter.apache.org/usermanual/remote-test.html
 
-You can specify which JMeter version to run by adding the following to the Docker run line.  By default, the `latest` tag is used.
-* --env JMETER_VERSION=3.3
-
-If you would like to run Lucy in AWS, and it is running in the same VPC as Gru, you can configure Lucy to use the Private IP address of Gru:
-* --env GRU_PRIVATE_IP=true
 
 ## Inspired by...
 https://en.wikipedia.org/wiki/Despicable_Me_2

aws-setup.sh

@@ -9,9 +9,13 @@ if [ "$CIDR_BLOCK" == '' ]; then
   # create a CIDR block at 10.74, the 74 being ASCII 'J'
   CIDR_BLOCK=10.74.0.0/16
 fi
-if [ "$SUBNET_CIDR_BLOCK" == '' ]; then
+if [ "$SUBNET_CIDR_BLOCK1" == '' ]; then
   # this CIDR limits us to 251 JMeter Minions - protection from a typo trying to create 1000 instances
-  SUBNET_CIDR_BLOCK=10.74.1.0/24
+  SUBNET_CIDR_BLOCK1=10.74.1.0/24
+fi
+if [ "$SUBNET_CIDR_BLOCK2" == '' ]; then
+  # this CIDR limits us to 251 JMeter Minions - protection from a typo trying to create 1000 instances
+  SUBNET_CIDR_BLOCK2=10.74.2.0/24
 fi
 if [ "$OWNER" == '' ]; then
   OWNER=jmeter-ecs
@@ -39,10 +43,12 @@ echo "Created VPC $VPC_ID"
 # enable DNS hostnames
 aws ec2 modify-vpc-attribute --vpc-id $VPC_ID --enable-dns-hostnames --output text
 
-# create a single subnet
-SUBNET_ID=$(aws ec2 create-subnet --vpc-id $VPC_ID --cidr-block $SUBNET_CIDR_BLOCK \
+# create a 2 subnets
+SUBNET_ID1=$(aws ec2 create-subnet --vpc-id $VPC_ID --cidr-block $SUBNET_CIDR_BLOCK1 \
+    --query 'Subnet.[SubnetId]' --output text | tr -d '\n')
+SUBNET_ID2=$(aws ec2 create-subnet --vpc-id $VPC_ID --cidr-block $SUBNET_CIDR_BLOCK2 \
     --query 'Subnet.[SubnetId]' --output text | tr -d '\n')
-echo "Created Subnet $SUBNET_ID"
+echo "Created Subnets $SUBNET_ID1,$SUBNET_ID2"
 
 # Step 2: Make Your Subnet Public
 IGW_ID=$(aws ec2 create-internet-gateway --query 'InternetGateway.[InternetGatewayId]' --output text | tr -d '\n')
@@ -59,12 +65,14 @@ if [ "$CREATE_ROUTE_RESULT" == 'True' ]; then
   echo "Created route for all traffic to the Internet Gateway"
 fi
 
-# make this a public subnet
-RTBASSOC_ID=$(aws ec2 associate-route-table  --subnet-id $SUBNET_ID --route-table-id $RTB_ID --output text | tr -d '\n')
-echo "Created Route Table Association $RTBASSOC_ID"
+# make these public subnet
+RTBASSOC_ID1=$(aws ec2 associate-route-table --subnet-id $SUBNET_ID1 --route-table-id $RTB_ID --output text | tr -d '\n')
+RTBASSOC_ID2=$(aws ec2 associate-route-table --subnet-id $SUBNET_ID2 --route-table-id $RTB_ID --output text | tr -d '\n')
+echo "Created Route Table Associations $RTBASSOC_ID1,$RTBASSOC_ID2"
 
 # we need public IP addresses so instances can register with ECS clusters
-aws ec2 modify-subnet-attribute --subnet-id $SUBNET_ID --map-public-ip-on-launch
+aws ec2 modify-subnet-attribute --subnet-id $SUBNET_ID1 --map-public-ip-on-launch
+aws ec2 modify-subnet-attribute --subnet-id $SUBNET_ID2 --map-public-ip-on-launch
 
 # create a security group for JMeter
 SG_ID=$(aws ec2 create-security-group --group-name "JMeter" --description "JMeter Security Group" --vpc-id $VPC_ID --output text | tr -d '\n')
@@ -74,10 +82,10 @@ JMETER_IP_PERMISSIONS='[{"IpProtocol": "tcp", "FromPort": 22, "ToPort": 22, "IpR
 aws ec2 authorize-security-group-ingress --group-id $SG_ID --ip-permissions "$JMETER_IP_PERMISSIONS"
 
 # tag all created resources
-aws ec2 create-tags --resources $VPC_ID $SUBNET_ID $IGW_ID $RTB_ID $SG_ID --tags $VPC_TAGS --output text
+aws ec2 create-tags --resources $VPC_ID $SUBNET_ID1 $SUBNET_ID2 $IGW_ID $RTB_ID $SG_ID --tags $VPC_TAGS --output text
 
 echo "******** Use these two enviroment variables in 'docker run'"
-echo "  --env SUBNET_ID=$SUBNET_ID"
+echo "  --env SUBNET_ID=$SUBNET_ID1,$SUBNET_ID2"
 echo "  --env SECURITY_GROUP=$SG_ID"
 echo "********"
 

lucy/lucy.sh

@@ -26,7 +26,7 @@ if [ "$KEY_NAME" == '' ]; then
   exit 2
 fi
 if [ "$SECURITY_GROUP" == '' ]; then
-  echo "Please set a SECURITY_GROUP that allows ports 22, 1099, 50000, 51000 (tcp) from all ports (e.g. sg-12345678)"
+  echo "Please set a SECURITY_GROUP that allows ports 22,1099,50000,51000/tcp and 4445/udp from all ports (e.g. sg-12345678)"
   exit 3
 fi
 if [ "$SUBNET_ID" == '' ]; then
@@ -53,6 +53,9 @@ fi
 if [ "$INSTANCE_TYPE" == '' ]; then
   INSTANCE_TYPE=t2.micro
 fi
+if [ "$MEM_LIMIT" == '' ]; then
+  MEM_LIMIT=950m
+fi
 if [ "$MINION_COUNT" == '' ]; then
   MINION_COUNT=2
 fi
@@ -84,7 +87,7 @@ do
   MINION_CONTAINER_INSTANCE_IDS=$(aws ecs list-container-instances --cluster $MINION_CLUSTER_NAME --output text |
       awk '{print $2}' | tr '\n' ' ')
   if [ "$MINION_CONTAINER_INSTANCE_IDS" == '' ]; then
-    echo "Waiting for Minion container instances IDs.."
+    echo "Waiting for Minion container instance IDs.."
     sleep 5
   fi
 done
@@ -98,7 +101,7 @@ do
   GRU_CONTAINER_INSTANCE_ID=$(aws ecs list-container-instances --cluster $GRU_CLUSTER_NAME --output text |
       awk '{print $2}' | tr '\n' ' ')
   if [ "$GRU_CONTAINER_INSTANCE_ID" == '' ]; then
-    echo "Waiting for Gru container instances ID.."
+    echo "Waiting for Gru container instance ID..."
     sleep 5
   fi
 done
@@ -107,8 +110,9 @@ GRU_INSTANCE_ID=$(aws ecs describe-container-instances --cluster $GRU_CLUSTER_NA
     --container-instances $GRU_CONTAINER_INSTANCE_ID --query 'containerInstances[*].[ec2InstanceId]' --output text)
 echo "Gru instances ID: $GRU_INSTANCE_ID"
 
-# Step 3 - Run the Minion task with the requested JMeter version and instance count
+# Step 3 - Run the Minion task with the requested JMeter version, instance count and memory
 sed -i 's/jmeter:latest/jmeter:'"$JMETER_VERSION"'/' /opt/jmeter/lucy.yml
+sed -i 's/950m/'"$MEM_LIMIT"'/' /opt/jmeter/lucy.yml
 ecs-cli compose --file /opt/jmeter/lucy.yml up --cluster $MINION_CLUSTER_NAME
 ecs-cli compose --file /opt/jmeter/lucy.yml --cluster $MINION_CLUSTER_NAME scale $MINION_COUNT