Skip to content

Commit 13bfb4b

Browse files
author
Shakir
committed
added 2021-11-14-aws-ec2-launch-instances-the-hard-way-with-cli.md
1 parent 779dccf commit 13bfb4b

File tree

3 files changed

+709
-6
lines changed

3 files changed

+709
-6
lines changed

‎_posts/aws/2021-10-16-aws-eks-setup-cluster-and-deploy-owasp-juice-shop.md‎

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ We have successfully installed kubectl. We can now remove kubectl and the checks
8181
└──╼ $rm kubectl*
8282
```
8383

84-
# AWS CLI
84+
## AWS CLI
8585
We are going to launch a cluster in AWS EKS, we can do this via CLI. For that purpose, we need to install the
8686
[aws cli](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-linux.html#cliv2-linux-install) and eksctl.
8787

@@ -114,7 +114,7 @@ We would need an access key to issue AWS commands, for which let's add an IAM us
114114
[URL](https://console.aws.amazon.com/iam/home#/users$new?step=details) it would look like
115115
![Add IAM User](/assets/aws-eks-setup-cluster-and-deploy-owasp-juice-shop-1.png)
116116

117-
On the next page, you can create a group, if there isnt one. ![Add IAM Group](/assets/aws-eks-setup-cluster-and-deploy-owasp-juice-shop-2.png)
117+
On the next page, you can create a group, if there isn't one. ![Add IAM Group](/assets/aws-eks-setup-cluster-and-deploy-owasp-juice-shop-2.png)
118118

119119
I am going to create a group for the Administrators with the AdministratorAccess policy.
120120
![Administrator Access](/assets/aws-eks-setup-cluster-and-deploy-owasp-juice-shop-3.png)
@@ -154,9 +154,6 @@ So we have done the essential settings for the AWS CLI. Let's now continue with
154154
└──╼ $sudo mv /tmp/eksctl /usr/local/bin/eksctl
155155
[sudo] password for nc:
156156
157-
┌─[nc@parrot]─[~]
158-
└──╼ $
159-
160157
┌─[nc@parrot]─[~]
161158
└──╼ $eksctl version
162159
0.70.0
@@ -289,7 +286,7 @@ $ kubectl delete -f /tmp/juice-shop.yaml
289286
And finally when you are done, if you wish to delete the cluster, you may do so as follows.
290287
```
291288
┌─[nc@parrot]─[~]
292-
└──╼ $eksctl delete cluster --name hacktoberfest-2021 --region us-west-2
289+
└──╼ $eksctl delete cluster --name my-cluster --region us-west-2
293290
```
294291

295292
--end-of-post--

‎_posts/aws/2021-11-14-aws-ec2-launch-instances-the-hard-way-with-cli.md‎

Lines changed: 257 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,257 @@
1+
---
2+
categories: aws ec2
3+
title: aws ec2 > launch instances the hard way with cli
4+
---
5+
6+
Hey All :wave:, in this post we shall launch 3 AWS EC2 instances and test SSH connectivity to those...
7+
8+
We are not going to use the GUI / Web console :relaxed: for this purpose, we would be using the CLI :sweat_drops:, and also create individual components along the way, that are required for the instances to function properly, instead of relying on default ones, and thus touch bits of networking and security areas. Hope this approach gives someone a better understanding of the different components(like it gave me) that glue together underhood / behind the scenes, which we don't usually notice when we quickly setup instances with all the default options.
9+
10+
Hence, please ensure you have the following installed and configured: [aws cli](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) and [jq](https://stedolan.github.io/jq/download/). Note that jq is used to parse JSON content at different places in this post, though you can use the AWS CLI's built in filters.
11+
12+
You would see the naming conventions such as kubeadm or k8s, as these nodes are created for the purpose of creating a kubernetes cluster, and I thought I would keep the instance launching setup as a separate post.
13+
14+
Alright, let's get started...
15+
16+
## Configuration
17+
I have configured us-west-2 as the default region. You can change this as required.
18+
```
19+
$ cat ~/.aws/config
20+
[default]
21+
region = us-west-2
22+
```
23+
24+
And you should also see a credentials file like this.
25+
```
26+
$cat ~/.aws/credentials
27+
[default]
28+
aws_access_key_id=<aws_access_key_id>
29+
aws_secret_access_key=<aws_secret_access_key>
30+
```
31+
32+
## Key Pair
33+
Create an [ssh key pair](https://docs.aws.amazon.com/cli/latest/userguide/cli-services-ec2-keypairs.html), as we need it to login via SSH, to the instances we are about to launch.
34+
```
35+
$ aws ec2 create-key-pair --key-name kubeadmKeyPair --query 'KeyMaterial' --output text > kubeadmKeyPair.pem
36+
37+
$ ls
38+
kubeadmKeyPair.pem
39+
```
40+
41+
You can set permission 400 so that only you can read it, beneficial if its a shared system.
42+
```
43+
$ chmod 400 kubeadmKeyPair.pem
44+
```
45+
46+
And move it to the .ssh directory, where ssh keys are usually stored.
47+
```
48+
$ mv kubeadmKeyPair.pem ~/.ssh/.
49+
```
50+
51+
## VPC
52+
Let's create a [VPC](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ec2/create-vpc.html) from which we can reserve IPs for the EC2 instances.
53+
```
54+
$ aws ec2 create-vpc --cidr-block 10.0.0.0/28
55+
```
56+
57+
This is a /28 subnet which should give a total of 2 ^ (32-28) = 2 ^ 4 = 16 IPs, including the gateway and broadcast IP. We can not create VPCs with lesser IPs, 16 IPs are minimum, no matter we need it or not.
58+
59+
Let's retreive the VPC ID and save it in a variable.
60+
```
61+
$ export KUBEADM_VPC_ID=$(aws ec2 describe-vpcs | jq -r '.Vpcs[] | select(.CidrBlock == "10.0.0.0/28") | .VpcId')
62+
$ echo $KUBEADM_VPC_ID
63+
<vpc-id>
64+
```
65+
66+
## Internet Gateway
67+
We shall create an [Internet Gateway](https://docs.aws.amazon.com/cli/latest/reference/ec2/create-internet-gateway.html), as we need internet access to our instances, to access them via SSH from the local machine.
68+
```
69+
$ aws ec2 create-internet-gateway --tag-specifications 'ResourceType=internet-gateway,Tags=[{Key=Name,Value=kubeadm-k8s-igw}]'
70+
```
71+
72+
Retrieve the gateway id.
73+
```
74+
$ export KUBEADM_IGW_ID=$(aws ec2 describe-internet-gateways --filters Name=tag:Name,Values=kubeadm-k8s-igw --query "InternetGateways[*].InternetGatewayId" --output text)
75+
76+
$ echo $KUBEADM_IGW_ID
77+
<igw-id>
78+
```
79+
80+
Now, let's [attach](https://docs.aws.amazon.com/cli/latest/reference/ec2/attach-internet-gateway.html) this to the VPC.
81+
```
82+
$ aws ec2 attach-internet-gateway --internet-gateway-id $KUBEADM_IGW_ID --vpc-id $KUBEADM_VPC_ID
83+
```
84+
85+
## Route Table
86+
87+
A route table will be associated with the VPC we created. Let' find it.
88+
```
89+
$ KUBEADM_RTB_ID=$(aws ec2 describe-route-tables | jq -r '.RouteTables[] | select(.VpcId == env.KUBEADM_VPC_ID) | .RouteTableId')
90+
91+
$ echo $KUBEADM_RTB_ID
92+
<rtb-id>
93+
```
94+
95+
We can add a default route in this table, that points to our Internet Gateway.
96+
```
97+
$ aws ec2 create-route --route-table-id $KUBEADM_RTB_ID --destination-cidr-block 0.0.0.0/0 --gateway-id $KUBEADM_IGW_ID
98+
{
99+
"Return": true
100+
}
101+
```
102+
103+
## Subnet
104+
A VPC can have more than one subnet, however the minimum number of IPs in the [EC2 subnet](https://docs.aws.amazon.com/cli/latest/reference/ec2/create-subnet.html?highlight=subnet) is also 16, hence we can only have one subnet in this case, which suits our requirement though. Sixteen IPs in the subnet is more than enough, as we plan to launch only 3 instances in this subnet.
105+
```
106+
$ aws ec2 create-subnet --cidr-block 10.0.0.0/28 --vpc-id $KUBEADM_VPC_ID
107+
```
108+
109+
The subnet is created, let's save the subnet id too.
110+
```
111+
$ export KUBEADM_SUBNET_ID=$(aws ec2 describe-subnets | jq -r '.Subnets[] | select(.CidrBlock == "10.0.0.0/28") | .SubnetId')
112+
113+
$ echo $KUBEADM_SUBNET_ID
114+
<subnet-id>
115+
```
116+
117+
Note that if you have duplicate subnets, then you may also have to add a condition above to check if the subnet belongs to the correct VPC.
118+
119+
We can check the availabilty zone of the subnet.
120+
```
121+
$ export KUBEADM_AVAILABILITY_ZONE=$(aws ec2 describe-subnets | jq -r '.Subnets[] | select(.CidrBlock == "10.0.0.0/28") | .AvailabilityZone')
122+
123+
$ echo $KUBEADM_AVAILABILITY_ZONE
124+
us-west-2b
125+
```
126+
127+
## Security Group
128+
We would need a [security group](https://docs.aws.amazon.com/cli/latest/userguide/cli-services-ec2-sg.html), that we can attach to our VPC, so that the instances launched in the subnet of the VPC, would leverage the rules defined in the security group.
129+
```
130+
$ aws ec2 create-security-group --group-name kubeadm-sg --description "kubeadm security group" --vpc-id $KUBEADM_VPC_ID
131+
```
132+
133+
Let's save the security group's id in a variable.
134+
```
135+
$ export KUBEADM_SG_ID=$(aws ec2 describe-security-groups | jq -r '.SecurityGroups[] | select(.GroupName == "kubeadm-sg") | .GroupId')
136+
137+
$ echo $KUBEADM_SG_ID
138+
<sg-id>
139+
```
140+
141+
## Image ID
142+
We need to choose an Image for our instances. I am going to choose Ubuntu 20.04 LTS on amd64 architecture, you can get the ami for Ubuntu from this [link](https://cloud-images.ubuntu.com/locator/ec2/), I am going to pick ami-036d46416a34a611c.
143+
144+
## Instance Type
145+
I am creating these instances to make a kubernetes cluser out of those. Hence, we need to choose an instance type that suffices kubernetes [resource requirements](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/), we need to have a minimum of 2 GB Ram and 2 CPUs. Hence I am going to filter for instance types that have 4 GB Ram, which is 4/1.074, i.e. approximately 3.75 GiB or 3840 MiB.
146+
```
147+
$ aws ec2 describe-instance-types | jq '.InstanceTypes[] | select(.MemoryInfo.SizeInMiB == 3840) | (.InstanceType, .VCpuInfo.DefaultVCpus)'
148+
"c3.large"
149+
2
150+
"c4.large"
151+
2
152+
"m3.medium"
153+
1
154+
```
155+
156+
Let's check for instance types with 4GiB RAM this time, which also has 2 Default CPUs
157+
```
158+
$ aws ec2 describe-instance-types | jq '.InstanceTypes[] | select(.MemoryInfo.SizeInMiB == 4096) | select (.VCpuInfo.DefaultVCpus == 2) | .InstanceType' | sort
159+
"a1.large"
160+
"c5ad.large"
161+
"c5a.large"
162+
"c5d.large"
163+
"c5.large"
164+
"c6gd.large"
165+
"c6g.large"
166+
"c6gn.large"
167+
"c6i.large"
168+
"t2.medium"
169+
"t3a.medium"
170+
"t3.medium"
171+
"t4g.medium"
172+
```
173+
174+
Let's check if t2.medium is available in our subnet's availability zone. Please refer to this [link](https://aws.amazon.com/premiumsupport/knowledge-center/ec2-instance-type-not-supported-az-error/) for more details.
175+
```
176+
$ aws ec2 describe-instance-type-offerings --location-type availability-zone | jq '.InstanceTypeOfferings[] | select(.Location == env.KUBEADM_AVAILABILITY_ZONE) | .InstanceType' | grep t2.medium
177+
"t2.medium"
178+
```
179+
So, let's fix t2.medium.
180+
181+
## Instances
182+
We can now go ahead and create the [EC2 instances](https://docs.aws.amazon.com/cli/latest/userguide/cli-services-ec2-instances.html).
183+
```
184+
$ aws ec2 run-instances --image-id ami-036d46416a34a611c --count 3 --instance-type t2.medium --key-name kubeadmKeyPair --security-group-ids $KUBEADM_SG_ID --subnet-id $KUBEADM_SUBNET_ID --associate-public-ip-address
185+
```
186+
187+
We have also enabled public IP address as we need to SSH in to the instances from our machine.
188+
189+
Great, so our instances are created finally.
190+
```
191+
$ aws ec2 describe-instances | jq -r '.Reservations[0] | .Instances[] | select(.SubnetId==env.KUBEADM_SUBNET_ID) | .InstanceId'
192+
<i-id1>
193+
<i-id2>
194+
<i-id3>
195+
```
196+
197+
Let's give these instances, some [names](https://docs.aws.amazon.com/cli/latest/userguide/cli-services-ec2-instances.html#tagging-instances).
198+
```
199+
$ aws ec2 create-tags --resources <i-id1> --tags Key=Name,Value=k8s-master
200+
$ aws ec2 create-tags --resources <i-id2> --tags Key=Name,Value=k8s-node1
201+
$ aws ec2 create-tags --resources <i-id3> --tags Key=Name,Value=k8s-node2
202+
```
203+
204+
We can get the public IP as follows.
205+
```
206+
$ export K8S_MASTER_IP=$(aws ec2 describe-instances --filter Name=tag:Name,Values=k8s-master --query "Reservations[*].Instances[*].PublicIpAddress" --output text)
207+
208+
$ echo $K8S_MASTER_IP
209+
<master-public-ip>
210+
211+
$ export K8S_NODE1_IP=$(aws ec2 describe-instances --filter Name=tag:Name,Values=k8s-node1 --query "Reservations[*].Instances[*].PublicIpAddress" --output text)
212+
213+
$ echo $K8S_NODE1_IP
214+
<node1-public-ip>
215+
216+
$ export K8S_NODE2_IP=$(aws ec2 describe-instances --filter Name=tag:Name,Values=k8s-node2 --query "Reservations[*].Instances[*].PublicIpAddress" --output text)
217+
218+
$ echo $K8S_NODE2_IP
219+
<node2-public-ip>
220+
```
221+
222+
## SSH
223+
We need to access the instances via SSH from the local machine, for which we need allow port 22 on the security group.
224+
225+
Get the local machine's public IP.
226+
```
227+
$ export MY_PUBLIC_IP=$(curl ifconfig.me --silent)
228+
$ echo $MY_PUBLIC_IP
229+
<public-ip>
230+
```
231+
232+
We can now :pencil: [add the rule](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/authorizing-access-to-an-instance.html).
233+
```
234+
$ aws ec2 authorize-security-group-ingress --group-id $KUBEADM_SG_ID --protocol tcp --port 22 --cidr $MY_PUBLIC_IP/32
235+
```
236+
237+
Let's add the node IPs to a file.
238+
```
239+
$ cat > k8s-node-ips.txt <<EOF
240+
$K8S_MASTER_IP
241+
$K8S_NODE1_IP
242+
$K8S_NODE2_IP
243+
EOF
244+
```
245+
246+
And then test the connection.
247+
```
248+
$ for ip in $ips; do ssh ubuntu@$ip -i ~/.ssh/kubeadmKeyPair.pem 'echo -n "Hello World!, my AWS EC2 hostname is "; hostname'; done
249+
Hello World!, my AWS EC2 hostname is ip-10-0-0-5
250+
Hello World!, my AWS EC2 hostname is ip-10-0-0-11
251+
Hello World!, my AWS EC2 hostname is ip-10-0-0-12
252+
```
253+
254+
## Summary
255+
So we have successfully launched 3 EC2 instances using the AWS CLI, you can customize the options in the commands used, to create instances with different configuration as required. Thank you !!! :congratulations:.
256+
257+
--end-of-post--

0 commit comments

Comments
 (0)