Skip to content

Commit ae1f79a

Browse files
authored
Merge pull request #47642 from mjpytlak/osdocs3819
OSDOCS-3819: Update AWS VPC endpoint guidance for restricted installations
2 parents d3febf2 + a05047b commit ae1f79a

4 files changed

+109
-51
lines changed

modules/installation-aws-user-infra-requirements.adoc

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,12 +32,36 @@ Alternatively, you can manually create the components or you can reuse existing
3232
* IAM roles
3333
* S3 buckets
3434

35-
If you are working in a disconnected environment or use a proxy, you cannot reach the public IP addresses for EC2 and ELB endpoints. To reach these endpoints, you must create a VPC endpoint and attach it to the subnet that the clusters are using. Create the following endpoints:
35+
If you are working in a disconnected environment, you are unable to reach the public IP addresses for EC2, ELB, and S3 endpoints. Depending on the level to which you want to restrict internet traffic during the installation, the following configuration options are available:
36+
37+
[discrete]
38+
[id="create-vpc-endpoints_{context}"]
39+
=== Option 1: Create VPC endpoints
40+
41+
Create a VPC endpoint and attach it to the subnets that the clusters are using. Name the endpoints as follows:
42+
43+
* `ec2.<region>.amazonaws.com`
44+
* `elasticloadbalancing.<region>.amazonaws.com`
45+
* `s3.<region>.amazonaws.com`
46+
47+
With this option, network traffic remains private between your VPC and the required AWS services.
48+
49+
[discrete]
50+
[id="create-proxy-without-vpc-endpoints_{context}"]
51+
=== Option 2: Create a proxy without VPC endpoints
52+
As part of the installation process, you can configure an HTTP or HTTPS proxy. With this option, internet traffic goes through the proxy to reach the required AWS services.
53+
54+
[discrete]
55+
[id="create-proxy-with-vpc-endpoints_{context}"]
56+
=== Option 3: Create a proxy with VPC endpoints
57+
As part of the installation process, you can configure an HTTP or HTTPS proxy with VPC endpoints. Create a VPC endpoint and attach it to the subnets that the clusters are using. Name the endpoints as follows:
3658

3759
* `ec2.<region>.amazonaws.com`
3860
* `elasticloadbalancing.<region>.amazonaws.com`
3961
* `s3.<region>.amazonaws.com`
4062

63+
When configuring the proxy in the `install-config.yaml` file, add these endpoints to the `noProxy` field. With this option, the proxy prevents the cluster from accessing the internet directly. However, network traffic remains private between your VPC and the required AWS services.
64+
4165
.Required VPC components
4266

4367
You must provide a suitable VPC and subnets that allow communication to your

modules/installation-configure-proxy.adoc

Lines changed: 15 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,9 @@ endif::[]
8484
ifeval::["{context}" == "installing-restricted-networks-aws"]
8585
:aws:
8686
endif::[]
87+
ifeval::["{context}" == "installing-aws-secret-region"]
88+
:aws:
89+
endif::[]
8790
ifeval::["{context}" == "installing-bare-metal"]
8891
:bare-metal:
8992
endif::[]
@@ -183,14 +186,6 @@ The `Proxy` object `status.noProxy` field is populated with the values of the `n
183186
For installations on Amazon Web Services (AWS), Google Cloud Platform (GCP), Microsoft Azure, and {rh-openstack-first}, the `Proxy` object `status.noProxy` field is also populated with the instance metadata endpoint (`169.254.169.254`).
184187
====
185188

186-
ifdef::aws[]
187-
* You have added the
188-
ifdef::aws-china[`ec2.<region>.amazonaws.com.cn`, ]
189-
ifndef::aws-china[`ec2.<region>.amazonaws.com`, ]
190-
`elasticloadbalancing.<region>.amazonaws.com`, and `s3.<region>.amazonaws.com` endpoints to your VPC endpoint. These endpoints are required to complete requests from the nodes to the AWS EC2 API. Because the proxy works on the container level, not the node level, you must route these requests to the AWS EC2 API through the AWS private network. Adding the public IP address of the EC2 API to your allowlist in your proxy server is not sufficient.
191-
endif::aws[]
192-
// TODO: xref installation-aws-user-infra-requirements.adoc#installation-aws-user-infra-other-infrastructure_{context} as a relative link
193-
194189
.Procedure
195190

196191
. Edit your `install-config.yaml` file and add the proxy settings. For example:
@@ -202,18 +197,24 @@ baseDomain: my.domain.com
202197
proxy:
203198
httpProxy: http://<username>:<pswd>@<ip>:<port> <1>
204199
httpsProxy: https://<username>:<pswd>@<ip>:<port> <2>
200+
ifndef::aws[]
205201
noProxy: example.com <3>
202+
endif::aws[]
203+
ifdef::aws[]
204+
noProxy: ec2.<region>.amazonaws.com,elasticloadbalancing.<region>.amazonaws.com,s3.<region>.amazonaws.com <3>
205+
endif::aws[]
206206
additionalTrustBundle: | <4>
207207
-----BEGIN CERTIFICATE-----
208208
<MY_TRUSTED_CA_CERT>
209209
-----END CERTIFICATE-----
210-
...
211210
----
212211
<1> A proxy URL to use for creating HTTP connections outside the cluster. The
213212
URL scheme must be `http`.
214213
<2> A proxy URL to use for creating HTTPS connections outside the cluster.
215-
<3> A comma-separated list of destination domain names, IP addresses, or
216-
other network CIDRs to exclude from proxying. Preface a domain with `.` to match subdomains only. For example, `.y.com` matches `x.y.com`, but not `y.com`. Use `*` to bypass the proxy for all destinations.
214+
<3> A comma-separated list of destination domain names, IP addresses, or other network CIDRs to exclude from proxying. Preface a domain with `.` to match subdomains only. For example, `.y.com` matches `x.y.com`, but not `y.com`. Use `*` to bypass the proxy for all destinations.
215+
ifdef::aws[]
216+
If you have added the Amazon `EC2`,`Elastic Load Balancing`, and `S3` VPC endpoints to your VPC, you must add these endpoints to the `noProxy` field.
217+
endif::aws[]
217218
ifdef::vsphere,vmc[]
218219
You must include vCenter's IP address and the IP range that you use for its machines.
219220
endif::vsphere,vmc[]
@@ -271,6 +272,9 @@ endif::[]
271272
ifeval::["{context}" == "installing-restricted-networks-aws"]
272273
:!aws:
273274
endif::[]
275+
ifeval::["{context}" == "installing-aws-secret-region"]
276+
:!aws:
277+
endif::[]
274278
ifeval::["{context}" == "installing-bare-metal"]
275279
:!bare-metal:
276280
endif::[]

modules/installation-creating-aws-bootstrap.adoc

Lines changed: 19 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,10 @@
77
[id="installation-creating-aws-bootstrap_{context}"]
88
= Creating the bootstrap node in AWS
99

10-
You must create the bootstrap node in Amazon Web Services (AWS) to use during {product-title} cluster initialization.
10+
You must create the bootstrap node in Amazon Web Services (AWS) to use during {product-title} cluster initialization. You do this by:
1111

12-
You can use the provided CloudFormation template and a custom parameter file to create a stack of AWS resources. The stack represents the bootstrap node that your {product-title} installation requires.
12+
* Providing a location to serve the `bootstrap.ign` Ignition config file to your cluster. This file is located in your installation directory. The provided CloudFormation Template assumes that the Ignition config files for your cluster are served from an S3 bucket. If you choose to serve the files from another location, you must modify the templates.
13+
* Using the provided CloudFormation template and a custom parameter file to create a stack of AWS resources. The stack represents the bootstrap node that your {product-title} installation requires.
1314
1415
[NOTE]
1516
====
@@ -30,50 +31,28 @@ have to contact Red Hat support with your installation logs.
3031
3132
.Procedure
3233

33-
. Provide a location to serve the `bootstrap.ign` Ignition config file to your
34-
cluster. This file is located in your installation directory. One way to do this
35-
is to create an S3 bucket in your cluster's region and upload the Ignition
36-
config file to it.
37-
+
38-
[IMPORTANT]
39-
====
40-
The provided CloudFormation Template assumes that the
41-
Ignition config files for your cluster are served from an S3 bucket. If you
42-
choose to serve the files from another location, you must modify the templates.
43-
====
44-
+
45-
[IMPORTANT]
46-
====
47-
If you are deploying to a region that has endpoints that differ from the AWS SDK, or you are providing your own custom endpoints, you must use a presigned URL for your S3 bucket instead of the `s3://` schema.
48-
====
49-
+
50-
[NOTE]
51-
====
52-
The bootstrap Ignition config file does contain secrets, like X.509 keys. The
53-
following steps provide basic security for the S3 bucket. To provide additional
54-
security, you can enable an S3 bucket policy to allow only certain users, such
55-
as the OpenShift IAM user, to access objects that the bucket contains. You
56-
can avoid S3 entirely and serve your bootstrap Ignition config file from any
57-
address that the bootstrap machine can reach.
58-
====
59-
60-
.. Create the bucket:
34+
. Create the bucket by running the following command:
6135
+
6236
[source,terminal]
6337
----
6438
$ aws s3 mb s3://<cluster-name>-infra <1>
6539
----
6640
<1> `<cluster-name>-infra` is the bucket name. When creating the `install-config.yaml` file, replace `<cluster-name>` with the name specified for the cluster.
41+
+
42+
You must use a presigned URL for your S3 bucket, instead of the `s3://` schema, if you are:
43+
** Deploying to a region that has endpoints that differ from the AWS SDK.
44+
** Deploying a proxy.
45+
** Providing your own custom endpoints.
6746

68-
.. Upload the `bootstrap.ign` Ignition config file to the bucket:
47+
. Upload the `bootstrap.ign` Ignition config file to the bucket by running the following command:
6948
+
7049
[source,terminal]
7150
----
7251
$ aws s3 cp <installation_directory>/bootstrap.ign s3://<cluster-name>-infra/bootstrap.ign <1>
7352
----
7453
<1> For `<installation_directory>`, specify the path to the directory that you stored the installation files in.
7554

76-
.. Verify that the file uploaded:
55+
. Verify that the file uploaded by running the following command:
7756
+
7857
[source,terminal]
7958
----
@@ -85,9 +64,13 @@ $ aws s3 ls s3://<cluster-name>-infra/
8564
----
8665
2019-04-03 16:15:16 314878 bootstrap.ign
8766
----
67+
+
68+
[NOTE]
69+
====
70+
The bootstrap Ignition config file does contain secrets, like X.509 keys. The following steps provide basic security for the S3 bucket. To provide additional security, you can enable an S3 bucket policy to allow only certain users, such as the OpenShift IAM user, to access objects that the bucket contains. You can avoid S3 entirely and serve your bootstrap Ignition config file from any address that the bootstrap machine can reach.
71+
====
8872

89-
. Create a JSON file that contains the parameter values that the template
90-
requires:
73+
. Create a JSON file that contains the parameter values that the template requires:
9174
+
9275
[source,json]
9376
----
@@ -188,6 +171,8 @@ deploying the cluster to an AWS GovCloud region.
188171
section of this topic and save it as a YAML file on your computer. This template
189172
describes the bootstrap machine that your cluster requires.
190173
174+
. Optional: If you are deploying the cluster with a proxy, you must update the ignition in the template to add the `ignition.config.proxy` fields. Additionally, If you have added the Amazon EC2, Elastic Load Balancing, and S3 VPC endpoints to your VPC, you must add these endpoints to the `noProxy` field.
175+
191176
. Launch the CloudFormation template to create a stack of AWS resources that represent the bootstrap node:
192177
+
193178
[IMPORTANT]

modules/installation-custom-aws-vpc.adoc

Lines changed: 50 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -67,25 +67,68 @@ The installation program modifies your subnets to add the `kubernetes.io/cluster
6767
+
6868
If you prefer to use your own Route 53 hosted private zone, you must associate the existing hosted zone with your VPC prior to installing a cluster. You can define your hosted zone using the `platform.aws.hostedZone` field in the `install-config.yaml` file.
6969

70-
ifndef::aws-china,aws-secret[]
71-
If you are working in a disconnected environment, you are unable to reach the public IP addresses for EC2 and ELB endpoints. To resolve this, you must create a VPC endpoint and attach it to the subnet that the clusters are using. The endpoints should be named as follows:
70+
ifndef::aws-secret[]
71+
If you are working in a disconnected environment, you are unable to reach the public IP addresses for EC2, ELB, and S3 endpoints. Depending on the level to which you want to restrict internet traffic during the installation, the following configuration options are available:
72+
endif::aws-secret[]
73+
74+
ifdef::aws-secret[]
75+
A cluster in an SC2S or C2S Region is unable to reach the public IP addresses for the EC2, ELB, and S3 endpoints. Depending on the level to which you want to restrict internet traffic during the installation, the following configuration options are available:
76+
endif::aws-secret[]
7277

78+
[discrete]
79+
[id="create-vpc-endpoints_{context}"]
80+
=== Option 1: Create VPC endpoints
81+
82+
Create a VPC endpoint and attach it to the subnets that the clusters are using. Name the endpoints as follows:
83+
84+
ifndef::aws-china,aws-secret[]
7385
* `ec2.<region>.amazonaws.com`
7486
* `elasticloadbalancing.<region>.amazonaws.com`
7587
* `s3.<region>.amazonaws.com`
7688
endif::aws-china,aws-secret[]
7789

7890
ifdef::aws-china[]
79-
If you are working in a disconnected environment, you are unable to reach the public IP addresses for EC2 and ELB endpoints. To resolve this, you must create a VPC endpoint and attach it to the subnet that the clusters are using. The endpoints should be named as follows:
91+
* `ec2.<region>.amazonaws.com.cn`
92+
* `elasticloadbalancing.<region>.amazonaws.com`
93+
* `s3.<region>.amazonaws.com`
94+
endif::aws-china[]
8095

96+
ifdef::aws-secret[]
97+
SC2S::
98+
** `elasticloadbalancing.<region>.sc2s.sgov.gov`
99+
** `ec2.<region>.sc2s.sgov.gov`
100+
** `s3.<region>.sc2s.sgov.gov`
101+
C2S::
102+
** `elasticloadbalancing.<region>.c2s.ic.gov`
103+
** `ec2.<region>.c2s.ic.gov`
104+
** `s3.<region>.c2s.ic.gov`
105+
endif::aws-secret[]
106+
107+
With this option, network traffic remains private between your VPC and the required AWS services.
108+
109+
[discrete]
110+
[id="create-proxy-without-vpc-endpoints_{context}"]
111+
=== Option 2: Create a proxy without VPC endpoints
112+
As part of the installation process, you can configure an HTTP or HTTPS proxy. With this option, internet traffic goes through the proxy to reach the required AWS services.
113+
114+
[discrete]
115+
[id="create-proxy-with-vpc-endpoints_{context}"]
116+
=== Option 3: Create a proxy with VPC endpoints
117+
As part of the installation process, you can configure an HTTP or HTTPS proxy with VPC endpoints. Create a VPC endpoint and attach it to the subnets that the clusters are using. Name the endpoints as follows:
118+
119+
ifndef::aws-china,aws-secret[]
120+
* `ec2.<region>.amazonaws.com`
121+
* `elasticloadbalancing.<region>.amazonaws.com`
122+
* `s3.<region>.amazonaws.com`
123+
endif::aws-china,aws-secret[]
124+
125+
ifdef::aws-china[]
81126
* `ec2.<region>.amazonaws.com.cn`
82127
* `elasticloadbalancing.<region>.amazonaws.com`
83128
* `s3.<region>.amazonaws.com`
84129
endif::aws-china[]
85130

86131
ifdef::aws-secret[]
87-
* A cluster in an SC2S or C2S Region is unable to reach the public IP addresses for the EC2 and ELB endpoints. You must create a VPC endpoint and attach it to the subnet that the clusters are using. Name the endpoints as follows:
88-
+
89132
SC2S::
90133
** `elasticloadbalancing.<region>.sc2s.sgov.gov`
91134
** `ec2.<region>.sc2s.sgov.gov`
@@ -96,6 +139,8 @@ C2S::
96139
** `s3.<region>.c2s.ic.gov`
97140
endif::aws-secret[]
98141

142+
When configuring the proxy in the `install-config.yaml` file, add these endpoints to the `noProxy` field. With this option, the proxy prevents the cluster from accessing the internet directly. However, network traffic remains private between your VPC and the required AWS services.
143+
99144
.Required VPC components
100145

101146
You must provide a suitable VPC and subnets that allow communication to your

0 commit comments

Comments
 (0)