Skip to content

Commit 8ddb09d

Browse files
aireillyaireilly
committed
networking module for CNF-1500 ZTP
1 parent d70c621 commit 8ddb09d

12 files changed

+403
-30
lines changed

‎_topic_map.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2015,8 +2015,8 @@ Topics:
20152015
- Name: Provisioning and deploying a distributed unit (DU)
20162016
File: cnf-provisioning-and-deploying-a-distributed-unit
20172017
Distros: openshift-webscale
2018-
- Name: Zero Touch Provisioning
2019-
File: ztp-zero-touch-provisioning
2018+
- Name: Deploying distributed units at scale in a disconnected environment
2019+
File: ztp-deploying-disconnected
20202020
Distros: openshift-webscale
20212021
---
20222022
Name: Backup and restore

‎modules/common-attributes.adoc‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,8 @@ endif::[]
2626
:cloud-redhat-com: Red Hat OpenShift Cluster Manager
2727
:rh-storage-first: Red Hat OpenShift Container Storage
2828
:rh-storage: OpenShift Container Storage
29+
:rh-rhacm-first: Red Hat Advanced Cluster Management (RHACM)
30+
:rh-rhacm: RHACM
2931
:sandboxed-containers-first: OpenShift sandboxed containers
3032
:sandboxed-containers-operator: OpenShift sandboxed containers Operator
3133
:rh-virtualization-first: Red Hat Virtualization (RHV)

‎modules/installation-creating-mirror-registry.adoc‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,11 @@ endif::[]
1313
Create a registry to host the mirrored content that you require for installing
1414
{product-title}.
1515

16+
[IMPORTANT]
17+
====
18+
Deploying a disconnected registry host based on the `docker.io/library/registry:2` API for {product-title} is not officially supported by Red Hat. You can create a mirror host based on the `docker.io/library/registry:2` API with the following unsupported procedure.
19+
====
20+
1621
ifdef::restricted[]
1722
For installation in a restricted network, you can place the mirror
1823
registry on a host that can be accessed from both the your network and

‎modules/ztp-acm-adding-images-to-mirror-registry.adoc‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
// CNF-1500 ZTP - preparing to install ACM
2+
// Module included in the following assemblies:
3+
//
4+
// scalability_and_performance/ztp-deploying-disconnected.adoc
5+
[id="ztp-acm-adding-images-to-mirror-registry_{context}"]
6+
= Adding {op-system} ISO and RootFS images to a disconnected mirror host
7+
8+
Before you install a cluster on infrastructure that you provision, you must create {op-system-first} machines for it to use. Use a disconnected mirror to host the {op-system} images you require to provision your distributed unit (DU) bare-metal hosts.
9+
10+
.Prerequisites
11+
12+
* Deploy and configure a HTTP server to host the {op-system} image resources on the network. You must be able to access the HTTP server from your computer, and from the machines that you create.
13+
14+
[IMPORTANT]
15+
====
16+
The {op-system} images might not change with every release of {product-title}. You must download images with the highest version that is less than or equal to the {product-title} version that you install. Use the image versions that match your {product-title} version if they are available. You require ISO and RootFS images to install {op-system} on the DU hosts. {op-system} qcow2 images are not supported for this installation type.
17+
====
18+
19+
.Procedure
20+
21+
. Log in to the mirror host.
22+
. Obtain the {op-system} ISO and RootFS images from link:https://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/pre-release/[mirror.openshift.com], for example:
23+
24+
.. Export the required image names and {product-title} version as environment variables:
25+
+
26+
[source,terminal]
27+
----
28+
$ export ISO_IMAGE_NAME=<iso_image_name> <1>
29+
----
30+
+
31+
[source,terminal]
32+
----
33+
$ export ROOTFS_IMAGE_NAME=<rootfs_image_name> <2>
34+
----
35+
+
36+
[source,terminal]
37+
----
38+
$ export OCP_VERSION=<ocp_version> <3>
39+
----
40+
<1> ISO image name, for example, `rhcos-4.8.0-fc.9-x86_64-live.x86_64.iso`
41+
<2> RootFS image name, for example, `rhcos-4.8.0-fc.9-x86_64-live-rootfs.x86_64.img`
42+
<3> {product-title} version, for example, `latest-4.8`
43+
44+
.. Download the required images:
45+
+
46+
[source,terminal]
47+
----
48+
$ sudo wget https://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/pre-release/${OCP_VERSION}/${ISO_IMAGE_NAME} -O /var/www/html/${ISO_IMAGE_NAME}
49+
----
50+
+
51+
[source,terminal]
52+
----
53+
$ sudo wget https://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/pre-release/${OCP_VERSION}/${ROOTFS_IMAGE_NAME} -O /var/www/html/${ROOTFS_IMAGE_NAME}
54+
----
55+
56+
.Verification steps
57+
58+
* Verify that the images downloaded successfully and are being served on the disconnected mirror host, for example:
59+
+
60+
[source,terminal]
61+
----
62+
$ wget http://$(hostname)/${ISO_IMAGE_NAME}
63+
----
64+
+
65+
.Expected output
66+
+
67+
[source,terminal]
68+
----
69+
...
70+
Saving to: rhcos-4.8.0-fc.9-x86_64-live.x86_64.iso
71+
rhcos-4.8.0-fc.8-x86_64- 11%[====> ] 10.01M 4.71MB/s
72+
...
73+
----

‎modules/ztp-acm-installing-disconnected-rhacm.adoc‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
// Module included in the following assemblies:
2+
//
3+
// scalability_and_performance/ztp-deploying-disconnected.adoc
4+
5+
[id="installing-disconnected-rhacm_{context}"]
6+
= Installing Red Hat Advanced Cluster Management in a disconnected environment
7+
8+
You use {rh-rhacm-first} on a hub cluster in the disconnected environment to manage the deployment of distributed unit (DU) profiles on multiple managed spoke clusters.
9+
10+
.Prerequisites
11+
12+
* Install the {product-title} CLI (`oc`).
13+
* Log in as a user with `cluster-admin` privileges.
14+
* Configure a disconnected mirror registry for use in the cluster.
15+
* Enable the disconnected Operator Lifecycle Manager (OLM). {rh-rhacm} is included in the OLM Red Hat Operator catalog. Follow the steps in xref:../operators/admin/olm-restricted-networks.adoc[Using Operator Lifecycle Manager on restricted networks].
16+
17+
.Procedure
18+
19+
* Install {rh-rhacm} on the hub cluster in the disconnected environment. See link:https://access.redhat.com/documentation/en-us/red_hat_advanced_cluster_management_for_kubernetes/2.2/html/install/installing#installing-in-a-disconnected-environment[Installing {rh-rhacm} in a disconnected environment].

‎modules/ztp-acm-preparing-to-install-disconnected-acm.adoc‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
// CNF-1500 ZTP - preparing to install ACM
2+
// Module included in the following assemblies:
3+
//
4+
// scalability_and_performance/ztp-deploying-disconnected.adoc
5+
[id="ztp-acm-preparing-to-install-disconnected-acm_{context}"]
6+
= Preparing the disconnected environment
7+
8+
Before you can provision distributed units (DU) at scale, you must install {rh-rhacm-first}, which handles the provisioning of the DUs.
9+
10+
{rh-rhacm} is deployed as an Operator on the {product-title} hub cluster. It controls clusters and applications from a single console with built-in security policies. {rh-rhacm} provisions and manage your DU hosts. To install {rh-rhacm} in a disconnected environment, you create a mirror registry that mirrors the Operator Lifecycle Manager (OLM) catalog that contains the required Operator images. OLM manages, installs, and upgrades Operators and their dependencies in the cluster.
11+
12+
You also use a disconnected mirror host to serve the {op-system} ISO and RootFS disk images that provision the DU bare-metal host operating system.
13+

‎modules/ztp-du-host-bios-requirements.adoc‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
// CNF-1500 ZTP
2+
// Module included in the following assemblies:
3+
//
4+
// scalability_and_performance/ztp-deploying-disconnected.adoc
5+
6+
[id="configuring_bios_for_du_hosts_{context}"]
7+
= Configuring BIOS for distributed unit bare-metal hosts
8+
9+
Distributed unit (DU) hosts require the BIOS to be configured before the host can be provisioned. The BIOS configuration is dependent on the specific hardware that runs your DUs and the particular requirements of your installation.
10+
11+
[IMPORTANT]
12+
====
13+
In this Developer Preview release, configuration and tuning of BIOS for DU bare-metal host machines is the responsibility of the customer. Automatic setting of BIOS is not handled by the zero touch provisioning workflow.
14+
====
15+
16+
.Procedure
17+
18+
. Set the *UEFI/BIOS Boot Mode* to `UEFI`.
19+
. In the host boot sequence order, set *Hard drive first*.
20+
. Apply the specific BIOS configuration for your hardware. The following table describes a representative BIOS configuration for an Intel Xeon Skylake or Intel Cascade Lake server, based on the Intel FlexRAN 4G and 5G baseband PHY reference design.
21+
+
22+
[IMPORTANT]
23+
====
24+
The exact BIOS configuration depends on your specific hardware and network requirements. The following sample configuration is for illustrative purposes only.
25+
====
26+
+
27+
.Sample BIOS configuration for an Intel Xeon Skylake or Cascade Lake server
28+
[cols=2*, width="90%", options="header"]
29+
|====
30+
|BIOS Setting
31+
|Configuration
32+
33+
|CPU Power and Performance Policy
34+
|Performance
35+
36+
|Uncore Frequency Scaling
37+
|Disabled
38+
39+
|Performance P-limit
40+
|Disabled
41+
42+
|Enhanced Intel SpeedStep (R) Tech
43+
|Enabled
44+
45+
|Intel Configurable TDP
46+
|Enabled
47+
48+
|Configurable TDP Level
49+
|Level 2
50+
51+
|Intel(R) Turbo Boost Technology
52+
|Enabled
53+
54+
|Energy Efficient Turbo
55+
|Disabled
56+
57+
|Hardware P-States
58+
|Disabled
59+
60+
|Package C-State
61+
|C0/C1 state
62+
63+
|C1E
64+
|Disabled
65+
66+
|Processor C6
67+
|Disabled
68+
|====
69+
70+
[NOTE]
71+
====
72+
Enable global SR-IOV and VT-d settings in the BIOS for the host. These settings are relevant to bare-metal environments.
73+
====

‎modules/ztp-du-host-networking-requirements.adoc‎

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
// CNF-1500 ZTP
2+
// Module included in the following assemblies:
3+
//
4+
// scalability_and_performance/ztp-deploying-disconnected.adoc
5+
6+
[id="ztp-du-host-networking-requirements_{context}"]
7+
= Distributed unit host networking requirements
8+
9+
The following tables provide a high level overview of the networking information and custom resources required by {rh-rhacm-first} to provision a DU bare-metal host:
10+
11+
.Required `AgentClusterInstall` networking fields
12+
[cols="2,4", width="90%", options="header"]
13+
|====
14+
|Field
15+
|Description
16+
17+
|`imageSetRef`
18+
| Installer image used to install {product-title} on the DU.
19+
20+
|`clusterNetwork`
21+
| Used to allocate an IPv4 or IPv6 IP address to each node. Ensure there is no overlap with `serviceNetwork`.
22+
23+
|`serviceNetwork`
24+
| Block of IPv4 or IPv6 IP addresses used for cluster services internal communication in {product-title}. Ensure there is no overlap with `clusterNetwork`.
25+
26+
|`machineNetwork`
27+
| Represents the network range for external communication. Also used to determine the API and Ingress VIP addresses for provisioning the cluster.
28+
|====
29+
30+
[NOTE]
31+
====
32+
Do not specify API and Ingress VIP addresses for DU single node clusters. Instead, when the host is provisioned by the assisted installer service, the `machineNetwork` field in the `AgentClusterInstall` CR is used to determine the API and Ingress VIP addresses.
33+
====
34+
35+
.Required `ClusterDeployment` networking fields
36+
[cols="2,4", width="90%", options="header"]
37+
|====
38+
|Field
39+
|Description
40+
41+
|`baseDomain`
42+
|Base domain for the hub cluster managing the individual DU single node clusters.
43+
44+
|`sshPrivateKeySecretRef`
45+
|SSH private key for secure transactions with the single node cluster DU.
46+
47+
|`pullSecretRef`
48+
|Pull secret for secure installation on the DU host.
49+
|====
50+
51+
.Required `BareMetalHost` networking fields
52+
[cols="2,4", width="90%", options="header"]
53+
|====
54+
|Field
55+
|Description
56+
57+
|`bmc`
58+
|BMC address and BMC username and password credentials.
59+
60+
|`bootMACAddress`
61+
|Boot MAC address for the bare-metal host.
62+
63+
|`bmac.agent-install.openshift.io/hostname`
64+
|Optional: Configures the cluster hostname. If this field is not used, a hostname is allocated by the cluster DHCP server.
65+
66+
|`spec.bmc.address`
67+
|Location of the installation ISO.
68+
69+
|`spec.bmc.credentialsName`
70+
|Name of the `bmcCredentials` secret used to access the ISO image.
71+
72+
|`userData.bootkey`
73+
|Reference to the `Secret` containing the user data to be passed to the host before it boots from the ISO image.
74+
75+
|====
76+
77+
.Required `InfraEnv` networking fields
78+
[cols="2,4", width="90%", options="header"]
79+
|====
80+
|Field
81+
|Description
82+
83+
|`additionalNTPSources`
84+
|IP address for a Network Time Protocol (NTP) server. NTP is required to ensure that the certificates are installed correctly on the DU host. The NTP server is only required during provisioning.
85+
86+
|`pullSecretRef`
87+
|Name of the pull secret created for the DU host.
88+
89+
|====
90+
91+
.Required `NMStateConfig` networking fields
92+
[cols="2,4", width="90%", options="header"]
93+
|====
94+
|Field
95+
|Description
96+
97+
|`dns-resolver`
98+
|Target cluster DNS server.
99+
100+
|`interfaces`
101+
|Configures `eno1` for IPv4 and IPv6 connectivity.
102+
103+
|`Routes`
104+
|Configures the default route for the target cluster.
105+
106+
|`mac-address`
107+
|Target bare-metal host MAC address. Must match the MAC address specified in the `BareMetalHost` custom resource (CR).
108+
109+
|`ip-address`
110+
|Target bare-metal host static IP address.
111+
112+
|`public-network-prefix`
113+
|Bare-metal host static IP address subnet.
114+
115+
|`gateway`
116+
|Target bare-metal host gateway.
117+
118+
|`Interfaces`
119+
|Target bare-metal host interface name and MAC address.
120+
|====
121+
122+
[NOTE]
123+
====
124+
`NMStateConfig` is an optional resource. Use `NMStateConfig` to configure network bonding for a pair of NICs, use a concrete VLAN, or to declare a static IP for the DU host. Each `NMState` profile has a one-to-one relationship with a related `InfraEnv` ISO profile used for installing {product-title} on the host. If used, the `NMStateConfig` resource must be created before the `ClusterDeployment` resource. The `NMStateConfig` resource is not required if DHCP is enabled for the cluster network.
125+
====

‎modules/ztp-provisioning-edge-sites-at-scale.adoc‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
// Module included in the following assemblies:
2+
//
3+
// scalability_and_performance/ztp-deploying-disconnected.adoc
4+
5+
[id="provisioning-edge-sites-at-scale_{context}"]
6+
= Provisioning edge sites at scale
7+
8+
Telco edge computing presents extraordinary challenges with managing hundreds to tens of thousands of clusters in hundreds of thousands of locations. These challenges require fully-automated management solutions with, as closely as possible, zero human interaction.
9+
10+
Zero touch provisioning (ZTP) allows you to provision new edge sites with declarative configurations of bare-metal equipment at remote sites. Template or overlay configurations install {product-title} features that are required for CNF workloads. End-to-end functional test suites are used to verify CNF related features. All configurations are declarative in nature.
11+
12+
You start the workflow by creating declarative configurations for ISO images that are delivered to the edge nodes to begin the installation process. The images are used to repeatedly provision large numbers of nodes efficiently and quickly, allowing you keep up with requirements from the field for far edge nodes.
13+
14+
Service providers are deploying a more distributed mobile network architecture allowed by the modular functional framework defined for 5G. This allows service providers to move from appliance-based radio access networks (RAN) to open cloud RAN architecture, gaining flexibility and agility in delivering services to end users.

‎modules/ztp-site-planning-for-du-deployments.adoc‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
// Module included in the following assemblies:
2+
//
3+
// scalability_and_performance/ztp-deploying-disconnected.adoc
4+
5+
[id="ztp-site-planning-for-du-deployments_{context}"]
6+
= Site planning considerations for distributed unit deployments
7+
8+
Site planning for distributed units (DU) deployments is complex. The following is an overview of the tasks that you complete before the DU hosts are brought online in the production environment.
9+
10+
* Develop a network model. The network model depends on various factors such as the size of the area of coverage, number of hosts, projected traffic load, DNS, and DHCP requirements.
11+
* Decide how many DU radio nodes are required to provide sufficient coverage and redundancy for your network.
12+
* Develop mechanical and electrical specifications for the DU host hardware.
13+
* Develop a construction plan for individual DU site installations.
14+
* Tune host BIOS settings for production, and deploy the BIOS configuration to the hosts.
15+
* Install the equipment on-site, connect hosts to the network, and apply power.
16+
* Configure on-site switches and routers.
17+
* Perform basic connectivity tests for the host machines.
18+
* Establish production network connectivity, and verify host connections to the network.
19+
* Provision and deploy on-site DU hosts at scale.
20+
* Test and verify on-site operations, performing load and scale testing of the DU hosts before finally bringing the DU infrastructure online in the live production environment.

0 commit comments

Comments
 (0)