@@ -66,7 +66,6 @@ focus on the challenge of provisioning an entire site the first time.
6666We comment on the simpler problem of incrementally provisioning
6767individual resources as relevant details emerge.
6868
69-
70693.1 Physical Infrastructure
7170---------------------------
7271
@@ -130,8 +129,8 @@ information is readily available on the NetBox web site.
130129.. _reading_netbox :
131130.. admonition :: Further Reading
132131
133- `NetBox: <https://netbox.readthedocs.io/en/stable >`_ Information
134- Resource Modeling Application.
132+ `NetBox: <https://docs. netbox.dev >`_ Information Resource Modeling
133+ Application.
135134
136135One of the key features of NetBox is the ability to customize the set
137136of models used to organize all the information that is collected. For
@@ -234,22 +233,23 @@ The following fields are also filled in when creating a Device:
234233* MAC Addresses
235234
236235Note there is typically both a primary and management (e.g., BMC/IPMI)
237- interface, where the *Device Type * implies the specific interfaces.
236+ interface. One convenience feature of Netbox is to use the *Device Type * as a
237+ template that will set the default naming of interfaces, power connections, and
238+ other equipment model specific characteristics.
238239
239- Finally, the virtual interfaces for the Device must be specified, with
240- its *Label * field set to the physical network interface that it is
241- assigned. IP addresses are then assigned to the physical and virtual
242- interfaces we have defined. The Management Server should always have
243- the first IP address in each range, and they should be incremental, as
244- follows:
240+ Finally, the virtual interfaces for the Device must be specified, with its
241+ *Label * field set to the physical network interface that it is assigned. IP
242+ addresses are then assigned to the physical and virtual interfaces we have
243+ defined. The Management Server should always have the first IP address within
244+ each prefix, and by convention they are assigned incrementally as follows:
245245
246246* Management Server
247247
248248 * ``eno1 `` - site provided public IP address, or blank if DHCP provided
249249 * ``eno2 `` - 10.0.0.1/25 (first of ADMIN) - set as primary IP
250250 * ``bmc `` - 10.0.0.2/25 (next of ADMIN)
251- * ``mgmt800 `` - 10.0.0.129/25 (first of MGMT)
252- * ``fab801 `` - 10.0.1.1/25 (first of FABRIC)
251+ * ``mgmt800 `` - 10.0.0.129/25 (first of MGMT, on VLAN 800 )
252+ * ``fab801 `` - 10.0.1.1/25 (first of FABRIC, on VLAN 801 )
253253
254254* Management Switch
255255
@@ -323,7 +323,7 @@ deployment currently include:
323323
324324* Configure the Management Server so it boots from a provided USB key.
325325
326- * Load Ansible roles and playbooks needed to complete configuration
326+ * Run Ansible roles and playbooks needed to complete configuration
327327 onto the Management Server.
328328
329329* Configure the Compute Servers so they boot from the Management
@@ -345,8 +345,13 @@ should be taken to *not* overload this step with configuration that
345345can be done later. For example, various radio parameters can be set on
346346the eNBs when it is physically installed, but those parameters will
347347become settable through the Management Platform once the cluster is
348- brought online. Configuration work done at this stage should be
349- minimized.
348+ brought online.
349+
350+ Manual configuration work done at this stage should be minimized, and most
351+ systems should be configured to use automated means of configuration. For
352+ example, using DHCP pervasively with MAC reservations for IP address assignment
353+ instead of manual configuration of each interface allows for management to be
354+ Zero Touch and simplifies future reconfiguration.
350355
351356The automated aspects of configuration are implemented as a set of
352357Ansible *roles * and *playbooks *, which in terms of the high-level
@@ -359,30 +364,32 @@ parameters that NetBox maintains.
359364
360365The general idea is as follows. For every network service (e.g., DNS,
361366DHCP, iPXE, Nginx) and every per-device subsystem (e.g., network
362- interfaces, Docker) that needs to be configured, there is a
363- corresponding Ansible role and playbook.\ [# ]_ This set is copied onto
364- the Management Server during the manual configuration stage summarized
365- above, and then executed once the management network is online.
367+ interfaces, Docker) that needs to be configured, there is a corresponding
368+ Ansible role and playbook.\ [# ]_ These configurations are applied to the
369+ Management Server during the manual configuration stage summarized above, once
370+ the management network is online.
366371
367372.. [# ] We gloss over the distinction between *roles * and *playbooks *
368373 in Ansible, and focus on the general idea of there being a
369374 *script * that runs with a set of input parameters.
370375
371- The Ansible playbooks instantiate the network services on the
372- Management Server. The role of DNS and DHCP are obvious. As for iPXE
373- and Nginx, they are boot servers for the rest of the infrastructure:
374- the compute servers are configured to boot from the former and the
375- fabric switches are configured to boot from the latter.
376+ The Ansible playbooks install and configure the network services on the
377+ Management Server. The role of DNS and DHCP are obvious. As for iPXE and Nginx,
378+ they are used to bootstrap the rest of the infrastructure: the compute servers
379+ are configured by iPXE delivered over DHCP/TFTP, then loading the scripted OS
380+ installation from a Nginx webserver, and the fabric switches receive their
381+ Stratum OS package from a webserver.
376382
377383In many cases, the playbooks use parameters—such as VLANs, IP
378384addresses, DNS names, and so on—extracted from NetBox. :numref: `Figure
379385%s <fig-ansible>` illustrates the approach, and fills in a few
380386details. For example, a home-grown Python program (``edgeconfig.py ``)
381- extracts data from NetBox and outputs a corresponding set of YAML
382- files, crafted to serve as input to yet another open source tool
383- (*Netplan *), which actually does the detailed work of configuring the
384- network subsystem on the various backend devices. More information
385- about Ansible and Netplan is available on their respective web sites.
387+ extracts data from NetBox using the REST API and outputs a corresponding
388+ set of YAML files, crafted to serve as input to Ansible, which creates yet
389+ more configuration on management and compute systems. One example of this
390+ is the *Netplan * file, which is used in Ubuntu to manage network interfaces.
391+ More information about Ansible and Netplan can be found on their respective web
392+ sites.
386393
387394.. _reading_ansible :
388395.. admonition :: Further Reading
0 commit comments