Skip to content

Commit ab58692

Browse files
matofederscoopex
matofeder
and
scoopex
authored
Add blog post - Automating SONiC switch provisioning and management with NetBox as a central source (#1045)
* Add blog post - Automating SONiC switch provisioning and management with NetBox as a central source Signed-off-by: Matej Feder <[email protected]> * Fix review comment Signed-off-by: Matej Feder <[email protected]> * Fix typos Signed-off-by: Matej Feder <[email protected]> --------- Signed-off-by: Matej Feder <[email protected]> Co-authored-by: Marc Schöchlin <[email protected]>
1 parent e8ebd0e commit ab58692

File tree

7 files changed

+215
-0
lines changed

7 files changed

+215
-0
lines changed

‎_assets/images/blog/sonic/netbox_config_compliant.png‎

42.7 KB
Loading

‎_assets/images/blog/sonic/netbox_config_diff.png‎

78.3 KB
Loading

‎_assets/images/blog/sonic/netbox_config_sync.png‎

61.5 KB
Loading

‎_assets/images/blog/sonic/netbox_config_sync_done.png‎

85.3 KB
Loading

‎_assets/images/blog/sonic/sonic_netbox.png‎

40.5 KB
Loading

‎_i18n/de/_posts/blog/2024-11-12-sonic.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
_i18n/en/_posts/blog/2024-11-12-sonic.md

‎_i18n/en/_posts/blog/2024-11-12-sonic.md‎

Lines changed: 214 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,214 @@
1+
---
2+
layout: post
3+
title: "Automating SONiC switch provisioning and management with NetBox as a central source"
4+
category: tech
5+
author:
6+
- "Matej Feder"
7+
avatar:
8+
- "matofeder.png"
9+
about:
10+
- "matofeder"
11+
---
12+
13+
Automating network infrastructure is essential for improving efficiency, scalability, and accuracy, particularly in
14+
large-scale environments. In our setup, we’ve developed a system that automates the provisioning and management of a fleet
15+
of SONiC switches by leveraging tools like Zero-Touch Provisioning (ZTP), Open Network Install Environment (ONIE),
16+
and NetBox as our single source of truth.
17+
This configuration reduces the need for manual setup, minimizes human error, and accelerates deployment by centralizing
18+
control within NetBox, ultimately streamlining device provisioning and configuration across the network.
19+
20+
## Provisioning
21+
22+
The architecture for automating the provisioning of SONiC switches relies on several key components:
23+
24+
- DHCP Server: Provides initial network configuration for new switches, directing them to ONIE and ZTP for loading the SONiC image and retrieving initial configuration.
25+
- HTTP Server: Hosts SONiC images, ZTP configuration file, and scripts, making them available for download during provisioning.
26+
- NetBox: Acts as the central source of truth, storing device information, configuration template, and scripts that automate ongoing switch management.
27+
28+
<figure class="figure mx-auto d-block" style="width:50%">
29+
<a href="{% asset "blog/sonic/sonic_netbox.png" @path %}">
30+
{% asset 'blog/sonic/sonic_netbox.png' class="figure-img w-100" %}
31+
</a>
32+
</figure>
33+
34+
### DHCP server
35+
36+
The DHCP server plays a critical role in the initial provisioning phase. When a new SONiC switch connects to the network,
37+
the DHCP server assigns it an IP address and provides options for ONIE and ZTP, guiding the switch to download the
38+
SONiC image and its configuration.
39+
40+
The DHCP server can employ various configuration strategies, such as using the vendor-class-identifier to filter devices.
41+
Some of these strategies are detailed in [ZTP docs](https://github.com/sonic-net/SONiC/blob/master/doc/ztp/ztp.md) and [ONIE docs](https://opencomputeproject.github.io/onie/user-guide/index.html).
42+
43+
Below is an example of a basic DHCP configuration for a host in the SCS lab environment:
44+
```text
45+
host <host> {
46+
hardware ethernet <mac-address>;
47+
server-name "<server-name>";
48+
option host-name "<host-name>";
49+
fixed-address <ipv4-address>;
50+
option default-url "<http-server-address>/sonic-image.bin";
51+
option bootfile-name "<http-server-address>/ztp.json";
52+
}
53+
```
54+
55+
### HTTP server
56+
57+
The HTTP server stores essential files, including the SONiC image and configuration scripts, which ZTP uses to set up
58+
the switch. This server needs to be accessible from the network where the switches are deployed.
59+
60+
### NetBox
61+
62+
NetBox serves as the backbone of this automated setup, providing a structured and centralized source of truth for device
63+
configurations and management scripts. The configuration for SONiC switches combines a configuration template,
64+
the device’s state as defined in NetBox, and configuration context with static settings such as IP addresses for NTP servers.
65+
66+
The following diagram shows the flow of information used to generate the final SONiC configuration:
67+
68+
```text
69+
Configuration context NetBox device state
70+
| |
71+
v v
72+
--------------------------------
73+
| SONiC configuration template |
74+
--------------------------------
75+
|
76+
v
77+
------------------------
78+
| SONiC configuration |
79+
------------------------
80+
```
81+
82+
The SONiC configuration is dynamically generated based on the information stored in NetBox and retrieved via its API.
83+
This approach ensures consistency, as the configuration for each switch can be managed centrally and tailored to each
84+
device’s specifications.
85+
86+
### ONIE
87+
88+
ONIE is an open-source boot loader designed to automate the installation of a network operating system (NOS) on bare-metal
89+
network switches. Developed by Cumulus Networks and now widely supported by the Open Compute Project (OCP), ONIE enables
90+
network flexibility by supporting various NOS options, including SONiC, across different hardware platforms.
91+
92+
When a switch with ONIE boots, it enters a discovery mode where it attempts to locate and install a NOS. This process can
93+
be configured through multiple methods, with DHCP being one of the most common. In our DHCP configuration, ONIE retrieves
94+
a designated NOS image URL, allowing it to automatically download and install the appropriate SONiC version as required.
95+
96+
### ZTP
97+
98+
ZTP is an automated method that streamlines the setup of SONiC network devices, allowing them to be deployed directly
99+
to a site, download configuration files, and begin operation without any manual intervention. This approach significantly
100+
reduces configuration time, minimizes human error, and simplifies scaling.
101+
102+
In our SCS LAB deployment, ZTP enables the initial configuration of network switches.
103+
The ZTP configuration file includes steps to execute essential scripts, retrieve configurations from NetBox (our source of truth),
104+
and validate connectivity. Specifically, the provisioning script within this file pulls the switch’s configuration from
105+
NetBox based on its hostname, ensuring each device receives the appropriate settings, such as interface configurations,
106+
VLANs, BGP configuration and more. Once the configuration is applied, the device automatically reboots, entering its
107+
operational state.
108+
109+
The ZTP configuration file we used is structured as follows:
110+
111+
```json
112+
{
113+
"ztp": {
114+
"01-prerequisites-script": {
115+
"plugin": {
116+
"url": "<http-server-address>/prerequisites.sh"
117+
}
118+
},
119+
"02-provisioning-script": {
120+
"plugin": {
121+
"url": "<http-server-address>/provision.sh",
122+
"ignore-section-data": true,
123+
"args": "--netbox-url <netbox-url> --netbox-token <netbox-token>"
124+
}
125+
},
126+
"03-connectivity-check": {
127+
"ping-hosts": [
128+
"<ip-address-that-should-be-reachable>"
129+
]
130+
},
131+
"reboot-on-success": true
132+
}
133+
}
134+
```
135+
136+
For a detailed example of scripts, visit the related [repository](https://github.com/SovereignCloudStack/hardware-landscape).
137+
138+
## Management
139+
140+
In modern network infrastructures, managing switch configurations can become a complex task, especially when dealing
141+
with scalable environments. To streamline network configuration for SONiC devices, we've implemented an MVP (Minimum Viable
142+
Product) integration between NetBox and SONiC. This integration leverages NetBox's data management capabilities with
143+
custom scripts that help automate configuration synchronization for SONiC devices, making it easier to keep network
144+
configurations consistent and compliant.
145+
146+
NetBox has been enhanced to support configuration management for SONiC devices through two custom scripts that automate
147+
configuration tasks. These scripts `SONiC Config Diff` and `SONiC Config Sync` are designed to align SONiC device
148+
configurations with what’s defined in NetBox. Currently, these scripts are executed manually and offer limited
149+
functionality, but they set the groundwork for more comprehensive network management.
150+
151+
```text
152+
+----------------------+
153+
| NetBox |
154+
+----------------------+
155+
/| Sonic Config Diff |\
156+
/ | Sonic Config Sync | \
157+
/ +----------------------+ \
158+
/ | \
159+
/ | \
160+
+-------------------+ +-------------------+ +-------------------+
161+
| SONiC SW 1 | | SONiC SW 2 | ... | SONiC SW N |
162+
+-------------------+ +-------------------+ +-------------------+
163+
```
164+
165+
### SONiC Config Diff
166+
167+
This script identifies configuration discrepancies between SONiC devices and NetBox. By comparing the running
168+
configuration on SONiC devices with the stored configuration in NetBox, Config Diff highlights differences that could
169+
affect compliance. This script builds on the `netbox-config-diff` plugin, extending its capabilities to support SONiC NOS.
170+
171+
<figure class="figure mx-auto d-block" style="width:50%">
172+
<a href="{% asset "blog/sonic/netbox_config_compliant.png" @path %}">
173+
{% asset 'blog/sonic/netbox_config_compliant.png' class="figure-img w-100" %}
174+
</a>
175+
</figure>
176+
177+
When run, SONiC Config Diff produces a report showing where device settings deviate from the intended configuration,
178+
allowing administrators to easily spot inconsistencies. These results are presented in a visual format, making it easy
179+
to assess compliance at a glance:
180+
181+
<figure class="figure mx-auto d-block" style="width:50%">
182+
<a href="{% asset "blog/sonic/netbox_config_diff.png" @path %}">
183+
{% asset 'blog/sonic/netbox_config_diff.png' class="figure-img w-100" %}
184+
</a>
185+
</figure>
186+
187+
### SONiC Config Sync
188+
189+
While Config Diff identifies differences, SONiC Config Sync enables administrators to apply updates to the SONiC device
190+
based on NetBox configurations. This synchronization process allows NetBox to push missing configuration elements to
191+
SONiC devices, helping bring them up to date with the desired setup. Config Sync is a step toward automated configuration
192+
management, though it only adds configurations and doesn’t support full bidirectional synchronization yet.
193+
Like Config Diff, it can be targeted to specific SONiC devices or run across multiple devices using filters like site,
194+
rack, or device role:
195+
196+
<figure class="figure mx-auto d-block" style="width:50%">
197+
<a href="{% asset "blog/sonic/netbox_config_sync.png" @path %}">
198+
{% asset 'blog/sonic/netbox_config_sync.png' class="figure-img w-100" %}
199+
</a>
200+
</figure>
201+
202+
The result is then visualized as follows:
203+
204+
<figure class="figure mx-auto d-block" style="width:50%">
205+
<a href="{% asset "blog/sonic/netbox_config_sync_done.png" @path %}">
206+
{% asset 'blog/sonic/netbox_config_sync_done.png' class="figure-img w-100" %}
207+
</a>
208+
</figure>
209+
210+
This blog covers the provisioning and management of SONiC switches,advancing the automation of scalable network environments.
211+
By combining NetBox's management capabilities with the flexible, open-source SONiC platform, we are laying the foundation
212+
for simplifying administration and reducing operational complexity.
213+
214+
⚙️ Happy automation!

0 commit comments

Comments
 (0)