Skip to content

Commit e9168e6

Browse files
YufengXinYufengXin
authored
Merge pull request #169 from atlanticwave-sdx/mcevik-production-deployment
Merge changes for production-deployment to master
2 parents 487a472 + 7a2a7f1 commit e9168e6

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

56 files changed

+3083
-600
lines changed

‎configuration/awave-production/README.md‎

Lines changed: 158 additions & 53 deletions
Original file line numberDiff line numberDiff line change
@@ -2,106 +2,211 @@
22

33
## Testbed Topology
44

5-
![alt text](figures/AW-SDX-Topology.png)
5+
![alt text](figures/AW-SDX-Production_Setup-1-Topology.png)
66

77
## Node Layout
88

9-
![alt text](figures/AW-SDX-Node-Layout.png)
9+
![alt text](figures/AW-SDX-Production_Setup-2-Node_Layout.png)
10+
11+
## Tunnel Layout
12+
13+
![alt text](figures/AW-SDX-Production_Setup-3-Tunnel_Layout-L2Multipoint.png)
1014

11-
Direct Link: https://www.draw.io/#G1C7yn7oQ9FrGdVVxNnJtcGu9MOPG7U2xL
1215

1316
```
1417
# Out-of-band management
18+
19+
- Miami
20+
- miami-corsa : 67.17.206.198
21+
- miami-vm : 190.103.186.106
22+
23+
- Atlanta
24+
- sox-corsa : 143.215.216.3
25+
- Baremetal server: 128.61.149.224
26+
- awsdx-ctrl (VM) : 128.61.149.223
27+
- awsdx-app (VM) : 128.61.149.224
28+
29+
- Santiago
30+
- santiago-corsa : 139.229.127.101
31+
- santiago-dtn : 200.0.207.86
32+
1533
# OpenFlow Conrol Connection
1634
1735
# SDX Controller
18-
192.168.201.156 atlanticwave-sdx-controller.renci.ben
36+
128.61.149.224 awsdx-app sdx.atlanticwave-sdx.net
1937
20-
# Local Controllers (running on BF40G Nodes)
21-
192.168.201.196 bf40g1.renci.ben atlanticwave-lc-renci.renci.ben
22-
192.168.202.39 bg40g1.duke.ben atlanticwave-lc-duke.renci.ben
23-
192.168.203.10 bg40g1.unc.ben atlanticwave-lc-unc.renci.ben
24-
192.168.204.21 bg40g1.ncsu.ben atlanticwave-lc-ncsu.renci.ben
38+
# Local Controllers
39+
128.61.149.223 awsdx-ctrl lc-atl.atlanticwave-sdx.net
40+
190.103.186.106 miami-vm lc-mia.atlanticwave-sdx.net
41+
200.0.207.86 santiago-vm lc-chl.atlanticwave-sdx.net
2542
2643
# Corsa Switches
27-
192.168.201.168 corsa-2.renci.ben
28-
192.168.201.169 corsa-1.renci.ben
29-
192.168.202.30 corsa-1.duke.ben
30-
192.168.203.30 corsa-1.unc.ben
31-
192.168.204.30 corsa-1.ncsu.ben
44+
143.215.216.3 sox-corsa corsa-atl.atlanticwave-sdx.net
45+
67.17.206.198 miami-corsa corsa-mia.atlanticwave-sdx.net
46+
139.229.127.101 santiago-corsa corsa-chl.atlanticwave-sdx.net
3247
3348
```
3449

3550
```
3651
# In-band management
37-
# VLAN 1411 is used for in-band management
52+
# VLANs across sites are shown on topology drawing.
3853
3954
# SDX Controller
40-
10.14.11.254 atlanticwave-sdx-controller.renci.ben
55+
10.14.11.254 sdx.atlanticwave-sdx.net
4156
4257
# Local Controllers
43-
10.14.11.1 atlanticwave-lc-renci.renci.ben
44-
10.14.11.2 atlanticwave-lc-duke.renci.ben
45-
10.14.11.3 atlanticwave-lc-unc.renci.ben
46-
10.14.11.4 atlanticwave-lc-ncsu.renci.ben
58+
10.14.11.1 lc-atl.atlanticwave-sdx.net
59+
10.14.11.2 lc-mia.atlanticwave-sdx.net
60+
10.14.11.3 lc-chl.atlanticwave-sdx.net
4761
4862
```
4963

5064
## Corsa Switches
5165

52-
Ports are set to `passthrough` mode. (ctag mode strips off the VLAN tag that prevent flows with `dl_vlan` match field being pushed properly. This can be a pure openflow behavior that needs to be elaborated.)
66+
Configuration on the switches prior to the deployment is saved on the files.
67+
- [sox-corsa](config/sox-corsa.cfg)
68+
- [miami-corsa](config/miami-corsa.cfg)
69+
- [chl-corsa](config/chl-corsa.cfg)
5370

54-
Openflow control connection is established through default namespace (and associated out-of-band management interface).
71+
Also, backup of the active-configuration is saved on the switches.
5572

56-
![alt text](figures/AW-SDX-Corsa-Tunnel-Layout.png)
73+
```
74+
corsa-sdx-56m# show file backup full
75+
.
76+
├── [3.8K Jun 1 3:00:02] corsa-sdx-56m.bkp.0.030002.12.2020.06.01.030002.tar.bz2
77+
├── [3.8K Jun 8 3:00:02] corsa-sdx-56m.bkp.0.030002.12.2020.06.08.030001.tar.bz2
78+
├── [5.1K Jun 9 18:08:06] corsa-sdx-56m.bkp.0.030002.12.2020.06.09.180806.tar.bz2
79+
├── [3.8K Jun 15 3:00:01] corsa-sdx-56m.bkp.0.030002.12.2020.06.15.030001.tar.bz2
80+
├── [3.8K Jun 22 3:00:02] corsa-sdx-56m.bkp.0.030002.12.2020.06.22.030002.tar.bz2
81+
└── [3.4K Jun 25 2:58:11] corsa-atl.atlanticwave-sdx.net.p12
82+
```
5783

5884

85+
Ports are set to `passthrough` mode. (ctag mode strips off the VLAN tag that prevent flows with `dl_vlan` match field being pushed properly. This can be a pure openflow behavior that needs to be elaborated.)
86+
87+
Openflow control connection is established through default namespace (and associated out-of-band management interface).
88+
5989

6090
### Set port tunnel-modes
6191

92+
6293
```
63-
# RENCI-1
64-
configure port 1 tunnel-mode passthrough
65-
configure port 2 tunnel-mode passthrough
66-
configure port 11 tunnel-mode passthrough
67-
configure port 12 tunnel-mode passthrough
68-
configure port 23 tunnel-mode passthrough
94+
#
95+
# sox-corsa
96+
#
97+
98+
# Primary forwarding ports
99+
configure port 1 tunnel-mode passthrough
100+
101+
102+
# Rate-limiting ports
103+
configure port 29 tunnel-mode passthrough
69104
configure port 30 tunnel-mode passthrough
70-
configure port 19 tunnel-mode passthrough
71-
configure port 20 tunnel-mode passthrough
105+
configure port 31 tunnel-mode ctag
106+
configure port 32 tunnel-mode ctag
107+
108+
109+
# Multipoint Rate-limiting ports
110+
configure port 25 tunnel-mode ctag
111+
configure port 26 tunnel-mode ctag
112+
configure port 27 tunnel-mode ctag
113+
configure port 28 tunnel-mode ctag
114+
115+
```
72116

73117
```
118+
#
119+
# miami-corsa
120+
#
121+
122+
# Primary forwarding ports
123+
amlight-corsa# configure port 1 tunnel-mode passthrough
124+
amlight-corsa# configure port 2 tunnel-mode passthrough
125+
amlight-corsa# configure port 3 tunnel-mode passthrough
126+
amlight-corsa# configure port 4 tunnel-mode passthrough
127+
amlight-corsa# configure port 10 tunnel-mode passthrough
128+
129+
# Rate-limiting ports
130+
amlight-corsa# configure port 23 tunnel-mode passthrough
131+
amlight-corsa# configure port 24 tunnel-mode passthrough
132+
amlight-corsa# configure port 25 tunnel-mode ctag
133+
amlight-corsa# configure port 26 tunnel-mode ctag
134+
135+
# Multipoint Rate-limiting ports
136+
amlight-corsa# configure port 13 tunnel-mode ctag
137+
amlight-corsa# configure port 14 tunnel-mode ctag
138+
amlight-corsa# configure port 15 tunnel-mode ctag
139+
amlight-corsa# configure port 16 tunnel-mode ctag
140+
```
141+
74142

75143

76-
### Create VFCs on all Corsa switches
144+
145+
### Create VFCs
77146

78147
```
79-
# RENCI-1
148+
#
149+
# sox-corsa
150+
#
151+
152+
# Create Primary VFC
80153
81154
configure bridge add br21 openflow resources 10
82155
configure bridge br21 dpid 0xC9
83156
configure bridge br21 tunnel attach ofport 1 port 1
84-
configure bridge br21 tunnel attach ofport 2 port 2
85-
configure bridge br21 tunnel attach ofport 11 port 11
86-
configure bridge br21 tunnel attach ofport 12 port 12
87-
configure bridge br21 tunnel attach ofport 23 port 23
157+
configure bridge br21 tunnel attach ofport 29 port 29
88158
configure bridge br21 tunnel attach ofport 30 port 30
89-
configure bridge br21 tunnel attach ofport 19 port 19
90-
configure bridge br21 tunnel attach ofport 20 port 20
91-
configure bridge br21 controller add CONTbr21 192.168.201.196 6681
159+
configure bridge br21 controller add CONTbr21 143.215.216.21 6681
160+
161+
# Create Multipoint Rate-limiting VFC
162+
163+
configure bridge add br19 l2-vpn resources 10
164+
configure bridge br19 controller add CONTbr19 172.17.1.1 6653
165+
166+
# Create L2Tunnel Rate-limiting VFC
167+
# by using scripts in ratelimiting-vfc directory
168+
python make-rate-limiting-switch-clean-sox.py
92169
93170
```
94171

172+
```
173+
#
174+
# miami-corsa
175+
#
176+
177+
# Create Primary VFC
178+
179+
amlight-corsa# configure bridge add br22 openflow resources 10
180+
amlight-corsa# configure bridge br22 dpid 0xCA
181+
amlight-corsa# configure bridge br22 tunnel attach ofport 1 port 1
182+
amlight-corsa# configure bridge br22 tunnel attach ofport 2 port 2
183+
amlight-corsa# configure bridge br22 tunnel attach ofport 3 port 3
184+
amlight-corsa# configure bridge br22 tunnel attach ofport 4 port 4
185+
amlight-corsa# configure bridge br22 tunnel attach ofport 10 port 10
186+
amlight-corsa# configure bridge br22 tunnel attach ofport 23 port 23
187+
amlight-corsa# configure bridge br22 tunnel attach ofport 24 port 24
188+
amlight-corsa# configure bridge br22 controller add CONTbr22 190.103.186.106 6682
189+
190+
191+
# Create Multipoint Rate-limiting VFC
192+
193+
amlight-corsa# configure bridge add br19 l2-vpn resources 10
194+
amlight-corsa# configure bridge br19 controller add CONTbr19 172.17.1.1 6653
195+
196+
# Create L2Tunnel Rate-limiting VFC
197+
# by using scripts in ratelimiting-vfc directory
198+
python make-rate-limiting-switch-clean-miami.py
199+
```
95200

96201

97202
### Rate Limiting VFC
98203

99204
Physical ports are in ctag tunnel-mode.
100-
On RENCI-1, DUKE, UNC, NCSU switches Port 21 and 22 are attached to the rate-limiting VFC.
101-
On RENCI-2 switch ports 27 and 28 are attached to the rate-limiting VFC.
205+
miami-corsa: Port 25 and 26 are attached to the rate-limiting VFC.
206+
sox-corsa: Port 31 and 32 are attached to the rate-limiting VFC.
102207

103208
```
104-
# Create rate limiting VFC (br20) on RENCI-1 | RENCI-2 | DUKE | UNC | NCSU
209+
# Create rate limiting VFC (br20)
105210
configure bridge add br20 vpws resources 2
106211
configure bridge br20 controller add Eline 172.17.2.1 6653
107212
application eline configure connection add atlanticwave 21 22 "Rate Limiting VFC"
@@ -110,11 +215,8 @@ application eline configure connection add atlanticwave 21 22 "Rate Limiting VFC
110215
Scripts in ratelimiting-vfc can be used to delete and create the rate-limiting VFCs.
111216

112217
```
113-
python make-rate-limiting-switch-clean-renci.py
114-
python make-rate-limiting-switch-clean-duke.py
115-
python make-rate-limiting-switch-clean-unc.py
116-
python make-rate-limiting-switch-clean-ncsu.py
117-
python make-rate-limiting-switch-clean-renci-2.py
218+
python make-rate-limiting-switch-clean-miami.py
219+
python make-rate-limiting-switch-clean-sox.py
118220
119221
or
120222
@@ -130,16 +232,19 @@ Type of the controller and site names are extracted from hostnames.
130232

131233
```
132234
# Build docker images
133-
/root/aw.sh -R <REPO> -B <BRANCH> -b
235+
/root/aw.sh -R <REPO> -B <BRANCH> -G <CONFIG> -H <MANIFEST> -b
134236
Default Repo: https://github.com/RENCI-NRIG/atlanticwave-proto.git
135-
Default Branch: master-rci
237+
Default Branch: master
238+
Configuration parameter maps to the directories under atlanticwave-proto/configuration (eg. renci_testbed, awave-production)
239+
Manifest can be selected from inside the selected configuration.
240+
136241
137-
/root/aw.sh -B renci-corsa-ben -b
242+
/root/aw.sh -B master -G awave-production -H awave-production.manifest -b
138243
139244
# Run docker containers (in detached mode)
140-
/root/aw.sh -m detached -r
245+
/root/aw.sh -m detached -G awave-production -H awave-production.manifest -r
141246
142247
# Run docker containers (interactive)
143-
/root/aw.sh -r
248+
/root/aw.sh -G awave-production -H awave-production.manifest -r
144249
```
145250

0 commit comments

Comments
 (0)