Skip to content

Commit deadf60

Browse files
dulekEmilienM
authored andcommitted
Add dsal-helper utility
This thing is a helper script automating generation of anaconda-ks.cfg, cloning dev-install and setting local-overrides.yaml there, and preparing cloud for OpenShift installation and generating install-config.yaml accordingly.
1 parent cfbb1d9 commit deadf60

File tree

13 files changed

+429
-0
lines changed

13 files changed

+429
-0
lines changed

tools/dsal/.gitignore

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
anaconda
2+
dev-install
3+
install-config
4+
__pycache__
5+
*.pyc
6+
.idea

tools/dsal/README.md

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# DSAL tools
2+
3+
Tool box helping with provisioning fresh DSAL boxes. It aims at providing full
4+
management of configurations required to configure the boxes.
5+
6+
## dsal_helper.py
7+
8+
Utility producing useful configs that can be used to provision nodes.
9+
10+
To use it you need to install deps:
11+
```
12+
$ sudo pip install -r requirements.txt
13+
```
14+
15+
Also make sure to run it from its own directory.
16+
17+
### Generating anaconda-ks.cfg
18+
19+
```
20+
./dsal_helper.py anaconda-cfg --disks sda,sdb --boot-disk sda <hostname>
21+
```
22+
23+
The file will get created as `anaconda/anaconda-ks-<hostname>`.
24+
25+
### Cloning and configuring dev-install
26+
27+
```
28+
./dsal_helper.py dev-install-cfg --fip-pool-start 10.1.10.51 --fip-pool-end 10.1.10.55 --ceph-disk /dev/sdb <hostname>
29+
```
30+
31+
The dev-install will be cloned into `dev-install/<hostname>` and be ready to
32+
run `make osp_full` there. The configuration will use first FIP as an OpenStack
33+
VIP.
34+
35+
### Generating install-config.yaml
36+
37+
```
38+
./dsal_helper.py openshift-install-cfg --pull-secret ../pull-secret.yaml --ssh-key ~/.ssh/id_rsa.pub <hostname>
39+
```
40+
41+
Script will attempt to find free FIPs to use and create new ones if there are
42+
none. The `install-config.yaml` will be created in
43+
`install-config/<hostname>/install-config.yaml` so you can immediately use that
44+
directory as `--dir` of `openshift-install`. `networkType` defaults to Kuryr,
45+
use `--network-type` to change that.
46+
47+
The command will print `/etc/hosts` entries required to access the cluster.
48+
49+
## Scripts
50+
51+
There are few useful scripts in `scripts` folder.
52+
53+
### open-ssh.sh
54+
55+
This will lookup OpenShift master and workers SGs and add rules allowing SSH
56+
traffic to the VMs.
57+
58+
### fix-down-ports.sh
59+
60+
This attempts to reattach subports in DOWN status to the trunks in order to fix
61+
them, so they become ACTIVE. Useful for Kuryr deployments on faulty OSPs.
62+
63+
### get-latest.sh <version>
64+
65+
This will look up and download latest openshift-install for a given version in
66+
form of `4.x`, e.g. `get-latest.sh 4.9`. The binary will be placed in a
67+
directory named with the nightly version.
68+
69+
## TODOs
70+
* Correctly finding paths relative to the place where the script is.
71+
* Converting the script to an installable module with entrypoint?
72+
* Supporting anything else than 8.4 and 16.2.
73+
* Automating other useful activities:
74+
* Downloading openshift-install.
75+
* Managing clouds.yaml correctly when using multiple boxes.
76+
* Setting KUBECONFIG.
77+
* Managing /etc/hosts automatically.

tools/dsal/dsal_helper/__init__.py

Whitespace-only changes.
Lines changed: 211 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,211 @@
1+
# Licensed under the Apache License, Version 2.0 (the "License");
2+
# you may not use this file except in compliance with the License.
3+
# You may obtain a copy of the License at
4+
#
5+
# http://www.apache.org/licenses/LICENSE-2.0
6+
#
7+
# Unless required by applicable law or agreed to in writing, software
8+
# distributed under the License is distributed on an "AS IS" BASIS,
9+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
10+
# See the License for the specific language governing permissions and
11+
# limitations under the License.
12+
13+
import argparse
14+
import os
15+
import pathlib
16+
import socket
17+
import subprocess
18+
import sys
19+
20+
import jinja2
21+
import netaddr
22+
import openstack
23+
24+
DEV_INSTALL_REPO = 'https://github.com/shiftstack/dev-install.git'
25+
26+
27+
class DSALHelper(object):
28+
def __init__(self):
29+
self.app_dir = os.path.dirname(os.path.realpath(__file__))
30+
31+
parser = self._get_arg_parser()
32+
args = parser.parse_args()
33+
self._prepare_jinja()
34+
if hasattr(args, 'func'):
35+
try:
36+
self._setup_openstack(args.cloud)
37+
except:
38+
pass
39+
args.func(args)
40+
return
41+
42+
parser.print_help()
43+
parser.exit()
44+
45+
def _prepare_jinja(self):
46+
self.jinja = jinja2.Environment(
47+
loader=jinja2.FileSystemLoader("templates"))
48+
49+
def _get_arg_parser(self):
50+
parser = argparse.ArgumentParser(
51+
description="Helper for deploying dev-install on DSAL boxes")
52+
parser.add_argument('-c', '--cloud',
53+
default=os.environ.get('OS_CLOUD'),
54+
help='name in clouds.yaml to use')
55+
56+
subparsers = parser.add_subparsers(help='supported commands')
57+
58+
anaconda_cfg = subparsers.add_parser(
59+
'anaconda-cfg', help='prepare anaconda-ks.cfg for the host')
60+
anaconda_cfg.add_argument('hostname', help='full hostname of DSAL box')
61+
anaconda_cfg.add_argument('template', help='template to use, there are'
62+
' no predefined ones as '
63+
'every machine has its own '
64+
'specifics')
65+
anaconda_cfg.add_argument('--disks',
66+
help='comma-separated list of disks to use',
67+
default='sda,sdb,sdc')
68+
anaconda_cfg.add_argument('--boot-disk', help='boot disk to use',
69+
default='sda')
70+
anaconda_cfg.set_defaults(func=self.anaconda_cfg)
71+
72+
dev_install_cfg = subparsers.add_parser(
73+
'dev-install-cfg', help='clone and prepare dev-install for the '
74+
'host')
75+
dev_install_cfg.add_argument('hostname',
76+
help='full hostname of DSAL box')
77+
dev_install_cfg.add_argument('--host-ip',
78+
help='IP of the host, if not set will '
79+
'attempt to autodetect',
80+
default=None)
81+
dev_install_cfg.add_argument('--fip-pool-start', required=True,
82+
help='first IP of the FIP pool assigned '
83+
'to the hostname')
84+
dev_install_cfg.add_argument('--fip-pool-end', required=True,
85+
help='last IP of the FIP pool assigned '
86+
'to the hostname')
87+
# TODO(dulek): ceph-disk could accept a list
88+
dev_install_cfg.add_argument('--ceph-disk', required=True,
89+
help='disk that CEPH should use, point '
90+
'to SSDs RAID here. E.g. /dev/sdc')
91+
dev_install_cfg.set_defaults(func=self.dev_install_cfg)
92+
93+
openshift_install_cfg = subparsers.add_parser(
94+
'openshift-install-cfg', help='creates FIPs and prepares '
95+
'install-config.yaml for the host')
96+
openshift_install_cfg.add_argument('hostname',
97+
help='full hostname of DSAL box')
98+
openshift_install_cfg.add_argument('--network-type',
99+
help='networkType to set, defaults '
100+
'to Kuryr', default='Kuryr')
101+
openshift_install_cfg.add_argument('--pull-secret', required=True,
102+
help='path to pull secret file')
103+
openshift_install_cfg.add_argument('--ssh-key',
104+
help='path to public SSH key,'
105+
'defaults to '
106+
'~/.ssh/id_rsa.pub',
107+
default=os.path.expanduser(
108+
'~/.ssh/id_rsa.pub'))
109+
openshift_install_cfg.add_argument('--external-network',
110+
help='external network to use for '
111+
'FIPs, defaults to "external"',
112+
default='external')
113+
openshift_install_cfg.add_argument('--api-fip',
114+
help='FIP to use for API, if not '
115+
'set will try to find or '
116+
'create one')
117+
openshift_install_cfg.add_argument('--ingress-fip',
118+
help='FIP to use for ingress, if '
119+
'not set will try to find or '
120+
'create one')
121+
openshift_install_cfg.set_defaults(func=self.openshift_install_cfg)
122+
123+
return parser
124+
125+
def _setup_openstack(self, cloud_name):
126+
conn = openstack.connection.from_config(cloud=cloud_name)
127+
self.os = conn
128+
self.neutron = conn.network
129+
130+
def _create_dir(self, directory):
131+
pathlib.Path(directory).mkdir(exist_ok=True, parents=True)
132+
133+
def _render(self, directory, template_file, file, vars):
134+
self._create_dir(directory)
135+
template = self.jinja.get_template(template_file)
136+
with open(f'{directory}/{file}', 'w') as f:
137+
print(template.render(**vars), file=f)
138+
139+
def anaconda_cfg(self, args):
140+
self._render('anaconda', args.template,
141+
f'anaconda-ks-{args.hostname}.cfg', args.__dict__)
142+
143+
def dev_install_cfg(self, args):
144+
vars = args.__dict__
145+
if args.host_ip is None:
146+
vars['host_ip'] = socket.gethostbyname(args.hostname)
147+
148+
range = netaddr.IPRange(args.fip_pool_start, args.fip_pool_end)
149+
vars['control_plane_ip'] = range[0]
150+
vars['external_start'] = range[1]
151+
vars['external_end'] = range[-1]
152+
153+
self._create_dir('dev-install')
154+
if not os.path.exists(f'dev-install/{args.hostname}'):
155+
subprocess.run(['git', 'clone', DEV_INSTALL_REPO,
156+
f'dev-install/{args.hostname}'],
157+
stdin=sys.stdin, stdout=sys.stdout)
158+
159+
# TODO(dulek): Choose template based on OSP version.
160+
self._render('dev-install', 'local-overrides-16.2.yaml.tpl',
161+
f'{args.hostname}/local-overrides.yaml', vars)
162+
163+
subprocess.run(['make', 'config', f'host={args.hostname}',
164+
'user=stack'], cwd=f'dev-install/{args.hostname}',
165+
stdin=sys.stdin, stdout=sys.stdout)
166+
167+
def openshift_install_cfg(self, args):
168+
vars = args.__dict__
169+
for key in ('pull_secret', 'ssh_key'):
170+
with open(vars[key], 'r') as f:
171+
content = f.read()
172+
vars[key] = content.strip()
173+
174+
fips = self.neutron.ips(port_id=None)
175+
for key in ('api_fip', 'ingress_fip'):
176+
if not vars[key]:
177+
# First try finding a free one.
178+
try:
179+
vars[key] = next(fips).floating_ip_address
180+
except StopIteration:
181+
pass
182+
if not vars[key]:
183+
# Then try creating one.
184+
network = self.neutron.find_network(args.external_network)
185+
vars[key] = self.neutron.create_ip(
186+
floating_network_id=network.id).floating_ip_address
187+
188+
self._render(f'install-config/{args.hostname}',
189+
'install-config.yaml.tpl',
190+
f'install-config.yaml', vars)
191+
192+
hosts = f"{vars['api_fip']} api.ostest.{args.hostname}\n" \
193+
f"{vars['ingress_fip']} *.apps.ostest.{args.hostname}\n" \
194+
f"{vars['ingress_fip']} console-openshift-console.apps." \
195+
f"ostest.{args.hostname}\n" \
196+
f"{vars['ingress_fip']} integrated-oauth-server-openshift-" \
197+
f"authentication.apps.ostest.{args.hostname}\n" \
198+
f"{vars['ingress_fip']} oauth-openshift.apps.ostest." \
199+
f"{args.hostname}\n"
200+
201+
# TODO(dulek): Save it automatically?
202+
print('Put this in /etc/hosts:')
203+
print(hosts)
204+
205+
206+
def main():
207+
DSALHelper()
208+
209+
210+
if __name__ == '__main__':
211+
DSALHelper()

tools/dsal/pyproject.toml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
[build-system]
2+
requires = [
3+
"setuptools>=42",
4+
"wheel"
5+
]
6+
build-backend = "setuptools.build_meta"

tools/dsal/requirements.txt

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
jinja2
2+
netaddr
3+
openstacksdk
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
#!/bin/bash -x
2+
3+
ports=`openstack port list --device-owner "trunk:subport" -f value | grep DOWN | cut -d " " -f 1`
4+
trunks=`openstack network trunk list -f value -c ID`
5+
6+
for port in $ports; do
7+
for trunk in $trunks; do
8+
vlan_id=`openstack network subport list --trunk $trunk -f value | grep $port | cut -d " " -f 3`
9+
if [[ $vlan_id ]]; then
10+
# Found it!
11+
echo "Port $port is in trunk $trunk with VLAN ID $vlan_id"
12+
openstack network trunk unset --subport $port $trunk
13+
openstack network trunk set --subport port=$port,segmentation-type=vlan,segmentation-id=$vlan_id $trunk
14+
break
15+
fi
16+
done
17+
done

tools/dsal/scripts/get-latest.sh

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
#!/bin/bash -x
2+
3+
version=$1
4+
latest_nightly=$(curl -s https://openshift-release-artifacts.apps.ci.l2s4.p1.openshiftapps.com/ | awk "match(\$0, /($version.0-0.nightly[-0-9]+)/, a) {print a[1]}" | tail -1)
5+
wget "https://openshift-release-artifacts.apps.ci.l2s4.p1.openshiftapps.com/$latest_nightly/openshift-install-linux-${latest_nightly}.tar.gz"
6+
mkdir $latest_nightly
7+
tar -C $latest_nightly -xvzf openshift-install-linux-${latest_nightly}.tar.gz openshift-install
8+
rm -rf openshift-install-linux-${latest_nightly}.tar.gz
9+

tools/dsal/scripts/open-ssh.sh

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
#!/bin/bash -x
2+
3+
sgs=`openstack security group list -f value | grep OpenShift | cut -f 1 -d " "`
4+
5+
for sg in $sgs; do
6+
openstack security group rule create --dst-port 22 --protocol tcp $sg
7+
done

tools/dsal/scripts/reprovision.sh

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
#!/bin/bash -x
2+
3+
host=$1
4+
5+
echo "Reprovisioning system"
6+
bkr system-provision --kickstart anaconda/anaconda-ks-${host}.cfg --distro-tree 118371 --kernel-options "console=tty0 console=ttyS1,115200n81" $host
7+
8+
echo "Waiting for SSH to succeed"
9+
until ssh root@${host} echo "I\'m in"; do
10+
sleep 30
11+
done
12+
echo "SSH succeeded!"
13+
14+
cd dev-install/$host
15+
make osp_full
16+
make local_os_client

0 commit comments

Comments
 (0)