Skip to content

Commit af02e6e

Browse files
markgoddardmarkgoddard
committed
libvirt: add nova-libvirt-cleanup command
Change Ia1239069ccee39416b20959cbabad962c56693cf added support for running a libvirt daemon on the host, rather than using the nova_libvirt container. It did not cover migration of existing hosts from using a container to using a host daemon. This change adds a kolla-ansible nova-libvirt-cleanup command which may be used to clean up the nova_libvirt container, volumes and related items on hosts, once it has been disabled. The playbook assumes that compute hosts have been emptied of VMs before it runs. A future extension could support migration of existing VMs, but this is currently out of scope. Change-Id: I46854ed7eaf1d5b5e3ccd8531c963427848bdc99 (cherry picked from commit 80b311b)
1 parent 2b78016 commit af02e6e

File tree

6 files changed

+138
-2
lines changed

6 files changed

+138
-2
lines changed

‎ansible/nova-libvirt-cleanup.yml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
- import_playbook: gather-facts.yml
3+
4+
- name: Remove nova_libvirt container
5+
gather_facts: false
6+
hosts:
7+
- compute
8+
serial: '{{ kolla_serial|default("0") }}'
9+
tags:
10+
- nova-libvirt-cleanup
11+
tasks:
12+
- import_role:
13+
name: nova-cell
14+
tasks_from: libvirt-cleanup.yml

‎ansible/roles/nova-cell/defaults/main.yml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -559,3 +559,14 @@ enable_shared_var_lib_nova_mnt: "{{ enable_cinder_backend_nfs | bool or enable_c
559559
###################################
560560

561561
nova_pci_passthrough_whitelist: "{{ enable_neutron_sriov | bool | ternary(neutron_sriov_physnet_mappings | dict2items(key_name='physical_network', value_name='devname'), []) }}"
562+
563+
##################
564+
# Libvirt cleanup
565+
##################
566+
567+
# The following options pertain to the kolla-ansible nova-libvirt-cleanup command.
568+
569+
# Whether to fail when there are running VMs.
570+
nova_libvirt_cleanup_running_vms_fatal: true
571+
# Whether to remove Docker volumes.
572+
nova_libvirt_cleanup_remove_volumes: false

‎ansible/roles/nova-cell/tasks/libvirt-cleanup.yml‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
---
2+
- name: Fail if nova_libvirt container is enabled
3+
fail:
4+
msg: >-
5+
The nova_libvirt container has not been cleaned up because it is enabled.
6+
It may be disabled by setting enable_nova_libvirt_container to false.
7+
when: enable_nova_libvirt_container | bool
8+
9+
- name: Get container facts
10+
become: true
11+
kolla_container_facts:
12+
name:
13+
- nova_libvirt
14+
register: container_facts
15+
16+
- block:
17+
- name: Check if there are any running VMs
18+
become: true
19+
shell:
20+
cmd: >
21+
set -o pipefail &&
22+
pgrep -l qemu | awk '!/qemu-ga/ && !/qemu-img/ {print $1}'
23+
register: running_vms
24+
25+
- name: Fail if there are any running VMs
26+
fail:
27+
msg: >-
28+
Refusing to remove nova_libvirt container with running VMs:
29+
{{ running_vms.stdout }}
30+
when:
31+
- running_vms.stdout != ''
32+
- nova_libvirt_cleanup_running_vms_fatal | bool
33+
34+
- name: Stop and remove nova_libvirt container
35+
become: true
36+
kolla_docker:
37+
action: "stop_and_remove_container"
38+
name: nova_libvirt
39+
when: container_facts['nova_libvirt'] is defined
40+
41+
- name: Remove nova_libvirt Docker volumes
42+
become: true
43+
kolla_docker:
44+
action: "remove_volume"
45+
name: "{{ item }}"
46+
loop:
47+
- libvirtd
48+
- nova_libvirt_qemu
49+
- nova_libvirt_secrets
50+
when: nova_libvirt_cleanup_remove_volumes | bool
51+
52+
- name: Remove config for nova_libvirt
53+
become: true
54+
file:
55+
path: "{{ node_config_directory }}/nova-libvirt"
56+
state: "absent"
57+
58+
# Revert the changes applied in config-host.yml.
59+
- block:
60+
- name: Remove udev kolla kvm rules
61+
become: true
62+
file:
63+
path: "/etc/udev/rules.d/99-kolla-kvm.rules"
64+
state: absent
65+
66+
- name: Reset /dev/kvm ownership
67+
become: true
68+
file:
69+
path: /dev/kvm
70+
group: kvm
71+
72+
- name: Unmask qemu-kvm service
73+
become: true
74+
systemd:
75+
name: qemu-kvm.service
76+
masked: false
77+
when:
78+
- ansible_facts.distribution == 'Ubuntu'
79+
when:
80+
- nova_compute_virt_type == 'kvm'

‎doc/source/reference/compute/libvirt-guide.rst‎

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -54,8 +54,27 @@ libvirt as a host daemon. However, since the Yoga release, if a libvirt daemon
5454
has already been set up, then Kolla Ansible may be configured to use it. This
5555
may be achieved by setting ``enable_nova_libvirt_container`` to ``false``.
5656

57-
Migration of hosts from a containerised libvirt to host libvirt is currently
58-
not supported.
57+
Migration from container to host
58+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
59+
60+
The ``kolla-ansible nova-libvirt-cleanup`` command may be used to clean up the
61+
``nova_libvirt`` container and related items on hosts, once it has
62+
been disabled. This should be run after the compute service has been disabled,
63+
and all active VMs have been migrated away from the host.
64+
65+
By default, the command will fail if there are any VMs running on the host. If
66+
you are sure that it is safe to clean up the ``nova_libvirt`` container with
67+
running VMs, setting ``nova_libvirt_cleanup_running_vms_fatal`` to ``false``
68+
will allow the command to proceed.
69+
70+
The ``nova_libvirt`` container has several associated Docker volumes:
71+
``libvirtd``, ``nova_libvirt_qemu`` and ``nova_libvirt_secrets``. By default,
72+
these volumes are not cleaned up. If you are sure that the data in these
73+
volumes can be safely removed, setting ``nova_libvirt_cleanup_remove_volumes``
74+
to ``true`` will cause the Docker volumes to be removed.
75+
76+
A future extension could support migration of existing VMs, but this is
77+
currently out of scope.
5978

6079
.. libvirt-tls:
6180

‎releasenotes/notes/nova-libvirt-cleanup-0d2c3b3156f2f657.yaml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
features:
3+
- |
4+
Adds a ``kolla-ansible nova-libvirt-cleanup`` command, which may be used to
5+
clean up the ``nova_libvirt`` container. This may be useful if switching to
6+
a host libvirt daemon.

‎tools/kolla-ansible‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -174,6 +174,7 @@ Commands:
174174
genconfig Generate configuration files for enabled OpenStack services
175175
prune-images Prune orphaned Kolla images
176176
chrony-cleanup Clean up disabled chrony containers
177+
nova-libvirt-cleanup Clean up disabled nova_libvirt containers
177178
EOF
178179
}
179180

@@ -217,6 +218,7 @@ upgrade
217218
upgrade-bifrost
218219
genconfig
219220
prune-images
221+
nova-libvirt-cleanup
220222
EOF
221223
}
222224

@@ -505,6 +507,10 @@ EOF
505507
ACTION="Cleanup disabled chrony containers"
506508
PLAYBOOK="${BASEDIR}/ansible/chrony-cleanup.yml"
507509
;;
510+
(nova-libvirt-cleanup)
511+
ACTION="Cleanup disabled nova_libvirt containers"
512+
PLAYBOOK="${BASEDIR}/ansible/nova-libvirt-cleanup.yml"
513+
;;
508514
(bash-completion)
509515
bash_completion
510516
exit 0

0 commit comments

Comments
 (0)