Heal PCI allocation during resize

During resize an instance with existing PCI allocation can be changed to
consume less, more, or different PCI devices. So the heal allocation
logic needs to handle the case when an existing instance is changed to
consume different PCI devices.

This patch adds support to change existing PCI allocations in placement
during resize.

There is one limitation of the healing logic. It assumes that there is
no in-progress migration when nova is upgraded. If there is an in
progress migration, then the PCI usage will not be healed in the
migration allocation. The placement view will be consistent after such
migration is completed or reverted.

blueprint: pci-device-tracking-in-placement
Change-Id: Icc968c567f9967d7449d6c6c1f57783098e63f55

Author: gibizer

doc/source/admin/pci-passthrough.rst

@@ -394,4 +394,14 @@ be added to the resource provider representing the matching PCI devices.
    (Zed) the nova-compute service will refuse to start with such configuration.
    It is suggested to use the PCI address of the device instead.
 
+The nova-compute service makes sure that already existing instances with PCI
+allocations in the nova DB will have a corresponding PCI allocation in
+placement. This allocation healing also acts on any new instances regardless of
+the status of the scheduling part of this feature to make sure that the nova
+DB and placement are in sync. There is one limitation of the healing logic.
+It assumes that there is no in-progress migration when the nova-compute service
+is upgraded. If there is an in-progress migration, then the PCI allocation on
+the source host of the migration will not be healed. The placement view will be
+consistent after such migration is completed or reverted.
+
 For deeper technical details please read the `nova specification. <https://specs.openstack.org/openstack/nova-specs/specs/zed/approved/pci-device-tracking-in-placement.html>`_

nova/compute/pci_placement_translator.py

@@ -268,6 +268,31 @@ def update_allocations(
         rp_uuid = provider_tree.data(self.name).uuid
 
         for consumer, amount in self._get_allocations().items():
+            if consumer not in allocations:
+                # We have PCI device(s) allocated to an instance, but we don't
+                # see any instance allocation in placement. This
+                # happens for two reasons:
+                # 1) The instance is being migrated and therefore the
+                #    allocation is held by the migration UUID in placement. In
+                #    this case the PciDevice is still allocated to the instance
+                #    UUID in the nova DB hence our lookup for the instance
+                #    allocation here. We can ignore this case as: i) We healed
+                #    the PCI allocation for the instance before the migration
+                #    was started. ii) Nova simply moves the allocation from the
+                #    instance UUID to the migration UUID in placement. So we
+                #    assume the migration allocation is correct without
+                #    healing. One limitation of this is that if there is in
+                #    progress migration when nova is upgraded, then the PCI
+                #    allocation of that migration will be missing from
+                #    placement on the source host. But it is temporary and the
+                #    allocation will be fixed as soon as the migration is
+                #    completed or reverted.
+                # 2) We have a bug in the scheduler or placement and the whole
+                #    instance allocation is lost. We cannot handle that here.
+                #    It is expected to be healed via nova-manage placement
+                #    heal_allocation CLI instead.
+                continue
+
             current_allocs = allocations[consumer]['allocations']
             current_rp_allocs = current_allocs.get(rp_uuid)