Merge pull request #109 from stackhpc/upstream/2023.1-2024-01-08

Synchronise 2023.1 with upstream

Author: mnasiadka

doc/source/admin/ovn/external_ports.rst

@@ -10,7 +10,7 @@ ML2/OVN leverages the use of OVN's `external ports
 feature.
 
 What is it
-~~~~~~~~~~
+----------
 
 The external ports feature in OVN allows for setting up a port that lives
 externally to the instance and is reponsible for replying to ARP requests
@@ -28,18 +28,107 @@ following VNICs:
 * macvtap
 * baremetal
 
-Also, ports of the type ``external`` will be scheduled on the gateway
-nodes (controller or networker nodes) in HA mode by the OVN Neutron
-driver. Check the `OVN Database information`_ section for more
-information.
+These ports can be listed in OVN with following command:
 
-OVN Database information
-~~~~~~~~~~~~~~~~~~~~~~~~
+.. code-block:: bash
+
+   $ ovn-nbctl find Logical_Switch_Port type=external
+   _uuid               : 105e83ae-252d-401b-a1a7-8d28ec28a359
+   ha_chassis_group    : [43047e7b-4c78-4984-9788-6263fcc69885]
+   type                : external
+   ...
+
+.. end
+
+The next section will talk more about the different configurations for
+scheduling these ports and how they are represented in the OVN database.
+
+Scheduling and database information
+-----------------------------------
+
+Ports of the type ``external`` will be scheduled on nodes
+marked to host these type of ports via the `ovn-cms-options
+<http://www.ovn.org/support/dist-docs/ovn-controller.8.html>`_
+configuration. There are two supported configurations for these nodes:
+
+1. ``enable-chassis-as-extport-host``
+2. ``enable-chassis-as-gw``
+
+These options can be set by running the following command locally on each
+node that will act as a candidate to host these ports:
+
+.. code-block:: bash
 
-the ML2/OVN driver identifies a gateway node by the
-``ovn-cms-options=enable-chassis-as-gw`` and ``ovn-bridge-mappings``
-options in the external_ids column from the ``Chassis`` table in the
-OVN Southbound database:
+   $ ovs-vsctl set Open_vSwitch . external-ids:ovn-cms-options=\"enable-chassis-as-extport-host\"
+
+   $ ovs-vsctl set Open_vSwitch . external-ids:ovn-cms-options=\"enable-chassis-as-gw\"
+
+.. end
+
+The sections below will explain the differences between the two
+configuration values.
+
+Configuration: ``enable-chassis-as-extport-host``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+When nodes in the cluster are marked with the
+``enable-chassis-as-extport-host`` configuration, the ML2/OVN driver
+will schedule the external ports onto these nodes. This configuration
+takes precedence over ``enable-chassis-as-gw``.
+
+With this configuration, the ML2/OVN driver will create one
+``HA_Chassis_Group`` per external port and it will be named as
+``neutron-extport-<Neutron Port UUID>``. For example:
+
+.. code-block:: bash
+
+   $ ovn-sbctl list Chassis
+   _uuid               : fa24d475-9664-4a62-bb1c-52a6fa4966f7
+   external_ids        : {ovn-cms-options=enable-chassis-as-extport-host, ...}
+   hostname            : compute-0
+   name                : "6fd9cef6-4e9d-4bde-ab82-016c2461957b"
+   ...
+   _uuid               : a29ee8f6-5301-45f5-b280-a43e533d4d65
+   external_ids        : {ovn-cms-options=enable-chassis-as-extport-host, ...}
+   hostname            : compute-1
+   name                : "4fa76c10-c6ea-4ae9-b31c-bc69103fe6f9"
+   ...
+
+.. end
+
+.. code-block:: bash
+
+   $ ovn-nbctl list HA_Chassis_Group neutron-extport-392a77f9-7c48-4ad0-bd06-8b55bba00bd1
+   _uuid               : 1249b761-24e3-414e-ae10-7e880e9d3cf8
+   external_ids        : {"neutron:availability_zone_hints"=""}
+   ha_chassis          : [0d6b9718-7718-45d2-a838-1deb40131442, ae6e64e7-f948-49b3-a171-c9cfb58c8b31]
+   name                : neutron-extport-392a77f9-7c48-4ad0-bd06-8b55bba00bd1
+
+.. end
+
+Also, for HA, there will be a limit of five Chassis per
+``HA_Chassis_Group``, meaning that even if there are more nodes marked
+with the ``enable-chassis-as-extport-host`` option, each group will
+contain up to five members. This limit has been imposed because OVN uses
+BFD to monitor the connectivity of each member in the group, and having
+an unlimited number of members can potentially put a lot of stress on OVN.
+
+In general, this option is used when there are specific requirements
+for ``external`` ports and they can not be scheduled on controllers or
+gateway nodes. The next configuration does the opposite and uses the
+nodes marked as gateway to schedule the ``external`` ports.
+
+Configuration: ``enable-chassis-as-gw``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For the majority of use cases where there are no special requirements
+for the ``external`` ports and they can be co-located with gateway ports,
+this configuration should be used.
+
+Gateway nodes are identified by the
+``enable-chassis-as-gw`` and `ovn-bridge-mappings
+<http://www.ovn.org/support/dist-docs/ovn-controller.8.html>`_
+configurations:
 
 .. code-block:: bash
 
@@ -52,67 +141,81 @@ OVN Southbound database:
 
 .. end
 
-For more information about both of these options, please
-take a look at the `ovn-controller documentation
-<http://www.ovn.org/support/dist-docs/ovn-controller.8.html>`_.
+As mentioned in the `What is it`_ section, every time a Neutron port
+with a certain VNIC is created the OVN driver will create a port of the
+type ``external`` in the OVN Northbound database.
 
-These options can be set by running the following command locally on each
-gateway node (note, the ``ovn-bridge-mappings`` will need to be adapted
-to your environment):
+When the ``enable-chassis-as-gw`` configuration is used, the ML2/OVN
+driver will create one ``HA_Chassis_Group`` per network (instead
+of one per external port in the previous case) and it will be named as
+``neutron-<Neutron Network UUID>``.
+
+All ``external`` ports belonging to this network will share the same
+``HA_Chassis_Group`` and the group is also limited to a maximum of five
+members for HA.
 
 .. code-block:: bash
 
-   $ ovs-vsctl set Open_vSwitch . external-ids:ovn-cms-options=\"enable-chassis-as-gw\" external-ids:ovn-bridge-mappings=\"public:br-ex\"
+   $ ovn-nbctl list HA_Chassis_Group
+   _uuid               : 43047e7b-4c78-4984-9788-6263fcc69885
+   external_ids        : {"neutron:availability_zone_hints"=""}
+   ha_chassis          : [3005bf84-fc95-4361-866d-bfa1c980adc8, 72c7671e-dd48-4100-9741-c47221672961]
+   name                : neutron-4b2944ca-c7a3-4cf6-a9c8-6aa541a20535
 
 .. end
 
-As mentioned in the `What is it`_ section, every time a Neutron port
-with a certain VNIC is created the OVN driver will create a port of the
-type ``external`` in the OVN Northbound database. These ports can be
-found by issuing the following command:
+High availability
+-----------------
+
+As hinted above, the ML2/OVN driver does provide high availability to the
+``external`` ports. This is done via the ``HA_Chassis_Group`` mechanism
+from OVN.
+
+On every ``external`` port there will be a column called
+``ha_chassis_group`` which points to the ``HA_Chassis_Group`` that the
+port belongs to:
 
 .. code-block:: bash
 
-   $ ovn-nbctl find Logical_Switch_Port type=external
-   _uuid               : 105e83ae-252d-401b-a1a7-8d28ec28a359
-   ha_chassis_group    : [43047e7b-4c78-4984-9788-6263fcc69885]
-   type                : external
-   ...
+  $ ovn-nbctl find logical_switch_port type=external
+  ha_chassis_group    : 924fd0fe-3e84-4eaa-aa1d-41103ec511e5
+  name                : "287040d6-0936-4363-ae0a-2d5a239e55fa"
+  type                : external
+  ...
 
 .. end
 
-The ``ha_chassis_group`` column indicates which HA Chassis Group that
-port belongs to, to find that group do:
+In the ``HA_Chassis_Group``, the members of each group are listed in the
+``ha_chassis`` column:
 
 .. code-block:: bash
 
-   # The UUID is the one from the ha_chassis_group column from
-   # the Logical_Switch_Port table
-   $ ovn-nbctl list HA_Chassis_Group 43047e7b-4c78-4984-9788-6263fcc69885
-   _uuid               : 43047e7b-4c78-4984-9788-6263fcc69885
-   external_ids        : {}
-   ha_chassis          : [3005bf84-fc95-4361-866d-bfa1c980adc8, 72c7671e-dd48-4100-9741-c47221672961]
-   name                : neutron-4b2944ca-c7a3-4cf6-a9c8-6aa541a20535
+  $ ovn-nbctl list HA_Chassis_Group 924fd0fe-3e84-4eaa-aa1d-41103ec511e5
+  _uuid               : 924fd0fe-3e84-4eaa-aa1d-41103ec511e5
+  external_ids        : {"neutron:availability_zone_hints"=""}
+  ha_chassis          : [3005bf84-fc95-4361-866d-bfa1c980adc8, 72c7671e-dd48-4100-9741-c47221672961]
+  name                : neutron-extport-287040d6-0936-4363-ae0a-2d5a239e55fa
 
 .. end
 
 .. note::
-  The external ports will be placed on a HA Chassis Group for the
-  network that the port belongs to. Those HA Chassis Groups are named as
-  ``neutron-<Neutron Network UUID>``, as seeing in the output above. You
-  can also use this "name" with the ``ovn-nbctl list`` command when
-  searching for a specific HA Chassis Group.
-
-The chassis that are members of the HA Chassis Group are listed in
-the ``ha_chassis`` column. Those are the gateway nodes (controller
-or networker nodes) in the deployment and it's where the ``external``
-ports will be scheduled. In order to find which gateway node the external
-ports are scheduled on use the following command:
+
+  There will be a maximum of five members for each group, this limit
+  has been imposed because OVN uses BFD to monitor the connectivity of
+  each member in the group, and having an unlimited number of members
+  can potentially put a lot of stress on OVN.
+
+.. end
+
+When listing the members of a group there will be a column called
+``priority`` that contains a numerical value, the member with the highest
+``priority`` is the chassis where the ports will be scheduled on. OVN
+will monitor each member via BFD protocol, and if the chassis that is
+hosting the ports goes down, the ports will be automatically scheduled
+on the next chassis with the highest priority that is alive.
 
 .. code-block:: bash
 
-   # The UUIDs are the UUID members of the HA Chassis Group
-   # (ha_chassis column from the HA_Chassis_Group table)
    $ ovn-nbctl list HA_Chassis 3005bf84-fc95-4361-866d-bfa1c980adc8 72c7671e-dd48-4100-9741-c47221672961
    _uuid               : 3005bf84-fc95-4361-866d-bfa1c980adc8
    chassis_name        : "1a462946-ccfd-46a6-8abf-9dca9eb558fb"
@@ -126,11 +229,6 @@ ports are scheduled on use the following command:
 
 .. end
 
-Note the ``priority`` column from the previous command, the chassis with
-the highest ``priority`` from that list is the chassis that will have
-the external ports scheduled on it. In our example above, the chassis
-with the UUID ``1a462946-ccfd-46a6-8abf-9dca9eb558fb`` is the one.
-
-Whenever the chassis with the highest priority goes down, the ports will
-be automatically scheduled on the next chassis with the highest priority
-which is alive. So, the external ports are HA out of the box.
+In the example above, the Chassis with the UUID
+``1a462946-ccfd-46a6-8abf-9dca9eb558fb`` is the one that is hosting the
+external port ``287040d6-0936-4363-ae0a-2d5a239e55fa``.

neutron/agent/linux/external_process.py

@@ -87,7 +87,19 @@ def enable(self, cmd_callback=None, reload_cfg=False, ensure_active=False):
         if not self.active:
             if not cmd_callback:
                 cmd_callback = self.default_cmd_callback
-            cmd = cmd_callback(self.get_pid_file_name())
+            # Always try and remove the pid file, as it's existence could
+            # stop the process from starting
+            pid_file = self.get_pid_file_name()
+            try:
+                utils.delete_if_exists(pid_file, run_as_root=self.run_as_root)
+            except Exception as e:
+                LOG.error("Could not delete file %(pid_file)s, %(service)s "
+                          "could fail to start. Exception: %(exc)s",
+                          {'pid_file': pid_file,
+                           'service': self.service,
+                           'exc': e})
+
+            cmd = cmd_callback(pid_file)
 
             ip_wrapper = ip_lib.IPWrapper(namespace=self.namespace)
             ip_wrapper.netns.execute(cmd, addl_env=self.cmd_addl_env,
@@ -99,12 +111,14 @@ def enable(self, cmd_callback=None, reload_cfg=False, ensure_active=False):
 
     def reload_cfg(self):
         if self.custom_reload_callback:
-            self.disable(get_stop_command=self.custom_reload_callback)
+            self.disable(get_stop_command=self.custom_reload_callback,
+                         delete_pid_file=False)
         else:
-            self.disable('HUP')
+            self.disable('HUP', delete_pid_file=False)
 
-    def disable(self, sig='9', get_stop_command=None):
+    def disable(self, sig='9', get_stop_command=None, delete_pid_file=True):
         pid = self.pid
+        delete_pid_file = delete_pid_file or sig == '9'
 
         if self.active:
             if get_stop_command:
@@ -118,10 +132,10 @@ def disable(self, sig='9', get_stop_command=None):
                 utils.execute(cmd, addl_env=self.cmd_addl_env,
                               run_as_root=self.run_as_root,
                               privsep_exec=True)
-                # In the case of shutting down, remove the pid file
-                if sig == '9':
-                    utils.delete_if_exists(self.get_pid_file_name(),
-                                           run_as_root=self.run_as_root)
+
+            if delete_pid_file:
+                utils.delete_if_exists(self.get_pid_file_name(),
+                                       run_as_root=self.run_as_root)
         elif pid:
             LOG.debug('%(service)s process for %(uuid)s pid %(pid)d is stale, '
                       'ignoring signal %(signal)s',

neutron/agent/linux/keepalived.py

@@ -427,15 +427,6 @@ def _output_config_file(self):
 
         return config_path
 
-    @staticmethod
-    def _safe_remove_pid_file(pid_file):
-        try:
-            os.remove(pid_file)
-        except OSError as e:
-            if e.errno != errno.ENOENT:
-                LOG.error("Could not delete file %s, keepalived can "
-                          "refuse to start.", pid_file)
-
     def get_vrrp_pid_file_name(self, base_pid_file):
         return '%s-vrrp' % base_pid_file
 
@@ -516,9 +507,6 @@ def callback(pid_file):
             if vrrp_pm.active:
                 vrrp_pm.disable()
 
-            self._safe_remove_pid_file(pid_file)
-            self._safe_remove_pid_file(self.get_vrrp_pid_file_name(pid_file))
-
             cmd = ['keepalived', '-P',
                    '-f', config_path,
                    '-p', pid_file,

neutron/agent/metadata/driver.py

@@ -269,7 +269,13 @@ def spawn_monitored_metadata_proxy(cls, monitor, ns_name, port, conf,
         pm = cls._get_metadata_proxy_process_manager(uuid, conf,
                                                      ns_name=ns_name,
                                                      callback=callback)
-        pm.enable()
+        try:
+            pm.enable()
+        except exceptions.ProcessExecutionError as exec_err:
+            LOG.error("Encountered process execution error %(err)s while "
+                      "starting process in namespace %(ns)s",
+                      {"err": exec_err, "ns": ns_name})
+            return
         monitor.register(uuid, METADATA_SERVICE_NAME, pm)
         cls.monitors[router_id] = pm
 
@@ -288,9 +294,8 @@ def destroy_monitored_metadata_proxy(cls, monitor, uuid, conf, ns_name):
                         pm.pid, SIGTERM_TIMEOUT)
             pm.disable(sig=str(int(signal.SIGKILL)))
 
-        # Delete metadata proxy config and PID files.
+        # Delete metadata proxy config.
         HaproxyConfigurator.cleanup_config_file(uuid, cfg.CONF.state_path)
-        linux_utils.delete_if_exists(pm.get_pid_file_name(), run_as_root=True)
 
         cls.monitors.pop(uuid, None)